Orchadmin Command : DataStage

Orchadmin is a command line utility provided by datastage to research on data sets.

The general callable format is : $orchadmin< command> [options] [descriptor file]

1. Before using orchadmin, you should make sure that either the working directory or the
$APT_ORCHHOME/etc  contains the file “config.apt” OR
The environment variable $APT_CONFIG_FILE  should be defined for your session.

Orchadmin  commands
 The various commands available with orchadmin are

1. CHECK: $orchadmin check

Validates the configuration file contents like  , accesibility of all nodes defined in the configuration file,
scratch disk definitions and accesibility of all the nodes etc. Throws an error when config file is not
found or not defined properly

2. COPY : $orchadmin copy< source.ds> <destination.ds>

Makes a complete copy of the datasets of source with new destination descriptor file name. Please not
that
a. You cannot use UNIX cp command as it justs copies the config file to a new name. The data is not
copied.
b. The new datasets will be arranged in the form of the config file that is in use but not according to the
old confing file that was in use with the source.

  3. DELETE : $orchadmin < delete | del | rm >  [-f | -x] descriptorfiles….

The unix rm utility cannot be used to delete the datasets. The orchadmin delete or rm command should
be used to delete one or more persistent data sets.
-f options makes a force delete. If some nodes are not accesible then -f forces to delete the dataset
partitions from accessible nodes and leave the other partitions in inaccesible nodes as orphans.
-x forces to use the current config file to be used while deleting than the one stored in data set.

4.  DESCRIBE: $orchadmin describe [options] descriptorfile.ds

This is the single most important command.

1. Without any option lists the no.of.partitions, no.of.segments, valid segments, and preserve
partitioning flag details of the persistent dataset.
-c : Print the configuration file that is written in the dataset  if any
-p: Lists down the partition level information.
-f: Lists down the file level information in each partition
-e:  List down the segment level information .
-s: List down the  meta-data schema of the information.
-v:  Lists all segemnts , valid or otherwise
-l : Long listing. Equivalent to -f -p -s -v -e

5. DUMP: $orchadmin dump [options] descriptorfile.ds

The dump command is used to dump(extract) the records from the dataset.
Without any options the dump command lists down all the records starting from first record from first
partition till  last record in last partition.
-delim ‘<string>’ : Uses the given string as delimtor for fields instead of space.
-field< name> : Lists only the given field instead of all fields.
-name : List all the values preceded by field name and a colon
-n numrecs : List only the given number of records per partition.
-p period(N) : Lists every  Nth record from each partition starting from first record.
-skip N: Skip the first N records from each partition.
-x : Use the current system configuration file rather than the one stored in dataset.

6. TRUNCATE: $orchadmin truncate [options] descriptorfile.ds

Without options deletes all the data(ie Segments) from the dataset.
-f: Uses force truncate. Truncate accessible segments and leave the inaccesible ones.
-x: Uses current system config file rather than the default one stored in the dataset.
-n N: Leaves the first N segments in each partition and truncates the remaining.

7. HELP: $orchadmin -help OR $orchadmin <command> -help

Help manual about the usage of orchadmin or orchadmin commands.

Grep command in Unix/Linux is the short form of ‘global search for the regular expression’.

The grep command is a filter that is used to search for lines matching a specified pattern and print
the matching lines to standard output.

Grep Command in Unix with Examples

Syntax:
grep [options] [pattern] [file]

The pattern is specified as a regular expression. A regular expression is a string of characters that is
used to specify a pattern matching rule. Special characters are used to define the matching rules and
positions.

#1) Anchor Characters: ‘^’ and ‘$’ at the beginning and end of the pattern are used to anchor the
pattern to the start of the line, and to the end of the line respectively.

Example: “^Name” matches all lines that start with the string “Name”. The strings “\<” and “\>” are
used to anchor the pattern to the start and end of a word respectively.

#2) Wildcard Character: ‘.’ Is used to match any character.

Example:“^.$” will match all lines with any single character.

#3) Escaped Characters: Any of the special characters can be matched as a regular character by
escaping them with a ‘\’.

Example: “\$\*” will match the lines that contain the string “$*”

#4) Character Range: A set of characters enclosed in a ‘[‘ and ‘]’ pair specify a range of characters to
be matched.

Example: “[aeiou]” will match all lines that contain a vowel. A hyphen can be used while specifying a
range to shorten a set of consecutive characters. E.g. “[0-9]” will match all lines that contain a digit.
A carat can be used at the beginning of the range to specify a negative range. E.g. “[^xyz]” will match
all lines that do not contain x, y or z.

#5) Repetition Modifier: A ‘*’ after a character or group of characters is used to allow matching zero
or more instances of the preceding pattern.

The grep command supports a number of options for additional controls on the matching:
•-i: performs a case-insensitive search.
•-n: displays the lines containing the pattern along with the line numbers.
•-v: displays the lines not containing the specified pattern.
•-c: displays the count of the matching patterns.

Examples:
•Match all lines that start with ‘hello’. E.g: “hello there”
$ grep “^hello” file1
•Match all lines that end with ‘done’. E.g: “well done”
$ grep “done$” file1
•Match all lines that contain any of the letters ‘a’, ‘b’, ‘c’, ‘d’ or ‘e’.
$ grep “[a-e]” file1
•Match all lines that do not contain a vowel
$ grep “[^aeiou]” file1
•Match all lines that start with a digit following zero or more spaces. E.g: “ 1.” or “2.”
$ grep “ *[0-9]” file1
•Match all lines that contain the word hello in upper-case or lower-case
$ grep -i “hello”

Conclusion

I’m sure that this tutorial would have helped you to get a good understanding of what is grep
command in Unix and how it is used in various conditions.

ls command in Linux/Unix
ls is a Linux shell command that lists directory contents of files and directories.
 ls syntax
 ls options
 ls examples

ls syntax
$ ls [options] [file|dir]

ls command options
ls command main options:
option description

ls -a list all files including hidden file starting with '.'

ls --color colored list [=always/never/auto]

ls -d list directories - with ' */'

ls -F add one char of */=>@| to enteries

ls -i list file's inode index number

ls -l list with long format - show permissions

ls -la list long format including hidden files

ls -lh list long format with readable file size

ls -ls list with long format with file size

ls -r list in reverse order

ls -R list recursively directory tree

ls -s list file size

ls -S sort by file size

ls -t sort by time & date

ls -X sort by extension name

ls command examples
You can press the tab button to auto complete the file or folder names.
List directory Documents/Books with relative path:
$ ls Documents/Books
 
List directory /home/user/Documents/Books with absolute path.
$ ls /home/user/Documents/Books
 
List root directory:
$ ls /
 
List parent directory:
$ ls ..
 
List user's home directory (e.g: /home/user):
$ ls ~
 
List with long format:
$ ls -l
 
Show hidden files:
$ ls -a
 
List with long format and show hidden files:
$ ls -la
 
Sort by date/time:
$ ls -t
 
Sort by file size:
$ ls -S
 
List all subdirectories:
$ ls *
 
Recursive directory tree list:
$ ls -R
 
List only text files with wildcard:
$ ls *.txt
 
ls redirection to output file:
$ ls > out.txt
 
List directories only:
$ ls -d */
 
List files and directories with full path:
$ ls -d $PWD/*