1.

 sudo command
The sudo command stands for "superuser do". It prompts you for your personal password and
confirms your request to execute a command by checking a file, called sudoers, which the system
administrator configures. Using the sudoers file, system administrators can give certain users or
groups access to some or all commands without those users having to know the root password. It
also logs all commands and arguments so there is a record of who used it for what, and when.
To use the sudo command, at the command prompt, enter:

sudo command

2. su command
The su command stands for "switch user", and allows you to become another user. To use
the su command on a per-command basis, enter:

su user

Replace user with the name of the account which you'd like to run the commands as.
The user feature is optional; if you don't provide a user, the su command defaults to
the root account, which in Unix is the system administrator account. In either case, you'll be
prompted for the password associated with the account for which you're trying to run the
command. If you supply a user, you will be logged in as that account until you exit it. To do so,
press Ctrl-d or type exit at the command prompt.

3. who-
who is used to find out who is logged on to the system, what line they’re on, and how long they’ve
been logged in.

4. whoami –
This command lists the account name associated with the current login.
5. who am i-
This command tells about my identity on the computer, including the name of the computer
itself, my account name, and where and when I logged in.

6. which
Locates the executable file associated with a given command.
The which command shows the full path for its given arguments. This command searchers for
an executable or script in the user’s PATH.
It does this by searching the paths in the PATH environment variable for executable files
matching the names of the arguments.

7. mail
Opens the mail program and displays the first message in the mailbox, if any

Starts a new e-mail addressed to xyzemailid.com. When finished composing the

message, type CTRL-D on a new line.
8. Mailx
cat QDMAUDT_20160419_1105372.log | head -2 | mailx -s "abc"
shreya.soni@amdocs.com

9. Hostname
hostname is used to display the system's DNSname, and to display or set
its hostname or NIS (Network Information Services) domain name.
The hostname command shows or sets the system hostname.
When called without any arguments, hostname will display the name of the
system as returned by the gethostname function.
 OPTIONS-
1. –i (IP address)
hostname –i prints the IP address of the computer.

2. –d (Domain name)
Print the domain name. To know the domain name where the computer resides, use the -d
option with hostname command.

3. –h (help)
To get help about the hostname command either use the man command or the -h
option with hostname command.

10. Uname
This command displays certain features of the operating system running on your machine.
By default, it displays name of the operating system
 OPTIONS-
1. –r ( the current version)
uname –r displays the version of operating system (here version number of kernel)

2. –n ( the machine name)

uname –n displays the host name.

11. Ps(process status)

The ps command on linux is used for viewing the processes running on the system. It
provides a snapshot of the current processes along with detailed information like user
id, cpu usage, memory usage, command name etc. 

By default, it will show us 4 columns of information.

 PID is a Process ID of the running command (CMD)

 TTY is a place where the running command runs
 TIME tell about how much time is used by CPU while running the command
 CMD is a command that run as current process

This will show all current processes.

x
While x will show all process even the current process is not associated with any TTY (terminal)

3. Filter processes by its user

For some situation we may want to filter processes by user. To do this, we can use -u option. Let
say we want to see what processes which run by user pungki. So the command will be like below

$ ps -u pungki

4. Filter processes by CPU or memory usage

Another thing that you might want to see is filter the result by CPU or memory usage. With this, you
can grab information about which processes that consume your resource. To do this, we can
use aux options. Here’s an example of it :

$ ps -aux | less
Since the result can be in a long list, we can pipe less command into ps command.
By default, the result will be in unsorted form. If we want to sort by particular column, we can add --
sortoption into ps command.

Sort by the highest CPU utilization in ascending order

$ ps -aux --sort -pcpu | less

Sort by the highest Memory utilization in ascending order

$ ps -aux --sort -pmem | less

Or we can combine itu a single command and display only the top ten of the result :

$ ps -aux --sort -pcpu,+pmem | head -n 10

5. Filter processes by its name or process ID

To to this, we can use -C option followed by the keyword. Let say, we want to show processes
named getty. We can type :

$ ps -C getty

If we want to show more detail about the result, we can add -f option to show it on full format listing.
The above command will looks like below :

$ ps -f -C getty
12. cd ( changing the current directory)
We can move around in the file system using cd command.

When used with an argument, it changes the current directory to the directory specified as
argument.

When you need to switch to the file present in another directory, absolute pathname must be
used as –

cd can also be used without any arguments. Cd used without arguments reverts to the home
directory.

cd..
An argument to cd to move to the parent directory.

cd.. moves one level up.

cd ../.. moves two levels up.

13. ls [options] [names]
OPTIONS-
1. –a (hidden files) – It lists all the files including hidden files that are present in your
directory
2. –l (long listing)-
Lists the total files in the directory and subdirectories, the names of the files in the current
directory, their permissions, the number of subdirectories in directories listed, the size of
the file, and the date of last modification.
3. –R (Recusive)-
This option list all subdirectories encountered.

4. –i (inode number)- This option displays inode number of the file.

5. –r (reverse order)-
This option sorts file names in reverse order (By default ASCII sequence)
6. –t (modification time)-
Sorts filenames by last modification time.
7. –u (access time)-
Sorts filenames by last access time.

8. –lt (listing by modification time)-

Sorts listing by last modification time. (most recently used to least recently used)

9. –lrt (listing by modification time -reverse)-

Sorts listing by modification time in reverse order (least recently used to most recently
used)
9. –F
This option marks executables with *, directories with / and symbolic links with @.
10. –lh
This option list long format with readable file sizes

11. -1
This option lists only one file in a line
13. Cat command
1. Display Contains of File

# cat /etc/passwd

root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
narad:x:500:500::/home/narad:/bin/bash

2. View Contains of Multiple Files in terminal

# cat test test1

Hello everybody
Hi world,

3. Create a File with Cat Command

# cat >test2

Awaits input from user, type desired text and press CTRL+D (hold down Ctrl
Key and type ‘d‘) to exit. The text will be written in test2 file. You can see contains
of file with following cat command.
# cat test2

hello everyone, how do you do?

4. Use Cat Command with More & Less Options

If file having large number of contains that won’t fit in output terminal and screen
scrolls up very fast, we can use parameters more and less with cat command as
show above.

# cat song.txt | more

# cat song.txt | less

5. Display Line Numbers in File

With -n option you could see the line numbers of a file song.txt in the output
terminal.

# cat -n song.txt

1 "Heal The World"

2 There's A Place In
3 Your Heart
4 And I Know That It Is Love
5 And This Place Could
6 Be Much
7 Brighter Than Tomorrow
8 And If You Really Try
9 You'll Find There's No Need
10 To Cry
11 In This Place You'll Feel
12 There's No Hurt Or Sorrow

6. Display $ at the End of File

In the below, you can see with -e option that ‘$‘ is shows at the end of line and also
in space showing ‘$‘ if there is any gap between paragraphs. This options is useful
to squeeze multiple lines in a single line.

# cat -e test

hello everyone, how do you do?$

$
Hey, am fine.$
How's your training going on?$
$

7. Display Tab separated Lines in File

In the below output, we could see TAB space is filled up with ‘^I‘ character.

# cat -T test

hello ^Ieveryone, how do you do?

Hey, ^Iam fine.

^I^IHow's your training ^Igoing on?
Let's do ^Isome practice in Linux.

8. Display Multiple Files at Once

In the below example we have three files test, test1 and test2 and able to view the

contains of those file as shown above. We need to separate each file with ; (semi
colon).

# cat test; cat test1; cat test2

This is test file

This is test1 file.
This is test2 file.

9. Use Standard Output with Redirection Operator

We can redirect standard output of a file into a new file else existing file with ‘>‘
(greater than) symbol. Careful, existing contains of test1 will be overwritten by
contains of test file.

# cat test > test1

10. Appending Standard Output with Redirection Operator

Appends in existing file with ‘>>‘ (double greater than) symbol. Here, contains
of test file will be appended at the end of test1 file.

# cat test >> test1

11. Redirecting Standard Input with Redirection Operator

When you use the redirect with standard input ‘<‘ (less than symbol), it use file
name test2 as a input for a command and output will be shown in a terminal.

# cat < test2

This is test2 file.

12. Redirecting Multiple Files Contain in a Single File

This will create a file called test3 and all output will be redirected in a newly created
file.

# cat test test1 test2 > test3

13. Sorting Contains of Multiple Files in a Single File

This will create a file test4 and output of cat command is piped to sort and result
will be redirected in a newly created file.

# cat test test1 test2 test3 | sort > test4

14. chmod
Used for setting file and directory permissions.
Syntax.
chmod [0-7][0-7][0-7] filename (Absolute Mode)

chmod [ugoa][+-=][rwx] filename (Symbolic Mode)

Following are the symbolic representation of three different roles:

 u is for user,
 g is for group,
 and o is for others.
Following are the symbolic representation of three different permissions:

 r is for read permission,

 w is for write permission,
 x is for execute permission.

1. Add single permission to a file/directory

$ chmod u+x filename

2. Add multiple permission to a file/directory

$ chmod u+r,g+x filename

3. Remove permission from a file/directory

$ chmod u-rx filename

4. Change permission for all roles on a file/directory

$ chmod a+x filename

5. Make permission for a file same as another file (using reference)

$ chmod --reference=file1 file2

6. Apply the permission to all the files under a directory recursively

$ chmod -R 755 directory-name/

7. Change execute permission only on the directories (files are not affected)
$ chmod u+X *

15. Gzip is used to compress a file in order to reduce disk space

Example Gzip Commands

 1. Compress a single file

This will compress file.txt and create file.txt.gz, note that this will remove the
original file.txt file.
 gzip file.txt

 2. Compress multiple files at once

This will compress all files specified in the command, note again that this will
remove the original files specified by turning file1.txt, file2.txt and file3.txt into
file1.txt.gz, file2.txt.gz and file3.txt.gz
 gzip file1.txt file2.txt file3.txt

To instead compress all files within a directory, see example 8 below.

 3. Compress a single file and keep the original

You can instead keep the original file and create a compressed copy.
 gzip -c file.txt > file.txt.gz

The -c flag outputs the compressed copy of file.txt to stdout, this is then sent to
file.txt.gz, keeping the original file.txt file in place. Newer versions of gzip may
also have -k or –keep available, which could be used instead with “gzip -k
file.txt”.

 4. Compress all files recursively

All files within the directory and all sub directories can be compressed
recursively with the -r flag
 [root@centos test]# ls -laR
 .:

 drwxr-xr-x. 2 root root 24 Jul 28 18:05 example

 -rw-r--r--. 1 root root 8 Jul 28 17:09 file1.txt

 -rw-r--r--. 1 root root 3 Jul 28 17:54 file2.txt

 -rw-r--r--. 1 root root 5 Jul 28 17:54 file3.txt

 ./example:

 -rw-r--r--. 1 root root 5 Jul 28 18:00 example.txt

 [root@centos test]# gzip -r *

 [root@centos test]# ls -laR

 .:

 drwxr-xr-x. 2 root root 27 Jul 28 18:07 example

 -rw-r--r--. 1 root root 38 Jul 28 17:09 file1.txt.gz

 -rw-r--r--. 1 root root 33 Jul 28 17:54 file2.txt.gz

 -rw-r--r--. 1 root root 35 Jul 28 17:54 file3.txt.gz

 ./example:

 -rw-r--r--. 1 root root 37 Jul 28 18:00 example.txt.gz

In the above example there are 3 .txt files in the test directory which is our
current working directory, there is also an example sub directory which
 contains example.txt. Upon running gzip with the -r flag over everything, all
files were recursively compressed.

This can be reversed by running “gzip -dr *”, where -d is used to decompress
and -r performs this on all of the files recursively.

 5. Decompress a gzip compressed file

To reverse the compression process and get the original file back that you have
compressed, you can use the gzip command itself or gunzip which is also part of
the gzip package.
 gzip -d file.txt.gz

OR

gunzip file.txt.gz

Both of these commands will produce the same result, decompressing file.txt.gz
to file.txt, removing the compressed file.txt.gz file.

Similar to example 3, it is possible to decompress a file and keep the original .gz
file as below.

gunzip -c file.txt.gz > file.txt

As mentioned in step 4, -d can be combined with -r to decompress all files

recursively.

 6. List compression information

With the -l or –list flag we can see useful information regarding a compressed
.gz file such as the compressed and uncompressed size of the file as well as the
compression ratio, which shows us how much space our compression is saving.
 [root@centos ~]# gzip -l linux-3.18.19.tar.gz

 compressed uncompressed ratio uncompressed_name

 126117045 580761600 78.3% linux-3.18.19.tar



 [root@centos ~]# ls -lah

 -rw-r--r--. 1 root root 554M Jul 28 17:24 linux-3.18.19.tar

 -rw-r--r--. 1 root root 121M Jul 28 17:25 linux-3.18.19.tar.gz

In this example, a gzipped copy of the Linux kernel has compressed to 78.3% of
its original size, taking up 121MB of space rather than 554MB.

16. Links:
Every file has a data structure known as an inode that stores information about the file,
and the filename is simply used as a reference to that data structure.
A link is simply a way to refer to the contents of the file. They are of two types:
1. Hard Links
A file can have multiple filenames. When that happens then the file has more
than one link.

A file is linked with the ln command.

ln emp.lst employee // employee must not exist

The above command links (hard) emp.lst with employee

Create a link using ln command

Since they have the same inode no. that means they are actually one and the
same file.(copy).
The link count is 2.

If the content of one file gets changed, it will be reflected in other (linked) file as
well.
 We can link third filename emp.dat

We can also link multiple files but in that case destination filename must be a
directory.

ln chap?? Projects_dir

The rm command is used to remove the link.

Now the link count has come down to two.

 Where to use hard links

a. Let’s consider we have written a no. of programs that read a file foo.txt in
$HOME/input_files. Later we moved foo.txt to $HOME/data instead. What
happens to all the programs that look for foo.txt at its original location.

Solution is to just link foo.txt to the directory input_files

ln data/foo.txt input_files //creates link in directory input_files
With this link available, existing programs will continue to find foo.txt in the
input_files directory.
b. Links provide some protection against accidental deletion. Example even though
a single file foo.txt exits on disk, we have made a backup of this file.
 So if you inadvertenely delete input_files/foo.txt, one link will still be available at
data/foo.txt and that file is not gone.

2. Symbolic Links

limitations of hard links:

 We can’t have two linked filenames in two file systems. Or we can simply
say that we can’t link a filename in the /usr file system to another in the
/home file system.

 We can’t link a directory even within the same file system.

These limitations were overcome by Symbolic Links.

Unlike the hard link, a symbolic link doesn’t have file contents, but simply
provides the pathname of the file that actually has contents. It is also
known as soft link.
Example – windows shortcuts

ln command is used to create a soft link with –s option.

We can identify symbolic link with l in the permission field. The pointer notation
(->) suggests that note.sym contains the pathname for the filename note.
Only note contains the data and not note.sym.

cat note.sym opens the file the link points to. The size of the symbolic link is 4
that is length of the pathname it contains.

We have two files and they are not identical. So removing note.sym won’t affect
us. But if we remove note, we would lose the file containing the data. In that case,
note.sym would point to a non-existent file and become a dangling symbolic link.

Symbolic links can be used with relative pathnames. They can also span multiple
file systems and also link directories. It can link all filenames in a directory to
another directory. Symbolic link has a separate directory entry with its own
inode number.
We can use rm to remove a symbolic link
17. mkdir ( making directories)
This command is used to create a directory. The command is followed by names of the
directories to be created.

We can create a number of subdirectories with one mkdir command.

In above example , we have created three directories.

We can also create the directory tree using mkdir command.

 Sometimes, system refuses to create a directory

This can happen because of below reasons:

 The directory may already exist.
 There may be an ordinary file by the same name in the current directory.
 The permissions set for the current directory don’t permit the creation of files and
directories by the user.

OPTIONS-

 mkdir –m mode
Sets the access mode for the new directory ( as with chmod command)

 mkdir -p
Create parent directories as necessary. When this option is used, no error is reported if a
specified DIRECTORY already exists.

18. rmdir ( removing directories)-

The rmdir ( remove directory) command removes directories.

rmdir directory_name But the directory must be empty

 Like mkdir, rmdir can also delete more than one directory in one shot. For instance, the three
directories and subdirectories that were just created with mkdir can be removed by using rmdir
with a reverse set of arguments.
rmdir student/pqr student/abc student

The below sequence used by rmdir is invalid-

Remember you cannot delete a directory unless it is empty.

OPTIONS-

 rmdir –p
-p, --parents remove DIRECTORY and its ancestors

2 ways-
1. rmdir ABC/abc ABC
2. rmdir –p ABC/abc

19. wc (counting lines, words and

characters)-
wc takes one or more filenames as arguments and displays a four- columnar output.
Emp789 contains 4 lines, 24words and 105 characters.

OPTIONS-
1. –l
To count no. of lines

2. –w
To count no. of words

3. –c
To count no. of characters

When used with multiple files, wc produces a line for each file as well as total count.

20. Sort:
Helpful to sort/order lines in text files. You can sort the data in text file and display the
output on the screen, or redirect it to a file. Based on your requirement, sort provides
several command line options for sorting data in a text file.

Sort Command Syntax:

$ sort [-options]

For example, here is a test file:

$ cat test

zzz

sss

qqq

aaa

BBB

ddd

AAA

And, here is what you get when sort command is executed on this file without any
option. It sorts lines in test file and displays sorted output.

$ sort test

aaa

AAA
BBB

ddd

qqq

sss

zzz

1. Perform Numeric Sort using -n option

If we want to sort on numeric value, then we can use -n or –numeric-sort option.

Create the following test file for this example:

$ cat test

22 zzz

33 sss

11 qqq

77 aaa

55 BBB

The following sort command sorts lines in test file on numeric value in first word of line
and displays sorted output.
$ sort -n test

11 qqq

22 zzz

33 sss

55 BBB

77 aaa

2. Sort Human Readable Numbers using -h option

If we want to sort on human readable numbers (e.g., 2K 1M 1G), then we can use -h or –
human-numeric-sort option.
Create the following test file for this example:

$ cat test

2K

2G

1K

6T

1T

1G
2M

The following sort command sorts human readable numbers (i.e 1K = 1 Thousand, 1M =
1 Million, 1G = 1 Giga, 1T = 1 Tera) in test file and displays sorted output.

$ sort -h test

1K

2K

2M

1G

2G

1T

6T

3. Sort Months of an Year using -M option

If we want to sort in the order of months of year, then we can use -M or –month-

sort option.
Create the following test file for this example:

$ cat test

sept
aug

jan

oct

apr

feb

mar11

The following sort command sorts lines in test file as per month order. Note, lines in file
should contain at least 3 character name of month name at start of line (e.g. jan, feb,
mar). If we will give, ja for January or au for August, then sort command would not
consider it as month name.

$ sort -M test

jan

feb

mar11

apr

aug

sept
oct

4. Check if Content is Already Sorted using -c option

If we want to check data in text file is sorted or not, then we can use -c or –check, –
check=diagnose-first option.
Create the following test file for this example:

$ cat test

The following sort command checks whether text file data is sorted or not. If it is not,
then it shows first occurrence with line number and disordered value.

$ sort -c test

sort: test:3: disorder: 1

5. Reverse the Output and Check for Uniqueness using -r and -u options

If we want to get sorted output in reverse order, then we can use -r or –reverseoption. If

file contains duplicate lines, then to get unique lines in sorted output, “-u” option can be
used.
Create the following test file for this example:
$ cat test

The following sort command sorts lines in test file in reverse order and displays sorted
output.

$ sort -r test

1
The following sort command sorts lines in test file in reverse order and removes
duplicate lines from sorted output.

$ sort -r -u test

6. Selectively Sort the Content, Customize delimiter, Write output to a file using  -k, -t,
-o options

If we want to sort on the column or word position in lines of text file, then “-k” option
can be used. If we each word in each line of file is separated by delimiter except ‘space’,
then we can specify delimiter using “-t” option. We can get sorted output in any
specified output file (using “-o” option) instead of displaying output on standard output.

Create the following test file for this example:

$ cat test

aa aa zz

aa aa ff

aa aa tt

aa aa kk
The following sort command sorts lines in test file on the 3rd word of each line and
displays sorted output.

$ sort -k3 test

aa aa ff

aa aa kk

aa aa tt

aa aa zz

$ cat test

aa|5a|zz

aa|2a|ff

aa|1a|tt

aa|3a|kk

Here, several options are used altogether. In test file, words in each line are separated by
delimiter ‘|’. It sorts lines in test file on the 2nd word of each line on the basis of numeric
value and stores sorted output into specified output file.

$ sort -n -t'|' -k2 test -o outfile

The contents of output file are shown below.

$ cat outfile

aa|1a|tt

aa|2a|ff

aa|3a|kk

aa|5a|zz

21. tail - Display of the file end (default 10 lines) on screen.

22. head - Display of the file beginning (default 10 lines) on screen.

23. grep : The grep utility searches text files for a pattern and prints all lines
that contain that pattern.
grep:

SYNOPSIS

/usr/bin/grep [-bchilnsvw] limited-regular-expression

[filename]...

DESCRIPTION

The grep utility searches text files for a pattern and

prints all lines that contain that pattern.

Be careful using the characters $, *, [, ^, |, (, ), and \

in the pattern_list because they are also meaningful to the

shell. It is safest to enclose the entire pattern_list in

single quotes a'...a'.

General syntax :
grep –options pattern filename

where options can be any combination of ( b,c,h,i,l,n,s,v,w )

OPTIONS

-c Prints only a count of the lines that contain the pat-

tern.

-h Prevents the name of the file containing the matching

line from being prepended to that line. Used when

searching multiple files.

-i Ignores upper/lower case distinction during comparis-

ons.

-l Prints only the names of files with matching lines,

separated by NEWLINE characters. Does not repeat the

names of files when the pattern is found more than

once.
 -n Precedes each line by its line number in the file

(first line is 1).

-v Prints all lines except those that contain the pat-

tern.

-w Searches for the expression as a word as if surrounded

by \< and \>.

test.txt test.txt
24. egrep (grep -E):

search a file for a pattern using full regular

expressions .it can be used to search more than one pattern at a time.

SYNOPSIS

/usr/bin/egrep [-bchilnsv] -e pattern_list [file...]

/usr/bin/egrep [-bchilnsv] -f file [file...]

/usr/bin/egrep [-bchilnsv] pattern [file...]

NOTE : Since | is working here as logical OR operator , & doesn’t work as logical AND.

Options:

-f to read the pattern from a file

egrep –f input_file file

this command will first read input file and then the content of this file will be considered as pattern and
then this pattern will be searched in file.
-e : What if a pattern has a leading ‘-’? in that case –e option is useful.

All other options work the same way as in grep.

fgrep : (grep -F)

 The fgrep (fixed grep) utility searches files for a charac-

ter string and prints all lines that contain that string. it

searches for a string, instead of searching for a pattern

that matches an expression.

The characters $, *, [, ^, |, (, ), and \ are interpreted

literally by fgrep, that is, fgrep does not recognize full

regular expressions as does egrep.

Options work the same way as in egrep.

Miscellaneous:

1) To find files that are containing empty lines.

grep ^$ *

2) To find files containing ^V or ^M characters.

 If we use ^V literally to find using grep it won’t work.

Rather we have to use Ctrl+v+v (for ^V) and Ctrl+v+m (for ^M), see below results:

3) We also have –A , –B , –C options but it works only in Linux not in unix box.

-A n ( to get n lines above the line containing pattern)

grep –A 5 pattern filename

-B n (to get n lines below the line containing pattern)

grep –B 5 pattern filename

-C n (to get n lines around the line containing pattern)

grep –C 5 pattern filename

25. Awk
Awk is one of the most powerful tools in Unix used for processing the rows and columns in a file.
Awk has built in string functions and associative arrays. Awk supports most of the operators,
conditional blocks, and loops available in C language. 
One of the good things is that you can convert Awk scripts into Perl scripts using a2p utility. 

The basic syntax of AWK:

awk 'BEGIN {start_action} {action} END {stop_action}' filename

Here the actions in the begin block are performed before processing the file and the actions in the
end block are performed after processing the file. The rest of the actions are performed while
processing the file.

Examples:

Create a file input_file with the following data. This file can be easily created using the output of ls -l.

-rw-r--r-- 1 center center  0 Dec  8 21:39 p1

-rw-r--r-- 1 center center 17 Dec  8 21:15 t1

-rw-r--r-- 1 center center 26 Dec  8 21:38 t2

-rw-r--r-- 1 center center 25 Dec  8 21:38 t3

-rw-r--r-- 1 center center 43 Dec  8 21:39 t4

-rw-r--r-- 1 center center 48 Dec  8 21:39 t5

From the data, you can observe that this file has rows and columns. The rows are separated by a
new line character and the columns are separated by a space characters. We will use this file as the
input for the examples discussed here. 

1. awk '{print $1}' input_file

Here $1 has a meaning. $1, $2, $3... represents the first, second, third columns... in a row
respectively. This awk command will print the first column in each row as shown below.

-rw-r--r--
-rw-r--r--

-rw-r--r--

-rw-r--r--

-rw-r--r--

-rw-r--r--

To print the 4th and 6th columns in a file use awk '{print $4,$5}' input_file

Here the Begin and End blocks are not used in awk. So, the print command will be executed for
each row it reads from the file. In the next example we will see how to use the Begin and End blocks.

2. awk 'BEGIN {sum=0} {sum=sum+$5} END {print sum}' input_file

This will prints the sum of the value in the 5th column. In the Begin block the variable sum is
assigned with value 0. In the next block the value of 5th column is added to the sum variable. This
addition of the 5th column to the sum variable repeats for every row it processed. When all the rows
are processed the sum variable will hold the sum of the values in the 5th column. This value is
printed in the End block.

3. In this example we will see how to execute the awk script written in a file. Create a file
sum_column and paste the below script in that file

#!/usr/bin/awk -f

BEGIN {sum=0}

{sum=sum+$5}

END {print sum}

Now execute the the script using awk command as 

awk -f sum_column input_file.

This will run the script in sum_column file and displays the sum of the 5th column in the input_file.

4. awk '{ if($9 == "t4") print $0;}' input_file

This awk command checks for the string "t4" in the 9th column and if it finds a match then it will print
the entire line. The output of this awk command is

-rw-r--r-- 1 pcenter pcenter 43 Dec  8 21:39 t4

5. awk 'BEGIN { for(i=1;i<=5;i++) print "square of", i, "is",i*i; }'

This will print the squares of first numbers from 1 to 5. The output of the command is

square of 1 is 1

square of 2 is 4

square of 3 is 9

square of 4 is 16

square of 5 is 25

Notice that the syntax of “if” and “for” are similar to the C language.

awk 'FNR==3;' list

Awk Built in Variables: 

You have already seen $0, $1, $2... which prints the entire line, first column, second column...
respectively. Now we will see other built in variables with examples. 

FS - Input field separator variable:

So far, we have seen the fields separted by a space character. By default Awk assumes that fields in
a file are separted by space characters. If the fields in the file are separted by any other character,
we can use the FS variable to tell about the delimiter. 

6. awk 'BEGIN {FS=":"} {print $2}' input_file

OR
awk -F: '{print $2} input_file

This will print the result as 

39 p1

15 t1

38 t2

38 t3

39 t4

39 t5

OFS - Output field separator variable: 

By default whenever we printed the fields using the print statement the fields are displayed with
space character as delimiter. For example

7. awk '{print $4,$5}' input_file

The output of this command will be

center 0

center 17

center 26

center 25

center 43

center 48

We can change this default behavior using the OFS variable as

awk 'BEGIN {OFS=":"} {print $4,$5}' input_file

center:0

center:17

center:26

center:25

center:43

center:48

Note: print $4,$5 and print $4$5 will not work the same way. The first one displays the output with
space as delimiter. The second one displays the output without any delimiter. 

NF - Number of fileds variable:

The NF can be used to know the number of fields in line

8. awk '{print NF}' input_file

This will display the number of columns in each row.

NR - number of records variable: 

The NR can be used to know the line number or count of lines in a file.

9. awk '{print NR}' input_file

This will display the line numbers from 1.

10. awk 'END {print NR}' input_file

This will display the total number of lines in the file.

String functions in Awk:

Some of the string functions in awk are:

index(string,search)
length(string)
split(string,array,separator)
substr(string,position)
substr(string,position,max)
tolower(string)
toupper(string)

Advanced Examples:

1. Filtering lines using Awk split function

The awk split function splits a string into an array using the delimiter.
The syntax of split function is
split(string, array, delimiter)

Now we will see how to filter the lines using the split function with an example.

The input "file.txt" contains the data in the following format

1 U,N,UNIX,000

2 N,P,SHELL,111

3 I,M,UNIX,222

4 X,Y,BASH,333

5 P,R,SCRIPT,444

Required output: Now we have to print only the lines in which whose 2nd column has the string
"UNIX" as the 3rd field( The 2nd filed in the line is separated by comma delimiter ).
The ouptut is:

1 U,N,UNIX,000

3 I,M,UNIX,222

The awk command for getting the output is:

awk '{

split($2,arr,",");

if(arr[3] == "UNIX")

print $0
} ' file.txt