Training Material

Unix Commands in a Nutshell

Practical Usage of Commands

(Please open this PPT in openOffice.org)

DC/ SC/ Geo/Practice: Hi-Tech

Name of the project/support fn: Numonyx
Name of the author: Rohit Chopra
Date Created:16-Jul-2009

July 17, 2009

Objectives

• Introduction
• Unix Shell
• Unix commands
• Vi Editor
• String Manipulation
• Unix Utilities Monitoring Tools
• Scheduling Jobs

July 17, 2009

Flavors of UNIX
Product Name Company
Xenix Microsoft’s UNIX operating system
to run on microcomputers
IBM AIX IBM

Zeus Zilog Computers

DEC Ultrix DEC

HP-UX HP

Solaris SUN

Linux Red Hat

Darwin Mac OSX

July 17, 2009

 UNIX Features

Feature Description

Multitasking Performing tasks simultaneously rather than sequentially

e.g., While printing a document , start editing another document
Each task is considered to be a process
Multi-user Several users can use the same computer simultaneously
i.e. , more than one keyboard and terminal can be connected
to one computer
Highly cost-effective
Portability Easier to modify the UNIX system code for installation on a new
computer rather than write from scratch a new operating system

July 17, 2009

UNIX Kernel

File Management
Networking And Security
Services

Input / Output
Date and Time
Services
Services

UNIX
SYSTEM Process
Signal
Scheduling
Handling KERNEL

System
Memory
Administration
Management
and Accounting

July 17, 2009

Different types of unix Shell

Bourne Shell (sh)

C Shell (csh)

TC Shell (tcsh)

Korn Shell (ksh)

Bourne Again Shell (bash)

July 17, 2009

 Unix Shell

Bourne Shell (sh)

This is the original UNIX shell

This shell doesn’t have any interactive features.

C Shell (csh)

This shell provides C-like language for shell scripts.

All features of modern shell like command line history, alias are included in this shell

TC Shell (tcsh)

Similar to C shell but have emacs style editing

All features of modern shell are available.

July 17, 2009

 Unix Shell
Korn Shell (ksh)

This shell is now standard shell on UNIX systems

This shell provides all the features of C shell along with scripting similar to original Bourne shell (sh)

All features of modern shell are available.

Unix Shells

Bourne Again Shell (bash)

This is the first public domain shell

This shell is full implementation of IEEE Posix shell and tool specifications.

All features of modern shell like command line history, alias, command line editing etc are included in this shell.

July 17, 2009

Classification of files in Unix

- Ordinary Files

- Directory Files

- Special Files

- Standard Files

July 17, 2009

Files and Directories

• A file can be addressed as container of data.

• Directory is a container of files.

• Every file is assigned to a Directory

$ls -ld /usr/bin

drwxr-xr-x 4 bin bin 17408 May 25 03:51 /usr/bin

$ls -ld /etc/passwd

-rw-r--r-- 1 root security 6374 May 09 07:56 /etc/passwd
$

July 17, 2009

Concept of Special Files

* Used to reference Physical devices such as terminals and disks

* Read and written like ordinary files

• Device Drivers associated with each file

$ls -ld /dev/null

crw-rw-rw- 1 root system 2, 2 Aug 14 18:30 /dev/null
$ls -ld /dev/cd0
br--r--r-- 1 root system 14, 0 Oct 28 2005 /dev/cd0
$ls -ld /dev/tty
crw-rw-rw- 1 root system 1, 0 Aug 14 18:29 /dev/tty
$

July 17, 2009

 Concept of Standard Files

• Helps display information on the screen

* Special Names for Communication channels

* Keyboard input channel is called Standard input

(stdin) - file id is 0

* Terminal Screen output is called Standard output

(stdout) - file id is 1

• Diagnostic error messages (generated by a program) are sent to

Standard error
(stderr) - file id is 2 ( shown on terminal screen )

* All three files are open by default at the time of login

July 17, 2009

Features
* Hierarchical

* Ownership on each file

- User / Owner
- Group
- Others

* Separate security for permissions

- read
- write and
- execute

* Removable

* File Independence

* Time stamp on each file

- Modification time
- Access time

July 17, 2009

Security and Access Methods

* Three levels of access

User/Owner, group, others

* Three types of access on Files and Directories

Read, Write, Execute

Access Mode Ordinary Directory

Read Examination of Listing of

File Contents files within
Directory

Write Allows changing Creating new

of file contents files within
Directory

Execute Executing file as Searching the

a command Directory allowed
allowed

July 17, 2009

Useful Commands

mkdir creates a new directory

rm removes a file

rmdir removes a directory

du displays disk usage

df displays number of free block

touch updates the time of last modification

find locates files that match certain area

file displays the type of file

pwd displays full pathname of current directory

July 17, 2009

File Contents
cat Concatenate & Print on screen or printer
$cat [Options] [Arguments]

Options - take input from stdin

-n no. of output lines
-s squeeze adj. blank lines
-v enable display of non-printing
characters
-b used with -n to avoid numbering
blank lines

e.g.,

$ cat try.c Display the contents of try.c on the screen

$ cat Takes input from stdin i.e. keyboard and

displays on screen

July 17, 2009

 Usages of cat command

$ cat f1 > f2 Takes input from file f1 & puts it

on file f2
$ cat f2 > f3 f3 contains the contents of f1
$ cat f4 >> f3 Appends the contents of f4 to file f3
$ cat try[0-3] > final The file final contains contents
of try0, try1, try2 try3
$ cat test* > report The file report contains all files
beginning with test

July 17, 2009

UNIX Commands

$ Is -I List the files along with the protection bits and the user

$ Is -1 symtab.c
symtab.o
treegen
test

$ Is -I -rw-r—r— 1 smj proj1 30766 Mar 3 15:08 symtab.c

-rw-r—r— 1 smj proj1 8759 Mar 3 15:12 symtab.o
-rwxr-xr-x 4 smj proj1 40743 Mar 3 15:23 treegen
drwxrwxr-x 1 smj proj1 53 Mar 1 09:15 test

$ Is -a List the files starting with .and ..also / hidden files

.
..
.profile
.cshrc
symtab.c
...

$ Is -iI 10936-rw-r—r—I smj proj1 3076 Mar 3 15:08 test.c

10936 - inode number of file test.c

July 17, 2009

Unix Command Continues...

wc Word Count
$wc [Options] filename

Options - Display no. of lines, words, characters

-I Display no. of lines
-w Display no. of words
-c Display no. of characters
e.g., $ wc test.c
20 200 5678
20 - lines
200 - words
5678- characters

July 17, 2009

Unix Command Continues...
cp copy a file

-i - user interactive mode

e.g.,
$ cp test.c test.c.bak
test.c and test.c.bak contain the same contents
Extra disk storage

In Create link

e.g.,

$ ln first.c second.c
The file is referenced by two different names
No Extra disk storage

July 17, 2009

Unix Command Continues...

mv Moves or renames files and directories

-i interactive mode
e.g.,
$ mv old.c new.c
Renames the file old.c as new.c
rm Deletes the indicated file(s) files

rm removes files and directories

-i remove interactively
-f forcible remove
-r remove recursively
• Dangerous
• used in conjunction with -i

touch Updates access, modification or change times of a file

-a update access time

-m update modification time
-c prevents creating the file
e.g., $ touch f1
* The current system date & time stamp is put on the file f1
* If f1 does not exist then it is created with 0 bytes

July 17, 2009

File Comparison

cmp Compare two files

If files are same no output is sent to the terminal, or else The line number
and the byte at which the first difference occurs is reported

-s Outputs nothing Registers return code

Return code

0 if files are identical

1 if files are different
2 on error

e.g.,
$ cmp test1 test2
test1 and test2 differ in char 36 line 3
$ cmp -s test1 test2
$ echo $?

outputs 1 indicating that the files are different

July 17, 2009

File Comparison continues..

diff - Reports more than one differences

$diff [Options] file1 file2

-b Ignores trailing blanks

-e Gives a list of ed commands so as to convert

file1 into file2.

e.g., $ diff test1 test2

Outputs: n1 a n3,n4
n1,n2 d n3
n1,n1 c n3,n4

where * n1 ,n2, n3 ,n4 are line numbers

* a ,d, c means append, delete ,change respectively

July 17, 2009

File Comparison continues..

comm Display common lines

$comm -[123] f1 f2
Prints a three column output:
- lines that occur only in f1
- lines that occur only in f2
- lines that occur in both
comm -12 - prints lines common to the two files
comm -23 - prints only lines in the first file but not in the
second
comm -123 - prints nothing
e.g.,

$ comm test1 test2Reports the common lines between

files test1, test2 and reports the lines differing

$ comm -12 test1 test2 Prints line common to both

July 17, 2009

File Compression

pack Compress the file

$ pack <filename>

e.g., $ pack try

- Creates a file try.z which is packed
- Normally the huge logs are packed
- The size is reduced by 25 - 40 %

unpack Uncompress packed file

or
pcat

e.g., $ unpack try.z

unpacks the file try.z
or
$ pcat try.z
read the contents of try.z

July 17, 2009

File Security

passwd To change the password or your

shell

chown To change the ownership of the file

$ chown owner filename

e.g., $ chown trg2 test.c

* Initially the owner is trg1
* Only the owner or the superuser can
change the ownership of the file

July 17, 2009

 File Security Continues...
umask Set file creation mode mask
$ umask nnn
(nnn set file creation mode)
umask can also be set as a shell variable
e.g.,
umask 022
- Files normally created with 666 mode is assigned
644 permission

The value of each digit is subtracted from the corresponding "digit" specified by
the system for the creation of a file.

July 17, 2009

File Operation

tail Displays the last lines of file

options : -n (n= no. of lines)
e.g., $ tail -30 test.c
Displays the last 30 lines of file test. c

head Displays the top lines of file

e.g., $ head -10 test.c Displays the first 10 lines of test.c

split Splits the file into different files as specified by the number of lines
e.g., $ split -20 test.c
Splits the file test.c in blocks of 20 lines and creates files xaa, xab
and so on, such that
xaa has first 20 lines of test.c
xab has the next 20 lines of test.c
...
The file test.c is unaffected
$ split-20 test.c try Generates files as tryaa , tryab , tryac

paste Joins the two or more files horizontally

e.g., $ paste xaa xab
File xaa and xab are joined horizontally and output to the terminal

July 17, 2009

The vi - Visual Editor

The editor
* Powerful full screen editor
* vi v/s ed, ex
* Mostly single key stroke commands
* Interface with ‘ex’
* Macro facility
* Ability to process text

Invoking $ vi filename
e.g., $ vi pgm.c

Modes :
* Command mode
* Insert mode
* From Insert mode Pressing <ESC>
remitts Command mode

July 17, 2009

Types of commands
* vi-commands (invisible)

Command mode

Commands can be categorised as :

* Cursor movement
* Text manipulations
- insert, delete, copy, change
* Marking/Selecting, Positioning
* Search

Objects of interest recognized by ‘vi’:

* characters
* words
* lines
* block

July 17, 2009

Format of commands

[count] command [operand]

Use “.” to repeat last command
Use “u” to undo last command

Cursor Movement
Line Oriented :

^ or (zero) beginning of line

$ end of line

Character oriented
h move left
l move right
j move down
k move up

July 17, 2009

 Format of commands (contd.)
Word Oriented :

e move to end of next word

w move to beginning of word

b move to beginning of previous word

E move to end of next word

ignoring punctuation

W move to beginning of word ignoring

punctuation

B move to previous word ignoring

punctuation

July 17, 2009

Text Manipulation

Searching for a string :

/string for searching forward

?string for searching backward

Use n to continue search in the same direction

Use N to continue search in the opposite direction

s/oldstr/newstr commands can be used in vi

July 17, 2009

Common Set commands

set ai Causes automatic indentation

set noai Nullifies the effect of auto-indent

set nu Causes line numbers to be displayed

set wrapmargin = n Sets n column right margin

set bf The beautify option Removes all

unimportant control characters

set Causes a displays of current set options

July 17, 2009

Shell Utilities and Commands

July 17, 2009

 Shell Startup Dot files

• Helps in customizing UNIX System

* Executes .profile for bsh, ksh and bash when user logs on

* Executes .login and .cshrc for C Shell when user logs on

* Executes .logout for C Shell when user logs out from C-Shell

* Keeps track of the history mechanism In C Shell through the .history

July 17, 2009

Shell Variables
User defined
assignment : name = value

e.g., (on Bourne shell)

$ x=”Hello how are you”

$ echo $x
Hello how are you

$ PATH=$PATH:/usr/games
$ export PATH
$ echo $PATH

:.:/usr/bin:/bin:/usr/games:

July 17, 2009

 Predefined Special Parameters

$# number of positional parameters

$? exit value of last command

$$ process number of a process

$* all the parameters

$! process id of last command started with

asynchronous execution i.e., &

July 17, 2009

Variables Used by Shell in .profile

HOME login dir

PATH search path for commands

PS1 primary system prompt

PS2 secondary system prompt (eg. With while

statements)

MAIL files containing mail messages

TERM terminal type

July 17, 2009

 To algebraically compare variable values

test $num1 -eq $num2

Other algebraic operators -ne, -gt -ge -lt, -le

File related

$ test -f file True if file exists and is an ordinary file

$ test -r file True if file exists and is readable file

$ test -w file True if file exists and is writable file

$ test -s file True if file exists and has size greater than zero

July 17, 2009

 FILTERS :

grep get regular expressions only

fgrep fast, several simple strings at one time

egrep extended grep, can handle more powerful

expressions like | - or operators

July 17, 2009

 Searching Files with grep

e.g.,

$ grep -ni func1 *.c Prints all the lines and line
numbers in files *.c that match pattern func1
ignoring the case

$ grep ‘*’ * Search for the pattern * in all the

files

$ ls -l | grep ‘^d’ Searches for all subdirectories

July 17, 2009

 Processing Tabular Data

Cut

* Deletes columns from a file producing a new file with

shorter lines

* Cuts out selected fields of each line in a file.

* Cuts columns from a table or fields from a file which can

be of type

- Fixed length fields or

- Delimited by some character

July 17, 2009

 Cut (contd.)
cut -c list { file1 file2 ...}
e.g.,
cut -c 5-70 file1
cut would pass 5-70 characters from file1

cut -flist {-d char } {file1 file2....}

e.g.,
who | cut -d” “ -f1
gives a list of user login names
cut -d: -f 1,5 /etc/passwd
gives a list of user ID and names

July 17, 2009

PS, kill
$ somecommand &

5511 - pid

$ps

pid tty time command

3432 2 0 : 24 -sh
5765 2 0 : 03 ps
5511 2 0 : 51 somecommand

$ kill 5511
$ ps
pid tty time command

3432 2 0 : 24 -sh
5985 2 0 : 03 ps

July 17, 2009

$ stubborn-cmd &

pid tty time command

3432 2 0 : 24 -sh
6004 2 0 : 03 ps
5995 2 0 : 44 stubborn-cmd
$ kill 5995

$ ps
pid tty time command
3432 2 0 : 24 -sh
6004 2 0 : 03 ps
5995 2 0 : 44 stubborn-cmd
$ kill -9 5995

$ ps
pid tty time command
3432 2 0 : 24 -sh
6103 2 0 : 03 ps

July 17, 2009

Communication Features

1. User communication commands

mail
write
talk
post
news
mesg
wall
2. Networking commands
uucp
uuto
rlogin
telnet
ftp
rcp
uux
sftp
scp

July 17, 2009

Practical Use of the Command (cat)

$cat /etc/passwd > temp

$sort < temp
$ rm temp
These statements first create a temporary file, Sort the file
and then delete the temporary file.This same result can be
achieved through a single unix statement.

$ cat /etc/passwd | sort

This requires No need of creation/deletion of a file and then sorts
the filepasswd as per the first entry in passwd.

July 17, 2009

Practical Use of the Command (find)

find . -name text.xml

This command will search a file text.xml from the
current path of the directory.
find / -atime -30 -print
This command will list all the files accessed less
30 days ago.
find / -atime +30 -print
This command will list all the files accessed after
30 days

July 17, 2009

Practical Use of the Command (find)

find . -type f -print | xargs grep -il StringName

This is a very useful command which is a
combination of find and grep commands. This
command will find a file which contains a matching
StringName in it.

find /directory -mtime +30 -print -exec rm {} \;

This command will remove all the files in /directory that have
not been modified in 30 days.

July 17, 2009

Practical Use of the Command (find)

find $SEARCH_PATH -size +5000000c -print

To search for files greater than 5MB we can use the
above command where the + (plus sign) specifies greater than,
and the c denotes bytes

find $SEARCH_PATH -mtime 1 -print

This command will find all files that have been modified, or
created, in the last 24 hours

TS=$(date +%m%d%y%H%M%S)
This date command gives two digits for month, day, year,
hour, minute, and second:

July 17, 2009

 Practical Use of the Command (SED)

chsh username new_default_shell

The chsh command, short for change shell, allows you to modify
the system environment for your login

SED is used to edit a large number of configuration files .

for each in `ls *.cfg`; do
mv $each $each.bak #Safety First!
sed '30,35s/^/\#/g' $each.bak > $each
done
This will flow through all of the config files in a certain directory
and add a # sign at the beginning of lines 30 through 35,
commenting those lines out and if all goes
well, delete all of the .bak files created as backups by the script.

July 17, 2009

Practical Use of the Command (AWK)

awk ' '{print $1}' file

This command is used to extract the first column from a file.
AWK command allows the user to manipulate files that are
structured as columns of data and strings.

ls files_list | awk '{print "mv "$1" "$1".new"}' | sh

This command will help in renaming the files (append .new
to "files_list"):

July 17, 2009