Lecture 03: Shell

Environment & Commands

Dr. Vincent Emeakaroha
vincent.emeakaroha@cit.ie

Semester 2, 2019 COMP7044: Systems Scripting 1

 Environment Variables
• There are two types of variables:
• Local variables
• Environment variables
• Local variables
• Set and initialized by programmers in a script
• Environment variables are set by the operating system and can usually
be viewed using the env command.
• Environment variables hold special values. They are also known as Shell
variables
• Important for writing shell scripts

Semester 2, 2019 COMP7044: Systems Scripting 2

 Local Variables
• Bash uses variables as in any other programming languages. Their
values are always stored as strings, but there are mathematical
operators in the Bash language that will convert variables to numbers
for calculations.
• We have no need to declare a variable, just assigning a value to its
reference will create it.
• Example
#!/bin/bash
STR= Hello World!
echo $STR

• Line 2 creates a variable called STR and assigns the string "Hello World!" to it.
Then the value of this variable is retrieved by putting ‘$’ sign at the beginning.
Semester 2, 2019 COMP7044: Systems Scripting 3
 Variables: Warning
• Bash scripting does not check variable types. This means that a
variable can hold either number or character data.
• Example:
count=0
count=“Money”
• Changing the variable value TYPE can lead to confusion for the
programmer or someone trying to modify a script, so it is
recommended to use a variable for only a single TYPE of data value in
a script.
• \ is the bash escape character and it preserves the literal value of the
next character that follows it.
Semester 2, 2019 COMP7044: Systems Scripting 4
 Environment Variables
Name Meaning
$HOME Absolute path to your home directory
$PATH A list of directories to search for commands
$SHELL Absolute path to login shell
$LOGNAME/$USER Your user/login name
$PS1 Primary prompt
$PS2 Secondary prompt
$MAIL Absolute path to mailbox
$TERM Type of your terminal

Semester 2, 2019 COMP7044: Systems Scripting 5

 Additional Shell Features
• Tilde file expansion
~ $HOME
~username home directory of user
~+ $PWD
~- $OLDPWD

Semester 2 - 2019 COMP7044: Systems Scripting 6

 Setting PATH Variable
• Usually, we type in the commands in the following way:
./command

• By setting PATH=$PATH:. Our working directory is included in the

search path for commands, and we simply type:
command

• If we type in
mkdir ~/bin
and we include the following lines in the ~/.bash_profile or ~/.bashrc:
PATH=$PATH:$HOME/bin
export PATH

• We obtain that the directory /home/userid/bin is permanently

included in the search path for commands.
Semester 2 - 2019 COMP7044: Systems Scripting 7
 Shell Parameters
• Positional parameters are assigned from the shell’s argument when it is
invoked. Positional parameter “N” may be referenced as “${N}”, or as
“$N” when “N” consists of a single digit.
• Positional (special) parameters
• $# is the number of parameters passed
• $0 returns the name of the shell script running as well as its location
in the file system
• $* gives a single word containing all the parameters passed to the
script
• $@ gives an array of words containing all the parameters passed to the script

• Example
• sparameter.sh

Semester 2, 2019 COMP7044: Systems Scripting 8

 sparameter.sh
#!/bin/bash
echo "$#; $0; $1; $2; $*; $@"

• Calling it as:
./sparameters.sh arg1 arg2
• Output:
2; ./sparameters.sh; arg1; arg2; arg1 arg2; arg1 arg2

Semester 2, 2019 COMP7044: Systems Scripting 9

 Overview of Useful Commands

Semester 2, 2019 COMP7044: Systems Scripting 10

 File and Directory Management
• cd Change the current directory. With no arguments "cd" changes to
the users home directory. (cd <directory path>)
• chmod Change the file permissions.
• Example: chmod 751 myfile : change the file permissions to rwx for
owner, rx for group and x for others
• Example: chmod go+r myfile : Add read permission for the group and
others (character meanings u-user, g-group, o-other, + add permission,-
remove,r read,w-write,x-execute)
• Example: chmod +s myfile - Setuid bit on the file which allows the
program to run with user or group privileges of the setted user ID.

Semester 2, 2019 COMP7044: Systems Scripting 11

 File and Directory Management
• chown Change owner.
• Example: chown <owner1> <filename> : Change ownership of a file
to owner1.
• chgrp Change group.
• Example: chgrp <group1> <filename> : Change group of a file to group1.
• cp Copy a file from one location to another.
• Example: cp file1 file2 : Copy file1 to file2
• Example: cp –R dir1 dir2 : Copy dir1 to dir2
• md5sum Prints the MD5 Checksum
Semester 2, 2019 COMP7044: Systems Scripting 12
 File and Directory Management
• ls List contents of a directory.
• Example: ls, ls –l , ls –al, ls –ld, ls –R
• (-rwxrwxr-x 1 juan juan 0 Sep 26 12:25 foo )
• |more will list page wise

• mkdir Make a directory.

• Example: mkdir <directory name> : Makes a directory
• Example mkdir –p /www/chache/var/log will create all the directories
starting from www.
• mv Move or rename a file or directory.
• Example: mv <source> <destination>

Semester 2, 2019 COMP7044: Systems Scripting 13

 File and Directory Management
• pwd Print or list the present working directory with full path.
• rm Delete files (Remove files). (rm –rf <directory/file>)
• rmdir Remove a directory. The directory must be empty. (rmdir
<directory>)
• touch Change file timestamps to the current time. Make the file if it
doesn't exist. (touch <filename>)
• whereis Locate the binary and man page files for a command.
(whereis <program/command>)
• which Show full path of commands where given commands reside.
(which <command>)
Semester 2, 2019 COMP7044: Systems Scripting 14
 File Viewing and Editing
• pico Simple text editor.
• nano Command-line text editor.
• vi Editor with a command mode and text mode. Starts in command mode.
• gedit GUI Text Editor
• tail Look at the last 10 lines of a file (Default behaviour).
• Example: tail –f <filename> , Wait for input at file end
• Example: tail –n100 <filename>, Display last 100 entries
• head Look at the first 10 lines of a file (Default behaviour).
• Example: head <filename>

Semester 2, 2019 COMP7044: Systems Scripting 15

 File Manipulation and Viewing
• cat or more View / output the content of a file.
• Example: cat <filename>
• cmp Compare two files.
• Example: cmp <file1> <file2>, Compare byte by byte file1 and file2
• cut Remove sections from each line of files.
• diff Show the differences between files.
• Example: diff <file1> <file2>, Find differences between file1 & file2.
• echo Display a line of text.

Semester 2, 2019 COMP7044: Systems Scripting 16

 File Manipulation and Viewing
• sleep Delay for a specified amount of time.
• Example: sleep 5, Pause for 5 seconds
• sort Sort a file alphabetically.
• Example: sort <filename>, Sort the content of file alphabetically and
output result to the screen but original file remain unchanged.
• uniq Remove duplicate lines from a sorted file.
• wc Count lines, words, characters in a file.
• Example: wc –c/w/l <filename>, Count character/word/lines in the
file.

Semester 2, 2019 COMP7044: Systems Scripting 17

 File Compression and Restoring
• gzip - zip a file to a gz file.
• Example: gzip <filename>
• gunzip - unzip a gz file.
• Example: gunzip <gz filename>
• tar Archives files and directories. Can store files and directories on
tapes.
• Example: tar -zcvf <destination> <files/directories> -
Archive copy groups of files.
• Example: tar –zxvf <compressed file> to uncompress
• zip – Compresses a file to a .zip file.
• unzip – Uncompresses a file with .zip extension.
Semester 2, 2019 COMP7044: Systems Scripting 18
 Pipes
• An important early development in Unix was the invention of "pipes,"
a way to pass the output of one command to the input of another.
eg. $ who | wc −l
By combining these two commands, giving the wc command the
output of who, you can build a new command to list the number of
users currently logged in on the system.

Semester 2, 2019 COMP7044: Systems Scripting 19

 Input/Output (I/O) Redirection
• All I/O are handled by the kernel via a mechanism called file
descriptor
• A file descriptor is an index into a file-descriptor table maintained and
used by kernel to reference open files and I/O streams.
• The first three file descriptors are 0, 1 and 2.
• 0 is standard input (stdin)
• 1 is standard output (stdout)
• 2 is standard error (stderr)
• When a file descriptor is assigned to something else, it is called I/O
redirection.
Semester 2, 2019 COMP7044: Systems Scripting 20
 Basic I/O Redirection Commands
< sample.txt The command will take input from sample.txt

> sample.txt The output result will be stored in sample.txt

>> sample.txt The successive outputs will be appended to sample.txt

• Examples:
• ls > log.txt # Redirects the listing output into log.txt
• wc < log.txt # Counts the number of words, characters or lines from log.txt
• echo “Hello” >> content.txt # Appends “Hello” to the file content.txt

Semester 2, 2019 COMP7044: Systems Scripting 21

 Regular Expressions
• A "regular expression" is a pattern that describes a set of strings.

• Regular expressions are used when you want to search for specific
lines of text containing a particular pattern.

• An important means of specifying search options for grep command.

Semester 2, 2019 COMP7044: Systems Scripting 22

 What is grep?
• grep stands for
• "general regular expression parser“
• A powerful search command for UNIX
• Used to search for text strings and regular expressions within one or
more files
• Full details with the command:
• man grep

Semester 2, 2019 COMP7044: Systems Scripting 23

 Common grep Command Options
grep [options] pattern [files]
• -b Display the block number at the beginning of each line.
• -c Display the number of matched lines.
• -h Display the matched lines, but do not display the filenames.
• -i Ignore case sensitivity.
• -l Display the filenames, but do not display the matched lines.
• -n Display the matched lines and their line numbers.
• -s Silent mode.
• -v Display all lines that do NOT match.
• -w Match whole word.
grep -c jane my_file.txt
Semester 2, 2019 COMP7044: Systems Scripting 24
 Example Usage
• Search file for a user
• grep bill /etc/passwd
• Search file ignoring word case
• grep -i bill /etc/passwd
• Search recursively all files and directories under given directory
• grep -r bill /etc/

Semester 2, 2019 COMP7044: Systems Scripting 25

 Example Usage
• Search for a specific word in file
• grep -w jane $HOME/course.txt
• Search for 2 different words in file
• grep -w ‘jane\|victor' $HOME/course.txt
• Count lines that matched in file
• grep -c 'word' $HOME/course.txt

Semester 2, 2019 COMP7044: Systems Scripting 26

 Example Usage
• Display lines that did not match a pattern
• grep -v ‘mark’ $HOME/course.txt
• Number of lines that contain matched pattern
• grep -n 'word' $HOME/course.txt
• Display filenames that matched pattern, but not lines from the files
• grep -l ‘word' *.txt

Semester 2, 2019 COMP7044: Systems Scripting 27

 Using Wildcards in grep
• Dot ( . ) matches 1 character
• Asterisks ( * ) matches multiple characters
• Examples:
• grep b.g myfile è finds the words “big”, “bag”

• grep b*k myfile è finds the word “back”, “buck”, “book”

Semester 2, 2019 COMP7044: Systems Scripting 28

 Regular Expression and grep
• ^ (Caret) = match expression at the start of a line.
• Example: ^A simple man -> matches this line.
• $ (Dollar Sign) = match expression at the end of a line.
• Example: Sample$ -> matches this line.
• \ (Back Slash) = turn off the special meaning of the next character.
• Example: \^A simple man -> does not recognise this as regular expression.
• [ ] (Brackets) = match any one of the enclosed characters.
• Example: [aeiou] -> matches any of the characters.
• Use Hyphen "-" for a range. E.g., [0-9].
• [^ ] = match any one character except those enclosed in [ ].
• Example: [^0-9] -> ignores the range of 0 to 9.

Semester 2, 2019 COMP7044: Systems Scripting 29

 Regular Expression and grep
• . (Period) = match a single character of any value, except end of line.
• * (Asterisk) = match zero or more of the preceding character or
expression.
• \{x,y\} = match x to y occurrences of the preceding.
• \{x\} = match exactly x occurrences of the preceding.
• \{x,\} = match x or more occurrences of the preceding.

Semester 2, 2019 COMP7044: Systems Scripting 30

 Regular Expression Usage Examples
• grep bob files {search files for lines with ‘bob'}
• grep '^bob' files {‘bob' at the start of a line}
• grep ‘bob$' files {‘bob' at the end of a line}
• grep '^bob$' files {lines containing only ‘bob'}
• grep '\^b' files {lines starting with '^b', "\" escapes the ^}
• grep '[Bb]mug' files {search for starting with ‘B' or ‘b'}
• grep 'B[oO][bB]' files {search for BOB, Bob, BOb or BoB }
• grep '^$' files {search for empty lines}
• grep '[0-9][0-9]' files {search for pairs of numeric digits}

Semester 2, 2019 COMP7044: Systems Scripting 31

Semester 2, 2019 COMP7044: Systems Scripting 32