UNIX Shell-Scripting Basics

Agenda
 What is a shell? A shell script?  Introduction to bash  Running Commands  Applied Shell Programming

What is a shell?

What is a shell?

/bin/bash

What is a shell?

#!/bin/bash

What is a shell?
INPUT

shell

OUTPUT

ERROR

What is a shell?
 Any Program  But there are a few popular shells

Bourne Shells

 /bin/sh  /bin/bash Bourne-Again Shell

Steve Bourne

Other Common Shells

 C Shell (/bin/csh)  Turbo C Shell (/bin/tcsh)  Korn Shell (/bin/ksh)

An aside: What do I mean by /bin ?

 C Shell (/bin/csh)  Turbo C Shell (/bin/tcsh)  Korn Shell (/bin/ksh)

An aside: What do I mean by /bin ?

 /bin, /usr/bin, /usr/local/bin  /sbin, /usr/sbin, /usr/local/sbin  /tmp  /dev  /home/borwicjh

What is a Shell Script?

 A Text File  With Instructions  Executable

What is a Shell Script?

% cat > hello.sh <<MY_PROGRAM #!/bin/sh echo Hello, world MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world

What is a Shell Script? A Text File

% cat > hello.sh <<MY_PROGRAM #!/bin/sh echo Hello, world MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world

An aside: Redirection
 cat > /tmp/myfile  cat >> /tmp/myfile  cat 2> /tmp/myerr  cat < /tmp/myinput  cat <<INPUT Some input INPUT  cat > /tmp/x 2>&1

INPUT 0 env OUTPUT 1 ERROR 2

What is a Shell Script? How To Run

% cat > hello.sh <<MY_PROGRAM #!/bin/sh echo Hello, world MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world

What is a Shell Script? What To Do

% cat > hello.sh <<MY_PROGRAM #!/bin/sh echo Hello, world MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world

What is a Shell Script? Executable

% cat > hello.sh <<MY_PROGRAM #!/bin/sh echo Hello, world MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world

What is a Shell Script? Running it

% cat > hello.sh <<MY_PROGRAM #!/bin/sh echo Hello, world MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world

Finding the program: PATH

 % ./hello.sh  echo vs. /usr/bin/echo  % echo $PATH /bin:/usr/bin:/usr/local/bin: /home/borwicjh/bin  % which echo /usr/bin/echo

Variables and the Environment

% hello.sh bash: hello.sh: Command not found % PATH=$PATH:. % hello.sh Hello, world

An aside: Quoting
% echo $USER $USER % echo $USER borwicjh % echo \ % echo deacnet\\sct deacnet\sct % echo \ \

Variables and the Environment

% env [variables passed to sub-programs] % NEW_VAR=Yes % echo $NEW_VAR Yes % env [PATH but not NEW_VAR] % export NEW_VAR % env [PATH and NEW_VAR]

Welcome to Shell Scripting!

How to Learn
 man
man bash man cat man man

 man k
man k manual

 Learning the Bash Shell, 2nd Ed.  Bash Reference Cards  http://www.tldp.org/LDP/abs/html/

Introduction to bash

Continuing Lines: \
% echo This \ Is \ A \ Very \ Long \ Command Line This Is A Very Long Command Line %

Exit Status
 $?  0 is True
% ls /does/not/exist % echo $? 1 % echo $? 0

Exit Status: exit

% cat > test.sh <<_TEST_ exit 3 _TEST_ % chmod +x test.sh % ./test.sh % echo $? 3

Logic: test
% % 0 % % 1 test 1 -lt 10 echo $? test 1 == 10 echo $?

Logic: test
 test [ ]
[ 1 lt 10 ]

 [[ ]]
[[ this string =~ this ]]

 (( ))
(( 1 < 10 ))

Logic: test
[ [ [ [

-f /etc/passwd ] ! f /etc/passwd ] -f /etc/passwd a f /etc/shadow ] -f /etc/passwd o f /etc/shadow ]

An aside: $(( )) for Math

% echo $(( 1 + 2 )) 3 % echo $(( 2 * 3 )) 6 % echo $(( 1 / 3 )) 0

Logic: if
if something then : # elif a contraction of else if: elif something-else then : else then : fi

Logic: if
if [ $USER eq borwicjh ] then : # elif a contraction of else if: elif ls /etc/oratab then : else then : fi

Logic: if
# see if a file exists if [ -e /etc/passwd ] then echo /etc/passwd exists else echo /etc/passwd not found! fi

Logic: for
for i in 1 2 3 do echo $i done

Logic: for
for i in /* do echo Listing $i: ls -l $i read done

Logic: for
for i in /* do echo Listing $i: ls -l $i read done

Logic: for
for i in /* do echo Listing $i: ls -l $i read done

Logic: C-style for

for (( expr1 expr2 expr3 do list done ; ; ))

Logic: C-style for

LIMIT=10 for (( a=1 ; a<=LIMIT ; a++ )) do echo n $a done

Logic: while
while something do : done

Logic: while
a=0; LIMIT=10 while [ "$a" -lt "$LIMIT" ] do echo -n "$a a=$(( a + 1 )) done

Counters
COUNTER=0 while [ -e $FILE.COUNTER ] do COUNTER=$(( COUNTER + 1)) done

 Note: race condition

Reusing Code: Sourcing

% cat > /path/to/my/passwords <<_PW_ FTP_USER=sct _PW_ % echo $FTP_USER % . /path/to/my/passwords % echo $FTP_USER sct %

Variable Manipulation
% FILEPATH=/path/to/my/output.lis % echo $FILEPATH /path/to/my/output.lis % echo ${FILEPATH%.lis} /path/to/my/output % echo ${FILEPATH#*/} path/to/my/output.lis % echo ${FILEPATH##*/} output.lis

It takes a long time to become a bash guru

Running Programs

Reasons for Running Programs

 Check Return Code
$?

 Get Job Output

OUTPUT=`echo Hello` OUTPUT=$(echo Hello)

 Send Output Somewhere

Redirection: <, > Pipes

Pipes
 Lots of Little Tools
echo Hello | \ wc -c
INPUT 0 echo OUTPUT 1 ERROR 2

A Pipe!
INPUT 0 wc OUTPUT 1 ERROR 2

Email Notification
% echo Message | \ mail s Heres your message \ borwicjh@wfu.edu

Dates
% DATESTRING=`date +%Y%m%d` % echo $DATESTRING 20060125 % man date

FTP the Hard Way

ftp n u server.wfu.edu <<_FTP_ user username password put FILE _FTP_

FTP with wget

 wget \ ftp://user:pass@server.wfu.edu/file  wget r \ ftp://user:pass@server.wfu.edu/dir/

FTP with curl

curl T upload-file \ -u username:password \ ftp://server.wfu.edu/dir/file

Searching: grep
% % % % % grep grep grep grep grep rayra /etc/passwd r rayra /etc r RAYRA /etc ri RAYRA /etc rli rayra /etc

Searching: find
% find /home/borwicjh \ -name *.lis [all files matching *.lis] % find /home/borwicjh \ -mtime -1 name *.lis [*.lis, if modified within 24h] % man find

Searching: locate
% locate .lis [files with .lis in path] % locate log [also finds /var/log/messages]

Applied Shell Programming

Make Your Life Easier

 TAB completion  Control+R  history  cd  Study a UNIX Editor

pushd/popd
% cd /tmp % pushd /var/log /var/log /tmp % cd .. % pwd /var % popd /tmp

Monitoring processes
 ps  ps ef  ps u oracle  ps C sshd  man ps

DOS Mode Files

 #!/usr/bin/bash^M  FTP transfer in ASCII, or  dos2unix infile > outfile

sqlplus
JOB=ZZZTEST PARAMS=ZZZTEST_PARAMS PARAMS_USER=BORWICJH sqlplus $BANNER_USER/$BANNER_PW << _EOF_ set serveroutput on set sqlprompt "" EXECUTE WF_SATURN.FZ_Get_Parameters('$JOB', '$PARAMS', '$PARAMS_USER'); _EOF_

sqlplus
sqlplus $USER/$PASS @$FILE_SQL \ $ARG1 $ARG2 $ARG3 if [ $? ne 0 ] then exit 1 fi if [ -e /file/sql/should/create ] then [use SQL-created file] fi
 Ask Amy Lamy!

Passing Arguments
% cat > test.sh <<_TEST_ echo Your name is \$1 \$2 _TEST_ % chmod +x test.sh % ./test.sh John Borwick ignorethis Your name is John Borwick

INB Job Submission Template

$1: user ID $2: password $3: one-up number $4: process name $5: printer name % /path/to/your/script $UI $PW \ $ONE_UP $JOB $PRNT

Scheduling Jobs
% 0 0 * 0 % % crontab -l 0 * * * daily-midnight-job.sh * * * * hourly-job.sh * * * * every-minute.sh 1 * * 0 1AM-on-sunday.sh EDITOR=vi crontab e man 5 crontab

Other Questions?
 Shells and Shell Scripts  bash  Running Commands  bash and Banner in Practice