Introduction to UNIX

UNIX & LINUX Fundamentals

1
 Purpose of This Lecture

● Fundamentals of using UNIX and UNIX-like

systems
● History of UNIX & Linux
● Basic system commands
● Data Management
● Constructing basic shell scripts

2
 History of UNIX
● "...the number of UNIX installations has grown to 10,
with more expected..."
– Dennis Ritchie and Ken Thompson, June 1972
● "... When BTL withdrew from the project, they
needed to rewrite an operating system (OS) in order
to play space war on another smaller machine (a
DEC PDP-7 [Programmed Data Processor] with 4K
memory for user programs). The result was a system
which a punning colleague called UNICS (UNiplexed
Information and Computing Service)--an
'emasculated Multics'; no one recalls whose idea the
change to UNIX was"

3
 Early Movers and Shakers
● Dennis Ritchie & Ken Thompson
● PDP11

4
 And then there was C…
● in 1972, Ritchie rewrote B and called the
new language C; Thompson created the
pipe--a "Write programs that do one thing
and do it well. Write programs to work
together. Write programs that handle text
streams, because that is a universal
interface."

5
 Bringing UNIX to the desktop
● Unix was very expensive
● Microsoft DOS was the mainstream OS
● MINIX, tried but was not a full port

● An Open Source solution was needed!

6
 Linux 0.02 – October 5, 1991
● “Do you pine for the nice days of minix-1.1, when
men were men and wrote their own device drivers?
Are you without a nice project and just dying to cut
your teeth on a OS you can try to modify for your
needs? Are you finding it frustrating when everything
works on minix? No more all-nighters to get a nifty
program working? Then this post might be just for
you :-)” - Linus Benedict Torvalds
● "I still maintain the point that designing a monolithic
kernel in 1991 is a fundamental error. Be thankful
you are not my student. You would not get a high
grade for such a design :-)"
(Andrew Tanenbaum to Linus Torvalds)

7
 1990’s Movers and Shakers
● Richard Stallman, father of the GNU Project
● Linus Torvalds

8
 Why UNIX/LINUX is Still Used
● 30+ years of development
● Many academic, scientific, and system tools
● Open Source
● System Stability
● Lightweight
● Easy Development

9
 The Basics
● Command-Line
– Interaction with UNIX & LINUX is based on
entering commands to a text terminal
– Oftentimes there are no ‘warnings’ with
commands, no ‘undo’
● The Shell
– The user environment that enables interaction
with the kernel, or lower-system OS.
– Windows Explorer would be a shell for Microsoft
Windows.

10
 Common Shells
● sh – the original UNIX shell; still located in /bin/sh
● Bash is a Unix shell written for the GNU Project and
is installed on most Linux systems.
● csh – C Shell, modeled after the C programming
language used by UNIX systems
● tcsh – C Shell with modern improvements such as
filename completion

● echo $SHELL – displays what shell your account is

using
● chsh – change your shell

11
 Before we go further…
● Read the Manual.
– man command
– man [section] command
– man –k keyword (search all manuals based on
keyword)

● Most commands have a built-in UNIX manual, even

the man command!

● Commands without manuals have help too, with –h,

--help, or /? option.

12
 The Manual
● The manual pages are divided into eight sections
depending on type of command.

– 1 commands and applications

– 2 system calls
– 3 C library functions
– 4 special files
– 5 file formats
– 6 games
– 7 misc.
– 8 system administration utilities

13
 Conventions for this lecture
● This lecture is too short to give you all of the
options, so look at the manual for specific
syntax for commands.
● Commands will be in bold, options will be in
italics.
– command -arguments
● Output will be shown in its own bordered
table

14
 Command Conventions
● In help files and manuals, commands will
have required input and option input

● cp [OPTION] source destination

– Optional arguments are in brackets, required
arguments are not.
● cp –R or cp - -recursive
– Short options ‘-’, long options ‘- -’

15
 Who’s Path is it Anyway?
● UNIX treats the directory structure as a
hierarchy of individual paths

dev home usr

bhecker bin

16
 Directories
● What is a working directory?
– The directory your shell is currently associated
with. At anytime in the system your login is
associated with a directory

● pwd – view the path of your working

directory
● ls – view your working directory

17
 Finding your home
● Each user has a home directory which can be found
with:
– cd
– cd ~bhecker
– cd $HOME

● The tilde character ‘~’ will tell the shell to auto-

complete the path statement for the cd command
● $HOME refers to an environment variable which
contains the path for home.

18
 More file commands

● cd directory
- change your current working directory to the new path
● ls –a
– Show hidden files
– Hidden files are files that begin with a period in the filename ‘.’
● mv
– moves one file to another
● cp
– copies files or directories
● rm
– remove files & directories
– rm –rf
● remove everything with no warnings

– rm –rf *
● most dangerous command you can run!

19
 Recursive Directories
● Often times a manual will refer to ‘recursive’
actions on directories. This means to
perform an action on the given directory and
recursively to all subdirectories.
– cp –R source destination
● Copy recursively all directories under source to
destination

20
 Relative vs. Absolute Path
● Commands expect you to give them a path to a file. Most
commands will let you provide a file with a relative path, or a
path relative to your working directory.

– ../directory - the ‘..’ refers to looking at our previous directory first

– ./executable - ‘.’ says this directory, or our working directory

● Absolute, or Full paths are complete. An easy way to know if a

path is absolute is does it contain the ‘/’ character at the
beginning?

– /home/user/directory/executable - a full path to file executable

21
 Poking around in $home
● How much space do I have?
– Quota
● command to see all quotas for your directories are,
if any.
● How much space am I taking up?
– du
● command to find out how much space a folder or
directory uses.
– df
● display space information for the entire system

22
 Helpful Hints on Space
● Almost all commands that deal with file
space will display information in Kilobytes,
or Bytes. Nobody finds this useful.
● Many commands will support a ‘-h’ option
for “Human Readable” formatting.
– ls –lh
● displays the working directory files with a long
listing format, using “human readable” notation for
space

23
 Representing Space
● Bit – either a 1 or 0
● Byte – 8 bits “0000 1111”, or “x0F”, or 16
● KB, Kilobyte - 1024 Bytes
● MB, Megabyte – 1024KB
● GB, Gigabyte – 1024MB
● TB, Terabyte – 1024 GB
● PB, Petabyte – 1024 TB

24
 Permissions

● The *NIX systems are multi-user environments

where many users run programs and share data.
Files and directories have three levels of
permissions: World, Group, and User.

● The types of permissions a file can contain are:

Read permissions Write Permissions Execute Permissions

r w x

25
 Permissions Cont.

● File permissions are arranged in three

groups of three characters.
● In this example the owner can read & write
a file, while others have read access

User (owner) Group Others (everyone else)

rw- r-- r--

26
 Changing Permissions
● chmod – change permissions on a file or
directory

● chown – change file ownership to another

user

– Both options support ‘-R’ for recursion.

27
 All About Me

● Every user id corresponds to a unique user or system

process

– whoami – returns the use rid of the current user

– passwd – change password

● What is my group? – support!

lslogin2$ ls -l ~eturner
total 391632
drwx------ 2 eturner support
-rwx------ 1 eturner support
drwx------ 5 eturner support
4096 Sep 12 15:41 anothertest
8487 Jun 7 13:12 a.out
4096 Apr 7 2006 automake-test

28
 What everyone else is up to

● top – show a detailed, refreshed, description of

running processes on a system.
● uptime – show the system load and how long the
system has been up.
– ‘load’ is a number based on utility of the cpu’s of the
system. A load of 1 indicates full load for one cpu.

lslogin2$ uptime
15:43:52 up 3 days, 20:45, 154 users, load average: 2.04, 2.00, 2.36

29
 Working With Programs
● Commands or programs on the system are
identified by their filename and by a process
ID which is a unique identifier.
– ps
● Display process information on the system
– kill pid
● Terminates the process id
– ^c (control+c) terminates the running program
– ^d (control+d) terminates your session.

● Only you and the superuser (root) has

permissions to kill processes you own.
30
 Advanced Program Options
● Oftentimes we must run a command in the
background with the ampersand ‘&’
character
– command –options &
● Runs command in background, prompt returns
immediately
● Match zero or more characters wildcard ‘*’
– cp * destination
● Copy everything to destination
– This option can get you into trouble if misused

31
 Editing Files
● emacs vs vi
– Among the largest ‘nerd battle’ in history. emacs relies
heavily on key-cords (multiple key strokes), while vi is mode
based. (editor mode vs command mode)
– vi users tend to enter and exit the editor repeatedly, and use
the Unix shell for complex tasks, whereas emacs users
usually remain within the editor and use emacs itself for
complex tasks

● pico (nano)
– editor originally used for the email client pine, simple no-frills
editor which resembles notepad for windows.

32
 Input and Output
● Programs and commands can contain an input and
output. These are called ‘streams’. UNIX
programming is oftentimes stream based.
– Programs also have an error output. We will see later how to
catch the error output.

● STDIN – ‘standard input,’ or input from the keyboard

● SDTOUT – ‘standard output,’ or output to the screen
● STDERR – ‘standard error,’ error output which is
sent to the screen.

33
 File Redirection
● Oftentimes we want to save output (stdout)
from a program to a file. This can be done with
the ‘redirection’ operator.
– myprogram > myfile – using the ‘>’ operator we
redirect the output from myprogram to file myfile

● Similarly, we can append the output to a file

instead of rewriting it with a double ‘>>’
– myprogram >> myfile – using the ‘>’ operator we
append the output from myprogram to file myfile

34
 Input Redirection
● Input can also be given to a command from
a file instead of typing it to the screen,
which would be impractical.

– cat programinput > mycommand - This

command series starts with the command ‘cat’cat
which prints a file to a screen. programinput is
printed to stdout, which is redirected to a
command mycommand.

35
 Redirecting stderr
● Performing a normal redirection will not
redirect sdterr. In Bash, this can be
accomplished with ‘2>’
– command 2> file1

● Or, one can merge stderr to stdout (most

popular) with ‘2>&1’
– command > file 2>&1

36
 Pipes
● Using a pipe operator ‘|’ commands can be
linked together. The pipe will link the
standard output from one command to the
standard input of another.

● Very helpful for searching files

37
 Searching

● A large majority of activity on UNIX systems involve

searching for files and information.
– find – utility to find files
– grep – the best utility ever written for UNIX, searches for
patterns inside files and will return the line, if found

lslogin2$ find ./ -name output-evansapp-n0.out534086

./test/run_0/output-evansapp-n0.out534086
lslogin2$ cat test/run_0/output-evansapp-n0.out534086 | grep CPU
CPU time : 0.03 sec.

38
 Packing Files
● When creating backups of files, or
transferring to other hosts, files must be
packed into larger files. This is needed for
ease of manipulation, transfer speeds, and
file management.

● tar – create or extract a packed file. tar

stands for ‘tape archive’.

39
 Compressing Files
● Compressing files can gain file space at the
expense of cpu time to compress and
decompress files.
● Compression works well for text files, but
not as well for binary files with random data
such as float values.
● Compression algorithms & commands
– gzip, gunzip, bzip2, bunzip2

40
 Using tar to create compressed files

● Tar will create compressed files for you

– tar –czvf mytarfile.tar.gz directory
● creates a compressed file named mytarfile.tar.gz
containing all of the files in the directory directory

– tar –xzvf mytarfile.tar.gz

● uncompresses all directories and files inside the file
mytarfile.tar.gz into the working directory

41
 Connecting to Another Machine
● Secure Shell vs. Restricted Shell
– ssh is an encrypted remote login program that is ‘secure’ to
trust across non secure networks.

● ssh userid@hostname

42
 Copying Files to Remote Hosts
● copy local file lfile to rfile on remote
machine rsys
– scp lfile rsys:rfile
● -p preserves modification time, access time and
mode from original
– scp -p lfile rsys:rfile

● copy rfile from remote machine rsys to local

file lfile
– scp -p rsys:rfile lfile

43
 Running Commands on a Remote Host

● Commands can be executed on a remote

host with ssh

● ssh userid@hostname “ls”

– Run ls on remote host hostname

44
 My Environment
● View all system variables by the command
‘env’
● Depending on shell, startup commands can
be managed with the files .profile for bash
and .cshrc with c shell

45
 Basic Shell Scripts
● Many times it is helpful to create a ‘script’ of
commands to run instead of typing them in
individually. Scripts can be made to aid in post-
processing, system administration, and automate
menial tasks

● #!/bin/bash
– First statement inside a script, will list which shell to run this
script in

● # - says what will follow is a comment and not to

execute

46
 Variables
● By convention system variables are capitalized
– HOME – location of the home directory
– OLDPWD – location of the previous working directory
– PATH – locations to look inside for executable files
● Setting system variables differs by shell. bash uses
export, csh uses setenv
● User defined variables in scripts are lower-case by
convention
– myvariable=10
● sets myvariable to 10
– echo $myvariable
● prints myvariable

47
 Conditionals
if condition
then
condition is zero (true - 0) execute all
commands up to else statement
else
if condition is not true then execute all
commands up to fi
fi

48
 Performing Loops
● Loops are statements that are repeated until the conditions are
met.

for { variable name } in { list }

do
execute one for each item in the list until the list is not finished
(And repeat all statement between do and done)
done

for i in 1 2 3 4 5
do
echo "Welcome $i times“
done

49
 Putting it Together
#!/bin/bash
#my first script
#scp replacement

remotefile=labresults
myserver=saguaro.fulton.asu.edu
mylsinfo=`ssh $myserver ls $remotefile 2>&1`
ismissing=`echo $mylsinfo | grep ERROR`
if [ "$ismissing" ]
then
echo "$remotefile not found! Exiting!"
else
ssh -n "cat < $remotefile" > $localfile
Fi 50
 More…
● mylsinfo=`ssh $myserver ls $remotefile
2>&1`
– `Backticks` are used to place output from a
command into a variable
● if [ "$ismissing" ]
– Is $ismissing set (has a value)? If so then the
expression is true, otherwise false

51
 References
The textbook for this course:

● Graham Glass & King Ables UNIX for

Programmers and Users

52