UNIX Fundamentals 2010

UNIX Fundamentals
CTO/SDE
2009-5
Agenda
UNIX Overview UNIX File System Process Control Introduction to vi Customizing Your Shell Environment Other Basic Commands
2 | Corporate Overview | January 2000
All Rights Reserved Alcatel-Lucent Shanghai Bell 2009
UNIX Overview
What Is UNIX?
The UNIX Operating System (OS) is a large program (mostly coded in C) that turns the computer into a useable machine.
UNIX is a multi-user, multi-tasking operating system.

UNIX is a machine independent operating system. UNIX is a software development environment.
A Brief History of UNIX

UNICS (1969) Fourth Edition (1973) Sixth Edition (1975) Seventh Edition (1979) SYSV (1983) Solaris/SunOs 5.x(SUN) AIX(IBM) IRIX(SGI) HP-UN(HP) Digital UNIX(DEC) SCO Unix(SCO) UnixWare(SCO) SVR4 (1993) *Linux (Open Source) (1991) BSD (1979) SunOs 4.x(SUN) ULTRIX(DEC) NextStep(NeXT) FreeBSD (Open Source) NetBSD (OPen Source) OpenBSD (Open Source)
Why Use UNIX?
multi-tasking / multi-user Distributed processing
rich set of tools

powerful networking capabilities stabile
portable (Hardware independence )

free! (FreeBSD, GNU) profitable
1996 Sales: US$34.5 Billion, up 12%
active community
Architecture of UNIX
Kernel
includes device driver
implements most BSD and SYSV system calls

Shells and GUIs
command line shell
GUIs: KDE, GNOME

Utilities & Application
Utilities : ls, cp, grep, awk, sed, bc, wc, more Application programs: emacs editor, gcc
compiler
Login to UNIX systems

login: ad Enter your ID and RETURN. Password: Enter your password and RETURN. It does not appear. $ The UNIX prompt (or similar). You can now enter commands.
If you login with a graphical terminal, you can look for menus or icons which mention the words "shell", "xterm", "console" or "terminal to open a shell prompt.
Logout from your account

logout or
^D
Press CONTROL and D together
Or
exit
Change your password

The command is $ passwd
It will ask you for the new password twice.
Password Tips NEVER tell anyone your password. Dont write it down. A good password is: - 8 (or more) characters long - uses a mix of uppercase and lowercase letters, numbers, and symbols (e.g. #, %).
General Command Syntax

The general syntax for a UNIX command is $ command [-options] targets
Example: ls -l /home/jones/dir1
Note: UNIX commands, options, and arguments are all Case sensitive!
UNIX Help
man cmd
man ls man man
Manual pages Spacebar to go on; ^C to stop
whatis cmd which cmd locate keyword
One-line description Location of command
List files with keyword in their name (or path)
UNIX File System
File types
The UNIX file system contains several different types of files
Regular file
Directory
Device Link
Socket
Named Pipe
A simplified UNIX Directory Structure
/ etc bin date . . . home cal will
...
dev
tmp
...
...
.......
joan
play
...
work
...
hobby.c
proj1
...
Some Typical UNIX directories

/ /bin /usr/bin /sbin The "root" directory Essential low-level system utilities Higher-level system utilities and application programs Superuser system utilities (for performing system administration tasks)
/lib
Program libraries (collections of system calls that can be included

in programs by a compiler) for low-level system utilities
/usr/lib
Program libraries for higher-level user programs
/tmp
/etc
Temporary file storage space (can be used by any user)

UNIX system configuration and information files
Some Typical UNIX directories (cont.)

/home or /homes User home directories containing personal file space for each user. Each directory is named after the login of the user.
/dev
/proc
Hardware devices
A pseudo-filesystem which is used as an interface to the kernel. Includes a sub-directory for each active program (or process).
The home directory

When you log into UNIX, your current working directory is your user home directory.
You can refer to your home directory at any time as "~" and the home directory of other users as "~<login>".
Will specify the directory proj1? / Wills home dir will / home /joan /proj1 home joan
~joan/proj1
play
...
work
...
~/play
hobby.c
proj1
...
Pathnames
Absolute Pathnames
The sequence of directory names between the top of the tree (the root) and the directory of interest.
Relative Pathnames
The sequence of directory names below the directory where you are now to the directory of interest.
examples: Absolute Pathnames /bin /etc/terminfo Relative Pathnames bin terminfo Comments if you are in / if you are in etc
/export/user/home/ad
../user/home/ad
if you are in /export/tma
. ..
the current directory the parent directory

File Permissions
There are three types of permissions supported by UNIX:
r (4)
read, view the contents of a file or a directory
w (2)
x (1)
write, edit file/directory contents

execute, run executable file
There are three levels of permissions:

User (owner) Group Other the person who owns the file. the group owns the file. the rest of the world
File Permissions
Heres an example
Suppose you type in ls -l and the result is
-rwxr-xr-- 1 hans doc 858 Aug 22 22:28 hw1

links
type
User permissions
Group permissions
owner
size group
Modification date/time
File name
Other Permissions
File Permissions
read=>4; write=>2; execute=>1
rwx r-x
r--
111
4 +2 +1
101
4 +0+1
100
4 +0+0
Changing Permissions
chmod [options] mode filename example: change file permissions with numerical mode
chmod 641 hw1
6
110
4
100
1
001
rw-
r--
--x
the owner of hw1 has rw-(6) permission, the group has r--(4) permission, others have --x permission.
23 | Corporate Overview | January 2000 All Rights Reserved Alcatel-Lucent Shanghai Bell 2009
Changing Permissions
chmod [options] who ops permission filename change file permissions with symbolic mode
options
R recursively change permissions who can be any combination of:
u (user)
o (other/world)
g (group)
a (all or ugo)
ops adds or takes away permission, and can be: + (add permission) (remove permission)
permission can be any combination of: r (read)

w (write)
x (execute)
Changing Permissions (cont.)

examples: chmod u+w test report add write permission on the files test and report for
their owners
chmod u-x abc.c chmod g+rwx myfile1 take away execute permission on abc.c from owner add read, write and execute permissions on myfile1
for the group

chmod ugo+rwx myfile2 add read,write and execute permissions on myfile2 for everyone
Permission Mask
umask [-S] [mode] Option: -S Example: file default permissions: directory default permissions: $ umask u=rwx,g=rwx,o= $ mkdir foo $ touch bar $ ls -l drwxrwx--- 2 user test 512 May 1 20:59 foo -rw-rw---- 1 user test 0 May 1 20:59 bar
User file-creation mode mask Produce symbolic output user group other rwrwrwrwx
rwx rwx
(or $ umask 007)
Changing owner and group

chown user file change file ownership to another user Must have the write permission on the file!
chgrp group file
change file group to another user

Only owner or supervisor can change the group!
examples:
chown joan test1

chgrp testteam test1
File Management
ls pwd (list) list files and directories
(print working directory display full pathname of current directory
mkdir
cd mv
(make directory
(change directory) (move)
create new directories

change to the directory move a file or directory
cp
rm touch cat more ln
(copy)
(remove)
copy a file or directory

remove a file and directory update Timestamp on File
(concatenate)
display files, or concatenate files control the display of files
(Link)
create link file or directory

Listing Contents
ls [options] dirname options: list the contents of dirname
-a
-l
examples: ls /home/user/temp
list all files including hidden files

[hidden files are preceded by a .; eg .cshrc] long listing showing ownership, permissions and links
view the contents of a directory with absolute pathname /home/user/temp
ls ../../temp
list the contents of a directory using a relative path.
pwd
display full pathname of current directory
Make Directory
mkdir dirname create the directory dirname
examples:
mkdir work create the directory work/ in the current working directory
mkdir work/proj1
mkdir /wrk/user2
create the directory proj1/ in the work/ directory

create the directory user2/ in the /wrk/ directory
Change Directory
cd cd dirname change to your home directory change to the directory dirname
examples:
cd ~tom cd /wrk/user2 change to toms home directory change to the directory /wrk/user2
cd ..
cd ../..
change to the parent directory

Get up two levels
Moving or Renaming
mv [options] presname newname mv [options] srcfile destdir rename a file move a file to another directory
options:
-i
example: mv ~user1/file ./outputfile
confirm overwrites
moves file from user1s home directory to the current working directory and renames it output file
Note: Be careful when overwriting files!
Copying
cp [options] srcfile destfile cp [options] srcfile destdir copy a file to another filename copy a file to another directory
options:
-i -R
example: cp -R proj1 proj2
confirm overwrites recursively copy a directory and its contents, copies symbolic links
copy the directory proj1/ and name it proj2/
Note: Be careful when overwriting files!
Removing
rm [options] filename options: remove a file
-f
-i -r
example: rm -rf /scratch/user2/dir1
remove without prompting

confirm removal recursively remove a directory and its contents
remove the directory /scratch2/user2/dir1 and its contents
!!!WARNING!!! This will DELETE EVERYTHING in that directory!!!

You can not recover your files after you removed them (unlike Windows OS).
Update Timestamp on File

touch [options] file Options:
-a
-m -c
update access and/or modification times of file
Changes the access time of the file

Changes the modification time of the file Does not create the file if it does not already exist
Examples:
ls -l -rw-r--r-- 1 user1 -rw-r--r-- 1 user1 touch newfile ls -l -rw-r--r-- 1 user1 -rw-r--r-- 1 user1 -rw-r--r-- 1 user1 touch secondfile ls l -rw-r--r-- 1 user1 -rw-r--r-- 1 user1 -rw-r--r-- 1 user1
users 25936 Apr 24 09:53 firstfile users 10245 Apr 24 09:53 secondfile users 25936 Apr 24 09:53 firstfile users 0 Apr 25 10:02 newfile users 10245 Apr 24 09:53 secondfile users 25936 Apr 24 09:53 firstfile users 0 Apr 25 10:02 newfile users 10245 Apr 25 10:05 secondfile
Catenate / Type
cat filename 1) displays the contents of file on the screen, one after the other .
2) You can also use it to create files from keyboard input

as follows (> is the output redirection operator) example:
cat > hello.txt hello world! [ctrl-d] ls hello.txt hello.txt cat hello.txt hello world!
Catenate with pause

more filename 1) displays the contents of file on the screen, pausing at the end of each screenful and asking the user to press a key
2)You can also use more to break up the output of commands

that produce more than one screenful of output example:
more output.c
ls -l | more (| is the pipe operator)
Making links file

ln filename linkname ln -s filename linkname Make hard links from one file or directory to another Make soft (or symbolic) links from one file or
directory to another
example: ln -s hello.txt bye.txt ls -l bye.txt lrwxrwxrwx 1 will finance 13 bye.txt -> hello.txt
Process Control
Processes
A process is a program in execution identified by a unique PID (process

identifier).
Every time you invoke a system utility or an application program from a shell,
one or more "child" processes are created by the shell in response to your command.
An important process that is always present is the init process. This is the first
process to be created when a UNIX system starts up and usually has a PID of 1. All other processes are said to be "descendants" of init.
A process may be in the foreground, in the background, or be suspended.
Listing Process
ps [options] display the status of the current processes and the process id-number
options:
-e -u all processes now running display processes owned by a particular user
jobs
shows any jobs that are currently running in the background or suspended
Process Control
<Ctrl>-C <Ctrl>-Z cancel a foreground job stop (interrupt) a foreground job
bg
fg &
run stopped job in the background

run stopped job in the foreground appended to the end of a command will place that job in the
background
examples: $ sleep 1000 & $ man ls <Ctrl>-Z $ jobs $ fg %2
$ jobs [2] + Stopped (SIGTSTP) man ls [1] - Running sleep 1000 &
?
Killing a process
kill id-number terminate a process owned by you; id-number (process id) can be found with the ps command
or
kill %job-number terminate a process owned by you; job-number can be found with the jobs command
Example:
ps
PID TTY TIME CMD 17717 pts/10 00:00:00 bash
27501 pts/10 00:00:01 find

27502 pts/10 00:00:00 ps
kill 27501 kill -9 27501

If a process refuses to be killed, uses the -9 option

STDINT,STDOUT and STDERR
File
Rese t Br e k a f1 Sp to f2
Device
f3 f4 Men u Use r S m yste f5 f6 f7 f8
File Descriptor
Inse t r line Dele te line
stdin
~ `
! 1 Q
@ 2 W
# 3 E
$ 4 R
% 5 T
^ 6 Y
& 7 U
* 8 I
( 9 O
) 0 P
_ { [ : ;
= + } ] " '
Back S ce pa
. 7 4 1 0
/ 8 5 2
+ 9 6 3 .
T ab
| \
Ins r t e
cha r
Dele te cha r
Cap s
CTRL
Re rn tu
Pe r v
,
T ab
DE L ESC
S hift
< ,
> .
? /
E xten d cha r
S hift
S elect
Next
Exten d cha r
keyboard stdout stderr terminal 1 2
a566137
Input Redirection
< command input redirection (from file)
Examples:
how to email a file to a collaborator mail -s results collab@alcatel-sbell.com.cn < new_blast_results
Output Redirection
> >> command output redirection (create new, overwrites existing file) command output redirection (append)
Examples: Create/Overwrite Create/Append
$ ls > filelist.out
$ cat seq1 seq2 > seq
$ ls >> filelist.out
$ cat seq1 >> seq2
(combine individual sequence files into one file)
Error Redirection
2> 2>> error output redirection (create new, overwrites existing file) error output redirection (append)
Examples:
$ cp 2> cp.err $ cp 2>> cp.err Create/Overwrite Create/Append
$ more cp.err
Usage: cp [-f|-i] [-p] source_file target_file cp [-f|-i] [-p] source_file ...target_directory cp [-f|-i] [-p] -R|-rsource_directory...target_directory Usage: cp [-f|-i] [-p] source_file target_file
cp [-f|-i] [-p] source_file ... target_directory

cp [-f|-i] [-p] -R|-r source_directory...target_directory
pipes
command1 | command2 | directs standard output of one command into standard input for the next command
examples:
ls -al | more cat hello.txt | sort | uniq look at the ls output one page at a time creates three processes (corresponding to cat,
sort and uniq) which execute concurrently. As

they execute, the output of the who process is passed on to the sort process which is in turn passed on to the uniq process. uniq displays its output on the screen (a sorted list of users with duplicate lines removed).
Introduction to vi
Introduction to vi
vi is the standard UNIX text editor very powerful
useful simple subset of commands

portable (PCs, mainframes, etc.) designed for slow networks
full-screen
Starting vi vi filename Changes are stored in a buffer, so you must save to change the file !If the file doesn't exist, vi will create it for you.
Three Modes
Command mode
move cursor, save, delete text, quit vi, etc.

Last-line mode
Initiated from command mode by entering advanced editing

commands like :, /, ?, !.
Commands are shown on the status line (Bottom line )

Input mode
for inserting text start by typing i; finish with ESC cannot quit, delete, etc. in this mode If in doubt, press ESC a few times. This will put you back in
command mode.
Cursor movement
h j k left down up
right
^ $
beginning of line end of line
1G G ^F ^B w b
top of document end of document page forward page backward word forwards word backwards
<n> G go to line <n>
Inserting and typing text

i a o insert text (and enter input mode) append text (and enter input mode) start a new line (and enter input mode)
No RETURN
Move to insertion point
Switch to input mode:

Start typing;
ESC
BACKSPACE or DELETE for
deletion
finish; back in command mode
Deleting
Backspace Delete character before cursor (only works in insert mode) Must be in command mode
x
dd <n>dd
Delete character that cursor is on.

Delete current line. Delete n lines
Delete from cursor position to end of line
:i,jd :23,29d :.,$d u
Delete lines i to j Delete lines 23 to 29 Delete from current line to the end of file.
Undo last command

Deleting and moving

Cut & Paste with Deleted Text
dd or <n>dd or D p
Copy & Paste
delete from screen and store text in a buffer
move cursor to new location

paste contents of buffer to right of cursor position
yy or <n>yy
yank/copy lines and store text in a buffer
move cursor to new location
paste contents of buffer to right of cursor position
Searching for Text type / and then a regular expression and press Enter
There is one here and one more here and yet one more but not this ONE nor this One
/one <cr> n next N previous

a56686
Global Search and replace

:s/old/new/g replace every occurrence of old by new
:i,js/old/new/g replace every occurrence of old by new between lines i and j
:s/Hat/Head/g
:2,200s/Andy/Andrew/g
:1,.s/fc/function/g
from line 1 to current
Finishing a vi Session
Get to :w :wq ZZ :q! command mode (press ESC) save file (not quit) save file and quit save file and quit quit without saving save as the file 'newfile', overwriting any existing newfile
:w! newfile
Customizing Your Shell Environment
Shells and Shell Scripts

A shell is a program that takes your commands from the keyboard and gives them to the operating system to perform. Shells also usually provide features such job control, input and output redirection and a command language for writing shell scripts. A shell script is simply an ordinary text file containing a series of commands in a shell command language (just like a "batch file" under MS-DOS).
There are many different shells available on UNIX systems (e.g. sh, bash, csh, ksh etc.), and they each support a different command language.
View your login shell

Determine your login shell: echo $SHELL
Change to another shell, just type the shell name:

sh (or Or csh or ksh or bash)
exec sh
(close the original shell, then open another shell sh)
Go back to your normal shell exit or press CTRL-D
Changing your Login Shell

To change your Unix shell, you will need to be at a Unix shell prompt. type the command
$ passwd -r nis -e
Enter login(NIS) password: Type your Unix password and press return. You will then see screen output similar to the following:
Old shell: /usr/local/bin/bash New shell:

enter the full name of your new shell.
Initialization files
Each time you login to a UNIX host, the system looks in your home directory for initialization files. You can change your environment by changing initialization files. Shell System initialization filesFirst /etc/profile /etc/profile User initialization filessecondly $HOME/.profile $HOME/.profile $HOME/.kshrc $HOME/.cshrc $HOME/.login $HOME/.bash_profile $HOME/.bashrc Template /etc/skel local.profile local.profile
sh(Bourne shell) ksh(Korn shell) csh(C shell)
/etc/.login
local.cshrc local.login local.profile
bash(Bourne Again shell )
/etc/profile
Setting shell variables

At login the C shell first reads .cshrc followed by .login
.login is to set conditions which will apply to the whole session and to perform
actions that are relevant only at login.
.cshrc is used to set conditions and perform actions specific to the shell and to
each invocation of it. The guidelines are to set ENVIRONMENT variables in the .login file and SHELL variables in the .cshrc file. Setting shell variables in the .cshrc file
$ vi ~/.cshrc Add the following line AFTER the list of other commands set history = 200 Save the file $ source .cshrc (force the shell to reread its .cshrc file ) $ echo $history (Check this has worked)
Setting Environment Variables

View all system variables by the command env By convention system variables are capitalized
PATH
HOME PWD OLDPWD
a list of directories that the shell uses to locate executable files for commands
Name of your home (login) directory Current directory Previous directory before the last cd command
Setting system variables differs by shell. bash uses export, csh uses setenv
varname=value command export varnames Example: PATH=$PATH:/sbin:/usr/sbin:/usr/bin:/etc:/usr/ucb:/local/bin EDITOR=/usr/local/bin/emacs export EDITOR PATH
Alias
alias new=original alias new original defines new to be an alias for original (ksh or bash) (csh)
example: alias ct=cleartool
alias ll='ls l'

alias logout='. ~/.ksh_logout; exit'
Other Basic Commands
Other Basic Commands

tar compress (tape archive) ; create archives compress files
gzip
find grep sort
compress files
find files and directories with a wealth of search criteria (general regular expression parser); text search sort information provided on standard input
ssh
(secure shell) ; remote login
Archives and Storage

tar create archives, and read/write from tapes and diskettes create a disk file tar archive
tar -cvf archivenamefilenames
tar -tvf archivename

tar -xvf archivename compress /gzip compress files
list the contents of a tar archive

restore files from a tar archive
compress file
uncompress file.Z gzip file gunzip file.gz
compress file and rename it file.Z

uncompress file.Z and rename it file compress file and rename it file.gz uncompress file.gz and rename it file
Finding Files
find Example: Find files and directories with a wealth of search criteria
find directory -name targetfile print

look for a file called targetfile in any part of the directory tree rooted at directory. find /home -name "*.txt" -print 2>/dev/null search all user directories for any file ending in ".txt" and output any matching files. The 2>/dev/null suppresses error messages. find . -name "*.txt" -exec wc -l '{}' ';'
counts the number of lines in every text file in and below the current directory. The '{}' is replaced by the name of each file found and the ';' ends the -exec clause.
Searching Text
grep options pattern files searches the named files (or standard input if no files are named) for lines that match a given pattern
Options:
-c (print a count of the number of lines that match) -i (ignore case) -v (print out the lines that don't match the pattern) -n (printout the line number before printing the matching line)
Searching Text (cont.)

Examples: grep -vi hello *.txt searches all text files in the current
directory for lines that do not contain

any form of the word hello (e.g. Hello, HELLO, or hELlO). grep hello `find . -name "*.txt" -print` search all text files in the directory tree rooted at the current directory for lines containing the word "hello". grep ^..[l-z]$ hello.txt matches any line in hello.txt that contains a three character sequence that ends with a lowercase letter from l to z.
Sorting files
sort [options] filenames Options: sort information provided on standard input
-r
Examples:
Reverses the sense of comparisons
sort input1.txt input2.txt > output.txt
outputs the sorted concentenation of files input1.txt and input2.txt to the file output.txt. sort input.txt | uniq > output.txt
Uniq removes duplicate adjacent lines from a file input.txt. It is most useful when combined with sort
Remote login
ssh [options] host secure shell; a program for logging into a remote host providing encrypted communications
between hosts
options: -l login name
-X
sets environment variables for porting X-display
example: ssh -l fengy sbardy12 open a secure connection for the user fengy on the host sbardy12 ssh load2 open a secure connection to host load2 with current username
Thanks!
www.alcatel-sbell.com.cn

UNIX Fundamentals 2010

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

UNIX Fundamentals 2010

Uploaded by

Copyright:

Available Formats

UNIX Fundamentals

2 | Corporate Overview | January 2000

All Rights Reserved Alcatel-Lucent Shanghai Bell 2009

3 | Corporate Overview | January 2000

All Rights Reserved Alcatel-Lucent Shanghai Bell 2009

UNIX is a multi-user, multi-tasking operating system.

4 | Corporate Overview | January 2000

All Rights Reserved Alcatel-Lucent Shanghai Bell 2009

A Brief History of UNIX

5 | Corporate Overview | January 2000

All Rights Reserved Alcatel-Lucent Shanghai Bell 2009

Why Use UNIX?

multi-tasking / multi-user Distributed processing

rich set of tools

portable (Hardware independence )

6 | Corporate Overview | January 2000

All Rights Reserved Alcatel-Lucent Shanghai Bell 2009

includes device driver

implements most BSD and SYSV system calls

command line shell

GUIs: KDE, GNOME

7 | Corporate Overview | January 2000

All Rights Reserved Alcatel-Lucent Shanghai Bell 2009

Login to UNIX systems

8 | Corporate Overview | January 2000

All Rights Reserved Alcatel-Lucent Shanghai Bell 2009

Logout from your account

Press CONTROL and D together

9 | Corporate Overview | January 2000

All Rights Reserved Alcatel-Lucent Shanghai Bell 2009

Change your password

It will ask you for the new password twice.

10 | Corporate Overview | January 2000

All Rights Reserved Alcatel-Lucent Shanghai Bell 2009

General Command Syntax

11 | Corporate Overview | January 2000

All Rights Reserved Alcatel-Lucent Shanghai Bell 2009

Manual pages Spacebar to go on; ^C to stop

whatis cmd which cmd locate keyword

One-line description Location of command

List files with keyword in their name (or path)

12 | Corporate Overview | January 2000

All Rights Reserved Alcatel-Lucent Shanghai Bell 2009

UNIX File System

13 | Corporate Overview | January 2000

All Rights Reserved Alcatel-Lucent Shanghai Bell 2009

14 | Corporate Overview | January 2000

All Rights Reserved Alcatel-Lucent Shanghai Bell 2009

A simplified UNIX Directory Structure

/ etc bin date . . . home cal will

15 | Corporate Overview | January 2000

All Rights Reserved Alcatel-Lucent Shanghai Bell 2009

Some Typical UNIX directories

Program libraries (collections of system calls that can be included

Program libraries for higher-level user programs

Temporary file storage space (can be used by any user)

16 | Corporate Overview | January 2000

All Rights Reserved Alcatel-Lucent Shanghai Bell 2009

Some Typical UNIX directories (cont.)

17 | Corporate Overview | January 2000

All Rights Reserved Alcatel-Lucent Shanghai Bell 2009

The home directory

18 | Corporate Overview | January 2000