lOMoARcPSD|27910594

Module 5 - simple filter commands in Linux and

understanding various servers.
Linux Os And Shell Programming (Mahatma Gandhi University)

Studocu is not sponsored or endorsed by any college or university

Downloaded by Haneef Zain (haneefzain2003@gmail.com)
 lOMoARcPSD|27910594

Module 5

Filter commands and Servers in Linux

Filters
A filter is a program that takes input from the standard input file, process it and sends the
output to the standard output file. Linux provides various filters which enable you to work
effectively with data.

Simple filter Commands - pr, head, tail, cut, sort, uniq, tr

Head

The head command prints the top N data of the given input. By default it
prints the first 10 lines of the specified file.
Syntax

head [OPTION]... [FILE]...

Consider a file having name state.txt which contains the names of Indian states.

cat state.txt

Output

Andhra Pradesh

Arunachal Pradesh

Assam

Bihar

Chhattisgarh

Goa

Gujarat

Haryana

Himachal Pradesh

Jammu and Kashmir

Jharkhand

Karnataka

Downloaded by Haneef Zain (haneefzain2003@gmail.com)

 lOMoARcPSD|27910594

Without any option head command will display only the first 10 lines of the file specified.

Example

head state.txt

Andhra Pradesh

Arunachal Pradesh

Assam

Bihar

Chhattisgarh

Goa

Gujarat

Haryana

Himachal Pradesh

Jammu and Kashmir

Options

1. -n num: Prints the first ‘num’ lines instead of first 10 lines. num is mandatory to be specified
in command otherwise it displays an error.

Syntax

head –n num filename

Example

head -n 5 state.txt

Andhra Pradesh

Arunachal Pradesh

Assam

Bihar

Chhattisgarh

2. -c num: Prints the first ‘num’ bytes from the file specified. num is mandatory to be specified
in command otherwise displays an error.

Downloaded by Haneef Zain (haneefzain2003@gmail.com)

 lOMoARcPSD|27910594

Syntax

head –c num filename

Example

head -c 6 state.txt

Andhra

3. -v: By using this option, data from the specified file is always preceded by its file name.

Syntax

head –v filename

Example

head -v state.txt

==> state.txt <==

Andhra Pradesh

Arunachal Pradesh

Assam

Bihar

Chhattisgarh

Goa

Gujarat

Haryana

Himachal Pradesh

Jammu and Kashmir

Tail
It is used to display the last or bottom part of the file. By default it will give 10 lines.

Syntax

tail [options] filename

Downloaded by Haneef Zain (haneefzain2003@gmail.com)

 lOMoARcPSD|27910594

Options

-n - specify how many lines you want to display.

consider the above file state.txt

tail state.txt

Assam

Bihar

Chhattisgarh

Goa

Gujarat

Haryana

Himachal Pradesh

Jammu and Kashmir

Jharkhand

Karnataka

tail -n 3 state.txt or tail -3 state.txt

Jammu and Kashmir

Jharkhand

Karnataka

uniq
It is used to handle duplicate lines in a file. If this command is used without any option, it
displays the lines by eliminating duplicates lines.

Syntax

uniq –options filename

Options are:

-u - displays only the non repeated lines.

Downloaded by Haneef Zain (haneefzain2003@gmail.com)

 lOMoARcPSD|27910594

-d - displays only the duplicated lines.

-c - displays each line by eliminating duplicate lines and prefixing the number of times it occurs.

Examples

cat emp

1 aa

2 bb

2 bb

3 cc

4 dd

4 dd

4 dd

5 ee

6 ff

6 ff

uniq emp

1 aa

2 bb

3 cc

4 dd

5 ee

6 ff

uniq –u emp

1 aa

3 cc

5 ee

uniq –d emp

2 bb

4 dd
5

Downloaded by Haneef Zain (haneefzain2003@gmail.com)

 lOMoARcPSD|27910594

6 ff

uniq –c emp

1 1 aa

2 2 bb

1 3 cc

3 4 dd

1 5 ee

2 6 ff

sort
It can be used for sorting the contents of a file. It can merge multiple sorted files and store
the result in the specified output file.

Syntax

sort myfile

This would sort the contents of myfile and display the sorted contents on the screen.

sort file1 file2 file3

It will sort the contents of several files at one shot.

Options

1. sort –o result file1 file2 file3

The above command sorts the 3 files file1, file2, file3 and saves the result in a file called

result.

2. sort –u –o result file1 file2 file3

Using –u option we can avoid repeated lines in each files.

3. sort –m file1 file2

If the files were already been sorted we can merge them using the above command.

pr
6

Downloaded by Haneef Zain (haneefzain2003@gmail.com)

 lOMoARcPSD|27910594

This command formats a file to make it look better when printed. It paginates files for printing.It
displays the contents of the specified file adding with suitable headers and footers. The header part
consists of last modification date and time along with filename and page number.

Syntax

pr [option]... [file]...
options

-l <number> - It changes the page size to specified <number> of lines (by default the page
size is 66 lines).
-<number> - prepares the output in <number> columns.
-n - Numbers lines.
Syntax:
-t - turns off the heading at the top of the page.
[ pr [option]... [file]...
cat emp
1005
1002
1003
1001
1004

pr emp

2012-01-06 09:40 e1 Page 1

1005
1002
1003
1001
1004
----Note that remaining lines are empty--------

pr -n emp

2012-01-06 09:40 e1 Page 1

1 1005
2 1002
3 1003
4 1001
5 1004
----Note that remaining lines are empty--------

pr -3 emp

2012-01-06 09:40 e1 Page 1

Downloaded by Haneef Zain (haneefzain2003@gmail.com)

 lOMoARcPSD|27910594

1005 1003
1002 1001 1004

----Note that remaining lines are empty--------

pr -t emp
1005
1002
1003
1001
1004

tr
This command translate or delete characters. Translate, squeeze, and delete characters from
standard input, writing to standard output. Note that this command gets input from the standard
input (keyboard), not from a file. But you can use pipe or redirection to use a file as input.

Syntax

tr <Character_Set1><Character_Set2><StandardInput>

cat > myfile.txt

computer
zoology
commerce
botany
physics

cat myfile.txt | tr "[a-z]" "[A-Z]"

COMPUTER
ZOOLOGY
COMMERCE
BOTANY
PHYSICS

tr "c,o" "C,O" < myfile.txt

COmputer
zOOlOgy
COmmerCe
bOtany
physics

Note -The tr command used with –d option deletes the characters specified in <character_set> from
input without translation.

tr -d "c,o" <myfile.txt

Downloaded by Haneef Zain (haneefzain2003@gmail.com)

 lOMoARcPSD|27910594

mputer
zlgy
mmere
btany
physis

cut
The cut command is for cutting out sections from each line of a file and write the result to
standard output. It can be used to cut parts of a line by byte position or character. Basically, the
cut command slices a line and extracts the text. It is necessary to specify option with the command
otherwise it gives error.

Syntax
cut OPTION... [FILE]...

Consider a file having name state.txt contain 5 names of the Indian states.

cat state.txt
Andhra Pradesh
Arunachal Pradesh
Assam
Bihar
Chhattisgarh

Without any option specified it displays error.

cut state.txt

cut: you must specify a list of bytes, characters, or fields

Try 'cut --help' for more information.

Options

1. -b(byte): This option is used to extract the specific bytes with the list of byte numbers
separated by comma. Range of bytes can also be specified using the hyphen (-). It is
necessary to specify list of byte numbers otherwise it gives error. Tabs and backspaces are
treated as a character of 1 byte.

#List without ranges

cut -b 1,2,3,4 state.txt

Andh
Arun
Assa
Biha
Chha

Downloaded by Haneef Zain (haneefzain2003@gmail.com)

 lOMoARcPSD|27910594

#List with ranges

cut -b 1-3,5-7 state.txt

Andra
Aruach
Assm
Bihr
Chhtti

Note - It uses an option “1-“for selecting bytes from beginning upto the end of the line.

cut -b 1- state.txt

Andhra Pradesh
Arunachal Pradesh
Assam
Bihar
Chhattisgarh

Note - “-n” option cuts bytes from beginning to nth byte.

In this, -3 indicate from 1st byte to 3rd byte of a line

cut -b -3 state.txt

And
Aru
Ass
Bih
Chh

2. -c (column): To cut by character use the -c option. This can be a list of numbers separated by
comma or a range of numbers separated by hyphen (-). Tabs and backspaces are treated as
characters. It is necessary to specify the list of character numbers, otherwise it gives error
with this option.

Syntax
cut -c [(k)-(n)/(k),(n)/(n)] filename

Here, k denotes the starting position of the character and n denotes the ending position of the
character.

cut -c 2,5,7 state.txt

nr
rah
10

Downloaded by Haneef Zain (haneefzain2003@gmail.com)

 lOMoARcPSD|27910594

sm
ir
hti

Above cut command prints second, fifth and seventh character from each line of the file.

cut -c 1-7 state.txt

Andhra
Arunach
Assam
Bihar
Chhatti

Note - Cut uses '1-’ for selecting characters from beginning to the end of the line.

cut -c 1- state.txt

Andhra Pradesh
Arunachal Pradesh
Assam
Bihar
Chhattisgarh

Note - Cut uses “-n” option to cut from the beginning to the nth position.

cut -c -5 state.txt

Andhr
Aruna
Assam
Bihar
Chhat

11

Downloaded by Haneef Zain (haneefzain2003@gmail.com)

 lOMoARcPSD|27910594

Filter using regular expressions

Grep

The grep filter searches a file for a particular pattern of characters, and displays all lines that
contain that pattern. The pattern that is searched in the file is referred to as the regular expression
(grep stands for Globally search for Regular Expression and Print). In the simplest terms, grep is
a small family of commands that search input files for a search string, and print the lines that
match it.grep is considered one of the most useful commands in any Unix system.
Grep is made up of three separate, yet connected commands, grep, egrep, and fgrep. All
three of the grep commands work the same way. Beginning at the first line in the file, grep
copies a line into a buffer, compares it against the regular expression, and if the match is found,
it prints the line to the screen. Grep will repeat this process until the file runs out of lines. Note
that in this process grep doesn’t store lines, change lines, or search only a part of a line.

Syntax

grep [options] pattern [files]

Options

-c - This prints only a count of the lines that match a pattern.

-h - Display the matched lines, but do not display the filenames.
-i - Ignores, case for matching.
-l - Displays list of a filenames only that matc hes the pattern.
-n - Display the matched lines and their line numbers.
-v - This prints out all the lines that do not matches the pattern.
-w - Match the whole word.
-o - Print only the matched parts of a matching line, with each such part on a separate output line.
-E - Treats pattern as an extended regular expression (ERE)

cat > geekfile.txt

unix is great os. unix is opensource. unix is free os.

learn operating system.
Unix linux which one you choose.
uNix is easy to learn. unix is a multiuser os. Learn unix. unix is powerful.

12

Downloaded by Haneef Zain (haneefzain2003@gmail.com)

 lOMoARcPSD|27910594

1. Case insensitive search: The” –i” option enables to search for a string case insensitively in
the given file.

grep -i "UNix" geekfile.txt

unix is great os. unix is opensource. unix is free os.

Unix linux which one you choose.
uNix is easy to learn. unix is a multiuser os. Learn unix .unix is powerful.

2. Displaying the count of number of matches: “-c” option finds the number of lines that
matches the given string/pattern.

grep -c "unix" geekfile.txt

3. Display the file names that matches the pattern: “-l” option display the files that contains
the given string/pattern.

grep -l "unix" *

geekfile.txt

4. Checking for the whole words in a file : By default, grep matches the given string/pattern
even if it found as a substring in a file. The “-w” option makes it match only the whole words.

grep -w "unix" geekfile.txt

unix is great os. unix is opensource. unix is free os.

unix is a multiuser os. Learn unix .unix is powerful.

5. Displaying only the matched pattern : By default, grep displays the entire line which has the
matched string. We can make the grep to display only the matched string by using the -o
option.

grep -o "unix" geekfile.txt

unix
unix
unix

13

Downloaded by Haneef Zain (haneefzain2003@gmail.com)

 lOMoARcPSD|27910594

unix
unix
unix

6. Inverting the pattern match : You can display the lines that are not matched with the
specified search sting pattern using the “-v” option.

grep -v "unix" geekfile.txt

learn operating system.

Unix linux which one you choose. uNix is easy to learn.

7. Matching the lines that start with a string: The” ^” regular expression pattern specifies the
start of a line. This can be used in grep to match the lines which start with the given string or
pattern.

grep "^unix" geekfile.txt

unix is great os. unix is opensource. unix is free os.

unix is a multiuser os. unix is powerful.

8. Matching the lines that end with a string: The “$” regular expression pattern specifies the
end of a line. This can be used in grep to match the lines which end with the given string or
pattern.

grep "os$" geekfile.txt

unix is great os. unix is free os.

unix is a multiuser os.

When grep performs its pattern matching, it expects you to provide a regular expression
for the pattern. Here are the most common types of regular expressions:

Regular Expression Meaning

abc Match the line containing the string “abc”
^abc Matches line starts with abc
abc$ Matches line ending with abc
a..c Match line containing a and c separated by any two
characters (dot matches any single character)
a.*c Matches line containing a and c containing any number of
characters (the dot – asterisk means match zero or more
characters)

14

Downloaded by Haneef Zain (haneefzain2003@gmail.com)

 lOMoARcPSD|27910594

egrep

egrep (extended grep) is a pattern searching command which belongs to the family
of grep functions. It works the same way as grep -E does. It treats the pattern as an extended
regular expression and prints out the lines that match the pattern.grep supports many regular
expression commands, but it does not support certain useful sequences such as the “+” and “?”
operators. egrep supports this kind of regular expressions.

Syntax
egrep [options] 'PATTERN' files

Note: The egrep command used mainly due to the fact that it is faster than the grep command.

Most of the options for this command are same as grep.

Note: fgrep is the third member of grep family. It stands for “fast grep”. It is equivalent to grep –F.

Sed
SED command stands for stream editor and it can perform lots of function on file like,
search, find and replace, insertion or deletion. By using SED you can edit files even without opening
it, which is much quicker way to find and replace something in file, in Vi Editor and then changing it .

Syntax

sed OPTIONS... [SCRIPT] [INPUTFILE...]

Consider the below text file as an input.

cat > geekfile.txt

unix is great os. unix is opensource. unix is free os.

learn operating system.
unix linux which one you choose.
unix is easy to learn.unix is a multiuser os.Learn unix .unix is powerful.

Options

1. Replacing or substituting string : Sed command is mostly used to replace the text in a file.
The below simple sed command replaces the word “unix” in the first line with “linux” in
the file.
15

Downloaded by Haneef Zain (haneefzain2003@gmail.com)

 lOMoARcPSD|27910594

sed 's/unix/linux/' geekfile.txt

linux is great os. unix is opensource. unix is free os.

learn operating system.
linux linux which one you choose.
linux is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.

2. Replacing the nth occurrence of a pattern in a line : Use the /1, /2 etc flags to replace the
first, second occurrence of a pattern in a line. The below command replaces the second
occurrence of the word “unix” with “linux” in a line.

sed 's/unix/linux/2' geekfile.txt

unix is great os. linux is opensource. unix is free os.

learn operating system.
unix linux which one you choose.
unix is easy to learn.linux is a multiuser os.Learn unix .unix is a powerful.
3. Replacing from nth occurrence to all occurrences in a line : Use the combination of /1, /2
etc and /g to replace all the patterns from the nth occurrence of a pattern in a line. The
following sed command replaces the third, fourth, fifth… “unix” word with “linux” word in
a line.

sed 's/unix/linux/3g' geekfile.txt

unix is great os. unix is opensource. linux is free os.

learn operating system.
unix linux which one you choose.
unix is easy to learn.unix is a multiuser os.Learn linux .linux is a powerful.

16

Downloaded by Haneef Zain (haneefzain2003@gmail.com)

 lOMoARcPSD|27910594

Servers
In computing, a server is a computer program or a device that provides functionality for
other programs or devices which are called clients. This architecture is called the client–server
model. Servers can provide various functionalities called services. These services include sharing
data or resources among multiple clients, or performing computation for a client. Multiple clients
can be served by a single server, and a single client can use multiple servers. A client process may
run on the same device. It can also connect over a network to a server to run on a different device.
Most frequently client–server systems are implemented by the request–response model.,
i.e., a client sends a request to the server. Server then performs some action and sends a response
back to the client, typically with a result or acknowledgement. Designating a computer as server-
class hardware means that it is specialized for running servers on it. This implies that it is more
powerful and reliable than standard personal computers.

Types of Servers and their applications:

1. Application server – These servers host web apps (computer programs that run inside a web
browser) allowing users in the network to run and use them.

2. Catalog server – These servers maintain an index or table of contents of information that can
be found across a large distributed network. Distributed network may include computers,
users, files shared on file servers, and web apps. Examples of catalog servers are Directory
servers and name servers. Their clients are any computer program that needs to find
something on the network.

3. Communications server – These servers maintain an environment needed for one

communication endpoint to find other endpoints and then communicates with them.

4. Computing server – These servers share vast amounts of computing resources which include
CPU and random-access memory over a network.

5. Database server – These servers maintain and share any form of database over a network.

6. Game server – These servers enable several computers or gaming devices to play
multiplayer games. Personal computers or gaming consoles are their clients.

7. Mail server – These servers makes email communication possible in the same way as a post
office makes snail mail communication possible. Clients of these servers are senders and
recipients of email.

17

Downloaded by Haneef Zain (haneefzain2003@gmail.com)

 lOMoARcPSD|27910594

8. Print server – These severs share one or more printers over a network which eliminates the
hassle of physical access. Their clients are computers in need of printing something.

9. Proxy server – This server acts as an intermediary between a client and a server accepting
incoming traffic from the client and sending it to the server. Reasons to use a proxy server
includes content control and filtering, improving traffic performance, preventing unauthorized
network access or simply routing the traffic over a large and complex network. Their clients
are any networked computer.

10. Web server – These servers hosts web pages. A web server is responsible for making the
World Wide Web possible. Each website has one or more web servers. Their clients are
computers with a web browser.

Understanding Various Servers in Linux

1. DHCP server

The dynamic host configuration protocol is a network configuration protocol for host on
internet protocol network. Computers that are connected to IP networks must be configured
(Must be given an IP address) before they can communicate with other host. The most essential
information needed is an IP address.
DHCP eliminates the manual task by a network administration. It also provides a
central database of devices that are connected to the network and eliminates duplicate
resource assignments. In addition to IP address DHCP also provides other configuration information
particularly the IP address, network route service or other service host. DHCP is used for IPv4&
IPv6.
DHCP was first defined as an extension to the boot strap protocol. The reason for extending
boot strap protocol was that boot strap protocol required manual intervention to add
configuration information for each client and did not provide a mechanism for reclaiming dimmed
IP addresses.
DHCP is useful because it makes it easy to add new machines to the network.
When a DHCP configured client connects to a network the DHCP client sends a broadcast
query requesting necessary information from a DHCP server. The DHCP server manages a
pool of IP addresses and information about client configuration parameters such as default
gateway, domain name, the name servers and other servers. On receiving a valid request the
server assign the computer and IP address a lease time (length of time the allocation in valid)
and other IP configuration parameters such as the subnet mask and the default gateway. The
query is typically initiated immediately after booting and must complete before the client can
initiate IP based communicate with other host.
Depending on implementation the DHCP server may have 3 methods of allocating IP
addresses.
 Dynamic allocation
 Automate allocation
 Static allocation

18

Downloaded by Haneef Zain (haneefzain2003@gmail.com)

 lOMoARcPSD|27910594

 Dynamic allocation

A network administrator assigns a range of IP addresses to DHCP and each client computer
on the LAN is configured to request an IP address from the DHCP server during network
initialization. The request & grand process uses a lease concept with a controllable time
period allowing the DHCP server to reclaim and then reallocate IP addresses that are not
renewed.

 Automatic allocation

A DHCP permanently assign a free IP address to a requesting client from the range defined by
the administrator. This is like dynamic allocation but the DHCP server keeps a table of part IP
address assignments so that it can preferentially assigned to a client the same IP address that
the client previously had.

 Static allocation
The DHCP server allocates an IP address based on a table with MAC (Medium Access
Control) address /IP address pair which are manually filled in. only requesting clients are
allocated with an IP address. This Feature (which is not supported by all DHCP servers) is
variously called static DHCP assignments, fixed address, address reservation, DHCP
reservation etc.

2. DNS Server
DNS server translate the domain names into machine readable IP address needed to locate the
requested web server on the internet.

Users can have their DNS server up and running correctly and securely. DNS server is a very
capable and full feature server and it has plenty of option for expert to make it just the way
they want DNS server can optionally server. DNS records directly from a SQL server.DNS
server comes with a DHCP server plug in, a DNS look up tool and many other features.

19

Downloaded by Haneef Zain (haneefzain2003@gmail.com)

 lOMoARcPSD|27910594

3. SQUID

SQUID is released in July 1996. It is a proxy server and a web server. Proxy server is a server
that acts as an intermediary for request from clients seeking resources from other servers. A
client connects to the proxy requesting some service such as a file, connection, web page on
other resources available from a different server. The proxy server evaluates the request
according to its filtering rules. If the request is validated by the filter the proxy server provide
the resource by connecting to the relevant server and requesting the service on behalf of the
client.
SQUID is a free software. It is used by Wikipedia. Squid is used by 100‘s of internet provides
worldwide to provide their users with the best possible web access. Squid optimizes the data
flow between client and server improves performance and caches frequently.

4. FTP Server

File Transfer Protocol (FTP) is a standard network protocol used to transfer files from one
host to another host over a TCP (Transmission Control Protocol) based network such as the
internet. FTP is built on client server architecture and utilizes separate control and data
connections between the client and server. FTP users may authenticate themselves using a clear
text sign in protocol but can connect anonymously if the server is config
ured to allow it.

5. Samba server

Samba is free software licensed under the GNU General Public License. It is developed in
1992. Samba has provided secure stable and fast file and print services for all clients using the
SMB / CIFS protocol. (Server Message Block / Common Internet File System) works as
application layer of the network. It works in multiplatform. It allows for interoperability between
UNIX or linux servers and windows based clients. It is a software package that gives network
administrator flexibility and freedom in terms of set up configuration and choice of system and
equipment.
20

Downloaded by Haneef Zain (haneefzain2003@gmail.com)

 lOMoARcPSD|27910594

6. Apache

Apache is the most widely used HTTP-server in the world today. It is also the most used
web server for a Linux system. A web server like Apache, in its simplest function, is a software that
displays and serves HTML pages hosted on a server to a client browser that understands the
HTML code.

7. Telnet

Telnet server is used to login into another system. You can use the telnet command to log in
remotely to another system on your network. The system can be on your local area network or
available through an Internet connection. Telnet operates as if you were logging in to another
system from a remote terminal. You will be asked for a login name and password. In effect,
you are logging in to another account on another system. In fact, if you have an account on
another system, you could use Telnet to log in to it.

21

Downloaded by Haneef Zain (haneefzain2003@gmail.com)