Part I !

Introduction to Perl Scripting

Dr. K. Najeeb

Professor and Head,

Department of Computer Science and Engg.
Govt. College of Engg. Kannur, Kerala

(Email: k.najeeb@gmail.com)

January 16, 2017

1/67
 Objectives of the Presentation

To expose participants the basic concepts about Perl

To Introduce the concept of Scripting using Perl and AWK

2/67
 Expected Outcomes

After the successful completion of the class,

participants shall be able to:
1 find applications of scripting in teaching
learning process
2 demonstrate the working Perl scripts to the
students
3 develop scripts for solving simple problems
4 identify problems to be given in the FOSS lab
introduces by KTU
5 adapt open source platform as the computational
framework
6 appraise importance of computing freedom for
research, development & technology

3/67
 Outline of the Presentation

1 Motivations

2 Basic concepts of Perl

3 Scripting using Perl

4 Regular Expressions in Perl

5 Introduction to AWK

4/67
 Outline

1 Motivations

2 Basic concepts of Perl

3 Scripting using Perl

4 Regular Expressions in Perl

5 Introduction to AWK

5/67
 Practice, Practice . . . . . . and Practice

Confucius 450BC

Tell me and I will forget. Show me, and I may remember.

Involve me, and I will understand.

We are what we repeatedly do. Excellence, then, is not an act,

but a habit

6/67
 Why do we Learn Different Programming Languages?

Learning new concept should help us to think differently

Help us to solve my problems more effectively
New concept may match our way of thinking
Improve our knowledge
To be successful, one must be willing to learn and apply new
concepts and not be afraid of changes

7/67
 Realities in Life

I am positive about all programming languages

Pick any language well suited to the task

Good developers are always most important, whatever

language is used

8/67
 Smart Scripting Languages

Julia: A language that walks like Python and Runs like C

Emerging star in the world of programming languages
It offers the coveted combination of high performance &
Productivity
Introduced by MIT
Go programming language
Combines the development speed of working in a dynamic
language like Perl with the performance and safety of complied
languages like C or C++
Introduced by Google
Python
Perl
Swift - by Apple Inc
Rust - by Mozilla Research
Scala
9/67
 Outline

1 Motivations

2 Basic concepts of Perl

3 Scripting using Perl

4 Regular Expressions in Perl

5 Introduction to AWK

10/67
 What is Perl?

Perl is an interpreted language

Perl=Practical Extraction and Report Language

11/67
 History of Perl

Perl was designed in the mid 1980s by Larry Wall

12/67
 Features of Perl

Very powerful in using Regular Expressions

It is free
Easy availability
All details are available on Web
Perl is available for Windows and Mac OS
Comprehensive Perl Archive network (CPAN)
Awesome community for support
Extensible library modules
Perl = Procedural+Functional+Object oriented

13/67
 Where we fit Perl?

Powerful level of abstraction

14/67
 What Perl Does Well?

Pattern matching with regular expressions

String processing
Hash tables
File I/O
Network and database access
CGI (website)

15/67
 Some of the Applications

Text processing - textual data, reports, email filtering, news,

log files etc
Linux administration tasks - scripts, file systems, networking
etc.
Web programming - to process and generate HTML,
Extraction, Text summarizing etc.
Database interaction - Perl’s DBI module interacts with all
kinds of databases
Internet programming - Perl modules for socket programming
High level programming
Research & development

16/67
 How is it different from C?

No need of defining data type

All variables are defined without explicit declaration

17/67
 Example 1

Pl. open a terminal and type perl -de1

18/67
 Outline

1 Motivations

2 Basic concepts of Perl

3 Scripting using Perl

4 Regular Expressions in Perl

5 Introduction to AWK

19/67
 How to Run Perl Program?
Option 1
Run Perl script with .pl extension from the command line with
the perl command

20/67
 How to Run Perl Program?
Option 1
Run Perl script with .pl extension from the command line with
the perl command

Option 2
Run Perl with -e option

20/67
 How to Run Perl Program?
Option 3
Run with eval perl

Press ctrl-d (∧d) on a new line to complete input

21/67
 How to Run Perl Program?
Option 3
Run with eval perl

Press ctrl-d (∧d) on a new line to complete input

Option 4
Run the script by adding execute permission to the script
$chmod 766 sum.pl
Execute the script by typing ./sum in the terminal
21/67
 Variable Naming Rules in Perl

1 Variable names can start with a letter, a number, or an

underscore, although they normally begin with a letter
2 Followed by any combination of letters, numbers, and the
underscore
3 Variable names are case sensitive
4 Variable name can be as long as you want but normally
limited to 255 characters

22/67
 Perl > Data Types

Scalar type
Scalars are simple variables
They are preceded by a dollar sign $
It can be of type:
integer, floating point, string,binary data,reference
Arrays or Lists:- Arrays are basically an ordered collection of
scalars and accessed by indices from zero on-wards
They are preceded by @
Can contain mixture of scalar types
Hashes / Dictionary:- are unordered sets of key/value pairs
Start with percent sign %
Keys are unique and can be any scalar value
Values can be any scalar value

23/67
 Example 1 ~ example1.pl

Go to the Perl folder

Open the file example1.pl
Analyze all the lines
Execute

24/67
 Dissection of example1.pl

25/67
 Example2 ~ example2.pl

#!/usr/bin/perl
$inputline = <STDIN>;
print( $inputline );

About the script

1 $inputline = <STDIN> ;reads one line in from the keyboard
including the newline character ( \n) and puts it into scalar
variable $inputline
2 Instead of <STDIN>, we can also use <>

26/67
 Try this Script!!!

Problem Definition
Write a script to read radius of a circle through keyboard and the
display radius, diameter, circumference and area of the circle on the
screen

27/67
 Try this Script!!!

Problem Definition
Write a script to read radius of a circle through keyboard and the
display radius, diameter, circumference and area of the circle on the
screen
Refer Refer example3.pl in the directory

#! usr/bin/perl
print "What is the radius of the circle?";
chomp ($r = <>);
$diameter = (2 * $r);
$area = (3.14 * ($r ** 2));
$cir = ($diameter * 3.14);
print "Radius: $r\n Diameter: $diameter\n
Circumference: $cir\n Area: $area \n";

27/67
 Array Creation and Use~ example4.pl

#! usr/bin/perl
@array = (1, 2, ’Hello’);
@days = qw/Mon Tue Wed Thu Fri Sat Sun/;
print "$days[0]\n";
print "$days[-1]\n";
@var_10 = (1..10);
print "@var_10\n";
@var_abc = (a..z);
print "@var_abc \n";
print "size of array days=",scalar @days,"\n";
print "max index of array days =",$#days, "\n";

28/67
 Adding and Removing Elements in the Array
~ example5.pl

#!/usr/bin/perl
@days=qw/mon tue wed thu fri sat sun/;
pop @days;
print "@days \n";
push @days,"sun";
print "@days \n";
shift @days;
print "@days \n";
unshift @days,"mon";
print "@days \n";
splice(@days, 2, 3, (’sun’,’sat’,’fri’));
print "After - @days\n";

29/67
 Try This

Problem Statement
How to remove an element from the middle of an array in Perl?

30/67
 Try This

Problem Statement
How to remove an element from the middle of an array in Perl?

Try example6.pl from the directory

my @days = (’mon’,’tue’,’wed’,’thu’,’fri’,’sat’,’sun’);
$size=int(@days/2);
splice @days, $size, 1;
print "@days\n"; #String Interpolation

30/67
 Slicing Array Elements ~ example7.pl

#!/usr/bin/perl
@days = qw/Mon Tue Wed Thu Fri Sat Sun/;
@weekdays = @days[3,4,5];
print "@weekdays\n";
@weekdays = @days[3..5];
print "@weekdays\n";
@weekdays = @days[-4..-2];
print "@weekdays\n";
@weekday = @days[4..2];
print "@weekday\n";

31/67
 Hashes ~ example8.pl

A hash is a set of key/value pairs

For ex:- %fruit = (’apples’, 3, ’oranges’, 6);

32/67
 Hashes ~ example8.pl

A hash is a set of key/value pairs

For ex:- %fruit = (’apples’, 3, ’oranges’, 6);

#!/usr/bin/perl
%data = (’Apple’, 145, ’Banana’, 30, ’Grapes’, 60);
%data1 = (’John Paul’ => 45, ’Lisa’ => 30, ’Kumar’ => 40);
print "\$data{’Apple’} = $data{’Apple’}\n";
print "\$data{’Banana’} = $data{’Banana’}\n";
print "\$data{’Grapes’} = $data{’Grapes’}\n";
print %data, "\n";
while ( ($k,$v) = each %data1 ) {
print "$k => $v\n";
}

32/67
 Operators in Perl

33/67
 while loop
until loop
for loop
for each loop
do while loop
nested loop

34/67
 while and for loop

while loop example -

example9.pl

#!/usr/local/bin/perl
$a = 10;
while( $a < 20 )
{
printf "Value of a: $a\n";
$a = $a + 1;
}

35/67
 while and for loop

while loop example -

example9.pl for loop example-
example10.pl
#!/usr/local/bin/perl
$a = 10; #!/usr/local/bin/perl
while( $a < 20 ) for($a=10; $a<20; $a=$a+1)
{ {
printf "Value of a: $a\n"; print "value of a: $a\n";
$a = $a + 1; }
}

35/67
 until and do while loop

until loop
example example11.pl

#!/usr/local/bin/perl
$a = 5;
until( $a > 10 )
{
printf "Value of a: $a\n";
$a = $a + 1;
}

36/67
 until and do while loop

until loop do while example

example example11.pl example12.pl

#!/usr/local/bin/perl #!/usr/local/bin/perl
$a = 5; $a = 5;
until( $a > 10 ) until( $a > 10 )
{ {
printf "Value of a: $a\n"; printf "Value of a: $a\n";
$a = $a + 1; $a = $a + 1;
} }

36/67
 if Statement

Example 2
example15.pl
Example 1
example14.pl print "What is your age? ";
my $age = <STDIN>;
print "What is your age? "; if ($age >= 18)
my $age = <STDIN>; {
if ($age >= 18) print "you can vote.\n";
{ }
print "you can vote.\n"; else
} {
print "You are young\n";
}

37/67
 Switch Statement in Perl

#!/usr/local/bin/perl
use Switch;
$var = 10;
@array = (10, 20, 30);
%hash = (’key1’ => 10, ’key2’ => 20);

switch($var){
case 10 {print "number 100\n"}
case "a" {print "string a" }
case [1..10,42] {print "no. in list"}
case (\@array) {print "no. in list"}
case (\%hash) {print "no. in hash"}
else {print "not true"}
}

38/67
 2D array example

my $Table;
for($i=0;$i <3;$i++) {
for($j=0;$j < 3; $j++) {
printf ("enter number Table[%d][%d] ==>",$i,$j);
$value=<STDIN>;
chop($value); # Chop last character
$Table->[$i][$j] = $value; } }
for($i=0;$i <3;$i++) {
for($j=0;$j < 3; $j++){
printf(" %d ",$Table->[$i][$j]); }
print " \n"; }

39/67
 What is the Output of the Script?

@list=(1..5);
$size= @list-1;
$size1= $#list;
printf("%d %d \n", $size,$size1);

40/67
 What is the Output of the Script?

@list=(1..5);
$size= @list-1;
$size1= $#list;
printf("%d %d \n", $size,$size1);

See example18.pl 4 4

40/67
 System Variables in Perl

Concept
Variables which have a predefined and special meaning in Perl

$/ : input line separator - execute script example20.pl

$” : Array element separator - execute example21.pl
$! : System Error message
$. : Line no. last line read from file
$ : Default scalar variable
The most commonly used global scalar variable is the $
variable.
Many Perl functions and operators modify the contents of $ if
we do not explicitly specify the scalar variable on which they
are to operate
refer example22.pl

41/67
 Pointer Concept in Perl ~ example23.pl

my $scalar = "This is a scalar";

my $scalar_ref = \$scalar;
print "Reference: " . $scalar_ref . "\n";
print "Dereferenced: " . $$scalar_ref . "\n";

42/67
 Function concept ~ example24.pl

sub check
{
my ($a,$b) = @_; #@_ is Array system variable
$k = $a+$b;
return $k;
}
$e= check(3,4);
print "\n $e\n";

43/67
 Identify Syntax Errors, if any
Script 1
if ($remaining > 0)
print $x;

44/67
 Identify Syntax Errors, if any
Script 1
if ($remaining > 0)
print $x;

Missing braces
Script 2
for ($i = 1; $i < $n; $i++)
print a[$i], "\n";

44/67
 Identify Syntax Errors, if any
Script 1
if ($remaining > 0)
print $x;

Missing braces
Script 2
for ($i = 1; $i < $n; $i++)
print a[$i], "\n";

Missing $ for a
Script 3
while ($c = 1)
{
$c = do_thing($m, $q);
44/67 }
 Identify Syntax Errors, if any
Script 1
if ($remaining > 0)
print $x;

Missing braces
Script 2
for ($i = 1; $i < $n; $i++)
print a[$i], "\n";

Missing $ for a
Script 3
while ($c = 1)
{ == instead of =
$c = do_thing($m, $q);
44/67 }
 Outline

1 Motivations

2 Basic concepts of Perl

3 Scripting using Perl

4 Regular Expressions in Perl

5 Introduction to AWK

45/67
 Regular Expressions in Perl
What is Regular Expression?
A regular expression is a string of characters that defines a
pattern
Regular expressions in Perl are similar to sed, grep, and awk
Perl has a powerful Reg Exp engine
Allows us to look for patterns in our data

Why RegExps?
Given a large text file. We have to delete all the blank lines
scattered here and there.
As system administrator, you have to delete all video and mp3
files recursively traversing all the user directories
Linux admin has to analyze the log file for security reasons
Matching passwords in applications based on the conditions
Text validation
46/67
 Why RegExps?

Consider the following password matching scenario:

Scenario
6 to 12 characters in length
Must have at least one uppercase letter
Must have at least one lower case letter
Must have at least one digit
Should contain other characters

47/67
 Why RegExps?

Consider the following password matching scenario:

Scenario
6 to 12 characters in length
Must have at least one uppercase letter
Must have at least one lower case letter
Must have at least one digit
Should contain other characters

Sample Pattern
^(?=.*[a-z])(?=.*[A-Z])(?=.*\d).{6,12}$

47/67
 Try this Problem

Problem statement
Write a Perl script to read a text file and find all matches for a string
foo

48/67
 Try this Problem

Problem statement
Write a Perl script to read a text file and find all matches for a string
foo

Refer example17.pl

48/67
 Example17.pl - An overview

printf("Enter text file name to read ==>");

$infile =<STDIN>;
undef $/; #
open (INFILE, $infile)||die "Could not open $infile: $!";
#$!If a system library function generates
#an error, the error code generated by the
#function is assigned to the $! variable
$string = <INFILE>;
print "$string \n \n";
while ($string =~ m/foo/g)
{
print "Found " ,$& , "Next attempt at character " ,
pos($string)+1 , "\n";
}
close INFILE;
$/="\n";
49/67
 Regular Expressions

50/67
 Reg Exp. Contd

51/67
 Predict the Output

$id = ’[a-zA-Z][a-zA-Z0-9_]*’;
$line = " and AND1 (a,b,c)";
if ($line=~/^\s*(and|or|nand|xor)\s*($id)\s*\((.*)\)/xigs)
# x i g s are match operator modifiers
# x --- Allows to use white space in the expression
# for clarity.
# i --- Makes the match case insensitive.
# g --- Globally finds all matches.
# s --- Allows use of . to match a newline character.
{
print "\nTrue\n";
}

52/67
 Do this task

1 Create a Directory with name Veerappan

2 Create and store some dummy pdf and other types files into it
3 See the absolute path name of the folder using Linux
command
4 Edit the perl script example26.pl from the folder by $path
variable assignment as the new path
5 Run the perl script and comment on the output

53/67
 Predict the Output ~ example27.pl

$string = " Oru nal varum ";

$string =~ s/^\s+//;
$string =~ s/\s+$//;
print "output is <<$string>> \n";

54/67
 Predict the Output ~ example27.pl

$string = " Oru nal varum ";

$string =~ s/^\s+//;
$string =~ s/\s+$//;
print "output is <<$string>> \n";

∧ and $ match the very start and very end of the string

54/67
 Linux Basics: Quick Review

What is this command for?

$ find . | grep ’[a-zA-Z]*\.sxw’

55/67
 Linux Basics: Quick Review

What is this command for?

$ find . | grep ’[a-zA-Z]*\.sxw’

55/67
 Part II: AWK Programming

56/67
 Outline

1 Motivations

2 Basic concepts of Perl

3 Scripting using Perl

4 Regular Expressions in Perl

5 Introduction to AWK

57/67
 Introduction

AWK is an interpreted programming language

Awk is a powerful language to manipulate and process text
files
It is a standard feature of Linux operating systems
Its name is derived from the family names of its authors
Alfred Aho, Peter Weinberger , and Brian Kernighan

In FOSS Lab
Cycles may be given for
Text processing
Producing formatted text reports
Performing string operations

58/67
 Example: AWK for producing formatted text output

Let us create a file data1.txt which contains the serial

number, name of the student, subject name, and number of
marks obtained
Let us now display the file contents with header by using
AWK script.

$ awk ’BEGIN{printf "Sr No\tName\tSub\tMarks\n"} {print}’

marks.txt

59/67
 Try this

Problem Statement
Assume that you have been recruited as a developer in a Company.
On the first day itself, Your BIG BOSS has given a large text file and
directed you to remove all blank lines and return the modified one
within say 3 minutes. Suggest four possible ways to find solutions
to this problem

60/67
 Try this

Problem Statement
Assume that you have been recruited as a developer in a Company.
On the first day itself, Your BIG BOSS has given a large text file and
directed you to remove all blank lines and return the modified one
within say 3 minutes. Suggest four possible ways to find solutions
to this problem

Solution 1
Load this file using one editor and manually delete all blank lines

60/67
 Try this

Problem Statement
Assume that you have been recruited as a developer in a Company.
On the first day itself, Your BIG BOSS has given a large text file and
directed you to remove all blank lines and return the modified one
within say 3 minutes. Suggest four possible ways to find solutions
to this problem

Solution 1
Load this file using one editor and manually delete all blank lines

Solution 2
Write a script and solve the problem. See the script example28.pl

60/67
 Try this
Problem Statement
Assume that you have been recruited as a developer in a Company.
On the first day itself, Your BIG BOSS has given a large text file and
directed you to remove all blank lines and return the modified one
within say 3 minutes. Suggest four possible ways to find solutions
to this problem

Solution 3
Use one powerful editor and use RegExp features to complete the
task

1,$ s/^ *\n//g

61/67
 Try this
Problem Statement
Assume that you have been recruited as a developer in a Company.
On the first day itself, Your BIG BOSS has given a large text file and
directed you to remove all blank lines and return the modified one
within say 3 minutes. Suggest four possible ways to find solutions
to this problem

Solution 3
Use one powerful editor and use RegExp features to complete the
task

1,$ s/^ *\n//g

Solution 4
awk ’NF’ data4.txt > data5.txt

61/67 NF is a built-in variable whose value is the number of fields in

 AWK example

$ awk -F: ’/mail/ {print $1}’ /etc/passwd

F : field separator
/mail/ is the pattern
print is the action
$1 indicates first field of record
/etc/passwd is the input file

62/67
 AWK Script Structure

A typical awk script has following three blocks

1 BEGIN block, which executed only once at the beginning
syntax is BEGIN { awk-commands }
The begin block is a good place to print report headers, and
initialize variables
The keyword BEGIN should be specified in upper case
Begin block is optional.
2 Body Block
syntax is /pattern/ {action}
The body block gets executed once for every line in the input
file
There is no keyword for the body block
3 END Block
Syntax is END { awk-commands }
The end block gets executed only once at the end, after awk
completes executing the body block for all the lines in the
input-file
End block is optional.

63/67
 AWK Workflow

64/67
 AWK Example ~ script2.awk

$ awk ’BEGIN { FS=":";print "---header---" } \

/mail/ {print $1} \
END { print "---footer---"}’ /etc/passwd

Run above script using the command

awk -f script2.awk /etc/passwd

65/67
 Part III: Make utility

66/67
67/67