PAAVENDHAR COLLEGE OF ARTS & SCIENCE, M.V.

SOUTH UNIT-V:
DEPARTMENT OF COMPUTER SCIENCE
II MSC COMPUTER SCIENCE Concurrency: Queues –Processes –Threads –Green Threads
OPEN SOURCE COMPUTING and gevent –twisted –Redis. Networks: Patterns –The Publish-
Subscribe Model –TCP/IP –Sockets –ZeroMQ –Internet Services –
UNIT-I: Web Services and APIs –Remote Processing –Big Fat Data and Map
Reduce –Working in the Clouds.
Python: Introduction–Numbers–Strings–Variables –Lists–
Tuples–Dictionaries–Sets –Comparison.
TEXT BOOK:
UNIT-II:
1. Bill Lubanovic, ―Introducing Python‖, O‟Reilly, First Edition-
Code Structures: if, elif, and else –Repeat with while–Iterate Second Release, 2014.
with for –Comprehensions –Functions –Generators –Decorators –
Namespaces and Scope –Handle Errors with try and except –User
Exceptions. Modules, Packages, and Programs: Standalone Programs
–Command-Line Arguments –Modules and the import Statement –
The Python Standard Library. Objects and Classes: Define a Class
with class –Inheritance –Override a Method –Add a Method –Get
Help from Parent with super –In self Defense –Get and Set Attribute
Values with Properties –Name Mangling for Privacy –Method Types
–Duck Typing –Special Methods –Composition.

UNIT-III:

Data Types: Text Strings –Binary Data. Storing and Retrieving

Data: File Input/Output –Structured Text Files –Structured Binary
Files -Relational Databases –No SQL Data Stores.

UNIT-IV:

Web: Web Clients –Web Servers –Web Services and

Automation –Systems: Files –Directories –Programs and Processes –
Calendars and Clocks.

1
 Python Syntax compared to other programming languages
Paavendhar College of Arts & Science, M.V.South  Python was designed for readability, and has some
Department of Computer Science similarities to the English language with influence from
Open Source Computing mathematics.
II MSC CS  Python uses new lines to complete a command, as opposed to
UNIT I other programming languages which often use semicolons or
INTRODUCTION parentheses.
What is Python?  Python relies on indentation, using whitespace, to define
Python is a popular programming language. It was created by scope; such as the scope of loops, functions and classes. Other
Guido van Rossum, and released in 1991. programming languages often use curly-brackets for this
It is used for: purpose.
 web development (server-side), Python Install
 software development,  Many PCs and Macs will have python already installed.
 mathematics,  To check if you have python installed on a Windows PC,
 system scripting. search in the start bar for Python or run the following on the
What can Python do? Command Line (cmd.exe):
 Python can be used on a server to create web applications. C:\Users\Your Name>python --version
 Python can be used alongside software to create workflows.
 Python can connect to database systems. It can also read and  To check if you have python installed on a Linux or Mac, then
modify files. on linux open the command line or on Mac open the Terminal
 Python can be used to handle big data and perform complex and type:
mathematics.  python --version
 Python can be used for rapid prototyping, or for production-
ready software development. Python Quickstart
Why Python? Python is an interpreted programming language, this means
 Python works on different platforms (Windows, Mac, Linux, that as a developer you write Python (.py) files in a text editor and
Raspberry Pi, etc). then put those files into the python interpreter to be executed.
 Python has a simple syntax similar to the English language.
 Python has syntax that allows developers to write programs The way to run a python file is like this on the command line:
with fewer lines than some other programming languages. C:\Users\Your Name>python helloworld.py
 Python runs on an interpreter system, meaning that code can Where "helloworld.py" is the name of your python file.
be executed as soon as it is written. This means that Let's write our first Python file, called helloworld.py, which
prototyping can be very quick. can be done in any text editor.
 Python can be treated in a procedural way, an object- helloworld.py
orientated way or a functional way. print("Hello, World!")

2
 Simple as that. Save your file. Open your command line, Whenever you are done in the python command line, you can
navigate to the directory where you saved your file, and run: simply type the following to quit the python command line interface:
C:\Users\Your Name>python helloworld.py exit().

The output should read: Execute Python Syntax

Hello, World! As we learned in the previous page, Python syntax can be
The Python Command Line executed by writing directly in the Command Line:
To test a short amount of code in python sometimes it is >>> print("Hello, World!")
quickest and easiest not to write the code in a file. This is made Hello, World!
possible because Python can be run as a command line itself. Or
by creating a python file on the server, using the .py file extension,
Type the following on the Windows, Mac or Linux command line: and running it in the Command Line:
C:\Users\Your Name>python myfile.py
C:\Users\Your Name>python
Python Indentations
From there you can write any python, including our hello  Where in other programming languages the indentation in code is
world example from earlier in the tutorial: for readability only, in Python the indentation is very important.
 Python uses indentation to indicate a block of code.
C:\Users\Your Name>python
Example
Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:04:45) [MSC v.1900 32 if 5 > 2:
bit (Intel)] on win32 print("Five is greater than two!")
Type "help", "copyright", "credits" or "license" for more information. Python will give you an error if you skip the indentation:
Example
>>> print("Hello, World!") if 5 > 2:
print("Five is greater than two!")
Which will write "Hello, World!" in the command line:
Comments
C:\Users\Your Name>python  Python has commenting capability for the purpose of in-code
documentation.
Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:04:45) [MSC v.1900 32  Comments start with a #, and Python will render the rest of
bit (Intel)] on win32 the line as a comment:
Type "help", "copyright", "credits" or "license" for more information. Example
Comments in Python:
>>> print("Hello, World!") #This is a comment.
Hello, World! print("Hello, World!")
3
Docstrings  A variable name must start with a letter or the underscore character
 Python also has extended documentation capability, called  A variable name cannot start with a number 
docstrings.  A variable name can only contain alpha-numeric characters and
 Docstrings can be one line, or multiline. underscores (A-z, 0-9, and _ )
 Python uses triple quotes at the beginning and end of the  Variable names are case-sensitive (age, Age and AGE are three
docstring: different variables)
Example Remember that variables are case-sensitive
Docstrings are also comments:
"""This is a Output Variables
multiline docstring.""" The Python print statement is often used to output variables.
print("Hello, World!") To combine both text and a variable, Python uses the + character:
Example
VARIABLES x = "awesome"
print("Python is " + x)
Creating Variables You can also use the + character to add a variable to another
 Unlike other programming languages, Python has no variable:
command for declaring a variable. Example
 A variable is created the moment you first assign a value to it. x = "Python is "
Example y = "awesome"
x=5 z=x+y
y = "John" print(z)
print(x) For numbers, the + character works as a mathematical operator:
print(y) Example
Variables do not need to be declared with any particular type x=5
and can even change type after they have been set. y = 10
print(x + y)
If you try to combine a string and a number, Python will give you an
Example error:
x = 4 # x is of type int Example
x = "Sally" # x is now of type str x=5
print(x) y = "John"
print(x + y)
Variable Names NUMBERS
A variable can have a short name (like x and y) or a more Python Numbers
descriptive name (age, carname, total_volume). Rules for Python There are three numeric types in Python:
variables:  int

4
 float print(type(y))
 complex print(type(z))
Variables of numeric types are created when you assign a value to Float can also be scientific numbers with an "e" to indicate the power
them: of 10.
Example Example
x = 1 # int Floats:
y = 2.8 # float x = 35e3
z = 1j # complex y = 12E4
To verify the type of any object in Python, use the type() function: z = -87.7e100
Example print(type(x))
print(type(x)) print(type(y))
print(type(y)) print(type(z))
print(type(z))
Complex
Int Complex numbers are written with a "j" as the imaginary part:
Int, or integer, is a whole number, positive or negative, Example
without decimals, of unlimited length. Complex:
Example x = 3+5j
Integers: y = 5j
x=1 z = -5j
y = 35656222554887711 print(type(x))
z = -3255522 print(type(y))
print(type(x)) print(type(z))
print(type(y))
print(type(z)) Specify a Variable Type
There may be times when you want to specify a type on to a
Float variable. This can be done with casting.
Float, or "floating point number" is a number, positive or Python is an object-orientated language, and as such it uses
negative, containing one or more decimals. classes to define data types, including its primitive types.
Example Casting in python is therefore done using constructor functions:
Floats:  int() - constructs an integer number from an integer literal, a
x = 1.10 float literal (by rounding down to the previous whole
y = 1.0 number), or a string literal (providing the string represents a
z = -35.59 whole number)
print(type(x))

5
  float() - constructs a float number from an integer literal, a a = "Hello, World!"
float literal or a string literal (providing the string represents a print(a[1])
float or an integer) Example
 str() - constructs a string from a wide variety of data types, Substring. Get the characters from position 2 to position 5 (not
including strings, integer literals and float literals included):
Example b = "Hello, World!"
Integers: print(b[2:5])
x = int(1) # x will be 1 Example
y = int(2.8) # y will be 2 The strip() method removes any whitespace from the beginning or
z = int("3") # z will be 3 the end:
Example a = " Hello, World! "
Floats: print(a.strip()) # returns "Hello, World!"
x = float(1) # x will be 1.0 Example
y = float(2.8) # y will be 2.8 The len() method returns the length of a string:
z = float("3") # z will be 3.0 a = "Hello, World!"
w = float("4.2") # w will be 4.2 print(len(a))
Example Example
Strings: The lower() method returns the string in lower case:
x = str("s1") # x will be 's1' a = "Hello, World!"
y = str(2) # y will be '2' print(a.lower())
z = str(3.0) # z will be '3.0' Example
The upper() method returns the string in upper case:
String Literals a = "Hello, World!"
String literals in python are surrounded by either single print(a.upper())
quotation marks, or double quotation marks. Example
'hello' is the same as "hello". The replace() method replaces a string with another string:
Strings can be output to screen using the print function. For a = "Hello, World!"
example: print("hello"). print(a.replace("H", "J"))
Like many other popular programming languages, strings in Example
Python are arrays of bytes representing unicode characters. The split() method splits the string into substrings if it finds instances
However, Python does not have a character data type, a single of the separator:
character is simply a string with a length of 1. Square brackets can be a = "Hello, World!"
used to access elements of the string. print(a.split(",")) # returns ['Hello', ' World!']
Example Command-line String Input
Get the character at position 1 (remember that the first  Python allows for command line input.
character has the position 0):  That means we are able to ask the user for input.
6
The following example asks for the user's name, then, by using the / Division x/y
input() method, the program prints the name to the screen: % Modulus x%y
Example
demo_string_input.py ** Exponentiation x ** y
print("Enter your name:") // Floor division x // y
x = input()
print("Hello, ", x) Python Assignment Operators
Save this file as demo_string_input.py, and load it through the Assignment operators are used to assign values to variables:
command line: Operator Example Same As Try it
C:\Users\Your Name>python demo_string_input.py = x=5 x=5
Our program will prompt the user for a string:
+= x += 3 x = x + 3
Enter your name:
The user now enters a name: -= x -= 3 x = x - 3
Linus *= x *= 3 x = x * 3
Then, the program prints it to screen with a little message: /= x /= 3 x = x / 3
Hello, Linus
%= x %= 3 x = x % 3
Python Operators //= x //= 3 x = x // 3
Operators are used to perform operations on variables and values. **= x **= 3 x = x ** 3
Python divides the operators in the following groups: &= x &= 3 x = x & 3
 Arithmetic operators |= x |= 3 x = x | 3
 Assignment operators
^= x ^= 3 x = x ^ 3
 Comparison operators
 Logical operators
>>= x >>= 3 x = x >> 3
 Identity operators <<= x <<= 3 x = x << 3
 Membership operators
 Bitwise operators Python Comparison Operators
Comparison operators are used to compare two values:
Python Arithmetic Operators Operator Name Example Try it
Arithmetic operators are used with numeric values to perform == Equal x == y
common mathematical operations:
!= Not equal x != y
Operator Name Example
> Greater than x>y
+ Addition x+y
< Less than x<y
- Subtraction x-y
>= Greater than or equal to x >= y
* Multiplication x * y
7
<= Less than or equal to x <= y Python Bitwise Operators
Bitwise operators are used to compare (binary) numbers:
Python Logical Operators Operator Name Description
Logical operators are used to combine conditional statements: & AND Sets each bit to 1 if both bits are 1
Operator Description Example Try it | OR Sets each bit to 1 if one of two bits is 1
Returns True if both ^ XOR Sets each bit to 1 if only one of two bits is 1
and x < 5 and x < 10
statements are true ~ NOT Inverts all the bits
or Returns True if one of the statements is true x < 5 or x < 4 Zero fill Shift left by pushing zeros in from the right and
<<
Reverse the result, returns False if the result not(x < 5 and x < left shift let the leftmost bits fall off
not
is true 10) Signed
Shift right by pushing copies of the leftmost bit in
>> right
Python Identity Operators from the left, and let the rightmost bits fall off
shift
Identity operators are used to compare the objects, not if they
are equal, but if they are actually the same object, with the same Python Collections (Arrays)
memory location: There are four collection data types in the Python
Operator Description Example Try it programming language:
Returns true if both  List is a collection which is ordered and changeable. Allows
is variables are the same x is y duplicate members.
object  Tuple is a collection which is ordered and unchangeable.

is not Returns true if both variables are not the same object x is not y Allows duplicate members.
 Set is a collection which is unordered and unindexed. No
duplicate members.
Python Membership Operators
 Dictionary is a collection which is unordered, changeable and
Membership operators are used to test if a sequence is presented in
indexed. No duplicate members.
an object:
Operator Description Example Try it When choosing a collection type, it is useful to understand the
Returns True if a properties of that type. Choosing the right type for a particular data
sequence with the set could mean retention of meaning, and, it could mean an increase
in x in y
specified value is in efficiency or security.
present in the object
not Returns True if a sequence with the specified value is x not List:
in not present in the object in y A list is a collection which is ordered and changeable. In
Python lists are written with square brackets.

8
Example thislist = ["apple", "banana", "cherry"]
Create a List: if "apple" in thislist:
thislist = ["apple", "banana", "cherry"] print("Yes, 'apple' is in the fruits list")
print(thislist)
List Length
Access Items To determine how many items a list has, use the len() method:
You access the list items by referring to the index number: Example
Example Print the number of items in the list:
Print the second item of the list: thislist = ["apple", "banana", "cherry"]
thislist = ["apple", "banana", "cherry"] print(len(thislist))
print(thislist[1])
Add Items
Change Item Value To add an item to the end of the list, use the append() method:
To change the value of a specific item, refer to the index number: Example
Example Using the append() method to append an item:
Change the second item: thislist = ["apple", "banana", "cherry"]
thislist = ["apple", "banana", "cherry"] thislist.append("orange")
thislist[1] = "blackcurrant" print(thislist)
print(thislist) To add an item at the specified index, use the insert() method:
Example
Insert an item as the second position:
Loop Through a List thislist = ["apple", "banana", "cherry"]
You can loop through the list items by using a for loop: thislist.insert(1, "orange")
Example print(thislist)
Print all items in the list, one by one:
thislist = ["apple", "banana", "cherry"] Remove Item
for x in thislist: There are several methods to remove items from a list:
print(x) Example
The remove() method removes the specified item:
Check if Item Exists thislist = ["apple", "banana", "cherry"]
To determine if a specified item is present in a list use the in thislist.remove("banana")
keyword: print(thislist)
Example Example
Check if "apple" is present in the list: The pop() method removes the specified index, (or the last
item if index is not specified):

9
 thislist = ["apple", "banana", "cherry"] The list() Constructor
thislist.pop() It is also possible to use the list() constructor to make a new
print(thislist) list.
Example Example
The del keyword removes the specified index: Using the list() constructor to make a List:
thislist = ["apple", "banana", "cherry"] thislist = list(("apple", "banana", "cherry")) # note the
del thislist[0] double round-brackets
print(thislist) print(thislist)
Example
The del keyword can also delete the list completely: List Methods
thislist = ["apple", "banana", "cherry"] Python has a set of built-in methods that you can use on lists.
del thislist Method Description
Example
append() Adds an element at the end of the list
The clear() method empties the list:
thislist = ["apple", "banana", "cherry"] clear() Removes all the elements from the list
thislist.clear() copy() Returns a copy of the list
print(thislist) Returns the number of elements with the
count()
specified value
Copy a List
You cannot copy a list simply by typing list2 = list1, because: Add the elements of a list (or any iterable), to the
extend()
list2 will only be a reference to list1, and changes made in list1 will end of the current list
automatically also be made in list2. Returns the index of the first element with the
There are ways to make a copy, one way is to use the built-in index()
specified value
List method copy().
insert() Adds an element at the specified position
Example
Make a copy of a list with the copy() method: pop() Removes the element at the specified position
thislist = ["apple", "banana", "cherry"] remove() Removes the item with the specified value
mylist = thislist.copy()
print(mylist) reverse() Reverses the order of the list
Another way to make a copy is to use the built-in method list(). sort() Sorts the list
Example
Make a copy of a list with the list() method:
thislist = ["apple", "banana", "cherry"] Tuple:
mylist = list(thislist) A tuple is a collection which is ordered and unchangeable. In
print(mylist) Python tuples are written with round brackets.

10
Example Tuple Length
Create a Tuple: To determine how many items a tuple has, use the len()
thistuple = ("apple", "banana", "cherry") method:
print(thistuple) Example
Print the number of items in the tuple:
Access Tuple Items thistuple = ("apple", "banana", "cherry")
You can access tuple items by referring to the index number, inside print(len(thistuple))
square brackets:
Example Add Items
Return the item in position 1: Once a tuple is created, you cannot add items to it. Tuples are
thistuple = ("apple", "banana", "cherry") unchangeable.
print(thistuple[1]) Example
You cannot add items to a tuple:
Change Tuple Values thistuple = ("apple", "banana", "cherry")
Once a tuple is created, you cannot change its values. Tuples thistuple[3] = "orange" # This will raise an error
are unchangeable. print(thistuple)

Loop Through a Tuple Remove Items

You can loop through the tuple items by using a for loop. Note: You cannot remove items in a tuple.
Example Tuples are unchangeable, so you cannot remove items from it, but
Iterate through the items and print the values: you can delete the tuple completely:
thistuple = ("apple", "banana", "cherry") Example
for x in thistuple: The del keyword can delete the tuple completely:
print(x) thistuple = ("apple", "banana", "cherry")
del thistuple
Check if Item Exists print(thistuple)
To determine if a specified item is present in a tuple use the in #this will raise an error because the tuple no longer exists
keyword: The tuple() Constructor
Example It is also possible to use the tuple() constructor to make a
Check if "apple" is present in the tuple: tuple.
thistuple = ("apple", "banana", "cherry") Example
if "apple" in thistuple: Using the tuple() method to make a tuple:
print("Yes, 'apple' is in the fruits tuple") thistuple = tuple(("apple", "banana", "cherry")) # note the
double round-brackets
print(thistuple)

11
Tuple Methods Change Items
Python has two built-in methods that you can use on tuples. Once a set is created, you cannot change its items, but you can
Method Description add new items.
Returns the number of times a specified value occurs
count() Add Items
in a tuple
To add one item to a set use the add() method.
Searches the tuple for a specified value and returns the
index() To add more than one item to a set use the update() method.
position of where it was found
Example
Add an item to a set, using the add() method:
Set: thisset = {"apple", "banana", "cherry"}
A set is a collection which is unordered and unindexed. In thisset.add("orange")
Python sets are written with curly brackets. print(thisset)
Example Example
Create a Set: Add multiple items to a set, using the update() method:
thisset = {"apple", "banana", "cherry"} thisset = {"apple", "banana", "cherry"}
print(thisset) thisset.update(["orange", "mango", "grapes"])
Note: Sets are unordered, so the items will appear in a random print(thisset)
order. Get the Length of a Set
To determine how many items a set has, use the len() method.
Access Items Example
You cannot access items in a set by referring to an index, since Get the number of items in a set:
sets are unordered the items has no index. thisset = {"apple", "banana", "cherry"}
But you can loop through the set items using a for loop, or ask print(len(thisset))
if a specified value is present in a set, by using the in keyword. Remove Item
Example To remove an item in a set, use the remove(), or the discard()
Loop through the set, and print the values: method.
thisset = {"apple", "banana", "cherry"}
Example
for x in thisset: Remove "banana" by using the remove() method:
print(x) thisset = {"apple", "banana", "cherry"}
Example thisset.remove("banana")
Check if "banana" is present in the set: print(thisset)
thisset = {"apple", "banana", "cherry"} Note: If the item to remove does not exist, remove() will raise an
print("banana" in thisset) error.
Example
Remove "banana" by using the discard() method:

12
 thisset = {"apple", "banana", "cherry"} Set Methods
thisset.discard("banana") Python has a set of built-in methods that you can use on sets.
print(thisset) Method Description
Note: If the item to remove does not exist, discard() will NOT raise add() Adds an element to the set
an error.
Removes all the elements from
You can also use the pop(), method to remove an item, but clear() the set
this method will remove the last item. Remember that sets are
unordered, so you will not know what item that gets removed. copy() Returns a copy of the set
The return value of the pop() method is the removed item. Returns a set containing the
Example difference() difference between two or more
Remove the last item by using the pop() method: sets
thisset = {"apple", "banana", "cherry"} Removes the items in this set that
x = thisset.pop() difference_update() are also included in another,
print(x) specified set
print(thisset) discard() Remove the specified item
Note: Sets are unordered, so when using the pop() method, you will
Returns a set, that is the
not know which item that gets removed. intersection()
Example intersection of two other sets
The clear() method empties the set: Removes the items in this set that
thisset = {"apple", "banana", "cherry"} intersection_update() are not present in other, specified
thisset.clear() set(s)
print(thisset) Returns whether two sets have a
isdisjoint()
Example intersection or not
The del keyword will delete the set completely: Returns whether another set
thisset = {"apple", "banana", "cherry"} issubset()
contains this set or not
del thisset Returns whether this set contains
print(thisset) issuperset()
another set or not
The set() Constructor pop() Removes an element from the set
It is also possible to use the set() constructor to make a set. remove() Removes the specified element
Example Returns a set with the symmetric
Using the set() constructor to make a set: symmetric_difference()
differences of two sets
thisset = set(("apple", "banana", "cherry")) inserts the symmetric differences
# note the double round-brackets symmetric_difference_update()
from this set and another
print(thisset)

13
 Return a set containing the union Change the "year" to 2018:
union()
of sets thisdict = {
Update the set with the union of "brand": "Ford",
update() "model": "Mustang",
this set and others
"year": 1964
}
thisdict["year"] = 2018
Dictionary
A dictionary is a collection which is unordered, changeable
Loop Through a Dictionary
and indexed. In Python dictionaries are written with curly brackets,
You can loop through a dictionary by using a for loop.
and they have keys and values.
When looping through a dictionary, the return value are the keys of
Example
the dictionary, but there are methods to return the values as well.
Create and print a dictionary:
thisdict = { Example
Print all key names in the dictionary, one by one:
"brand": "Ford",
"model": "Mustang", for x in thisdict:
"year": 1964 print(x)
Example
}
Print all values in the dictionary, one by one:
print(thisdict)
for x in thisdict:
print(thisdict[x])
Accessing Items
Example
You can access the items of a dictionary by referring to its key
You can also use the values() function to return values of a
name, inside square brackets:
dictionary:
Example
for x in thisdict.values():
Get the value of the "model" key:
print(x)
x = thisdict["model"]
Example
There is also a method called get() that will give you the same result:
Loop through both keys and values, by using the items() function:
Example
for x, y in thisdict.items():
Get the value of the "model" key:
print(x, y)
x = thisdict.get("model")
Check if Key Exists
To determine if a specified key is present in a dictionary use
Change Values
the in keyword:
You can change the value of a specific item by referring to its
key name: Example
Check if "model" is present in the dictionary:
Example

14
 thisdict = { thisdict.pop("model")
"brand": "Ford", print(thisdict)
"model": "Mustang", Example
"year": 1964 The popitem() method removes the last inserted item (in versions
} before 3.7, a random item is removed instead):
if "model" in thisdict: thisdict = {
print("Yes, 'model' is one of the keys in the thisdict dictionary") "brand": "Ford",
"model": "Mustang",
Dictionary Length "year": 1964
To determine how many items (key-value pairs) a dictionary }
has, use the len() method. thisdict.popitem()
Example print(thisdict)
Print the number of items in the dictionary: Example
print(len(thisdict)) The del keyword removes the item with the specified key name:
thisdict = {
Adding Items "brand": "Ford",
Adding an item to the dictionary is done by using a new "model": "Mustang",
index key and assigning a value to it: "year": 1964
Example }
thisdict = { del thisdict["model"]
"brand": "Ford", print(thisdict)
"model": "Mustang", Example
"year": 1964 The del keyword can also delete the dictionary completely:
} thisdict = {
thisdict["color"] = "red" "brand": "Ford",
print(thisdict) "model": "Mustang",
"year": 1964
Removing Items }
There are several methods to remove items from a dictionary: del thisdict
Example print(thisdict) #this will cause an error because "thisdict" no longer
The pop() method removes the item with the specified key name: exists.
thisdict = { Example
"brand": "Ford", The clear() keyword empties the dictionary:
"model": "Mustang", thisdict = {
"year": 1964 "brand": "Ford",
} "model": "Mustang",
15
 "year": 1964 thisdict = dict(brand="Ford", model="Mustang", year=1964)
} # note that keywords are not string literals
thisdict.clear() # note the use of equals rather than colon for the assignment
print(thisdict) print(thisdict)

Copy a Dictionary Dictionary Methods

You cannot copy a dictionary simply by typing dict2 = dict1, Python has a set of built-in methods that you can use on dictionaries.
because: dict2 will only be a reference to dict1, and changes made in Method Description
dict1 will automatically also be made in dict2. clear() Removes all the elements from the dictionary
There are ways to make a copy, one way is to use the built-in
copy() Returns a copy of the dictionary
Dictionary method copy().
Example fromkeys() Returns a dictionary with the specified keys and values
Make a copy of a dictionary with the copy() method: get() Returns the value of the specified key
thisdict = { Returns a list containing the a tuple for each key value
"brand": "Ford", items()
pair
"model": "Mustang", keys() Returns a list containing the dictionary's keys
"year": 1964
pop() Removes the element with the specified key
}
mydict = thisdict.copy() popitem() Removes the last inserted key-value pair
print(mydict) Returns the value of the specified key. If the key does not
setdefault()
Another way to make a copy is to use the built-in method dict(). exist: insert the key, with the specified value
Example Updates the dictionary with the specified key-value
update()
Make a copy of a dictionary with the dict() method: pairs
thisdict = { values() Returns a list of all the values in the dictionary
"brand": "Ford",
"model": "Mustang",
"year": 1964
} UNIT I COMPLETED
mydict = dict(thisdict)
print(mydict)

The dict() Constructor

It is also possible to use the dict() constructor to make a new
dictionary:
Example

16
 print "1 - Got a true expression value"
UNIT -II print var1
var2 = 0
1) Explain briefly about Conditional statement in Python. (5 if var2:
Marks) print "2 - Got a true expression value"
print var2
Single Statement Suites print "Good bye!"
If the suite of an if clause consists only of a single line, it may
go on the same line as the header statement. When the above code is executed, it
Here is an example of a one-line if clause − produces the following result −
1 - Got a true
#!/usr/bin/python expression value
var = 100 100
if ( var == 100 ) : print "Value of expression is 100" Good bye!
print "Good bye!" IF ..... ELIF statement

When the above code is executed, it produces the following result − An else statement can be combined with an if statement. An
Value of expression is 100 else statement contains the block of code that executes if the
Good bye! conditional expression in the if statement resolves to 0 or a FALSE
IF Statement value.
It is similar to that of other languages. The if statement The else statement is an optional statement and there could be
contains a logical expression using which data is compared and a at most only one else statement following if.
decision is made based on the result of the comparison. Syntax
Syntax if expression:
if expression: statement(s)
statement(s) else:
statement(s)
If the boolean expression evaluates to TRUE, then the block of Flow Diagram
statement(s) inside the if statement is executed.
If boolean expression evaluates to FALSE, then the first set of Example
code after the end of the if statement(s) is executed. #!/usr/bin/python
var1 = 100
Flow Diagram
if var1:
Example
print "1 - Got a true expression value"
#!/usr/bin/python
print var1
var1 = 100
else:
if var1:
17
 print "1 - Got a false expression value" Example
print var1 #!/usr/bin/python
var2 = 0 var = 100
if var2: if var == 200:
print "2 - Got a true expression value" print "1 - Got a true expression value"
print var2 print var
else: elif var == 150:
print "2 - Got a false expression value" print "2 - Got a true expression value"
print var2 print var
print "Good bye!" elif var == 100:
print "3 - Got a true expression value"
When the above code is executed, it produces the following result − print var
1 - Got a true expression value else:
100 print "4 - Got a false expression value"
2 - Got a false expression value print var
0 print "Good bye!"
Good bye!
The elif Statement When the above code is executed, it produces the following result −
The elif statement allows you to check multiple expressions 3 - Got a true expression value
for TRUE and execute a block of code as soon as one of the 100
conditions evaluates to TRUE. Good bye!
Similar to the else, the elif statement is optional. However,
unlike else, for which there can be at most one statement, there can Nested If Statement
be an arbitrary number of elif statements following an if. There may be a situation when you want to check for another
Syntax condition after a condition resolves to true. In such a situation, you
if expression1: can use the nested if construct.
statement(s) In a nested if construct, you can have an if...elif...else
elif expression2: construct inside another if...elif...else construct.
statement(s) Syntax
elif expression3:
statement(s) if expression1:
else: statement(s)
statement(s) if expression2:
Core Python does not provide switch or case statements as in statement(s)
other languages, but we can use if..elif...statements to simulate elif expression3:
switch case as follows − statement(s)

18
 elif expression4: Here, statement(s) may be a single statement or a block of
statement(s) statements. The condition may be any expression, and true is any
else: non-zero value. The loop iterates while the condition is true.
statement(s) When the condition becomes false, program control passes to
else: the line immediately following the loop.
statement(s) In Python, all the statements indented by the same number of
Example character spaces after a programming construct are considered to be
#!/usr/bin/python part of a single block of code. Python uses indentation as its method
var = 100 of grouping statements.
if var < 200:
print "Expression value is less than 200" Flow Diagram
if var == 150:
print "Which is 150" Here, key point of the while loop
elif var == 100: is that the loop might not ever run.
print "Which is 100"
When the condition is tested and the
elif var == 50: result is false, the loop body will be
print "Which is 50"
skipped and the first statement after the
elif var < 50:
while loop will be executed.
print "Expression value is less than 50"
else:
print "Could not find true expression"
print "Good bye!"

When the above code is executed, it produces following result −

#!/usr/bin/python The count is: 0
Expression value is less than 200
count = 0 The count is: 1
Which is 100
while (count < 9): The count is: 2
Good bye!
print 'The count is:', count The count is: 3
count = count + 1 The count is: 4
LOOPING print "Good bye!" The count is: 5
A while loop statement in Python programming language
The Infinite Loop
repeatedly executes a target statement as long as a given condition is
A loop becomes infinite loop if a condition never becomes
true.
FALSE. You must use caution when using while loops because of the
Syntax
possibility that this condition never resolves to a FALSE value.
while expression:
This results in a loop that never ends. Such a loop is called an
statement(s)
infinite loop.

19
 An infinite loop might be useful in client/server #!/usr/bin/python 0 is less than 5
programming where the server needs to run continuously so that count = 0 1 is less than 5
while count < 5: 2 is less than 5
client programs can communicate with it as and when required. print count, " is less than 5" 3 is less than 5
#!/usr/bin/python Enter a number :20 count = count + 1 4 is less than 5
var = 1 You entered: 20 else: 5 is not less than 5
while var == 1 : # This constructs an infinite loop Enter a number :29 print count, " is not less than 5" You entered: 3
print "Good bye!"
num = raw_input("Enter a number :") You entered: 29
print "You entered: ", num Enter a number :3
When the above code is executed, it produces the following result −
print "Good bye!" You entered: 3
Single Statement Suites
Similar to the if statement syntax, if your while clause
Enter a number between :Traceback (most recent call last): consists only of a single statement, it may be placed on the same line
File "test.py", line 5, in <module>
as the while header.
num = raw_input("Enter a number :")
Here is the syntax and example of a one-line while clause −
KeyboardInterrupt #!/usr/bin/python
Above example goes in an infinite loop and you need to use flag = 1
CTRL+C to exit the program. while (flag): print 'Given flag is really true!'
Using else Statement with Loops print "Good bye!"
Python supports to have an else statement associated with a It is better not try above example because it goes into infinite loop
loop statement. and you need to press CTRL+C keys to exit.
 If the else statement is used with a for loop, the else statement For Loop Statement
is executed when the loop has exhausted iterating the list. It has the ability to iterate over the items of any sequence,
 If the else statement is used with a while loop, the else such as a list or a string.
statement is executed when the condition becomes false. Syntax
The following example illustrates the combination of an else for iterating_var in sequence:
statements(s)
#!/usr/bin/python Current Letter : P
If a sequence contains an expression list, it is evaluated first.
for letter in 'Python': # First Example Current Letter : y
print 'Current Letter :', letter Current Letter : t Then, the first item in the sequence is assigned to the iterating
fruits = ['banana', 'apple', 'mango'] Current Letter : h variable iterating_var.
for fruit in fruits: # Second Example Current Letter : o Next, the statements block is executed. Each item in the list is
print 'Current fruit :', fruit Current Letter : n assigned to iterating_var, and the statement(s) block is executed until
print "Good bye!" Current fruit : banana
Current fruit : apple
the entire sequence is exhausted.
Current fruit : mango
Good bye! Iterating by Sequence Index
An alternative way of iterating through each item is by index
statement with a while statement that prints a number as long as it is offset into the sequence itself. Following is a simple example −
less than 5, otherwise else statement gets executed. #!/usr/bin/python
20
 fruits = ['banana', 'apple', 'mango']
for index in range(len(fruits)): Nested Loop Statement
print 'Current fruit :', fruits[index]
Python programming language allows to use one loop inside
print "Good bye!" another loop. Following section shows few examples to illustrate the
When the above code is executed, it produces the following result − concept.
Current fruit : banana
Syntax
Current fruit : apple Iterating For Loop Nested While loop
Current fruit : mango for iterating_var in sequence: while expression:
Good bye! for iterating_v ar in sequence: while expression:
statements(s) statement(s)
Here, we took the assistance of the len() built-in function, statements(s) statement(s)
which provides the total number of elements in the tuple as well as
the range() built-in function to give us the actual sequence to iterate 2) Explain about Repeat with while .
over. The while statement
Using else Statement with Loops The keyword while followed by a test expression (which can
Python supports to have an else statement associated with a be any valid expression),and a colon. Following the header is an
loop statement indented body.
 If the else statement is used with a for loop, the else statement The test expression is evaluated. If it evaluates to True, then
is executed when the loop has exhausted iterating the list. the body of the loop is executed. After executing the body, the test
 If the else statement is used with a while loop, the else expression is evaluated again. While test expression evaluates to
statement is executed when the condition becomes false.
#!/usr/bin/python Output:
for num in range (10,20): #t o it e rat e be t we e n 10 t o 20
for i in range (2,num ): #t o it e rat e on t he fac t ors of t he 10 equals 2 * 5
number 11 is a prim e num be r
if num%i == 0: #t o de t e rm ine t he first fac t or 12 equals 2 * 6
j=num/i #t o c alc ulat e t he se c ond fac t or 13 is a prime number
print '%d equals %d * %d' % (num,i,j)
break #to move to the next number, the #first FOR 14 e quals 2 * 7
#!/usr/bin/python Output:
else: # else part of the loop 15 e quals 3 * 5
i=2 print num, 'is a prime number' 16 e quals 2 * 8
while(i < 100): 2 is prime 17 is a prim e num be r
j=2 3 is prime 18 equals 2 * 9
while(j <= (i/j)): 5 is prime 19 is a prime number

if not(i%j): break 7 is prime

j=j+ 1 11 is prime True, the body of the loop is executed.
if (j > i/j) : print i, " is prime" 13 is prime When the test expression evaluates to False, the loop is
i=i+1 17 is prime terminated and execution continues with the statement following the
print "Good bye!" 19 is prime body
23 is prime

21
 The continue statement is used to skip the rest of the code inside a
loop for the current iteration only. Loop does not terminate but
continues on with the next iteration.
The working of continue statement
in for and while loop is shown
below.

The condition for this loop is n != 1, so the loop will continue

until n is 1, which makes the condition false. Each time through the
loop, the program outputs the value of n and then checks
whether it is even or odd. If it is even, n is divided by 2. If it is
odd, the value of n is replaced with n*3+1.
For example, if the argument passed to sequence is 3, the resulting
sequence is 3,10, 5, 16, 8, 4,2, 1.
3) Explain about continue and break statement
The break statement terminates the loop containing it. Control of 4) Explain the following
the program flows to the statement immediately after the body of the i) Iterate with Multiple sequences with zip()
loop. ii) range()
If break statement is inside a nested loop (loop inside another loop), iii)pass statement
break will terminatethe innermost loop.The working of break i) Iterate with Multiple sequences with zip()
statement in for loop and while loop is shown below. Iterating over multiple sequences in parallel by using the zip()
function:
>>> days = ['Monday', 'Tuesday', 'Wednesday']
>>> fruits = ['banana', 'orange', 'peach']
>>> drinks = ['coffee', 'tea', 'beer']
22
 >>> desserts = ['tiramisu', 'ice cream', 'pie', 'pudding'] This function does not store all the values in memory, it
>>> for day, fruit, drink, dessert in zip(days, fruits, drinks, would be inefficient.
desserts): range( start, stop, step ).
... print(day, ": drink", drink, "- eat", fruit, "- enjoy", dessert)
... If you omit start, the range begins at 0. The only required
Monday : drink coffee - eat banana - enjoy tiramisu value is stop; as with slices, the last value created will be just before
Tuesday : drink tea - eat orange - enjoy ice cream stop. The default value of step is 1, but you can go backward with -1.
Wednesday : drink beer - eat peach - enjoy pie Like zip(), range() returns an iterable object, so you need to step
zip() stops when the shortest sequence is done. One of the lists through the valueswith for ... in, or convert the object to a sequence
(desserts) was longer than the others, so no one gets any pudding like a list. Let’s make the range 0,1, 2:
unless we extend the other lists. >>> for x in range(0,3):
... print(x)
>>> english = 'Monday', 'Tuesday', 'Wednesday' ...
>>> french = 'Lundi', 'Mardi', 'Mercredi' 0
1
Now, use zip() to pair these tuples. The value returned by 2
zip() is itself not a tuple or list, but an iterable value that can be >>> list( range(0, 3) )
turned into one: [0, 1, 2]
>>> list( zip(english, french) ) Here’s how to make a range from 2 down to 0:
[('Monday', 'Lundi'), ('Tuesday', 'Mardi'), ('Wednesday', >>> for x in range(2, -1, -1):
'Mercredi')] ... print(x)
Feed the result of zip() directly to dict() and voila: a tiny English- ...
French dictionary! 2
>>> dict( zip(english, french) ) 1
{'Monday': 'Lundi', 'Tuesday': 'Mardi', 'Wednesday': 0
'Mercredi'} >>> list( range(2, -1, -1) )
[2, 1, 0]
ii) range(): The following snippet uses a step size of 2 to get the even numbers
If you do need to iterate over a sequence of numbers, the built- from 0 to 10:
in function range() comes in handy. It generates arithmetic >>> list( range(0, 11, 2) )
progressions: [0, 2, 4, 6, 8, 10]

Eg: iii) pass statement :

# Prints out the numbers 0,1,2,3,4 The pass statement does nothing. It can be used when a
for x in range(5): statement is required
print(x) syntactically but the program requires no action.

23
Eg: Global Variables and Global Scope
>>> while True: Global Scope
... pass # Busy-wait for keyboard interrupt (Ctrl+C)  The scope of a variable refers to the places that you can see or
... access a variable.
This is commonly used for creating minimal classes:  A variable with global scope can be used anywhere in the
>>> class MyEmptyClass:
... pass program.
...  It can be created by defining a variable outside the function.
Another place pass can be used is as a place-holder for a function or
conditional bodywhen you are working on new code, allowing you Local Scope A variable with local scope can be used only within the
function. A local variable is a variable that is only accessible from within
to keep thinking at a more abstract level. The pass is silently ignored:
a given function. Such variables are said to have local scope .
>>> def initlog(*args):
... pass # Remember to implement this! Namespace
 Python programs have various namespaces within which a
6) Explain about namespace and scope of a variable. particular name is unique and unrelated to the same name in
 Scope of a variable is the portion of a program where the other namespaces.
variable is recognized. Parameters and variables defined  Each function defines its own namespace. If you define a
inside a function is not visible from outside. Hence, they have variable called x in a main program and another variable
a local scope. called x in a function
 Lifetime of a variable is the period throughout which the Python provides two functions to access the contents of your
variable exits in the memory. namespaces:
 The lifetime of variables inside a function is as long as the • locals() returns a dictionary of the contents of the local namespace.
• globals() returns a dictionary of the contents of the global
function executes.
namespace.
 They are destroyed once we return from the function. Hence,
a function does not remember the value of a variable from its The local namespace within change_local() contained only the
previous calls. local variable animal. The global namespace contained the separate
global variable animal and a number of other things.
Eg:
def my_func():
x = 10 Section –B(Ten marks)
print("Value inside function:",x) 1) Discuss about Comprehension
x = 20 A comprehension is a compact way of creating a Python data
structure from one or more iterators. Comprehensions make it
my_func()
print("Value outside function:",x) possible for you to combine loops and conditional tests with a less
Output: verbose syntax.
Value inside function: 10
Value outside function: 20

24
List Comprehension: >>> pow2 = [2 ** x for x in range(10) if x > 5]
 List comprehensions provide a concise way to apply >>> pow2
operations on a list. [64, 128, 256, 512]
 It creates a new list in which each element is the result of >>> odd = [x for x in range(20) if x % 2 == 1]
applying a given operation in a list. It consists of brackets >>> odd
containing an expression followed by a ―for‖ clause, then a [1, 3, 5, 7, 9, 11, 13, 15, 17, 19]
list. >>> [x+y for x in ['Python ','C '] for y in ['Language','Programming']]
 The list comprehension always returns a result list.
Syntax ['Python Language', 'Python Programming', 'C Language', 'C
Programming']

Dictionary Comprehensions: Similar to list comprehensions,

dictionary comprehensions can also have if tests and multiple for
clauses:

{ key_expression : value_expression for expression in iterable }

Similar to list comprehensions, dictionary comprehensions

can also have if tests and multiple for clauses:

List comprehension consists of an expression followed by for >>> word = 'letters'

statement inside square brackets. >>> letter_counts = {letter: word.count(letter) for letter in word}
>>> letter_counts
Here is an example to make a list with each item being
{'l': 1, 'e': 2, 't': 2, 'r': 1, 's': 1}
increasing power of 2.
pow2 = [2 ** x for x in range(10)]
# Output: [1, 2, 4, 8, 16, 32, 64, 128, 256, 512] We are running a loop over each of the seven letters in the
string 'letters' and counting how many times that letter appears.
print(pow2)
This code is equivalent to Two of our uses of word.count (letter) are a waste of time
pow2 = [] because we have to count all the e’s twice and all the t’s twice.
for x in range(10): But, when we count the e’s the second time, we do no harm
pow2.append(2 ** x) because we just replace the entry in the dictionary that was already
A list comprehension can optionally contain more for or if there; the same goes for counting the t’s.
statements. An optional if statement can filter out items for the new So, the following would have been a teeny bit more Pythonic:
list. >>> word = 'letters'
Here are some examples.
25
 >>> letter_counts = {letter: word.count(letter) for letter 2
in set(word)} 3
>>> letter_counts 4
{'t': 2, 'l': 1, 'e': 2, 'r': 1, 's': 1} 5
The dictionary’s keys are in a different order than the We can wrap a list() call around a generator comprehension to make
previous example, because iterating set(word) returns letters in a it work like a list comprehension:
different order than iterating the string word. >>> number_list = list(number_thing)
>>> number_list
Set Comprehensions : [1, 2, 3, 4, 5]
It looks like the list and dictionary comprehensions that A generator can be run only once. Lists, sets, strings, and
you’ve just seen: dictionaries exist in memory, but a generator creates its values on the
{ expression for expression in iterable } fly and hands them out one at a time through an iterator. It doesn’t
The longer versions (if tests, multiple for clauses) are also remember them, so you can’t restart or back up a generator.If you try
valid for sets: to re-iterate this generator, you’ll find that it’s tapped out:
>>> a_set = {number for number in range(1,6) if number % 3 >>> try_again = list(number_thing)
== 1} >>> try_again
>>> a_set []
{1, 4} 2) List down the types of arguments functions in Python
Generator Comprehensions : 1. Required Arguments
Tuples do not have comprehensions, that is changing the 2. Keyword Arguments
square brackets of a list comprehension to parentheses would create 3. Default Arguments
a tuple comprehension. 4. Variable length Arguments
And it would appear to work because there’s no exception if 1. Required Arguments: The number of arguments in the function
you type this: call should match exactly with the function definition.
>>> number_thing = (number for number in range(1, 6)) def my_details( name, age ):
The thing between the parentheses is a generator comprehension, and it print("Name: ", name)
returns a generator print("Age ", age)
object: return
>>> type(number_thing) my_details("george",56)
<class 'generator'> Output:
A generator is one way to provide data to an iterator. Name: george Age 56
We can iterate over this generator object directly, as illustrated here: 2. Keyword Arguments:
>>> for number in number_thing: Python interpreter is able to use the keywords provided to
... print(number) match the values with parameters even though if they are arranged
... in out of order.
1 def my_details( name, age ):

26
 print("Name: ", name) • Return the modified function for use
print("Age ", age) Here’s what the code looks like:
return >>> def document_it(func):
my_details(age=56,name="george") ... def new_function(*args, **kwargs):
Output: ... print('Running function:', func. name )
Name: george Age 56 ... print('Positional arguments:', args)
3. Default Arguments: ... print('Keyword arguments:', kwargs)
Assumes a default value if a value is not provided in the ... result = func(*args, **kwargs)
function call for that argument. ... print('Result:', result)
def my_details( name, age=40 ): ... return result
print("Name: ", name) ... return new_function
print("Age ", age) Whatever func you pass to document_it(), you get a new
return function that includes the extra statements that document_it() adds.
my_details(name="george") A decorator doesn’t actually have to run any code from func,
Output: but document_it() calls func part way through so that you get the
Name: george Age 40 results of func as well as all the extras.apply the decorator manually:
4. Variable length Arguments >>> def add_ints(a, b):
If we want to specify more arguments than specified while ... return a + b
defining the function, variable length arguments are used. It is ...
denoted by * symbol before parameter. >>> add_ints(3, 5)
def my_details(*name ): 8
print(*name) >>> cooler_add_ints = document_it(add_ints) # manual
my_details("rajan","rahul","micheal", decorator assignment
ärjun") >>> cooler_add_ints(3, 5)
Output: rajan rahul micheal ärjun Running function: add_ints
3) Discuss about Decorators. Positional arguments: (3, 5)
A decorator is a function that takes one function as input and Keyword arguments: {}
returns another function. Result: 8
• *args and **kwargs 8
• Inner functions As an alternative to the manual decorator assignment above,
• Functions as arguments just add @decorator_name before the function that you want to
The function document_it() defines a decorator that will do the decorate:
following: >>> @document_it
• Print the function’s name and the values of its arguments ... def add_ints(a, b):
• Run the function with the arguments ... return a + b
• Print the result ...

27
 >>> add_ints(3, 5) >>> add_ints(3, 5)
Start function add_ints Running function: add_ints
Positional arguments: (3, 5) Positional arguments: (3, 5)
Keyword arguments: {} Keyword arguments: {}
Result: 8 Result: 8
8 64
You can have more than one decorator for a function. Let’s write 4) Explain briefly about Functions
another decorator called square_it() that squares the result: Function is a sub program which consists of set of instructions
>>> def square_it(func): used to perform a specific task. A large program is divided into basic
... def new_function(*args, **kwargs): building blocks called function.
... result = func(*args, **kwargs) Need For Function:
... return result * result When the program is too complex and large they are divided
... return new_function into parts. Each partis separately coded and combined into single
... program. Each subprogram is called as function.
The decorator that’s used closest to the function (just above Debugging, Testing and maintenance becomes easy when the
the def) runs first and then the one above it. program is divided into subprograms.
Either order gives the same end result, but you can see how  Functions are used to avoid rewriting same code again and
the intermediate again in a program.
steps change:
 Function provides code re-usability
>>> @document_it
... @square_it  The length of the program is reduced.
... def add_ints(a, b):
Types of function:
... return a + b
... Functions can be classified into two categories:
>>> add_ints(3, 5) i) Built in function
Running function: new_function ii) user defined function
Positional arguments: (3, 5)
Keyword arguments: {} i) Built in functions
Result: 64 Built in functions are the functions that are already created
64 and stored in python.
These built in functions are always available for usage and
Let’s try reversing the decorator order:
>>> @square_it accessed by a programmer. It cannot be modified.
... @document_it
... def add_ints(a, b):
... return a + b
...

28

ii) User Defined Functions:
User defined functions are the functions that programmers
create for their requirement and use.
These functions can then be combined to form module
which can be used in other programs by importing them.
Advantages of user defined functions:
Programmers working on large project can divide the
workload by making different functions.
If repeated code occurs in a program, function can be used
to include those codes and execute when needed by calling
that function.
Function definition: (Sub program)
def keyword is used to define a function.
Give the function name after def keyword followed by
parentheses in which arguments are given.
End with colon (:)
Inside the function add the program statements to be
executed
29
End with or without return statement
Function Calling: (Main Function)
Once we have defined a function, we can call it from
another function, program or even the Python prompt.
To call a function we simply type the function name with
appropriate arguments.
Example:
x=5
y=4
Flow of Execution:
The order in which statements are executed is called the
flow of execution
Execution always begins at the first statement of the
program.
Statements are executed one at a time, in order, from top to
bottom.
Function definitions do not alter the flow of execution of the
program, but remember that statements inside the function
are not executed until the function is called.
Function calls are like a bypass in the flow of execution.
Instead of going to the next statement, the flow jumps to the
first line of the called function, executes all the statements
there, and then comes back to pick up where it left off.
5) Describe about Function prototypes.
i. Function without arguments and without return type
ii. Function with arguments and without return type
 iii. Function without arguments and with return type These are the values required by function to work.
iv. Function with arguments and with return type If there is more than one value required, all of them will be listed
i) Function without arguments and without return type in parameter list separated by comma.
o In this type no argument is passed through the function call Example: def my_add(a,b):
and no output is return to main function
o The sub function will read the input values perform the Arguments :
operation and print the result in the same block Arguments are the value(s) provided in function call/invoke
ii) Function with arguments and without return type statement.
o Arguments are passed through the function call but output List of arguments should be supplied in same way as parameters
is not return to the main function are listed.
iii) Function without arguments and with return type Bounding of parameters to arguments is done 1:1, and so there
o In this type no argument is passed through the function call should be same number and type of arguments as mentioned in
but output is return to the main function. parameter list.
iv) Function with arguments and with return type Example: my_add(x,y)
o In this type arguments are passed through the function call RETURN STATEMENT
and output is return to the main function The return statement is used to exit a function and go back to the
place from where it was called.
If the return statement has no arguments, then it will not return
any values. But exits from function.
Syntax:
return[expression]
Example:
def my_add(a,b):
c=a+b
return c
x=5
y=4
print(my_add(x,y))
Output:
9
6) How to handling errors with try and except in python

Python (interpreter) raises exceptions when it encounters

Parameters: errors. Error caused by not following the proper structure (syntax) of
Parameters are the value(s) provided in the parenthesis when we the language is called syntax error or parsing error.
write function header. >>> if a < 3

30
 File "<interactive input>", line 1 Exception Cause of Error
if a < 3
^
SyntaxError: invalid syntax
Errors can also occur at runtime and these are called
exceptions. They occur, for example, when a file we try to open does
not exist (FileNotFoundError), dividing a number by zero
(ZeroDivisionError), module we try to import is not found
(ImportError) etc.
Whenever these type of runtime error occur, Python creates
an exception object. If not handled properly, it prints a traceback to
that error along with some details about why that error occurred.
>>> 1 / 0
Traceback (most recent call last):
File "<string>", line 301, in runcode
File "<interactive input>", line 1, in <module>
ZeroDivisionError: division by zero
>>> open("imaginary.txt")
Traceback (most recent call last):
File "<string>", line 301, in runcode
File "<interactive input>", line 1, in <module>
FileNotFoundError: [Errno 2] No such file or directory: 'imaginary.txt'

Python Built-in Exceptions

Illegal operations can raise exceptions. There are plenty of
built-in exceptions in Python that are raised when corresponding
errors occur. We can view all the built-in exceptions using the
local() built-in functions as follows.
>>> locals()[' builtins ']
This will return us a dictionary of built-in exceptions,
functions and attributes.
Some of the common built-in exceptions in Python
programming along with the error that
cause then are tabulated below.

Python Built-in Exceptions

31
 A critical operation which can raise exception is placed inside the try
clause and the code that handles exception is written in except
clause.
It is up to us, what operations we perform once we have
caught the exception. Here is a simple example.
# import module sys to get the type of exception
import sys
randomList = ['a', 0, 2]
for entry in randomList:
try:
print("The entry is", entry)
r = 1/int(entry)
break
except:
print("Oops!",sys.exc_info()[0],"occured.")
print("Next entry.")
print()
print("The reciprocal of",entry,"is",r)

Output
We can handle these built-in and user-defined exceptions in The entry is a
Python using try, except and finally statements. Oops! <class 'ValueError'> occured.
Next entry.
Python Exception Handling The entry is 0
Python has many built-in exceptions which forces your Oops! <class 'ZeroDivisionError' > occured.
program to output an error when something in it goes wrong. Next entry.
When these exceptions occur, it causes the current process to The entry is 2
stop and passes it to the calling process until it is handled. If not The reciprocal of 2 is 0.5
handled, our program will crash. In this program, we loop until the user enters an integer that
For example, if function A calls function B which in turn calls has a valid reciprocal. The portion that can cause exception is placed
function C and an exception occurs in function C. If it is not handled inside try block.
in C, the exception passes to B and then to A. If never handled, an If no exception occurs, except block is skipped and normal
error message is spit out and our program come to a sudden, flow continues. But if any exception occurs, it is caught by the except
unexpected halt. block. Here, we print the name of the exception using ex_info()
function inside sys module and ask the user to try again. We can see
Catching Exceptions in Python
In Python, exceptions can be handled using a try statement.

32
that the values 'a' and '1.3' causes ValueError and '0' causes  Define only what you need to add or change in the new class,
ZeroDivisionError. and this overrides the behavior of the old class.
 The original class is called a parent, superclass, or base class;
Try...finally  The new class is called a child, subclass, or derived class.
The try statement in Python can have an optional finally  To define an empty class called Car. Next, define a subclass of
clause. This clause is executed no matter what, and is generally used Car called Yugo.
to release external resources.  To define a subclass by using the same class keyword but
For example, we may be connected to a remote data center with the parent class name inside the parentheses (class
through the network or working with a file or working with a Yugo(Car) below):
Graphical User Interface (GUI).In all these circumstances, we must >>> class Car():
clean up the resource once used, whether it was successful or not. ... pass
These actions (closing a file, GUI or disconnecting from ...
network) are performed in the finally clause to guarantee execution. >>> class Yugo(Car):
Here is an example of file operations to illustrate this. ... pass
try: ...
f = open("test.txt",encoding = 'utf-8')  Next, create an object from each class:
# perform file operations >>> give_me_a_car = Car()
finally: >>> give_me_a_yugo = Yugo()
f.close()  A child class is a specialization of a parent class; in object-
UNIT II COMPLETED oriented lingo, Yugo is-a Car.
 The object named give_me_a_yugo is an instance of class
Yugo
 New class definitions
>>> class Car():
UNIT - 3 ... def exclaim(self):
1) Explain the following ... print("I'm a Car!")
i)Inheritance ii)Duck Typing ...
>>> class Yugo(Car):
i) Inheritance ... pass
Inheritance: ...
 inheritance: creating a new class from an existing class but with  one object from each class and call the exclaim method:
some additions or changes. It’s an excellent way to reuse code. >>> give_me_a_car = Car()
 When you use inheritance, the new class can automatically >>> give_me_a_yugo = Yugo()
use all the code from the old class but without copying any of >>> give_me_a_car.exclaim()
it. I'm a Car!
>>> give_me_a_yugo.exclaim()
33
 I'm a Car! QuestionQuote and ExclamationQuote.
 Without doing anything special, Yugo inherited the exclaim() >>> hunter = Quote('Elmer Fudd', "I'm hunting wabbits")
method from Car >>> print(hunter.who(), 'says:', hunter.says())
Elmer Fudd says: I'm hunting wabbits.
ii) Duck Typing >>> hunted1 = QuestionQuote('Bugs Bunny', "What's up,
Duck Typing : doc")
 Python has a loose implementation of polymorphism; this >>> print(hunted1.who(), 'says:', hunted1.says())
means that it applies the same operation to different objects, Bugs Bunny says: What's up, doc?
regardless of their class >>> hunted2 = ExclamationQuote('Daffy Duck', "It's rabbit
 same init () initializer for all three Quote classes now, but season")
add two new functions: >>> print(hunted2.who(), 'says:', hunted2.says())
 who() just returns the value of the saved person string Daffy Duck says: It's rabbit season!
 says() returns the saved words string with the specific  Three different versions of the says() method provide
punctuation different behavior for the three classes.
>>> class Quote():  Let’s define a class called BabblingBrook that has no relation to
... def init (self, person, words): our previouswoodsy hunter and huntees (descendants of the
... self.person = person Quote class):
... self.words = words >>> class BabblingBrook():
... def who(self): ... def who(self):
... return self.person ... return 'Brook'
... def says(self): ... def says(self):
... return self.words + '.' ... return 'Babble'
... ...
>>> class QuestionQuote(Quote): >>> brook = BabblingBrook()
... def says(self):  Now, run the who() and says() methods of various objects,
... return self.words + '?' one (brook) completely unrelated to the others:
... >>> def who_says(obj):
>>> class ExclamationQuote(Quote): ... print(obj.who(), 'says', obj.says())
... def says(self): ...
... return self.words + '!' >>> who_says(hunter)
... Elmer Fudd says I'm hunting wabbits.
>>> >>> who_says(hunted1)
 Python then automatically calls the init () method of the Bugs Bunny says What's up, doc?
parent class Quote to store the instance variables person and >>> who_says(hunted2)
words. Daffy Duck says It's rabbit season!
 We can access self.words in objects created from the subclasses >>> who_says(brook)
34
 Brook says Babble ... ['Rosa', 'Klebb'],
 This behavior is sometimes called duck typing, ... ['Mister', 'Big'],
2) Explain about Structured text file. ... ['Auric', 'Goldfinger'],
 There are many formats, Structured Text Files | 181
o A separator, or delimiter, character like tab ('\t'), comma www.it-ebooks.info
(','), or vertical bar ('|'). This is an example of the ... ['Ernst', 'Blofeld'],
comma-separated values (CSV) format. ... ]
o '<' and '>' around tags. Examples include XML and >>> with open('villains', 'wt') as fout: # a context
HTML. manager
o Punctuation. An example is JavaScript Object Notation ... csvout = csv.writer(fout)
(JSON). ... csvout.writerows(villains)
o Indentation. An example is YAML (which depending This creates the file villains with these lines:
on the source you use means ―YAML Ain’t Markup Doctor,No
Language;‖ you’ll need to research that one yourself). Rosa,Klebb
o Miscellaneous, such as configuration files for Mister,Big
programs. Auric,Goldfinger
CSV Ernst,Blofeld
 Delimited files are often used as an exchange format for  Now, we’ll try to read it back in:
spreadsheets and databases. >>> import csv
 You could read CSV files manually, a line at a time, splitting >>> with open('villains', 'rt') as fin: # context manager
each line into fields at comma separators, and adding the ... cin = csv.reader(fin)
results to data structures such as lists and dictionaries. ... villains = [row for row in cin] # This uses a list
 Some have alternate delimiters besides a comma: '|' and '\t' comprehension
(tab) are common. Some have escape sequences. ...
 If the delimiter character can occur within a field, the entire >>> print(villains)
field might be surrounded by quote characters or preceded by [['Doctor', 'No'], ['Rosa', 'Klebb'], ['Mister', 'Big'],
some escape character. ['Auric', 'Goldfinger'], ['Ernst', 'Blofeld']]
 Files have different line-ending characters. Unix uses '\n',  Using reader() and writer() with their default options, the
Microsoft uses '\r \n', and Apple used to use '\r' but now columns are separated bycommas and the rows by line feeds.
uses '\n'.  The data can be a list of dictionaries rather than a list of lists.
 There can be column names in the first line.First, we’ll see Let’s read the villains file again, this time using the new
how to read and write a list of rows, each containing a list of DictReader() function and specifying the column names:
columns: >>> import csv
>>> import csv >>> with open('villains', 'rt') as fin:
>>> villains = [ ... cin = csv.DictReader(fin, fieldnames=['first', 'last'])
... ['Doctor', 'No'], ... villains = [row for row in cin]
35
 ...
>>> print(villains)
[{'last': 'No', 'first': 'Doctor'},
{'last': 'Klebb', 'first': 'Rosa'},
{'last': 'Big', 'first': 'Mister'},
{'last': 'Goldfinger', 'first': 'Auric'},
{'last': 'Blofeld', 'first': 'Ernst'}]
 Let’s rewrite the CSV file by using the new DictWriter()
function. We’ll also call write header() to write an initial line
of column names to the CSV file:
import csv
villains = [
{'first': 'Doctor', 'last': 'No'},
{'first': 'Rosa', 'last': 'Klebb'},
{'first': 'Mister', 'last': 'Big'},
{'first': 'Auric', 'last': 'Goldfinger'},
{'first': 'Ernst', 'last': 'Blofeld'},
] <item price="$5.00">hamburger</item>
with open('villains', 'wt') as fout:
cout = csv.DictWriter(fout, ['first', 'last'])
cout.writeheader()
cout.writerows(villains)
That creates a villains file with a header line:
first,last
Doctor,No
Rosa,Klebb
Mister,Big
Auric,Goldfinger
Ernst,Blofeld
 By omitting the fieldnames argument in the DictReader()
call,we instruct it to use the values in the first line of the file
(first,last) as column labels and matching dictionary keys:
>>> import csv
>>> with open('villains', 'rt') as fin:
... cin = csv.DictReader(fin)
... villains = [row for row in cin]
36
...
>>> print(villains)
[{'last': 'No', 'first': 'Doctor'},
{'last': 'Klebb', 'first': 'Rosa'},
{'last': 'Big', 'first': 'Mister'},
{'last': 'Goldfinger', 'first': 'Auric'},
{'last': 'Blofeld', 'first': 'Ernst'}]
XML
 XML is the most prominent markup format that suits the bill. It
uses tags to delimit data, as in this sample menu.xml file:
<?xml version="1.0"?>
<menu>
<breakfast hours="7-11">
<item price="$6.00">breakfast burritos</item>
<item price="$4.00">pancakes</item>
</breakfast>
<lunch hours="11-3">
</lunch>
<dinner hours="3-10">
<item price="8.00">spaghetti</item>
</dinner>
</menu>
 Following are a few important characteristics of XML:
• Tags begin with a < character. The tags in this sample were
menu, breakfast, lunch,dinner, and item.
• Whitespace is ignored.
• Usually a start tag such as <menu> is followed by other content
and then a final matching end tag such as </menu>.
• Tags can nest within other tags to any level. In this example,
item tags are childrenof the breakfast, lunch, and dinner tags;
they, in turn, are children of menu.
• Optional attributes can occur within the start tag. In this
example, price is an attribute of item.
• Tags can contain values. In this example, each item has a value,
such as pancakes for the second breakfast item.
• If a tag named thing has no values or children, it can be
expressed as the single tag by including a forward slash just
before the closing angle bracket, such as <thing/ >, rather than a
start and end tag, like <thing></thing>.
• The choice of where to put data—attributes, values, child
tags—is somewhat arbitrary.
 For instance, we could have written the last item tag as <item
price="$8.00" food="spaghetti"/>.
 XML is often used for data feeds and messages, and has
subformats like RSS and Atom.Some industries have many
specialized XML formats, such as the finance field.
 XML’s uber-flexibility has inspired multiple Python libraries
that differ in approach and capabilities.
 The simplest way to parse XML in Python is by using
ElementTree. Here’s a little program to parse the menu.xml file
and print some tags and attributes:
>>> import xml.etree.ElementTree as et
>>> tree = et.ElementTree(file='menu.xml')
>>> root = tree.getroot()
>>> root.tag
'menu'
>>> for child in root:
... print('tag:', child.tag, 'attributes:', child.attrib)
... for grandchild in child:
... print('\ttag:', grandchild.tag, 'attributes:',
grandchild.attrib)
...
tag: breakfast attributes: {'hours': '7-11'}
tag: item attributes: {'price': '$6.00'}
tag: item attributes: {'price': '$4.00'}
tag: lunch attributes: {'hours': '11-3'}
tag: item attributes: {'price': '$5.00'}
tag: dinner attributes: {'hours': '3-10'}
tag: item attributes: {'price': '8.00'}
>>> len(root) # number of menu sections
3 ... "hours": "7-11",
37
>>> len(root[0]) # number of breakfast items
2
 For each element in the nested lists, tag is the tag string and
attrib is a dictionary of its attributes. ElementTree has many
other ways of searching XML-derived data, modifyingit, and
even writing XML files. The ElementTree documentation has
the details.
 Other standard Python XML libraries include:
 Xml.dom :The Document Object Model (DOM), familiar to
JavaScript developers, representsWeb documents as
hierarchical structures. This module loads the entire XML file
into memory and lets you access all the pieces equally
 .xml.sax :Simple API for XML, or SAX, parses XML
HTML
 Enormous amounts of data are saved as Hypertext Mark up
Language (HTML), the basic document format of the Web.
 The problem is so much of it doesn’t follow the HTML rules,
which can make it difficult to parse. Also, much of HTML is
intended more to format output than interchange data
JSON
 JavaScript Object Notation (JSON) has become a very popular
data interchange format,beyond its JavaScript origins.
 The JSON format is a subset of JavaScript, and often
legalPython syntax as well. Its close fit to Python makes it a
good choice for data interchange among programs.
 Unlike the variety of XML modules, there’s one main JSON
module, with the unforgettable name json.
 This program encodes (dumps) data to a JSON string and
decodes(loads) a JSON string back to data.
 In this example, let’s build a Python data structurecontaining
the data from the earlier XML example:
>>> menu = \
... {
... "breakfast": {
 ... "items": { 'hours': '11-3'}, 'dinner': {'items': {'spaghetti': '$8.00'}, 'hours': '3-
... "breakfast burritos": "$6.00", 10'}}
... "pancakes": "$4.00"  menu and menu2 are both dictionaries with the same keys
... } and values. might be annoying to make these special
... }, conversions.
... "lunch" : { YAML
... "hours": "11-3",  Similar to JSON, YAML has keys and values, but handles
... "items": { more data types such as datesand times.
... "hamburger": "$5.00"  The standard Python library does not yet include YAML
... } handling, so you need to install a third-party library named
... }, yaml to manipulate it. load() converts aYAML string to
... "dinner": { Python data, whereas dump() does the opposite.
... "hours": "3-10",  The following YAML file, mcintyre.yaml, contains information
... "items": { on the Canadian poet James McIntyre, including two of his
... "spaghetti": "$8.00" poems:
... } name:
... } first: James
... } last: McIntyre
. dates:
 Next, encode the data structure (menu) to a JSON string birth: 1828-05-25
(menu_json) by using dumps(): death: 1906-03-31
>>> import json details:
>>> menu_json = json.dumps(menu) bearded: true
>>> menu_json themes: [cheese, Canada]
'{"dinner": {"items": {"spaghetti": "$8.00"}, "hours": "3-10"}, books:
"lunch": {"items": {"hamburger": "$5.00"}, "hours": "11-3"}, url: http://www.gutenberg.org/files/36068/36068-
"breakfast": {"items": {"breakfast burritos": "$6.00", h/36068-h.htm
"pancakes": poems:
"$4.00"}, "hours": "7-11"}}' - title: 'Motto'
 String menu_json back into a Python data structure (menu2) text: |
by using loads(): Politeness, perseverance and pluck,
>>> menu2 = json.loads(menu_json) To their possessor will bring good luck.
>>> menu2 - title: 'Canadian Charms'
{'breakfast': {'items': {'breakfast burritos': '$6.00', 'pancakes': text: |
'$4.00'}, 'hours': '7-11'}, 'lunch': {'items': {'hamburger': '$5.00'}, Here industry is not in vain,
For we have bounteous crops of grain,
38
 And you behold on every field <!ENTITY lol1
Of grass and roots abundant yield, "&lol;&lol;&lol;&lol;&lol;&lol;&lol;&lol;&lol;&lol;">
But after all the greatest charm <!ENTITY lol2
Is the snug home upon the farm, "&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;">
And stone walls now keep cattle warm. <!ENTITY lol3
 Values such as true, false, on, and off are converted to Python "&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;">
Booleans. Integers and strings are converted to their Python <!ENTITY lol4
equivalents. Other syntax creates lists and dictionaries: "&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;">
>>> import yaml <!ENTITY lol5
>>> with open('mcintyre.yaml', 'rt') as fin: "&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;">
>>> text = fin.read() <!ENTITY lol6
>>> data = yaml.load(text) "&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;">
>>> data['details'] Structured Text Files | 189
{'themes': ['cheese', 'Canada'], 'bearded': True} www.it-ebooks.info
>>> len(data['poems']) <!ENTITY lol7
2 "&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;">
 The data structures that are created match those in the YAML <!ENTITY lol8
file, which in this case are more than one level deep in places. "&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;">
You can get the title of the second poem with this <!ENTITY lol9
dict/list/dict reference: "&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;">
>>> data['poems'][1]['title'] ]>
'Canadian Charms' <lolz>&lol9;</lolz>
 PyYAML can load Python objects from strings, and this is  Defused XML lists this attack and others, along with the
dangerous.Use safe_load() instead of load() if you’re vulnerability of Python libraries. The link shows how to
importing YAML that you don’t trust. Better yet, always use change the settings for many of the libraries to avoid these
safe_load(). problems.
A Security Note  Use the defusedxml library as a security frontend for the other
 It’s possible to exploit this process and cause security libraries:
problems. >>> # insecure:
 For example, the following XML snippet from the billion >>> from xml.etree.ElementTree import parse
laughs Wikipedia page defines ten nested entities, each >>> et = parse(xmlfile)
expanding the lower level ten times for a total expansion of >>> # protected:
one billion: >>> from defusedxml.ElementTree import parse
<?xml version="1.0"?> >>> et = parse(xmlfile)
<!DOCTYPE lolz [
<!ENTITY lol "lol">
39
Configuration Files Serialize by Using pickle
 Most programs offer various options or settings. Dynamic ones  Saving data structures to a file is called serializing. Formats
can be provided as program arguments such as JSON might require some custom converters to
 Need to maintain both the writer program and the reader serialize all the data types from a Python program.
program (sometimes called a parser).  Python provides the pickle module to save and restore any
 Use the standard configparser module, which handles object in a special binary format.
Windows-style .ini files. Such files have sections of key = value
definitions. Here’s a minimal settings.cfg file: >>> import pickle
[english] >>> import datetime
greeting = Hello >>> now1 = datetime.datetime.utcnow()
[french] >>> pickled = pickle.dumps(now1)
greeting = Bonjour >>> now2 = pickle.loads(pickled)
[files] >>> now1
home = /usr/local datetime.datetime(2014, 6, 22, 23, 24, 19, 195722)
# simple interpolation: >>> now2
bin = %(home)s/bin datetime.datetime(2014, 6, 22, 23, 24, 19, 195722)
 Here’s the code to read it into Python data structures:  Pickle works with your own classes and objects, too. We’ll
>>> import configparser define a little class calledTiny that returns the string 'tiny'
>>> cfg = configparser.ConfigParser() when treated as a string:
>>> cfg.read('settings.cfg') >>> import pickle
['settings.cfg'] >>> class Tiny():
>>> cfg ... def str (self):
<configparser.ConfigParser object at 0x1006be4d0> ... return 'tiny'
>>> cfg['french'] ...
<Section: french> >>> obj1 = Tiny()
>>> cfg['french']['greeting'] >>> obj1
'Bonjour' < main .Tiny object at 0x10076ed10>
>>> cfg['files']['bin'] >>> str(obj1)
'/usr/local/bin' 'tiny'
Other Interchange Formats >>> pickled = pickle.dumps(obj1)
 These binary data interchange formats are usually more >>> pickled
compact and faster than XML or JSON: b'\x80\x03c main \nTiny\nq\x00)\x81q\x01.'
• MsgPack >>> obj2 = pickle.loads(pickled)
• Protocol Buffers >>> obj2
• Avro < main .Tiny object at 0x10076e550>
• Thrift >>> str(obj2)
40
 'tiny' >>> the_bytes[1] = 127
 Pickled is the pickled binary string made from the object obj1. Traceback (most recent call last):
We converted that backto the object obj2 to make a copy of File "<stdin>", line 1, in <module>
obj1. TypeError: 'bytes' object does not support item
 Use dump() to pickle to a file, and load() to unpicked from assignment
one.  But a bytearray variable is mellow and mutable:
3) Discuss about binary data types. >>> the_byte_array = bytearray(blist)
Binary Data >>> the_byte_array
 Binary file formats or network packets to extract or even bytearray(b'\x01\x02\x03\xff')
change data. >>> the_byte_array[1] = 127
bytes and bytearray >>> the_byte_array
 Python 3 introduced the following sequences of eight-bit bytearray(b'\x01\x7f\x03\xff')
integers, with possible values from 0 to 255, in two types:  Each of these would create a 256-element result, with values
• bytes is immutable, like a tuple of bytes from 0 to 255:
• bytearray is mutable, like a list of bytes >>> the_bytes = bytes(range(0, 256))
 Beginning with a list called blist, this next example creates a >>> the_byte_array = bytearray(range(0, 256))
bytes variable called the_bytes and a bytearray variable called  When printing bytes or bytearray data, Python uses \x xx for
the_byte_array: non-printable bytes and their ASCII equivalents for printable
>> blist = [1, 2, 3, 255] ones (plus some common escape characters, such as \n instead
>>> the_bytes = bytes(blist) of \x0a).
>>> the_bytes Convert Binary Data with struct
b'\x01\x02\x03\xff'  Table : Endian specifiers
>>> the_byte_array = bytearray(blist)
>>> the_byte_array
bytearray(b'\x01\x02\x03\xff')
 The representation of a bytes value begins with a b and a quote
character, followed by hex sequences such as \x02 or ASCII  The type specifiers follow the endian character. Any specifier
characters, and ends with a matching quote character. may be preceded by a number that indicates the count; 5B is
 Python converts the hex sequences or ASCII characters to little the same as BBBBB.
integers, but shows byte values that are also valid ASCII  You can use a count prefix instead of >LL:
encodings as ASCII characters.
>>> struct.unpack('>2L', data[16:24])
>>> b'\x61' (154,141)
b'a'
>>> b'\x01abc\xff'
b'\x01abc\xff'
 Can’t change a bytes variable:
41
 >>> fmt = Struct('png',
... Magic(b'\x89PNG\r\n\x1a\n'),
... UBInt32('length'),
... Const(String('type', 4), b'IHDR'),
... UBInt32('width'),
BInt32('height')
... )
>>> data = b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR' + \
...
b'\x00\x00\x00\x9a\x00\x00\x00\x8d\x08\x02\x00\x00\x00\xc0'
>>> result = fmt.parse(data)
>>> print(result)
 Used the slice data[16:24] to grab the interesting bytes directly. Container:
We could also use the x specifier to skip the uninteresting length = 13
parts: type = b'IHDR'
>>> struct.unpack('>16x2L6x', data) width = 154
(154, 141) height = 141
 This means: >>> print(result.width, result.height)
• Use big-endian integer format (>) 154, 141
• Skip 16 bytes (16x) Convert Bytes/Strings with binascii()
• Read 8 bytes—two unsigned long integers (2L)  The standard binascii module has functions with which you
• Skip the final 6 bytes (6x) can convert between binary data and various string
Other Binary Data Tools : representations: hex (base 16), base 64, uuencoded, and others.
 Defining and extracting binary data:  For example, in the next snippet, let’s print that 8-byte PNG
• bitstring header as a sequence of hex values, instead of the mixture of
• construct ASCII and \x xx escapes that Python uses to display bytes
• hachoir variables:
• binio >>> import binascii
 How to download and install external packages such as these. >>> valid_png_header = b'\x89PNG\r\n\x1a\n'
$ pip install construct >>> print(binascii.hexlify(valid_png_header))
 Here’s how to extract the PNG dimensions from our data b'89504e470d0a1a0a'
bytestring by using construct:  Hey, this thing works backwards, too:
... U>>> from construct import Struct, Magic, UBInt32, >>> print(binascii.unhexlify(b'89504e470d0a1a0a'))
Const, String b'\x89PNG\r\n\x1a\n'
>>> # adapted from code at https://github.com/construct

42
 Bit Operators >>> db['mustard'] = 'yellow'
 Python provides bit-level integer operators, similar to those in >>> db['ketchup'] = 'red'
the C language. >>> db['pesto'] = 'green'
4) Explain about NoSql Datasto  Let’s pause and check what we have so far:
>>> len(db)
3
>>> db['pesto']
b'green'
 Now close, then reopen to see if it actually saved what we gave it:
>>> db.close()
>>> db = dbm.open('definitions', 'r')
>>> db['mustard']
b'yellow'
 Keys and values are stored as bytes. You cannot iterate over the
re
database object db, but you can get the number of keys by using
 Some databases are not relational and don’t support SQL. These len()
were written to handle very large data sets, allow more flexible Memcached
data definitions, or support custom data operations.  memcached is a fast in-memory key-value cache server. It’s often
 They’ve been collectively labeled NoSQL (formerly meaning no put in front of a database,or used to store web server session
SQL; now the less confrontational not only SQL). data.
The dbm Family  It can be download the versions for Linux and OS X, and for
 The dbm formats were around long before NoSQL was coined. Windows.
They’re key-value stores, often embedded in applications such as  There are many Python drivers; one that works with Python 3 is
web browsers to maintain various settings.
python3-memcached, which you can install by using this
 A dbm database is like a Python dictionary in the following command:
ways: $ pip install python-memcached
• You can assign a value to a key, and it’s automatically saved  To use it, connect to a memcached server, after which you can do
to the database on disk.
the following:
• You can get a value from a key. • Set and get values for keys
 The following is a quick example. The second argument to the • Increment or decrement a value
following open() method is 'r' to read, 'w' to write, and 'c' for • Delete a key
both, creating the file if it doesn’t exist:
 Data is not persistent, and data that you wrote earlier might
>>> import dbm disappear. This is inherent in memcached, being that it’s a cache
>>> db = dbm.open('definitions', 'c') server.
 To create key-value pairs, just assign a value to a key just as you  It avoids running out of memory by discarding old data.
would a dictionary:  You can connect to multiple memcached servers at the same time
43
 >>> import redis
>>> import memcache >>> conn = redis.Redis()
>>> db = memcache.Client(['127.0.0.1:11211'])  redis.Redis('localhost') or redis.Redis('localhost', 6379) would
>>> db.set('marco', 'polo') have given the same result.
True  List all keys (none so far):
>>> db.get('marco') >>> conn.keys('*')
'polo' []
>>> db.set('ducks', 0)  Set a simple string (key 'secret'), integer (key 'carats'), and float
True (key 'fever'):
>>> db.get('ducks') >>> conn.set('secret', 'ni!')
0 True
>>> db.incr('ducks', 2) >>> conn.set('carats', 24)
2 True
>>> db.get('ducks') >>> conn.set('fever', '101.5')
2 True
Redis Get the values back by key:
 Redis is a data structure server. Like memcached, all of the data in >>> conn.get('secret')
a Redis server should fit in memory (although there is now an b'ni!'
option to save the data to disk). >>> conn.get('carats')
 Unlike memcached,Redis can do the following: b'24'
• Save data to disk for reliability and restarts >>> conn.get('fever')
• Keep old data b'101.5'
• Provide more data structures than simple strings  Here, the setnx() method sets a value only if the key does not
 The Redis data types are a close match to Python’s, and a Redis exist:
server can be a useful intermediary for one or more Python >>> conn.setnx('secret', 'icky-icky-icky-ptang-zoop-boing!')
applications to share data False
 The Python driver redis-py has its source code and tests on  It failed because we had already defined 'secret':
GitHub, as well as online documentation. You can install it by >>> conn.get('secret')
using this command: b'ni!'
$ pip install redis  The getset() method returns the old value and sets it to a new one
 The Redis server itself has good documentation. If you install and at the same time:
start the Redis server on your local computer >>> conn.getset('secret', 'icky-icky-icky-ptang-zoop-boing!')
Strings b'ni!'
 A key with a single value is a Redis string. Simple Python data  Get a substring by using getrange() (as in Python, offset 0=start, -
types are automatically converted. Connect to a Redis server at 1=end):
some host (default is localhost) and port (default is 6379): >>> conn.getrange('secret', -6, -1)
44
 b'boing!' Lists
 Replace a substring by using setrange() (using a zero-based  Redis lists can contain only strings. The list is created when you
offset): do your first insertion.Insert at the beginning by using lpush():
>>> conn.setrange('secret', 0, 'ICKY') >>> conn.lpush('zoo', 'bear')
32 1
>>> conn.get('secret')  Insert more than one item at the beginning:
b'ICKY-icky-icky-ptang-zoop-boing!' >>> conn.lpush('zoo', 'alligator', 'duck')
 Set multiple keys at once by using mset(): 3
>>> conn.mset({'pie': 'cherry', 'cordial': 'sherry'})  Insert before or after a value by using linsert():
True >>> conn.linsert('zoo', 'before', 'bear', 'beaver')
 Get more than one value at once by using mget(): 4
>>> conn.mget(['fever', 'carats']) >>> conn.linsert('zoo', 'after', 'bear', 'cassowary')
[b'101.5', b'24'] 5
 Delete a key by using delete():  Insert at an offset by using lset() (the list must exist already):
>>> conn.delete('fever') >>> conn.lset('zoo', 2, 'marmoset')
True True
 Increment by using the incr() or incrbyfloat() commands, and  Insert at the end by using rpush():
decrement with decr(): >>> conn.rpush('zoo', 'yak')
>>> conn.incr('carats') 6
25  Get the value at an offset by using lindex():
>>> conn.incr('carats', 10) >>> conn.lindex('zoo', 3)
35 b'bear'
>>> conn.decr('carats')  Get the values in an offset range by using lrange() (0 to -1 for all):
34 >>> conn.lrange('zoo', 0, 2)
>>> conn.decr('carats', 15) [b'duck', b'alligator', b'marmoset']
19  Trim the list with ltrim(), keeping only those in a range of offsets:
>>> conn.set('fever', '101.5') >>> conn.ltrim('zoo', 1, 4)
True True
>>> conn.incrbyfloat('fever')  Get a range of values (use 0 to -1 for all) by using lrange():
102.5 >>> conn.lrange('zoo', 0, -1)
>>> conn.incrbyfloat('fever', 0.5) [b'alligator', b'marmoset', b'bear', b'cassowary']
103.0 Hashes
 There’s no decrbyfloat(). Use a negative increment to reduce the  Redis hashes are similar to Python dictionaries but can contain
fever: only strings. create and play with a Redis hash called song:
>>> conn.incrbyfloat('fever', -2.0)  Set the fields do and re in hash song at once by using hmset():
101.0 >>> conn.hmset('song', {'do': 'a deer', 're': 'about a deer'})
45
 True {b'duck', b'goat', b'turkey'}
 Set a single field value in a hash by using hset():  Remove a value from the set:
>>> conn.hset('song', 'mi', 'a note to follow re') >>> conn.srem('zoo', 'turkey')
1 True
 Get one field’s value by using hget():  Let’s make a second set to show some set operations:
>>> conn.hget('song', 'mi') >>> conn.sadd('better_zoo', 'tiger', 'wolf', 'duck')
b'a note to follow re' 0
 Get multiple field values by using hmget():  Intersect (get the common members of) the zoo and better_zoo
>>> conn.hmget('song', 're', 'do') sets:
[b'about a deer', b'a deer'] >>> conn.sinter('zoo', 'better_zoo')
 Get all field keys for the hash by using hkeys(): {b'duck'}
>>> conn.hkeys('song')  Get the intersection of zoo and better_zoo, and store the result in
[b'do', b're', b'mi'] the set fowl_zoo:
 Get all field values for the hash by using hvals(): >>> conn.sinterstore('fowl_zoo', 'zoo', 'better_zoo')
>>> conn.hvals('song') 1
[b'a deer', b'about a deer', b'a note to follow re']  Who’s in there?
 Get the number of fields in the hash by using hlen(): >>> conn.smembers('fowl_zoo')
>>> conn.hlen('song') {b'duck'}
3  Get the union (all members) of zoo and better_zoo:
 Get all field keys and values in the hash by using hgetall(): >>> conn.sunion('zoo', 'better_zoo')
>>> conn.hgetall('song') {b'duck', b'goat', b'wolf', b'tiger'}
{b'do': b'a deer', b're': b'about a deer', b'mi': b'a note to follow  Store that union result in the set fabulous_zoo:
re'} >>> conn.sunionstore('fabulous_zoo', 'zoo',
 Set a field if its key doesn’t exist by using hsetnx(): 'better_zoo')
>>> conn.hsetnx('song', 'fa', 'a note that rhymes with la') 4
1 >>> conn.smembers('fabulous_zoo')
Sets {b'duck', b'goat', b'wolf', b'tiger'}
 Redis sets are similar to Python sets, Add one or more values to a  What does zoo have that better_zoo doesn’t? Use sdiff() to get the
set: set difference, and sdiffstore() to save it in the zoo_sale set:
>>> conn.sadd('zoo', 'duck', 'goat', 'turkey') >>> conn.sdiff('zoo', 'better_zoo')
3 {b'goat'}
 Get the number of values from the set: >>> conn.sdiffstore('zoo_sale', 'zoo', 'better_zoo')
>>> conn.scard('zoo') 1
3 >>> conn.smembers('zoo_sale')
 Get all the set’s values: {b'goat'}
>>> conn.smembers('zoo')
46
Sorted sets Bits :
 Sorted set, or zset. It’s a set of unique values, but each value has an  Bits are more compact and faster. few user IDs:
associated floating point score. Itcan access each item by its value >>> days = ['2013-02-25', '2013-02-26', '2013-02-27']
or score. Sorted sets have many uses: >>> big_spender = 1089
• Leader boards >>> tire_kicker = 40459
• Secondary indexes >>> late_joiner = 550212
• Timeseries, using timestamps as scores  Each date is a separate key. Set the bit for a particular user ID for
 Tracking user logins via timestamps. Python time() function: that date. For example, on the first date (2013-02-25), we had
>>> import time visits from big_spender (ID 1089) and tire_kicker (ID 40459):
>>> now = time.time() >>> conn.setbit(days[0], big_spender, 1)
>>> now 0
1361857057.576483 >>> conn.setbit(days[0], tire_kicker, 1)
 Let’s add our first guest, looking nervous: 0
>>> conn.zadd('logins', 'smeagol', now)  The next day, big_spender came back:
1 >>> conn.setbit(days[1], big_spender, 1)
 Five minutes later, another guest: 0
>>> conn.zadd('logins', 'sauron', now+(5*60))  The next day had yet another visit from our friend, big_spender,
1 and a new person whom we’re calling late_joiner:
 Two hours later: >>> conn.setbit(days[2], big_spender, 1)
>>> conn.zadd('logins', 'bilbo', now+(2*60*60)) 0
1 >>> conn.setbit(days[2], late_joiner, 1)
 One day later, not hasty: 0
>>> conn.zadd('logins', 'treebeard', now+(24*60*60))  Let’s get the daily visitor count for these three days:
1 >>> for day in days:
 In what order did bilbo arrive? ... conn.bitcount(day)
>>> conn.zrank('logins', 'bilbo') ...
2 2
 Let’s see everyone in login order: 1
>>> conn.zrange('logins', 0, -1) 2
[b'smeagol', b'sauron', b'bilbo', b'treebeard']  Did a particular user visit on a particular day?
 With their times, please: >>> conn.getbit(days[1], tire_kicker)
>>> conn.zrange('logins', 0, -1, withscores=True) 0
[(b'smeagol', 1361857057.576483), (b'sauron',  How many users visited every day?
1361857357.576483), >>> conn.bitop('and', 'everyday', *days)
(b'bilbo', 1361864257.576483), (b'treebeard', 1361943457.576483)] 68777
>>> conn.bitcount('everyday')
47
 1 Section –B(Ten marks)
Caches and expiration 1)Explain about Python standard Library
 All Redis keys have a time-to-live, or expiration date. By default, Handle Missing Keys with setdefault() and defaultdict() :
this is forever. We canuse the expire() function to instruct Redis  Using the dictionary get() function to return a default value
how long to keep the key. avoids an exception.
>>> import time  The setdefault() function is like get(), but also assigns an item
>>> key = 'now you see it' to the dictionary ifthe key is missing:
>>> conn.set(key, 'but not for long') >>> periodic_table = {'Hydrogen': 1, 'Helium': 2}
True >>> print(periodic_table)
>>> conn.expire(key, 5) {'Helium': 2, 'Hydrogen': 1}
True  If the key was not already in the dictionary, the new value is
>>> conn.ttl(key) used:
5 >>> carbon = periodic_table.setdefault('Carbon', 12)
>>> conn.get(key) >>> carbon
b'but not for long' 12
>>> time.sleep(6) >>> periodic_table
>>> conn.get(key) {'Helium': 2, 'Carbon': 12, 'Hydrogen': 1}
>>>  If we try to assign a different default value to an existing key,
 The expireat() command expires a key at a given epoch time. Key the original value is returned and nothing is changed:
expiration is useful to keep caches fresh and to limit login >>> helium = periodic_table.setdefault('Helium', 947)
sessions. >>> helium
Other NoSQL 2
 The NoSQL servers listed here handle data larger than memory, >>> periodic_table
and many of them use multiple computers. {'Helium': 2, 'Carbon': 12, 'Hydrogen': 1}
 Table NoSQL databases  defaultdict() is similar, but specifies the default value for any
new key up front, whenthe dictionary is created. Its argument
is a function.
 In this example, we pass the function int, which will be called
as int() and return the integer 0:
>>> from collections import defaultdict
>>> periodic_table = defaultdict(int)
 Now, any missing value will be an integer (int), with the
value 0:
>>> periodic_table['Hydrogen'] = 1
>>> periodic_table['Lead']
0
48
 >>> periodic_table
defaultdict(<class 'int'>, {'Lead': 0, 'Hydrogen': 1})
 The argument to defaultdict() is a function that returns the
value to be assigned toa missing key.
 In the following example, no_idea() is executed to return a
value when needed:
>>> from collections import defaultdict
>>>
>>> def no_idea():
... return 'Huh?'
...
>>> bestiary = defaultdict(no_idea)
>>> bestiary['A'] = 'Abominable Snowman'
>>> bestiary['B'] = 'Basilisk'
>>> bestiary['A']
'Abominable Snowman'
>>> bestiary['B']
'Basilisk'
>>> bestiary['C']
'Huh?'
 You can use the functions int(), list(), or dict() to return default
empty values forthose types: int() returns 0, list() returns an
empty list ([]), and dict() returns anempty dictionary ({}).
 If you omit the argument, the initial value of a new key will
be set to None.By the way, you can use lambda to define your
default-making function right inside the call:
>>> bestiary = defaultdict(lambda: 'Huh?')
>>> bestiary['E']
'Huh?'
 Using int is one way to make your own counter:
>>> from collections import defaultdict
>>> food_counter = defaultdict(int)
>>> for food in ['spam', 'spam', 'eggs', 'spam']:
... food_counter[food] += 1
...
>>> for food, count in food_counter.items():
49
... print(food, count)
...
eggs 1
spam 3
 In the preceding example, if food_counter had been a normal
dictionary instead of adefaultdict,
 Python would have raised an exception every time we tried to
increment the dictionary element food_counter[food] because
it would not have been initialized.
Count Items with Counter()
Speaking of counters, the standard library has one that does
the work of the previous example and more:
>>> from collections import Counter
>>> breakfast = ['spam', 'spam', 'eggs', 'spam']
>>> breakfast_counter = Counter(breakfast)
>>> breakfast_counter
Counter({'spam': 3, 'eggs': 1})
The most_common() function returns all elements in
descending order, or just the top count elements if given a
count:
>>> breakfast_counter.most_common()
[('spam', 3), ('eggs', 1)]
>>> breakfast_counter.most_common(1)
[('spam', 3)]
You can combine counters. First, let’s see again what’s in
breakfast_counter:
>>> breakfast_counter
>>> Counter({'spam': 3, 'eggs': 1})
This time, we’ll make a new list called lunch, and a counter
called lunch_counter:
>>> lunch = ['eggs', 'eggs', 'bacon']
>>> lunch_counter = Counter(lunch)
>>> lunch_counter
Counter({'eggs': 2, 'bacon': 1})
The first way we combine the two counters is by addition,
using +:
 >>> breakfast_counter + lunch_counter >>> quotes = OrderedDict([
Counter({'spam': 3, 'eggs': 3, 'bacon': 1}) ... ('Moe', 'A wise guy, huh?'),
Subtract one counter from another : ... ('Larry', 'Ow!'),
>>> breakfast_counter - lunch_counter ... ('Curly', 'Nyuk nyuk!'),
Counter({'spam': 3}) ... ])
>>>
>>> lunch_counter - breakfast_counter >>> for stooge in quotes:
Counter({'bacon': 1, 'eggs': 1}) ... print(stooge)
Get common items by using the intersection operator &: ...
>>> breakfast_counter & lunch_counter Moe
Counter({'eggs': 1}) Larry
The intersection picked the common element ('eggs') with the Curly
lower count. This makessense: breakfast only offered one egg, Stack + Queue == deque :
so that’s the common count.  A deque (pronounced deck) is a double-ended queue, which
Finally, you can get all items by using the union operator |: has features of both a stack and a queue.
>>> breakfast_counter | lunch_counter  When you want to add and delete items from either end of a
Counter({'spam': 3, 'eggs': 2, 'bacon': 1}) sequence
The item 'eggs' was again common to both.  The function popleft() removes the leftmost item from the
Order by Key with OrderedDict() deque and returnsit; pop() removes the rightmost item and
 add keys a, b, and c in that order, but keys() might return c, a, returns it
b. >>> def palindrome(word):
>>> quotes = { ... from collections import deque
... 'Moe': 'A wise guy, huh?', ... dq = deque(word)
... 'Larry': 'Ow!', ... while len(dq) > 1:
... 'Curly': 'Nyuk nyuk!', ... if dq.popleft() != dq.pop():
... } ... return False
>>> for stooge in quotes: ... return True
... print(stooge) ...
... ...
Larry >>> palindrome('a')
Curly True
Moe >>> palindrome('racecar')
 An OrderedDict() remembers the order of key addition and True
returns them in the sameorder from an iterator. >>> palindrome('')
 Creating an OrderedDict from a sequence of (key, value)tuples: True
>>> from collections import OrderedDict >>> palindrome('radar')
50
 True 2
>>> palindrome('halibut') .
False .
 To just compare a string with its reverse. .
 Python doesn’t have a reverse() function for strings, but it …and so on.
does have a way to reverse a string with a slice, as illustrated  accumulate() calculates accumulated values. By default, it
here: calculates the sum:
>>> def another_palindrome(word): >>> import itertools
... return word == word[::-1] >>> for item in itertools.accumulate([1, 2, 3, 4]):
... ... print(item)
>>> another_palindrome('radar') ...
True 1
>>> another_palindrome('halibut') 3
False 6
Iterate over Code Structures with itertools : 10
 itertools contains special-purpose iterator functions.  You can provide a function as the second argument to
 Each returns one item at a time when called within a for … in accumulate(), and it will be used instead of addition.
loop, and remembers its state between calls.  The function should take two arguments and return a single
 chain() runs through its arguments as though they were a result.
single iterable:  This example calculates an accumulated product:
>>> import itertools >>> import itertools
>>> for item in itertools.chain([1, 2], ['a', 'b']): >>> def multiply(a, b):
... print(item) ... return a * b
... ...
1 >>> for item in itertools.accumulate([1, 2, 3, 4],
2 multiply):
a ... print(item)
b ...
 cycle() is an infinite iterator, cycling through its arguments: 1
>>> import itertools 2
>>> for item in itertools.cycle([1, 2]): 6
... print(item) 24
...  The itertools module has many more functions, notably some
1 for combinations andpermutations that can be time savers
2 when the need arises.
1
51
Print Nicely with pprint(): >>> someone = Person()
 A pretty printer such as pprint():  In this case, Person() creates an individual object from the
>>> from pprint import pprint Person class and assigns it the name someone.
>>> quotes = OrderedDict([  special Python object initialization method init :
... ('Moe', 'A wise guy, huh?'), >>> class Person():
... ('Larry', 'Ow!'), ... def init (self):
... ('Curly', 'Nyuk nyuk!'), ... pass
... ])  init () is the special Python name for a method that
>>> initializes
 Plain old print() just dumps things out there: an individual object from its class definition.
>>> print(quotes)  The self argument specifies that it refers to the individual
OrderedDict([('Moe', 'A wise guy, huh?'), ('Larry', object itself.
'Ow!'), ('Curly', 'Nyuk nyuk!')])  When you define init () in a class definition, its first
 However, pprint() tries to align elements for better parameter should be self. Although self is not a reserved word
readability: in Python, it’s common usage
>>> pprint(quotes)  add the parameter name to the initialization method:
{'Moe': 'A wise guy, huh?', >>> class Person():
'Larry': 'Ow!', ... def init (self, name):
'Curly': 'Nyuk nyuk!'} ... self.name = name
2)Discuss about Python Objects and classes. ...
 An object contains both data (variables, called attributes) and >>>
code (functions, called methods).  Now, we can create an object from the Person class by passing
 It represents a unique instance of some concrete thing. a string for the name parameter:
Define a Class with class : >>> hunter = Person('Elmer Fudd')
 class is like the mold that makes that box. For instance, a  Here’s what this line of code does:
String is the built-in Python class that makes string objects • Looks up the definition of the Person class
such as 'cat' and 'duck'. • Instantiates (creates) a new object in memory
 Python has many other built-in classes to create the other • Calls the object’s init method, passing this
standard data types, including lists, dictionaries, and so on. newly-created object as self and the other argument
 To create your own custom object in Python, you first need to ('Elmer Fudd') as name
define a class by using the class keyword. • Stores the value of name in the object
 To define a class called Person as the mold • Returns the new object
>>> class Person(): • Attaches the name hunter to the object
... pass  This new object is like any other object in Python. You can use
 You create an object from a class by calling the class name as it as an element of a list,tuple, dictionary, or set. You can pass
though it were a function: it to a function as an argument, or return it as a result.
52
  Saved with the object as an attribute.  New class definitions
 You can read and write it directly: >>> class Car():
>>> print('The mighty hunter: ', hunter.name) ... def exclaim(self):
The mighty hunter: Elmer Fudd ... print("I'm a Car!")
 inside the Person class definition, you access the name ...
attribute as self.name. >>> class Yugo(Car):
 When you create an actual object such as hunter, you refer to ... pass
it as hunter.name. ...
Inheritance:  one object from each class and call the exclaim method:
 inheritance: creating a new class from an existing class but with >>> give_me_a_car = Car()
some additions or changes. It’s an excellent way to reuse code. >>> give_me_a_yugo = Yugo()
 When you use inheritance, the new class can automatically >>> give_me_a_car.exclaim()
use all the code from the old class but without copying any of I'm a Car!
it. >>> give_me_a_yugo.exclaim()
 Define only what you need to add or change in the new class, I'm a Car!
and this overrides the behavior of the old class.  Without doing anything special, Yugo inherited the exclaim()
 The original class is called a parent, superclass, or base class; method from Car
 The new class is called a child, subclass, or derived class. Override a Method :
 To define an empty class called Car. Next, define a subclass of  A new class initially inherits everything from its parent class
Car called Yugo.  exclaim() method works for a Yugo:
 To define a subclass by using the same class keyword but >>> class Car():
with the parent class name inside the parentheses (class ... def exclaim(self):
Yugo(Car) below): ... print("I'm a Car!")
>>> class Car(): ...
... pass >>> class Yugo(Car):
... ... def exclaim(self):
>>> class Yugo(Car): ... print("I'm a Yugo! Much like a Car, but more
... pass Yugo-ish.")
... ...
 Next, create an object from each class:  Now, make two objects from these classes:
>>> give_me_a_car = Car() >>> give_me_a_car = Car()
>>> give_me_a_yugo = Yugo() >>> give_me_a_yugo = Yugo()
 A child class is a specialization of a parent class; in object-
oriented lingo, Yugo is-a Car. >>> give_me_a_car.exclaim()
 The object named give_me_a_yugo is an instance of class I'm a Car!
Yugo
53
 >>> give_me_a_yugo.exclaim() ... def exclaim(self):
I'm a Yugo! Much like a Car, but more Yugo-ish. ... print("I'm a Car!")
 In these examples, we overrode the exclaim() method. ...
 An override any methods,including init (). Here’s another >>> class Yugo(Car):
example that uses our earlier Person class. ... def exclaim(self):
 Let’smake subclasses that represent doctors (MDPerson) and ... print("I'm a Yugo! Much like a Car, but more Yugo-ish.")
lawyers (JDPerson): ... def need_a_push(self):
>>> class Person(): ... print("A little help here?")
... def init (self, name): ...
... self.name = name  Next, make a Car and a Yugo:
... >>> give_me_a_car = Car()
>>> class MDPerson(Person): >>> give_me_a_yugo = Yugo()
... def init (self, name):  A Yugo object can react to a need_a_push() method call:
... self.name = "Doctor " + name >>> give_me_a_yugo.need_a_push()
... A little help here?
>>> class JDPerson(Person):  But a generic Car object cannot:
... def init (self, name): >>> give_me_a_car.need_a_push()
... self.name = name + ", Esquire" Traceback (most recent call last):
 The initialization method init () takes the same arguments File "<stdin>", line 1, in <module>
as the parent Person class but stores the value of name AttributeError: 'Car' object has no attribute 'need_a_push'
differently inside the object instance: Get Help from Your Parent with super :
>>> person = Person('Fudd') >>> class Person():
>>> doctor = MDPerson('Fudd') ... def init (self, name):
>>> lawyer = JDPerson('Fudd') ... self.name = name
>>> print(person.name) ...
Fudd  The init () call in the following subclass has an additional
>>> print(doctor.name) email parameter:
Doctor Fudd >>> class EmailPerson(Person):
>>> print(lawyer.name) ... def init (self, name, email):
Fudd, Esquire ... super(). init (name)
Add a Method : ... self.email = email
 The child class can also add a method that was not present in  When you define an init () method for your class, you’re
its parent class. replacing the init ()method of its parent class, and the
 To define the new method need_a_push() for class Yugo only: latter is not called automatically anymore.
>>> class Car():

54
  As a result,we need to call it explicitly. Here’s what’s >>> car.exclaim()
happening: I'm a Car!
• The super() gets the definition of the parent class, Person.  Here’s what Python actually does, under the hood:
• The init () method calls the Person. init () method. It • Look up the class (Car) of the object car.
takes care of passing the self argument to the superclass, so • Pass the object car to the exclaim() method of the Car class
you just need to give it any optional arguments. In our case, as the self parameter.
the only other argument Person() accepts is name.  It will work the same as the normal (car.exclaim()) syntax:
• The self.email = email line is the new code that makes this >>> Car.exclaim(car)
EmailPerson different I'm a Car!
from a Person. Get and Set Attribute Values with Properties :
>>> bob = EmailPerson('Bob Frapples', 'bob@frapples.com')  Programmers often need to write getter and setter method to
 We should be able to access both the name and email read and write the values of such private attributes.
attributes:  Python doesn’t need getters or setters, because all attributes
>>> bob.name and methods are public,
'Bob Frapples'  To define a Duck class with a single attribute called
>>> bob.email hidden_name.
'bob@frapples.com' >>> class Duck():
... def init (self, input_name):
>>> class EmailPerson(Person): ... self.hidden_name = input_name
... def init (self, name, email): ... def get_name(self):
... self.name = name ... print('inside the getter')
... self.email = email ... return self.hidden_name
 Used super() to make Person do its work, the same as a plain ... def set_name(self, input_name):
Person object would. ... print('inside the setter')
 There’s another benefit: if the definition of Person changes in ... self.hidden_name = input_name
the future, using super() will ensure that the attributes and ... name = property(get_name, set_name)
methods that EmailPerson inherits from Person will reflect the  To define two methods: agetter (get_name()) and a setter
change. (set_name()).
 Use super() when the child is doing something its own way  It defines the two methods as properties of the attribute called
but still needs something from the parent . name.
 The first argument to property() isthe getter method, and the
In self Defense : second is the setter
 Python uses the self argument to find the right object’s  Calling the get_name() method to return it:
attributes and methods >>> fowl = Duck('Howard')
>>> car = Car() >>> fowl.name

55
 inside the getter ... print('inside the setter')
'Howard' ... self.hidden_name = input_name
 You can still call get_name() directly, too, like a normal getter  You can still access name as though it were an attribute, but
method: there are no visible get_name() or set_name() methods:
>>> fowl.get_name() >>> fowl = Duck('Howard')
inside the getter >>> fowl.name
'Howard' inside the getter
 When you assign a value to the name attribute, the set_name() 'Howard'
method will be called: >>> fowl.name = 'Donald'
>>> fowl.name = 'Daffy' inside the setter
inside the setter >>> fowl.name
>>> fowl.name inside the getter
inside the getter 'Donald'
'Daffy'  Let’s define a Circle class that has a radius attribute and a
 You can still call the set_name() method directly: computeddiameter property:
>>> fowl.set_name('Daffy') >>> class Circle():
inside the setter ... def init (self, radius):
>>> fowl.name ... self.radius = radius
inside the getter ... @property
'Daffy' ... def diameter(self):
 Another way to define properties is with decorators. In this ... return 2 * self.radius
next example, we’ll definetwo different methods, each called ...
name() but preceded by different decorators:  We create a Circle object with an initial value for its radius:
• @property, which goes before the getter method >>> c = Circle(5)
• @name.setter, which goes before the setter method >>> c.radius
Code: 5
>>> class Duck():  We can refer to diameter as if it were an attribute such as
... def init (self, input_name): radius:
... self.hidden_name = input_name >>> c.diameter
... @property 10
... def name(self):  we can change the radius attribute at any time, and the
... print('inside the getter') diameter property will be computed from the current value of
... return self.hidden_name radius:
... @name.setter >>> c.radius = 7
... def name(self, input_name): >>> c.diameter

56
 14 Traceback (most recent call last):
 If you don’t specify a setter property for an attribute, you File "<stdin>", line 1, in <module>
can’t set it from the outside.This is handy for read-only AttributeError: 'Duck' object has no attribute ' name'
attributes:  This naming convention doesn’t make it private, but Python
>>> c.diameter = 20 does mangle the name to make it unlikely for external code to
Traceback (most recent call last): stumble upon it
File "<stdin>", line 1, in <module> Method Types :
AttributeError: can't set attribute  There are three types of methods
 There’s one more big advantage of using a property over 1) Instance method
direct attribute access. 2) Class method
Name Mangling for Privacy: 3) Static method
 Python has a naming convention for attributes that should 1) Instance method:
notbe visible outside of their class definition: begin by using  An initial self argument in methods within a class definition
with two underscores ( ).  The first parameter of an instance method is self, and Python
 Let’s rename hidden_name to name, as demonstrated here: passes the object to the method when you call it.
2) Class Method
>>> class Duck():  In contrast, a class method affects the class as a whole. Any
... def init (self, input_name): change you make to the classaffects all of its objects.
... self. name = input_name  Within a class definition, a preceding @classmethod
... @property decorator indicates that that following function is a class
... def name(self): method. Also, the first parameter to themethod is the class
... print('inside the getter') itself.
…  The Python tradition is to call the parameter cls, because class
 Take a moment to see if everything still works: is a reserved word and can’t be used here.
>>> fowl = Duck('Howard')  Let’s define a class method for A thatcounts how many object
>>> fowl.name instances have been made from it:
inside the getter >>> class A():
'Howard' ... count = 0
>>> fowl.name = 'Donald' ... def init (self):
inside the setter ... A.count += 1
>>> fowl.name ... def exclaim(self):
inside the getter ... print("I'm an A!")
'Donald' ... @classmethod
 Can’t access the name attribute: ... def kids(cls):
>>> fowl. name ... print("A has", cls.count, "little objects.")

57
 ...  says() returns the saved words string with the specific
>>> easy_a = A() punctuation
>>> breezy_a = A() >>> class Quote():
>>> wheezy_a = A() ... def init (self, person, words):
>>> A.kids() ... self.person = person
A has 3 little objects. ... self.words = words
 Referred to A.count (the class attribute) rather than self.count ... def who(self):
(whichwould be an object instance attribute). ... return self.person
 In the kids() method, we used cls.count, but we could just as ... def says(self):
well have used A.count. ... return self.words + '.'
3) Static method : ...
 A third type of method in a class definition affects neither the >>> class QuestionQuote(Quote):
class nor its objects; it’s just in there for convenience instead of ... def says(self):
floating around on its own. ... return self.words + '?'
 It’s a static method,preceded by a @staticmethod decorator, ...
with no initial self or class parameter. >>> class ExclamationQuote(Quote):
 Here’s an example that serves as a commercial for the class ... def says(self):
CoyoteWeapon: ... return self.words + '!'
>>> class CoyoteWeapon(): ...
... @staticmethod >>>
... def commercial():  Python then automatically calls the init () method of the
... print('This CoyoteWeapon has been brought to you parent class Quote to store the instance variables person and
by Acme') words.
...  We can access self.words in objects created from the subclasses
>>> QuestionQuote and ExclamationQuote.
>>> CoyoteWeapon.commercial() >>> hunter = Quote('Elmer Fudd', "I'm hunting wabbits")
This CoyoteWeapon has been brought to you by Acme >>> print(hunter.who(), 'says:', hunter.says())
Duck Typing : Elmer Fudd says: I'm hunting wabbits.
 Python has a loose implementation of polymorphism; this >>> hunted1 = QuestionQuote('Bugs Bunny', "What's up,
means that it applies the same operation to different objects, doc")
regardless of their class >>> print(hunted1.who(), 'says:', hunted1.says())
 same init () initializer for all three Quote classes now, but Bugs Bunny says: What's up, doc?
add two new functions: >>> hunted2 = ExclamationQuote('Daffy Duck', "It's rabbit
 who() just returns the value of the saved person string season")
>>> print(hunted2.who(), 'says:', hunted2.says())
Daffy Duck says: It's rabbit season!
58
  Three different versions of the says() method provide  That is, a Word containing the value 'ha' would be considered
different behavior for the three classes. equal to one containing 'HA'.
 Let’s define a class called BabblingBrook that has no relation to  The example that follows is a first attempt, with a normal
our previouswoodsy hunter and huntees (descendants of the method we’re calling equals().self.text is the text string that
Quote class): this Word object contains, and the equals() method compares
>>> class BabblingBrook(): it with the text string of word2 (another Word object):
... def who(self): >>> class Word():
... return 'Brook' ... def init (self, text):
... def says(self): ... self.text = text
... return 'Babble' ...
... ... def equals(self, word2):
>>> brook = BabblingBrook() ... return self.text.lower() == word2.text.lower()
 Now, run the who() and says() methods of various objects, ...
one (brook) completely unrelated to the others:  Then, make three Word objects from three different text
>>> def who_says(obj): strings:
... print(obj.who(), 'says', obj.says()) >>> first = Word('ha')
... >>> second = Word('HA')
>>> who_says(hunter) >>> third = Word('eh')
Elmer Fudd says I'm hunting wabbits.  When strings 'ha' and 'HA' are compared to lowercase, they
>>> who_says(hunted1) should be equal:
Bugs Bunny says What's up, doc? >>> first.equals(second)
>>> who_says(hunted2) True
Daffy Duck says It's rabbit season!  But the string 'eh' will not match 'ha':
>>> who_says(brook) >>> first.equals(third)
Brook says Babble False
 This behavior is sometimes called duck typing,  We defined the method equals() to do this lowercase
Special Methods conversion and comparison.
 When you type something such as a = 3 + 8, how do the  Itwould be nice to just say if first == second, just like Python’s
integer objects with values 3 and 8 know how to implement built-in types. We change the equals() method to the special
+? Also, how does a know how to use = to get the result? name eq ()
 You can get at these operators by using Python’s special >>> class Word():
methods ... def init (self, text):
 The names of these methods begin and end with double ... self.text = text
underscores ( ). ... def eq (self, word2):
 Suppose that you have a simple Word class, and you want an ... return self.text.lower() == word2.text.lower()
equals() method that compares two words but ignores case ...
59
 Let’s see if it works: < main .Word object at 0x1006ba3d0>
>>> first = Word('ha') >>> print(first)
>>> second = Word('HA') < main .Word object at 0x1006ba3d0>
>>> third = Word('eh')  Let’s add both str () and repr () methods to the Word
>>> first == second class to make it prettier:
True >>> class Word():
>>> first == third ... def init (self, text):
False ... self.text = text
 Magic! All we needed was the Python’s special method name ... def eq (self, word2):
for testing equality, eq (). Tables 1 and 2 list the names of ... return self.text.lower() == word2.text.lower()
the most useful magic methods. ... def str (self):
 Table1. Magic methods for comparison ... return self.text
eq ( self, other ) self == other ... def repr (self):
ne ( self, other ) self != other ... return 'Word("' self.text '")'
lt ( self, other ) self < other ...
gt ( self, other ) self > other >>> first = Word('ha')
le ( self, other ) self <= other >>> first # uses repr
ge ( self, other ) self >= other Word("ha")
 Table-2. Magic methods for math >>> print(first) # uses str
add ( self, other ) self + other ha
sub ( self, other ) self - other
mul ( self, other ) self * other Composition
floordiv ( self, other ) self // other  Inheritance is a good technique to use when you want a child
truediv ( self, other ) self / other class to act like its parentclass most of the time (when child is-
mod ( self, other ) self % other a parent).
pow ( self, other ) self ** other  It’s tempting to build elaborate inheritancehierarchies, but
 Table-3. Other, miscellaneous magic methods sometimes composition or aggregation (when x has-a y)
str ( self ) str( self )  A duck is-a bird, but has-a tail. A tail is not a kind of duck, but
repr ( self ) repr( self ) part of a duck.
len ( self ) len( self )  In this next example, let’s make bill and tail objects and
 The interactive interpreter uses the repr () function to echo provide them to a new duck object:
variables to output. >>> class Bill():
 If you fail to define either str () or repr (), you get ... def init (self, description):
Python’s default string version of your object: ... self.description = description
>>> first = Word('ha') ...
>>> first >>> class Tail():
60
 ... def init (self, length): • w means write. If the file doesn’t exist, it’s created. If
... self.length = length the file does exist, it’s overwritten.
... • x means write, but only if the file does not already
>>> class Duck(): exist.
... def init (self, bill, tail): • a means append (write after the end) if the file exists.
... self.bill = bill  The second letter of mode is the file’s type:
... self.tail = tail • t (or nothing) means text.
... def about(self): • b means binary.
... print('This duck has a', bill.description, 'bill  After opening the file, you call functions to read or write data;
and a', these will be shown in the examples that follow.
tail.length, 'tail')  Last, you need to close the file.
... Write a Text File with write() :
>>> tail = Tail('long') >>> poem = '''There was a young lady named Bright,
>>> bill = Bill('wide orange') ... Whose speed was far faster than light;
>>> duck = Duck(bill, tail) ... She started one day
>>> duck.about() ... In a relative way,
 This duck has a wide orange bill and a long tail ... And returned on the previous night.'''
3) Explain briefly about File Input and Output >>> len(poem)
 The simplest kind of persistence is a plain old file, sometimes 150
called a flat file. This is just a sequence of bytes stored under a  The following code writes the entire poem to the file
filename. 'relativity' in one call:
 You read from a file into memory andwrite from memory to a >>> fout = open('relativity', 'wt')
file. >>> fout.write(poem)
 Python makes these jobs easy. Its file operations weremodeled 150
on the familiar and popular Unix equivalents. >>> fout.close()
 Before reading or writing a file, you need to open it:  The write() function returns the number of bytes written. It
fileobj = open( filename, mode ) does not add any spaces or newlines, as print() does. You can
 Here’s a brief explanation of the pieces of this call: also print() to a text file:
• fileobj is the file object returned by open() >>> fout = open('relativity', 'wt')
• filename is the string name of the file >>> print(poem, file=fout)
• mode is a string indicating the file’s type and what >>> fout.close()
you want to do with it  print() adds a space after each argument and a newline at the
 The first letter of mode indicates the operation: end. In the previous example, it appendeda newline to the
• r means read. relativity file. To make print() work like write(), pass
thefollowing two arguments:

61
 • sep (separator, which defaults to a space, ' ') ... fout.write('stomp stomp stomp')
• end (end string, which defaults to a newline, '\n') ... except FileExistsError:
... print('relativity already exists!. That was a close
 print() uses the defaults unless you pass something else. We’ll one.')
pass empty strings to suppress all of the normally added by ...
print(): relativity already exists!. That was a close one.
>>> fout = open('relativity', 'wt')
>>> print(poem, file=fout, sep='', end='') Read a Text File with read(), readline(), or readlines()
>>> fout.close()
 If you have a large source string, you can also write chunks  Call read() with no arguments to the entire file at once, as
until the source is done: shown inthe example that follows.
>>> fout = open('relativity', 'wt') >>> fin = open('relativity', 'rt' )
>>> size = len(poem) >>> poem = fin.read()
>>> offset = 0 >>> fin.close()
>>> chunk = 100 >>> len(poem)
>>> while True: 150
... if offset > size:  You can provide a maximum character count to limit how
... break much read() returns at onetime. Let’s read 100 characters at a
... fout.write(poem[offset:offset+chunk]) time and append each chunk to a poem string to rebuild the
... offset += chunk original:
... >>> poem = ''
100 >>> fin = open('relativity', 'rt' )
50 >>> chunk = 100
>>> fout.close() >>> while True:
 This wrote 100 characters on the first try and the last 50 ... fragment = fin.read(chunk)
characters ... if not fragment:
>>> fout = open('relativity', 'xt') ... break
Traceback (most recent call last): ... poem += fragment
File "<stdin>", line 1, in <module> ...
FileExistsError: [Errno 17] File exists: 'relativity' >>> fin.close()
 You can use this with an exception handler: >>> len(poem)
>>> try: 150
... fout = open('relativity', 'xt')]  Read() will return an emptystring (''), which is treated as False
File Input/Output | 175 in if not fragment. This breaks out of the while True loop.
www.it-ebooks.info

62
 You can also read the file a line at a time by using readline(). >>> print(len(lines), 'lines read')
In this next example,we’ll append each line to the poem string 5 lines read
to rebuild the original: >>> for line in lines:
>>> poem = '' ... print(line, end='')
>>> fin = open('relativity', 'rt' ) ...
>>> while True: There was a young lady named Bright,
... line = fin.readline() Whose speed was far faster than light;
... if not line: She started one day
... break In a relative way,
... poem += line And returned on the previous night.>>>
...  print() to suppress the automatic newlines because the first
>>> fin.close() four lines alreadyhad them.
>>> len(poem)  The last line did not, causing the interactive prompt >>> to
150 occur right after the last line.
 For a text file, even a blank line has a length of one (the
newline character), and isevaluated as True.
 When the file has been read, readline() (like read()) also Write a Binary File with write()
returns an empty string, which is also evaluated as False.  If you include a 'b' in the mode string, the file is opened in
 The easiest way to read a text file is by using an iterator. This binary mode. In this case, you read and write bytes instead of
returns one line at a time.It’s similar to the previous example, a string.
but with less code:  We don’t have a binary poem lying around, so we’ll just
>>> poem = '' generate the 256 byte values from 0 to 255:
>>> fin = open('relativity', 'rt' ) >>> bdata = bytes(range(0, 256))
>>> for line in fin: >>> len(bdata)
... poem += line 256
...  Open the file for writing in binary mode and write all the
>>> fin.close() data at once:
>>> len(poem) >>> fout = open('bfile', 'wb')
150 >>> fout.write(bdata)
 All of the preceding examples eventually built the single 256
string poem. The read lines() call reads a line at a time, and >>> fout.close()
returns a list of one-line strings:  Again, write() returns the number of bytes written.As with
>>> fin = open('relativity', 'rt' ) text, you can write binary data in chunks:
>>> lines = fin.readlines() >>> fout = open('bfile', 'wb')
>>> fin.close() >>> size = len(bdata)

63
 >>> offset = 0 Change Position with seek()
>>> chunk = 100  Python keeps track of where you are in the file.
>>> while True:  The tell() function returns your current offset from the
... if offset > size: beginning of the file, in bytes.
... break  The seek()function lets you jump to another byte offset in the
... fout.write(bdata[offset:offset+chunk]) file.
... offset += chunk  This means that you don’t have to read every byte in a file to
... read the last one; you can seek() to the last one and just read
100 one byte.
100  For this example, use the 256-byte binary file 'bfile' that you
56 wrote earlier:
>>> fout.close() >>> fin = open('bfile', 'rb')
Read a Binary File with read() >>> fin.tell()
 This one is simple; all you need to do is just open with 'rb': 0
>>> fin = open('bfile', 'rb')  Use seek() to one byte before the end of the file:
>>> bdata = fin.read() >>> fin.seek(255)
>>> len(bdata) 255
256  Read until the end of the file:
>>> fin.close() >>> bdata = fin.read()
 Close Files Automatically by Using with >>> len(bdata)
 If you forget to close a file that you’ve opened, it will be 1
closed by Python after it’s nolonger referenced. This means >>> bdata[0]
that if you open a file within a function and don’t closeit 255
explicitly,  seek() also returns the current offset.Call seek() with a second
 It will be closed automatically when the function ends. The argument: seek( offset, origin ):
file should be closed to force any remaining writes to be  If origin is 0 (the default), go offset bytes from the start
completed.  If origin is 1, go offset bytes from the current position
 Python has context managers to clean up things such as open  If origin is 2, go offset bytes relative to the end
files. You use the form with expression as variable:  These values are also defined in the standard os module:
>>> with open('relativity', 'wt') as fout: >>> import os
... fout.write(poem) >>> os.SEEK_SET
... 0
 After the block of code under the context manager (in this >>> os.SEEK_CUR
case, one line)completes (normally or by a raised exception), 1
the file is closed automatically. >>> os.SEEK_END

64
 2 4) Explain briefly about Modules and Import statement
 Read the last byte in different ways:  Creating and using Python code in more than one file.
>>> fin = open('bfile', 'rb')  A module is just a file of Python code.
 One byte before the end of the file:  Refer to code of other modules by using the import statement
>>> fin.seek(-1, 2) Import a Module
255  The simplest use of the import statement is import module,
>>> fin.tell() where module is the name of another Python file, without the
255 .py extension.
 Read until the end of the file:  Let’s simulate a weather station andprint a weather report.
>>> bdata = fin.read()  One main program prints the report, and a separate module
>>> len(bdata) witha single function returns the weather description used by
1 the report.
>>> bdata[0]  Here’s the main program (call it weatherman.py):
255 import report
 Here’s an example of seeking from the current position in the description = report.get_description()
file: print("Today's weather:", description)
>>> fin = open('bfile', 'rb')  And here is the module (report.py):
 This next example ends up two bytes before the end of the def get_description(): # see the docstring below?
file: """Return random weather, just like the pros"""
>>> fin.seek(254, 0) from random import choice
254 possibilities = ['rain', 'snow', 'sleet', 'fog', 'sun',
>>> fin.tell() 'who knows']
254 return choice(possibilities)
 Go forward one byte:  If you have these two files in the same directory and instruct
>>> fin.seek(1, 1) Python to run weatherman.py as the main program, it will
255 access the report module and run its get_description()
>>> fin.tell() function.
255  We wrote this version of get_description() to return a random
 Finally, read until the end of the file: resultfrom a list of strings, so that’s what the main program
>>> bdata = fin.read() will get back and print:
>>> len(bdata) $ python weatherman.py
1 Today's weather: who knows
>>> bdata[0] $ python weatherman.py
255 Today's weather: sun
 These functions are most useful for binary files. $ python weatherman.py

65
 Today's weather: sleet  Some people prefer to put all their imports at the top of the
 We used imports in two different places: file, just to make all the dependencies of their code explicit
o The main program weatherman.py imported the module Either way works.
report.
o In the module file report.py, the get_description() Import a Module with Another Name
function imported the choice function from Python’s  In our main weatherman.py program, we called import report.
standard random module. But what if you have another module with the same name or
 We also used imports in two different ways: want to use a name that is more mnemonic or shorter
o The main program called import report and then ran  In such a situation, you can import using an alias. Let’s use the
report.get_description(). alias wr:
o The get_description() function in report.py called from import report as wr
random import choice and then ran choice(possibilities). description = wr.get_description()
def get_description(): print("Today's weather:", description)
import random  Import Only What You Want from a ModuleWith Python,
possibilities = ['rain', 'snow', 'sleet', 'fog', 'sun', 'who you can import one or more parts of a module.
knows']  Each part can keep itsoriginal name or you can give it an alias.
return random.choice(possibilities) First, let’s import get_description() from the report module
 Themodule-qualified name (random.choice) is safer but with its original name:
requires a little more typing. from report import get_description
 These get_description() examples showed variations of what description = get_description()
to import, but but not where to do the importing—they all print("Today's weather:", description)
called import from inside the function.  Now, import it as do_it:
 We couldhave imported random from outside the function: from report import get_description as do_it
>>> import random description = do_it()
>>> def get_description(): print("Today's weather:", description)
... possibilities = ['rain', 'snow', 'sleet', 'fog', 'sun', 'who knows'] Module Search Path
... return random.choice(possibilities)  It uses a list of directory names and ZIP archive files stored in
... the standard sys module as the variable path.
>>> get_description()  It can access and modify this list. Here’s the value of sys.path
'who knows' for Python 3.3 on my Mac:
>>> get_description() >>> import sys
'rain' >>> for place in sys.path:
 You should consider importing from outside the function if ... print(place)
the imported code might be used in more than one place, and ...
from inside if you know its use will be limited.

66
 /Library/Frameworks/Python.framework/Versions/3.3/lib
/python33.zip
/Library/Frameworks/Python.framework/Versions/3.3/lib
/python3.3
/Library/Frameworks/Python.framework/Versions/3.3/lib
/python3.3/plat-darwin
/Library/Frameworks/Python.framework/Versions/3.3/lib
/python3.3/lib-dynload
/Library/Frameworks/Python.framework/Versions/3.3/lib
/python3.3/site-packages
 That initial blank output line is the empty string '', which
stands for the current directory.If '' is first in sys.path
 Python looks in the current directory first when youtry to
import something: import report looks for report.py.
 The first match will be used. This means that if you define a
module named random and it’s in the search path before the
standard library.
5) Describe about Relational Database.
 Access to data by multiple simultaneous users
 Protection from corruption by those users
 Efficient methods to store and retrieve the data
 Data defined by schemas and limited by constraints
 Joins to find relationships across diverse types of data
 A declarative (rather than imperative) query language: sql
 These are called relational because they show relationships
among different kinds of data in the form of tables
 A table is a grid of rows and columns, similar to
spreadsheet. To create a table, name it and specify the order,
names, and types of its columns.
 Each row has the same columns,although a column may be
defined to allow missing data
 A column or group of columns is usually the table’s primary
key; its values must beunique in the table. This prevents
adding the same data to the table more than once.
 This key is indexed for fast lookups during queries. An index
works a little like a book index, making it fast to find a
particular row.
 Each table lives within a parent database, like a file within a
directory.
 To find rows by some non-key column value, define a
secondary index onthat column. Otherwise, the database server
must perform a table scan—a brute-forcesearch of every row
for matching column values.
 Tables can be related to each other with foreign keys, and
column values can be constrained to these keys.
SQL
 SQL is not an API or a protocol, but a declarative language:
you say what you want ratherthan how to do it.
 It’s the universal language of relational databases
 There are two main categories of SQL statements:
 DDL (data definition language) :Handles creation, deletion,
constraints, and permissions for tables, databases, and uses
 DML (data manipulation language) :Handles data insertions,
selects, updates, and deletions
 Table-1. Basic SQL DDL commands
a
67
  The main DML operations of a relational database are often
known by the acronym CRUD:
• Create by using the SQL INSERT statement
• Read by using SELECT
• Update by using UPDATE
• Delete by using DELETE
 Table 2. Basic SQL DML commands
DB-API :
 DB-API is Python’s standard API for accessing relational
databases.
 Using it, you can write a single program that works with
multiple kinds of relational databases instead of writing a
separate program for each one.
 It’s similar to Java’s JDBC or Perl’s dbi.
 Its main functions are the following:
 connect() :Make a connection to the database; this can
include arguments such as username, password, server
address, and others.
 cursor() :Create a cursor object to manage queries.execute()
and executemany()Run one or more SQL commands
against the database.
68
 fetchone(), fetchmany(), and fetchall() :Get the results from
execute.
SQLite
 SQLite is a good, light, open source relational database. It’s
implemented as a standardPython library, and stores
databases in normal files.
 These files are portable across machinesand operating
systems, making SQLite a very portable solution for simple
relational database applications.
 It isn’t as full-featured as MySQL or PostgreSQL, but it
doessupport SQL, and it manages multiple simultaneous
users. Web browsers, smart phones,and other applications use
SQLite as an embedded database.
 Begin with a connect() to the local SQLite database file that
you want to use or create.
 This file is the equivalent of the directory-like database that
parents tables in other servers.
 The special string ':memory:' creates the database in memory
only; this is fast and useful for testing but will lose data when
your program terminates or if your computer goes down.
 For the next example, let’s make a database called
enterprise.db and the table zoo to manage our thriving
roadside petting zoo business. The table columns are as
follows:
 Critter :A variable length string, and our primary key
 Count :An integer count of our current inventory for
this animal
 Damages :The dollar amount of our current losses from
animal-human interactions
>>> import sqlite3
>>> conn = sqlite3.connect('enterprise.db')
>>> curs = conn.cursor()
>>> curs.execute('''CREATE TABLE zoo
(critter VARCHAR(20) PRIMARY KEY,
count INT,
 damages FLOAT)''') >>> curs.fetchall()
<sqlite3.Cursor object at 0x1006a22d0> [('weasel', 1, 2000.0)]
 Python’s triple quotes are handy when creating long strings  To close them when we’re done:
such as SQL queries.Now, add some animals to the zoo: >>> curs.close()
>>> curs.execute('INSERT INTO zoo VALUES("duck", 5, 0.0)') >>> conn.close()
<sqlite3.Cursor object at 0x1006a22d0> MySQL :
>>> curs.execute('INSERT INTO zoo VALUES("bear", 2,  Unlike SQLite, it’s an actual server, so clients can access it
1000.0)') from different devices across the network.
<sqlite3.Cursor object at 0x1006a22d0>  MysqlDB has been the most popular MySQL driver, but it has
 There’s a safer way to insert data, using a placeholder: not yet been ported toPython 3. Table lists the drivers you can
>>> ins = 'INSERT INTO zoo (critter, count, damages) use to access MySQL from Python.
VALUES(?, ?, ?)'
>>> curs.execute(ins, ('weasel', 1, 2000.0))
<sqlite3.Cursor object at 0x1006a22d0>
 If we can get all our animals out again:
>>> curs.execute('SELECT * FROM zoo')
<sqlite3.Cursor object at 0x1006a22d0>
>>> rows = curs.fetchall()
>>> print(rows)
[('duck', 5, 0.0), ('bear', 2, 1000.0), ('weasel', 1, 2000.0)]
 Ordered by their counts: PostgreSQL:
>>> curs.execute('SELECT * from zoo ORDER BY count')  PostgreSQL is a full-featured open source relational database,
<sqlite3.Cursor object at 0x1006a22d0> in many ways more advanced than MySQL.
>>> curs.fetchall()  Table presents the Python drivers you can use to access it.
[('weasel', 1, 2000.0), ('bear', 2, 1000.0), ('duck', 5, 0.0)]
 Descending order:
>>> curs.execute('SELECT * from zoo ORDER BY
count DESC')
<sqlite3.Cursor object at 0x1006a22d0>
>>> curs.fetchall()
[('duck', 5, 0.0), ('bear', 2, 1000.0), ('weasel', 1, 2000.0)]
 Which type of animal is costing us the most? SQLAlchemy:
>>> curs.execute('''SELECT * FROM zoo WHERE  The most popular cross-database Python library is
... damages = (SELECT MAX(damages) FROM zoo)''') SQLAlchemy.
<sqlite3.Cursor object at 0x1006a22d0>

69
 It isn’t in the standard library, but it’s well known and used
by many people. You caninstall it on your system by using
this command:
$ pip install sqlalchemy
 You can use SQLAlchemy on several levels:
 The lowest level handles database connection pools,
executing SQL commands, andreturning results. This is
closest to the DB-API.
 Next up is the SQL expression language, a Pythonic SQL
builder.
 Highest is the ORM (Object Relational Model) layer,
which uses the SQL Expression Language and binds
application code with relational data structures.
 The initial connection string you provide to SQLAlchemy will
determine it. That string looks like this:
dialect + driver :// user : password @ host : port / dbname
 The values you put in this string are as follows:
 Dialect : The database type
 Driver :The particular driver you want to use for that
database
 user and password :Your database authentication
strings
 host and port :The database server’s location
 dbname :The database to initially connect to on the
server
 Table SQLAlchemy connection
70
dialect driver
The engine layer :
 The connection string for SQLite skips the host, port, user, and
password.
 The dbname informs SQLite as to whatfile to use to store your
database.
 If you omit the dbname, SQLite builds a database in memory.
 If the dbname starts with a slash (/), it’s an absolute filename
on your computer
 The following is an example of an import alias,
>> import sqlalchemy as sa
 Connect to the database and create the storage for it in
>>> conn = sa.create_engine('sqlite://')
 Create a database table called zoo that comprises three
columns:
>>> conn.execute('''CREATE TABLE zoo
... (critter VARCHAR(20) PRIMARY KEY,
... count INT,
... damages FLOAT)''')
<sqlalchemy.engine.result.ResultProxy object at 0x1017efb10>
 Running conn.execute() returns a SQLAlchemy object called a
ResultProxy
 Insert three sets of data into your new empty table:
>>> ins = 'INSERT INTO zoo (critter, count, damages)
VALUES (?, ?, ?)'
>>> conn.execute(ins, 'duck', 10, 0.0)
<sqlalchemy.engine.result.ResultProxy object at 0x1017efb50>
>>> conn.execute(ins, 'bear', 2, 1000.0)
<sqlalchemy.engine.result.ResultProxy object at 0x1017ef090>
>>> conn.execute(ins, 'weasel', 1, 2000.0)
<sqlalchemy.engine.result.ResultProxy object at 0x1017ef450>
 Next, ask the database for everything that we just put in:
>>> rows = conn.execute('SELECT * FROM zoo')
In SQLAlchemy, rows is not a list; it’s that special ResultProxy
thing that we can’t print directly:
 >>> print(rows)  Check out the parentheses in that multiline call in the
<sqlalchemy.engine.result.ResultProxy object at 0x1017ef9d0> preceding example. The structureof the Table() method
 However, you can iterate over it like a list, so we can get a matches the structure of the table.
row at a time:  Just as our table containsthree columns, there are three calls to
>>> for row in rows: Column() inside the parentheses of the Table()method call.
... print(row)  Meanwhile, zoo is some magic object that bridges the SQL
... database world and the Python data structure world.
('duck', 10, 0.0)  Insert the data with more Expression Language functions:
('bear', 2, 1000.0)
('weasel', 1, 2000.0) ... conn.execute(zoo.insert(('bear', 2, 1000.0)))
The SQL Expression Language : <sqlalchemy.engine.result.ResultProxy object at 0x1017ea910>
 The next level up is SQLAlchemy’s SQL Expression >>> conn.execute(zoo.insert(('weasel', 1, 2000.0)))
Language. <sqlalchemy.engine.result.ResultProxy object at 0x1017eab10>
 It introduces functions to create the SQL for various >>> conn.execute(zoo.insert(('duck', 10, 0)))
operations. <sqlalchemy.engine.result.ResultProxy object at 0x1017eac50>
 The Expression Language handles more of theSQL dialect  Next, create the SELECT statement (zoo.select() selects
differences than the lower-level engine layer does. everything from the tablerepresented by the zoo object, such
 It can be a handy middleground approach for relational as SELECT * FROM zoo would do in plain SQL):
database applications. >>> result = conn.execute(zoo.select())
 Here’s how to create and populate the zoo table. Again, these  Finally, get the results:
are successive fragments of a single program.The import and >>> rows = result.fetchall()
connection are the same as before: >>> print(rows)
>>> import sqlalchemy as sa [('bear', 2, 1000.0), ('weasel', 1, 2000.0), ('duck',
>>> conn = sa.create_engine('sqlite://') 10, 0.0)]
 To define the zoo table, we’ll begin using some of the The Object-Relational Mapper :
Expression Language instead of  At the top layer of SQLAlchemy, the Object-Relational
SQL: Mapper (ORM) uses theSQL Expression Language but tries to
>>> meta = sa.MetaData() make the actual database mechanisms invisible.
>>> zoo = sa.Table('zoo', meta,  The basic idea behind that complicated phrase, ―object-
... sa.Column('critter', sa.String, primary_key=True), relational mapper,‖ is that you can refer to objects in your
... sa.Column('count', sa.Integer), code, and thus stay close to the way Python likes to
... sa.Column('damages', sa.Float) operatewhile still using a relational database.
... )  We’ll define a Zoo class and hook it into the ORM. This time,
>>> meta.create_all(conn) we’ll make SQLite use thefile zoo.db so that we can confirm
that the ORM worked.

71
 The initial import is the same  The add() function adds one object, and add_all() adds a list:
 >>> import sqlalchemy as sa >>> session.add(first)
>>> from sqlalchemy.ext.declarative import declarative_base >>> session.add_all([second, third])
 Here, we make the connection:  Finally, we need to force everything to complete:
>>> conn = sa.create_engine('sqlite:///zoo.db') >>> session.commit()
 Get into SQLAlchemy’s ORM. We define the Zoo class and  Use command-line sqlite3 program to check it:
associate its attributes with table columns: $ sqlite3 zoo.db
>>> Base = declarative_base() SQLite version 3.6.12
>>> class Zoo(Base): Enter ".help" for instructions
... tablename = 'zoo' Enter SQL statements terminated with a ";"
... critter = sa.Column('critter', sa.String, primary_key=True) sqlite> .tables
... count = sa.Column('count', sa.Integer) zoo
... damages = sa.Column('damages', sa.Float) sqlite> select * from zoo;
... def init (self, critter, count, damages): duck|10|0.0
... self.critter = critter bear|2|1000.0
... self.count = count weasel|1|2000.0
... self.damages = damages
... def repr (self):
... return "<Zoo({}, {}, {})>".format(self.critter, self.count, UNIT III COMPLETED
self.damages)
 The following line magically creates the database and table:
>>> Base.metadata.create_all(conn)
 You can then insert data by creating Python objects. The ORM
manages these internally: UNIT - IV
>>> first = Zoo('duck', 10, 0.0) Section –A(Five marks)
>>> second = Zoo('bear', 2, 1000.0)
>>> third = Zoo('weasel', 1, 2000.0) 1) Explain about Web clients using telnet
>>> first  The Web is a client-server system. The client makes a request
<Zoo(duck, 10, 0.0)> to a server: it opens a TCP/IP connection, sends the URL and
 Get the ORM ,we create a session to talk to the database: other information via HTTP, and receives a response.
>>> from sqlalchemy.orm import sessionmaker  The format of the response is also defined by HTTP. It
>>> Session = sessionmaker(bind=conn) includes the status of the request,and (if the request
>>> session = Session() succeeded) the response’s data and format.
 Within the session, we write the three objects that we created  The most well-known web client is a web browser.
to the database.

72
  The most well-known web client is a web browser. It can make  or our first test, we’ll use the HTTP command HEAD, which
HTTP requests in a number of ways. just retrieves some basic information about the resource:
 HTTP Request manually by typing a URL into the location HEAD / HTTP/1.1
 bar or clicking on a link in a web page. Very often, the data  That HEAD / sends the HTTP HEAD verb (command) to get
returned is used to display a website—HTML documents, information about the home page (/).
JavaScript files, CSS files, and images—but it can be anytype  Add an extra carriage return to send a blank line so the
of data, not just that intended for display. remote server knows you’re all done and want a response.
 An important aspect of HTTP is that it’s stateless. Each HTTP  You’ll receive a response such as
connection that you make is independent of all the others. this HTTP/1.1 200 OK
This simplifies basic web operations but complicates others. Date: Sat, 26 Oct 2013 17:05:17 GMT
 Here are just a few samples of the challenges: Expires: -1
o Caching :Remote content that doesn’t change should be saved Cache-Control: private, max-age=0
by the web client and used to avoid downloading from the Content-Type: text/html; charset=ISO-8859-
server again. 1 Set-Cookie:
o Sessions : A shopping website should remember the contents PREF=ID=962a70e9eb3db9d9:FF=0:TM=1382807117:LM=1382807117:
of your shopping cart. S=y...
o Authentication :Sites that require your username and password expires=Mon, 26-Oct-2015 17:05:17 GMT;
should remember them while you’re logged in. path=/;
Test with telnet : domain=.google.com
 Telnet program connect to any server and port and Set-Cookie:
type commands. Google, some basic information about NID=67=hTvtVC7dZJmZzGktimbwVbNZxPQnaDijCz716B1L56GM
9qvsqqeIGb...
its home page. Type this:
expires=Sun, 27-Apr-2014 17:05:17 GMT
$ telnet www.google.com 80 path=/;
domain=.google.com;
 If there is a web server on port 80 at google.com telnet will
print some reassuring information and then display a final HttpOnly
blank line that’s your cue to type something else: P3P: CP="This is not a P3P policy! See
Trying 74.125.225.177... http://www.google.com/support/accounts...
Server: gws
Connected to www.google.com.
Escape character is '^]'. X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
 Now, type an actual HTTP command for telnet to send to the
Alternate-Protocol: 80:quic
Google web server.
Transfer-Encoding: chunked
 The most common HTTP is GET. This retrieves the contents
of the specified resource, such as an HTML file, and returns it
to the client

73
  These are HTTP response headers and their values. Some, like
Date and Content- Type, are required. Others, such as Set-
Cookie, are used to track your activity acrossmultiple visits .
 When you make an HTTP HEAD request, you get back only
headers. If you had used the HTTPGET or POST commands,
you would also receive data from the home page
 To close telnet, type the following:
q digit:
2) Explain about Python’s Standard Web Libraries
 Python 3 has modules into two packages
• http manages all the client-server HTTP details:
—client does the client-side stuff
—server helps you write Python web servers
—cookies and cookiejar manage cookies, which save data
between site visits
• urllib runs on top of http:
—request handles the client request
—response handles the server response
—parse cracks the parts of a URL
 Standard library to get something from a website. The URL in
the following example returns a random text quote, similar to
a fortune cookie:
>>> import urllib.request as ur
>>> url = 'http://www.iheartquotes.com/api/v1/random'
>>> conn = ur.urlopen(url)
>>> print(conn)
<http.client.HTTPResponse object at 0x1006fad50>
In the official documentation, we find that conn is an HTTPResponse
object with a number of methods, and that its read() method will
give us data from the web page:
>>> data = conn.read()
>>> print(data)
b'You will be surprised by a loud noise.\r\n\n[codehappy]
http://iheartquotes.com/fortune/show/20447\n'
74
This little chunk of Python opened a TCP/IP connection to the
remote quote server, made an HTTP request, and received an HTTP
response. The response contained morethan just the page data . One
of the most important parts of the response is the HTTP status code:
>>> print(conn.status)
200
A 200 means that everything was peachy. There are dozens of
HTTP status codes, grouped into five ranges by their first (hundreds)
1xx (information)
The server received the request but has some extra
information for the client.
2xx (success)
It worked; every success code other than 200 conveys extra
details.
3xx (redirection)
The resource moved, so the response returns the new URL to
the client.
4xx (client error)
Some problem from the client side, such as the famous 404
(not found). 418 (I’m a teapot) was an April Fool’s joke.
5xx (server error)
500 is the generic whoops; you might see a 502 (bad gateway)
if there’s some disconnect between a web server and a backend
application server.
Web servers can send data back to you in any format they
like. It’s usually HTML (and usually some CSS and JavaScript), but
in our fortune cookie example it’s plain text. The data format is
specified by the HTTP response header value with the name Content-
Type, which we also saw in our google.com example:
>>> print(conn.getheader('Content-Type'))
text/plain
That text/plain string is a MIME type, and it means plain old
text. The MIME type for HTML, which the google.com example sent,
is text/html.
>>> for key, value in conn.getheaders(): True
... print(key, value) The webbrowser makes your browser do all the work.
... Web APIs and Representational State Transfer
Beyond the Standard Library: Requests  Often, data is only available within web pages. If you want to
 First, install the requests library into your Python access it, you need to access the pages through a web browser
environment. From a terminal window (Windows users, type and read it.
cmd to make one), type the following command to make the  If the authors of the website made any changes since the last
 Python package installer pip download the latest version of time you visited, the location and style of the data might have
the requests package and install it: changed.
$ pip install requests  Instead of publishing web pages, you can provide data
Let’s redo our previous call to the quotes service with requests: through a web application programming interface (API).
>>> import requests  Clients access your service by making requests to URLs and
>>> url = 'http://www.iheartquotes.com/api/v1/random'
>>> resp = requests.get(url) getting back responses containing status and data. Instead of HTML
>>> resp pages, the data is in formats that are easier for programs to consume,
<Response [200]> such as JSON or XML
>>> print(resp.text) Representational State Transfer (REST) was defined by Roy Fielding in
[codehappy] http://iheartquotes.com/fortune/show/21465 his doctoral thesis. Many products claim to have a REST interface or a
RESTful interface. In practice,this often only means that they have a
3) Discuss about Web services and Automation. web interface—definitions of URLs to access a web service.
The webbrowser Module A RESTful service uses the HTTP verbs in specific ways, as is
Start a Python session in a terminal window and type the following: described here:
>>> import antigravity HEAD
This secretly calls the standard library’s webbrowser module and Gets information about the resource, but not its data.
directs your browser to an enlightening Python link.1 GET
You can use this module directly. This program loads the main As its name implies, GET retrieves the resource’s data from
Python site’s page in your browser: the server. This is the standard method used by your browser. Any
>>> import webbrowser time you see a URL with a question mark (?) followed by a bunch of
>>> url = 'http://www.python.org/' arguments, that’s a GET request. GET should not be used to create,
>>> webbrowser.open(url) change, or delete data.
True POST
This opens it in a new window: This verb updates data on the server. It’s often used by HTML
>>> webbrowser.open_new(url) forms and web APIs.
True PUT
And this opens it in a new tab, if your browser supports tabs: This verb creates a new resource.
>>> webbrowser.open_new_tab('http://www.python.org/') DELETE
75
 This one speaks for itself: DELETE deletes. Truth in
advertising!
A RESTful client can also request one or more content types
from the server by using HTTP request h eaders. For example, a
complex service with a REST interface might prefer its input and
output to be JSON strings.
JSON
JSON is especially well suited to web clientserver data interchange.
It’s especially popular in web-based APIs, such as OpenStack.

Crawl and Scrape

1. Type the URL into your browser.
2. Wait for the remote page to load.
3. Look through the displayed page for the information you want. Saved this program as links.py and then ran this command:
4. Write it down somewhere.
$ python links.py http://boingboing.net
5. Possibly repeat the process for related URLs.
Here are the first few lines that it printed:
 An automated web fetcher is called a crawler or spider
Links in http://boingboing.net/
(unappealing terms to arachnophobes). After the contents
1 http://boingboing.net/suggest.html
have been retrieved from the remote web servers, a scraper
2 http://boingboing.net/category/feature/
parses it to find the needle in the haystack
3 http://boingboing.net/category/review/
 If you need an industrial-strength combined crawler and
4 http://boingboing.net/category/podcasts
scraper, Scrapy is worth downloading: 5 http://boingboing.net/category/video/
$ pip install scrapy
6 http://bbs.boingboing.net/
Scrapy is a framework, not a module such as BeautifulSoup. It 7 javascript:void(0)
does more, but it’s more complex to set up. 8 http://shop.boingboing.net/
Scrape HTML with BeautifulSoup 9 http://boingboing.net/about
HTML parsing is harder than it sounds. This is because much
10 http://boingboing.net/contact
of the HTML on public web pages is technically invalid: unclosed
tags, incorrect nesting, and other complications.
4) Explain briefly about Python Directories
To install BeautifulSoup, type the following
 In most operating systems, files exist in a hierarchy of
$ pip install beautifulsoup4
directories
The HTML a element represents alink, and href is its attribute
 The container of all of these files and directories is a file system
representing the link destination. In the following example,we’ll
 The standard os module deals with operating specifics such as
define the function get_links() to do the grunt work, and a main
these and provides the following functions with which you
program to get one or more URLs as command-line arguments:
can manipulate them.

76
Create with mkdir() With this function, you can go from one directory to another. Let’s
This example shows how to create a directory called poems to store leave the current directory and spend a little time in poems:
that precious verse: >>> import os
>>> os.mkdir('poems') >>> os.chdir('poems')
>>> os.path.exists('poems') >>> os.listdir('.')
True ['mcintyre']
Delete with rmdir()
This example shows how to delete a directory List Matching Files with glob()
>>> os.rmdir('poems') The glob() function matches file or directory names by using
>>> os.path.exists('poems') Unix shell rules rather than the more complete regular expression
False syntax.
List Contents with listdir() Here are those rules:
It shows listing the directory • * matches everything (re would expect .*)
>>> os.mkdir('poems') • ? matches a single character
Now, get a list of its contents (none so far): • [abc] matches character a, b, or c
>>> os.listdir('poems') • [!abc] matches any character except a, b, or c
[]
Next, make a subdirectory: Try getting all files or directories that begin with m:
>>> os.mkdir('poems/mcintyre') >>> import glob
>>> os.listdir('poems') >>> glob.glob('m*')
['mcintyre'] ['mcintyre']
Create a file in this subdirectory
>>> fout = open('poems/mcintyre/the_good_man', 'wt') How about any two-letter files or directories?
>>> fout.write('''Cheerful and happy was his mood, >>> glob.glob('??')
... He to the poor was kind and good, []
... And he oft' times did find them food,
... Also supplies of coal and wood, I’m thinking of an eight-letter word that begins with m and ends
... He never spake a word was rude, with e:
... 'Tis sad, but such is world's ways. >>> glob.glob('m??????e')
... ''') ['mcintyre']
344
>>> fout.close() What about anything that begins with a k, l, or m, and ends with e?
>>> os.listdir('poems/mcintyre') >>> glob.glob('[klm]*e')
['the_good_man'] ['mcintyre']
Change Current Directory with chdir()
5) Write a short notes on Python Process.
77
  When you run an individual program, your operating system >>> ret = subprocess.getoutput('date')
creates a single process. >>> ret
'Sun Mar 30 22:54:37 CDT 2014'
 A process is isolated from other processes
 The operating system keeps track of all the running processes, getoutput() is a string representing a complete shell command,
giving each a little time to run and then switching to another, you can include arguments, pipes, < and > I/O redirection, and so
with the twin goals of spreading the work around fairly and on:
being responsive to the user. >>> ret = subprocess.getoutput('date -u')
 It can viewed the state of your processes with graphical >>> ret
interfaces such as the Mac’s Activity Monitor (OS X), or Task 'Mon Mar 31 03:55:01 UTC 2014'
Manager onWindows-based computers.
 It can be easily access process data from your own programs. Piping that output string to the wc command counts one line,
 The standard library’s os module provides a common way of six ―words,‖ and 29 characters:
accessing some system information >>> ret = subprocess.getoutput('date -u | wc')
 For instance,the following functions get the process ID and the >>> ret
current working directory of the running ' 1 6 29'
 Python interpreter:
>>> import os A variant method called check_output() takes a list of the
>>> os.getpid() command and arguments.By default it only returns standard output
76051 as type bytes rather than a string and does not use the shell:
>>> os.getcwd() >>> ret = subprocess.check_output(['date', '-u'])
'/Users/williamlubanovic' >>> ret
And these get my user ID and group ID: b'Mon Mar 31 04:01:50 UTC 2014\n'
>>> os.getuid()
501 To show the exit status of the other program,
>>> os.getgid() getstatusoutput() returns a tuple with the status code and output:
20 >>> ret = subprocess.getstatusoutput('date')
>>> ret
Create a Process with subprocess (0, 'Sat Jan 18 21:36:23 CST 2014')
 It can start and stop other existing programs from Python by
using the standard library’s subprocess module. If you don’t want to capture the output but might want to
 If you just want to run another program in a shell and grab know its exit status, use call():
whatever output it created use the getoutput() function. >>> ret = subprocess.call('date')
 Here, we’ll get the output of the Unix date program: Sat Jan 18 21:33:11 CST 2014
>>> import subprocess >>> ret
0
78
 That date and time was printed to output but not captured
within our program. So, we saved the return code as ret.
You can run programs with arguments in two ways. The first
is to specify them in a single string. Our sample command is date -u,
which prints the current date and time in UTC
Kill a Process with terminate()
>>> ret = subprocess.call('date -u', shell=True)
Tue Jan 21 04:40:04 UTC 2014
You need that shell=True to recognize the command line date -u,
splitting it into separate strings and possibly expanding any
wildcard characters such as * .The second method makes a list of the
arguments, so it doesn’t need to call the shell:
>>> ret = subprocess.call(['date', '-u'])
Tue Jan 21 04:41:59 UTC 2014
Create a Process with multiprocessing
 You can run a Python function as a separate process or even
run multiple independent processes in a single program with
the multiprocessing module. save it as mp.py and then run it
by typing python
Output
 The Process() function
spawned a new process and
ran the do_this() function in
it.Because we did this in a
loop that had four passes, we
generated four new
79
processes that executed do_this() and then exited.
 The multiprocessing module has more activities; for example,
downloading web pages for scraping, resizing images, and so
on.
 It includes ways to queue tasks, enable intercommunication
among processes, and wait for all the processes to finish.
 If you created one or more processes and want to terminate
one for some reason use terminate().
 In the example that follows, our process would count to a
million, sleeping at each step for a second, and printing an
irritating message. However, our main program runs out of
patience in five seconds and nukes it from orbit:
Section –B(Ten marks)
1) Explain about Files
 File is a named location on disk to store related information. It
is used to permanently store data in a non-volatile memory
(e.g. hard disk).
 Since, random access memory (RAM) is volatile which loses
its data when computer is turned off, we use files for future
use of the data.
 When we want to read from or write to a file we need to open
it first. When we are done, itneeds to be closed, so that
resources that are tied with the file are freed.
 Hence, in Python, a file operation takes place in the following
order.
o Open a file
o Read or write (perform operation)
o Close the file
Opening a file
 Python has a built-in function open() to open a file. This
function returns a file object, also called a handle, as it is used
to read or modify the file accordingly.
 
>>> f = open("test.txt") # open file in current directory

>>> f = open("C:/Python33/README.txt") # specifying full path

 We can specify the mode while opening a file. In mode, we
specify whether we want to read'r', write 'w' or append 'a' to Hence, when working with files in text mode, it is highly
the file. recommended to specify the encoding type.
 We also specify if we want to open the file in text mode or
binary mode.
 The default is reading in text mode. In this mode, we get
strings when reading from the file. Closing a File
 On the other hand, binary mode returns bytes and this is the  When we are done with operations to the file, we need to
mode to be used when dealingwith non-text files like image properly close it.
or exe files.  Closing a file will free up the resources that were tied with the
file and is done using the close() method.

 Python has a garbage collector to clean up unreferenced
objects but, we must not rely on it to close the file.

f = open("test.txt",encoding = 'utf-8')
# perform file operations
f.close()

 This method is not entirely safe. If an exception occurs when

we are performing some operation with the file, the code exits
without closing the file. A safer way is to use a try...finally
block.
try:
f = open("test.txt",encoding = 'utf-8')
# perform file operations
finally:
f.close()
 This way, we are guaranteed that the file is properly closed
even if an exception is raised,causing program flow to stop.
 The best way to do this is using the with statement. This
ensures that the file is closed when the block inside with is
exited.

80
We don't need to explicitly call the close() method. It is done
internally.
open("test.txt",encoding = 'utf-8') as f:
# perform file operations
Reading and writing
 A text file is a sequence of characters stored on a permanent
medium like a hard drive, flash memory, or CD-ROM.
 To write a file, you have to open it with mode 'w' as a second
parameter:
>>> fout = open('output.txt', 'w')
>>> print fout
<open file 'output.txt', mode 'w' at 0xb7eb2410>
 If the file already exists, opening it in write mode clears out
the old data and starts fresh,so be careful! If the file doesn’t
exist, a new one is created.
 The write method puts data into the file.
>>> line1 = "This here's the wattle,\n"
>>> fout.write(line1)
Again, the file object keeps track of where it is, so if you call write
again, it adds the new datato the end.
>>> line2 = "the emblem of our land.\n"
>>> fout.write(line2)
When you are done writing, you have to close the file.
>>> fout.close()
Format operator
 The argument of write has to be a string, so if we want to put
other values in a file, we haveto convert them to strings. The
easiest way to do that is with str:
>>> x = 52
>>> fout.write(str(x))
 An alternative is to use the format operator, %. When applied
to integers, % is the modulusoperator. But when the first
operand is a string, % is the format operator.
 The first operand is the format string, which contains one or
more format sequences, whichspecify how the second
operand is formatted. The result is a string.
81
 For example, the format sequence '%d' means that the second
operand should be formatted as an integer (d stands for
―decimal‖):
>>> camels = 42
>>> '%d' % camels
'42'
 The result is the string '42', which is not to be confused with
the integer value 42.
 A format sequence can appear anywhere in the string, so you
can embed a value in a sentence:
>>> camels = 42
>>> 'I have spotted %d camels.' % camels
'I have spotted 42 camels.'
 If there is more than one format sequence in the string, the
second argument has to be a tuple.
 Each format sequence is matched with an element of the
tuple, in order.The following example uses '%d' to format an
integer, '%g' to format a floating-point numberand '%s' to
format a string:
>>> 'In %d years I have spotted %g %s.' % (3, 0.1, 'camels')
'In 3 years I have spotted 0.1 camels.'
 The number of elements in the tuple has to match the number
of format sequences in thestring. Also, the types of the
elements have to match the format sequences:
>>> '%d %d %d' % (1, 2)
TypeError: not enough arguments for format string
>>> '%d' % 'dollars'
TypeError: illegal argument type for built-in
operation
Filenames and paths
 Files are organized into directories (also called ―folders‖).
Every running program has a ―current directory,‖ which is
the default directory for most operation s. For example,when
you open a file for reading,
 Python looks for it in the current directory. The os module  To demonstrate these functions, the following example
provides functions for working with files and directories (―os‖ ―walks‖ through a directory, prints the names of all the files,
stands for and calls itself recursively on all the directories.
―operating system‖). os.getcwd returns the name of the def walk(dirname):
current directory: for name in os.listdir(dirname):
>>> import os path = os.path.join(dirname, name)
>>> cwd = os.getcwd() if os.path.isfile(path):
>>> print cwd print path
/home/dinsdale else:
 cwd stands for ―current working directory.‖ The result in this walk(path)
example is /home/dinsdale, which is the home directory of a  os.path.join takes a directory and a file name and joins them
user named dinsdale. into a complete path.
 A string like cwd that identifies a file is called a path.
 A relative path starts from the current directory; an absolute
path starts from the topmost directory in the file system. Check Existence with exists()
 The paths we have seen so far are simple filenames, so they  To verify whether the file or directory is exits or not usin
are relative to the current directory. exists(), with a relative or absolute pathname, as
 To find the absolute path to a file, you can use demonstrated here:
os.path.abspath: >>> import os
>>> os.path.abspath('memo.txt') >>> os.path.exists('oops.txt')
'/home/dinsdale/memo.txt' True
 os.path.exists checks whether a file or directory exists: >>> os.path.exists('./oops.txt')
>>> os.path.exists('memo.txt') True
True >>> os.path.exists('waffles')
 If it exists, os.path.isdir checks whether it’s a directory: False
>>> os.path.exists('.')
>>> os.path.isdir('memo.txt')
True
False
>>> os.path.exists('..')
>>> os.path.isdir('music')
True True
Check Type with isfile()
 Similarly, os.path.isfile checks whether it’s a file.os.listdir
 The functions in this section check whether a name refers to a
returns a list of the files (and other directories) in the given
file, directory, or symboliclink
directory:
>>> name = 'oops.txt'
>>> os.listdir(cwd)
>>> os.path.isfile(name)
['music', 'photos', 'memo.txt']
True
Here’s how you determine a directory:

82
 >>> os.path.isdir(name)  A symbolic link is an alternative method that stores the new
False name as its own file, making it possible for you to get both the
 A single dot (.) is shorthand for the current directory, and two original and new names at once.
dots (..) stands for the parent directory. These always exist, so  The link() call creates a hard link, and symlink() makes a
a statement such as the following will always report True: symbolic link
>>> os.path.isdir('.')  The islink() function checks whether the file is a symbolic link.
True  Here’s how to make a hard link to the existing file oops.txt
 The os module contains many functions dealing with from the new file yikes.txt:
pathnames One such function, isabs(), determines whether its >>> os.link('oops.txt', 'yikes.txt')
argument is an absolute pathname. The argument doesn’t >>> os.path.isfile('yikes.txt')
need to be the name of a real file: True
>>> os.path.isabs(name)  To create a symbolic link to the existing file oops.txt from the
False new file jeepers.txt, usethe following:
>>> os.path.isabs('/big/fake/name') >>> os.path.islink('yikes.txt')
True False
>>> os.path.isabs('big/fake/name/without/a/leading/slash') >>> os.symlink('oops.txt', 'jeepers.txt')
False >>> os.path.islink('jeepers.txt')
True
Copy with copy() Change Permissions with chmod()
The copy() function comes from another module, shutil. This  On a Unix system, chmod() changes file permissions.There are
example copies the file oops.txt to the file ohno.txt: read, write, and execute permissions for the user the main
>>> import shutil group that
>>> shutil.copy('oops.txt', 'ohno.txt') the user is in, and the rest of the world.
The shutil.move() function copies a file and then removes the  The command takes an intensely compressed octal (base 8)
original. value that combines user, group, and other permissions.
Change Name with rename()This function does exactly what it says.  For instance, to make oops.txt only readable by its owner, type
In the example here, it renames ohno.txt to ohwell.txt: the following:
>>> import os
>>> os.rename('ohno.txt', 'ohwell.txt') >>> os.chmod('oops.txt', 0o400)
Link with link() or symlink()
 If you don’t want to deal with cryptic octal values and would
 In Unix, a file exists in one place, but it can have multiple rather deal with obscure cryptic symbols, you can import
names, called links.
some constants from the stat module and use a statement such
 In lowlevel hard links, it’s not easy to find all the names for a as the following:
given file.
>>> import stat
>>> os.chmod('oops.txt', stat.S_IRUSR)

83
Change Ownership with chown()  Use setfirstweekday() to set the first day of the week to
 This function is also Unix/Linux/Mac–specific. You can Sunday (6) or to any other weekday. Parameters that specify
change the owner and/or group ownership of a file by dates are given as integers.
specifying the numeric user ID (uid) and group ID (gid): Name Description
>>> uid = 5 Return an iterator for the weekdays
>>> gid = 22 iterweekdays() method
numbers that will be used for one week.
>>> os.chown('oops', uid, gid)
Get a Pathname with abspath() The itermonthdates() method is used to
itermonthdates()
 This function expands a relative name to an absolute one. If return an iterator for the month month (1-
method
your current directory is /usr/gaberlunzie and the file oops.txt is 12) in the year year.
there, also, you can type the following: The itermonthdays2() method is used to
itermonthdays2()
>>> os.path.abspath('oops.txt') return an iterator for the month month in
method
'/usr/gaberlunzie/oops.txt' the year year similar to itermonthdates().
The itermonthdays() method is used to
Get a symlink Pathname with realpath() itermonthdays() method return an iterator for the month month in
 A symbolic link to oops.txt from the new file jeepers.txt. In the year year similar to itermonthdates().
circumstances such as this, you can get the name of oops.txt The monthdatescalendar() method is used to
from jeepers.txt by using the realpath() function, as shown monthdatescalendar() return a list of the weeks in the month
here: method month of the year as full weeks. Weeks are
>>> os.path.realpath('jeepers.txt') lists of seven datetime.date objects.
'/usr/gaberlunzie/oops.txt'
The monthdays2calendar() method is used
to return a list of the weeks in the month
Delete a File with remove() monthdays2calendar()
month of the year as full weeks. Weeks are
 In this snippet, we use the remove() function and say farewell method
lists of seven tuples of day numbers and
to oops.txt:
weekday numbers.
>>> os.remove('oops.txt')
>>> os.path.exists('oops.txt') The monthdayscalendar() method is used
False monthdayscalendar() to return a list of the weeks in the month
method month of the year as full weeks. Weeks are
2) Describe about Calendar lists of seven day numbers.
 The calendar module allows you to output calendars like the The yeardatescalendar() method is used to
Unix cal program and provides additional useful functions yeardatescalendar() return the data for the specified year ready
related to the calendar. method for formatting. The return value is a list of
 By default, these calendars have Monday as the first day of month rows.
the week, and Sunday as the last (the European convention). yeardays2calendar() The yeardays2calendar() method is used to

84
method return the data for the specified year ready The isleap() method is used to returns True
isleap() method
for formatting (similar to if year is a leap year, otherwise False.
yeardatescalendar()). The leapdays() method is used to returns
The yeardayscalendar() method is used to the number of leap years in the range from
leapdays() method
yeardayscalendar() return the data for the specified year ready y1 to y2 (exclusive), where y1 and y2 are
method for formatting (similar to years.
yeardatescalendar()) The weekday() method is used to returns
The formatmonth() method is used to weekday() method the day of the week (0 is Monday) for year
text-calendar-
return a month’s calendar in a multi-line (1970–...), month (1–12), day (1–31).
formatmonth() method
string. The weekheader() method is used to return
The prmonth() method is used to Print a a header containing abbreviated weekday
text-calendar-prmonth() weekheader() method
month’s calendar as returned by names. n specifies the width in characters
method
formatmonth(). for one weekday.
The formatyear() method is used to return The monthrange() method is used to
text-calendar-
a m-column calendar for an entire year as a returns weekday of first day of the month
formatyear() method monthrange() method
multi-line string. and number of days in month, for the
The pryear() method is used to Print the specified year and month.
text-calendar-pryear()
calendar for an entire year as returned by The monthrange() method is used to
method monthcalendar()
formatyear(). returns a matrix representing a month’s
method
The formatmonth() method is used to calendar.
html-calendar-
return a month’s calendar as an HTML The prmonth() method is used to prints a
formatmonth() method prmonth() method
table. month’s calendar as returned by month().
html-calendar- The formatyear() method is used to return The month() method is used to returns a
formatyear()method a year’s calendar as an HTML table. month’s calendar in a multi-line string
month() method
html-calendar- The formatyearpage() method is used to using the formatmonth() of the
formatyearpage() return a year’s calendar as a complete TextCalendar class.
method HTML page. The prcal() method is used to prints the
setfirstweekday() Sets the weekday (0 is Monday, 6 is prcal() method calendar for an entire year as returned by
method Sunday) to start each week. calendar().
The firstweekday() method is used to The calendar() method is used to returns a
firstweekday() method returns the current setting for the weekday 3-column calendar for an entire year as a
calendar() method
to start each week. multi-line string using the formatyear() of
the TextCalendar class.

85
3) Expalin about date and time modules

 The datetime module offers functions and classes for working

with date and time parsing, formatting, and arithmetic.
 Two kinds of date and time objects exist- naïve and aware.
 To use this module, we must first import it.

>>> import datetime

 It defines four main objects,each with many methods:

 date for years, months, and days
 time for hours, minutes, seconds, and fractions
 datetime for dates and times together
 timedelta for date and/or time intervals
Date Object
• The range of date is from date.min (year=1, month=1, day=1)
to date.max (year=9999,month=12, day=31).
• The datetime module’s time object is used to represent a time
of day:

• Print a date with its isoformat() method:

o The iso refers to ISO 8601, an international standard for
representing dates and times

• This example uses the today() method to generate today’s

date: • The datetime object includes both the date and time of day.
You can create one directly, such as the one that follows,
which is for January 2, 2014, at 3:04 A.M., plus 5 seconds and 6
microseconds:

• Use of a timedelta object to add some time interval to a date:

86
• The datetime object also has an isoformat() method

Using the time Module

 Python has a datetime module with a time object, and a
• That middle T separates the date and time parts. separate
• datetime has a now() method with which you can get the time module
current date and time:  The time module’s time() function returns the current time as
an epoch value:
>>> import time
>>> now = time.time()
>>> now
1391488263.664645
 Convert an epoch value to a string by using ctime():
>>> time.ctime(now)
'Mon Feb 3 22:31:03 2014'
 Time provides as struct_time objects. localtime() provides
thetime in your system’s time zone, and gmtime() provides it
in UTC:
>>> time.localtime(now)
time.struct_time(tm_year=2014, tm_mon=2, tm_mday=3,
tm_hour=22, tm_min=31,
tm_sec=3, tm_wday=0, tm_yday=34, tm_isdst=0)
 Merge a date object and a time object into a datetime object by
using combine(): >>> time.gmtime(now)
time.struct_time(tm_year=2014, tm_mon=2, tm_mday=4,
tm_hour=4, tm_min=31,
tm_sec=3, tm_wday=1, tm_yday=35, tm_isdst=0)

 In my (Central) time zone, 22:31 was 04:31 of the next day in

UTC (formerly called Greenwich time or Zulu time). If you omit
 Yank the date and time from a datetime by using the date() the argument to localtime() or gmtime(), they assume the
and time() methods: current time.
87
  The opposite of these is mktime(), which converts a  Here’s the strftime() function provided by the time module. It
struct_time object to epoch seconds: converts a struct_timeobject to a string.
>>> tm = time.localtime(now)
>>> time.mktime(tm)
1391488263.0
Read and Write Dates and Times
 isoformat() is not the only way to write dates and times.
ctime()function in the time module, which you can use to
convert epochs to strings:
>>> import time
>>> now = time.time()  If we try this with a date object, only the date parts will work,
>>> time.ctime(now) and the time defaults to midnight:
>>> from datetime import date
'Mon Feb 3 21:14:36 2014' >>> some_day = date(2014, 7, 4)
 Convert dates and times to strings by using strftime(). >>> fmt = "It's %B %d, %Y, local time %I:%M:%S%p"
 This is provided as a method in the datetime, date, and time >>> some_day.strftime(fmt)
objects, and as a function in the timemodule. strftime() uses "It's Friday, July 04, 2014, local time 12:00:00AM"
format strings to specify the output, which you can see in  For a time object, only the time parts are converted:
>>> from datetime import time
>>> some_time = time(10, 35)
>>> some_time.strftime(fmt)
"It's Monday, January 01, 1900, local time 10:35:00AM"
 Convert a string to a date or time, use strptime() with the
sameformat string. There’s no regular expression pattern
matching; the nonformat parts ofthe string (without %) need
to match exactly.
 Let’s specify a format that matches yearmonth-day, such as
2012-01-29.
 Names are specific to your locale—internationalization
settings for your operating system.
 To print different month and day names, change your locale
by usingsetlocale(); its first argument is locale.LC_TIME for
dates and times, and the secondis a string combining the
language and country abbreviation.
 Numbers are zero-padded on the left. >>> import locale
>>> from datetime import date

88
 >>> halloween = date(2014, 10, 31) ['de_at', 'de_de', 'de_ch', 'de_lu', 'de_be']
>>> for lang_country in ['en_us', 'fr_fr', 'de_de',
'es_es', 'is_is',]: Alternative Modules
... locale.setlocale(locale.LC_TIME, arrow
lang_country) This combines many date and time functions with a simple API.
... halloween.strftime('%A, %B %d')
... dateutil
'en_us' This module parses almost any date format and handles relative
'Friday, October 31' dates and times well.
'fr_fr'
'Vendredi, octobre 31' iso8601
'de_de' This fills in gaps in the standard library for the ISO8601 format.
'Freitag, Oktober 31'
'es_es' fleming
'viernes, octubre 31' This module offers many time zone functions.
'is_is'
'fostudagur, oktober 31' 4) Discuss about Web servers
>>>  Web developers have found Python to be an excellent
>>> import locale
language for writing web servers and server-side programs.
>>> names = locale.locale_alias.keys()
 This has led to such a variety of Python-based web frameworks
 From names, let’s get just locale names that seem to work that it can be hard to navigate among them and make choices
with setlocale(), such as the ones we used in the preceding
 A web framework provides features with which you can build
example—a two-character language code followed by an
websites, so it does more than a simple web (HTTP) server.,
underscore and a two-character country code:
The Simplest Python Web Server
>>> good_names = [name for name in names if Run a simple web server by typing just one line of Python:
\ $ python -m http.server
len(name) == 5 and name[2] == '_']
This implements a bare-bones Python HTTP server. If there are
no problems, this willprint an initial status message:
 What do the first five look like?
Serving HTTP on 0.0.0.0 port 8000 ...
>>> good_names[:5]
['sr_cs', 'de_at', 'nl_nl', 'es_ni', 'sp_yu']  That 0.0.0.0 means any TCP address,
 So, if you wanted all the German language locales, try this:
 Request files, with pa ths relative to your current
>>> de = [name for name in good_names if
directory, and they will be returned. If you type
name.startswith('de')]
http://localhost:8000 in your web browser, you should see
>>> de

89
 a directory listing there, and the server will print access
log lines such as this:
127.0.0.1 - - [20/Feb/2013 22:02:37] "GET / HTTP/1.1" 200 -
 localhost and 127.0.0.1 are TCP synonyms for your local
computer, so this works regardless of whether you’re
connected to the Internet. You can interpret this line as
follows:
• 127.0.0.1 is the client’s IP address
• The first "-" is the remote username, if found
• The second "-" is the login username, if required
• [20/Feb/2013 22:02:37] is the access date and time
• "GET / HTTP/1.1" is the command sent to the web
server:
—The HTTP method (GET)
—The resource requested (/, the top)
—The HTTP version (HTTP/1.1)
• The final 200 is the HTTP status code returned by the
web server
 Click any file. If your browser can recognize the format
(HTML, PNG, GIF, JPEG, and so on) it should display it,
and the server will log the request.
 For instance, if you have the file oreilly.png in your current
directory, a request for http://localhost:8000/oreilly.
o png should return the image of the unsettling
127.0.0.1 - - [20/Feb/2013 22:03:48] "GET /oreilly.png HTTP/1.1"
200 -
 If you have other files in the same directory on your
computer, they should show up in a listing on your
display, and you can click any one to download it. If your
browser is configured to display that file’s format, you’ll
see the results on your screen;
 otherwise,your browser will ask you if you want to
download and save the file.The default port number used
is 8000, but you can specify another:
90
$ python -m http.server 9999
 You should see this:
Serving HTTP on 0.0.0.0 port 9999 ...
 This Python-only server is best suited for quick tests. You can
stop it by killing itsprocess; in most terminals, press Ctrl+C.
 Traditional webservers such as Apache and Nginx are much
faster for serving static files.
.
Web Server Gateway Interface
 In the early days of the Web, the Common Gateway Interface (CGI)
was designed for clients to make web servers run external
programs and return the results.
 CGI also handled getting input arguments from the client
through
the server to the external programs. However, the programs were
started anew for each client access. This could not scale well,
because even small programs have appreciable startup time.
 To avoid this startup delay, people began merging the language
interpreter into the web server.
 Apache ran PHP within its mod_php module, Perl in mod_perl,
and Python in mod_python. Then, code in these dynamic
languages could be executed within the long running
 Apache process itself rather than in external programs.
 An alternative method was to run the dynamic language within a
separate long-running program and have it communicate with
the web server.
 FastCGI and SCGI are examples.
 Python web development made a leap with the definition of Web
Server Gateway Interface (WSGI), a universal API between Python
web applications and web servers
 A web framework handles, at a minimum, client requests and
server responses. It might provide some or all of these features:
Routes : Interpret URLs and find the corresponding server files or
Python server code

Templates : Merge server-side data into pages of HTML
Authentication and authorization : Handle usernames, passwords,
permissions
Sessions : Maintain transient data storage during a user’s visit to
the website
Bottle
 Bottle consists of a single Python file, so it’s very easy to
try out, and it’s easy to deploy later. Bottle isn’t part of
standard
 Python, so to install it, type the following command:
$ pip install bottle
 Here’s code that will run a test web server and return a
line of text when your browser accesses the URL
http://localhost:9999/. Save it as bottle1.py:
from bottle import route, run
@route('/')
def home():
return "It isn't fancy, but it's my home page"
run(host='localhost', port=9999)
 Bottle uses the route decorator to associate a URL with the
following function; in this case, / (the home page) is
handled by the home() function. Make Python run this
server script by typing this:
$ python bottle1.py
 You should see this on your browser when you access
http://localhost:9999:
 The run() function executes bottle’s built-in Python test
web server. You don’t need to use this for bottle programs,
but it’s useful for initial development and testing.
 Now, instead of creating text for the home page in code,
let’s make a separate HTML file called index.html that
contains this line of text:
 My <b>new</b> and <i>improved</i> home page!!!
91
 Make bottle return the contents of this file when the home
page is requested. Save this script as bottle2.py:
from bottle import route, run, static_file
@route('/')
def main():
return static_file('index.html', root='.')
run(host='localhost', port=9999)
 In the call to static_file(), we want the file index.html in the
directory indicated by root (in this case, '.', the current directory).
 If your previous server example code wasstill running, stop it.
Now, run the new server:
$ python bottle2.py
 When you ask your browser to get http:/localhost:9999/, you
should see:
My new and improved home page!!!Let’s add one last example
that shows how to pass arguments to a URL and use them.Of
course, this will be bottle3.py:
from bottle import route, run, static_file
@route('/')
def home():
return static_file('index.html', root='.')
@route('/echo/<thing>')
def echo(thing):
return "Say hello to my little friend: %s!" % thing
run(host='localhost', port=9999)
 We have a new function called echo() and want to pass it a string
argument in a URL.
 That’s what the line @route('/echo/<thing>') in the preceding
example does. That <thing> in the route means that whatever
was in the URL after /echo/ is assigned to the string argument
thing, which is then passed to the echo function.
 To see what happens, stop the old server if it’s still running, and
start it with the new code:
$ python bottle3.py
 Then, access http://localhost:9999/echo/Mothra in your web browser.
You should see the following:
Say hello to my little friend: Mothra!
 Now, leave bottle3.py running for a minute so that we can try
something else. You’ve been verifying that these examples work
by typing URLs into your browser and looking at the displayed
pages.
 You can also use client libraries such as requests to do your work
for you. Save this as bottle_test.py:
import requests
resp =
requests.get('http://localhost:9999/echo/Mothr
a')
if resp.status_code == 200 and \
resp.text == 'Say hello to my little friend:
Mothra!':
print('It worked! That almost never happens!')
else:
print('Argh, got this:', resp.text)
Great! Now, run it:
$ python bottle_test.py
arguments when you call run():
• debug=True creates a debugging page if you get an HTTP
error;
• reloader=True reloads the page in the browser if you change
any of the Python code.
Flask
 Bottle is a good initial web framework.
 It started in 2010 as an April Fools’ joke, but enthusiastic
response encouraged the author, Armin Ronacher, to make
it a real framework. He named the result Flask as a
wordplay on bottle.
 Flask is about as simple to use as Bottle, but it supports
many extensions that are useful in professional web
92
development, such as Facebook authentication and
database integration.
 It’s my personal favorite among Python web frameworks
because it balances ease of use with a rich feature set.
 The Flask package includes the werkzeug WSGI library
and the jinja2 template library.
 You can install it from a terminal:
$ pip install flask
Non-Python Web Servers
Faster web server. The usual choices are the following:
• apache with the mod_wsgi module
• nginx with the uWSGI app server
Both work well; apache is probably the most popular, and nginx
has a reputation for stability and lower memory use.
Apache
 The apache web server’s best WSGI module is mod_wsgi.
This can run Python code within the Apache process or in
separate processes that communicate with Apache.You
should already have apache if your system is Linux or OS
X.
 For Windows, you’llneed to install apache.
 Finally, install your preferred WSGI-based Python web
framework. Let’s try bottle here. Almost all of the work
involves configuring Apache, which can be a dark art.
Create this test file and save it as /var/www/test/home.wsgi:
import bottle
application = bottle.default_app()
@bottle.route('/')
def home():
return "apache and wsgi, sitting in a tree"
 Do not call run() this time, because that starts the built-in
Python web server. We need to assign to the variable
 application because that’s what mod_wsgi looks for to $ WSGIDaemonProcess domain-name user=user-name
marrythe web server and the Python code. group=group-name threads=25
 If apache and its mod_wsgi module are working correctly, WSGIProcessGroup domain-name
we just need to connect them to our Python script. We  In the preceding example, user-name and group-name are the
want to add one line to the file that defines the default operating system user and group names, and the domain-name is
website the name of your Internet domain.
 for this apache server, but finding that file is a task in and  A minimal apache config might look like this:
of itself. It could be /etc/apache2/httpd.conf, or <VirtualHost *:80>
/etc/apache2/sites-available/default, DocumentRoot /var/www
 Let’s assume for now that you understand apache and Web Servers | 233
found that file. Add this line inside the <VirtualHost> www.it-ebooks.info
section that governs the default website: WSGIScriptAlias / /var/www/test/home.wsgi
WSGIDaemonProcess mydomain.com user=myuser
group=mygroup threads=25
WSGIScriptAlias / /var/www/test/home.wsgi WSGIProcessGroup mydomain.com
<Directory /var/www/test>
<VirtualHost *:80> Order allow,deny
DocumentRoot /var/www
WSGIScriptAlias / /var/www/test/home.wsgi
<Directory /var/www/test> Allow from all
Order allow,deny </Directory>
Allow from all </VirtualHost>
</Directory> The nginx Web Server
</VirtualHost>  The nginx web server does not have an embedded Python
module. Instead, it communicates by using a separate
 Start apache, or restart it if it was running to make it use WSGI server such as uWSGI.
this new configuration.  Together they make a very fast and configurable platform
 If youthen browse to http://localhost/, you should see: for Python web development.
apache and wsgi, sitting in a tree  You can install nginx from its website. You also need to
 This runs mod_wsgi in embedded mode, as part of apache install uWSGI. uWSGI is a large system, with many levers
itself. and knobs to adjust
 You can also run it in daemon mode: as one or more Other Frameworks
processes, separate from apache. To do this, add two new The current main contenders include:
directive lines to your apache config file: django

93
  This is the most popular, especially for large sites. It’s  It started in 2010 as an April Fools’ joke, but enthusiastic
worth learning for many reasons, among them the response encouraged the author, Armin Ronacher, to make
frequent requests for django experience in Python job ads. it a real framework. He named the result Flask as a
 It includes ORM (―The Object-Relational Mapper‖)to wordplay on bottle.
create automatic web pages for the typical database CRUD  Flask is about as simple to use as Bottle, but it supports
functions (create, replace, update, delete) many extensions that are useful in professional web
web2py development, such as Facebook authentication and
 This covers much the same ground as django, with a database integration.
different style.  It’s my personal favorite among Python web frameworks
pyramid because it balances ease of use with a rich feature set.
 This grew from the earlier pylons project, and is similar to  The Flask package includes the werkzeug WSGI library
django in scope. and the jinja2 template library.
turbogears  You can install it from a terminal:
 This framework supports an ORM, many databases, and $ pip install flask
multiple template languages.
wheezy.web
 This is a newer framework optimized for performance. It
was faster than the others in a recent test.
Let’s replicate the final bottle example code in flask. First, though,
we need to make a few changes:
• Flask’s default directory home for static files is static, and URLs
Other Python Web Servers for files there also begin with /static. We change the folder to '.'
Following are some of the independent Python-based WSGI (current directory) and the URL prefix to '' (empty) to allow the
servers that work like apache or nginx, using multiple processes URL / to map to the file index.html.
and/or threads to handle simultaneous requests: • In the run() function, setting debug=True also activates the
• uwsgi automatic reloader; bottle used separate arguments for
• cherrypy debugging and reloading.
• pylons
Here are some event-based servers, which use a single process but  Save this file to flask1.py:
avoid blocking on any single request: from flask import Flask
• tornado app = Flask( name , static_folder='.',
• gevent static_url_path='')
• gunicorn @app.route('/')
5) Describe about Flask: def home():
Flask return app.send_static_file('index.html')
 Bottle is a good initial web framework.
94
 @app.route('/echo/<thing>') <body>
def echo(thing): Say hello to my little friend: {{ thing }}
return "Say hello to my little friend: %s" % thing </body>
app.run(port=9999, debug=True) </html>
 Then, run the server from a terminal or window:  Next, we’ll write the server code to grab this template, fill in the
$ python flask1.py value of thing that wepassed it, and render it as HTML (I’m
 Test the home page by typing this URL into your browser: dropping the home() function here to save space).
http://localhost:9999/ Save this as flask2.py:
 You should see the following from flask import Flask, render_template
My new and improved home page!!! app = Flask( name )
 Try the /echo endpoint: @app.route('/echo/<thing>')
http://localhost:9999/echo/Godzilla def echo(thing):
 You should see this: return render_template('flask2.html', thing=thing)
Say hello to my little friend: Godzilla app.run(port=9999, debug=True)
 There’s another benefit to setting debug to True when calling  That thing = thing argument means to pass a variable named
run. If an exception occurs in the server code, thing to the template,with the value of the string thing.
 Flask returns a specially formatted page with useful details  Ensure that flask1.py isn’t still running, and start flask2.py:
about $ python flask2.py
what went wrong, and where. Even better, you can type some Now, type this URL:
commands to see the values of variables in the server http://localhost:9999/echo/Gamera
program. You should see the following:
 Do not set debug = True in production web servers. It exposes Say hello to my little friend: Gamera
too  Let’s modify our template and save it in the templates
much information about your server to potential intruders. directory as flask3.html:
 Flask example just replicates what we did with bottle. What <html>
can Flask do that bottle can’t? Flask includes jinja2, a more <head>
extensive templating system. Here’s a tiny example of how to <title>Flask3 Example</title>
use jinja2 and flask together. </head>
<body>
 Create a directory called templates, and a file within it called Say hello to my little friend: {{ thing }}.
flask2.html: Alas, it just destroyed {{ place }}!
<html> </body>
<head> </html>
<title>Flask2 Example</title>  You can pass this second argument to the echo URL in
</head> many ways.

95
Pass an argument as part of the URL path  When a GET command is used for a URL, any arguments
are passed in the form
from flask import Flask, render_template &key1=val1&key2=val2&...
app = Flask( name )
@app.route('/echo/<thing>/<place>')  You can also use the dictionary ** operator to pass
def echo(thing, place): multiple arguments to a template from a single dictionary
return render_template('flask3.html', thing=thing, (call this flask3c.py):
place=place)
app.run(port=9999, debug=True) from flask import Flask, render_template, request
As usual, stop the previous test server script if it’s still running app = Flask( name )
and then try this new one: @app.route('/echo/')
$ python flask3a.py def echo():
 The URL would look like this: kwargs = {}
http://localhost:9999/echo/Rodan/McKeesport kwargs['thing'] = request.args.get('thing')
 And you should see the following: kwargs['place'] = request.args.get('place')
Say hello to my little friend: Rodan. Alas, it just destroyed return render_template('flask3.html', **kwargs)
McKeesport!Or, you can provide the arguments as GET app.run(port=9999, debug=True)
parameters (save this as flask3b.py):  That **kwargs acts like thing=thing, place=place.
from flask import Flask, render_template, request  It saves some typing if there are a lot of input arguments.
app = Flask( name )
@app.route('/echo/') UNIT IV COMPLETED
def echo():
thing = request.args.get('thing')
place = request.args.get('place')
return render_template('flask3.html', thing=thing,
place=place)
UNIT - V
app.run(port=9999, debug=True)
1) Explain the following
Run the new server script:
i) Python Working in clouds
$ python flask3b.py
ii)Big Fat Data and Map Reduce
 This time, use this URL:
i) Python Working in clouds :
http://localhost:9999/echo?thing=Gorgo&place=Wilmerding
 We would buy your own servers, bolt them into racks in data
 You should get back what you see here: centers, and install layers of software on them: operating
Say hello to my little friend: Gorgo. Alas, it just destroyed systems, device drivers, file systems,databases, web servers,
Wilmerding!
email servers, name servers, load balancers, monitors, and
more.

96
  Any initial novelty wore off as you tried to keep multiple
systems alive and responsive.And you worried constantly
about security.
 Many hosting services offered to take care of your servers for
a fee, but you still leasedthe physical devices and had to pay
for your peak load configuration at all times.
 With more individual machines, failures are no longer
infrequent: they’re very common.You need to scale services
horizontally and store data redundantly.
 You can’t assume that the network operates like a single
machine. The eight fallacies of distributed computing,
according to Peter Deutsch, are as follows:
 The network is reliable.
 Latency is zero.
 Bandwidth is infinite.
 The network is secure.
 Topology doesn’t change.
 There is one administrator.
 Transport cost is zero.
 The network is homogeneous
• Instead of building, you can rent servers in the cloud. By
adopting this model, maintenance is someone else’s problem,
and you can concentrate on your service, or blog, or whatever
you want to show the world.
• Using web dashboards and APIs, you can spinup servers with
whatever configuration you need, quickly and easily—they’re
elastic.
• You can monitor their status, and be alerted if some metric
exceeds a given threshold.Clouds are currently a pretty hot
topic, and corporate spending on cloud componentshas
spiked.
Python interacts with some popular clouds.
Google
• Google uses Python a lot internally, and it employs some
prominent Python developers (even Guido van Rossum
himself, for some time).
97
• Go to the App Engine site and then, under ―Choose a
Language,‖ click in the Pythonbox. You can type Python code
into the Cloud Playground and see results just below.
• Just after that are links and directions to download the Python
SDK to your machine.This allows you to develop against
Google’s cloud APIs on your own hardware.
• Google’s main cloud page, you can find details on its services,
including these:
• App Engine : A high-level platform, including Python tools
such as flask and django.
• Compute Engine :Create clusters of virtual machines for large
distributed computing tasks.
• Cloud Storage : Object storage (objects are files, but there are no
directory hierarchies).
• Cloud Datastore :A large NoSQL database.
• Cloud SQL :A large SQL database.
• Cloud Endpoints :Restful access to applications.
• BigQuery :Hadoop-like big data.
• Google services compete with Amazon and OpenStack, a
segue if there ever was one.
Amazon
• As Amazon was growing from hundreds to thousands to
millions of servers, developers ran into all the nasty problems
of distributed systems.
• AmazonWeb Services (AWS), which now dominates the
market. It now contains dozens of services, but the most
relevant are the following:
• Elastic Beanstalk :High-level application platform
• EC2 (Elastic Compute) :Distributed computing
• S3 (Simple Storage Service):Object storage
• RDS :Relational databases (MySQL, PostgreSQL, Oracle,
MSSQL)
• DynamoDB :NoSQL database
• Redshift :Data warehouse
• EMR :Hadoop
OpenStack
• The second most popular cloud service provider has been
Rackspace. In 2010, it formedan unusual partnership with
NASA to merge some of their cloud infrastructure
intoOpenStack.
• This is a freely available open source platform to build public,
private, andhybrid clouds. A new release is made every six
months, the most recent containing over1.25 million lines of
Python from many contributors.
• OpenStack is used in productionby a growing number of
organizations, including CERN and PayPal.
• OpenStack’s main APIs are RESTful, with Python modules
providing programmaticinterfaces, and command-line Python
programs for shell automation. Here are someof the standard
services in the current release:
• Keystone Identity service, providing authentication (for
example, user/password), authorization (capabilities), and
service discovery.
• Nova Compute service, distributing work across networked
servers.
• Swift Object storage, such as Amazon’s S3. It’s used by
Rackspace’s Cloud Files service.
• Glance :Mid-level image storage service.
• Cinder :Low-level block storage service.
• Horizon:Web-based dashboard for all the services.
• Neutron :Network management service.
• Heat :Orchestration (multicloud) service.
• Ceilometer:Telemetry (metrics, monitoring, and metering)
service.
• Other services are proposed from time to time, which then go
through an incubation process and might become part of the
standard OpenStack platform.
• OpenStack runs on Linux or within a Linux virtual machine
(VM). The installation of its core services is still somewhat
involved.
• he fastest way to install OpenStack on Linux is to use
98
Devstack and watch all the explanatory text flying by as it
runs.
ii) Big Fat Data and Map Reduce
• As Google and other Internet companies grew, they found
that traditional computingsolutions didn’t scale.
• Software that worked for single machines, or even a few
dozen,could not keep up with thousands.
• Disk storage for databases and files involved too much
seeking, which requires mechanical movement of disk heads.
• But you could stream consecutive segments of the disk more
quickly.
• Developers found that it was faster to distribute and analyze
data on many networked machines than on individual ones.
They could use algorithms that sounded simplistic, but
actually worked better overall with massively distributed
data.
• One of these is Map‐Reduce, which spreads a calculation
across many machines and then gathers the results.It’s
similar to working with queues.
• After Google published its results in a paper, Yahoo followed
with an open source Javabased package named Hadoop
• The phrase big data applies here. Often it just means ―data too
big to fit on my machine‖:data that exceeds the disk, memory,
CPU time, or all of the above.
• To some organizations,if big data is mentioned somewhere in
aquestion, the answer is always Hadoop.
• Hadoop copies data among machines, running them through
map and reduce programs, and saving the results on disk at
each step.
• This batch process can be slow. A quicker method called
Hadoop streaming works like Unix pipes, streaming the data
through programs without requiring disk writes at each step.
 • You can write Hadoop streaming programs in any language, server = context.socket(zmq.REP)
including Python. server.bind("tcp://%s:%s" % (host, port))
• Many Python modules have been written for Hadoop. while True:
# Wait for next request from client
2) Write a short notes on ZeroMQ request_bytes = server.recv()
• ZeroMQ is a library. Sometimesdescribed as sockets on request_str = request_bytes.decode('utf-
steroids, 8')
• ZeroMQ sockets do the things that you sort of expectedplain print("That voice in my head says: %s" %
sockets to do: request_str)
 Exchange entire messages
 Retry connections reply_str = "Stop saying: %s" %
 Buffer data to preserve it when the timing between request_str
senders and receivers doesn’t line up reply_bytes = bytes(reply_str, 'utf-8')
• ZeroMQ is like a Lego set, construct networks from a few server.send(reply_bytes)
sockettypes and patterns. • We create a Context object: this is a ZeroMQ object that
• The basic ―Lego pieces‖ presented in the following list are maintains state. Then, we make a ZeroMQ socket of type
theZeroMQ socket types REP (for REPly).
 REQ (synchronous request) • We call bind() to make it listen on a particular IP address
 REP (synchronous reply) and port. Notice that they’re specified in a string such as
 DEALER (asynchronous request) 'tcp://localhost:6789' rather than a tuple, as in the plain
 ROUTER (asynchronous reply) socket examples.
 PUB (publish) • This example keeps receiving requests from a sender and
 SUB (subscribe) sending a response. The messages can be very long—
 PUSH (fanout) ZeroMQ takes care of the details
 PULL (fanin) • Following is the code for the corresponding request
• To install the Python ZeroMQ library by typing this (client), zmq_client.py. Its type isREQ (for REQuest), and it
command: calls connect() rather than bind().
$ pip install pyzmq • Start the serverin one window in the background:
• The simplest pattern is a single request-reply pair. This is $ python zmq_server.py &
synchronous: one socket makes a request and then the • Start the client in the same window:
other replies. First, the code for the reply (server), $ python zmq_client.py
zmq_server.py:
import zmq • You’ll see these alternating output lines from the client and
host = '127.0.0.1' server:
port = 6789
context = zmq.Context()
99
 • Our client ends after sending its fifth message, but we didn’t • For example, use DEALER and ROUTER sockets to connect
tell the server to quit, so it sits by the phone, waiting for multiplesources and/or destinations asynchronously.
another message. • Multiple REQ sockets connect to a single ROUTER, which
• If you run the client again, it will printthe same five lines, and passes each request to a DEALER, which then contacts any
the server will print its five also. REP sockets that have connected to it (Figure)
• If you don’t kill the zmq_server.py process and try to run • This is similar to a bunch of browsers contacting a proxy
another one, Python will complain that the address is already server in front of a web server farm. It lets you add multiple
is use: clients and servers as needed.
$ python zmq_server.py &
[2] 356
Traceback (most recent call last):
File "zmq_server.py", line 7, in <module>
server.bind("tcp://%s:%s" % (host, port))
File "socket.pyx", line 444, in zmq.backend.cython.socket.Socket.bind
(zmq/backend/cython/socket.c:4076)
File "checkrc.pxd", line 21, in zmq.backend.cython.checkrc._check_rc
(zmq/backend/cython/socket.c:6032)
zmq.error.ZMQError: Address already in use
• The messages need to be sent as byte strings, so we encoded
our example’s text stringsin UTF-8 format
• We used simple text strings as the source of our messages, so
encode() and decode() were enough to convert to and from
byte strings.
• If your messages have otherdata types, you can use a library
such as MessagePack. • The REQ sockets connect only to the ROUTER socket; the
• Even this basic REQ-REP pattern allows for some fancy DEALER connects to the multiple REP sockets behind it.
communication patterns, because any number of REQ clients • ZeroMQ takes care of the nasty details, ensuring that the
can connect() to a single REP server. requests are load balanced and that the replies go back to the
• The server handles requests one at a time, synchronously, but right place.
doesn’t drop other requests that are arriving in the meantime. • Another networking pattern called the ventilator uses PUSH
• ZeroMQ buffers messages, up to some specified limit, until sockets to farm out asynchronous tasks, and PULL sockets to
they can get through; that’s where it earns the Q in its name. gather the results.
The Q stands for Queue, the M stands for Message, and the • The last notable feature of ZeroMQ is that it scales up and
Zero means there doesn’t need to be any broker. down, just by changing theconnection type of the socket when
it’s created:
• tcp between processes, on one or more machines
100
 • ipc between processes on one machine
• inproc between threads in a single process

3) Explain briefly about Redis

• An another approach to queues that can run on a single
machine or across a network.
• Even with multiple singing processes and dancing threads,
sometimes one machine isn’t enough,
• A Redis server runs on one machine; this can be the same one
as its clients, or another that the clients can access through a
network.
• In either case, clients talk to the server via TCP, so they’re
networking.
• One or more provider clients pushes messages onto one end
of the list.
• One or more client workers watches this list with a blocking
pop operation.
• Process- and thread-based examples, redis_washer.py generates
a sequence of dishes:
• This code waits for messages whose first token is ―dishes‖
and prints that each one isdried. It obeys the quit message
by ending the loop.
• Start the dryer, and then the washer. Using the & at the
end puts the first program in thebackground; it keeps
running, but doesn’t listen to the keyboard anymore.
• This works on Linux, OS X, and Windows, although you
might see different output on the next line.
• we start the washer process normally You’ll see the
mingled
output of the two processes:

• The loop generates four messages containing a dish name,

followed by a final messagethat says ―quit.‖
• It appends each message to a list called dishes in the Redis
server, similarto appending to a Python list.
• And as soon as the first dish is ready, redis_dryer.py does its
work:
• To add a lot more program logic, such as the following:
 Agreeing ahead of time on some maximum dish number,
which would kind of be a sentinel anyway.

101
  Doing some special out-of-band (not in the data stream)
interprocess communication.
 Timing out after some interval with no new data.
Let’s make a few last changes:
• Create multiple dryer processes.
• Add a timeout to each dryer rather than looking for a
sentinel.
The new redis_dryer2.py:

• One dryer process reads the quit ID and quits:

Dryer process 44448 is done
• After 20 seconds, the other dryer processes get a return value
of None from their blpop calls, indicating that they’ve timed
out. They say their last words and exit:
Dryer process 44447 is done
Dryer process 44446 is done
• After the last dryer subprocess quits, the main dryer program
ends:
[1]+ Done python redis_dryer2.py

4) Describe about the Publish–Subscribe network Model

• Start the dryer processes in the background, and then the
washer process in the foreground:
Redis
• Publish-subscribe is not a queue but a broadcast. One or more
processes publish messages.
• Each subscriber process indicates what type of messages it
would like to receive.
• A copy of each message is sent to each subscriber that
matched its type. Thus, a given message might be processed
once, more than once, or not at all.
• Each publisher is justbroadcasting and doesn’t know who—if
anyone—is listening.
• You can build a quick pub-sub system by using Redis.

102
• The publisher emits messages with a topic and a value, and • Next, start the publisher. It will send 10 messages, and then
subscribers say which topics they want to receive. quit:
• Here’s the publisher, redis_pub.py: $ python redis_pub.py
Publish: maine coon wears a stovepipe
import redis Publish: norwegian forest wears a stovepipe
import random Publish: norwegian forest wears a tam-o-shanter
conn = redis.Redis() Publish: maine coon wears a bowler
cats = ['siamese', 'persian', 'maine coon', 'norwegian forest'] Publish: siamese wears a stovepipe
hats = ['stovepipe', 'bowler', 'tam-o-shanter', 'fedora'] Publish: norwegian forest wears a tam-o-shanter
for msg in range(10): Publish: maine coon wears a bowler
cat = random.choice(cats) Publish: persian wears a bowler
hat = random.choice(hats) Publish: norwegian forest wears a bowler
print('Publish: %s wears a %s' % (cat, hat)) Publish: maine coon wears a stovepipe
conn.publish(cat, hat)
• Each topic is a breed of cat, and the accompanying message is • The subscriber cares about only two types of cat:
a type of hat. $ python redis_sub.py
• Here’s a single subscriber, redis_sub.py: Subscribe: maine coon wears a stovepipe
import redis Subscribe: maine coon wears a bowler
conn = redis.Redis() Subscribe: maine coon wears a bowler
topics = ['maine coon', 'persian'] Subscribe: persian wears a bowler
sub = conn.pubsub() Subscribe: maine coon wears a stovepipe
sub.subscribe(topics) • We didn’t tell the subscriber to quit, so it’s still waiting for
for msg in sub.listen(): messages.
if msg['type'] == 'message': • If you restart the publisher, the subscriber will grab a few
cat = msg['channel'] more messages and print them.
hat = msg['data'] • You can have as many subscribers (and publishers) as you
print('Subscribe: %s wears a %s' % (cat, hat)) want. If there’s no subscriber for a message, it disappears
• The subscriber just shown wants all messages for cat types from the Redis server.
'maine coon' and'persian', and no others. • However, if there are subscribers, the messages stay in the
• The listen() method returns a dictionary. If its type is server until all subscribers have retrieved them.
'message', it was sent by the publisher and matches our ZeroMQ
criteria. • ZeroMQ has no central server, so each publisher writes to all
• The 'channel' key isthe topic (cat), and the 'data' key contains subscribers.
the message (hat). • Let’s rewrite the cat-hat pub-sub for ZeroMQ. The publisher,
• If you start the publishe so start the subscriber first: zmq_pub.py, looks like this:
$ python redis_sub.py

103
 import zmq print('Subscribe: %s wears a %s' % (cat, hat))
import random • In this code, we subscribe to two different byte values:
import time the two strings in topics, encoded as UTF-8.
host = '*' • It seems a little backward, but if you want all topics,
port = 6789 you need to subscribe to the empty bytestring b''; if you
ctx = zmq.Context() don’t, you’ll get nothing.
pub = ctx.socket(zmq.PUB) • Notice that we call send_multipart() in the publisher
pub.bind('tcp://%s:%s' % (host, port)) and recv_multipart() in the
cats = ['siamese', 'persian', 'maine coon', subscriber.
'norwegian forest'] • This makes it possible for us to send multipart messages, and
hats = ['stovepipe', 'bowler', 'tam-o-shanter', use the firstpart as the topic.
'fedora'] • We could also send the topic and message as a single string or
time.sleep(1) bytestring, but it seems cleaner to keep cats and hats separate.
for msg in range(10): • Start the subscriber:
cat = random.choice(cats)
$ python zmq_sub.py
cat_bytes = cat.encode('utf-8')
• Start the publisher. It immediately sends 10 messages, and
hat = random.choice(hats)
then quits:
hat_bytes = hat.encode('utf-8')
print('Publish: %s wears a %s' % (cat, hat)) $ python zmq_pub.py
pub.send_multipart([cat_bytes, hat_bytes])
Publish: norwegian forest wears a stovepipe
• Notice how this code uses UTF-8 encoding for the topic
Publish: siamese wears a bowler
and value strings. Publish: persian wears a stovepipe
• The file for the subscriber is zmq_sub.py: Publish: norwegian forest wears a fedora
import zmq Publish: maine coon wears a tam-o-shanter
host = '127.0.0.1' Publish: maine coon wears a stovepipe
port = 6789 Publish: persian wears a stovepipe
ctx = zmq.Context() Publish: norwegian forest wears a fedora
sub = ctx.socket(zmq.SUB)
Publish: norwegian forest wears a bowler
sub.connect('tcp://%s:%s' % (host, port)) Publish: maine coon wears a bowler
topics = ['maine coon', 'persian']
for topic in topics:
• The subscriber prints what it requested and received:
sub.setsockopt(zmq.SUBSCRIBE, •
topic.encode('utf-8'))
Subscribe: persian wears a stovepipe
while True: Subscribe: maine coon wears a tam-o-shanter
cat_bytes, hat_bytes = sub.recv_multipart()
Subscribe: maine coon wears a stovepipe
cat = cat_bytes.decode('utf-8')
Subscribe: persian wears a stovepipe
hat = hat_bytes.decode('utf-8')
104
 Subscribe: maine coon wears a bowler for job in jobs:
print(job.value)
Other Pub-sub Tools • There’s a one-line for-loop in the preceding example. Each
RabbitMQ :This is a well-known messaging broker, and pika is a hostname is submitted in turn to a gethostbyname() call, but
Python API they can run asynchronously because it’s the gevent version
pypi.python.org: Go to the upper-right corner of the search window of gethostbyname().
and type pubsub to find Pythonpackages like pypubsub. • Run gevent_test.py with Python 2 by typing the following (in
Pubsubhubbub :This mellifluous protocol enables subscribers to bold):
register callbacks with publishers. $ python2 gevent_test.py
66.6.44.4
5) Explain the following 74.125.142.121
i) Green Thread and gevent 78.136.12.50
ii) twisted • gevent.spawn() creates a greenlet (a green thread or a
i) Green Thread and gevent : microthread) to execute each
• The gevent library is event-based and accomplishes a cool gevent.socket.gethostbyname(url).
trick: you write normal imperative code, and it magically • The difference from a normal thread is that it doesn’t block. If
converts pieces to coroutines. something occurred thatwould have blocked a normal thread,
• These are like generators that can communicate with one gevent switches control to one of the other greenlets.
another and keep track of where they are. gevent modifies • The gevent.joinall() method waits for all the spawned jobs to
many of Python’s standard objects such as socket to use its finish.
mechanism insteadof blocking. • Finally, we dump the IP addresses that we got for these
• This does not work with Python add-in code that was written hostnamesInstead of the gevent version of socket, you can use
in C, as some database drivers are. its evocatively named monkeypatching functions.
• You install gevent by using the Python 2 version of pip: • These modify standard modules such as socket to use
$ pip2 install gevent greenlets rather than calling the gevent version of the module.
• gethostbyname() function is synchronous,so you wait while it • At the top of your program, add the following call:
chases name servers around the world tolook up that address. from gevent import monkey
• Save this as gevent_test.py: monkey.patch_socket()
import gevent • This inserts the gevent socket everywhere the normal socket is
from gevent import socket called, anywhere inyour program, even in the standard
hosts = ['www.crappytaxidermy.com', library.
'www.walterpottertaxidermy.com', • Again, this works only for Python code, not libraries written
'www.antique-taxidermy.com'] in C.
jobs = [gevent.spawn(gevent.socket.gethostbyname, host) for host • Another function monkey-patches even more standard library
in hosts] modules:
gevent.joinall(jobs, timeout=5) from gevent import monkey
105
 monkey.patch_all() $ pip2 install twisted
• Use this at the top of your program to get as many gevent • twisted is a large package, with support for many Internet
speedups as possible.Save this program as gevent_monkey.py: protocols on top of TCP andUDP. To be short and simple,
import gevent server, knock_server.py (notice the Python2 syntax for print()):
from gevent import monkey; monkey.patch_all() from twisted.internet import protocol, reactor
import socket class Knock(protocol.Protocol):
hosts = ['www.crappytaxidermy.com', def dataReceived(self, data):
'www.walterpottertaxidermy.com', print 'Client:', data
'www.antique-taxidermy.com'] if data.startswith("Knock knock"):
jobs = [gevent.spawn(socket.gethostbyname, host) for host in response = "Who's there?"
hosts] else:
gevent.joinall(jobs, timeout=5) response = data + " who?"
for job in jobs: print 'Server:', response
print(job.value) self.transport.write(response)
• Again, using Python 2, run the program: class KnockFactory(protocol.Factory):
$ python2 gevent_monkey.py def buildProtocol(self, addr):
66.6.44.4 return Knock()
74.125.192.121 reactor.listenTCP(8000, KnockFactory())
reactor.run()
78.136.12.50
• There are potential dangers when using gevent. As with any • Now, let’s take a glance at its trusty companion,
event-based system, each chunk of code that you execute knock_client.py:
should be relatively quick. from twisted.internet import reactor, protocol
class KnockClient(protocol.Protocol):
• Although it’s non blocking,code that does a lot of work is still
def connectionMade(self):
slow.
self.transport.write("Knock knock")
ii) twisted
• twisted is an asynchronous, event-driven networking def dataReceived(self, data):
if data.startswith("Who's there?"):
framework.
response = "Disappearing client"
• To connect functions to events such as data received or
self.transport.write(response)
connection closed, and those functions are called when those
else:
events occur.
self.transport.loseConnection()
• This is a callback design, and if you’ve written anything in
reactor.stop()
JavaScript, it might seem familiar. class KnockFactory(protocol.ClientFactory):
• For somedevelopers, callback-based code becomes harder to protocol = KnockClient
manage as the application grows.
def main():
• Like gevent, twisted has not yet been ported to Python 3.. f = KnockFactory()
Type the following to install it:
106
 reactor.connectTCP("localhost", 8000, f)  Suppose that you’re washing dishes. If you’re stuck with the
reactor.run() entire job, you need to wash each dish, dry it, and put it away.
if name == ' main ': You can do this in a number of ways.
main()  You might wash the first dish, dry it, and then put it away.
• Start the server first:  You then repeat with the second dish, and soon.
$ python2 knock_server.py  Or, you might batch operations and wash all the dishes, dry
• Then start the client: them all, and then put them away;
$ python2 knock_client.py  In general, queues transport messages, distributed task
• The server and client exchange messages, and the server management, also known as workqueues, job queues, or task
prints the conversation: queues.
Client: Knock knock  This can be synchronous (workers wait for a dish to handle
Server: Who's there? and another worker to whom to give it),
Client: Disappearing client  Asynchronous (dishes are stacked between workers with
Server: Disappearing client who? different paces)
• Our trickster client then ends, keeping the server waiting for Process
the punch line.  You can implement queues in many ways. For a single
machine, the standard library’s multiprocessing
Section –B(Ten marks) modulecontains a Queue function.
1) How to achieve Concurrency in Python programming  Let’s simulate just a single washer and multiple dryer
• In computers, if you’re waiting for something, it’s usually for processes (someone can put the dishes away later) and an
one of two reasons: intermediate dish_queue.
 I/O bound :  Call this program dishes.py:
This is by far more common. Computer CPUs are ridiculously
fast—hundreds of times faster than computer memory and
many thousands of times faster than disksor networks.
 CPU bound :This happens with number crunching tasks such as
scientific or graphic calculations.
• Two more terms are related to concurrency:
Synchronous :One thing follows the other, like a funeral
procession.
Asynchronous :Tasks are independent, like party-goers
dropping in and tearing off in separate cars.
Queues
 A queue is like a list: things are added at one end and taken
away from the other. Themost common is referred to as FIFO
(first in, first out).

107
  Run your new program  Here prints

 This queue looked a lot like a simple Python iterator,  Reproduce our process-based dish example by using
producing a series of dishes. threads:
 Itactually started up separate processes along with the
communication between the washerand dryer.
 Using JoinableQueue and the final join() method to let the
washer know that all the dishes have been dried.
Threads
 A thread runs within a process with access to everything in the
process, similar to a multiple personality.
 The multiprocessing module has a cousin called threading
that uses threads instead of processes .Example with threads:

108
 • This does not work with Python add-in code that was written
in C, as some database drivers are.
• You install gevent by using the Python 2 version of pip:
$ pip2 install gevent
• gethostbyname() function is synchronous,so you wait while it
 One difference between multiprocessing and threading is chases name servers around the world tolook up that address.
that threading does not have a terminate() function. • Save this as gevent_test.py:
 There’s no easy way to terminate a running thread, import gevent
because it can cause all sorts of problems in your code, and from gevent import socket
possibly in the space-time continuum itself. hosts = ['www.crappytaxidermy.com',
 Threads can be dangerous, To use threads, all the code in 'www.walterpottertaxidermy.com',
the program—and in external libraries that it uses—must 'www.antique-taxidermy.com']
be threadsafe jobs = [gevent.spawn(gevent.socket.gethostbyname, host) for host
 Threads can be useful and safe when global data is not in hosts]
involved. In particular, threads are useful for saving time gevent.joinall(jobs, timeout=5)
while waiting for some I/O operation to complete. for job in jobs:
 In these cases, they don’t have to fight over data, because print(job.value)
each has completely separate variables. • There’s a one-line for-loop in the preceding example. Each
 But threads do sometimes have good reasons to change hostname is submitted in turn to a gethostbyname() call, but
global data. they can run asynchronously because it’s the gevent version
 In fact, one common reason to launch multiple threads is of gethostbyname().
to let them divide up the work on some data, so a certain • Run gevent_test.py with Python 2 by typing the following (in
degree of change to the data is expected bold):
 So for Python, the recommendations are as follows: $ python2 gevent_test.py
• Use threads for I/O bound problems 66.6.44.4
• Use processes, networking, or events for 74.125.142.121
CPU- bound problems 78.136.12.50
• gevent.spawn() creates a greenlet (a green thread or a
Green Thread and gevent : microthread) to execute each
• The gevent library is event-based and accomplishes a cool gevent.socket.gethostbyname(url).
trick: you write normal imperative code, and it magically • The difference from a normal thread is that it doesn’t block. If
something occurred thatwould have blocked a normal thread,
converts pieces to coroutines.
• These are like generators that can communicate with one gevent switches control to one of the other greenlets.
another and keep track of where they are. gevent modifies • The gevent.joinall() method waits for all the spawned jobs to
many of Python’s standard objects such as socket to use its finish.
mechanism insteadof blocking.
109
• Finally, we dump the IP addresses that we got for these • There are potential dangers when using gevent. As with any
hostnamesInstead of the gevent version of socket, you can use event-based system, each chunk of code that you execute
its evocatively named monkeypatching functions. should be relatively quick.
• These modify standard modules such as socket to use • Although it’s non blocking,code that does a lot of work is still
greenlets rather than calling the gevent version of the module. slow.
• At the top of your program, add the following call: twisted
from gevent import monkey • twisted is an asynchronous, event-driven networking
monkey.patch_socket() framework.
• This inserts the gevent socket everywhere the normal socket is • To connect functions to events such as data received or
called, anywhere inyour program, even in the standard connection closed, and those functions are called when those
library. events occur.
• Again, this works only for Python code, not libraries written • This is a callback design, and if you’ve written anything in
in C. JavaScript, it might seem familiar.
• Another function monkey-patches even more standard library • For somedevelopers, callback-based code becomes harder to
modules: manage as the application grows.
from gevent import monkey • Like gevent, twisted has not yet been ported to Python 3..
monkey.patch_all() Type the following to install it:
• Use this at the top of your program to get as many gevent $ pip2 install twisted
speedups as possible.Save this program as gevent_monkey.py: • twisted is a large package, with support for many Internet
import gevent protocols on top of TCP andUDP. To be short and simple,
from gevent import monkey; monkey.patch_all() server, knock_server.py (notice the Python2 syntax for print()):
import socket from twisted.internet import protocol, reactor
hosts = ['www.crappytaxidermy.com', class Knock(protocol.Protocol):
'www.walterpottertaxidermy.com', def dataReceived(self, data):
'www.antique-taxidermy.com'] print 'Client:', data
jobs = [gevent.spawn(socket.gethostbyname, host) for host in if data.startswith("Knock knock"):
hosts] response = "Who's there?"
gevent.joinall(jobs, timeout=5) else:
for job in jobs: response = data + " who?"
print(job.value) print 'Server:', response
• Again, using Python 2, run the program: self.transport.write(response)
$ python2 gevent_monkey.py class KnockFactory(protocol.Factory):
66.6.44.4 def buildProtocol(self, addr):
74.125.192.121 return Knock()
78.136.12.50 reactor.listenTCP(8000, KnockFactory())
reactor.run()

110
• Now, let’s take a glance at its trusty companion, • Even with multiple singing processes and dancing threads,
knock_client.py: sometimes one machine isn’t enough,
from twisted.internet import reactor, protocol Redis
class KnockClient(protocol.Protocol): • A Redis server runs on one machine; this can be the same one
def connectionMade(self): as its clients, or another that the clients can access through a
self.transport.write("Knock knock") network.
def dataReceived(self, data): • In either case, clients talk to the server via TCP, so they’re
if data.startswith("Who's there?"): networking.
response = "Disappearing client" • One or more provider clients pushes messages onto one end
self.transport.write(response) of the list. One or more client workers watches this list with a
else: blocking pop operation.
self.transport.loseConnection() • Process- and thread-based examples, redis_washer.py generates
reactor.stop() a sequence of dishes:
class KnockFactory(protocol.ClientFactory):
protocol = KnockClient
def main():
f = KnockFactory()
reactor.connectTCP("localhost", 8000, f)
reactor.run()
if name == ' main ':
main()
• Start the server first:
$ python2 knock_server.py
• The loop generates four messages containing a dish name,
• Then start the client: followed by a final messagethat says ―quit.‖
$ python2 knock_client.py • It appends each message to a list called dishes in the Redis
• The server and client exchange messages, and the server server, similarto appending to a Python list.
prints the conversation: Beyond Queues :
Client: Knock knock  Some techniques available that you can apply. They include
Server: Who's there? the following:
Client: Disappearing client  Fire and forget :Just pass things on and don’t worry about the
Server: Disappearing client who? consequences, even if no one is there.That’s the dishes-on-the-
• Our trickster client then ends, keeping the server waiting for
floor approach.
the punch line.
 Request-reply :The washer receives an acknowledgement from
• An another approach to queues that can run on a single
the dryer, and the dryer from the put-away-er, for each dish
machine or across a network.
in the pipeline.

111
  Back pressure or throttling :This technique directs a fast worker • Publish-subscribe is not a queue but a broadcast. One or more
to take it easy if someone downstream can’t keep up. processes publish messages.
 Some Python-based queue packages that add this extra level • Each subscriber process indicates what type of messages it
of management—some of which use Redis—include: would like to receive.
 Celery :This particular package is well worth a look. It can • A copy of each message is sent to each subscriber that
execute distributed tasks synchronously or asynchronously matched its type. Thus, a given message might be processed
 Thoonk :This package builds on Redis to provide job queues once, more than once, or not at all.
and pub-sub • Each publisher is justbroadcasting and doesn’t know who—if
 Rq :This is a Python library for job queues, also based on anyone—is listening.
Redis. Redis
 Queues :This site offers a discussion of queuing software, • You can build a quick pub-sub system by using Redis.
Python-based and otherwise. • The publisher emits messages with a topic and a value, and
subscribers say which topics they want to receive.
2) Explain about Networking in Python • Here’s the publisher, redis_pub.py:
Network Patterns:
 The most common pattern is request-reply, also known as import redis
client-server. This pattern is synchronous: the client waits import random
until the server responds. conn = redis.Redis()
 Another common pattern is push, or fanout: you send data cats = ['siamese', 'persian', 'maine coon', 'norwegian forest']
to any available worker in a pool of processes. An example hats = ['stovepipe', 'bowler', 'tam-o-shanter', 'fedora']
is a web server behind a load balancer. for msg in range(10):
 The opposite of push is pull, or fanin: you accept data from cat = random.choice(cats)
one or more sources. Anexample would be a logger that hat = random.choice(hats)
takes text messages from multiple processes and writes print('Publish: %s wears a %s' % (cat, hat))
them to a single log file. conn.publish(cat, hat)
 One pattern is similar to radio or television broadcasting: • Each topic is a breed of cat, and the accompanying message is
publish-subscribe, or pub-sub. With this pattern, a publisher a type of hat.
sends out data. • Here’s a single subscriber, redis_sub.py:
 In a simple pub-sub system, all subscribers would receive import redis
a copy. More often, subscribers can indicate that they’re conn = redis.Redis()
interested onlyin certain types of data (often called a topic), topics = ['maine coon', 'persian']
sub = conn.pubsub()
and the publisher will send just those.
sub.subscribe(topics)
 So,unlike the push pattern, more than one subscriber
might receive a given piece of data. If there’s no subscriber for msg in sub.listen():
if msg['type'] == 'message':
for a topic, the data is ignored.
cat = msg['channel']
Publish-Subscribe Network Model :
112
 hat = msg['data'] • You can have as many subscribers (and publishers) as you
print('Subscribe: %s wears a %s' % (cat, hat)) want. If there’s no subscriber for a message, it disappears
• The subscriber just shown wants all messages for cat types from the Redis server.
'maine coon' and'persian', and no others. • However, if there are subscribers, the messages stay in the
• The listen() method returns a dictionary. If its type is server until all subscribers have retrieved them.
'message', it was sent by the publisher and matches our ZeroMQ
criteria. • ZeroMQ has no central server, so each publisher writes to all
• The 'channel' key isthe topic (cat), and the 'data' key contains subscribers.
the message (hat). • Let’s rewrite the cat-hat pub-sub for ZeroMQ. The publisher,
• If you start the publishe so start the subscriber first: zmq_pub.py, looks like this:
$ python redis_sub.py import zmq
• Next, start the publisher. It will send 10 messages, and then import random
quit: import time
$ python redis_pub.py host = '*'
Publish: maine coon wears a stovepipe port = 6789
Publish: norwegian forest wears a stovepipe ctx = zmq.Context()
Publish: norwegian forest wears a tam-o-shanter pub = ctx.socket(zmq.PUB)
Publish: maine coon wears a bowler pub.bind('tcp://%s:%s' % (host, port))
Publish: siamese wears a stovepipe cats = ['siamese', 'persian', 'maine coon',
Publish: norwegian forest wears a tam-o-shanter 'norwegian forest']
Publish: maine coon wears a bowler hats = ['stovepipe', 'bowler', 'tam-o-shanter',
Publish: persian wears a bowler 'fedora']
Publish: norwegian forest wears a bowler time.sleep(1)
Publish: maine coon wears a stovepipe for msg in range(10):
cat = random.choice(cats)
• The subscriber cares about only two types of cat: cat_bytes = cat.encode('utf-8')
$ python redis_sub.py hat = random.choice(hats)
Subscribe: maine coon wears a stovepipe hat_bytes = hat.encode('utf-8')
Subscribe: maine coon wears a bowler print('Publish: %s wears a %s' % (cat, hat))
Subscribe: maine coon wears a bowler pub.send_multipart([cat_bytes, hat_bytes])
Subscribe: persian wears a bowler • Notice how this code uses UTF-8 encoding for the topic
Subscribe: maine coon wears a stovepipe and value strings.
• We didn’t tell the subscriber to quit, so it’s still waiting for • The file for the subscriber is zmq_sub.py:
messages. import zmq
• If you restart the publisher, the subscriber will grab a few host = '127.0.0.1'
more messages and print them. port = 6789

113
 ctx = zmq.Context()
sub = ctx.socket(zmq.SUB)
sub.connect('tcp://%s:%s' % (host, port))
topics = ['maine coon', 'persian']
for topic in topics:
sub.setsockopt(zmq.SUBSCRIBE,
topic.encode('utf-8'))
while True:
cat_bytes, hat_bytes = sub.recv_multipart()
cat = cat_bytes.decode('utf-8')
hat = hat_bytes.decode('utf-8')
print('Subscribe: %s wears a %s' % (cat, hat))
• In this code, we subscribe to two different byte values:
the two strings in topics, encoded as UTF-8.
• It seems a little backward, but if you want all topics,
you need to subscribe to the empty bytestring b''; if you
don’t, you’ll get nothing.
• Call send_multipart() in the publisher and
recv_multipart() in the subscriber.
• This makes it possible for us to send multipart messages, and
use the firstpart as the topic.
• We could also send the topic and message as a single string or
bytestring, but it seems cleaner to keep cats and hats separate.
• Start the subscriber:
$ python zmq_sub.py
• Start the publisher. It immediately sends 10 messages, and
then quits:
$ python zmq_pub.py
Publish: norwegian forest wears a stovepipe
Publish: siamese wears a bowler
Publish: persian wears a stovepipe
Publish: norwegian forest wears a fedora
Publish: maine coon wears a tam-o-shanter
Publish: maine coon wears a stovepipe
Publish: persian wears a stovepipe
114
Publish: norwegian forest wears a fedora
Publish: norwegian forest wears a bowler
Publish: maine coon wears a bowler
• The subscriber prints what it requested and received:
Subscribe: persian wears a stovepipe
Subscribe: maine coon wears a tam-o-shanter
Subscribe: maine coon wears a stovepipe
Subscribe: persian wears a stovepipe
Subscribe: maine coon wears a bowler
Other Pub-sub Tools
RabbitMQ :This is a well-known messaging broker, and pika is a
Python API
pypi.python.org: Go to the upper-right corner of the search window
and type pubsub to find Pythonpackages like pypubsub.
Pubsubhubbub :This mellifluous protocol enables subscribers to
register callbacks with publishers.
TCP/IP :
 The Internet is based on rules about how to make connections,
exchange data, terminate connections, handle timeouts, and
so on. These are called protocols, and they are arranged in
layers.
 The IP (Internet Protocol) layer, which specifies how network
locations are addressed and how packets (chunks) of data flow.
 In the layer above that, two protocols describe how to move
bytes between locations:
 UDP (User Datagram Protocol): This is used for short
exchanges. A datagram is a tiny message sent in a single
burst,like a note on a postcard.
 TCP (Transmission Control Protocol) :This protocol is used
for longer-lived connections. It sends streams of bytes and
ensures that they arrive in order without duplication
 UDP messages are not acknowledged, so you’re never sure if
they arrive at their destination.If you wanted to tell a joke
over UDP:
Here's a UDP joke. Get it?
 TCP sets up a secret handshake between sender and receiver
to ensure a good connection.A TCP joke would start like this:
Do you want to hear a TCP joke?
Yes, I want to hear a TCP joke.
Okay, I'll tell you a TCP joke.
Okay, I'll hear a TCP joke.
Okay, I'll send you a TCP joke now.
Okay, I'll receive the TCP joke now.
... (and so on)
 Your local machine always has the IP address 127.0.0.1 and
the name localhost. Youmight see this called the loopback
interface. If it’s connected to the Internet, your machine will
also have a public
Sockets :
 The lowest level of network programming uses a socket,
borrowed from the C language and the Unix operating
system. Socket-level coding is tedious.
 For instance,messages about sockets often turn up when
networking errors take place.
 Let’s write a very simple client-server exchange. The client
sends a string in a UDP datagram to a server, and the server
returns a packet of data containing a string.
 The server needs to listen at a particular address and port—
like a post office and a post office box. The client needs to
know these two values to deliver its message, and receive any
reply.
 In the following client and server code, address is a tuple of
(address, port). The address is a string, which can be a name or
an IP address.
115
 When your programs are just talking to one another on the
same machine, you can use the name 'localhost' or the
equivalent address '127.0.0.1'.
 Here’s the first program, udp_server.py:
from datetime import datetime
import socket
server_address = ('localhost', 6789)
max_size = 4096
Networks | 281
www.it-ebooks.info
print('Starting the server at', datetime.now())
print('Waiting for a client to call.')
server = socket.socket(socket.AF_INET,
socket.SOCK_DGRAM)
server.bind(server_address)
data, client = server.recvfrom(max_size)
print('At', datetime.now(), client, 'said', data)
server.sendto(b'Are you talking to me?', client)
server.close()
 The server has to set up networking through two methods
imported from the socket package.
 The first method, socket.socket, creates a socket, and the
second, bind, binds to it (listens to any data arriving at that IP
address and port). AF_INET means we’ll create an Internet
(IP) socket.
 SOCK_DGRAM means we’ll send and receive datagrams—in
other words, we’ll use UDP.
 At this point, the server sits and waits for a datagram to come
in (recvfrom). When one arrives, the server wakes up and gets
both the data and information about the client.
 The client variable contains the address and port combination
needed to reach the client. The server ends by sending a reply
and closing its connection.
 Let’s take a look at udp_client.py:  there were any problems, it tries to send it again. Let’s
shoot a few packets from client to server and back with TCP.
import socket  tcp_client.py acts like the previous UDP client, sending only
from datetime import datetime one string to the server, but there are small differences in
server_address = ('localhost', 6789) the socket calls, illustrated here:
max_size = 4096 import socket
print('Starting the client at', datetime.now()) from datetime import
client = socket.socket(socket.AF_INET, datetime address = ('localhost',
socket.SOCK_DGRAM) 6789) max_size = 1000
client.sendto(b'Hey!', server_address) print('Starting the client at',
data, server = client.recvfrom(max_size) datetime.now()) client =
print('At', datetime.now(), server, 'said', data) socket.socket(socket.AF_INET,
client.close() socket.SOCK_STREAM)
 The client has most of the same methods as the server (with client.connect(address)
the exception of bind()). The client sends and then receives, client.sendall(b'Hey!')
whereas the server receives first. data = client.recv(max_size)
 Start the server first, in its own window. It will print its print('At', datetime.now(), 'someone replied', data)
greeting and then wait a client sends it some data: client.close()
$ python udp_server.py  We’ve replaced SOCK_DGRAM with SOCK_STREAM to get
 Starting the server at 2014-02-05 21:17:41.945649 the streaming protocol, TCP. We also added a connect() call
Waiting for a client to call. to set up the stream.
 Next, start the client in another window. It will print its  We didn’t need that for UDP because each datagram was on
greeting, send data to the server,print the reply, and then exit: its own in the wild, wooly Internet.tcp_server.py also differs
$ python udp_client.py from its UDP cousin:
Starting the client at 2014-02-05 21:24:56.509682 from datetime import datetime
At 2014-02-05 21:24:56.518670 ('127.0.0.1', 6789) import socket
said b'Are you talking to me?' address = ('localhost', 6789)
 Finally, the server will print something like this, and then exit: At max_size = 1000
2014-02-05 21:24:56.518473 ('127.0.0.1', 56267) said b'Hey!' Networks | 283
 The client needed to know the server’s address and port www.it-ebooks.info
number but didn’t need to specify a port number for itself. print('Starting the server at', datetime.now())
That was automatically assigned by the system—in this case, print('Waiting for a client to call.')
server =
it was 56267.
socket.socket(socket.AF_INET,
 TCP is used for longer-lived connections, such as the Web.
socket.SOCK_STREAM)
TCP delivers data in the order in which you send it.
server.bind(address)
116
server.listen(5)
client, addr = server.accept()

117
 data = client.recv(max_size)
print('At', datetime.now(), client, 'said', data)
client.sendall(b'Are you talking to me?')
client.close()
server.close()
 server.listen(5) is configured to queue up to five client
connections before refusing new ones. server.accept() gets the
first available message as it arrives.
 The client.recv(1000) sets a maximum acceptable message
length of 1,000 bytes. start the server and then the client, and
watch the fun. First, theserver:
$ python tcp_server.py
Starting the server at 2014-02-06 22:45:13.306971
Waiting for a client to call.
At 2014-02-06 22:45:16.048865 <socket.socket object,
fd=6, family=2, type=1,
proto=0> said b'Hey!'
 Now, start the client. It will send its message to the server,
receive a response, and then exit:
$ python tcp_client.py
Starting the client at 2014-02-06 22:45:16.038642
At 2014-02-06 22:45:16.049078 someone replied b'Are
you talking to me?'
 The server collects the message, prints it, responds, and then
quits:
At 2014-02-06 22:45:16.048865 <socket.socket object,
fd=6, family=2, type=1,
proto=0> said b'Hey!'
 Notice that the TCP server called client.sendall() to respond,
and the earlier UDP server called client.sendto(). TCP
maintains the client-server connection across multiple socket
calls and remembers the client’s IP address.
• UDP sends messages, but their size is limited, and they’re not
guaranteed to reach their destination.
118
• TCP sends streams of bytes, not messages. You don’t know
how many bytes the system will send or receive with each call.
• To exchange entire messages with TCP, you need some extra
information to reassemble the full message from its segments: a
fixed message size (bytes), or the size of the full message, or some
delimiting character.
• Because messages are bytes, not Unicode text strings,
Internet Services :
 Python has an extensive networking toolset.
Domain Name System
 Computers have numeric IP addresses such as 85.2.101.94, but
we remember names
 better than numbers. The Domain Name System (DNS) is a
critical Internet service that
 converts IP addresses to and from names via a distributed
database
 Some DNS functions are found in the low-level socket
module. gethostbyname()returns the IP address for a domain
name, and the extended edition gethostbyname_ex() returns
the name, a list of alternative names, and a list of addresses:
>>> import socket
>>> socket.gethostbyname('www.crappytaxidermy.com')
'66.6.44.4'
>>> socket.gethostbyname_ex('www.crappytaxidermy.com')
('crappytaxidermy.com', ['www.crappytaxidermy.com'],
['66.6.44.4'])
 The getaddrinfo() method looks up the IP address, but it also
returns enough information to create a socket to connect to it:
>>> socket.getaddrinfo('www.crappytaxidermy.com', 80)
[(2, 2, 17, '', ('66.6.44.4', 80)), (2, 1, 6, '', ('66.6.44.4', 80))]
 The preceding call returned two tuples, the first for UDP, and
the second for TCP (the6 in the 2, 1, 6 is the value for TCP).
 You can ask for TCP or UDP information only:
>>> socket.getaddrinfo('www.crappytaxidermy.com',
80, socket.AF_INET,
 socket.SOCK_STREAM) www.it-ebooks.info
[(2, 1, 6, '', ('66.6.44.4', 80))] data = response.json()
 Some TCP and UDP port numbers are reserved for certain for video in data['feed']['entry'][0:6]:
services by IANA, and areassociated with service names. For print(video['title']['$t'])
example, HTTP is named http and is assigned TCP port 80.  APIs are especially useful for mining well-known social
 These functions convert between service names and port media sites such as Twitter,Facebook, and LinkedIn.
numbers:  All these sites provide APIs that are free to use, but they
>>> import socket require you to register and get a key to use when connecting.
>>> socket.getservbyname('http')  The key lets a site determine who’s accessing its data. It can
80 also serve as a way to limit request traffic to servers.
>>> socket.getservbyport(80)  The YouTube example you just looked at did not require an
'http' API key for searching, but it would if you made calls that
Python Email Modules updated data at YouTube.
 The standard library contains these email modules:  Here are some interesting service APIs:
 smtplib for sending email messages via Simple Mail  New York Times
Transfer Protocol (SMTP)  YouTube
 email for creating and parsing email messages  Twitter
 poplib for reading email via Post Office Protocol 3  Facebook
(POP3)  Weather Underground
 imaplib for reading email via Internet Message Access  Marvel Comics
Protocol (IMAP) Remote Procedure Call :
Other protocols  Remote Procedure Calls (RPCs) look like normal functions but
 Using the standard ftplib module, you can push bytes around execute on remote machinesacross a network.
by using the File TransferProtocol (FTP).  Instead of calling a RESTful API with arguments encoded in
Web Services and APIs :  the URL or request body, you call an RPC function on your own
 The easiest API is a web interface, but one that provides data machine.
in a structured format such as JSON or XML rather than plain  RPC client:
text or HTML. o It converts your function arguments into bytes (sometimes
 The API might be minimalor a full-fledged RESTful API. this is called marshalling,or serializing, or just encoding).
 Read about web requests, JSON, dictionaries, lists, and slices: o It sends the encoded bytes to the remote machine.
import requests  Remote machine:
url = "https://gdata.youtube.com/feeds/api/ 1. It receives the encoded request bytes.
standardfeeds/top_rated?alt=json" 2. After receiving the bytes, the RPC client decodes the bytes
response = requests.get(url) back to the original data structures
Networks | 291

119
 3. The client then finds and calls the local function with the
decoded data.
4. Next, it encodes the function results.
5. Last, the client sends the encoded bytes back to the caller.
And finally, the machine that started it all decodes the bytes
to return values.
 RPC implementation that uses XML as the exchangeformat:
xmlrpc. You define and register functions on the server, and
the client calls themas though they were imported.
 First, let’s explore the file xmlrpc_server.py:
from xmlrpc.server import SimpleXMLRPCServer
def double(num):
return num * 2
server = SimpleXMLRPCServer(("localhost", 6789))
server.register_function(double, "double")
server.serve_forever()
 The function we’re providing on the server is called double().
It expects a number as an argument and returns the value of
that number times two.
 The server starts up onan address and port. We need to
register the function to make it available to clients viaRPC.
Finally, start serving and carry on., xmlrpc_client.py:
import xmlrpc.client
proxy =
xmlrpc.client.ServerProxy("http://localhost:6789/")
num = 7
result = proxy.double(num)
print("Double %s is %s" % (num, result))
 The client connects to the server by using ServerProxy(). Then,
it calls the functionproxy.double().
120
 The RPC machinery magically hooks this function name into a
call to the remote server.
 Start the server and then run the client:
$ python xmlrpc_server.py
 Next, run the client:
$ python xmlrpc_client.py
Double 7 is 14
 The server then prints the following:
127.0.0.1 - - [13/Feb/2014 20:16:23] "POST / HTTP/1.1" 200 -
 Popular transport methods are HTTP and ZeroMQ. Common
encodings besides XMLinclude JSON, Protocol Buffers, and
MessagePack.
 There are many Python packages for JSON-based RPC, but
many of them either don’t support Python 3
 Here’s how to install it: Python RPC
$ pip install msgpack-rpc-python
 This will also install tornado, a Python event-based web
server that this library uses asa transport. As usual, the server
comes first (msgpack_server.py):
from msgpackrpc import Server, Address
class Services():
def double(self, num):
return num * 2
server = Server(Services())
server.listen(Address("localhost", 6789))
server.start()
 The Services class exposes its methods as RPC services. Go
ahead and start the client,msgpack_client.py:
from msgpackrpc import Client, Address
client = Client(Address("localhost", 6789))
num = 8
result = client.call('double', num)
print("Double %s is %s" % (num, result))
 To run these, follow the usual drill: start the server, start the
client, see the results:
 $ python msgpack_server.py $ fab -f fab2.py -H localhost iso
$ python msgpack_client.py [localhost] Executing task 'iso'
Double 8 is 16 [localhost] local: date -u
fabric : Sun Feb 23 05:22:33 UTC 2014
 The fabric package lets you run remote or local commands, Done.
upload or download files,and run as a privileged user with Disconnecting from localhost... done.
sudo. The package uses Secure Shellto run programs on  The remote counterpart of local() is run(). Here’s fab3.py:
remote machines. from fabric.api import run
 You write functions (in Python) in a so-called fabric file and def iso():
indicate if they should be run locally or remotely. run('date -u')
 When you run these with the fabric program (called Salt :
fab)  Salt started as a way to implement remote execution, but it
 First, install fabric by typing the following: grew to a full-fledged systems management platform.
$ pip2 install fabric  Based on ZeroMQ rather than SSH, it can scale to thousands
 You can run Python code locally from a fabric file directly of servers.
without SSH. Save this first file as fab1.py:  Salt has not yet been ported to Python 3., Alternative products
def iso(): include puppet and chef, which are closely tiedto Ruby.
from datetime import date  The ansible package, which like Salt is written in Python,is
print(date.today().isoformat()) also comparable.
 Now, type the following to run it:  salt and ansible are both functional supersets of fabric,
$ fab -f fab1.py -H localhost iso handlinginitial configuration, deployment, and
[localhost] Executing task 'iso' remote execution.
2014-02-22 Python interacts with some popular clouds :
Done. Google
 The -f fab1.py option specifies to use fabric file fab1.py instead • Google uses Python a lot internally, and it employs some
of the default fabfile. py. prominent Python developers (even Guido van Rossum
 The -H localhost option indicates to run the command on himself, for some time).
your local machine.Finally, iso is the name of the function in • Go to the App Engine site and then, under ―Choose a
the fab file to run. Language,‖ click in the Pythonbox. You can type Python code
 Reuse the function name iso, but this time have it run a into the Cloud Playground and see results just below.
command by using local(). Here’s the command and its • Just after that are links and directions to download the Python
output: SDK to your machine.This allows you to develop against
from fabric.api import local Google’s cloud APIs on your own hardware.
def iso(): • Google’s main cloud page, you can find details on its services,
local('date -u') including these:

121
 • App Engine : A high-level platform, including Python tools
such as flask and django.
• Compute Engine :Create clusters of virtual machines for large
distributed computing tasks.
• Cloud Storage : Object storage (objects are files, but there are no
directory hierarchies).
• Cloud Datastore :A large NoSQL database.
• Cloud SQL :A large SQL database.
• Cloud Endpoints :Restful access to applications.
• BigQuery :Hadoop-like big data.
• Google services compete with Amazon and OpenStack, a
segue if there ever was one.
Amazon
• As Amazon was growing from hundreds to thousands to
millions of servers, developers ran into all the nasty problems
of distributed systems.
• AmazonWeb Services (AWS), which now dominates the
market. It now contains dozens of services, but the most
relevant are the following:
• Elastic Beanstalk :High-level application platform
• EC2 (Elastic Compute) :Distributed computing
• S3 (Simple Storage Service):Object storage
• RDS :Relational databases (MySQL, PostgreSQL, Oracle,
MSSQL)
• DynamoDB :NoSQL database
• Redshift :Data warehouse
• EMR :Hadoop
OpenStack
• The second most popular cloud service provider has been
Rackspace. In 2010, it formed an unusual partnership with
NASA to merge some of their cloud infrastructure
intoOpenStack.
• This is a freely available open source platform to build public,
private, andhybrid clouds. A new release is made every six
months, the most recent containing over1.25 million lines of
Python from many contributors.
122
• OpenStack is used in production by a growing number of
organizations, including CERN and PayPal.
• OpenStack’s main APIs are RESTful, with Python modules
providing programmatic interfaces, and command-line
Python programs for shell automation. Here are someof the
standard services in the current release:
• Keystone Identity service, providing authentication (for
example, user/password), authorization (capabilities), and
service discovery.
• Nova Compute service, distributing work across networked
servers.
• Swift Object storage, such as Amazon’s S3. It’s used by
Rackspace’s Cloud Files service.
• Glance :Mid-level image storage service.
• Cinder :Low-level block storage service.
• Horizon:Web-based dashboard for all the services.
• Neutron :Network management service.
• Heat :Orchestration (multicloud) service.
• Ceilometer:Telemetry (metrics, monitoring, and metering)
service.
• Other services are proposed from time to time, which then go
through an incubation process and might become part of the
standard OpenStack platform.
• OpenStack runs on Linux or within a Linux virtual machine
(VM). The installation of its core services is still somewhat
involved.
• The fastest way to install OpenStack on Linux is to use
Devstack and watch all the explanatory text flying by as it
runs.
UNIT V COMPLETED