Python Network Programming

Cookbook Practical solutions to

overcome real world networking
challenges 2nd Edition Pradeeban
Kathiravelu
Visit to download the full and correct content document:
https://textbookfull.com/product/python-network-programming-cookbook-practical-sol
utions-to-overcome-real-world-networking-challenges-2nd-edition-pradeeban-kathirav
elu/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Python Network Programming Cookbook Kathiravelu

https://textbookfull.com/product/python-network-programming-
cookbook-kathiravelu/

Practical Packet Analysis Using Wireshark to Solve Real

World Network Proble 3rd Edition Safari

https://textbookfull.com/product/practical-packet-analysis-using-
wireshark-to-solve-real-world-network-proble-3rd-edition-safari/

Python GUI Programming Cookbook Meier

https://textbookfull.com/product/python-gui-programming-cookbook-
meier/

Programming Excel with VBA A Practical Real World Guide

Flavio Morgado

https://textbookfull.com/product/programming-excel-with-vba-a-
practical-real-world-guide-flavio-morgado/
Body Area Network Challenges and Solutions R. Maheswar

https://textbookfull.com/product/body-area-network-challenges-
and-solutions-r-maheswar/

Python GUI Programming Cookbook Burkhard A. Meier

https://textbookfull.com/product/python-gui-programming-cookbook-
burkhard-a-meier/

Python Real World Machine Learning: Real World Machine

Learning: Take your Python Machine learning skills to
the next level 1st Edition Joshi

https://textbookfull.com/product/python-real-world-machine-
learning-real-world-machine-learning-take-your-python-machine-
learning-skills-to-the-next-level-1st-edition-joshi/

Practical Python Design Patterns: Pythonic Solutions to

Common Problems Badenhorst

https://textbookfull.com/product/practical-python-design-
patterns-pythonic-solutions-to-common-problems-badenhorst/

Network Analysis Using Wireshark 2 Cookbook: Practical

recipes to analyze and secure your network using
Wireshark 2, 2nd Edition Nagendra Kumar Nainar

https://textbookfull.com/product/network-analysis-using-
wireshark-2-cookbook-practical-recipes-to-analyze-and-secure-
your-network-using-wireshark-2-2nd-edition-nagendra-kumar-nainar/
Python Network Programming Cookbook
Second Edition

Overcome real-world networking challenges

Pradeeban Kathiravelu
Dr. M. O. Faruque Sarker

BIRMINGHAM - MUMBAI
Python Network Programming Cookbook
Second Edition
Copyright © 2017 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or
transmitted in any form or by any means, without the prior written permission of the
publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the
information presented. However, the information contained in this book is sold without
warranty, either express or implied. Neither the authors, nor Packt Publishing, and its
dealers and distributors will be held liable for any damages caused or alleged to be caused
directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the
companies and products mentioned in this book by the appropriate use of capitals.
However, Packt Publishing cannot guarantee the accuracy of this information.

First published: March 2014

Second edition: August 2017

Production reference: 1080817

Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.

ISBN 978-1-78646-399-9

www.packtpub.com
 Credits

Authors Copy Editors

Pradeeban Kathiravelu Safis Editing
Dr. M. O. Faruque Sarker Juliana Nair

Reviewers Project Coordinator

Dr. S. Gowrishankar Judie Jose
Michael Bright

Commissioning Editor Proofreader

Kartikey Pandey Safis Editing

Acquisition Editor Indexer

Rahul Nair Aishwarya Gangawane

Content Development Editor Graphics

Abhishek Jadhav Kirk D'Penha

Technical Editor Production Coordinator

Mohd Riyan Khan Aparna Bhagat
 About the Author
Pradeeban Kathiravelu is an open source evangelist. He is a Ph.D. researcher at INESC-ID
Lisboa/Instituto Superior Tecnico, Universidade de Lisboa, Portugal, and Universite
Catholique de Louvain, Belgium. He is a Fellow of Erasmus Mundus Joint Degree in
Distributed Computing (EMJD-DC), researching a software-defined approach to quality of
service and data quality in multi-tenant clouds.

Pradeeban holds a master of science degree, Erasmus Mundus European Master in

Distributed Computing (EMDC), from Instituto Superior Tecnico, Portugal and KTH Royal
Institute of Technology, Sweden. He also holds a first class bachelor of science in
engineering (Hons) degree, majoring in computer science and engineering, from the
University of Moratuwa, Sri Lanka. His research interests include Software-Defined
Networking (SDN), distributed systems, cloud computing, web services, big data in
biomedical informatics, Network Functions Virtualizations (NFV), and data mining. He is
very interested in free and open source software development and has been an active
participant in the Google Summer of Code (GSoC) program since 2009, as a student and as a
mentor.

Pradeeban has published several conference papers and co-authored a few book chapters.
He has also worked on OpenDaylight Cookbook and Learning OpenDaylight as a technical
reviewer. Python Network Programming Cookbook, Second Edition (2017) is his first book as an
author, and he is quite excited about it.

I would like to thank my readers for the interest in the book. Please feel free to contact me if
you need any assistance in the topics or the recipes, beyond what we have discussed in the
book. I would like to thank the entire editorial team at Packt, including Abhishek Jadhav,
Rahul Nair, and Mohd Riyan Khan. I would like to extend my thanks to the Linux
Foundation for their open source projects on softwarization of networks and systems. I
would like to thank my friends and colleagues who helped me in various ways. I would like
to thank Prof. Luís Veiga (INESC-ID Lisboa), my MSc and Ph.D. advisor, for sharing his
wisdom and encouragement throughout my stay in Instituto Superior Técnico. I would
like to thank him for being my mentor since 2012. I would also like to thank Prof. Ashish
Sharma (Emory University, Atlanta) for his guidance and motivation.

My special thanks go to my loving wife, Juejing Gu. This book would not be a reality
without her continuous support and creative suggestions. Her tireless efforts helped me
always be on time without missing the deadlines.
I would like to thank my mom, Selvathie Kathiravelu, for her support.
Dr. M. O. Faruque Sarker is a software architect based in London, UK, where he has been
shaping various Linux and open source software solutions, mainly on cloud computing
platforms, for commercial companies, educational institutions, and multinational
consultancies. Over the past 10 years, he has been leading a number of Python software
development and cloud infrastructure automation projects. In 2009, he started using
Python, where he was responsible for shepherding a fleet of miniature E-puck robots at the
University of South Wales, Newport, UK. Later, he honed his Python skills, and he was
invited to work on the Google Summer of Code (2009/2010) programs for contributing to
the BlueZ and Tahoe-LAFS open source projects. He is the author of Python Network
Programming Cookbook and Learning Python Network Programming both by Packt Publishing.

He received his Ph.D. in multi-robot systems from the University of South Wales. He is
currently working at University College London. He takes an active interest in cloud
computing, software security, intelligent systems, and child-centric education. He lives in
East London with his wife, Shahinur, and daughter, Ayesha.

All praises and thanks to Allah, the God who is the Merciful and the Beneficent. I would
not be able to finish this book without the help of God.I would like to thank everyone who
has contributed to the publication of this book, including the publisher, technical reviewers,
editors, my family and friends for their sacrifice of time, encouraging words, and smiles,
especially my wife Shahinur Rijuani for her love and support in my work. I also thank the
readers who have patiently been waiting for this book and who have given me lots of
valuable feedback.
 About the Reviewers
Dr. S. Gowrishankar is currently working as an associate professor in the Department of
Computer Science and Engineering at Dr. Ambedkar Institute of Technology, Bengaluru,
Karnataka, India.

He received his Ph.D. in Engineering from Jadavpur University, Kolkata, West Bengal,
India in 2010, MTech in software engineering and BE in computer science and engineering
from Visvesvaraya Technological University (VTU), Belagavi, Karnataka, India in the year
2005 and 2003 respectively.

From 2011 to 2014 he worked as a senior research scientist and tech lead at Honeywell
Technology Solutions, Bengaluru, Karnataka, India.

He has published several papers in various reputed international journals and conferences.
He is serving as an editor and reviewer for various prestigious international journals. He is
also a member of IEEE, ACM, CSI, and ISTE.

He has delivered many keynote addresses and invited talks throughout India on a variety
of subjects related to computer science and engineering. He was instrumental in organizing
several conferences, workshops, and seminars. He has also served on the panel of a number
of academic bodies of universities and autonomous colleges as a BOS and BOE member.

His current research interests are mainly focused on data science, including its technical
aspects as well as its applications and implications. Specifically, he is interested in the
applications of Machine Learning, Data Mining, and Big Data Analytics in Healthcare.

I would like to acknowledge my earnest gratitude to my wife, Roopa K M, for her constant
source of support and encouragement throughout this assignment. I’m truly thankful to
almighty God for having her in my life and give her my deepest expression of love and
appreciation.
Michael Bright, RHCE/RHCSA, is a solution architect working in the HPE EMEA Customer
Innovation Center.

He has strong experience across Cloud and Container technologies (Docker, Kubernetes,
AWS, GCP, Azure) as well as NFV/SDN.

Based in Grenoble, France, he runs a Python user group and is a co-organizer of the Docker
and FOSS Meetup groups. He has a keen interest in Container, Orchestration, and
Unikernel technologies on which he has presented and run training tutorials in several
conferences.
He has presented many times on subjects diverse as NFV, Docker, Container Orchestration,
Unikernels, Jupyter Notebooks, MongoDB, and Tmux.

Michael has a wealth of experience across pure research, R&D and pre-sales consulting
roles.
 www.PacktPub.com
For support files and downloads related to your book, please visit www.PacktPub.com.

Did you know that Packt offers eBook versions of every book published, with PDF and
ePub files available? You can upgrade to the eBook version at www.PacktPub.comand as a
print book customer, you are entitled to a discount on the eBook copy. Get in touch with us
at service@packtpub.com for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a
range of free newsletters and receive exclusive discounts and offers on Packt books and
eBooks.

https://www.packtpub.com/mapt

Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt
books and video courses, as well as industry-leading tools to help you plan your personal
development and advance your career.

Why subscribe?
Fully searchable across every book published by Packt
Copy and paste, print, and bookmark content
On demand and accessible via a web browser
 Customer Feedback
Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial
process. To help us improve, please leave us an honest review on this book's Amazon page
at https://www.amazon.com/dp/1786463997.

If you'd like to join our team of regular reviewers, you can e-mail us at
customerreviews@packtpub.com. We award our regular reviewers with free eBooks and
videos in exchange for their valuable feedback. Help us be relentless in improving our
products!
I dedicate this book to the world, in memory of my dad, Kanapathipillai Kathiravelu.
 Table of Contents
Preface 1
Chapter 1: Sockets, IPv4, and Simple Client/Server Programming 11
Introduction 12
Printing your machine's name and IPv4 address 12
Getting ready 12
How to do it... 13
How it works... 15
Retrieving a remote machine's IP address 16
How to do it... 16
How it works... 17
Converting an IPv4 address to different formats 18
How to do it... 18
How it works... 19
Finding a service name, given the port and protocol 19
Getting ready 19
How to do it... 19
How it works... 20
Converting integers to and from host to network byte order 20
How to do it... 20
How it works... 21
Setting and getting the default socket timeout 22
How to do it... 22
How it works... 23
Handling socket errors gracefully 23
How to do it... 23
How it works... 25
Modifying a socket's send/receive buffer sizes 26
How to do it... 26
How it works... 28
Changing a socket to the blocking/non-blocking mode 28
How to do it... 28
How it works... 29
Reusing socket addresses 29
How to do it... 30
 How it works... 32
Printing the current time from the internet time server 32
Getting ready 32
How to do it... 33
How it works... 34
Writing an SNTP client 34
How to do it... 34
How it works... 35
Writing a simple TCP echo client/server application 36
How to do it... 36
How it works... 39
Writing a simple UDP echo client/server application 39
How to do it... 39
How it works... 42
Chapter 2: Multiplexing Socket I/O for Better Performance 43
Introduction 43
Using ForkingMixIn in your socket server applications 44
How to do it... 44
How it works... 47
Using ThreadingMixIn in your socket server applications 48
Getting ready 48
How to do it... 48
How it works... 50
Writing a chat server using select.select 50
How to do it... 51
How it works... 58
Multiplexing a web server using select.epoll 58
How to do it... 58
How it works... 61
Multiplexing an echo server using Diesel concurrent library 62
Getting ready 62
How to do it... 64
How it works... 66
Chapter 3: IPv6, Unix Domain Sockets, and Network Interfaces 67
Introduction 67
Forwarding a local port to a remote host 68
How to do it... 68
How it works... 71

[ ii ]
Pinging hosts on the network with ICMP 72
Getting ready 72
How to do it... 72
How it works... 76
Waiting for a remote network service 77
How to do it... 77
How it works... 80
Enumerating interfaces on your machine 80
Getting ready 81
How to do it... 81
How it works... 82
Finding the IP address for a specific interface on your machine 83
Getting ready 83
How to do it... 83
How it works... 84
Finding whether an interface is up on your machine 85
Getting ready 85
How to do it... 85
How it works... 86
Detecting inactive machines on your network 87
Getting ready 87
How to do it... 87
How it works... 89
Performing a basic IPC using connected sockets (socketpair) 89
Getting ready 89
How to do it... 90
How it works... 91
Performing IPC using Unix domain sockets 91
How to do it... 91
How it works... 94
Finding out if your Python supports IPv6 sockets 94
Getting ready 94
How to do it... 95
How it works... 96
Extracting an IPv6 prefix from an IPv6 address 97
How to do it... 97
How it works... 99
Writing an IPv6 echo client/server 99
How to do it... 99

[ iii ]
 How it works... 102
Chapter 4: Programming with HTTP for the Internet 103
Introduction 104
Downloading data from an HTTP server 104
How to do it... 104
How it works... 107
Serving HTTP requests from your machine 107
How to do it... 108
How it works... 110
Extracting cookie information after visiting a website 110
How to do it... 110
How it works... 113
Submitting web forms 114
Getting ready 114
How to do it... 114
How it works... 116
Sending web requests through a proxy server 116
Getting ready 116
How to do it... 116
How it works... 118
Checking whether a web page exists with the HEAD request 118
How to do it... 119
How it works... 121
Spoofing Mozilla Firefox in your client code 121
How to do it... 121
How it works... 122
Saving bandwidth in web requests with the HTTP compression 122
How to do it... 123
How it works... 125
Writing an HTTP fail-over client with resume and partial downloading 125
How to do it... 126
How it works... 127
Writing a simple HTTPS server code with Python and OpenSSL 128
Getting ready 128
How to do it... 128
How it works... 129
Building asynchronous network applications with Twisted 130
Getting ready 130
How to do it... 131

[ iv ]
 How it works... 134
Building asynchronous network applications with Tornado 134
Getting ready 134
How to do it... 134
How it works... 136
Building concurrent applications with Tornado Future 137
Getting ready 137
How to do it... 137
How it works... 140
Chapter 5: Email Protocols, FTP, and CGI Programming 141
Introduction 141
Listing the files in a remote FTP server 142
Getting ready 142
How to do it... 142
How it works... 144
Common error 144
Uploading a local file to a remote FTP server 146
Getting ready 146
How to do it... 146
How it works... 147
Emailing your current working directory as a compressed ZIP file 148
Getting ready 148
How to do it... 148
How it works... 151
See also 152
Downloading your Google email with POP3 152
Getting ready 152
How to do it... 152
How it works... 153
Checking your remote email with IMAP 153
Getting ready 154
How to do it... 154
How it works... 155
Sending an email with an attachment via Gmail SMTP server 156
Getting ready 156
How to do it... 156
How it works... 158
Writing a guestbook for your (Python-based) web server with CGI 158
Getting ready 158

[v]
 How to do it... 158
How it works... 162
Finding the mail server from an email address 163
Getting ready 163
How to do it... 164
How it works... 166
Writing a simple SMTP server 166
Getting ready 166
How to do it... 166
How it works... 169
Writing a secure SMTP client using TLS 170
Getting ready 170
How to do it... 170
How it works... 172
Writing an email client with POP3 173
Getting ready 173
How to do it... 174
How it works... 175
Chapter 6: Programming Across Machine Boundaries 176
Introduction 176
Executing a remote shell command using telnet 177
Getting ready 177
How to do it... 177
How it works... 179
Copying a file to a remote machine by SFTP 179
Getting ready 179
How to do it... 180
How it works... 181
Printing a remote machine's CPU information 182
Getting ready 182
How to do it... 182
How it works... 186
Installing a Python package remotely 186
Getting ready 187
How to do it... 187
How it works... 189
Running a MySQL command remotely 189
Getting ready 189
How to do it... 190

[ vi ]
 How it works... 193
Transferring files to a remote machine over SSH 193
Getting ready 193
How to do it... 194
How it works... 196
Configuring Apache remotely to host a website 196
Getting ready 197
How to do it... 197
How it works... 199
Chapter 7: Working with Web Services – XML-RPC, SOAP, and REST 201
Introduction 201
Querying a local XML-RPC server 202
Getting ready 202
How to do it... 202
How it works... 204
Writing a multithreaded, multicall XML-RPC server 205
How to do it... 205
How it works... 207
Running an XML-RPC server with a basic HTTP authentication 208
How to do it... 208
How it works... 212
Collecting some photo information from Flickr using REST 213
How to do it... 213
How it works... 217
Searching for SOAP methods from an Amazon S3 web service 217
Getting ready 217
How to do it... 218
How it works... 219
Searching Amazon for books through the product search API 219
Getting ready 220
How to do it... 220
How it works... 222
Creating RESTful web applications with Flask 222
Getting ready 223
How to do it... 223
How it works... 226
Chapter 8: Network Monitoring and Security 227
Introduction 227

[ vii ]
 Sniffing packets on your network 228
Getting ready 228
How to do it... 228
How it works... 230
Saving packets in the pcap format using the pcap dumper 230
How to do it... 231
How it works... 234
Adding an extra header in HTTP packets 235
How to do it... 235
How it works... 236
Scanning the ports of a remote host 237
How to do it... 237
How it works... 239
Customizing the IP address of a packet 239
How to do it... 239
How it works... 241
Replaying traffic by reading from a saved pcap file 241
How to do it... 241
How it works... 243
Scanning the broadcast of packets 244
How to do it... 244
How it works... 246
Chapter 9: Network Modeling 247
Introduction 247
Simulating networks with ns-3 248
Getting ready 248
How to do it... 250
How it works... 252
Emulating networks with Mininet 252
Getting ready 253
How to do it... 253
How it works... 255
Distributed network emulation with MaxiNet 255
Getting ready 256
How to do it... 257
How it works... 258
Emulating wireless networks with Mininet-WiFi 259
Getting ready 259
How to do it... 259

[ viii ]
 How it works... 263
Extending Mininet to emulate containers 264
Getting ready 265
How to do it... 266
How it works... 270
Chapter 10: Getting Started with SDN 271
Introduction 271
SDN emulation with Mininet 272
Getting ready 272
How to do it... 272
How it works... 275
Developing Software-Defined Networks with OpenDaylight controller 276
Getting ready 276
How to do it... 278
How it works... 281
Developing Software-Defined Networks with ONOS controller 281
Getting ready 282
How to do it... 284
How it works... 286
Developing Software-Defined Networks with Floodlight controller 286
Getting ready 287
How to do it... 290
How it works... 293
Developing Software-Defined Networks with Ryu controller 293
Getting ready 293
How to do it... 294
How it works... 298
Developing Software-Defined Networks with POX controller 299
Getting ready 299
How to do it... 301
How it works... 302
Developing Software-Defined Networks visually with MiniEdit 303
Getting ready 303
How to do it... 303
How it works... 307
Chapter 11: Authentication, Authorization, and Accounting (AAA) 308
Introduction 309
Finding DNS names of a network 309

[ ix ]
 Getting ready 309
How to do it... 309
How it works... 311
Finding DNS host information 311
Getting ready 311
How to do it... 311
How it works... 313
Finding DNS resource records 313
Getting ready 315
How to do it... 315
How it works... 318
Making DNS zone transfer 319
Getting ready 319
How to do it... 319
How it works... 323
Querying NTP servers 323
Getting ready 323
How to do it... 323
How it works... 325
Connecting to an LDAP server 325
Getting ready 326
How to do it... 327
How it works... 330
Making LDAP bind 331
Getting ready 331
How to do it... 331
How it works... 333
Reading and writing LDAP 333
Getting ready 333
How to do it... 334
How it works... 335
Authenticating REST APIs with Eve 336
Getting ready 336
How to do it... 336
How it works... 338
Throttling requests with RequestsThrottler 338
Getting ready 339
How to do it... 339
How it works... 342

[x]
Chapter 12: Open and Proprietary Networking Solutions 343
Introduction 343
Configuring Red PNDA 344
Getting ready 344
How to do it... 344
How it works... 347
Configuring VMware NSX for vSphere 6.3.2 348
Getting ready 348
How to do it... 349
How it works... 350
Configuring Juniper Contrail Server Manager 351
Getting ready 351
How to do it... 354
How it works... 355
Configuring OpenContrail controller 355
Getting ready 356
How to do it... 356
How it works... 357
Configuring OpenContrail cluster 357
How to do it... 358
How it works... 363
Interacting with devices running Cisco IOS XR 363
Getting ready 364
How to do it... 364
How it works... 364
Collaborating with Cisco Spark API 365
Getting ready 365
How to do it... 367
How it works... 368
Chapter 13: NFV and Orchestration – A Larger Ecosystem 369
Introduction 369
Building VNFs with OPNFV 370
Getting ready 370
How to do it... 371
How it works... 373
Packet processing with DPDK 376
Getting ready 377
How to do it... 379

[ xi ]
 How it works... 379
Parsing BMP messages with SNAS.io 380
Getting ready 381
How to do it... 382
How it works... 386
Controlling drones with a wireless network 387
Getting ready 387
How to do it... 387
How it works... 390
Creating PNDA clusters 390
Getting ready 391
How to do it... 394
How it works... 395
Chapter 14: Programming the Internet 396
Introduction 396
Checking a website status 396
Getting ready 397
How to do it... 397
How it works... 398
Benchmarking BGP implementations with bgperf 398
Getting ready 398
How to do it... 400
How it works... 400
BGP with ExaBGP 401
Getting ready 401
How to do it... 401
Looking glass implementations with Python 402
Getting ready 404
How to do it... 405
How it works... 407
Understanding the internet ecosystem with Python 407
Getting ready 408
How to do it... 409
How it works... 410
Establishing BGP connections with yabgp 410
Getting ready 411
How to do it... 411
How it works... 413

[ xii ]
Index 414

[ xiii ]
 Preface
It has been more than 3 years since Python Network Programming Cookbook was first
published. In this second edition, we extend our book to discuss the recent advancements in
the networking industry and network softwarization. The widespread use of Software-
Defined Networking (SDN), Network Functions Virtualization (NFV), and orchestration
have been addressed in detail in the latter chapters while the first eight chapters were taken
from the first edition, improved with a few new recipes based on the feedback from the
readers.

This book is an exploratory guide to network programming in Python. It has touched a

wide range of networking protocols such as TCP/UDP, HTTP/HTTPS, FTP, SMTP, POP3,
IMAP, and CGI. With the power and interactivity of Python, it brings joy and fun to
develop various scripts for performing real-world tasks on network and system
administration, web application development, interacting with your local and remote
network, low-level network packet capture and analysis, and so on. The primary focus of
this book is to give you a hands-on experience on the topics covered. So, this book covers
less theory, but it is packed with practical materials.

This book is written with a DevOps mindset, where a developer is also more or less in
charge of operation, that is, deploying the application and managing various aspects of it,
such as remote server administration, monitoring, scaling-up, and optimizing for better
performance. This book introduces you to a bunch of open-source, third-party Python
libraries, which are ideal to be used in various use cases. We elaborate in detail the
configurations of complex networking systems with helpful hints to ensure that the reader
can follow them without getting stuck.

We hope you will enjoy the recipes presented in this book and extend them to make them
even more powerful and enjoyable.

What this book covers

Chapter 1, Sockets, IPv4, and Simple Client/Server Programming, introduces you to Python's
core networking library with various small tasks and enables you to create your first client-
server application.

Chapter 2, Multiplexing Socket I/O for Better Performance, discusses various useful techniques
for scaling your client/server applications with default and third-party libraries.
Preface

Chapter 3, IPv6, Unix Domain Sockets, and Network Interfaces, focuses more on administering
your local machine and looking after your local area network.

Chapter 4, Programming with HTTP for the Internet, enables you to create a mini command-
line browser with various features such as submitting web forms, handling cookies,
managing partial downloads, compressing data, and serving secure content over HTTPS.

Chapter 5, Email Protocols, FTP, and CGI Programming, brings you the joy of automating
your FTP and e-mail tasks such as manipulating your Gmail account, and reading or
sending emails from a script or creating a guest book for your web application. We learn to
write email clients with SMTP and POP3.

Chapter 6, Programming Across Machine Boundaries, gives you a taste of automating your
system administration and deployment tasks over SSH. You can run commands, install
packages, or set up new websites remotely from your laptop.

Chapter 7, Working with Web Services – XML-RPC, SOAP, and REST, introduces you to
various API protocols such as XML-RPC, SOAP, and REST. You can programmatically ask
any website or web service for information and interact with them. For example, you can
search for products on Amazon or Google.

Chapter 8, Network Monitoring and Security, introduces you to various techniques for
capturing, storing, analyzing, and manipulating network packets. This encourages you to
go further to investigate your network security issues using concise Python scripts.

Chapter 9, Network Modeling, introduces you to the world of network simulations and
emulations. You learn to simulate networks with NS-3, and emulate networking systems
with Mininet and its extensions.

Chapter 10, Getting Started with SDN, discusses the enterprise SDN controllers, configuring
them to use in Software-Defined Networks. We learn to develop SDN visually with
MiniEdit, and configure the networks with OpenDaylight, ONOS, Floodlight, Ryu, and
POX controllers.

Chapter 11, Authentication, Authorization, and Accounting (AAA), introduces how the
networks are secured, and discusses configuring LDAP clients with Python, accounting
aspects of the network, and authentication and access of network services.

Chapter 12, Open and Proprietary Networking Solutions, discusses in detail, configuring large-
scale enterprise networking projects, including a few projects from Cisco, Juniper, VMware,
and the Linux Foundation.

[2]
Preface

Chapter 13, NFV and Orchestration – A Larger Ecosystem, discusses configuring complex
NFV and orchestration systems of the Linux Foundation, such as OPNFV, DPDK, SNAS.io,
Dronekit, and PNDA. We elaborate the use of Python in these complex systems.

Chapter 14, Programming the Internet, presents you various Python libraries for BGP
protocol and implementations developed for the internet scale. We learn to use and
benchmark libraries such as exabgp and yabgp, and also discuss the looking glass
implementations with Python.

What you need for this book

You need a working PC or laptop, preferably with a modern Linux operating system. The
installation instructions are written and tested on Ubuntu 16.04 LTS and would work on
any recent Debian-based Linux operating system without modification. We developed for
Python 3. However, we have maintained backward-compatibility with Python 2 in our
recipes as much as we can. On the other hand, some open source projects used in this book
do not yet support Python 3. So, ideally, you will need both Python 2 and Python 3 to test
all the recipes in this book.

Most of the recipes in this book will run on other platforms such as Windows and Mac OS
with some changes in the configuration steps. Some of the recipes require two or more
computers in a cluster to test the distributed systems. You may use Amazon Web Services
(AWS) to initiate a cluster inside a single placement group to test these recipes.

You also need a working internet connection to install the third-party software libraries
mentioned with respective recipes. If you do not have a stable or continuous internet
connection, you can download the third-party libraries and install them in one go.
However, it is highly recommended to test some of these recipes with the internet
connection, as it would make the configuration task minimal and more interesting, than
having to download a bulk of software in bunch. Moreover, testing the application in an
AWS cluster would certainly require the internet connectivity.

The following is a list of the Python third-party libraries with their download URLs:

ntplib: https://pypi.python.org/pypi/ntplib/
diesel: https://pypi.python.org/pypi/diesel/
nmap: https://pypi.python.org/pypi/python-nmap
scapy: https://pypi.python.org/pypi/scapy
netifaces: https://pypi.python.org/pypi/netifaces/

[3]
Preface

netaddr: https://pypi.python.org/pypi/netaddr
pyopenssl: https://pypi.python.org/pypi/pyOpenSSL
pygeocoder: https://pypi.python.org/pypi/pygocoder
pyyaml: https://pypi.python.org/pypi/PyYAML
requests: https://pypi.python.org/pypi/requests
feedparser: https://pypi.python.org/pypi/feedparser
paramiko: https://pypi.python.org/pypi/paramiko/
fabric: https://pypi.python.org/pypi/Fabric
supervisor: https://pypi.python.org/pypi/supervisor
xmlrpclib: https://pypi.python.org/pypi/xmlrpclib
SOAPpy: https://pypi.python.org/pypi/SOAPpy
bottlenose: https://pypi.python.org/pypi/bottlenose
construct: https://pypi.python.org/pypi/construct/
libpcap: https://pypi.python.org/pypi/pcap
setup tools: https://pypi.python.org/pypi/setuptools
exabgp: https://pypi.python.org/pypi/exabgp
traixroute: https://pypi.python.org/pypi/traixroute
dronekit: https://pypi.python.org/pypi/dronekit
dronekit-sitl: https://pypi.python.org/simple/dronekit-sitl/
ryu: https://pypi.python.org/pypi/ryu
Flask: https://pypi.python.org/pypi/Flask
smtpd: https://pypi.python.org/pypi/secure-smtpd
twisted: https://pypi.python.org/pypi/Twisted
tornado: https://pypi.python.org/pypi/tornado
dnspython: https://pypi.python.org/pypi/dnspython
ldap3: https://pypi.python.org/pypi/ldap3
Eve: https://pypi.python.org/pypi/Eve
RequestsThrottler: https://pypi.python.org/pypi/RequestsThrottler
PyNSXv: https://pypi.python.org/pypi/pynsxv
vmware-nsx: https://pypi.python.org/pypi/vmware-nsx

[4]
Preface

Other software needed to run some recipes are as follows:

postfix: http://www.postfix.org/
openssh server: http://www.openssh.com/
mysql server: http://downloads.mysql.com/
apache2: http://httpd.apache.org/download.cgi/
virtualenv: https://virtualenv.pypa.io/
filezilla: https://filezilla-project.org/
vsftpd: https://security.appspot.com/vsftpd.html
telnetd: telnetd.sourceforge.net/
curl: https://curl.haxx.se/
NS-3: https://www.nsnam.org/ns-3-26/download/
Mininet: mininet.org/
Ansible: https://www.ansible.com/
Git: https://git-scm.com/
aptitude: https://www.openhub.net/p/aptitude
Node-ws / wscat: https://www.npmjs.com/package/wscat
MaxiNet: https://github.com/MaxiNet/MaxiNet/
Mininet-WiFi: https://github.com/intrig-unicamp/mininet-wifi
ContainerNet: https://github.com/containernet/containernet.git
Ant: ant.apache.org/
Maven: https://maven.apache.org/
OpenDaylight: https://www.opendaylight.org/downloads
ONOS: https://wiki.onosproject.org/display/ONOS/Downloads
Floodlight: http://www.projectfloodlight.org/download/
POX: http://github.com/noxrepo/pox
libnl-3-dev: https://packages.debian.org/sid/libnl-3-dev
libnl-genl-3-dev: https://packages.debian.org/sid/libnl-genl-3-dev
libnl-route-3-dev: https://packages.debian.org/sid/libnl-route-3-dev
pkg-config: https://www.freedesktop.org/wiki/Software/pkg-config/
python-tz: pytz.sourceforge.net/
libpcap-dev: https://packages.debian.org/libpcap-dev
libcap2-dev: https://packages.debian.org/jessie/libcap2-dev

[5]
Another random document with
no related content on Scribd:
 Hindu Rags

The Hindu songs presided over by special gods were called rags,
but of course, the name has no relationship whatever to our ragtime.
It seems that religious feeling has dominated everything in the
Orient; for this reason every idea connected with music has a
corresponding idea in Hindu mythology. The rags were all named
after the gods who brought music down from heaven to comfort
man. The character of each god or goddess was supposed to be
reflected in the rag, and it was not the result of scientific study as
were, for instance, the scales of the Chinese. Our knowledge of the
rag has come down to us from what Sanskrit writers on Indian music
have said, and what is practised today by the modern Hindu. No
doubt Arabia and Persia once had a music system very much like this
one of the Hindus.
Rags, ragas, or raginis were neither airs nor modes in our sense
of the words, but something like the modes of the Arab songs, they
were melodic forms, or themes, on which musicians either
improvised or composed new songs, by using them in rhythms of
endless variety.
There were many many rags, and they were under the guidance of
the gods of the rainy season, the cold season, the mild, the hot, etc.,
and could only be sung during their special seasons. It was thought
that these songs sung at the wrong time would bring down calamity.
Again, as with the American Indian (Chapter II), we find tribes who
are most careful to sing certain music at certain times.
It is told that a Hindu nobleman, long ago, tried to sing a night
song in the daytime and darkness covered all things within the sound
of his voice!
As important as was the song from the earliest time, instrumental
music held almost an equal place.
 Orchestra

The Hindu orchestra is sometimes large and sometimes small. Its

dances are lively and vigorous, and very seldom slow and romantic.
Nevertheless they have many kinds of songs, some lively and some
not, such as: songs in honor of Krishna (one of their principal gods),
official odes, war hymns, love songs, evening songs, wedding songs,
cradle and patriotic songs. In some, the Arab seems to have
influenced the Hindu music, because they have the lively rhythm and
the variety of the Arab music, yet it is difficult to know which one
influenced the other. The Arab music has the variety and luxurious
soft beauty in popular dances equal to anything our modern
musicians or poets have composed.
The playing of instruments, accompanying songs with Sanskrit
texts, was supposed to give energy, develop heroism, make a peaceful
heart, and drive away harm and impurity.
The members of the different castes or sects had and still have
meetings held sometimes in private houses and sometimes in the
temples, when they sang religious hymns. Among the higher classes,
they went to the expense of having good musicians and of giving
artistic performances, but among the common people, their idea was
that the greater the noise the more they showed their devotion, so
they sang, beat drums and blew whistles without any regard for time
or melody, and you can imagine the effect was pandemonium.
 Notation

It is impossible to write the Hindu music in European notation,

because instead of dividing the scale into semi-tones or half-steps,
they use quarter tones.
Margaret Glyn, in her book, Evolution of Musical Form, writes:
“In the East notation is in an elementary condition, the staff being
unknown. The Hindus, Chinese, and Abyssinians have ancient note-
signs, consisting of a kind of letter to which some indication of time
is added, but in this respect the Chinese system is wanting, having
practically no time-notation. Probably note-signs existed in Persia
and Arabia, but these do not appear to have survived. The modern
Arabic notation is but three hundred years old, and is said to have
been invented by one Demetrius de Cantemir who adapted the letters
of the Turkish alphabet for the purpose. This has eighteen tones to
the octave (we have twelve), and is used in Turkey and other
countries of the near East.”
The Hindus seem to like both triple (3 beats) and duple (2 or 4
beats) rhythm. The scheme changes according to the poetry of the
song, and the pitch and the length of tones are shown by Sanskrit
characters and special signs or words.
 Instruments

The Hindus have spent much time in studying music and the
instruments of which they have many kinds. There are the strings,
there are skins sounded by beating, instruments struck together in
pairs, and those that are blown. They like the strings best. The
characteristic instrument is the vina, usually of wood or bamboo
strengthened by one to three gourds as sounding boards, and having
five or seven wire strings played something like the zither, but
sometimes with a bow. (Figure 16.) There are many varieties of the
sitar, an instrument like a lute, and many viols of which the sarinda
sarunja is typical and is played with a bow. (Figure 17.)
Among the percussion instruments are tambourines, castanets and
cymbals; they had wind instruments such as flutes (seldom
transverse or blown in from the side, and often played by blowing on
them through the nostrils), trumpets, horns, bagpipes and oboes.
Certain instruments were used only by the priests, others by beggars,
and others by dancing girls. Imagine how weird a story could be told
in music if a modern symphony orchestra played a piece of music
telling of life among the Hindus. Maybe some of the readers of this
book will get an idea for a Hindu tone poem, who knows?
Courtesy of the Metropolitan Museum of
Art, New York City.

Hindu Instruments.

Fig. 15.—Tabla—
drum.

Fig. 16.—Kinnari—
Vina, a stringed
instrument.
 Fig. 17.

Sitar (Strings) Trumpet.

Fig. 18.

Hindu Instruments.
Childhood of Music
 CHAPTER VII
What Church Music Imported from Greece

During the centuries when the Eastern nations were powerful the
European continent was inhabited by primitive men, who had
gradually formed tribes. They had rude songs, dances, and crude
instruments. They used their music in religious ceremonies, to
celebrate war victories and successful hunting expeditions, to sing to
their sweethearts, and to accompany their work in the fields and
homes, much as the American Indian did. Many manners and
customs of the Anglo-Saxon (English), Teutonic (German), Norse
(Danish, Swedish and Norwegian), Celtic (Irish) and Gallic (French)
races may be traced back to these barbaric days, and even the
beginning of national schools of music may be found.
Although a thousand years passed between the Greek musical era
and the “Golden Age” of Christian Church music, much that
happened in that time is hidden in darkness. The nations and tribes
were fighting for existence and were developing into the nations
which we know today.
Islam or the Mohammedan religion, and not Christianity, was the
great influence.
Julius Cæsar (100–44 B.C.), the great Roman General, conquered
Gallia (France), then invaded the land of the Teutons (Germany) and
even reached England. In parts of northern Europe, one still sees the
remains of great roadways, aqueducts or water works, and bridges,
that the Romans built during their invasions.
In the Cluny Museum in Paris is a great hall built as a bath by the
Romans. In Bath, England, the city was named for the ancient
Roman baths still existing, and you can see the pipe lines which
carried the water.
 Dark Ages

The world at that time was not a happy place in which to live.
There was constant warfare between the once powerful Roman
Empire and these barbaric tribes. The poorest people were oppressed
and many were slaves, bought and sold by the rich land owners and
army leaders.
Into such a world was the child, Jesus, born—a world with little
love for humanity, little unselfishness, little sympathy for the down-
trodden and unhappy, few kind words for the poor or the sick, little
justice and less mercy. No wonder that His teachings brought new
life and gave hope to the people!
For several centuries following the birth of Christ, the world went
through a period called the Dark Ages. Rome, the city of glorious
victories and brilliant culture became the prey of the barbaric tribes
—Huns, Goths, Visigoths, Vandals, Franks, Saxons, and Slavs,—until
it seemed that civilization would be wiped out and people would
become primitive again.
Music was saved during the Dark Ages through a small band of
faithful followers of Jesus Christ, who founded a church in His name.
That their music should have been made up of existing tunes and
words is very natural. Jesus, himself, brought up in the religion of
the Hebrews, often sang the Psalms of David. The beautiful
traditional music of the Jewish synagogues found its way into the
services of the early Christian Church, because many of the believers
were Hebrews. Soon the Hebrew Bible texts were translated into
Latin—the everyday language of the Romans, and as most of these
early Christians lived in Rome, they followed the rules of music the
Romans learned from the Greeks. So, our Church music was
influenced both by the Hebrews and Greeks.
For about three hundred years the early Christians had to hold
their services in secret, as they were punished even by death when
caught, for not worshipping Jupiter and the Roman gods. They were
not rich and influential, but just humble folk to whom the teachings
of Jesus came as a joyous comfort. They had no beautiful palaces
where they could hold services, and at the same time hide from the
Roman centurions, so they worshipped in dark and secret places and
could not have much music as it would have attracted the attention
of their enemies. The early Christians shunned music, too, because it
had been used for the wild dances and festivals of their pagan
oppressors. As they were poor and uneducated, they had had little
training and lacked money to buy instruments, so all in all music had
a hard time to keep alive.
From what we can gather, they chanted their Psalms much as did
the Hebrews and had responses which sounded like soft and
monotonous droning.
As time went on emperors like Constantine, began to take away the
death penalty from those believing in Christ and gradually as the
Romans saw the beauty of His teachings they became Christians in
increasing numbers. Many of these Romans came from the upper
classes, and as Greek was the language of culture they had had a
thorough Greek education and owned many instruments, so they
brought their Greek musical inheritance to the growing band.
Thus, the chants composed in Rome for the kithara were the direct
ancestors of our Christian hymns. These early hymns were also a
bridge between the single melody line of the Orient and Greece and
Rome, and the many melody lines, called polyphonic music, of
Europe.
In 325 A.D., Emperor Constantine made Christianity the national
religion of Rome. He also founded the Christian Church in
Byzantium, later called Constantinople, and all through the Dark
Ages, in many parts of Europe, the cathedrals and church schools
were the only gleam of learning in a time of darkness and struggle.
After the Roman Empire reached its greatest height, in the 2nd
century, it gradually grew weaker and during the 4th and 5th
centuries, the Goths, Vandals and Huns drove the government from
Rome to Constantinople. In the 7th century Mohammedanism rose
and swept over Syria, Egypt and North Africa, and reached Spain in
the 8th century.
 Answering Music

It is related that St. Ignatius (49–107 A.D.) one of the early

Christian fathers, had a vision in which he heard the Heavenly choirs
praising the Holy Trinity, in alternating chants, and he was so
impressed by it, that he introduced into the Church the idea of two
choirs of singers answering each other.
In singing the Psalms of David, the Hebrews used this idea of
antiphonal music. (Anti—against, phonal—sounding: antiphonal—
sounding against each other.) We see it too in the Greek choruses, in
the Roman kitharoedic chants (chants accompanied by the kithara),
and now in the early Christian hymns. From this antiphonal music to
the later polyphony (poly—many, phony—sounds: many voices or
parts) is a natural step. Here again is an instance of the influence of
one nation on another.
 The Patron Saint of Music

Among the martyrs to the cause of the Christian faith, was St.
Cecilia, a member of a noble Roman family, who was put to death for
becoming a Christian about 177 A.D. She is believed to be buried in
the Roman catacombs (underground burial chambers) and is the
patron saint of music, and she is supposed to have invented the
organ.
It was St. Ambrose (333–397 A.D.) who worked out the first system
for church music and put it on a foundation that lasted for two
hundred years.
 Greek Modes as Models

St. Ambrose built scales modeled on the old Greek modes and they
were given Greek names, but somehow the names became mixed so
that the mode called by Ambrose, Dorian (from D to D on the white
keys of the piano) was the Greek Phrygian; and the Phrygian (E to E)
of Ambrose was called Dorian by the Greeks; F to F is the Lydian
mode; G to G, the Mixolydian. These were the four authentic
Ecclesiastical or Church Modes.
St. Ambrose felt it his duty to make over the church music because
popular street songs had crept in with the Hebrew Psalms and Greek
and Roman chants! It was much the same effect as if you entered a
church and heard the organ and choir performing “Yes, We Have No
Bananas.” This is a funny comparison because it is said that a part of
“Yes, We Have No Bananas” is stolen from Handel’s great
“Hallelujah” Chorus from the oratorio The Messiah.
About this time, schools were formed to train singers in these new
hymns and church services, and a way to write down the music
composed by St. Ambrose and his followers was needed. The Greek
letters had been used in Rome, but now a new system called neumes
appeared; this word comes from the Greek and means “breath” and
the neumes simply marked where one should breathe in chanting the
hymns. There were eight signs with Latin names which gave full
directions when to raise and lower the voice.
The system of Neumes notation looks like our present day
shorthand and was a help, though, should we use it now we would
think it anything but a help.
While the Neumes writing showed how to mark the time, it had a
serious shortcoming, because it did not outline the melody exactly. It
indicated whether the melody rose or fell, but just how much was a
question not definitely shown. An unfamiliar chant could not be sung
until the notation had been worked out. It took five years for a choir
singer to be able to sing the music!
When the singers sang solos, they ornamented the songs and sang
anything they pleased! This made variety but it must also have
caused much confusion! The people may have learned this
ornamental singing of the Arab, from The Gloss. (Page 59.)
The next step was the Gregorian chant which even today is sung in
the Roman Catholic Churches.
To the four authentic scales of St. Ambrose, St. Gregory who was
Pope from 590 to 604 added four more called plagal. He did not
invent these scales but based them on the old Greek and Ambrosian
modes. To each authentic scale, he added a plagal scale starting four
tones below it, and to the name of the authentic mode is added the
prefix hypo.
Each authentic mode and its hypo are related.
Here is a table of these related modes, and their names:
After a painting by
Garofalo, National
Gallery at Rome.

Saint Cecilia—Patron Saint of Music.

(Holding a portative organ.)
After a painting by
Maxence, in Paris.

Book of Peace.
 Authentic Scales or Modes Plagal Scales or Modes
(St. Ambrose’s Scales) (St. Gregory’s Scales)

I. Dorian: II. Hypo-Dorian:

d ef͡ g a bc͡ d a bc͡ d ef͡ g a
III. Phrygian: IV. Hypo-Phrygian:
ef͡ g a bc͡ d e bc͡ d ef͡ g a b
V. Lydian: VI. Hypo-Lydian:
f g a bc͡ d ef͡ c d ef͡ g a bc͡
VII. Mixo-Lydian: VIII. Hypo-Mixo-Lydian:
g a bc͡ d ef͡ g d ef͡ g a bc͡ d

We have marked the half-steps bc͡ and ef͡ , and in every mode they
fall on different degrees of the scale. This shifting of the half-steps
tells us the name of the mode.
In order to try to give you a definite idea of how the church modes
worked, we have written the familiar national hymn America in each
mode (see page 74). Play them, and you will see how one differs from
the other.
Katherine Ruth Heyman has used a similar idea in her little book
The Relation of Ultramodern to Archaic Music in explaining Greek
Modes.
It took many years to establish this music and it was not until the
time of Charlemagne (742–814) that it became a real system called
Plain Chant or Plain-song (from the Latin, cantus planus).
Pope Gregory founded the Schola Cantorum, school of singers, at
Rome, and with these trained people he tried to establish for all
Christian churches, a way to sing systematically and well. They
studied nine years, and everything had to be memorized, for only the
leader had a song book. Books were written by hand and were hard
to get. The teacher had a monochord, the instrument invented by
Pythagoras, to give the pitch, for all the singing was done without
accompaniment. The singing must have improved greatly after
Gregory became Pope, for before his reform, music had become a
stunt with no solemnity, and people in the churches waved
handkerchiefs if the stunt pleased them!
America