A Python Data Analyst's Toolkit: Learn Python and Python-Based Libraries With Applications in Data Analysis and Statistics Gayathri Rajagopalan

A Python Data Analyst’s Toolkit: Learn
Python and Python-based Libraries

with Applications in Data Analysis and
Statistics Gayathri Rajagopalan
Visit to download the full and correct content document:
https://textbookfull.com/product/a-python-data-analysts-toolkit-learn-python-and-pytho
n-based-libraries-with-applications-in-data-analysis-and-statistics-gayathri-rajagopala
n/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...
Data Analysis from Scratch with Python Peters Morgan
https://textbookfull.com/product/data-analysis-from-scratch-with-
python-peters-morgan/
Data Analysis with Python and PySpark (MEAP V07)

Jonathan Rioux
https://textbookfull.com/product/data-analysis-with-python-and-
pyspark-meap-v07-jonathan-rioux/
Python for Data Analysis Data Wrangling with Pandas

NumPy and IPython Wes Mckinney
https://textbookfull.com/product/python-for-data-analysis-data-
wrangling-with-pandas-numpy-and-ipython-wes-mckinney/
Python Data Analysis: Perform data collection, data

processing, wrangling, visualization, and model
building using Python 3rd Edition Avinash Navlani
https://textbookfull.com/product/python-data-analysis-perform-
data-collection-data-processing-wrangling-visualization-and-
model-building-using-python-3rd-edition-avinash-navlani/
Practical Python Data Visualization: A Fast Track
Approach To Learning Data Visualization With Python
Ashwin Pajankar
https://textbookfull.com/product/practical-python-data-
visualization-a-fast-track-approach-to-learning-data-
visualization-with-python-ashwin-pajankar/
Python 2 and 3 Compatibility: With Six and Python-

Future Libraries Nanjekye
https://textbookfull.com/product/python-2-and-3-compatibility-
with-six-and-python-future-libraries-nanjekye/
Learning Data Mining with Python Layton
https://textbookfull.com/product/learning-data-mining-with-
python-layton/
A Tour Of Data Science: Learn R And Python In Parallel

Nailong Zhang
https://textbookfull.com/product/a-tour-of-data-science-learn-r-
and-python-in-parallel-nailong-zhang/
Hands-on Scikit-Learn for machine learning

applications: data science fundamentals with Python
David Paper
https://textbookfull.com/product/hands-on-scikit-learn-for-
machine-learning-applications-data-science-fundamentals-with-
python-david-paper/
A Python Data
Analyst’s Toolkit
Learn Python and Python-based
Libraries with Applications in Data
Analysis and Statistics
—
Gayathri Rajagopalan
A Python Data
Analyst’s Toolkit
Learn Python and Python-based
Libraries with Applications in Data
Analysis and Statistics
A Python Data Analyst’s Toolkit: Learn Python and Python-based Libraries with
Applications in Data Analysis and Statistics
ISBN-13 (pbk): 978-1-4842-6398-3 ISBN-13 (electronic): 978-1-4842-6399-0

https://doi.org/10.1007/978-1-4842-6399-0
Copyright © 2021 by Gayathri Rajagopalan

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the
material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now
known or hereafter developed.
Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with
every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an
editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the
trademark.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not
identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to
proprietary rights.
While the advice and information in this book are believed to be true and accurate at the date of publication,
neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or
omissions that may be made. The publisher makes no warranty, express or implied, with respect to the
material contained herein.
Managing Director, Apress Media LLC: Welmoed Spahr
Acquisitions Editor: Celestin Suresh John
Development Editor: James Markham
Coordinating Editor: Aditee Mirashi
Cover designed by eStudioCalamar
Cover image designed by Freepik (www.freepik.com)
Distributed to the book trade worldwide by Springer Science+Business Media New York, 1 New York Plaza,
Suite 4600, New York, NY 10004-1562, USA. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail
orders-ny@springer-sbm.com, or visit www.springeronline.com. Apress Media, LLC is a California LLC
and the sole member (owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc). SSBM
Finance Inc is a Delaware corporation.
For information on translations, please e-mail booktranslations@springernature.com; for reprint,
paperback, or audio rights, please e-mail bookpermissions@springernature.com.
Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook versions and
licenses are also available for most titles. For more information, reference our Print and eBook Bulk Sales
web page at http://www.apress.com/bulk-sales.
Any source code or other supplementary material referenced by the author in this book is available to
readers on GitHub via the book’s product page, located at www.apress.com/978-1-4842-6398-3. For more
detailed information, please visit http://www.apress.com/source-code.
Printed on acid-free paper
This book is dedicated to my daughter.
Table of Contents
About the Author�� xiii
About the Technical Reviewer��xv

Acknowledgments��xvii
Introduction��xix
Chapter 1: Getting Familiar with Python�� 1

Technical requirements�� 1
Getting started with Jupyter notebooks�� 2
Shortcuts and other features in Jupyter�� 5
Tab Completion�� 7
Magic commands used in Jupyter�� 7
Python Basics�� 8
Comments, print, and input�� 8
Variables and Constants�� 11
Operators�� 12
Data types�� 15
Working with Strings�� 20
Conditional statements�� 25
Loops�� 26
Functions�� 29
Syntax errors and exceptions�� 31
Working with files�� 32
Reading from a file�� 33
Writing to a file�� 34
Modules in Python�� 35
v
Table of Contents
Python Enhancement Proposal (PEP) 8 – standards for writing code�� 36

Summary�� 38
Review Exercises�� 39
Chapter 2: Exploring Containers, Classes, and Objects�� 45

Containers�� 45
Lists�� 45
Tuples�� 56
Dictionaries�� 59
Sets�� 63
Object-oriented programming�� 65
Object-oriented programming principles�� 67
Summary�� 70
Chapter 3: Regular Expressions and Math with Python�� 77

Regular expressions�� 77
Steps for solving problems with regular expressions�� 77
Python functions for regular expressions�� 79
Using Sympy for math problems�� 86
Factorization of an algebraic expression�� 86
Solving algebraic equations (for one variable)�� 87
Solving simultaneous equations (for two variables)�� 87
Solving expressions entered by the user�� 88
Solving simultaneous equations graphically�� 89
Creating and manipulating sets�� 90
Union and intersection of sets�� 90
Finding the probability of an event�� 91
Solving questions in calculus�� 92
Summary�� 94
vi
Table of Contents
Chapter 4: Descriptive Data Analysis Basics�� 101

Descriptive data analysis - Steps�� 101
Structure of data�� 104
Classifying data into different levels�� 104
Visualizing various levels of data�� 106
Plotting mixed data�� 110
Summary�� 113
Chapter 5: Working with NumPy Arrays�� 117

Getting familiar with arrays and NumPy functions�� 117
Creating an array�� 118
Reshaping an array�� 121
Combining arrays�� 125
Testing for conditions�� 127
Broadcasting, vectorization, and arithmetic operations�� 130
Obtaining the properties of an array�� 133
Slicing or selecting a subset of data�� 136
Obtaining descriptive statistics/aggregate measures�� 138
Matrices�� 140
Summary�� 140
Chapter 6: Prepping Your Data with Pandas�� 147

Pandas at a glance�� 147
Building blocks of Pandas�� 149
Examining the properties of a Series�� 152
DataFrames�� 156
Creating DataFrames by importing data from other formats�� 158
Accessing attributes in a DataFrame�� 160
Modifying DataFrame objects�� 161
vii
Table of Contents
Indexing�� 169
Type of an index object�� 170
Creating a custom index and using columns as indexes�� 171
Indexes and speed of data retrieval�� 173
Immutability of an index�� 174
Alignment of indexes�� 176
Set operations on indexes�� 177
Data types in Pandas�� 178
Obtaining information about data types�� 179
Indexers and selection of subsets of data�� 182
Understanding loc and iloc indexers�� 183
Other (less commonly used) indexers for data access�� 188
Boolean indexing for selecting subsets of data�� 192
Using the query method to retrieve data�� 192
Operators in Pandas�� 193
Representing dates and times in Pandas�� 194
Converting strings into Pandas Timestamp objects�� 195
Extracting the components of a Timestamp object�� 196
Grouping and aggregation�� 197
Examining the properties of the groupby object�� 199
Filtering groups�� 201
Transform method and groupby�� 202
Apply method and groupby�� 204
How to combine objects in Pandas�� 204
Append method for adding rows�� 205
Concat function (adding rows or columns from other objects)�� 207
Join method – index to index�� 210
Merge method – SQL type join based on common columns�� 211
viii
Table of Contents
Restructuring data and dealing with anomalies�� 213

Dealing with missing data�� 214
Data duplication�� 218
Tidy data and techniques for restructuring data�� 220
Conversion from wide to long format (tidy data)�� 221
Stack method (wide-to-long format conversion)�� 223
Melt method (wide-to-long format conversion)�� 226
Pivot method (long-to-wide conversion)�� 228
Summary�� 229
Chapter 7: Data Visualization with Python Libraries�� 243

External files�� 244
Commonly used plots�� 245
Matplotlib�� 248
Approach for plotting using Matplotlib�� 251
Plotting using Pandas�� 253
Scatter plot�� 254
Histogram�� 255
Pie charts�� 256
Seaborn library�� 257
Box plots�� 258
Adding arguments to any Seaborn plotting function�� 259
Kernel density estimate�� 259
Violin plot�� 260
Count plots�� 261
Heatmap�� 262
Facet grid�� 263
Regplot�� 265
ix
Table of Contents
lmplot�� 266
Strip plot�� 267
Swarm plot�� 268
Catplot�� 269
Pair plot�� 270
Joint plot�� 272
Summary�� 273
Chapter 8: Data Analysis Case Studies�� 279

Methodology�� 280
Case study 8-1: Highest grossing movies in France – analyzing unstructured data�� 281
Case study 8-2: Use of data analysis for air quality management�� 288
Case study 8-3: Worldwide COVID-19 cases – an analysis�� 308
Summary�� 320
Chapter 9: Statistics and Probability with Python�� 325

Permutations and combinations�� 325
Probability�� 327
Rules of probability�� 328
Conditional probability�� 330
Bayes theorem�� 330
Application of Bayes theorem in medical diagnostics�� 331
Another application of Bayes theorem: Email spam classification�� 333
SciPy library�� 334
Probability distributions�� 335
Binomial distribution�� 335
Poisson distribution�� 338
Continuous probability distributions�� 341
x
Table of Contents
Normal distribution�� 341

Standard normal distribution�� 343
Measures of central tendency�� 347
Measures of dispersion�� 348
Measures of shape�� 349
Sampling�� 353
Probability sampling�� 353
Non-probability sampling�� 354
Central limit theorem�� 355
Estimates and confidence intervals�� 356
Types of errors in sampling�� 357
Hypothesis testing�� 358
Basic concepts in hypothesis testing�� 358
Key terminology used in hypothesis testing�� 359
Steps involved in hypothesis testing�� 361
One-sample z-test�� 362
Two-sample sample z-test�� 364
Hypothesis tests with proportions�� 366
Two-sample z-test for the population proportions�� 368
T-distribution�� 370
One sample t-test�� 372
Two-sample t-test�� 372
Two-sample t-test for paired samples�� 373
Solved examples: Conducting t-tests using Scipy functions�� 373
ANOVA�� 376
Chi-square test of association�� 379
Summary�� 383
Index�� 393
xi
About the Author
Gayathri Rajagopalan works for a leading Indian
multinational organization, with ten years of experience
in the software and information technology industry.
She has degrees in computer engineering and business
adminstration, and is a certified Project Management
Professional (PMP). Some of her key focus areas include
Python, data analytics, machine learning, statistics, and
deep learning. She is proficient in Python, Java, and C/C++
programming. Her hobbies include reading, music, and
teaching programming and data science to beginners.
xiii
About the Technical Reviewer
Manohar Swamynathan is a data science practitioner
and an avid programmer, with over 14 years of experience
in various data science related areas that include data
warehousing, Business Intelligence (BI), analytical tool
development, ad hoc analysis, predictive modeling, data
science product development, consulting, formulating
strategy, and executing analytics programs. He’s had a
career covering the life cycle of data across different
domains such as US mortgage banking, retail/ecommerce,
insurance, and industrial IoT. He has a bachelor’s degree
with a specialization in physics, mathematics, and
computers, and a master’s degree in project management. He’s currently living in
Bengaluru, the Silicon Valley of India.
xv
Acknowledgments
This book is a culmination of a year-long effort and would not have been possible
without my family’s support. I am indebted to them for their patience, kindness, and
encouragement.
I would also like to thank my readers for investing their time and money in this book. It is
my sincere hope that this book adds value to your learning experience.
xvii
Introduction
I had two main reasons for writing this book. When I first started learning data science,
I could not find a centralized overview of all the important topics on this subject.
A practitioner of data science needs to be proficient in at least one programming
language, learn the various aspects of data preparation and visualization, and also
be conversant with various aspects of statistics. The goal of this book is to provide
a consolidated resource that ties these interconnected disciplines together and
introduces these topics to the learner in a graded manner. Secondly, I wanted to provide
material to help readers appreciate the practical aspects of the seemingly abstract
concepts in data science, and also help them to be able to retain what they have learned.
There is a section on case studies to demonstrate how data analysis skills can be applied
to make informed decisions to solve real-world challenges. One of the highlights of
this book is the inclusion of practice questions and multiple-choice questions to help
readers practice and apply whatever they have learned. Most readers read a book and
then forget what they have read or learned, and the addition of these exercises will help
readers avoid this pitfall.
The book helps readers learn three important topics from scratch – the Python
programming language, data analysis, and statistics. It is a self-contained introduction
for anybody looking to start their journey with data analysis using Python, as it focuses
not just on theory and concepts but on practical applications and retention of concepts.
This book is meant for anybody interested in learning Python and Python-based libraries
like Pandas, Numpy, Scipy, and Matplotlib for descriptive data analysis, visualization,
and statistics. The broad categories of skills that readers learn from this book include
programming skills, analytical skills, and problem-solving skills.
The book is broadly divided into three parts – programming with Python, data analysis
and visualization, and statistics. The first part of the book comprises three chapters. It
starts with an introduction to Python – the syntax, functions, conditional statements,
data types, and different types of containers. Subsequently, we deal with advanced
concepts like regular expressions, handling of files, and solving mathematical problems
xix
Introduction
with Python. Python is covered in detail before moving on to data analysis to ensure that
the readers are comfortable with the programming language before they learn how to
use it for purposes of data analysis.
The second part of the book, comprising five chapters, covers the various aspects of
descriptive data analysis, data wrangling and visualization, and the respective Python
libraries used for each of these. There is an introductory chapter covering basic concepts
and terminology in data analysis, and one chapter each on NumPy (the scientific
computation library), Pandas (the data wrangling library), and the visualization
libraries (Matplotlib and Seaborn). A separate chapter is devoted to case studies to
help readers understand some real-world applications of data analysis. Among these
case studies is one on air pollution, using data drawn from an air quality monitoring
station in New Delhi, which has seen alarming levels of pollution in recent years. This
case study examines the trends and patterns of major air pollutants like sulfur dioxide,
nitrogen dioxide, and particulate matter for five years, and comes up with insights and
recommendations that would help with designing mitigation strategies.
The third section of this book focuses on statistics, elucidating important principles in
statistics that are relevant to data science. The topics covered include probability, Bayes
theorem, permutations and combinations, hypothesis testing (ANOVA, chi-squared
test, z-test, and t-test), and the use of various functions in the Scipy library to enable
simplification of tedious calculations involved in statistics.
By the end of this book, the reader will be able to confidently write code in Python, use
various Python libraries and functions for analyzing any dataset, and understand basic
statistical concepts and tests. The code is presented in the form of Jupyter notebooks
that can further be adapted and extended. Readers get the opportunity to test their
understanding with a combination of multiple-choice and coding questions. They
also get an idea about how to use the skills and knowledge they have learned to make
evidence-based decisions for solving real-world problems with the help of case studies.
xx
CHAPTER 1
Getting Familiar
with Python
Python is an open source programming language created by a Dutch programmer
named Guido van Rossum. Named after the British comedy group Monty Python,
Python is a high-level, interpreted, open source language and is one of the most sought-
after and rapidly growing programming languages in the world today. It is also the
language of preference for data science and machine learning.
In this chapter, we first introduce the Jupyter notebook – a web application for running
code in Python. We then cover the basic concepts in Python, including data types,
operators, containers, functions, classes and file handling and exception handling, and
standards for writing code and modules.
The code examples for this book have been written using Python version 3.7.3 and
Anaconda version 4.7.10.
T echnical requirements
Anaconda is an open source platform used widely by Python programmers and data
scientists. Installing this platform installs Python, the Jupyter notebook application, and
hundreds of libraries. The following are the steps you need to follow for installing the
Anaconda distribution.
1. Open the following URL: https://www.anaconda.com/products/

individual
2. Click the installer for your operating system, as shown in Figure 1-1.
The installer gets downloaded to your system.
1
© Gayathri Rajagopalan 2021
G. Rajagopalan, A Python Data Analyst’s Toolkit, https://doi.org/10.1007/978-1-4842-6399-0_1
Chapter 1 Getting Familiar with Python
Figure 1-1. Installing Anaconda
3. Open the installer (file downloaded in the previous step) and run it.
4. After the installation is complete, open the Jupyter application

by typing “jupyter notebook” or “jupyter” in the explorer (search
bar) next to the start menu, as shown in Figure 1-2 (shown for
Windows OS).
Figure 1-2. Launching Jupyter
Please follow the following steps for downloading all the data files used in this book:
• Click the following link: https://github.com/DataRepo2019/

Data-files
• Select the green “Code” menu and click on “Download ZIP” from the
dropdown list of this menu
• Extract the files from the downloaded zip folder and import these
files into your Jupyter application
Now that we have installed and launched Jupyter, let us understand how to use this
application in the next section.
Getting started with Jupyter notebooks

Before we discuss the essentials of Jupyter notebooks, let us discuss what an integrated
development environment (or IDE) is. An IDE brings together the various activities
involved in programming, like including writing and editing code, debugging, and
2
creating executables. It also includes features like autocompletion (completing what

the user wants to type, thus enabling the user to focus on logic and problem-solving)
and syntax highlighting (highlighting the various elements and keywords of the
language). There are many IDEs for Python, apart from Jupyter, including Enthought
Canopy, Spyder, PyCharm, and Rodeo. There are several reasons for Jupyter becoming
a ubiquitous, de facto standard in the data science community. These include ease
of use and customization, support for several programming languages, platform
independence, facilitation of access to remote data, and the benefit of combining output,
code, and multimedia under one roof.
JupyterLab is the IDE for Jupyter notebooks. Jupyter notebooks are web applications that
run locally on a user’s machine. They can be used for loading, cleaning, analyzing, and
modeling data. You can add code, equations, images, and markdown text in a Jupyter
notebook. Jupyter notebooks serve the dual purpose of running your code as well as
serving as a platform for presenting and sharing your work with others. Let us look at the
various features of this application.
1. Opening the dashboard
Type “jupyter notebook” in the search bar next to the start menu.
This will open the Jupyter dashboard. The dashboard can be used
to create new notebooks or open an existing one.
2. Creating a new notebook
Create a new Jupyter notebook by selecting New from the upper

right corner of the Jupyter dashboard and then select Python 3
from the drop-down list that appears, as shown in Figure 1-3.
Figure 1-3. Creating a new Jupyter notebook
3. Entering and executing code
Click inside the first cell in your notebook and type a simple line
of code, as shown in Figure 1-4. Execute the code by selecting Run
Cells from the “Cell” menu, or use the shortcut keys Ctrl+Enter.
3
Figure 1-4. Simple code statement in a Jupyter cell
4. Adding markdown text or headings

In the new cell, change the formatting by selecting Markdown
as shown in Figure 1-5, or by pressing the keys Esc+M on your
keyboard. You can also add a heading to your Jupyter notebook by
selecting Heading from the drop-down list shown in the following
or pressing the shortcut keys Esc+(1/2/3/4).
Figure 1-5. Changing the mode to Markdown
5. Renaming a notebook
Click the default name of the notebook and type a new name, as
shown in Figure 1-6.
Figure 1-6. Changing the name of a file
You can also rename a notebook by selecting File ➤ Rename.

6. Saving a notebook
Press Ctrl+S or choose File ➤ Save and Checkpoint.
7. Downloading the notebook
You can email or share your notebook by downloading your
notebook using the option File ➤ Download as ➤ notebook
(.ipynb), as shown in Figure 1-7.
4
Figure 1-7. Downloading a Jupyter notebook
Shortcuts and other features in Jupyter

Let us look at some key features of Jupyter notebooks, including shortcuts, tab
completions, and magic commands.
Table 1-1 gives some of the familiar icons found in Jupyter notebooks, the corresponding
menu functions, and the keyboard shortcuts.
5
Table 1-1. Jupyter Notebook Toolbar Functions

Icon in Toolbar Function Keyboard shortcut Menu function
Saving a Jupyter notebook Esc+s File ➤ Save as
Adding a new cell to a Esc+b (adding a cell below the Insert ➤ Insert Cell
Jupyter notebook current cell), or Esc+a (adding Above or Insert ➤
a cell above the current cell) Insert Cell Below
Cutting a selected cell Esc+x Edit ➤ Cut Cells
Copying the selected cell Esc+c Edit ➤ Copy Cells
Pasting a cell above or Esc+v Edit ➤ Paste Cells

below another selected cell Above or Edit ➤
Paste Cells Below
Running a given cell Ctrl+Enter (to run selected cell); Cell ➤ Run
Shift+Enter (to run selected cell Selected Cells
and insert a new cell)
Interrupting the kernel Esc+ii Kernel ➤ Interrupt
Rebooting the kernel Esc+00 Kernel ➤ Restart
If you are not sure about which keyboard shortcut to use, go to: Help ➤ Keyboard
Shortcuts, as shown in Figure 1-8.
Figure 1-8. Help menu in Jupyter

6
Commonly used keyboard shortcuts include
• Shift+Enter to run the code in the current cell and move to the next
cell.
• Esc to leave a cell.
• Esc+M changes the mode for a cell to “Markdown” mode.
• Esc+Y changes the mode for a cell to “Code”.
T ab Completion
This is a feature that can be used in Jupyter notebooks to help you complete the code
being written. Usage of tab completions can speed up the workflow, reduce bugs, and
quickly complete function names, thus reducing typos and saving you from having to
remember the names of all the modules and functions.
For example, if you want to import the Matplotlib library but don’t remember the
spelling, you could type the first three letters, mat, and press Tab. You would see a drop-
down list, as shown in Figure 1-9. The correct name of the library is the second name in
the drop-down list.
Figure 1-9. Tab completion in Jupyter
Magic commands used in Jupyter

Magic commands are special commands that start with one or more % signs, followed by
a command. The commands that start with one % symbol are applicable for a single line
of code, and those beginning with two % signs are applicable for the entire cell (all lines
of code within a cell).
7
One commonly used magic command, shown in the following, is used to display
Matplotlib graphs inside the notebook. Adding this magic command avoids the need
to call the plt.show function separately for showing graphs (the Matplotlib library is
discussed in detail in Chapter 7).
CODE:
%matplotlib inline
Magic commands, like timeit, can also be used to time the execution of a script, as shown
in the following.
CODE:
%%timeit
for i in range(100000):
i*i
Output:
16.1 ms ± 283 μs per loop (mean ± std. dev. of 7 runs, 100 loops each)
Now that you understand the basics of using Jupyter notebooks, let us get started with
Python and understand the core aspects of this language.
P
ython Basics
In this section, we get familiar with the syntax of Python, commenting, conditional
statements, loops, and functions.
Comments, print, and input

In this section, we cover some basics like printing, obtaining input from the user, and
adding comments to help others understand your code.
C
omments
A comment explains what a line of code does, and is used by programmers to help others
understand the code they have written. In Python, a comment starts with the # symbol.
8
Proper spacing and indentation are critical in Python. While other languages like Java
and C++ use brackets to enclose blocks of code, Python uses an indent of four spaces
to specify code blocks. One needs to take care of indents to avoid errors. Applications
like Jupyter generally take care of indentation and automatically add four spaces at the
beginning of a block of code.
Printing
The print function prints content to the screen or any other output device.
Generally, we pass a combination of strings and variables as arguments to the print

function. Arguments are the values included within the parenthesis of a function, which
the function uses for producing the result. In the following statement, “Hello!” is the
argument to the print function.
CODE:
print("Hello!")
To print multiple lines of code, we use triple quotes at the beginning and end of the
string, for example:
CODE:
print('''Today is a lovely day.

It will be warm and sunny.
It is ideal for hiking.''')
Output:
Today is a lovely day.

It will be warm and sunny.
It is ideal for hiking.
Note that we do not use semicolons in Python to end statements, unlike some other
languages.
The format method can be used in conjunction with the print method for embedding
variables within a string. It uses curly braces as placeholders for variables that are passed
as arguments to the method.
Let us look at a simple example where we print variables using the format method.
9
CODE:
weight=4.5
name="Simi"
print("The weight of {} is {}".format(name,weight))
Output:
The weight of Simi is 4.5
The preceding statement can also be rewritten as follows without the format method:
CODE:
print("The weight of",name,"is","weight")
Note that only the string portion of the print argument is enclosed within quotes. The name
of the variable does not come within quotes. Similarly, if you have any constants in your
print arguments, they also do not come within quotes. In the following example, a Boolean
constant (True), an integer constant (1), and strings are combined in a print statement.
CODE:
print("The integer equivalent of",True,"is",1)
Output:
The integer equivalent of True is 1
The format fields can specify precision for floating-point numbers. Floating-point
numbers are numbers with decimal points, and the number of digits after the decimal
point can be specified using format fields as follows.
CODE:
x=91.234566
print("The value of x upto 3 decimal points is {:.3f}".format(x))
Output:
The value of x upto 3 decimal points is 91.235
We can specify the position of the variables passed to the method. In this example, we
use position “1” to refer to the second object in the argument list, and position “0” to
specify the first object in the argument list.
10
CODE:
y='Jack'
x='Jill'
print("{1} and {0} went up the hill to fetch a pail of water".format(x,y))
Output:
Jack and Jill went up the hill to fetch a pail of water
I nput
The input function accepts inputs from the user. The input provided by the user is stored
as a variable of type String. If you want to do any mathematical calculations with any
numeric input, you need to change the data type of the input to int or float, as follows.
CODE:
age=input("Enter your age:")

print("In 2010, you were",int(age)-10,"years old")
Output:
Enter your age:76

In 2010, you were 66 years old
Further reading on Input/Output in Python: https://docs.python.org/3/tutorial/

inputoutput.html
V
ariables and Constants
A constant or a literal is a value that does not change, while a variable contains a value
can be changed. We do not have to declare a variable in Python, that is, specify its data
type, unlike other languages like Java and C/C++. We define it by giving the variable a
name and assigning it a value. Based on the value, a data type is automatically assigned
to it. Values are stored in variables using the assignment operator (=). The rules for
naming a variable in Python are as follows:
• a variable name cannot have spaces
• a variable cannot start with a number
11
• a variable name can contain only letters, numbers, and underscore

signs (_)
• a variable cannot take the name of a reserved keyword (for example,

words like class, continue, break, print, etc., which are predefined
terms in the Python language, have special meanings, and are invalid
as variable names)
Operators
The following are some commonly used operators in Python.
Arithmetic operators: Take two integer or float values, perform an operation, and return
a value.
The following arithmetic operators are supported in Python:
• **(Exponent)
• %(modulo or remainder),
• //(quotient),
• *(multiplication)
• -(subtraction)
• +(addition)
The order of operations is essential. Parenthesis takes precedence over exponents,

which takes precedence over division and multiplication, which takes precedence
over addition and subtraction. An acronym was designed - P.E.D.M.A.S.(Please Excuse
My Dear Aunt Sally) - that can be used to remember the order of these operations to
understand which operator first needs to be applied in an arithmetic expression. An
example is given in the following:
CODE:
(1+9)/2-3
Output:
2.0
12
In the preceding expression, the operation inside the parenthesis is performed first,
which gives 10, followed by division, which gives 5, and then subtraction, which gives the
final output as 2.
Comparison operators: These operators compare two values and evaluate to a true or
false value. The following comparison operators are supported in Python:
• >: Greater than
• < : Less than
• <=: Less than or equal to
• >=: Greater than or equal to
• == : equality. Please note that this is different from the assignment
operator (=)
• !=(not equal to)
Logical (or Boolean) operators: Are similar to comparison operators in that they
also evaluate to a true or false value. These operators operate on Boolean variables or
expressions. The following logical operators are supported in Python:
• and operator: An expression in which this operator is used evaluates

to True only if all its subexpressions are True. Otherwise, if any of
them is False, the expression evaluates to False
An example of the usage of the and operator is shown in the following.
CODE:
(2>1) and (1>3)
Output:
False
• or operator: An expression in which the or operator is used, evaluates

to True if any one of the subexpressions within the expression is True.
The expression evaluates to False if all its subexpressions evaluate to
False.
An example of the usage of the or operator is shown in the following.
CODE:
(2>1) or (1>3)
13
Another random document with
no related content on Scribd:
to about two hundred young priests and theological students on “The
Personal Qualifications for a Minister of Religion.” The address was
in no important respect different from that which would be suitable on
the same subject for an audience of theological students in England
or the United States; nor did its reception and appropriation seem
any less thorough and sincere.
After inspecting the work in drawing and water-colours of which—so
the posted notice read—“An Exhibition is given in honour of ——,”
Mrs. Ladd returned to Tokyo; but I remained to carry out my purpose
of spending a full day and night among my priestly Buddhist friends.
In our many confidential talks while we were in the relations of
teacher and pupil, the latter had avowed his life-work to be the moral
reform and improved mental culture of the priesthood of his sect. It
had then seemed to me a bold, even an audacious undertaking. But
seeming audacity was quite characteristic of the youth of all those
very men who now, in middle life and old-age, are holding the posts
of leadership in Japan in a way to conserve the best results of the
earlier period of more rapid change. Besides, I knew well that my
pupil had the necessary courage and devotion; for he was not only a
priest but also a soldier, and had been decorated for his bravery in
the Chino-Japanese war. And again, toward the close of the Russo-
Japanese war, when he had been called out with the reserves, he
had once more left the position of priestly student and teacher to
take his place at arms in the defence of his country.
How wholesome and thoroughly educative of their whole manhood
was the training which was being given to these young temple boys,
I had abundant reason to know before leaving the Nichiren College
at Osaki. After tea and welcome-addresses by one of the teachers
and two of the pupils, followed by a response by the guest, an
exhibition of one side of this training was given in the large dining-
hall of the school. For as it was in ancient Greece, so it is now in
Japan; arms and music must not be neglected in the preparation to
serve his country of the modern Buddhist priest. Sword-dancing—
one of the chants which accompanied the action being Saigo’s
celebrated “death song”—and a duet performed upon a flute and a
harp constructed by the performer out of split bamboo and strings of
silk, followed by banzais for their guest, concluded the
entertainment.
Of the nine who sat down to dinner that evening in a private room
belonging to another building of the school, four besides the host
were priests of the Nichiren sect. They constituted the body of the
more strictly religious or theological instructors; the courses in
literature and the sciences being taught for the most part by
professors from the Imperial University or from the private university
founded by Japan’s great teacher of youth, the late Mr. Fukuzawa.
Of the priests the most conspicuous and communicative was proud
to inform me that he had been the chaplain of General Noghi at the
siege of Port Arthur. With reference to the criticisms passed at the
time upon that great military leader he said with evident emotion that
General Noghi was “as wise as he was undoubtedly brave.” This
same priest had also interesting stories to tell of his experiences in
China. In speaking of the ignorance of the teachers of religion in that
country he declared, that of the hundreds of Tâoist priests he had
met, the vast majority could not even read the Chinese ideographs
when he wrote them; and none of the numbers he had known could
make any pretence to scholarship. They were quite universally
ignorant, superstitious, and physically and morally filthy. Among the
Buddhist priests in China, however, the case was somewhat better;
for perhaps three or four in every ten could make some pretence of
education; and there were even a very few who were real scholars.
But neither Tâoists nor Buddhists had much influence for good over
the people; and “priest, priest,” was a cry of insult with which to
follow one. As to their sincerity, at one of the Tâoist temples he had
asked for meat and wine, but had been told that none could be had,
because they abstained religiously from both. But when he replied
that he had no scruples against either, but needed them for his
health and wished to pay well for them, both were so quickly
produced he knew they could not have come from far away. (I may
remark in this connection that if the experiences and habits of the
Chinese in Manchuria resemble at all closely the experiences and
customs of the Koreans in their own country, the unwillingness to
furnish accommodations to travelling strangers is caused rather by
the fear of having them requisitioned without pay than to any
scruples, religious or otherwise, as to what they themselves eat and
drink or furnish to others for such purposes).
The same subject which had been introduced at the priests-house,
on occasion of the all-night festival at Ikegami, was now brought
forward again. What had been my impressions received from the
spectacle witnessed at that time? When to the inquiry I made a
similar answer,—namely, that only a portion of the vast crowd
seemed to be sincere worshippers, but that with the exception of a
few rude young men in the procession, who appeared to have had
too much saké, I saw no immoral or grossly objectional features—all
the priests expressed agreement with my views. Where the
superstitions connected with the celebration were not positively
harmful, it was the policy of the reforming and progressive party of
the sect to leave them to die away of themselves as the people at
large became more enlightened.
After a night of sound sleep, Japanese fashion, on the floor of the
study in my pupil’s pretty new home, we rose at six and hastened
across the fields to attend the morning religious services in the
chapel of the school. Here for a full half-hour, or more, what had
every appearance of serious and devout religious worship was held
by the assembled teachers and pupils. All were neatly dressed in
black gowns; no evidences of having shuffled into unbrushed
garments, with toilets only half-done or wholly neglected, were
anywhere to be seen, nor was there the vacant stare, the loud
whisper, the stolen glance at newspaper or text-book; but all
responded to the sutras and intoned the appointed prayers and
portions of the Scriptures, while the time was accented by the not too
loud beating of a musical gong. Certainly, the orderliness and
apparent devotion quite exceeded that of any similar service at
“morning prayers” in the average American college or university.
A brief exhibition of judo, (a modified form of jiujitsu), and of
Japanese fencing, which was carried on in the dining-room while the
head-master was exchanging his priestly for his military dress, in
order to take part in a memorial service to deceased soldiers, at
which General Noghi was expected to be present, terminated my
entertainment at this Buddhist school for the training of temple boys.
As we left the crowd of them who had accompanied us thus far on
the way, and stood shouting banzais on the platform of the station,
there was no room for doubting the heartiness of their friendly feeling
toward the teacher of their teacher; although the two, while sharing
many of the most important religious views, were called by names
belonging to religions so different as Christianity and Buddhism.
The impressions from these two visits to Ikegami regarding the
changes going on in Buddhistic circles in Japan, and in the attitude
of Buddhism toward Christianity, were amply confirmed by
subsequent experiences. At Kyoto, the ancient capital and religious
centre of the empire, I was invited by the Dean of the Theological
Seminary connected with the Nishi Honwangi to address some six
hundred young priests of various sects on the same topic as that on
which the address was given at the Nichiren College near Ikegami. It
should be explained that this temple is under the control of the Shin-
shu, the most numerous and probably the most wealthy sect in the
Empire. The high priest of this sect is an hereditary count and
therefore a member of the House of Peers. He is also a man of
intelligence and of a wide-spreading interest in religion. At the time of
my visit, indeed, the Count was absent on a missionary tour in
China. This address also was listened to with the same respectful
attention by the several hundred Buddhist priests who had gathered
at the temple of Nishi Honwangi. Here again Mrs. Ladd and I were
made the recipients of the same courteous and unique hospitality.
Before the lecture began, we were entertained in the room which
had been distinguished for all time in the estimate of the nation by
the fact that His Majesty the Emperor held within its walls the first
public reception ever granted to his subjects by the Mikado; and after
the lecture we were further honoured by being the first outsiders ever
invited to a meal with the temple officers within one of the temple
apartments.
Later on at Nagoya, further evidence was afforded of the important
fact that the old-time religious barriers are broken down or are being
overridden, wherever the enlightenment and moral welfare of the
people seem likely to be best served in this way. Now Nagoya has
hitherto been considered one of the most conservative and even
bigoted Buddhist centres in all Japan. Yet a committee composed of
Buddhists and of members of the Young Men’s Christian Association
united in arrangements for a course of lectures on education and
ethics. This was remarked upon as the first instance of anything of
the sort in the history of the city.
When we seek for the causes which have operated to bring about
these important and hopeful changes in the temper and practises of
the Buddhism which is fast gaining currency and favour in Japan, we
are impressed with the belief that the greatest of them is the
introduction of Christianity itself. This influence is obvious in the
following three essential ways. Christian conceptions and doctrines
are modifying the tenets of the leading Buddhistic thinkers in Japan.
As I listened for several hours to his exposition of his conception of
the Divine Being, the divine manner of self-revelation, and of his
thoughts about the relations of God and man, by one of the most
notable theologians of the Shin Shu (the sect which I have already
spoken of as the most popular in Japan), I could easily imagine that
the exponent was one of the Alexandrine Church-Fathers, Origen or
Clement, discoursing of God the Unrevealed and of the Logos who
was with God and yet who became man. But Buddhism is also giving
much more attention than formerly to raising the moral standards of
both priests and people. It is sharing in the spirit of ethical quickening
and revival which is so important an element of the work of Christian
missions abroad, but which is alas! so woefully neglected in the so-
called Christian nations at home. Japanese Buddhism is feeling now
much more than formerly the obligation of any religion which asks
the adherence and support of the people, to help the people, in a
genuine and forceful way, to a nobler and better way of living.
Hitherto in Japan it has been that peculiar development of Confucian
ethics called Bushidō, which has embodied and cultivated the nobler
moral ideals. Religion, at least in the form which Buddhism has taken
in Japan, has had little to do with inspiring and guiding men in the life
which is better and best, here and now. But as its superstitions with
regard to the future are falling away and are ceasing practically to
influence the body of the people, there are some gratifying signs that
its influence upon the spiritual interests of the present is becoming
purer and stronger.
That Buddhism is improving its means of educating its followers, and
is feeling powerfully the quickening of the national pulse, due to the
advancing strides in educational development, is obvious enough to
any one able to compare its condition to-day with its condition not
more than a score of years ago. There are, of course, in the ranks of
all the Buddhist sects leaders who are ready to cry out against
heresies and the mischief of changes concealed under the guise of
reforms. The multitudes of believers are still far below the desirable
standard of either intelligence in religious matters, or of morals as
controlled by religious motives. But the old days of stagnation and
decay seem to be passing away; and the outlook now is that the
foreign religion, instead of speedily destroying the older native
religion, will have helped it to assume a new and more vigorous and
better form of life.
As the period of more bitter conflict and mutual denunciation gives
way to a period of more respectful and friendly, and even co-
operative attitude in advancing the welfare of the nation, the future of
both Buddhism and Christianity in Japan affords a problem of more
complicated and doubtful character. The nation is awakening to its
need of morals and religion,—in addition to a modern army and
navy, and to an equipment for teaching and putting to practical uses,
the physical sciences,—as never before. The awakening is
accompanied there, as elsewhere in the modern world, by a thirst for
reality. Whatever can satisfy this thirst, however named, will find
acceptance and claim the allegiance of both the thoughtful and the
multitudes of the common people; for in Japan, as elsewhere in the
modern world, men are not easily satisfied or permanently satisfied
with mere names.
CHAPTER X
HIKONÉ AND ITS PATRIOT MARTYR
Among the feudal towns of Japan which can boast of a fine castle
still standing, and of an illustrious lord as its former occupant, there
are few that can rival Hikoné. Picturesquely seated on a wooded hill
close to the shores of Lake Biwa, with the blue waters and almost
equally blue surrounding mountains in full sight, the castle enjoys the
advantages of strength combined with beauty; while the lords of the
castle are descended from a very ancient family, which was awarded
its territory by the great Iyéyasu, the founder of the Tokugawa
Shōgunate, in return for the faithful services of their ancestor,
Naomasa, in bringing the whole land under the Tokugawa rule. They
therefore belonged to the rank of the Fudai Daimio, or Retainer
Barons, from whom alone the Roju, or Senators, and other officers of
the first class could be appointed. Of these lords of Hikoné much the
most distinguished was Naosuké, who signed the treaty with the
United States negotiated in 1857 and 1858. And yet, so strange are
the vicissitudes of history, and so influential the merely incidental
occurrences in human affairs, that only a chance visit of the Mikado
saved this fine feudal castle from the “general ruin of such buildings
which accompanied the mania for all things European and the
contempt of their national antiquities, whereby the Japanese were
actuated during the past two decades of the present régime.” Nor
was it until recent years that Baron Ii Naosuké’s memory has been
rescued from the charge of being a traitor to his country and a
disobedient subject of its Emperor, and elevated to a place of
distinction and reverence, almost amounting to worship, as a clear-
sighted and far-seeing statesman and patriot.
“PICTURESQUELY SEATED ON A WOODED HILL”
However we may regard the unreasonableness of either of these two
extreme views of Naosuké’s character, one thing seems clear. In
respect to the laying of foundations for friendly relations between the
United States and Japan, we owe more to this man than to any other
single Japanese. No one can tell what further delays and resulting
irritation, and even accession of blood-shed, might have taken place
in his time had it not been for his courageous and firm position
toward the difficult problem of admitting foreigners to trade and to
reside within selected treaty-ports of Japan. This position cost him
his life. For a generation, or more, it also cost him what every true
Japanese values far more highly than life; it cost the reputation of
being loyal to his sovereign and faithful to his country’s cause. Yet
not five Americans in a million, it is likely, ever heard the name of
Baron Ii Kamon-no-Kami, who as Tairō, or military dictator, shared
the responsibility and should share the fame of our now celebrated
citizen, then Consul General at Shimoda, Townsend Harris. My
purpose, therefore, is two-fold: I would gladly “have the honour to
introduce” Ii Naosuké to a larger audience of my own countrymen;
and by telling the story of an exceedingly interesting visit to Hikoné, I
would equally gladly introduce to the same audience certain ones of
the great multitude of Japanese who still retain the knightly courtesy,
intelligence and high standards of living—though in their own way—
which characterised the feudal towns of the “Old Japan,” now so
rapidly passing away.
Baron Ii Naosuké, better known in foreign annals as Ii Kamon-no-
Kami, was his father’s fourteenth son. He was born November 30,
1815. The father was the thirteenth feudal lord from that Naomasa
who received his fief from the great Iyéyasu. Since the law of
primogeniture—the only exceptions being cases of insanity or bodily
defect—was enforced throughout the Empire, the early chances that
Naosuké would ever become the head of the family and lord of
Hikoné, seemed small indeed. But according to the usage of the Ii
clan, all the sons except the eldest were either given as adopted
sons to other barons, or were made pensioned retainers of their
older brother. All his brothers, except the eldest, had by adoption
become the lords of their respective clans. But from the age of
seventeen onward, Naosuké was given a modest pension and
placed in a private residence. He thus enjoyed years of opportunity
for training in arms, literature, and reflective study, apart from the
corrupting influences of court life and the misleading temptations to
the exercise of unrestricted authority—both of which are so injurious
to the character of youth. Moreover, he became acquainted with the
common people. That was also true of him, which has been true of
so many of the great men of Japan down to the present time. He
made his friend and counsellor of a man proficient in the military and
literary education of the day. And, indeed, it has been the great
teachers who, more than any other class, through the shaping of
character in their pupils, have influenced mankind to their good. It
was Nakagawa Rokurō who showed to Naosuké, when a young
man, the impossibility of the further exclusion of Japan from foreign
intercourse. It was he also who “influenced the future Tairō to make a
bold departure from the old traditions” of the country.
On the death, without male issue, of his oldest brother, Naosuké was
declared heir-apparent of the Hikoné Baronetcy. And on Christmas
day of 1850 he was publicly authorised by the Shōgunate to assume
the lordly title of Kamon-no-Kami. It is chiefly through the conduct of
the man when, less than a decade later, he came to the position
which was at the same time the most responsible, difficult and
honourable but dangerous of all possible appointments in “Old
Japan,” that the character of Baron Ii must be judged. On the side of
sentiment—and only when approached from this side can one
properly appreciate the typical knightly character of Japanese
feudalism—we may judge his patriotism by this poem from his own
hand:
Omi no mi kishi utsu nami no iku tabimo,

Miyo ni kokoro wo kudaki nuru kana;
or as freely translated by Dr. Griffis:—
“As beats the ceaseless wave on Omi’s strand

So breaks my heart for our beloved land.”
(Omi is the poetical appellation of Lake Biwa, on which the feudal

castle of the lords of Hikoné has already been said to be situated.)
How the sincerity of this sentiment may be reconciled with the act
which for an entire generation caused the baron to be stigmatised a
traitor is made clear through the following story told by the great
Ōkubo. In the troubled year of 1858, the Viscount, just before
starting on an official errand to the Imperial Court at Kyoto, called on
Baron Ii, who was then chief in command under the Shōgun, to
inform him of his expected departure on the morrow. He had
embodied his own views regarding the vexed question of foreign
affairs, on his “pocket paper,” in the form of a poem. This paper the
Viscount handed to the Baron and asked him whether his views
were the same as those of the poem. Having carefully read it Ii
approved and instructed Ōkubo to act up to the spirit of the poem,
which reads:
“However numerous and diversified the nations of the earth may be,
the God who binds them together can never be more than one.”
Whatever differences of view prevailed, between his political
supporters and his political enemies, as to the purity of Naosuké’s
patriotic sentiments, there was little opportunity for difference as to
certain other important elements of his character. He had
conspicuously the qualities needed for taking a position of dictatorial
command in times of turbulence and extreme emergency. Serious in
purpose, but slow in making up his mind, he had undaunted firmness
in carrying out his plans, such that “no amount of difficulties would
make him falter or find him irresolute.”
The burning question of foreign intercourse which the coming of
Commodore Perry had forced upon the Shōgunate in 1853, had
afterward been referred to the barons of the land. They favoured
exclusion by a large majority; and some of them were ready to
enforce it at the expense of a foreign war. But the recent experience
of China at the hands of the allied forces was beginning to teach the
Far East that lesson of preparedness by foreign and modern
education which Japan has since so thoroughly learned; and to the
fuller magnitude of which China herself is just awakening. To take
the extreme position of complete and final resistance to the demands
of the foreign forces seemed obviously to court speedy and
inevitable ruin for the country at large. Yet none of the barons,
except the Baron of Hikoné, had a plan to propose by which to
exclude alike the peaceful foreigner come to trade and the armed
foreigner come to enforce his country’s demand for peaceful
intercourse by the use of warlike means.
It is interesting to notice that Naosuké answered the question of the
Shōgunate in a manner to indicate the consistent policy of his
country from 1853 down to the present time. He did not, it is
probable, love or admire the personality of the foreign invader more
than did his brother barons; or more than does the average Chinese
official at the present time. On consulting with his own retainers, he
found the “learned Nakagawa” the sole supporter of his views. All the
clan, with the exception of this teacher and scholar, favoured
exclusion at any cost. “The frog in the well knows not the great
ocean,” says the Japanese proverb. And as to the Japanese people,
who at that time were kept “in utter ignorance of things outside of
their own country,” Count (now Prince) Yamagata said in 1887, with
reference to the superior foresight of Baron Ii: “Their condition was
like that of a frog in a well.”
In spite of the almost complete loneliness of his position among the
barons of the first rank, Naosuké advised the Shōgunate that the
tendencies of the times made it impossible longer to enforce the
traditional exclusiveness of Japan. But he also—and this is most
significant of his far-sighted views—advised the repeal of the law,
issued early in the seventeenth century, which prohibited the building
of vessels large enough for foreign trade; and this advice he coupled
with the proposal that Japan should build navies for the protection, in
future, of her own coasts. “Thus prepared,” he writes, “the country
will be free from the menaces and threatenings of foreign powers,
and will be able to uphold the national principle and polity at any
time.”
The division of opinion, and the bloody strifes of political parties, in
Japan, over the question of exclusion were not settled by the
Convention for the relief of foreign ships and sailors which followed
upon the return of the war-ships of the United States, and of other
foreign countries, in 1854. Quite the contrary was the truth. When
Mr. Townsend Harris arrived as Consul General in 1856, and began
to press the question of foreign trade and residence in a more
definite form, the party favouring exclusion was stronger, more bitter,
and more extreme than before. In their complete ignorance of the
very nature of a commercial treaty, the rulers of Japan quite
generally mistook the American demand to open Kanagawa, Yedo,
Osaka, Hiogo, and Niigata for an extensive scheme of territorial
aggression. This they were, of course, ready to resist to their own
death and to the ruin of the country. When the senators prepared a
memorial to the Imperial Cabinet, stating their difficulty and the
necessity of conforming to the foreign demand, and sent it to the
Imperial Capital by the hand of their president, Baron Hotta, they
were therefore instructed to delay, and to consult further with the
Tokugawa Family and with the Barons of the land, before again even
venturing to refer the matter to the Government at Kyoto. These
instructions were, under the circumstances, equivalent to a flat and
most dangerous refusal to allow the opening of the country at all.
It has not been generally recognised in his own country, how
extremely important and yet how difficult was the position of Mr.
Townsend Harris during the years, 1857-1858. Nor has he, in my
judgment, been awarded his full relative share of credit for laying in
friendly foundations the subsequent commercial and other forms of
intercourse between the United States and Japan. Mr. Harris’ task
was in truth larger and more complicated than that of Commodore
Perry. The factors of Japanese politics opposed to its
accomplishment were more manifold and vehement. Moreover, the
question of foreign intercourse was then complicated by two other
questions of the most portentous magnitude for the internal politics
and political development of Japan. These were, the question of who
should be the heir-apparent to the then ruling Shōgun; and the yet
more important, and even supremely important question of how the
Shōgunate should in the future stand related to the virtual—and not
merely nominal—supremacy of the Imperial House. The opposition
on both these questions was substantially the same as the
opposition to permitting foreign trade and residence in the land. If
then Commodore Perry deserves the gratitude of all for making the
first approaches, in a way without serious disruption and lasting
hatred, to begin the difficult task of opening Japan, Townsend Harris
certainly deserves no less gratitude for enlarging and shaping into
more permanent form the same “opening,” while quite as skilfully
and effectively avoiding the exasperation of similar and even greater
political evils.
His many embarrassments forced upon the somewhat too timid and
hesitating Shōgun the necessity of selecting some one man upon
whom the responsibility and the authority for decisive action could be
confidently reposed. Seeing this man in the person only of Ii Kamon-
no-Kami, Lord of Hikoné, he appointed him to the position of Tairō.
Now, this position of Tairō, or “Great Elder,” which may be
paraphrased by “President-Senator,” was one of virtual dictatorship.
Only the Shōgun, who appointed him, could remove the Tairō or
legally resist his demands. Naosuké was the last to hold this office;
for fortunately for Japan the Shōgunate itself soon came to an end;
but he will be known in history as Go-Tairō,—the dictator especially
to be honoured, because he was bold, clear-sighted, and ready to
die in his country’s behalf. On June 5, 1858, Baron Ii was installed in
the position which gave him the power to conclude the treaty, and
which at the same time made him responsible for its consequences
of weal or woe, to individuals and to the entire nation,—even to the
world at large. In this important negotiation the Japanese Baron
Naosuké, and the American gentleman, Harris, were henceforth the
chief actors.
It is not my intention to recite in detail the history of the negotiations
of 1858, or of the difficulties and risks which the Tairō had to face in
his conduct of them. While the Mikado’s sanction for concluding the
treaty with Mr. Harris was still anxiously awaited, two American men-
of-war arrived at Shimoda; and a few days later these were followed
by Russian war-ships and by the English and French squadrons
which had so recently been victorious in their war with China. It was
by such arguments that America and Europe clinched the consent of
reluctant Japan to admit them to trade and to reside within her
boundaries!
It seemed plain enough now that the Yedo Government could not
longer wait for permission from the Imperial Government to abandon
its policy of exclusion. Two of its members, Inouyé and Iwasé, were
forthwith sent to confer with the Consul General at Shimoda. When
Mr. Harris had pointed out the impossibility of continuing the policy of
exclusion, the dangers of adhering obstinately to the traditions of the
past, and had assured them of America’s friendly intervention to
secure favourable terms with the other powers of the West, the
commissioners returned to Yedo to report. But still the opposing
party grew; and still the Imperial Government delayed its consent.
Meantime the bitterness against Baron Ii was increased by the
failure of his enemies to secure the succession to the Shōgunate for
their favoured candidate. None the less, the Tairō took upon himself
the responsibility of despatching the same men with authority to sign
that Convention between the United States and Japan which, in spite
of the fact that it bore the name of the “Temporary Kanagawa Treaty”
and was subject to revision after a specified term of years, remained
unchanged until as late as 1895. This important event bore date of a
little more than a half-century ago—namely, July 29, 1858.
It is foreign to my purpose to examine the charges, urged against Ii
Kamon-no-Kami, of disobedience to the Imperial Government and of
traitorous conduct toward his country. The latter charge has long
since been withdrawn; and for this has been substituted the praise
and homage due to the patriot who is able to oppose public opinion,
to stand alone, to be “hated even by his relatives,” and to sacrifice
his life in his country’s behalf. That the Tairō did not obey the
Imperial command to submit again the question of exclusion to a
council of the Tokugawa princes and the Barons of the land is indeed
true. On the other hand, it is to be said that the Imperial Government,
by not forbidding the Treaty, had thrown back upon the Shōgunate
the responsibility for deciding this grave question; and that the
appearance of the foreign war-ships gave no further opportunity, in
wisdom, for continuing the policy of procrastination and delay. The
hour demanded a man of audacity, of clear vision into the future, and
of willingness to bear the full weight of a responsible decision. The
hour found such a man in the Japanese Naosuké, hereditary feudal
lord of Hikoné, but by providence in the position of Tairō, or military
dictator. It was fortunate, indeed, for the future relations of the United
States and Japan, and for the entire development of the Far East
under European influences, that an American of such patience,
kindliness, tactful simplicity, and sincere moral and religious
principle, met at the very critical point of time a Japanese of such
knightly qualities of honour, fearlessness, and self-centred force of
character. This point of turning for two political hemispheres, this
pivot on which swung the character of the intercourse between Far
East and Occident, owes more, I venture to think, to Townsend
Harris and to Ii Kamon-no-Kami than to any other two men.
The concluding of the Treaty did not allay the excitement of the
country over the intrusion of foreigners, or discourage the party of
the majority which favoured the policy of either risking all in an
immediate appeal to arms, or of continuing the effort to put off the
evil day by a policy of prevarication and temporising. Less than a
fortnight after its signing, the Shōgun became suddenly ill, and four
days later he died. Two days before his death, the three English
ships had anchored at Shinagawa, a suburb of the capital of the
Shōgunate; while the Russians had invaded the city of Yedo itself
and established themselves in one of its Buddhist temples.
Everything was now in confusion. The influence of the party for
exclusion—forceful, if necessary—was now greatly strengthened
among the Imperial Councillors at Kyoto; and intrigues for the
deposition of the Tairō and even for his assassination went on
apace. A serious and wide-spreading rebellion was threatened. The
resort of the Baron of Hikoné to force in order to crush or restrain his
enemies served, as a natural and inevitable result, to combine them
all in the determination to effect his overthrow—a result which his
opponents suggested he should forestall by committing harakiri, after
acknowledging his mistakes; and which his friends urged him to
prevent by resigning his office at Tairō.
Since Ii Kamon-no-Kami was not the man to retreat in either of these
two cowardly ways, he was destined to perish by assassination. On
March 25, 1860, one of the five annual festivals at which the princes
and barons of the land were in duty bound to present themselves at
the Shōgun’s Castle to offer congratulations, the procession of the
Tairō left his mansion at “half-past the fifth watch,” or 9 o’clock a. m.
Near the “Cherry-Field” gate of the castle, they were attacked by
eighteen armed men, who were all, except one, former retainers of
the Mito Clan, whose princes had been the most powerful enemy of
Baron Ii, but who had resigned from the clan, and become ronin, or
“wave-men,” in order not to involve in their crime the lord of the clan.
The suddenness of the attack, and the fact that the defenders were
impeded by the covered swords and flowing rain-coats which the
weather had made necessary, gave the attacking party a temporary
advantage. Baron Ii was stabbed several times through the sides of
his palanquin, so that when dragged out for further wounding and
decapitation, he was already dead. Thus perished the man who
signed the treaty with Townsend Harris, fifty years ago, in the forty-
sixth year of his age.
The motives of the two parties—that of the majority who favoured
exclusion and that of the minority who saw the opening of the
country to be inevitable—can best be made clear by stating them in
the language of each, as they were proclaimed officially to the
Japanese of that day. Fortunately, we are able to do this. So bitter
was the feeling against their feudal lord, even after his death, that it
seemed necessary, in order to prevent complete ruin from falling
upon the whole Clan of Hikoné, that all his official papers and
records should be burned. But Viscount Ōkubo, at no inconsiderable
danger to himself, managed “to save the precious documents”; for,
said he, “There will be nothing to prove the sincerity and unmixed
fidelity of Lord Naosuké, if the papers be destroyed. Whatever may
come I dare not destroy them.”
From one of these papers we quote the following sentences which
show why Baron Ii as Tairō signed, on his own responsibility, this
detested treaty with the hated and dreaded foreigners. “The question
of foreign intercourse,” it says, “is pregnant with serious
consequences. The reason why the treaty was concluded with the
United States was because of the case requiring an immediate
answer. The English and French Squadrons, after their victory over
China, were very soon expected to our coasts; and the necessity of
holding conferences with different nations at the same time might
cause confusion from which little else than war could be expected.
These foreigners are no longer to be despised. The art of navigation,
their steam-vessels and their military and naval preparations have
found full development in their hands. A war with them might result in
temporary victories on our part; but when our country should come to
be surrounded by their combined navies, the whole land would be
involved in consequences which are clearly visible in China’s
experience.... Trying this policy for ten or twelve years, and making
full preparation for protection of the country during that period, we
can then determine whether to close up or open the country to
foreign trade and residence.... If it were only one nation with which
we had to deal, it would be much easier; but several nations, coming
at this time with their advanced arts, it is entirely impossible to refuse
their requests to open intercourse with our country. The tendency of
the times makes exclusion an entire impossibility.”
But the assassins, on their part, before entering on their bloody
deed, had drawn up a paper which, as signed by seventeen, or all
except one of their number, they wished to have go down to posterity
in justification of their course. They, too, all met death either on the
spot, or subsequently by public execution, for their crime of
assassination. “While fully aware,” says this manifesto, “of the
necessity of some change in policy since the coming of the
Americans to Uraga, it is entirely against the interest of the country
and a shame to the sacred dignity of the land, to open commercial
relations, to admit foreigners into the castle, to conclude a treaty, to
abolish the established custom of trampling on the picture of Christ,
to permit foreigners to build places of worship of their evil religion,
Christianity, and to allow three Foreign Ministers to reside in the
land. Under the excuse of keeping the peace, too much compromise
has been made at the sacrifice of national honour. Too much fear
has been shown in regard to the foreigners’ threatening.”
This remarkable paper then goes on to charge the Tairō, Baron Ii,
with being responsible for so dishonourable an act of compromise.
He has assumed “unbridled power”; he has proved himself “an
unpardonable enemy of his nation,” a “wicked rebel.” “Therefore we
have consecrated ourselves to be the instruments of Heaven to
punish this wicked man; we have assumed on ourselves the duty of
putting an end to a serious evil by killing this atrocious autocrat.” The
assassins then go on to swear before Heaven and earth, gods and
men, that their act was motived by loyalty to the Emperor, and by the
hope to see the national glory manifested in the expulsion of
foreigners from the land.
At this distance of half a century, and considering the spirit of the
former age, we need not judge between Naosuké and his murderers
as regards the sincerity of their patriotism. But as to which of the two
parties followed the path of wisdom, there can be no manner of
doubt. Both Japan and its foreign invaders still owe a great debt of
gratitude and a tribute of wisdom, to Baron Ii Kamon-no-Kami. While
over all our clouded judgment hangs serene the truth of the
autograph of four Chinese characters with which, years afterwards,
the Imperial Prince Kitashirakawa honoured the book written to
vindicate the Tairō: “Heaven’s ordination baffles the human.”
How the memory of its former feudal lord is cherished in Hikoné, and
how his spirit still survives and in some sort dominates its citizens, I
had occasion to know during two days of early February, 1907. The
little city, headed by Mr. Tanaka, the steward of the present Count Ii,
by letter and then by a personal visit from the Christian pastor, Mr.
Sonoda, had urgently invited us to visit them, with the promise that
we should see the castle and other reminders of its former feudal
lord. I, on my part, was to speak to them on education and morality,
the two subjects about which the serious people of Japan are just
now most eager to hear. The same gentleman who had been the
medium of the invitation, was to be our escort from Kyoto to Hikoné.
But on the way, although the wind was piercing and light snow was
falling, we saw again the familiar objects of interest about the lower
end of Lake Biwa;—Miidera Temple, with its relics of the legendary
giant Benkei, such as the bell which he carried part way up the hill
and then dropped and cracked, and the huge kettle out of which he
ate his rice; then the wonderful pine-tree at Karasaki, the sail down
the lake and under the bridge of Seta; and, finally, the sights of
Ishiyama.
At a tea-house near the station here we were met by Mr. Tanaka,
who had come by train to extend the welcome of the city and who
emphasised this welcome by referring to the interest which we, as
Americans, in common with all our countrymen, must feel in the
place that had been the residence of the great Tairō. For had not he
“influenced the Shōgunate to open the country to the United States,
and lost his life for his advanced views?”
As the train conveyed us into the uplands, the snow began to fall
more heavily until it lay nearly a foot deep upon the plain and
wooded hill, crowned with its castle, of the ancient feudal town. Just
as the setting sun was making the mountains and the clouds aglow
with a rose colour, as warm and rich as anything to be seen in
Switzerland, we reached the station of Hikoné, and were at once
taken into its waiting-room to receive and return greetings of some
thirty of the principal citizens who had come out to welcome the city’s
guests. On account of the deep snow it was a jinrikisha ride of nearly
half an hour to the place where we were to be lodged—the Raku-
raku-tei, just beside the castle-moat, under its hill, and almost in the
lake itself. Here a beautiful but purely Japanese house, which was
built by the lord of the castle as a villa, stands in one of the finest
gardens of all Japan.
The fear that their foreign guests would not be entirely comfortable,
even if entertained in the best Japanese style, made it difficult for us
at first to discard or neglect the accessories especially provided, and
disport ourselves as though we were really cherishing, and not
feigning, the wish to be treated by them as their feudal lord would
have treated his friends at the beginning of the half century now
gone by. In the end, however, we succeeded fairly well in the effort to
merge ourselves, and our modern Western habits and feelings, in
the thoughts, ways and emotions of the so-called “Old Japan.”
Flags were hung over the quaint Japanese doorway of the villa; and
the manager, the landlord, and all the servants, were in proper array
to greet the long line of jinrikishas which were escorting the guests.
Our shoes removed, we were ushered through numerous rooms and
corridors, made attractive with the quiet beauty of choice screens
and the finest of mats, into the best apartment of the house. Here
bright red felt had been spread over the mats; a tall lacquer hibachi,
daimyo style, stood in the middle of the chamber; and large lacquer
or brass candlesticks, with fat Hikoné candles and wicks nearly a
half-inch thick, stood on either side of the hibachi and in each of the
corners of the room.
Thus far, the surroundings were well fitted to carry our imaginations
back to the time of Ii Kamon-no-Kami himself. But there were two
articles of the furnishing sure to cause a disillusionment. These were
a pair of large arm-chairs, arranged throne fashion behind the
hibachi, and covered with green silk cushions (or zabuton) which
were expected to contribute both to our comfort and to our sense of
personal dignity, while we were “officially receiving”—so to say.
Without offending our kind hosts, I trust, and certainly to the increase
of our own satisfaction, we begged permission to slip off from our
elevated position, so calculated to produce the feelings of social

A Python Data Analyst's Toolkit: Learn Python and Python-Based Libraries With Applications in Data Analysis and Statistics Gayathri Rajagopalan

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

A Python Data Analyst's Toolkit: Learn Python and Python-Based Libraries With Applications in Data Analysis and Statistics Gayathri Rajagopalan

Uploaded by

Copyright:

Available Formats

A Python Data Analyst’s Toolkit: Learn

Python and Python-based Libraries

Data Analysis from Scratch with Python Peters Morgan

Data Analysis with Python and PySpark (MEAP V07)

Python for Data Analysis Data Wrangling with Pandas

Python Data Analysis: Perform data collection, data

Python 2 and 3 Compatibility: With Six and Python-

Learning Data Mining with Python Layton

A Tour Of Data Science: Learn R And Python In Parallel

Hands-on Scikit-Learn for machine learning

ISBN-13 (pbk): 978-1-4842-6398-3 ISBN-13 (electronic): 978-1-4842-6399-0

Copyright © 2021 by Gayathri Rajagopalan

About the Technical Reviewer���������������������������������������������������������������������������������xv

Chapter 1: Getting Familiar with Python������������������������������������������������������������������ 1

Python Enhancement Proposal (PEP) 8 – standards for writing code����������������������������������������� 36

Chapter 2: Exploring Containers, Classes, and Objects������������������������������������������ 45

Chapter 3: Regular Expressions and Math with Python����������������������������������������� 77

Chapter 4: Descriptive Data Analysis Basics�������������������������������������������������������� 101

Chapter 5: Working with NumPy Arrays��������������������������������������������������������������� 117

Chapter 6: Prepping Your Data with Pandas��������������������������������������������������������� 147

Restructuring data and dealing with anomalies����������������������������������������������������������������������� 213

Chapter 7: Data Visualization with Python Libraries�������������������������������������������� 243

Chapter 8: Data Analysis Case Studies����������������������������������������������������������������� 279

Chapter 9: Statistics and Probability with Python����������������������������������������������� 325

1. Open the following URL: https://www.anaconda.com/products/

Figure 1-1. Installing Anaconda

4. After the installation is complete, open the Jupyter application

Figure 1-2. Launching Jupyter

• Click the following link: https://github.com/DataRepo2019/

Getting started with Jupyter notebooks

creating executables. It also includes features like autocompletion (completing what

1. Opening the dashboard

2. Creating a new notebook

Create a new Jupyter notebook by selecting New from the upper

Figure 1-3. Creating a new Jupyter notebook

3. Entering and executing code

Figure 1-4. Simple code statement in a Jupyter cell

4. Adding markdown text or headings

Figure 1-5. Changing the mode to Markdown

Figure 1-6. Changing the name of a file

You can also rename a notebook by selecting File ➤ Rename.

Figure 1-7. Downloading a Jupyter notebook

Shortcuts and other features in Jupyter

Table 1-1. Jupyter Notebook Toolbar Functions

Saving a Jupyter notebook Esc+s File ➤ Save as

Cutting a selected cell Esc+x Edit ➤ Cut Cells

Copying the selected cell Esc+c Edit ➤ Copy Cells

Pasting a cell above or Esc+v Edit ➤ Paste Cells

Interrupting the kernel Esc+ii Kernel ➤ Interrupt

Rebooting the kernel Esc+00 Kernel ➤ Restart

Figure 1-8. Help menu in Jupyter

Commonly used keyboard shortcuts include

• Esc to leave a cell.

• Esc+M changes the mode for a cell to “Markdown” mode.

• Esc+Y changes the mode for a cell to “Code”.

Figure 1-9. Tab completion in Jupyter

Magic commands used in Jupyter

Comments, print, and input

Generally, we pass a combination of strings and variables as arguments to the print

print('''Today is a lovely day.

Today is a lovely day.

The weight of Simi is 4.5

print("The weight of",name,"is","weight")

About the Technical Reviewer��xv

Chapter 1: Getting Familiar with Python�� 1

Python Enhancement Proposal (PEP) 8 – standards for writing code�� 36

Chapter 2: Exploring Containers, Classes, and Objects�� 45

Chapter 3: Regular Expressions and Math with Python�� 77

Chapter 4: Descriptive Data Analysis Basics�� 101

Chapter 5: Working with NumPy Arrays�� 117

Chapter 6: Prepping Your Data with Pandas�� 147

Restructuring data and dealing with anomalies�� 213

Chapter 7: Data Visualization with Python Libraries�� 243

Chapter 8: Data Analysis Case Studies�� 279

Chapter 9: Statistics and Probability with Python�� 325

Getting started with Jupyter notebooks

Shortcuts and other features in Jupyter

Magic commands used in Jupyter

Comments, print, and input