Kai-Feng Chen

National Taiwan University

PROGRAMMING &
NUMERICAL ANALYSIS 16
0
Lecture 01: Introduction 2

1
OVERVIEW

■ Actually this is a quasi-laboratory course, since no one can learn

how to do programing by only listening to the lectures and take
notes (and only do the homework once a while!).
■ PRACTICE is extremely important:
You will never learn the calculus without doing lots of
differential/integral exercises, right?
■ You are strongly recommended to bring your laptop to this
lecture and practice during the lecture. 

(hopefully the battery life of your laptop can run over 3 hours!)
■ If you do not have a laptop, you are encouraged to work with your
classmate who has laptop during the lecture.

2
A QUASI-LABORATORY
LECTURE

One will never learn piano without real practice;

simply watching a couple of great performances never works.
3
A QUASI-LABORATORY
LECTURE
■ I will not just “blah-blah” throughout the whole 3 hours. 

Instead, 1/2~2/3 of the total time will be devoted to an
introductory lecture with slides.
■ Rest of time will be used for practice/exercise/problem-solving,
just like your laboratory courses!
■ There will be also some short “trial periods” during the main
lecture, which allows you to try something easy.
■ Please also stop me when you run into any difficulties or troubles
throughout the whole lecture.

4
THE GOAL OF
THIS COURSE
■ Learn how to solve a problem with computers rather than 

a pen and papers.
■ Learn how to program.
■ Learn how to utilize the existing tools/functionalities, or build
your own tool.
■ Learn how to formulate a problem into a simple program that can
give you an answer clearly and quickly.
■ And have fun with them! (most important!)

5
WHAT ARE WE GOING
TO DO?
■ We will learn PYTHON from scratch.

(well, python is probably the easiest computer language to learn within half of
semester and powerful enough to do many things!)

■ We will discuss how to use python and the numerical/graphical

libraries to solve scientific problems, which could be beneficial to
your own physics (experimental/theoretical) studies in the near
future.
■ It does not mean you do not need to learn other computer
languages (e.g. C++, fortran, php, etc.) in the future for your own
work. Hopefully you will get some “taste of computing” in this
semester.

6
SCIENTIFIC COMPUTING
WITH PYTHON
■ We will use (some of the) SciPy (pronounced like “sign pie”)
packages in this lecture:
SciPy library: 

fundamental library for scientific computing.
NumPy: 

base N-dimensional array package.
Matplotlib: 

Comprehensive plot making
■ You can also consider the IPython as a nice enhanced interactive
console.
■ More information: http://www.scipy.org
7
OUTLINE
■ Python, the programming language
▫ The basis
▫ Control flow
▫ Types and data structure
▫ Functions and modules
▫ Input & Output
▫ Classes and others
■ Numerical analysis with Python
▫ Numerical differential and integration
▫ Random numbers
▫ Show it graphically We will try our best to go
▫ Linear algebra through all of these topics
▫ Root finding and minimum finding during this semester!
▫ Differential equations
▫ Curve fitting and data modeling
8
IF YOU ARE ALREADY
FAMILIAR WITH CODING...
■ I’m pretty sure some of you already well experienced in
programming.
■ The first part of the course can be too easy for you in this case, and
you can probably learn the later part by yourself without any
difficulties.
■ If you are in this situation, I would recommend:
Discuss with me and maybe we can do something beyond the
scope (e.g. a more challenging project).
Still come to the class and act as a helper for your classmates
(especially during the exercises period!).

9
HOMEWORK SYSTEM

In order not to have our TA

overloaded, we are going to
introduce an online homework
assignment system!

10
HOMEWORK SYSTEM (II)

■ We have the newly built KaKiX web for this semester:


http://hep12.phys.ntu.edu.tw/

⇓your ID here

You may try the web now!


⇑set your password first!
Please modify your password
✸ If you do not change your password within today, 
 IMMEDIATELY!✸
we will lock your account! 11
 due day!

your ID & name

Select one of the You can edit your

assignments homework here.

Submit your code to the

server and verify it on the fly!

12
ONLINE VERIFICATION

■ You may already try the 0th assignment, the Hello World:

Simply insert capital “HELLO WORLD” here and

submit the code.
13
ONLINE VERIFICATION (II)

■ This is what you should see if everything is working fine:

y o u 

r w i se la ints!
Othe c o m p
s o m e
e t
will g

14
EVALUATION

■ Homework:
▫ Exercises will be assigned every subject (every 1–2 weeks).
▫ Please hand back (upload) the code within 2 week.
■ Quizzes:
▫ Will be assigned during the first week of May (tentative).
▫ Time limit: 2 week.
▫ Googling the answer is allowed. Discussions are also allowed.

Surely this sounds too relaxed...

15
EVALUATION (II)

■ From the basic grading toward the final goal:

If you fulfill all of the

minimal requirements
(homework & quizzes, 

no delayed hand back) You have to collect 

3 gold coins just like the
Super Mario game!

16
EVALUATION (III)

■ How to collect the “golden coins”:

▫ Finish all of the homework assignments, all on-time!
▫ Be the first 2 people uploading the answer to any one of the
quizzes during the midterm week. Or hand-in the best (most
elegant) answer!
▫ Tournament: entering the semi-final round.
▫ Additional project presentation (especially if you are already
familiar with coding).

By default everyone can get B+~A+ easily.

Surely you may still loose the grading if:
1) Delayed/No hand-back of the homework
2) No hand-back of the examines

17
TEXTBOOK &
REFERENCES
■ For python, most of the information can be found online. Getting a
printed textbook is not really required. A couple of nice 

online books/documents are available:
▫ Python.org tutorial:

http://docs.python.org/2/tutorial/index.html
▫ Think python (✸slides are based on this book):

http://www.greenteapress.com/thinkpython/html/index.html
▫ A byte of python: 

http://swaroopch.com/notes/python/

Caveat: the documents/books may be prepared for

python 2 or python 3. In this lecture we will use
python 2.7 as the default version.
18
TEXTBOOK &
REFERENCES (II)
■ For SciPy (and NumPy), there are already some document
available on the official website and some online e-books:
▫ Official web document:

http://docs.scipy.org/doc/
▫ NumPy Beginner’s Guide:

http://it-ebooks.info/book/2847/
▫ SciPy and NumPy book:

http://it-ebooks.info/book/1280/

In principle you can always find the help online, so it is

not really required to have a printed book.

19
TEXTBOOK &
REFERENCES (III)
■ References for computational physics & algorithms 

(python and non-python):

An Introduction to
Introduction to Computation
Computational Physics
and Programming Using Python
by Tao Pang
by John V. Guttag(2013)
2nd Edition (2006)

Computational Physics: Numerical Recipes:

Problem solving with The Art of Scientific Computing
computers by William H. Press
by Rubin H. Landau et al. 3rd Edition (2007)
2nd Edition (2007) http://www.nr.com/
There are still many other computational physics or
algorithm text book can be found on the market.
20
THE ULTIMATE REFERENCE

Whatever, Google tells you everything...

21
GETTING START

■ If you are using any unix-like operation system, such as Linux or

Mac OSX, usually a python is pre-installed in your system:






 You can simply start

 a terminal and type
“python”.



■ For Windows, in principle you can download the python from the
official download area (please avoid python 3 for now):

http://www.python.org/download/

But wait – this is not enough!

22
GETTING START (II)

■ Since in this lecture we will use SciPy and NumPy, it will be much
easier if you can install all of them together. There are some
integrated package available:
▫ Option #1: Get the “Canopy Express” (free version):

https://www.enthought.com/store/

Just download it and install. It comes with all the needed
packages already, together with a nice IDE ready to go.
▫ Option #2: Get the “Anaconda” 

http://continuum.io/downloads.html

Also simply download it and install. It requires a little bit more
command line working experience but the support is good.

Basically these two options support Windows, Linux, and Max OSX.
23
GETTING START (III)

■ If you have installed the Canopy Express, this is what you will get:
Just click the “Editor” to start
your coding work with the IDE!

You may also want to check the

command line integration:

24
GETTING START (IV)

■ If you installed the Anaconda, this is what you will see:

The ipython-notebook is
something similar to Matlab
(also available in Canopy).

You should see a similar command

line integration:

If you want an IDE (spyder)

you have to install it first!
25
COMMENTS

■ In principle you can install all of the required packages (Python

+SciPy+MumPy+Matplotlib) by yourself without the integrated
package like Canopy and Anaconda. But it will take much more
efforts before you can actually work. This is not very
straightforward for beginners.
■ IDE (integrated development environment) –– In principle this is
not really a requirement. We will mostly use terminal (command
line) in this course. However, a good IDE can be easier for some
people. You can use it if you like. You can try the one came with
Canopy, or the spyder, or the IDLE (which was developed by the
original python author Guido van Rossum).

26
COMMENTS (CONT.)

■ Python 2 versus Python 3:


If you check the documents (and the official site) carefully, you
may find there are some issues between python 2 and 3. Basically
python 3 does not have the backward capability with version 2,
and the integration of python 3 has not yet been fully established.
The syntax is slightly different. In this lecture we will use python
2.7 as the default version.

Don’t worry about the exact version for now.

The key idea is to learn how to solve a problem
with programming, not the language itself.

27
INTERMISSION

■ Now it’s the time to get your working environment ready!


(switch on your laptop now!)
■ If you already have a python (whatever version/bundle) installed
in your laptop/desktop, you may proceed immediately until we
start to talk about SciPy/NumPy.
■ It will take a while to install Canopy or Anaconda. You can do it
later today.
■ If you only have a pad/phone, you can even try some of the online
python interpreter, e.g.:

http://www.compileonline.com/execute_python_online.php

http://repl.it

28
PYTHON:
AN INTRODUCTION
■ Quote from Wikipedia: “Python is a widely used general-purpose,
high-level programming language. Its design philosophy
emphasizes code readability, and its syntax allows programmers to
express concepts in fewer lines of code than would be possible in
languages such as C. The language provides constructs intended to
enable clear programs on both a small and large scale.”
■ The core philosophy (as called “The Zen of Python”):
▫ Beautiful is better than ugly.
▫ Explicit is better than implicit.
▫ Simple is better than complex. In short: 

Must be simple and beautiful
▫ Complex is better than complicated.
▫ Readability counts.
29
PYTHON:
AN INTRODUCTION (II)
■ Python development was started in December 1989.
■ Python's principal author: Guido van Rossum.
■ The origin of the name comes from Monty Python (Monty
Python's Flying Circus, a British television comedy show), 

not the elongated legless animal.

Guido van Rossum Monty Python's Flying Circus

30
PYTHON,
THE PROGRAMMING LANGUAGE
■ Python is a high-level language; others: C/C++, Fortran, Java, etc.
■ The computers can only run the “machine language”, which is not
human readable (well, if you are a real human).
■ The programs written in a high-level language are shorter and
easier to read, and they are portable (not restricted to only one
kind of computer). However, they have to be processed
(“translated”) before they can run.

31
“HELLO WORLD” IN
C/C++
Well, the common starting point for any programming language.

#include <stdio.h>

int main()
{
printf("hello world!\n");
} helloworld.cc

■ If you run it in a unix-like system with a gcc compiler:

% gcc helloworld.cc
% ./a.out
hello world!
%

32
“HELLO WORLD” IN
PYTHON

print 'hello world!'

helloworld.py

■ Run it in your terminal with python interpreter:

% python helloworld.py
hello world!
%

All you need is just a single line!

The standard filename extension for python is “.py”.

33
THE COMPILER (& LINKER)
■ A compiler reads the source code and translates it before the
program starts running; the translated program is the object code
or the executable.
■ Once a program is compiled, it can be executed it repeatedly
without further translation.

This is what we did for the 


C version of “hello world”.
34
THE INTERPRETER

■ An interpreter reads a high-level program (the source code) and

executes it.
■ It processes the program little-by-little at a time, reading lines and
performing computations.

The python version of “hello

world” program is executed by
the python interpreter.

35
PYTHON,
THE INTERPRETER
■ Python is considered an interpreted language because Python
programs are executed by an interpreter.
▫ Rapid turn around: no needs of compilation and linking as in C
or C++ for any modification.
▫ Hybrid Python programs are compiled automatically to an
intermediate form called bytecode, which the interpreter then
reads.
▫ This gives Python the development speed of an interpreter
without the performance loss in purely interpreted languages.
▫ However for serious speed limited computations you still need
to use C, or even Fortran.

36
PYTHON,
THE INTERPRETER (II)
■ There are actually two ways to use the interpreter: 

interactive mode and script mode.
■ Running your program in script mode:
▫ As we did in the previous “hello world” example: put your
code into a file, process it with the python interpreter.
▫ The Unix/Linux “#!” trick:
#!/usr/bin/env python
print 'hello world!' helloworld.py

% chmod +x helloworld.py ⇐ Make it as an executable file

% ./helloworld.py
hello world!

37
PLAY WITH PYTHON
INTERACTIVELY (I)
■ Let’s try some simple Python commands in the interactive mode:
% python
>>> 2+2
4
>>> # I’m a comment. I will not do anything here.
... 2+2 # anything after ‘#’ is a comment.
4
>>> 6/3
2
>>> (50-5*6)/4
5 Use the python interpreter as a calculator!
>>> 6/-3 The regular precedence is followed:
-2
>>> 2**8 ⇐ this is exponentiation (28) (,) ⇒ ** ⇒ *,/ ⇒ % ⇒ +,-
256
>>> 10%3 ⇐ this is modulus
1
38
PLAY WITH PYTHON
INTERACTIVELY (II)
■ Float point number versus integer:
>>> 8/3
2
>>> 9998/9999 ⇐ rounding down to integer (floor)
0
>>> 8./3
2.6666666666666665
>>> 8/3.
2.6666666666666665
>>> 8/(6.6/2.2)
2.666666666666667
>>> 8/3+1.0
3.0
>>> round(3.1415927,3) ⇐ rounding off to 3rd digit
3.142
39
PLAY WITH PYTHON
INTERACTIVELY (III)
■ You can assign a variable with “=”:
>>> width = 20
>>> height = 5*9
>>> area = width * height
>>> area
900

■ You can assign multiple variables as well:

>>> x = y = z = 1
>>> x
1
>>> x + y + z
3

40
PLAY WITH PYTHON
INTERACTIVELY (IV)
■ Complex number is also supported. The imaginary part can be
written with a suffix “j”.

>>> a = 2j
>>> a*a
(-4+0j)
>>> b = complex(3,4) ⇐ Using the function complex(real, imag)
>>> a+b
(3+6j)
>>> 8j/3j
(2.6666666666666665+0j) ⇐ Complex numbers are always float.
>>> b.real + b.imag ⇐ Exact the real/imaginary part
7.0 with .real and .imag
>>> abs(b) ⇐ absolute value
5.0

41
FIRST VISIT WITH
STRINGS
■ Python supports for strings are actually very powerful. See several
examples below:
>>> 'I am a string!'
'I am a string!'
>>> "me too!"
'me too!'
>>> 'anything quoted with two \' is a string.'
"anything quoted with two ' is a string."
>>> "\" also works."
'" also works.'
>>> "pyt" 'hon' ⇐ Two string literals next to each other 

'python' are automatically concatenated
>>> sentence = 'I am a string variable.'
>>> sentence
'I am a string variable.'
42
FIRST VISIT WITH
STRINGS (II)
■ More: multiline strings
>>> lines = "line1\nline2\nline3"
>>> print lines
line1
line2
line3
>>> reply = '''I ⇐ Use ''' or """ to start/end multiline strings.
... also have
... 3 lines'''
>>> print reply
I
also have
3 lines

43
FIRST VISIT WITH
STRINGS (III)
■ More: operators “+”, “*”, and type conversions
>>> 'oh'+'my'+'god'
'ohmygod'
>>> 'It\'s s'+'o'*20+' delicious!!'
"It's soooooooooooooooooooo delicious!!"
>>> '1234'+'1234'
'12341234'
>>> int('1234')+int('1234')
2468
>>> float('1234')/100
12.34
>>> a = b = 2
>>> str(a)+' plus '+str(b)+' is '+str(a+b)
'2 plus 2 is 4'
We will talk more on python strings afterwards.
44
VALUES AND TYPES

■ A value is one of the basic things a program works with, like a

letter or a number.
▫ 1234 is an integer. 1234 ⇒ value; integer ⇒ type.
▫ ‘hello’ is a string: ‘hello’ ⇒ value; string ⇒ type.
■ The interpreter can also tell you what type a value has:
>>> type('hello!')
<type 'str'>
>>> type(1234)
<type 'int'>
>>> type(3.14159)
<type 'float'>

45
TYPES & MORE
■ Remember there are two more numerical types supported:
>>> type(123456789012345L)
<type 'long'>
>>> type(3+4j)
<type 'complex'>

■ Type casting is easy:

>>> type('1234')
<type 'str'>
>>> type(int('1234'))
<type 'int'>

46
TYPES & MORE (II)
■ Few more complicated (but very useful) types are also supported.
Will be discussed in a near future lecture.

>>> type(3<4)
<type 'bool'>
>>> type(open(‘tmpfile’,’w’))
<type 'file'>
>>> type([1,2,3,4,[’five’,6.0]])
<type 'list'>
>>> type((1,2,‘three’,4))
<type 'tuple'>
>>> type({'name':'Chen','phone':33665153})
<type 'dict'>
>>> type({'a','b',’c’})
<type 'set'>
47
VARIABLES

■ A variable is a name that refers to a value.

■ An assignment statement creates new variables and gives them
values (you have seen this already):

>>> pi = 3.1415926536
>>> last_message = ‘what a beautiful day’

■ Programmers generally choose names for their variables that are

meaningful – they document what the variable is used for.
■ Variable names can be arbitrarily long, can be with both letters and
numbers.
■ The underscore character “_” can appear in a name as well.
48
VARIABLES (CONT.)

■ The variable names have to start with a letter or “_”

■ It is a good idea to begin variable names with a lowercase letter.
■ Python’s keywords: keywords are used to recognize the structure
of the program, and cannot be used as variable names.

and del from not while

as elif global or with
assert else if pass yield
break except import print
class exec in raise
continue finally is return
def for lambda try

49
INTERMISSION

■ Now it’s the time to play with your python interpreter:

▫ As a simple calculator.
▫ Play with the strings.
▫ Examine the python built-in types.
▫ Define your variables and check any possible break-down cases.

50
A STEP TOWARD
PROGRAMMING
■ A program is a sequence of instructions that specifies how to
perform a mathematical/symbolic computation.
■ This is a simple example of mathematical computation:
s = t = 1.
for n in range(1,21):
t /= n ⇐ t /= n is equivalent to t = t/n
s += t
print "1/0!+1/1!+1/2!+1/3!+...+1/20! =",s

1/0!+1/1!+1/2!+1/3!+...+1/20! = 2.71828182846

51
THE PRINT STATEMENT
■ As you already seen in the “hello world!” program, as well as the
previous example, the print statement simply present the output to
the screen (your terminal).
■ A couple of examples:
>>> print 'show it now!'
show it now!
>>> print 1,2,(10+5j),-0.999,‘STRING’ ⇐ connect w/ comma
1 2 (10+5j) -0.999 STRING
>>> print 1,2,3,4,5
1 2 3 4 5
>>> print str(1)+str(2)+str(3)+str(4)+str(5)
12345
Remark: for python 3, the print becomes a function, you
have to use something like print(‘hello world!’).
52
STRUCTURE OF
A PROGRAM
■ Input: 

get data from the keyboard, a file, or some other device.
■ Output: 

display data on the screen or send data to a file or other device.
■ Math: 

perform basic mathematical operations like addition and
multiplication.
■ Conditional execution: 

check for certain conditions and execute the appropriate code.
■ Repetition: 

perform some action repeatedly, usually with some variation.

Every program you’ve ever used, no matter how complicated, is

made up of instructions that look pretty much like these.
53
GIVE ME A FLYSWATTER

■ Programming is error-prone. Programming errors are called bugs

and the process of tracking them down is called debugging.
■ Three kinds of errors can occur in a program:
▫ Syntax errors
▫ Runtime errors
▫ Semantic errors

54
SYNTAX ERRORS

■ Syntax refers to the structure of a program and the rules about that
structure.
■ Python can only execute a program if the syntax is correct;
otherwise, the interpreter displays an error message.
■ An example:
>>> y = 1 + 4x
File "<stdin>", line 1
y = 1 + 4x
^
SyntaxError: invalid syntax You probably already see a
>>> couple of syntax errors in
your previous trials.

55
RUNTIME ERRORS

■ A runtime error does not appear until after the program has
started running.
■ These errors are also called exceptions because they usually
indicate that something exceptional (and bad) has happened.
■ The following “ZeroDivisionError” only occur when it runs.
>>> a = b = 2
>>> r = ((a + b)/(a - b))**0.5 * (a - b)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ZeroDivisionError: integer division or modulo by zero
>>>

56
SEMANTIC ERRORS

■ If there is a semantic error in your program, it will run successfully,

but it will not do the right thing.
■ It does something else.
■ The problem is that the program you wrote is not the exactly
program you wanted to write.
■ The meaning of the program (its semantics) is wrong.
■ Identifying semantic errors can be tricky because it requires you to
work backward by looking at the output of the program and trying
to figure out what it is doing.
■ (Much) more difficult to debug.

57
SEMANTIC ERRORS

■ An example: Just one “dot” gives you a

s = t = 1. completely different answer.

s = t = 1
for n in range(1,21):
t /= n
s += t
print "1/0!+1/1!+1/2!+1/3!+...+1/20! =",s

1/0!+1/1!+1/2!+1/3!+...+1/20! = 2

2.71828182846
58
DEBUGGING
■ It is very difficult to write a bug free program at the first shot.
■ Programming and debugging can be the same thing. 

Programming is the process of debugging a program until it does
exactly what you want.
■ Having a good programming/coding habit sufficiently helps.
■ Re-run a buggy program never helps!

59
 A FUN DEMO
■ Continue our lecture with a demo problem: show you how things
can be sorted out easily, if you know how some coding + google.
■ Let me ask –– how many people are there in this photo, roughly?

You may reply: are you nuts? I do know how to count!

60
But how about this one? Ya…it might just take a while…
61
LET’S SOLVE THIS WITH
PROGRAMMING
■ You may start to think it may take a while to work out a code to
count number of people in a photo.
■ There might be some existing programs or app that can do such a
thing (in some of the cases you will even need to pay!).
■ In fact if you know how to do it, it won’t cost you more than 10
lines of coding!
■ Let me show you a small piece of python code which adopt the
OpenCV library + face detection.

62
LESS10 LINES TO WORK IT
OUT!
import cv2 ⇐ include OpenCV

img = cv2.imread('solvay.jpg') ⇐ load the image

face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_alt.xml')
faces = face_cascade.detectMultiScale(img, 1.1, 5)

print 'How many faces found:',len(faces) ⇐ print out how many ‘faces’

for (x,y,w,h) in faces: add a small red box 


cv2.rectangle(img,(x,y),(x+w,y+h),(0,0,255),2) ⇐ to the faces found
cv2.imwrite('out.jpg',img) ⇐ save the output
count_faces_ex1.py

■ Just run it directly, if you have all the needed files!

% python count_faces_ex1.py
How many faces found: 29

63
Yes, there are 29 people in total!

64
APPLY IT TO THE BIG
PHOTO?
■ Let’s see what we can get –– 211 faces in total?

Lots of “unlabeled” people.

Need some tuning!
65
Loosen the face detecting criteria: now
we find 418 people, w/ some missing
people plus some mistakes !
66
COMMENTS

■ With only few lines of code one can already get a very rough guess
of how many people (more precise –– how many faces!) in the input
photo!

■ You many know that in many of the album program can do a

similar thing as well: locate and identify the faces!

■ Just want to show you how a (looks-like-to-be) difficult task can be

worked out easily if you know there is such a tool existing.

■ Surely in the real life problem solving will take some real
efforts, not just magical few lines of codes!

67
Please do not flunk me, professors!

68
HANDS-ON SESSION
■ Up to now we have gone through:
▫ The basic python interface
▫ Variables, types, and operators
■ Now you should be able to do some useful calculations!
■ Let’s start our first round of 

hands-on session now!

69
HANDS-ON SESSION

■ Practice 1: 

Get your working environment ready now, and type in your first
“hello world!”.

70
HANDS-ON SESSION

■ Practice 2: 

Use the python interpreter as a calculator, and calculate the
following simple math problems:

2
▫ Given z = 1 + 3i , what’s the real part and imaginary part?

p

▫ Calculate the magnitude of the total gravitational force that is

acted on point mass A: 
 4 kg

 G = 6.67384 × 10–11 m3kg–1s–2

 5m
3m


A: 5 kg 3 kg
4m

71