Algorithm analysis

Elementary numerical analysis: algorithm analysis

Algorithm analysis
“Good program is a unity of thought-out algorithm and
efficient data structures”.
N. Wirth.

“Software is getting slower more rapidly than

hardware becomes faster”.
Wirth’s law (1995).
1969
Niklaus Wirth (born Feb. 15, 1934) is a Swiss
computer scientist, known for designing several
programming languages, including Pascal,
and for pioneering several classic topics
in software engineering.

In 1984 he won the Turing Award

for developing a sequence of
innovative computer languages.

Elementary numerical analysis: algorithm analysis

Algorithm analysis
“Algorithm” and “data structure” are key terms in computer science.
Which algorithms and data structures are high quality and effective?
We need to use accurate methods of algorithm analysis.
First, natural criterion: time of execution.
Also important:
– amount of resourced required: memory and disk space;
– reliability and validity of solutions;
– stability of solutions (robustness)

In computer science algorithm is robust if it continues

to operate despite abnormalities in input, calculations, etc.
Various commercial products perform robustness testing of
software systems.
There are many examples of robustness in
biology as well.

Elementary numerical analysis: algorithm analysis

 Algorithm analysis
At first, consider a time concept. How to evaluate a time for an arbitrary algorithm?
Time of execution of algorithm or data structure operation depends on several factors.
The easiest way to determine the time required to perform the algorithm
is to measure the time before start and after completion of the algorithm.

#include <time.h>
#include <stdlib.h>
#include <stdio.h>

void main (void)

{
time_t before = time (NULL);
for (int i=0; i < 10E8; i++)
(void)rand ();
printf ("time was taken about %d seconds",time(NULL)- before);
}

The result (for example):

time was taken about 17 seconds

Elementary numerical analysis: algorithm analysis

Algorithm analysis
Direct measuring of execution time is not exact!

Reasons:
– modern operating systems are multitasking;
– multiple evaluation needed (due to random factors presence)
– different input data sets must be processed.

Always needed Time Experimental data

for non trivial

algorithms.

Interpolation
and approximation Trend

of results is possble

Number of elements

Elementary numerical analysis: algorithm analysis

Algorithm analysis
Generally:

– the larger amount of data, the longer execution time;

– execution time depends on data structure;
– hardware is important (CPU, RAM, HDD, etc.);
– software is important (OS, programming language, compiler, etc.)

So, the analysis of algorithms empirically, is not really reliable.

Elementary numerical analysis: algorithm analysis

Algorithm analysis
The main disadvantages of empirical analysis:

1. Limited data set

Results obtained using another set are not taken into account

Example: algorithm for calculating the sum of a series with

given accuracy delta

Algorithm will give wrong solution (equal to a finite number)

because this series is divergent, and its sum is infinite.

Elementary numerical analysis: algorithm analysis

Algorithm analysis
Code example:
#include <math.h>
#include <stdio.h>
#include <conio.h>

void main(void)
{
double sum = 0 , ai;
int f = 1 , i = 1;

do {sum += (ai=1.0/f); f*=i++; }while (fabs(ai) >= 1E-5);

printf ("exp = %lf" , sum);

getch ();
}

Elementary numerical analysis: algorithm analysis

Algorithm analysis
The main disadvantages of empirical analysis:

2. To compare two algorithms

the same hardware and software are required

3. It is necessary to implement and execute an algorithm for

the experimental study.
So, we need general methods for algorithm analyzing.

Requirements:
– has to take into account the different types of input data;

– hardware and software independence;

– can be performed with algorithm description, no implementation
or run-time testing required.

Elementary numerical analysis: algorithm analysis

Algorithm analysis
General idea:

Algorithm f (n1, …, nm)

– number of data items for processing;
– accuracy;
– validity (for large numbers computing, for example);
–…

The first stage of the algorithm developing is writing a pseudocode.

Elementary numerical analysis: algorithm analysis

Algorithm analysis
Pseudocode is a compact language of algorithm description.

It uses keywords of a programming language, but omits not significant details.

The main purpose of pseudocode using is to provide understanding
of an algorithm, make it.
Example:

Title: An algorithm for counting the number of array elements equal to zero
Baseline data: Array A
Required: the number of elements equal to zero
Start
Put the number of K = 0
CYCLE for all elements of the array from 1 to ElementsNumber(A)

if the element A(i) = 0 THEN K = K + 1

END OF CYCLE
End
Displaying K

Elementary numerical analysis: algorithm analysis

Algorithm analysis
Logarithmic and exponential functions are used widely for
analyzing algorithms and data structures:

Elementary numerical analysis: algorithm analysis

Algorithm analysis
Cycles lead to series appearance, so
the series are used widely in algorithm analysis:

F(N) = A(1) + A(2) + … + A(N)

If integer N>=0 and real 0 < a < > 1, then:

geometric series example

arithmetic series example

Elementary numerical analysis: algorithm analysis

 Algorithm analysis
The order of the algorithm is a function, dominating over exact
expression of time complexity.
It describes algorithm efficiency as a function of number of
processed data.

“Big O” is used in practice.

For example, counting zero elements of an array

can be described as O(n),
where n – number of elements in array.

During algorithm analyzing (counting the operation and running time)

“small details” are not important .
For example, constant multipliers can be neglected.

Elementary numerical analysis: algorithm analysis

Algorithm analysis
Let k is a constant, f and g are functions.

1. O k  f   O f 
(constant multipliers are not important)

2. O f  g   O f   O g  , O f g   O f  O g 

 
Example: O 17  N   N   O17  N   O N   O N   O N   O N  N   O N 2

3. O f  g  is equal to dominant of O f  and O g 


Example: O N 5  N 2  O N 5   

Elementary numerical analysis: algorithm analysis

Algorithm analysis
Examples of algorithm complexity

O(1): Most of the operations in the program are executed only once or only a few times.
Algorithms of constant complexity.

О(N): Every element of input data is to be processed only a linear number of times.

О(N2), О(N3), О(Na): Polynomial complexity.

O(N2) – quadratic complexity, O(N3) – cubic complexity, ... .

О(log(N)): The program works much more slowly with increasing N.

Occurs in programs that divide a large problem into smaller problems and
solve them individually.

O(2N): Exponential complexity. Such algorithms are often the result

of the method of “brutal force”.

Elementary numerical analysis: algorithm analysis

Algorithm analysis
Example 1:

Evaluate complexity of the algorithm:

Pn(x) = anxn + an-1xn-1 + ... + aixi + ... + a1x1 + a0

Computing of the i-th item (i = 1..n) requires i multiplications.

Totally: 1 + 2 + 3 + ... + n = n(n+1)/2 multiplications.

n+1 additions.

Totally: n(n+1)/2 + n + 1= n2/2 + 3n/2 + 1 operations. O n 2  

1. O  k  f   O f 
2. O f  g   O f   O g  , O f g   O f  O g 
3. O f  g  is equal to dominant of O f  and O g 
Elementary numerical analysis: algorithm analysis
Algorithm analysis
Example 2: Pn(x) = anxn + an-1xn-1 + ... + aixi + ... + a1x1 + a0

Reduce the expression as follows:

Pn(x) = a0 + x(a1 + x(a2 + ... ( ai + .. x(an-1 + anx))).

For example,
P3(x) = a3x^3 + a2x^2 + a1x^1 + a0 = a0 + x(a1 + x(a2 + a3x))

1*, 1+ 1 *, 1+
n-1 brackets totally
Totally: n multiplications + n additions = 2n. O n 

1. O  k  f   O f 
2. O f  g   O f   O g  , O f g   O f  O g 
3. O f  g  is equal to dominant of O f  and O g 
Elementary numerical analysis: algorithm analysis
Algorithm analysis
Exercise:

1) 20n 3  7.2n 2  21.78n  5 O ? O n3 

2) 3  log n   log log n   O  ? O log n 

3) 2100 O  ? O1

1
4) 5n O  ?  O 
n

1. O  k  f   O f 
2. O f  g   O f   O g  , O f g   O f  O g 
3. O f  g  is equal to dominant of O f  and O g 
Elementary numerical analysis: algorithm analysis
Algorithm analysis
Example:

A problem: find a name in a telephone book of N pages.

Evaluate a complexity of search algorithm.

Opening a book in the middle halves “the rest” of the problem.

(We assume that names are sorted alphabetically).

So, the search requires time no more than log2 N.

Time complexity of the algorithm is O log 2 n 

If N = 1000 pages, we have to open the book log2 1000 ≈ 10 times.

If N = 100 000 pages, we have to open the book log2 100000 ≈ 17 times.

Elementary numerical analysis: algorithm analysis

Algorithm analysis
Donald Knuth “The Art of Computer Programming”
http://www-cs-faculty.stanford.edu/~knuth/taocp.html
The Art of Computer Programming 1:
기초 알고리즘 ( 개정 3 판 ) 도널드 커누스
(Donald E. Knuth) 지음 | 류광 역 | 번역서
| 2006 년 09 월 | 한빛미디어 ( 주 )

Donald Knuth at a reception for the Open Content Alliance,

October 25, 2005 (From Wikipedia).

Donald Knuth
born January 10, 1938
Homepage: http://www-cs-staff.stanford.edu/~uno/

Elementary numerical analysis: algorithm analysis

Algorithm analysis: Homework 1
Homework 1
Write a program tabulating a function.
The function Y(X) is defined in your program code.

User enters three numbers: X1, X2 and S.

User enters a way of result output (print on a screen or write to a text file).
The program computes values of Y(X) between X1 and X2 with the step S and
prints a table on a screen or to a file.

C:\>program.exe computer screen

Num | X | Y(X) |
-------------------------------
0 | 1.5 | 2.74 |
1 | 1.6 | 3.21 |
...
...

Elementary numerical analysis: algorithm analysis

Algorithm analysis: Homework 1
Deadline: 14/09/2010
Present your working program (your_file.exe) and
source code (files *.c (or *.cpp) and *.h).

Note: write your student number and english name in the first
line of your program as a comment:

/* <student number> <your name>*/

Send the files to e-mail: dsmsoft@yandex.ru

or bring me your files on USB flash drive. E-mail is the best.

Elementary numerical analysis: algorithm analysis

Algorithm analysis: Homeworks
Homeworks (details will be given later)
1.Tabulating a function (by 14.09)
2. Finding prime numbers (by 28.09)
3.Vector operations (addition, subtraction, length, scalar product,
vector product, use structures for vectors storing) (by 12.10)

•Sorting (bubble sort, insertion sort or selection sort, shell sort;

illustrate the difference in time efficiency of the used algorithms) (by 2.11)

5.Matrix operations (input from keyboard and file, adding/subtracting,

multiplying, transpose, output to screen and to file) for arbitrary-sized
matrices (by 16.11)
6.Solving a system of linear equation using Gaussian elimination
method. Using pivoting is a bonus (by 7.12)
Elementary numerical analysis: algorithm analysis