Chapter 1

Introduction to Algorithms
Gedamu Alemu
alemugedamu@yahoo.com
Image processing and Computer vision SIG
Some of the sides are exported from different sources to clarify the topic
Computer Science and Engineering Program: DAA

Gedamu Alemu
Introduction
 What is Algorithm
 Definition and Basic Concept
 Fundamental of Algorithmic Problem Solving
 Understanding and choose b/n exact &approximate
problem solving
 Important Problem Type
 Sorting and searching
 String processing and geometrical problem
 Graph and Tree problem
 Fundamental Data Structure
 Liner data Structure
 Non Liner Data stracture

Gedamu Alemu
Analysis of algorithms
 Issues:
 Correctness
 space efficiency
 time efficiency
 optimality
 Approaches:
 theoretical analysis
 empirical analysis

Gedamu Alemu
4
Introduction
5
Problem solving in computer Science

Gedamu Alemu
6
What is an algorithm?
 An algorithm is a sequence of unambiguous instructions for
solving a problem, i.e., for obtaining a required output for any
legitimate input in a finite amount of time.
problem
algorithm
input “computer” output

Gedamu Alemu
7
 Procedure for getting answers to a specific problem
 Problem solving strategy even if computers are not involved
 Scope is limited  I can not have a procedure for you to have very
happy life or becoming rich and famous

Gedamu Alemu
Notion/Concept of algorithm and 8
problem
 Nonambiguity for each step
 Range of inputs is specified carefully
 The same algorithm can be represented in several ways
 Several algorithms for the same problem may exist
 Several algorithms with different ideas can solve the same
problem with different speed.
problem
algorithm
Input “computer ” output

Gedamu Alemu
Example of computational problem: 1-9
sorting
 Statement of problem:
 Input: A sequence of n numbers <a1, a2, …, an>
 Output: A reordering of the input sequence <a´1, a´2, …, a´n> so that a´i ≤
a´j whenever i < j
 Instance: The sequence <5, 3, 2, 8, 3>
 Algorithms:
 Selection sort
 Insertion sort
 Merge sort
 (many others)

Gedamu Alemu
Some Well-known Computational Problems 1-10
 Sorting and Searching

 Graph Problems
 Combinatorial problems
 Geometrical Problems
 Traveling salesman problem
 Knapsack problem
 Chess
 Towers of Hanoi
Some of problems don’t have efficient algorithms, or algorithms at all!

Gedamu Alemu
1-11
Basic Issues Related to Algorithms
 How to design algorithms
 How to express algorithms
 Proving correctness
 Efficiency (or complexity) analysis

 Theoretical analysis
 Empirical/ experimental analysis
 Optimality

Gedamu Alemu
1-12
Algorithm design strategies
  Transform and conquer
Brute force
 Try all possible combinations • a simpler instance of the same problem, or
• a different representation of the same problem, or
 Divide and conquer • an instance of a different problem
breaking down a problem into two or more 
sub-problems of the same (or related) Greedy approach
type, until these become simple • The problem could be solved in iterations
enough to be solved directly.
 Dynamic programming
 Decrease and conquer • An instance is solved using the solutions for
 Change an instance into one smaller smaller instances.
instance of the problem. • The solution for a smaller instance might be
 Solve the smaller instance. needed multiple times.
 Convert the solution of the smaller • The solutions to smaller instances are stored in a
instance into a solution for the larger table, so that each smaller instance is solved only
instance. once.
• Additional space is used to save time.
 Backtracking and branch-and-
bound

Gedamu Alemu
1-13
Analysis of Algorithms
 How good is the algorithm?

 Correctness
 Time efficiency
 Space efficiency
 Does there exist a better algorithm?

 Lower bounds
 Optimality

Gedamu Alemu
1-14
 Recipe, process, method, technique, procedure, routine,… with the
following requirements:
1. Finiteness
 terminates after a finite number of steps
2. Definiteness
 rigorously and unambiguously specified
3. Clearly specified input
 valid inputs are clearly specified
4. Clearly specified/expected output
 can be proved to produce the correct output given a valid input
5. Effectiveness
 steps are sufficiently simple and basic

Gedamu Alemu
1-15
Algorithms Examples
1-16
Addition and Subtraction

1-17
2- Digit Addition
 Add the following (try it from left to right):
 Simply , think of 32 add ( 30+2) ; then add it to 47 ; then add

2 to the results

Gedamu Alemu
1-18
2- Digit Addition (cont. )
 Add the following (try it from left to right):

Gedamu Alemu
1-19
2- Digit Addition (cont. )
 Try this on your own:

Gedamu Alemu
1-20
3- Digit Addition
Add the following :
Simply :
- Add the hundreds digit of the second number to the first number
- Add the tens digit to the result
- Add the unit digit to the result

Gedamu Alemu
1-21
3- Digit Addition
Try this by yourself :

Gedamu Alemu
1-22
3- Digit Addition
 Try this one :
 Try it this way (subtraction instead of addition)

Gedamu Alemu
1-23
Part of your first assignment
 Find and write the best algorithm to sum up the

numbers from 1 to 100 ? What is the complexity
of your algorithm ?
 Write an algorithm to add 3-digit numbers (take

the sign in your account- Allow the addition of
positive or negative numbers)? Compare your
algorithm to the computer does in similar cases?

Gedamu Alemu
1-24
2-Digit Subtraction

Gedamu Alemu
1-25
3-Digit Subtraction

Gedamu Alemu
1-26
More about numbers Euclid’s Algorithm

1-27
Euclid’s Algorithm
Problem: Find gcd(m,n), the greatest common divisor of two nonnegative, not
both zero integers m and n
Examples: gcd(60,24) = 12, gcd(60,0) = 60, gcd(0,0) = ?
Euclid’s algorithm is based on repeated application of equality

gcd(m,n) = gcd(n, m mod n)
until the second number becomes 0, which makes the problem trivial.
Example: gcd(60,24) = gcd(24,12) = gcd(12,0) = 12

Gedamu Alemu
28
Descriptions of Euclid’s algorithm
 Step 1 If n = 0, return m and stop; otherwise go to Step 2
 Step 2 Divide m by n and assign the value of the remainder to r
 Step 3 Assign the value of n to m and the value of r to n. Go to
Step 1.
while n ≠ 0 do
r ← m mod n
m← n
n←r
return m
Gedamu Alemu
29
Other methods for gcd(m , n) [cont.]
Middle-school procedure
 Step 1 Find the prime factorization of m
 Step 2 Find the prime factorization of n
 Step 3 Find all the common prime factors
 Step 4 Compute the product of all the common prime factors and return
it as gcd(m,n)
E.g. find gcd(60, 24)

Step 1: 60 = 2 . 2 . 3 . 5
Step 2: 24 = 2 . 2 . 2 . 3 Is this an algorithm?
Step 3: 2 . 2 . 3 not qualified  how do you get the
prime factorization? How do you get
Step 4: 2 . 2 . 3 = 12 common elements in two stored lists?
How efficient is it?
Time complexity: O(sqrt(n))

Gedamu Alemu
1-30
More about numbers Sieve of

Eratosthenes
31
Sieve of Eratosthenes
 To
find all the prime numbers less than or
equal to a given integer n by Eratosthenes'
method:
 Create a list of consecutive integers from two to n: (2, 3, 4, ..., n).
 Initially, let p equal 2, the first prime number.
 Strike from the list all multiples of p less than or equal to n. (2p, 3p,
4p, etc.)
 Find the first number remaining on the list greater than p (this number
is the next prime); replace p with this number.
 Repeat steps 3 and 4 until p2 is greater than n.
 All the remaining numbers on the list are prime.

Gedamu Alemu
1-32
Sieve of Eratosthenes
Input: Integer n ≥ 2
Output: List of primes less than or equal to n
for p ← 2 to n do A[p] ← p
for p ← 2 to √n do
if A[p]  0 //p hasn’t been previously eliminated from the list
j ← p* p
while j ≤ n do
A[j] ← 0 //mark element as eliminated
j←j+p
Example: 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Gedamu Alemu
Steps of Designing and Analyzing an 33
Algorithm
 Understand the Problem
 Decide on the machine to be used (sequential vs. parallel)
 Design an algorithm: select a specific techniques such as divide and conquer .
 Also, decide on how do you represent the algorithm for instance pseudocode ,
flowchart , etc..
 Prove correctness may be by example or mathematically
 Analyze the algorithm for instance in space and time efficiency

Gedamu Alemu
1-34
Important Problem Types
 Sorting
 Searching
 String processing
 Graph problems
 Combinatorial problems (counting objects number of sub graph in a

graph )
 Geometric problems( closest pair and convex-hull)

 Numerical problems (mathematical equation)

Gedamu Alemu
1-35
Sorting (I)
 Rearrange the items of a given list in ascending order.
 Input: A sequence of n numbers <a1, a2, …, an>
 Output: A reordering <a´ , a´ , …, a´ > of the input sequence such that a ´ ≤ a´ ≤
1 2 n 1 2
… ≤ a n.
´
 Why sorting?
 Help searching
 Algorithms often use sorting as a key subroutine.
 Sorting key
 A specially chosen piece of information used to guide sorting. E.g., sort student
records by names.

Gedamu Alemu
1-36
Sorting (II)
 Examples of sorting algorithms
 Selection sort
 Bubble sort
 Insertion sort
 Merge sort
 …..
 Evaluate sorting algorithm complexity: the number of key comparisons.

 Two properties
 Stability: A sorting algorithm is called stable if it preserves the relative
order of any two equal elements in its input.
 In place : A sorting algorithm is in place if it does not require extra
memory, except, possibly for a few memory units.

Gedamu Alemu
1-37
Selection Sort
Algorithm SelectionSort(A[0..n-1])
//The algorithm sorts a given array by selection sort
//Input: An array A[0..n-1] of orderable elements
//Output: Array A[0..n-1] sorted in ascending order
for i  0 to n – 2 do
min  i
for j  i + 1 to n – 1 do
if A[j] < A[min]
min  j
swap A[i] and A[min]

Gedamu Alemu
1-38
Searching
 Find a given value, called a search key, in a given set.
 Examples of searching algorithms
 Sequential search
 Binary search …
input: sorted array a_i < … < a_j and

key x;
m (i+j)/2;
while i < j and x != a_m do
if x < a_m then j  m-1
else i  m+1;
if x = a_m then output a_m;
Time: O(log n)

Gedamu Alemu
1-39
String Processing
 A string is a sequence of characters from an alphabet.
 Text strings: letters, numbers, and special characters.
 String matching: searching for a given word/pattern in a text.
Examples:
(i) searching for a word or phrase on WWW or in a Word
document
(ii) searching for a short read in the reference genomic
sequence

Gedamu Alemu
1-40
String Matching
 Given a text string T of length n and a pattern string P
of length m, the exact string matching problem is to
find all occurrences of P in T.
 Example: T=“AGCTTGA” P=“GCT”
 Applications:
 Searching keywords in a file
 Searching engines (like Google and Openfind)
 Database searching (GenBank)
 More string matching algorithms (with source codes):
http://www-igm.univ-mlv.fr/~lecroq/string/

Gedamu Alemu
1-41
String Matching
 Brute Force algorithm
 The brute force algorithm consists in checking, at all positions in the text
between 0 and n-m, whether an occurrence of the pattern starts there or not.
Then, after each attempt, it shifts the pattern by exactly one position to the
right.
Time: O(mn) where m=|P| and n=|

T|.

Gedamu Alemu
1-42
Main Features
 No preprocessing phase;
 Constant extra space needed;
 Always shifts the window by exactly 1 position to the right;
 Comparisons can be done in any order;
 Searching phase in O(mn) time complexity;

Gedamu Alemu
1-43
Graph Problems
 Informal definition
 A graph is a collection of points called vertices, some of which
are connected by line segments called edges.
 Modeling Real-life Problems
 Modeling WWW
 Communication networks
 Project scheduling …
 Examples of Graph Algorithms
 Graph traversal algorithms
 Shortest-path algorithms
 ……..

Gedamu Alemu
44
….
 Data Structures:
 An implementation of an ADT is a translation into
statements of a programming language,
─ the declarations that define a variable to be of that
ADT type
─ the operations defined on the ADT (using procedures
of the programming language)
 Each data structure is built up from the basic data types of the
underlying programming language using the available data
structuring facilities, such as
 arrays, records (structures in C), pointers, files, sets, etc.

Gedamu Alemu
1-45
Fundamental data structures
 list
 array
 linked list  graph
 string  tree and binary tree
 stack
 queue
 priority queue

Gedamu Alemu
1-46
Linear Data Structures
 Arrays  Arrays
 A sequence of n items of the same data type that
are stored contiguously in computer memory and fixed length (need

made accessible by specifying a value of the preliminary reservation of

array’s index. memory)
 contiguous memory
 Linked List locations
 A sequence of zero or more nodes each
containing two kinds of information: some data
 direct access
and one or more links called pointers to other  Insert/delete
nodes of the linked list.
 Singly linked list (next pointer)
 Doubly linked list (next + previous pointers)
 Linked Lists
 dynamic length
 arbitrary memory
a1 a2 an
locations
 access by following links
 Insert/delete
Gedamu Alemu
1-47
Stacks and Queues
 Stacks
 A stack of plates
─ insertion/deletion can be done only at the top.
─ LIFO
 Two operations (push and pop)
 Queues
 A queue of customers waiting for services
─ Insertion/enqueue from the rear and deletion/dequeue from the
front.
─ FIFO
 Two operations (enqueue and dequeue)

Gedamu Alemu
1-48
Priority Queue and Heap

Priority queues (implemented using heaps)
 A data structure for maintaining a set of elements, each
associated with a key/priority, with the following
operations:
 Finding the element with the highest priority
 Deleting the element with the highest priority
 Inserting a new element
 Scheduling jobs on a shared computer

Gedamu Alemu
1-49
Graphs
 Formal definition
 A graph G = <V, E> is defined by a pair of two sets: a finite set V of
items called vertices and a set E of vertex pairs called edges.
 Undirected and directed graphs (digraphs).
 Complete, dense, and sparse graphs

 A graph with every pair of its vertices connected by an edge is called
complete, K|V|
 In Dense graph, the number of edges is close to the maximal number
of edges.
 1  2
 3  4
Dense
Complete

Gedamu Alemu
1-50
Graph Representation
 Adjacency matrix
 n x n boolean matrix if |V| is n.
 The element on the ith row and jth column is 1 if there’s an edge from
ith vertex to the jth vertex; otherwise 0.
 The adjacency matrix of an undirected graph is symmetric.
 Adjacency linked lists
 A collection of linked lists, one for each vertex, that contain all the
vertices adjacent to the list’s vertex.
 Which data structure would you use if the graph is a 100-node star
shape?
2 3 4
0111 4
0001
0001
0000

Gedamu Alemu
1-51
Weighted Graphs
 Graphs or digraphs with numbers assigned to the edges.
 1
5  2
6 7
9
3  4
8

Gedamu Alemu
1-52
Graph Properties -- Paths and Connectivity
 Paths
 A path from vertex u to v of a graph G is defined as a sequence of
adjacent (connected by an edge) vertices that starts with u and ends
with v.
 Simple paths: a path in a graph which does not have repeating
vertices..
 Path lengths: the number of edges, or the number of vertices – 1.
 Connected graphs
 A graph is said to be connected if for every pair of its vertices u and v
there is a path from u to v.
 Connected component
 The maximum connected subgraph of a given graph.

Gedamu Alemu
1-53
Graph Properties – A cyclicity
 Cycle
 A simple path of a positive length that starts and ends a the
same vertex.
 Acyclic graph
 A graph without cycles
 DAG (Directed Acyclic Graph)  1  2
 3  4

Gedamu Alemu
1-54
Trees
 Trees
 A tree (or free tree) is a connected acyclic graph.
 Forest: a graph that has no cycles but is not necessarily connected.
 Properties of trees
 For every two vertices in a tree there always existsexactly
rooted one simple
path from one of these vertices to the other.  3
 1  3  5
─ Rooted trees: The above property makes it possible to select an

arbitrary vertex in a free tree and consider it as the root of
 the
2 so 4
 4  1  5
called rooted tree.
─ Levels in a rooted tree.  2
Forest

Gedamu Alemu
1-55
Rooted Trees (I)
 Ancestors/predecessors
 For any vertex v in a tree T, all the vertices on the simple path from the root to
that vertex are called ancestors.
 Descendants/children
rooted
 All the vertices for which a vertex v is an ancestor are said to be descendants of
v.  3
 Parent, child and siblings
 If (u, v) is the last edge of the simple path from the root to vertex v,4u issaid
1 to 5
be the parent of v and v is called a child of u.
 Vertices that have the same parent are called siblings.  2
 Leaves
 A vertex without children is called a leaf.
 Subtree
 A vertex v with all its descendants is called the subtree of T rooted at v.

Gedamu Alemu
1-56
Rooted Trees (II)
 Depth of a vertex
 The length of the simple path from the root to the vertex.
 Height of a tree
 The length of the longest simple path from the root to a
leaf.
h=2
 3
 4  1  5
 2

Gedamu Alemu
1-57
Ordered Trees
 Ordered trees
 An ordered tree is a rooted tree in which all the children of each vertex
are ordered.
 Binary trees
 A binary tree is an ordered tree in which every vertex has no more than
two children and each children is designated either a left child or a
right child of its parent. 9
 Binary search trees 6 8
 Each vertex is assigned a number. 5 2 3
 A number assigned to each parental vertex is larger than all the

numbers in its left subtree and smaller than all the numbers in its right
subtree.
 6
 3  9
 2  5  8

Gedamu Alemu

Chapter 1

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 1

Uploaded by

Copyright:

Available Formats

Introduction to Algorithms

Computer Science and Engineering Program: DAA

Computer Science and Engineering Program: DAA

Computer Science and Engineering Program: DAA

Computer Science and Engineering Program: DAA

input “computer” output

Computer Science and Engineering Program: DAA

 Problem solving strategy even if computers are not involved

Computer Science and Engineering Program: DAA

Input “computer ” output

Computer Science and Engineering Program: DAA

 Instance: The sequence <5, 3, 2, 8, 3>

Computer Science and Engineering Program: DAA

 Sorting and Searching

Some of problems don’t have efficient algorithms, or algorithms at all!

Computer Science and Engineering Program: DAA

 How to express algorithms

 Efficiency (or complexity) analysis

 Empirical/ experimental analysis

Computer Science and Engineering Program: DAA

Computer Science and Engineering Program: DAA

 How good is the algorithm?

 Does there exist a better algorithm?

Computer Science and Engineering Program: DAA

Computer Science and Engineering Program: DAA

Addition and Subtraction

 Simply , think of 32 add ( 30+2) ; then add it to 47 ; then add

Computer Science and Engineering Program: DAA

Computer Science and Engineering Program: DAA

Computer Science and Engineering Program: DAA

Add the following :

Computer Science and Engineering Program: DAA

Try this by yourself :

Computer Science and Engineering Program: DAA

 Try it this way (subtraction instead of addition)

Computer Science and Engineering Program: DAA

 Find and write the best algorithm to sum up the

 Write an algorithm to add 3-digit numbers (take

Computer Science and Engineering Program: DAA

Computer Science and Engineering Program: DAA

Computer Science and Engineering Program: DAA

More about numbers Euclid’s Algorithm

Examples: gcd(60,24) = 12, gcd(60,0) = 60, gcd(0,0) = ?

Euclid’s algorithm is based on repeated application of equality

Example: gcd(60,24) = gcd(24,12) = gcd(12,0) = 12

Computer Science and Engineering Program: DAA

E.g. find gcd(60, 24)

Computer Science and Engineering Program: DAA

More about numbers Sieve of

Computer Science and Engineering Program: DAA

Computer Science and Engineering Program: DAA

Computer Science and Engineering Program: DAA

 Combinatorial problems (counting objects number of sub graph in a

 Geometric problems( closest pair and convex-hull)

Computer Science and Engineering Program: DAA

Computer Science and Engineering Program: DAA

 Evaluate sorting algorithm complexity: the number of key comparisons.

Computer Science and Engineering Program: DAA

Computer Science and Engineering Program: DAA

input: sorted array a_i < … < a_j and

Computer Science and Engineering Program: DAA

Computer Science and Engineering Program: DAA