Data Structures

Lecture 1: Introduction
Algorithms

Algorithm
a well-defined computational procedure that takes some
value, or a set of values, as input and produces some
value, or a set of values, as output

Algorithm
sequence of computational steps that transform the
input into the output.

Algorithms research
design and analysis of algorithms and data structures
for computational problems
Data structures

Data Structure
a way to store and organize data to facilitate access and
modifications

Abstract data structure

describes functionality (which operations are supported)

Implementation
a way to realize the desired functionality
 how is the data stored (array, linked list, …)
 which algorithms implement the operations
The course
 Design and analysis of efficient algorithms for some
basic computational problems:

 Basic algorithm design techniques and paradigms

 Algorithms analysis: O-notation, recursions, …

 Basic data structures

 Basic graph algorithms

Incremental algorithms

Incremental algorithms
process the input elements one-by-one and maintain the
solution for the elements processed so far.
Incremental algorithms

In pseudo-code:
IncAlg (A) comment
► incremental algorithm which computes the solution of a problem
with input A = { x1,…,xn }
1. initialize: compute the solution for { x1 }
2. for i ← 2 to n assignment (Pascal :=)
3. do begin
4. compute the solution for { x1,…,xj } using the (already
computed) solution for { x1,…,xj-1 }.
5. end the book uses indentation only
An incremental sorting algorithm

InsertionSort (A)
► incremental algorithm that sorts array A[1..n] in non-decreasing
order
1. initialize: sort A[1]
2. for j ← 2 to length[A]
3. do begin
4. sort A[1..j] using the fact that A[1.. j-1] is already sorted.
5. end
An incremental sorting algorithm

InsertionSort (A)
► incremental algorithm that sorts array A[1..n] in non-decreasing
order
1. initialize: sort A[1]
2. for j ← 2 to length[A]
3. do begin
4. sort A[1..j] using the fact that A[1.. j-1] is already sorted.
5. end
key ← A[ j ] ; i ← j -1
1 j n while i > 0 and A[ i ] > key
1 3 14 17 28 6 … do A[ i+1] ← A[ i ]; i ← i -1
A[ i +1] ← key
Correctness proof via an invariant
InsertionSort (A)
1. initialize: ---
2. for j ← 2 to length[A]
3. do begin
4. key ← A[ j ] ; i ← j -1
while i > 0 and A[ i ] > key
do begin A[ i+1] ← A[ i ]; i ← i -1 end
A[ i +1] ← key
5. end

Inv(k): subarray A[1..k] consists of the elements originally A[1..k] but in

sorted order.

 Inv(1) holds after initialization, hence Inv(j-1) holds before step 4 is

executed.
Correctness proof via an invariant
InsertionSort (A)
1. initialize: ---
2. for j ← 2 to length[A]
3. do begin
4. key ← A[ j ] ; i ← j -1
while i > 0 and A[ i ] > key
do begin A[ i+1] ← A[ i ]; i ← i -1 end
A[ i +1] ← key
5. end

Inv(k): subarray A[1..k] consists of the elements originally A[1..k] but in

sorted order.

 If Inv(j-1) holds before step 4 is executed, then Inv(j) holds after

step 4 is executed once.
Correctness proof via an invariant
InsertionSort (A)
1. initialize: ---
2. for j ← 2 to length[A]
3. do begin
4. key ← A[ j ] ; i ← j -1
while i > 0 and A[ i ] > key
do begin A[ i+1] ← A[ i ]; i ← i -1 end
A[ i +1] ← key
5. end

Inv(k): subarray A[1..k] consists of the elements originally A[1..k] but in

sorted order.

 After the algorithm has terminated, Inv(n) holds. Inv(n) implies that
the output of the algorithm is correct.
Divide-and-conquer

Divide-and-conquer
break the problem into two or more subproblems, solve
the subproblems recursively, and then combine these
solutions to create a solution to the original problem.
Divide-and-conquer

D&CAlg (A)
► divide-and-conquer algorithm that computes the solution of a
problem with input A = { x1,…,xn }
1. if # elements of A is small enough (for example 1)
2. then compute Sol (the solution for A) brute-force
3. else begin
4. split A in, for example, 2 non-empty subsets A1 and A2
5. Sol1 ← D&CAlg (A1) ; Sol2 ← D&CAlg (A2 )
6. compute Sol (the solution for A) from Sol1 and Sol2
7. end
8. return Sol
Sorting by divide-and-conquer: MergeSort

MergeSort (A)
► divide-and-conquer algorithm that sorts array A[1..n]
1. if length[A] = 1
2. then compute Sol (the solution for A) brute-force
3. else begin
4. split A in 2 non-empty subsets A1 and A2

5. Sol1 ← MergSort (A1) ; Sol2 ← MergeSort (A2)

6. compute Sol (the solution for A) from Sol1 en Sol2
7. end
Sorting by divide-and-conquer: MergeSort

MergeSort (A)
► divide-and-conquer algorithm that sorts array A[1..n]
1. if length[A] = 1
2. then skip
3. else begin
4. n ← length[A] ; n1 ← floor(n/2); n2 ← ceil(n/2);
copy A[1.. n1] to auxiliary array A1[1.. n1]
copy A[n1+1..n] to auxiliary array A2[1.. n2]

• MergeSort (A1); MergeSort (A2)

• Merge (A, A1, A2)

• end
Sorting by divide-and-conquer: MergeSort

3 14 1 28 17 8 21 7 4 35

1 3 4 7 8 14 17 21 28 35

3 14 1 28 17 8 21 7 4 35

1 3 14 17 28 4 7 8 21 35

3 14 1 28 17

3 14 1 17 28

3 14
Sorting by divide-and-conquer: MergeSort
 Merging

A1 1 3 14 17 28 A2 4 7 8 21 35

A 1 3 4 7 8 14 17 21 28 35
MergeSort: correctness proof

Induction on n (# of input elements)

 proof that the base case (n small) is solved correctly

 proof that if all subproblems are solved correctly, then

the complete problem is solved correctly
MergeSort: correctness proof
Lemma
MergeSort sorts the array A[1..n] correctly.

Proof (by induction on n)

Base case: n = 1, trivial.
Inductive step: assume n > 1. Note that n1 < n and n2 < n.
Inductive hypothesis ➨ arrays A1 and A2 are sorted
correctly
Remains to show: Merge (A, A1, A2 ) correctly con- structs
a sorted array A out of the sorted arrays A1 and A2 …
etc.
■
Sorting by divide-and-conquer: QuickSort
QuickSort (A)
► divide-and-conquer algorithm that sorts array A[1..n]
• if length[A] ≤ 1
• then skip
• else begin
• pivot ← A[1]
• move all A[i] with A[i] < pivot into auxiliary array A1

• move all A[i] with A[i] > pivot into auxiliary array A2

• move all A[i] with A[i] = pivot into auxiliary array A3

• QuickSort (A1); QuickSort (A2)

• A ← “A1 followed by A3 followed by A2”

• end
Analysis of algorithms
 Can we say something about the running time of an
algorithm without implementing and testing it?

InsertionSort (A)
1. for j ← 2 to length[A]
2. do begin
3. key ← A[ j ] ; i ← j -1

4. while i > 0 and A[ i ] > key

5. do begin A[ i+1] ← A[ i ]; i ← i -1 end

6. A[ i +1] ← key

7. end
Analysis of algorithms
 Analyze the running time as a function of n (# of input
elements)

 best case
 average case
 worst case

An algorithm has worst case running time T(n) if

for any input of size n the maximal number of
elementary operations executed is T(n).

elementary operations
add, subtract, multiply, divide, load, store, copy, conditional
and unconditional branch, return …
Analysis of algorithms: example
n=10 n=100 n=1000
InsertionSort: 15 n2 + 7n – 2 1568 150698 1.5 x 107
10466 204316 3.0 x 106
MergeSort: 300 n lg n + 50 n

InsertionSort InsertionSort
6 x faster 1.35 x faster

MergeSort
5 x faster

n = 1,000,000 InsertionSort 1.5 x 1013

MergeSort 6 x 109 2500 x faster !
Analysis of algorithms
 It is extremely important to have efficient algorithms for
large inputs

 The rate of growth (or order of growth) of the running

time is far more important than constants

InsertionSort: Θ (n2)
MergeSort: Θ (n log n)
Θ-notation

Intuition: concentrate on the leading term, ignore constants

19 n3 + 17 n2 - 3n becomes Θ(n3)

2 n lg n + 5 n1.1 - 5 becomes Θ(n1.1)

n - ¾ n √n becomes ---
Some rules and notation
 lg n denotes log2 n

 We have for a, b, c > 0 :

1. logc (ab) =
logc a + logc b

2. logc ( ab ) =
b logc a

3. loga b =
logc b / logc a
Find the leading term
 lg35n vs. √n ?

 logarithmic functions grow slower than polynomial

functions
 lga n grows slower than nb for all constants a > 0 and
b>0

 n100 vs. 2 n ?

 polynomial functions grow slower than exponential

functions
 na grows slower than bn for all constants a > 0 and
b>1