ITEC 2620

Introduction to
Data Structures
:Time Complexity
Khairul Bashar
School of Information Technology
York University, Toronto.
Estimation
• How long will it take to…
• Re-sort the company phone book?
• Print the company payroll?
• Update a single record?
• Should it be run now or overnight?
• When will the changes be available?
• Now? Next day? Next pay cycle?
Estimation
• Re-sort n employees
• n2 cost – could be very slow
• Process n employees
• n cost – slow, but not too bad
• Process (find) 1 employee
• logn cost – should be pretty fast
Estimation
• Factors in Cost Estimation
• How many lines of code?
• What does each line of code entail?
• Size of data
• Nature of data
Estimation
• Math.max(a, b);
• two numbers -> constant time
• maxValue(anArray);
• n numbers -> varies with array size
• insertionSort(anArray);
• n numbers -> varies with array size
Estimation
• Constant time programs
• Run once, always the same
• Estimation not really required
• Variable time programs
• Run once
• Future runs depend on relative size of input
Complexity Analysis
• Selection Sort
• n elements in outer loop
• n-1, n-2, …, 2, 1 elements in inner loop
• average n/2 elements for each pass of the outer loop
• swap at the end of each loop
• n * n/2 compares + n swaps
Complexity Analysis
• The “cost” to run selection sort
• 10,000 records -> 50,000,000 compares + 10,000 swaps
• 100,000 records -> 5,000,000,000 compares + 100,000 swaps
• do the swaps really matter? 100,000 initializations?
Asymptotic Analysis
• Formally
• “What is the ultimate growth rate of an algorithm as a function of its input size?”
• Simplified
• “If the problem size doubles, approximately how much longer will it take?”
Asymptotic Analysis
• From n to 2n…
• Quadratic (iterative sorting)
• four times longer
• Linear (linear search)
• twice as long
• Logarithmic (binary search)
• one time unit longer
Big O Notation
• Big-Oh represents the “order of” the cost function
• Ignoring all constants, find the largest function of n in the cost function

• As n gets large, this term will dominate all others. This term will be your asymptotic
growth rate.
Big O Notation
• Selection Sort
• n * n/2 compares + n swaps
• O(n2)
• Linear Search
• n compares + n increments + 1 initialization
• O(n)
Practice
• Efficient algorithm matters!!

• Problem: Write a java program to add n consecutive numbers.

Simplifying Conventions
• Only focus on the largest function of n
• Ignore smaller terms
• Ignore constants
Simplifying Conventions
• size of functions of n
• n! – factorial growth
• cn – exponential growth
• nc – polynomial growth
• logn – logarithmic growth
• c – constant
Simplifying Conventions
• size of functions, as n grows from 10 to 50
• 10! ≈ 108 -> 50! ≈ 1060
• 210 ≈ 103 -> 250 ≈ 1015
• 102 = 102 -> 502 ≈ 103
• log10 ≈ 3 -> log50 ≈ 5
• 10 -> 1 -> 50 -> 1
Simplifying Conventions
• Sequential code sections
• addition, find largest term
• Nested code sections (e.g. looping)
• multiplication
• n times through loop -> n/2 compares + 1 swap
• n*n/2 compares + n swaps
Example 1
for i = 0 to n-1
for j = 0 to n-1
c[i][j] = 0
for k = 0 to n-1
c[i][j] += a[k][j] * b[j][k]
Example 2
for i = 0 to n-1
min <- a[i]
for j = i to n-1
if a[j] < a[i]
min = a[j]
minLocation = j
swap(I, minLocation)
Example 3
a is an array of n elements

selectionSort(a)
for i = 0 to n-1
binarySearch(I, a)
Best, Worst and Avg Cases
• Why do people buy lottery tickets?
• Win millions!
• Case?
• Best
Best, Worst and Avg Cases
• Why should you invest your retirement savings in the stock market?
• Best long-term investment (compared to cash and bonds)
• Case?
• Average
Best, Worst and Avg Cases
• Why should you not fly a kite in a thunderstorm?
• Get killed!
• Case?
• Worst
Best, Worst and Avg Cases
• Buying lottery tickets
• Average case
• Waste of money
Best, Worst and Avg Cases
• Investing in the stock market
• Worst case
• Market crash!
Best, Worst and Avg Cases
• Average Case?
• Most often
• Weighted average of all possible cases
• Only (best + worst)/2 when they have equal probability
• Best Case?
• When reasonable expectation exists
• Minimal execution
• Worst Case?
• When guarantees are required
• Maximal execution
Best, Worst and Avg Cases
• Best, Worst, and Average cases exist when there is conditional execution
• while loops
• Branches with different cost alternatives
• If it runs the same way every time, there is only one case
Complexity Estimation
• The most important thing about algorithm analysis is to determine Big O – the
approximate complexity
• Allows us to estimate program run times
• Do we need all the math?
• Not really…
Complexity Estimation
• Two key questions
• How many times?
• What does it cost?
Complexity Estimation
• “How many times?” is the summation
• “What does it cost?” is what’s inside the summation
• more accurate questions
• What does it cost each time?
• What does it cost on average?
Example
for (int i = 0; i < n; i++)
Is there a best, worst, and average case?
if (Math.random() > 0.5) • Yes
if (i%2 == 0) • Branches with different cost alternatives
O(logn)
else
O(1)
else
for (int j = 0; j < i; j++)
for (int k = 0; k < i; k++)
O(1)
Example
• Best case
Math.random() always > 0.5
How many times? •n
• What does it cost (on average)? •(1+logn) / 2
• Overall? •n(1+logn)/2 -> O(nlogn)
Example
• Worst case
• Math.random() never > 0.5
• How many times? •n
• What does it cost (on
average)?
• Go straight to big-Oh notation on each line
for (int i = 0; i < n; i++) // O(n)
for (int j = 0; j < i; j++) // O(n)
for (int k = 0; k < i; k++) // O(n)
O(1)
Example
• Worst case is O(n) * O(n) * O(n) * O(1)
• O(n3)

• Average case is ( O(nlogn) + O(n3) ) /2

• O(n3)
 Example 1 (Prev)

for i = 0 to n-1 // O(n)

for j = 0 to n-1 // O(n)
c[i][j] = 0 // O(1)
for k = 0 to n-1 // O(n)
c[i][j] += a[k][j] * b[j][k] // O(1)
Example 1 (Prev)
• How many times (does the first for loop happen)?
•n
• What does it cost (each time through the loop?
• Another loop
• The stuff inside the loop
Example 1 (Prev)
• O(n) * O(n) * { O(1) + [ O(n) * O(1) ] }

Third loop Third loop Third loop

Example 3 (Prev)

a is an array of n elements

selectionSort(a) // O(n2)
for i = 0 to n-1 // O(n)
binarySearch(I, a) // O(logn)
Quicksort
Pivot = 10
13 16 9 1 2 15 10

10
Pivot = 2 Pivot = 15
9 1 2 13 16 15

2 15

1 9 13 16
Practice
Function quicksort (arr, low, high) // low = lowest index of arr
if low < high // high = highest index of arr
pi <- partition(arr, low, high)
quicksort(arr, low, pi-1)
quicksort(arr, pi + 1, high)

Function partition(arr, low, high)

pivot <- arr[high]
i <- low -1
for j = low to high -1
if arr[j] < pivot
i++
swap(arr, i, j)
swap(arr, i+1, high)
return (i+1)
That’s all for Today!
Thank you