Algorithmic Costs

COMP 103 #9
Menu

• Cost of operations and measuring

efficiency
• ArrayList: remove, add
• Typical Big ‘O’ results

• Assignment 3
• Undo: Rect/Oval -> Photo/Border

COMP103 2006/T3 2
Analysing Algorithms

• How well does an implementation

perform?
• What is the cost of its algorithms?
• isEmpty, size, iterator,
• Add, Remove, ensureCapacity?
• How do we measure cost?

COMP103 2006/T3 3
Analysing Costs
The costs of a program:
• Time:
• Run the program and count the milliseconds/minutes/days.
• Count number of steps/operations the algorithm will take.

• Space:
• Measure the amount of memory the program occupies
• Count the number of elementary data items the algorithm
stores.

• Question:
• Programs or Algorithms?
• Answer:
• Both are valuable:
• programs: benchmarking
• algorithms: analysis
COMP103 2006/T3 4
Benchmarking: program cost
• Measure: (Assignment 4 !!!)
• actual programs
• on real machines
• on specific input
• measure elapsed time
• measure real memory use

• Problems:
• what input ⇒ choose test sets carefully
use large data sets
don’t include user input
• other users/processes ⇒ minimise
average over many runs
• which computer? ⇒ specify details

COMP103 2006/T3 5
Benchmarking: program cost

• Write benchmarking programs that:

• Run the program/method many times
• Use large input (i.e. 1 Million items in a collection)
• Use the system clock to measure the time before and
after:

… // set up the input data

long start = System.currentTimeMillis();
… // run the method n times
double time = (System.currentTimeMillis() – start)/n;
… // output result

• Make sure you don’t measure extra actions like user

input.
• Have to report costs carefully – one test environment
might not be comparable with another (multi vs uni
processor).
COMP103 2006/T3 6
 Analysis: Algorithm complexity
• Abstract away from the details of
• the hardware (computers vary in speed and architecture)
• the operating system
• the programming language
• the compiler
• the program
• the input (choosing ‘valid’ test data can hard)

• Measure number of “steps” as a function of the data

size.
• worst case (easier)
• average case (harder)
• best case (easy, but useless)

• Next problem, what is a “step”

COMP103 2006/T3 7
What is a “step”?
• Count Representative Actions, Identify the key step(s)
• actions in the innermost loop (happen the most times)
• actions that happen every time round (not inside an “if”)
• actions involving “data values” (rather than indexes)
• representative actions (don’t need every action)

public E remove (int index){

    if (index < 0 || index >= count) throw new ….Exception();
    E ans = data[index];
    for (int i=index+1; i< count; i++) ← in the innermost loop
      data[i-1]=data[i];     
←Key Step
    count--;
data[count] = null;
    return ans;
 }

COMP103 2006/T3 8
Pragmatics
• Count the most expensive actions:
• Adding 2 numbers is cheap
• Raising to a power is not so cheap
• Comparing 2 strings may be expensive
• Reading a line from a file may be very expensive
• Waiting for input from a user or another program may take
forever…

• Sometimes we need to know about how the underlying

operations are implemented in the computer to choose well
(COMP203 and 206).

COMP103 2006/T3 9
Asymptotic Notation
• Big ‘O’
• We don’t care about small contributors.
• Focus on large contributors
• This is the dominant cost as n -> inf.
• So don’t need to care about any constants.

• Drop the constant factors and smaller terms:

• cost(n) = 3.47 n2 - 67n + 53 → cost is O(n2)

COMP103 2006/T3 10
ArrayList: get, set, remove
• Assume List contains n items.
• Cost of get and set:
• best, worst, average: O(1)
• ⇒ constant number of steps, regardless of n

• Cost of Remove:
n
• worst case:
• what is the worst case? from the front
• how many steps? n-1
• average case:
from the middle
• what is the average case?
• how many steps? n/2

COMP103 2006/T3 11
ArrayList: add
• Cost of add(index, value):
• what’s the key step? n
• worst case:

• average case:

public void add(int index, E item){

    if (index < 0 || index > count) throw new
IndexOutOfBoundsException();
    ensureCapacity(); But, what about this
    for (int i=count; i > index; i--)
      data[i]=data[i-1];
    data[index]=item;
    count++;
 } 
COMP103 2006/T3 12
ArrayList: add at end
• Cost of add(value):
• what’s the key step? n
• worst case:

• average case:

public void add (E item){

    ensureCapacity();
    data[count++] = item;
}
private void ensureCapacity () {
    if (count < data.length) return;
    E [ ] newArray = (E[ ]) (new Object[data.length * 2]);
    for (int i = 0; i < count; i++)
      newArray[i] = data[i];
    data = newArray;
  } 
COMP103 2006/T3 13
ArrayList: add at end
• Average case:
• average over all possible states and input. OR
• amortised cost over time.

• Amortised cost: total cost of adding n:

• first 10: cost = 1 each total = 10
• 11th: cost = 10+1 total = 21
• 12-20: cost = 1 each total = 30
• 21st: cost = 20+1 total = 51
• 22-40: cost = 1 each total = 70
• 41st: cost = 40+1 total = 111
• 42-80: cost = 1 each total = 150
• :
• -n total =

COMP103 2006/T3 14
ArrayList costs: Summary
• get O(1)
• set O(1)
• remove O(n)
• add (at i) O(n) (worst and average)
• add (at end) O(1) (average)
O(n) (worst)
O((2n-10)/n)
= O(1) (amortised average)
• Question:
• what would the cost be if the array size
is increased by 10 each time?
COMP103 2006/T3 15
Typical Costs in Big ‘O’
If data/input is size n

O(1) “constant” cost is independent of n

O(log(n)) “logarithmic” doubling size -> add a little to the cost
O(n) “linear” doubling size -> double the cost
O(n log(n)) “en-log-en” doubling size -> slightly more than double
cost
O(n2) “quadratic” doubling size -> 4 * the cost
O(n3) “cubic” doubling size -> 8 * the cost
.
:

O(2n) “exponential” adding one to size -> doubles the cost

=> You don’t want to run this algorithm!
O(n!) “factorial” adding one to size -> n the cost
=> You definitely don’t want this
algorithm!

COMP103 2006/T3 16
Typical Costs

COMP103 2006/T3 17