1

Algorithm Analysis

Algorithm:

Algorithm analysis is a
field of classical study
and research involving
scientific data and
complex Mathematics.
Offered as a separate
course!
 Algorithm Analysis

Algorithm:

- An algorithm is a sequence of computational steps that transform

the input into the desired output
- An algorithm is a tool for solving a well –specified computational
problem

 An algorithm is said to be correct if, for every input instance, it

halts with the correct output
 An incorrect algorithm might not halt at all on some input
instance, or it might halt with other than the desired output.
 Algorithm Analysis

Program:
- A program is the expression of an algorithm in a programming
language
- A set of instructions which the computer will follow to solve a
problem
Problem:
- Description of input-output relationship
Data Structure:
- An organized method of storing and retrieving data
What we need to do:
- Given a problem, design a correct and robust algorithm that solves it
- To design a good algorithm, find a amenable data structure
 Algorithm Analysis

A Problem and its Solution:

Problem: Input is a sequence of integers stored in an array and output
the minimum.
 Algorithm Analysis

Analyzing Algorithm:
- Correctness
 Does the input/output relation match algorithm requirement?
- Complexity
 Basic operations involved
- Space Used
 Memory used
- Simplicity
 Verification and implementation
Optimality
 Is it possible to do better?
 Algorithm Analysis

Analysis of Algorithms:

Questions:

- How long will my program take?

- Why does my program run out of memory?

Answers:
- Computer being used
- Data being processed
- Algorithm being used
- ……..
 Algorithm Analysis

Analysis of Algorithms:

Scientific method:

- Observe: feature of the nature with precise measurements

- Hypothesize (supposition): a model consistent with observation
- Predict: events using hypothesis
- Verify: predictions by further observations
- Validate: repeat until hypothesis and observations agree

Experiments must be
reproducible
 Algorithm Analysis

Analysis of Algorithms:
Observation:

How long will my

program take
(Running Time)?

Problem
size!
 Algorithm Analysis

Observation: Example 3-Sum

Problem (3-Sum):
Given N distinct integers, how many triples sum to exactly zero?
 Algorithm Analysis

3-Sum: brute-force algorithm:

 Algorithm Analysis

Measuring the running time:

Manual:
 Algorithm Analysis

Standard plot: Plot running time T(N) vs. input size N.

 Algorithm Analysis

Log-log plot: Plot running time T(N) vs. input size N using log-log scale.
 Algorithm Analysis

Prediction and validation:

 Algorithm Analysis

Mathematical models for running time:

Total running time: Time for each operation x frequency for all
operations.

- Need to analyze program to determine set of operations

- Cost depends on machine, compiler
- Frequency depends on algorithm, input data
 Algorithm Analysis

Cost of basic operations: How to estimate constants

 Algorithm Analysis

Observation: Most primitive operations take constant time

 Algorithm Analysis

Example: 1- Sum
 Algorithm Analysis

Example: 2 – Sum
 Algorithm Analysis
Example: 3- Sum
 Algorithm Analysis

Simplification 1: tilde (asymptotic approximation) notation:

 Algorithm Analysis

Mathematical models for running time:

We use: T(N) ~ c N3
 Algorithm Analysis

Order-of-growth classifications:
The order of growth of the running time of this code is N 3
 Algorithm Analysis

Common order-of-growth classifications:

 Algorithm Analysis

Order-of-growth classifications:
The set of functions
 Algorithm Analysis

Asymptotic Performance:
- How does the algorithm behaves as the problem size
gets very large?
 Running time
 Memory storage requirements
 Bandwidth/power/logic gates etc.,
 Algorithm Analysis

Asymptotic Analysis:
- Worst case
 Provides an upper bound on running time
 An absolute guarantee
- Average case
 Provides the expected running time
 Very useful, but with care: what is “average”?
 Random inputs
 Rea-life inputs
- Best case
 Algorithm Analysis

Memory: Basics
 Algorithm Analysis

Typical memory usage:

 Algorithm Analysis

Typical memory usage for objects in java:

 Algorithm Analysis

Typical memory usage summary:

 Algorithm Analysis

Example:
 Acknowledgements

1. Dr. Abul Kashem Mia@buet

2. Algs4.cs@princeton