Algorithms

Design and Analysis

 Course Objectives
• This course introduces students to the analysis and
design of computer algorithms. Upon completion of this
course, students will be able to do the following:
• Analyze the asymptotic performance of algorithms.
• Demonstrate a familiarity with major algorithms and
data structures.
• Apply important algorithmic design paradigms and
methods of analysis.
• Synthesize efficient algorithms in common
engineering design situations.
 Course Objectives
In this course we will cover the following topics:
 Understand foundations of algorithms &&
design and analysis various variants
algorithms
 Accuracy
 Efficiency
 Comparing efficiencies
 Algorithms
• The word algorithm comes from the name of a Persian
mathematician Abu Ja’far Mohammed ibn-i Musa al
Khowarizmi.

• In computer science, this word refers to a special

method useable by a computer for solution of a problem.
The statement of the problem specifies in general terms
the desired input/output relationship.

• For example, sorting a given sequence of numbers into

nondecreasing order provides fertile ground for
introducing many standard design techniques and
analysis tools.
 What is a program?

 A program is the expression of an algorithm in a

programming language

 a set of instructions which the computer will follo

w to solve a problem
 Problem-Solving (Science and Engineering)

• Analysis
• How does it work?
• Breaking a system down to known components
• How the components relate to each other
• Breaking a process down to known functions

• Synthesis
• Building tools and toys!
• What components are needed
• How the components should be put together
• Composing functions to form a process
 Problem Solving Using Computers

• Problem:
• Strategy:
• Algorithm:
• Input:
• Output:
• Step:
• Analysis:
• Correctness:
• Time & Space:
• Optimality:
• Implementation:
• Verification:
 Why need algorithm analysis ?

 Writing a working program is not good enough

 The program may be inefficient!

 If the program is run on a large data set, then

the running time becomes an issue
 Analyzing Algorithms and Problems

• We analyze algorithms with the intention of

improving them, if possible, and for choosing
among several available for a problem.

• Correctness
• Amount of work done, and space used
• Optimality, Simplicity
 Correctness can be proved!

• An algorithm consists of sequences of steps

(operations, instructions, statements) for
transforming inputs (preconditions) to outputs
(postconditions)

• Proving
• if the preconditions are satisfied,
• then the postconditions will be true,
• when the algorithm terminates.
 Amount of work done

• We want a measure of work that tells us

something about the efficiency of the method
used by the algorithm

• independent of computer, programming

language, programmer, and other
implementation details.

• Usually depending on the size of the input

 Amount of work done

•Counting passes through loops

•Basic Operation
• Identify a particular operation fundamental to the
problem
• the total number of operations performed is roughly
proportional to the number of basic operations
•Identifying the properties of the inputs that affect
the behavior of the algorithm
 Space Usage

•If memory cells used by the algorithms depends

on the particular input,

• then worst-case and average-case analysis can be

done.
•Time and Space Tradeoff.
 Optimality “the best possible”

•Each problem has inherent complexity

• There is some minimum amount of work required to
solve it.
•To analyze the complexity of a problem,
• we choose a class of algorithms, based on which
• prove theorems that establish a lower bound on the
number of operations needed to solve the problem.
•Lower bound (for the worst case)
 Simplicity

•Simplicity in an algorithm is a virtue.

 Designing Algorithms

•Problem solving using Computer

•Algorithm Design Techniques

• divide-and-conquer
• greedy methods
• depth-first search (for graphs)
• dynamic programming
The problem of sorting
 Analysis of algorithms

The theoretical study of computer-program

performance and resource usage.

What’s more important than performance?

• modularity
• correctness
• maintainability
• functionality
• robustness
• user-friendliness
• programmer time
• simplicity
• extensibility
• reliability
 Analysis of algorithms
Why study algorithms and performance?

• Algorithms help us to understand scalability.

• Performance often draws the line between what is feasible

and what is impossible.

• Algorithmic mathematics provides a language for talking

about program behavior.

• The lessons of program performance generalize to other

computing resources.

• Speed is fun!
 Running Time

• Therunning time depends on the input: an already

sorted sequence is easier to sort.

• Parameterize the running time by the size of the

input, since short sequences are easier to sort than
long ones.

• Generally, we seek upper bounds on the running

time, because everybody likes a guarantee.
 Kinds of analyses

Worst-case: (usually)
• T(n) = maximum time of algorithm on any input of
size n.

Average-case: (sometimes)
• T(n) = expected time of algorithm over all inputs of
size n.
• Need assumption of statistical distribution of inputs.

Best-case:
• Cheat with a slow algorithm that works fast on some
input.
 Machine-Independent time

The RAM Model

Machine independent algorithm design depends on a

hypothetical computer called Random Acces Machine (RAM).

Assumptions:
• Each simple operation such as +, -, if ...etc takes exactly
one time step.
• Loops and subroutines are not considered simple
operations.
• Each memory acces takes exactly one time step.
 Machine-independent time

What is sort’s worst-case time?

• It depends on the speed of our computer,

• relative speed (on the same machine),
• absolute speed (on different machines).

BIG IDEA:
• Ignore machine-dependent constants.
• Look at growth of T ( n) as n  
“Asymptotic Analysis”
 Asymptotic Analysis
Engineering:

•Drop low-order terms;

•ignore leading constants.

•Example:
Asymptotic performance