Professional Documents
Culture Documents
Design and Analysis of Algorithms: Text Book
Design and Analysis of Algorithms: Text Book
Text Book:
Introduction to the Design and Analysis of Algorithms. - Anany Levitin
UNIT I
Introduction Notion of Algorithm Fundamentals of Algorithmic Solving Important Problem Types Fundamentals of the Analysis Framework Asymptotic Notations Basic Efficiency Classes
Introduction
Algorithmics is more than a branch of computer science.
It is the core of computer science, and, in all fairness, can be said to be relevant to most of science, business, and technology.
Notion of Algorithm
An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.
Notion of Algorithm
GCD of 2 Numbers
Euclids algorithm for computing gcd(m, n) Step 1: If n = 0, return the value of m as the answer and stop; otherwise, proceed to Step 2.
Step 2: Divide m by n and assign the value of the remainder to r. Assign the value of n to m and the value of r to n. Go to Step 1.
Step 3:
GCD of 2 Numbers
ALGORITHM Euclid(m, n) //Computes gcd (m, n) by Euclids algorithm //Input: Two nonnegative, not-both-zero integers m and n //Output: Greatest common divisor of m and n while n != 0 do r m mod n mn nr return m
Notion of Algorithm
Consecutive integer checking algorithm for computing gcd(m, n) Step 1:
Step 2:
Notion of Algorithm
Step 3: Divide n by t. If the remainder of this division is 0, return the value of t as the answer and stop; otherwise, proceed to Step 4.
Decrease the value of t by 1. Go to Step 2.
Step 4:
Notion of Algorithm
Middle-school procedure for computing gcd(m, n)
Step 1: Step 2: Find the prime factors of m. Find the prime factors of n.
Notion of Algorithm
Middle-school procedure for computing gcd(m, n)
Step 3: Identify all the common factors in the two prime expansions found in Step 1 and Step 2.
Step 4:
Compute the product of all the common factors and return it as the greatest common divisor of the numbers given.
Notion of Algorithm
For the numbers 60 and 24, we get
60 = 2 . 2 . 3 . 5 24 = 2 . 2 . 2 . 3 gcd(60, 24) = 2 . 2 . 3 = 12.
Notion of Algorithm
Sieve of Eratosthenes method of generating prime numbers:
Notion of Algorithm
ALGORITHM Sieve(n)
//Implements the sieve of Eratosthenes //Input: A positive integer n > 1 //Output: Array L of all prime numbers less than or equal to n for p2 to n do A[p]p
Notion of Algorithm
for p2 to n do if A[p] !=0 jpp while j n do A[j ]0 j j + p end while End for
Notion of Algorithm
//copy the remaining elements of A to array L of the primes i 0 for p2 to n do if A[p] = 0 L[i]A[p] i i + 1 return L
Notion of Algorithm
by
applying
Notion of Algorithm
gcd(31415, 14142) gcd(14142, 3131) gcd(3131, 1618) gcd(1618, 1513) gcd(1513, 105) gcd(1513, 105) gcd(105, 43) gcd(43, 19) = gcd(19, 5) = gcd(5, 4) gcd(4, 1) = gcd(1, 0) = 1.
Choosing between Exact and Approximate Problem Solving Algorithm Design Techniques
Examples
Write an algorithm for the sorting problem that sorts an array by counting, for each of its elements, the number of smaller elements and then uses this information to put the element in its appropriate position in the sorted array. Apply this algorithm to sorting the list 60, 35, 81, 98, 14, 47.
Examples
ALGORITHM ComparisonCountingSort(A[0..n 1]) //Sorts an array by comparison counting //Input: Array A[0..n 1] of orderable values //Output: Array S[0..n 1] of As elements sorted in nondecreasing order
Examples
for i 0 to n 1 do Count[i]0 for i 0 to n 2 do for j i + 1 to n 1 do if A[i]<A[j ] Count[j ]Count[j ]+ 1 else Count[i]Count[i]+ 1
Examples
for i 0 to n 1 do S[Count[i]] = a[i] Return S
For example, it takes longer to sort larger arrays, multiply larger matrices, and so on. Therefore, it is logical to investigate an algorithms efficiency as a function of some parameter n indicating the algorithms input size.
Of the four, the most time-consuming operation is division, followed by multiplication and then addition and subtraction, with the last two usually considered together.
Orders of Growth
The worst-case analysis provides very important information about an algorithms efficiency by bounding its running time from above.
Amortized Efficiency
It applies not to a single run of an algorithm but rather to a sequence of operations performed on the same data structure. It turns out that in some situations a single operation can be expensive, but the total time for an entire sequence of n such operations is always significantly better than the worst-case efficiency of that single operation multiplied by n.
Amortized Efficiency
So we can amortize the high cost of such a worst-case occurrence over the entire sequence in a manner similar to the way a business would amortize the cost of an expensive item over the years of the items productive life.
To compare and rank such orders of growth, computer scientists use three notations:
O(big oh), (big omega), and (big theta).
O - Notation
A function t (n) is said to be in O(g(n)), denoted t (n) O(g(n)), if t (n) is bounded above by some constant multiple of g(n) for all large n, i.e., if there exist some positive constant c and some nonnegative integer n0 such that t (n) cg(n) for all n n0.
O - Notation
Example:
100n + 5 O(n2)
O - Notation
- Notation
A function t (n) is said to be in (g(n)), denoted t (n) (g(n)), if t (n) is bounded below by some positive constant multiple of g(n) for all large n, i.e., if there exist some positive constant c and some nonnegative integer n0 such that t (n) cg(n) for all n n0.
- Notation
Example
n3 (n2)
- Notation
- Notation
A function t (n) is said to be in (g(n)), denoted t (n) (g(n)), if t (n) is bounded both above and below by some positive constant multiples of g(n) for all large n, i.e., if there exist some positive constants c1 and c2 and some nonnegative integer n0 such that c2g(n) t (n) c1g(n) for all n n0.
- Notation
Example:
- Notation
THEOREM
If t1(n) O(g1(n)) and t2(n) O(g2(n)), then
PROOF
- Notation
RULES
(a)
(b)
RULES
Log k n = O(n) for any constant k. This tells us that logarithms grow very slowly.
RULES
If two programs are expected to take similar times, probably the best way to decide which is faster is to code them both up and run them.
A much more convenient method for doing so is based on computing the limit of the ratio of two functions in question.
Log 2 n and n
n! And 2n
ASSIGNMENT
1. What is an algorithm? Explain the notion of an algorithm with a neat diagram. 2. Give and explain the important characteristics of an algorithm. 3. Write and explain the Euclids algorithm for computing GCD of any two integers. Hence find gcd(31415, 14142).
ASSIGNMENT
4. Explain with an example the consecutive integer checking algorithm for computing GCD of any two integers.
5. Implement Sieves algorithm to generate prime numbers between 1 and n. Trace the algorithm for n = 25. 6. Explain with a neat diagram, the sequence of steps to be followed in designing and analyzing an algorithm.
ASSIGNMENT
7. Write an algorithm for finding the distance between the two closest elements in an array of numbers.
8. Explain the important problem types. Give two examples for each. 9. Implement an algorithm that sorts an array by counting, for each of its elements, the number of smaller elements and then uses this information to put the element in its appropriate position in the sorted array. Apply this algorithm to sorting the list 60, 35, 81, 98, 14, 47.
ASSIGNMENT
10. Explain the terms: time and space efficiency.
11. Explain the units for measuring the running time. 12. Compare the orders of growth of various algorithms. 13. Define the following: worst case, best case and average case analysis of an algorithm.
ASSIGNMENT
14. Give the worst case, best case and average case analysis of sequential search algorithm. 15. What is amortized efficiency? Explain with an example.
ASSIGNMENT
16. For each of the following algorithms, indicate (i) a natural size metric for its inputs, (ii) its basic operation, and (iii) whether the basic operation count can be different for inputs of the same size:
a. computing the sum of n numbers b. computing n! c. finding the largest element in a list of n numbers d. Euclids algorithm e. sieve of Eratosthenes f. pen-and-pencil algorithm for multiplying two n-digit decimal integers
ASSIGNMENT
17. Explain the different asymptotic notations with an example for each notation.
18. If If t1(n) O(g1(n)) and t2(n) O(g2(n)), then show that t1(n) + t2(n) O(max{g1(n), g2(n)}). 19. Compare the orders of growth for the following: a). n(n 1) and n2 b) Log 2 n and n c) n! And 2n
ASSIGNMENT
20. Indicate whether the first function of each of the following pairs has a smaller, same or larger order of growth than second function.
a. b. c. d. e. f. n (n+1) and 2000n2 100n2 and 0.01n3 Log 2 n and ln n 2n-1 and 2n Log 2 n and log 2 n2 (n-1)! And n!
ASSIGNMENT
21. For each of the following functions, indicate the class (g(n)) the function belongs to. Prove your assertions.
22. Prove that if t1(n) (g1(n)) and t2(n) (g2(n)), then t1(n) + t2(n) (max{g1(n), g2(n)}).
ASSIGNMENT
23. Prove that every polynomial of degree k, p(n) = aknk + ak1nk1+ . . . + a0 with ak > 0, belongs to (nk). 24.List the following functions according to their order of growth from the lowest to the highest:
ASSIGNMENT
25. Find the Big oh notation for the following:
a. b. c. Log n + n n + n log n 6n + 2n4 + 4n5
26. With the help of a flowchart, explain the various stages of algorithm design and analysis process. 27. Explain the different problem types and give one algorithm for each type.
ASSIGNMENT
28. Use the informal definitions of O,, and to determine whether the following assertions are true or false.