Module 2 Divide and Conquer: General method, Binary search, Recarrence equation for divide and conquer Finding the maximum and minimum (T2:3.1, 3.3,3.4), Merge sort, Quick sort (T1:4.1, 4.2), Strassens matrix multiplication (T2:3.8), Advantages and Disadvantages of divide and conquer. Decrease and| Conquer Approach: Topological Sort. (T1:5.3). RBT:L1, 12,13 Contents Divide and Conquer: ....u.nnninnn nnn sans A General Method a Master Theorem for Divide and Conquet.n.nrsswnnnninnninnnnnsnns5 Binary searchenrnnn wl Mergesort. 14 Quicksort.. Strassen’s Matrix Multiplication. 23 ‘Advantages of Divide & Conquer technique: 25 Limitations of divide & conquer technique: Decrease-and-Conquer. Topological Sorting. Divide and Conquer: General Method Many useful algorithms are recursive in nature, To solve a given problem, they call themselves recursively one or more times, These algorithms typically follow a divide & Conquer approach. A divide & Conquer method works by recursively breaking down a problem into two or more sub-problems of the same type, until these become simple enough (ie. smaller in size w.r.t. original problem) to be solved directly. The solutions to the sub-problems are then combines to give a solution to the original problem. Geethalaxmi, Department of ISE,CECStep 1: Divide the given big problem into a number of sub-problems that are similar tothe original problem but smaller in size. A sub-problem may be further divided into its sub- problems, A Boundary stage reaches when either a direct solution of a sub-problem atsome stage is available or it is not further sub-divided, When no further sub-division ispossible, we have a direct solution for the sub-problem. Step 2:Conquer(Solve) each solutions of each sub-problem (independently) by recursive calls; and then Step 3: Combine the solutions of each sub-problems to generate the solutions of original problem.in this unit we will solve the problems such as Binary Search, Searching -QuickSort, MergeSort, integer multiplication etc., by using Divide and Conquer method. Hence recurssive algorithms are used in divide and conquer strategy. Diagram 1 shows the general divide & conquer plan ey S/ “ 7 Staten oa ‘Seca oad peti oben Pseudo code Representation of Divide and conquer rule for problem “P” Algorithm: Control abstraction for divide-and-conquer Geethalaxmi, Department of ISE,CECAlgorithm DAndC(P) { if small(P) then return S(P) else divide P into smaller instances P1.P2.P3...Pk: apply DAndC to cach of these subprograms; // means DAndC(P1), DAndC(P2)... DAndC(Pk) rerum combine(DAndC(P1), DAndC(P2)..... DAndC(PK)): 3 } 1/P->Problem (“Mere sinall(P)> Boolean value function. If it fs true, then the function $ is Himvoked DAndC is lly invoked as DAndC(P),where P is the problem to be solved, Small(P)is a Boolean-valued function that determines whether the input size is small enough that the answer can be computed without splitting If this is so,the function Sis invoked. Otherwise the problem Ps divided i recursive applications of DAndC. Combine is a function that determines the }o smaller subproblems. These subproblems P1,P2,.....Pk are solved by solution to P using the solutions to the k subproblems. Ifthe size of P is n and the sizes of the kx subproblems arent,n2,...nk_respectively,then the computing time of DAndC is described by the recurrence relation T(n) = { g(n) n small T(r) +T(n2)+---+T (re) + f(n) otherwise where T(n) is the time for DAndC on any inpu t of sizen and g(n) is the time to compute the answer directly for small inputs. The function f(n) is the time for dividing P and combining the solutions to subproblem Example for GENERAL METHOD: As an example, let us consider the problem of computing the sum of n numbers ao, « ao Ifn> 1, we can divide the problem into two instances of the same problem. They are sum of the first | n/2|numbers Compute the sum of the 1st [n/2] numbers, and then compute the sum of another n/2 numbers. Combine the answers of two n/2 numbers sum. ie. Geethalaxmi, Department of ISE,CEC0+... han =(a0+ ot an/2) + (an/2+....+an-1) Assuming that size n is a power of b, to simplify our analysis, we get the following recurrence for the running time T(n).. T(n)=aT(n/b) +f(n) ‘This is called the general divide and-conquer recurrence. f[n) is a function that accounts for the time spent on dividing the problem into smaller ones and on combining their solutions. (For the summation example, 2andf(n)= 1. Advantages of DAnd¢: ‘The time spent on executing the problem using DAndC is smaller than other method. ‘This technique is ideally suited for parallel computation. ‘This approach provides an efficient algorithm in computer science. ‘Time Complexity of DAndC algorithm: Ta) ifn=1 Tn) = for (2) +f(m) itn> 1 where a,b arecontants This is called the general divide and-conquer recurrence. Recurrence Relations ‘The ecurence relations an equation or Iequalty that describes a function n terms of ts values of smaller inputs. The main ool for analysing the time efficiency of a recurrence algorthm sto setup a sum expressing the number execution fits base operation and ascertain the solutions order of growth, The methods here used for solving recurrence elation are 1. Substitution method 2. Master theorem Solving recurrences by Substitution method Idea: Make a guess for the form of the solution and prove by induction, Can be used to prove both upper bounds Of) and lower bounds 2). if in case order of growth is not given in the problem, Geethalaxmi, Department of ISE,CECExample 3.1 Consider the case in which a = 2 and b = 2. Let T(1) = 2 and f(n) =n. We have Tin) = 27 (n/2)+n = 22T(n/4) +n/2+n AT (n/4) + 2n 427 (n/8) +n/4]+2n 87 (n/8) +3n In general, we see that T(n) = 2°T(n/2!) + in, for any logyn > @ > 1. In particular, then, T(n) = 2!°%"T(n/2!®2") + nlogyn, corresponding to the choice of i = logan. Thus, T(n) = nT(1) + nloggn = nlogn + 2n. a Solving Reccurence using Master Theorem for Divide and Conquer An Instance of size n can be divided into b instances of size n/b, with “a” of them needing tobe solved. [a2 1,b > 1} Assume size n is a power of b. The recurrence for the running time T(n) is as follows: TC) = aT (a/b) +f) where: f(n) ~ a fune f that accounts for the time spent on dividing the problem into smaller ones and on combining their solutions Therefore, the order of growth of T(n) depends on the values of the constants a & b and the order of growth of the function f{n). Geethalaxmi, Department of ISE,CECTheorem: If fin) € © (n*) with d> 0 in recurrence equation Tin) = aT) + fim), the! 9 (n’) ab* Therefore: a>blie,2>2° Case 3 of master theorem holds good. Therefore: Tan) € @ cn".5) €0 (ns) € O(a) Geethalaxmi, Department of ISE,CECExample 3.3 Next consider the case in which a 2, and f(n) For this recurrenee, log, a = 1 and h(n) = f(n)/ @((logn}9). Hence. u(n} = Ollogn) and T(n) = nfT(1) + @(logn)] = O(relogn). o Example 3.4 As another example, consider the recurrence T(n) = TT (n/2)+ 18n?, n 2 2 and a power of 2. We obtain a = 7, b= 2, and f(n) = 18n?. So, logy, logy 7 © 2.81 and A(n) = 18n?/nloma7 = 18n?—l827 = O(n”), where log) T < 0. So, u(n) = O(1). The expression for T(r) is Tin) = nlomTir() +0) = O(n'ess7} as T(1) is assumed to be a constant. a Example 3.5 Asa final example, consider the reeurrence T(n) = 97 (n/3)+ 4n®, n > 3 and a power of 3. Comparing with (3.2), we obtain a = 9, b= 3. and f(n) = 4n®. So, logya = 2 and h(n) = 4n®/n® = 4n* = O2(n4). From ‘Table 3.1, we see that u(n) = O(h(n)) = O(n"). So, T(n) = w(T(1) + O(n')] = on") as T(1) can be assumed stant. a Exercise: 1 Solve the recurrence relation for the following choices of ab, and f{n) (c being a constant) (a)a=1b=2, and f{n)=en (b) a=5, b= 4,and f{n)= en? (6) a= 28,b = 3,and fn) = en? Applications of Divide and conquer rule or algorithm: Binary search, BQuick sort, Merge sort, GStrassen’s matrix multiplication, Finding The Maximum And Minimum This is a simple problem that can be solved by the divide and-conquer technique. The problem is to find the maximum and minimum items in a set of n elements. Geethalaxmi, Department of ISE,CECALGORITHM-1 Algorithm StraightMaxMin (a, n, max, min) //Set _mac’ to the maximum and _min‘ to the minimum of af Ln}; { max:=min:=al 1: tondo if (a{i]>max) then max:= afi; if a{i]min1 then min:=min1; i } MaxMin is a recursive algorithm that finds the maximum and minimum of the set of elements {a(i).a(H41),....00.a(j)}The situation of set sizes one (isj) and two (Isj-1) are handled separately. For sets containing more than two elements, the midpoint is determined (just as in binary search) and two new subproblems are generated, When the maxima and minima of these subproblems are generated, the two maxima are compared and the two minima are compared to achieve the solution for the entire set. Eg:- Suppose we simulate MaxMin on the following nine elements Geethalaxmi, Department of ISE,CECa [1] [2] [3] [4] [5] (6) [7] [8] [9] 22:13 -5 -8 15 60 17 3147 A good way of keeping track of recursive calls is to build a tree by adding a node each time anew call is made, For this algorithm each node has four items of information: i,j, max, and min, On trhe array a[] above, the tree is produced. ‘Trees of recursive calls of MaxMin 1,9,60,8 1,5,22,-8 6,9,60,17 1322-5 | [asa5-8 8,9,47, 31 12,22,13 33,5,5 We see that the root node contains 1 and 9 as the values of i and j corresponding to the initial call to MaxMin. This execution produces new call to MaxMin where i and j have the values 1,3 and 6,9, respectively, and thus split the set into two subsets of approximately the same size, From the tree, we can immediately see that the maximum depth of recursion is four(including the first call). ‘The order in which max and min are assigned values are follows: [1,2,22,13] [3,3,-5,-5] [1,3.22,-5] [4.5,15,-8] [1.5,22,-8] [6,7,60,17] [8.9,47.31] [6,9,60,17] [1.9,60,-8}. ‘What is the Number of element comparisons needed for MaxMin: If T(n) represents the number of element comparisons needed, then the resulting recurrence relation is: T([n/2])+T (In/2]) +2 m2 Tms< 1 0 Geethalaxmi, Department of ISE,CECWhen nis a power of 2, n=2 we can solve this equation by successive substitutions: ‘T(n) = 2T(n/2)+2 = 2(2T(n/4)+2) +2 =AT(n/A}+ 4 +2 = WT(2) +Y cates = 2h 42k -2 = 3n/2-2 When n is a power of 2, the number of comparisons in the best , average and worst case is 3n/2-2, COMPLEXITY ANALYSIS OF FINDING MAXIMUM&MINIMUM Consider the following nine elements to simulate MaxMin 2 13 5 8 1S Os B47 Fig below shows the tree constructed for the algorithm. Each node has four items of information: i, js max and min, 1,9, 60,-8 1,5,22,8 6,9, 60,17 is 3 1,3,22,5, 4,5, 16-8 667, 60,17 1,9, 0,8 L 1,2, 22,13 Geethalaxmi, Department of ISE,CECConsider the count when element comparisons have the same cost as comparisons between i andj. Let € (n) be this number. Assuming n=: for some positive integer k, we get Ch) { (nt) +3. n>2 2 n=2 C(n) = 2C (0/2) +3 =4C (n/4) +643 =2kic (ayy 7 =2K43*21.3 =5n/2-3 This is larger: if comparisons between array elements are costlier than comparisons of integer variable, then divide and conquer method is more efficient In both cases mentioned above the best case, average case and worst case complexity is O(n) Binary search 1, This algorithm finds the position of a specified input value (the search “key") within an array sorted by key value, 2 Ineach step, the algorithm compares the search key value with the key value of the middle element of the array. 3, Ifthe keys match, then a matching element has been found and its index, or position, is returned. Geethalaxmi, Department of ISE,CEC4, Otherwise, ifthe search key is less than the middle element's key, then the algorithm repeats its action on the sub-array to the left of the middle element or, ifthe search key is greater, then the algorithm repeats on sub array to the right of the middle element. 5. If the search element is less than the minimum position element or greater than the ‘maximum position element then this algorithm returns not found. We can write algorithm for binary search in 2 ways: 1, Recursive Manner 2. Iterative Manner Recursive Binary Search: 1 Algorithm BinSrch(a, i,i,:2) 2 ven an array ali : 1] of elements in nondecreasing 3 Jf order, 1 (log n) (O(log n):= forall cases. erage case> 60g n) [Best case> (1) Binary search algorithm by using iterative methodology: Geethalaxmi, Department of ISE,CEC1 Algorithm BinSearch(a,n,) 2 // Given an array a[1 - n] of elements in nondecreasing 3 /J order, n> 0, determine whether x is present, and 4 // if so, return j such that x = aly); else return 0. 5 6 low = 1; high = ns 7 while (Jow < high) do 8 { 9 mid == |(low + high)/2\; 10 if (x almid}) then low := mid + 1; 12 else return mid; 13 4 return 0; 15 } Algorithm 3.3 Iterative binary search Mergesort Mergesortis a perfect example of a successful application of the divide-and-conquer technique. It sorts a given array A[0.n - 1] by dividing it into two halves A[0. m/2 - 1] and A[ n/2 «n ~ 1], sorting each of them recursively, and then merging the two smaller sorted arrays into a single sorted one. ALGORITHM Mergesort(A[0.n~ 1) //Sorts array A{Oun~ 1) by recursive mergesort //input: An array A[0.n~ 1] of orderable elements //Output: Array A[0..2 - 1] sorted in nondecreasing order ifn>1 copy A[0.. n/2 ~ 1] to B[0..n/2 ~ 1] copy A[ n/2 .n ~ 1] to €[0..n/2~ 1] Mergesort(B{0..n/2~1]) Geethalaxmi, Department of ISE,CECMergesort(C[0.. n/2- 1)) Merge(B, C A) //see below The merging of two sorted arrays can be done as follows. Two pointers (array indices) are initialized to point to the first elements of the arrays being merged. The elements pointed to are compared, and the smaller of them is added to a new array being constructed; after that, the index of the smaller element is incremented to point to its immediate successor inthe array it was copied from, This operation is repeated until one of the two given arrays Is exhausted, and then the remaining elements of the other array are copied to the end of the new array. ALGORITHM Merge(B[0..p - 1], €[0..q - 1], A[0.p+q- 1) //Merges two sorted arrays into one sorted array //Input: Arrays B[0..p ~ 1] and €[0.g ~ 1] both sorted //Output: Sorted array A[0..p + q~ 1] of the elements of B and € 1-0; fe 0;kO0 while i< pandj1, C1) =0. Let us analyze Cmeng(n), the number of key comparisons performed during the merging stage. Ateach step, exactly one comparison is made, after which the total number of elements in the two arrays still needing to be processed is reduced by 1. In the worst case, neither of the two arrays becomes empty before the other one contains just one element (e.g,, smaller elements Geethalaxmi, Department of ISE,CECmay come from the alternating arrays). Therefore, for the worst case, Gnerae() have the recurrence -1,andwe Curorst(1) = 2Cworsln/2) +n — 1 forn>1, Cyors(l) =0. Hence, according to the Master Theorem, Gworst (n) € (n log n) (why?)- In fact, its easy to find the exact solution to the worst-case recurrence for m = 2: Gworst (n) = nlogen-n+1, A noteworthy advantage of mergesort over quick-sort and heapsort -is its stability. The principal short-coming of mergesort is the linear amount of extra storage the algorithm requires, Though merging can be done in-place, the resulting algorithm is quite complicated and of theoretical interest only, Exercises 1 Find the order of growth for solutions of the following recurrences. T(n)= 47 (n/2)+n, T(1)=1 T(n)= AT (n/2) +m, T (1 T (n) = 4T (n/2) +3, 7 (1, 2, Apply mergesort to sort the list E, X. A, M, P, L, E in alphabetical order. Quicksort Quicksort is the other important sorting algorithm that is based on the divide-and-conquer approach, Unlike mergesort, which divides its input elements according to their position in the array, quicksort divides them according to their value. A partition is an arrangement of the array’s elements so that all the elements to the left of some clement Als] are less than or equal to Als], and all the elements to the right of A[s] are greater than or equal to it: AO]... Afs—1] Als] Als +1]... Ala—1] ‘all are Als} Geethalaxmi, Department of ISE,CECObviously, after a partition is achieved, A[s] will be in ts final position in the sorted array, and we can continue sorting the two subarrays to the left and to the right of A[s] independently (eg. by the same method). Note the difference with mergesort: there, the division of the problem into two subproblems is immediate and the entire work happens in combining their solutions; here, the entire work happens in the division stage, with no work required to combine the solutions to the subproblems. Here is pseudocode of quicksort: call Quicksort(A[0..n ~ 1]) where ALGORITHM Quicksort(A(L.r) //Sorts a subarray by quicksort //tuput: Subarray of array A[Oun ~ 1}, defined by its left and right indices Tand //Output: Subarray A[L.r] sorted in nondecreasing order ifler s Partition(A[L.r]) //s sa split position Quicksort(A\L.s ~ 1) Quicksort(Als + 1.) Partition: Asa partition algorithm, Can partition A(0..n ~ 1] and, more generally, its subarray A[l.r] (0.<1 j, we will have partitioned the subarray after exchanging the pivot with A[j} ej in P all are sp. sp [2p all are2 p {at Finally, if the scanning indices stop while pointing to the same element, i.c., the value they are pointing to must be equal to p (why?). Thus, we have the subarray partitioned, with the split position s 5 tjzi> P allare

p We can combine the last case with the case of crossed-over indices (i > j) by exchanging the pivot with A[j] whenever i > j. Here is pseudocode implementing this partitioning procedure. ALGORITHM, HoarePartition(A[L.r]) //Partitions a subarray by Hoare’s algorithm, using the first element asa pivot //lwput: Subarray of array A[0..n ~ 1], defined by its left and right indices Fand r (or) //Output: Partition of A[L.r], with the split position returned as this function's value Geethalaxmi, Department of ISE,CECpoAll ick jerst repeat repeat i i+ 1 until Afi] p repeat j~ j-1 until Alj]

1, Chesi(1)=0. According to the Master Theorem, Cpe,:(12) € @(n log, n); solving it exactly for n= 2" yields Ches:(n) =n logy n. In the worst case, all the splits will be skewed to the extreme: one of the two subarrays will be empty, and the size of the other will be just 1 less than the size of the subarray being partitioned; The total number of key comparisons made will be equal to (n+ D422) 3 -3€ O(n’) Coorst(n) = (n+ VD $m pe$3= ‘Thus, the question about the utility of quicksort comes down to its average-case behavior. Geethalaxmi, Department of ISE,CECLet Cow(n) be the average number of key comparisons made by quicksort on a randomly ordered array of size n. A partition can happen in any position s (0 s ssn ~ 1) after n+ 1 comparisons are made to achieve the partition. After the partition, the left and right subarrays will have s and n~ 1 ~s elements, respectively. Assuming that the partition split can happen each position s with the same probability 1/n, we get the following recurrence relation: LS Yer + 1) + Cougs) + Cong — n = S)] forn>1, CaugO) =0, Cayg(l) =0. Its solution, which is much trickier than the worst- and best-case analyses, turns out to be Cayeltt) © 2n In n © 1.39n logy n. An example of sorting an array by quicksortis given in Figure 5.3. Geethalaxmi, Department of ISE,CECo12 3 «567 531 9 824 } i i sos 6 8 2 be i j sat 4 82 87 53 1 4 @ 4 9 7 531 4 28 9 7 ; 1 so 1 4 bb 9 7 231 4 § @ 97 i j (ara 251 4 sot 23 1 4 Pil (ea een rida zs a 243 4 1203 4 ‘ (eacr=d] fea] [Res] «TT s=2 iy 3 4 i i a4 art] | RTA 4 4 4 eo 7 ) i | 27 4 1 273 7 8 8 7 9 (a) FIGURE 5.3 Example of quicksort operation. (a) Array’s transformations with pivots shown in bold. (b) Tree of recursive calls to Quicksort with input values / and r of subarray bounds and split positions of a partition obtained. Ao) | Att) = [Am Geethalaxmi, Department of ISE,CECExercises 1. Apply quicksort to sort the list E, X. A, M, P, L, Ein alphabetical order. Draw the tree of the recursive calls made. Strassen’s Matrix Multiplication ‘The Strassen’s Matrix Multiplication find the product C of two 2 x 2 matrices A and B with just seven multiplications as opposed to the eight required by the brute-force algorithm. [2 2)-[2 = )[% 2) co cn ay ay bo bu [mtg ms tm, m3+ms me | Mia eee —_ ‘my = (yp + 441) * Boy +P) mg =a, + (by — boo), (a9 + aon) #1, (ayo — a9) * (boo + bor), my = (ayy — 443) + (by + 1). ms Thus, to multiply two 2x 2 matrices, Strassen’ algorithm makes 7 multiplications and 18 additions/subtrac ns, whereas the brute-force algorithm requires 8 multiplications and 4 additions, These numbers should not lead us to multiplying 2 x 2 matrices by Strassen’s algorithm. Its importance stems from its asymptotic superiority as matrix order n goes to infinity. Let A and B be two n x n matrices where n is a power of 2. (If n is not a power of 2, matrices can be padded with rows and columns of zeros.) We can divide A, B, and their product ¢ into four n/2 n/2 submatrices each as follows: Geethalaxmi, Department of ISE,CECCoo | Cor] _ [Aco | Aor Boo | Bor ColCud Lo! And Lo! Bn. ‘The value Coo can be computed either as Aoo * Boo + Aoi * Bio or as Mi + Ma~_Ms-+ Mz where Mi, Ms, Ms and My are found by Strassen’s formulas, with the numbers replaced by the corresponding submatrices. ‘The seven products of n/2 * n/2 matrices are computed recursively by Strassen's matrix multiplication algorithm, The asymptotic efficiency of Strassen's matrix multiplication algorithm If M(n) Is the number of multiplications made by Strassen’s algorithm in multiplying two nxn matrices, where n is a power of 2, The recurrence relation is M(n) = 7M(n/2) for n> 1,M(1)=1 ens 2k, M(2*)= 7M(2*4) = 77M(22)] = PM(2"4) = 7IM(2e) = 7M (24) = 7M(29) = TM() = 71) (Since M(1}=1) M(2*) = 7%, Since k= loge n, M(n) = 7%." nly? = 2807 which is smaller than n required by the brute-force algorithm. Since this savings in the number of multiplications was achieved at the expense of making extra additions, we must check the number of additions A(n) made Geethalaxmi, Department of ISE,CECby Strassen's algorithm. To multiply two matrices of order n>1, the algorithm needs to multiply seven matrices of order n/2 and make 18 additions/subtractions of matrices of size n/2; when n= 1, no additions are made since two numbers are simply multiplied. These observations yield the following recurrence relation: A(n) = 7A(n/2) + 18(n/2)?for n > 1, A(L) = 0. By closed-form solution to this recurrence and the Master Theorem, A(n) € @(n¥%”). which is a better efficiency class than @(n®)of the brute-force method, Advantages of Divide & Conquer technique: + For solving conceptually difficult problems like Tower Of Hanoi, divide & conquer is a powerful tool + Results in efficient algorithms + Divide & Conquer algorithms are adapted foe execution in multi-processor machines, + Results in algorithms that use memory eache efficiently Limitations of di ide & conquer technique: * Recursion is slow + Very simple problem may be more complicated than an iterative approach. Example: adding n numbers ete Decrease-and-Conquer The decrease-and-conquer technique is based on exploiting the relationship between a solution to a given instance of a problem and a solution to its smaller instance. Once such a relationship is established, it can be exploited either top down or bottom up. ‘The bottom-up variation is usually implemented iteratively, starting with a solution to the smallest instance of the problem; itis called sometimes the incremental approach. There are three major variations of decrease-and-conquer: Geethalaxmi, Department of ISE,CEC1. decrease by a constant 2, decrease by a constant factor 3. variable size decrease In the decrease-by-a-constant variation, the size of an instance is reduced by the same constant on each iteration of the algorithm. Typically, this constant is equal to one (Figure 4.1), although other constant size reductions do happen occasionally. Consider, as an example, the exponentiation problem of computing a" where a = 0 and nisa nonnegative integer, The relationship between a solution to an instance of size m and an instance of size n ~ 1 is obtained by the obvious formula a” = at-1-a, So the function f(n) = a” can be computed either “top down” by using its recursive definition a) the Subproblem FIGURE 4.1 Decrease-(by one}-anc-conquer technique. f(n=1)-a ifn >0, fO=]4 ifn =0, ay Geethalaxmi, Department of ISE,CECor “bottom up” by multiplying 1 by a n times. The decrease-by-a-constant-factor technique suggests reducing a problem instance by the same constant factor on each iteration of the algorithm, In most applications, this constant factor is equal to two, The decrease-by-half idea is illustrated in Figure 4.2, Foran example, let us revisit the exponentiation problems Ifthe instance of size n is to compute 4, the instance of half its size is to compute a*/, with the obvious relationship between the two: a = (a"”?)2, But since we consider here instances with integer exponents only, the former does not work for odd n. Ifm is odd, we have to compute a by using the rule for even-valued exponents and then multiply the result by a, To summarize, we have the following formula: Cees Solution to FIGURE 4.2 Decrease-{by half)-and-conquer technique, (ary? if nis even and positive, a=) a"-Y2.a ifnis odd, (42) 1 ifn=0. Geethalaxmi, Department of ISE,CECIfwe compute a” recursively according to formula (4.2) and measure the algo-rithm's efficiency by the number of multiplications, we should expect the algorithm to be in (log n) because, on each iteration, the size is reduced by about a half at the expense of one or two multiplications, Finally, in the variable-size-decrease variety of decrease-and-conquer, the size-reduction pattern varies from one iteration of an algorithm to another. Eu-clid’s algorithm for computing, the greatest common divisor provides a good ex-ample of such a situation, Recall that this algorithm is based on the formula ged(m, n) = ged(n, m mod n). ‘Though the value of the second argument is always smaller on the right-hand side than on the left-hand side, it decreases neither by a constant nor by a constant factor. Topological Sorting A directed graph, or digraph for short, is a graph with directions specified for all its edges. The adjacency matrix and adjacency lists are still two principal means of representing a digraph, There are only two notable differences between undirected and directed graphs in representing them: (1) the adjacency matrix of a directed graph does not have to be symmetric; (2) an edge ina directed graph has just one (not two) corresponding nodes in the digraph's adjacency lists. VS 6 4s © @) (b) FIGURE 4.5 (a) Digraph. (b) DFS forest of the digraph for the DFS traversal started at a. Geethalaxmi, Department of ISE,CECDepth-first search and breadth-first search are principal traversal algorithms for traversing digraphs as well, but the structure of corresponding forests can be more complex than for undirected graphs. Thus, even for the simple example of Figure 4.5a, the depth-first search forest (Figure 4.5b) exhibits all four types of edges possible in a DF'S forest of a directed graph: tree edges (ab, be, de), back edges (ba) from vertices to their ancestors, forward edges (ac) from vertices to their descendants in the tree other than their children, and cross edges (dc), which are none of the aforementioned types. Note that a back edge in a DFS forest of a directed graph can connect a vertex to its parent. Whether or not it is the case, the presence of a back edge indicates that the digraph has a directed cycle. Adirected cycle in a digraph is a sequence of three or more of its vertices that starts and ends with the same vertex and in which every vertex is connected to its immediate predecessor by an edge directed from the predecessor to the successor. For example, a, b, ais a directed cycle in the digraph in Figure 4.54, Conversely, if a DFS forest of a digraph has no back edges, the digraph is a dag, an acronym for directed acyclic graph. ‘Asa motivating example, consider a set of five required courses {C1, C2, C3, C4, C5} a part-time student has to take in some degree program. The courses can be taken in any order as long as the following course prerequisites are met: C1 and C2 have no prerequisites, C3 requires C1 and C2, C4 requires C3, and C5 requires C3 and C4. The student can take only one course per term. Inwhich order should the student take the courses? The situation can be modeled by a digraph in which vertices represent courses and directed edges indicate prerequisite requirements (Figure 4.6). In terms of this digraph, the question is whether we can list its vertices in such an order that for every edge in the graph, the vertex where the edge starts is listed before the vertex where the edge ends. (Can you find such an ordering of this digraph’s vertices?) This problem is called topological sorting. Topological Sorting: Thus, for topological sorting to be possible, a digraph in question must be a dag, ifa digraph has no directed cycles, the topological sorting problem for it has a solution. Geethalaxmi, Department of ISE,CECMoreover, there are two efficient algorithms that both verify whether a digraph is a dag and, if itis, produce an ordering of vertices that solves the topological sorting problem. The first algorithm is a simple application of depth-first search: 1, perform a DFS traversal and note the order in which vertices become dead-ends (ie, popped off the traversal stack). 2, Reversing this order yields a solution to the topological sorting problem, provided, of course, no back edge has been encountered during the traversal. If a back edge has been encountered, the digraph is not a dag, and topological sorting of its vertices is impossible. Q 4) {es all FIGURE 4.6 Digraph representing the prerequisite structure of five courses. @Q 4) C5) The popping-off order: og Cay C5, C4, C3, C1, C2 C3, The topologically sorted list: & ¢) C1, C25 C2 C1-»g3-»c4-c5 Ai all a ane co) ©) © FIGURE 4.7 (a) Digraph for which the topological sorting problem needs to be solved. (b) DFS traversal stack with the subscript numbers indicating the popping- off order. (c) Solution to the problem. Why does the algorithm work? When a vertex v is popped off'a DFS stack, no vertex u with an edge from u to v can be among the vertices popped off before v. Hence, any such vertex u will be listed after v in the popped- off order list, and before v in the reversed list. Geethalaxmi, Department of ISE,CECCc ¢c og delete C1 delete C2 a] dewreca © delete C4 delete C5 —> —> —> © @ ‘The solution obtained is C1, C2, C3, C4, C5 FIGURE 4.8 Illustration of the source-removal algorithm for the topological sorting problem. On each iteration, a vertex with no incoming edges is deleted from the digraph. ‘The second algorithm is based on a direct implementation of the decrease-(by one)-and- conquer technique: 1. Repeatedly, identify in a remaining digraph a source, which is a vertex with no incoming edges, and delete it along with all the edges outgoing from it. (If there are several sources, break the tie arbitrarily. Ifthere are none, stop because the problem cannot be solved—see Problem 6a in this section's exercises.) 2. The order in which the vertices are deleted yields a solution to the topological sorting problem. The application of this algorithm to the same digraph representing the five courses is given in Figure 4.8. Note that the solution obtained by the source-removal algorithm is different from the one obtained by the DFS-based algorithm, Both of them are correct, of course; the topological sorting problem may have several alternative solutions. As to applications of topological sorting in computer science, they include instruction scheduling in program compilation, cell evaluation ordering in spread-sheet formulas, and resolving symbol dependencies in linkers, Geethalaxmi, Department of ISE,CECExercises 4.2 1. Apply the DFS-based algorithm to solve the topological sorting problem for the following digraphs: Geethalaxmi, Department of ISE,CEC