ELuIsS HOROWITZ SARTAJ SAHNI SANGUTHEVAR RAJASEKARANCOMPUTER ALGORITHMSCOMPUTER SCIENCE PRESS Allred V. Aho, Columbia University Jeffrey D Ullman, Stanford University Foundations of Computer Science: Pascal Edition Foundations of Computer Science: C Edition Michael J. Clancy, University of California at Berkeley Marcia C. Linn, University of California at Berkeley Designing Pascal Solutions: A Case Study Approach Designing Pascal Solutions: Case Studies Using Data Structures ‘A. K. Dewdney, University of Western Ontario The New Turing Omnibus: 66 Excursions in Computer Science Introductory Computer Science: Bits of Theory, Bytes of Practice Robert Floyd, Stanford University Richard Beigel, Yale University The Language of Machines: An Introduction to ‘Computability and Formal Languages Michael R. Garey, Bell Laboratories David S. Johnson, Bell Laboratories Computers and Intractability: A Guide to the Theory of NP-Completeness Judith L. Gersting, University of Hawaii at Hilo Mathematical Structures for Computer Science, Third Edition Visual Basic® Programming: A Laboratory Approach Ellis Horowitz, University of Southern California Sartaj Sahni, University of Florida Fundamentals of Data Structures in Pascal, Fourth Edition Ellis Horowitz, University of Southern California Sartaj Sahni, University of Florida Susan Anderson-Freed, Illinois Wesleyan University Fundamentals of Data Structures in C Ellis Horowitz, University of Southern California Sartaj Sahni, University of Florida Dinesh Mehta, University of Tennessee Space Institute Fundamentals of Data Structures in C++ Ellis Horowitz, University of Southern California Sartaj Sahni, University of Florida Sanguthevar Rajasekaran, University of Florida Computer Algorithms Ellis Horowitz, University of Southern California Sartaj Sahni, University of Florida Sanguthevar Rajasekaran, University of Florida Computer Algorithms/C++ ‘Thomas W. Parsons, Hofstra University Introduction to Compiler Construction Gregory J. E. Rawlins, Indiana University Compared to What?: An Introduction to the Analysis of Algorithms Wei-Min Shen, Microelectronics and Computer ‘Technology Corporation Autonomous Learning from the Environment James A. Storer, Brandeis University Data Compression: Methods and Theory Steven Tanimoto, University of Washington Elements of Artificial Intelligence Using Common Lisp, Second Edition Kim W. Tracy, Bell Labs/Lucent Technologies, Ine, Peter Bouthoorn, Griningen University Object-Oriented Artificial Intelligence Using CH Jeffrey D. Ullman, Stanford University Principles of Database and Knowledge-Base Systems, Vol I: Classical Database Systems Principles of Database and Knowledge-Base Systems, Vol I: The New TechnologiesCOMPUTER ALGORITHMS Ellis Horowitz University of Southern California Sartaj Sahni University of Florida Sanguthevar Rajasekaran University of Florida & Computer Science Press An imprint of W. H. Freeman and Company New YorkAcqisitions Editor: Richard Bonacci Project Editor: Penelope Hull Text Designer: The Authors Text Illustrations: The Authors Cover Designer: A Good Thing Cover Illustration: Tomek Olbinski Production Coordinator: Sheila Anderson Composition: The Authors Manufacturing: R R Donnelley & Sons Company Library of Congress Cataloging-in-Publication Data Horowitz, Ellis. Computer algorithms / Ellis Horowitz, Sartaj Sahni, Sanguthevar Rajasekaran. P. om, Includes bibliographical references and index. ISBN 0-7167-8316-9 1. Computer algorithms. 2. Pseudocode (Computer program language). I. Sahni, Sartaj. II. Rajasekaran, Sanguthevar. III, Title. QA76.9.443H67 1998 005.1NDC21 97-20318 cIp © 1998 by W. H. Freeman and Company. Alll rights reserved. No part of this book may be reproduced by any mechanical, photographic, or electronic process, or in the form of a phonographic recording, nor may it be stored in a retrieval system, transmitted, or otherwise copied for public or private use, without written permission from the publisher. Printed in the United States of America First printing, 1997 Computer Science Press An imprint of W. H. Freeman and Company 41 Madison Avenue, New York, New York 10010 Houndmills, Basingstoke RG21 6XS, EnglandTo my nuclear family, MARYANNE, PIPL, CHAN OCH, and TRA — Ellis Horowitz To, NEETA, AGAM, NEHA, and PARAM ~ Sartaj Sahni KEERAN, KRISHNA, PANDT, and PONNUTHAL ~— Sanguthevar RajasekaranContents PREFACE 1 INTRODUCTION ll ea] 14 15 WHAT IS AN ALGORITHM? .........-.-5 ALGORITHM SPECIFICATION . 1.2.1 Pseudocode Conventions 1.2.2 Recursive Algorithms PERFORMANCE ANALYSIS 1.3.1 Space Complexity 1.3.2. Time Complexity . 1 Asymptotic Notation (O, 2, @) 1.3.4 Practical Complexitie 1.3.5 Performance Measurement . . RANDOMIZED ALGORITHMS . . 1.4.1 Basics of Probability Theory 1.4.2. Randomized Algorithms: An Informal Description . . 1.4.3 Identifying the Repeated Element 1.4.4 Primality Testing . Le 145 Advantages and Disadvantages: REFERENCES AND READINGS 2 ELEMENTARY DATA STRUCTURES 21 2 STACKS AND Queues TREES 221 Terminology 2.2.2 Binary Trees DICTIONARIES . 2.3.1 Binary Search ‘Trees. 2.3.2 Cost Amortization . . xv Be 69 69 76 oe 78 83 89viii CONTENTS 24 PRIORITY QUEUES 91 24.1 Heaps...... 0. 92 Heapsort 99 25 SETS AND DISJOINT SET UNION . 101 Introduction 7 101 . Union and Find Opera - 102 2.6 GRAPHS . . +. 2 2.6.1 Introduction : ee 1D) 2.6.2 Definitions . . . ee 112 Graph Representations na0 eis 2.7. REFERENCES AND READINGS ......... poco hd DIVIDE-AND-CONQUER 127 GENERAL METHOD . . . .. 27 BINARY SEARCH... .. . 131 FINDING THE MAXIMUM AND MINIMUM... 2... 139 MERGE SORT . 145 QUICKSORT . 154 3.5.1 Performance Measurement 3.5.2. Randomized Sorting Algorithms 159 30; SELECTION sete et 165 3.6.1 A Worst-Case _ Algorithm 169 62 Tuplamentation cfSelecta) ere t 72 3.7 STRASSEN’S MATRIX MULTIPLICATION ........- 179 3.8 CONVEX HULL 3.8.1 Some Geometric Primitives... . 3.8.2 The QuickHull Algorithm . . . 185 3.8.3 Graham’s Scan... . 187 4 An O(nlogn) Divide-and-Conquer Algorithm... .. 188 3.9 REFERENCES AND READINGS ... . . 193 3.10 ADDITIONAL EXERCISES 194 THE GREEDY METHOD 197 4.1 THE GENERAL METHOD 4.2 KNAPSACK PROBLEM . tee we 198 4.3 TREE VERTEX SPLITTING. ..........000- 77203 4.4 JOB SEQUENCING WITH DEADLINES... ...... + 208 4.5 MINIMUM-COST SPANNING TREES . : 216 4.5.1 Prim’s Algorithm. .........-. aaa neCONTENTS ix 45.2 Kruskal’s Algorithm... 02.2.0 .000008 . . 220 453 An Optimal Randomized Algorithm («) ........ 225 4.6 OPTIMAL STORAGE ON TAPES . . , . 229 4.7 OPTIMAL MERGE PATTERNS... . . . . 234 4.8 SINGLE-SOURCE SHORTEST PATHS . 241 4.9 REFERENCES AND READINGS . See ee 249 4.10 ADDITIONAL EXERCISES ...........-... - 250 5 DYNAMIC PROGRAMMING 253 5.1 THE GENERAL METHOD................... 253 5.2. MULTISTAGE GRAPHS 257 5.3 ALL PAIRS SHORTEST PATHS . 265 5.4 SINGLE-SOURCE SHORTEST PATHS: . GENERAL WEIGHTS ......... 270 5.5 OPTIMAL BINARY SEARCH TREES (+) 275 5.6 STRING EDITING - + 284 5.7 O/1-KNAPSACK . . . 5 . 287 5.8 RELIABILITY DESIGN. ..............5 sees 295 5.9 THE TRAVELING SALESPERSON PROBLEM . . 298 5.10 FLOW SHOP SCHEDULING . .. 301 5.11 REFERENCES AND READINGS . .. 307 5.12 ADDITIONAL EXERCISES - 308 6 BASIC TRAVERSAL AND SEARCH TECHNIQUES 6.1 TECHNIQUES FOR BINARY TREES 6.2. TECHNIQUES FOR GRAPHS . 6.2.1 Breadth First Search and ‘Trav 6.2.2 Depth First Search and Traversal . : 6.3 CONNECTED COMPONENTS AND SPANNING TREES : 6.4 BICONNECTED COMPONENTS AND DFS 6.5 REFERENCES AND READINGS 7 BACKTRACKING 339 7.1 THE GENERAL METHOD..... 339 7.2. THE 8-QUEENS PROBLEM .. . 353 7.3 SUM OF SUBSETS. 7.4 GRAPH COLORING 7.5 HAMILTONIAN CYCLES 7.6 KNAPSACK PROBLEMx CONTENTS 7.7 REFERENCES AND READINGS . . 7.8 ADDITIONAL EXERCISES - 374 - 375 8 BRANCH-AND-BOUND Ble CHE METHOD) fest es Least Cost (LC) Search... . The 15-puzzle: An Example . . Control Abstractions for LC-Search Bounding 15 FIFO Branch-and-Bound 8.1.6 LC Branch-and-Bound 8.2. 0/1 KNAPSACK PROBLEM 8.2.1 LC Branch-and-Bound Solution 8.2.2 FIFO Branch-and-Bound Solution . . 8.3. TRAVELING SALESPERSON (+) 8.4 EFFICIENCY CONSIDERATIONS 8.5 REFERENCES AND READINGS 9 ALGEBRAIC PROBLEMS 417 9.1 THE GENERAL METHOD 417 9.2 EVALUATION AND INTERPOLATION 420 9.3 THE FAST FOURIER TRANSFORM 430 9.3.1 An In-place Version of the FFT » 435 9.3.2 Some Remaining Points ... . 438 9.4 MODULAR ARITHMETIC . 440 95 EVEN FASTER EVALUATION AND INTERPOLATION. 448 9.6 REFERENCES AND READINGS ............005 456 10 LOWER BOUND THEORY 457 10.1 COMPARISON TREES 458 10.1.1 Ordered Searching . -. 459 10.1.2 Sorting . . 459 O13 Selection ete . 464 10.2 ORACLES AND ADVERSARY ARGUMENTS - 466 10.2.1 Merging 467 10.22 Largest and Second Largest... 0.0. 468 10.2.3 State Space Method ...... 470 10.2.4 Selection 471 10.3 LOWER BOUNDS THROUGH REDUCTIONS 474