Algorithm Design and Analysis

(ADA)
CSE222
 Divide and Conquer Algorithms
1. Divide the problem into smaller
subproblems

2. Conquer the subproblems via recursive

calls

3. Combine the subproblem answers

 Counting Inversions in an Array
Input : An array A of (distinct) natural numbers,
suppose 1,2,3, … n (for this lecture) in any
arbitrary order

Output : The number of inversions in the array

= number of pairs (i, j) such that i < j , but A[i]
> A[j]
 Counting Inversions : Example
an
Input : 1, 3, 5, 2, 4, 6
UV

Output : 3
 Counting Inversions : Example
Input : 1, 3, 5, 2, 4, 6

Output :

Trivial algorithm solves in 𝑂(𝑛2 )

 Counting Inversions : Example
Input : 1, 3, 5, 2, 4, 6

Output :

Trivial algorithm solves in 𝑂(𝑛2 )

Today : An 𝑂(𝑛 log 𝑛) time algorithm
 Counting Inversions : Example
Input : 1, 3, 5, 2, 4, 6

Output :

Trivial algorithm solves in 𝑂(𝑛2 )

Today : An 𝑂(𝑛 log 𝑛) time algorithm
Q. Can we output all inversions in O(nlogn)?
 Counting Inversions : Motivation
E-commerce Websites : Collaborative filtering

Use inversion counting to determine ‘similarity’

between two customers
 Key Idea for Divide and Conquer
Suppose A is divided in to X and Y . Each is
of size
An inversion pair (i, j) is : Mz
 Key Idea for Divide and Conquer
Suppose A is divided in to X and Y .
An inversion pair (i, j) is :
A i
1. Left inversion : Both (i,j) in X
 Key Idea for Divide and Conquer
Suppose A is divided in to X and Y .
An inversion pair (i, j) is :

1. Left inversion : Both (i,j) in X

2. Right inversions : Both (i,j) in Y

 Key Idea # 1
Suppose A is divided in to X and Y .
An inversion pair (i, j) is :

1. Left inversion : Both (i,j) in X

2. Right inversions : Both (i,j) in Y

3. Split inversions : i in X, j in Y
 Example Again..
1 5 3 2 4 6

X Y
1. Left inversion : H2 XG
2. Right inversions : 11

3. Split inversions : X 27 Y 2
 Key Idea for Divide and Conquer
Suppose A is divided in to X and Y .
An inversion pair (i, j) is :

1. Left inversion : Both (i,j) in X

2. Right inversions : Both (i,j) in Y present

3. Split inversions : i in X, j in Y Design 0Th time
algorithm
Runtime KT n 2 T M2 to K
 Algorithm : High Level Idea
CountInv (array A, length n)
If n==1 return 0
X = A[1,2,…n/2] ,Y = A[n/2+1, …n ]

x = CountInv (X, n/2)

y = CountInv (Y, n/2)
z = CountSplitInv (X, Y, n)
Return x+y+z
 Key Idea # 2
How do we count split inversions in linear time?
 Key Idea # 2
How do we count split inversions in linear time?
unsorted their
if X Y are tricky
It
Aha ! Perhaps easier if X and Y were sorted
(We shall see in a minute why)
 Key Idea # 2
How do we count split inversions in linear time?

Aha ! Perhaps easier if X and Y were sorted

(We see in a minute why)

Idea : Ask for more from the recursive calls

 Key Idea # 2 : Why sorted subarrays help ?
Recap of ‘Merge’ from Mergesort
X ,Y : Sorted arrays of length n/2
Z : Output array of length n
i, j = 1
for k=1 to n
if X(i) < Y(j)
D(k) = X(i)
i++
else
D(k) = Y(j)
j++
 Key Idea # 2 : Why sorted subarrays help ?
Recap of ‘Merge’ from Mergesort
X ,Y : Sorted arrays of length n/2 Merge exposes split inversions :
Z : Output array of length n Y

f
i, j = 1 𝟏 𝟑 𝟓 𝟐 𝟒 𝟔
for k=1 to n
if X(i) < Y(j) I j
D(k) = X(i)
i++
else
D(k) = Y(j)
I
j++
 Key Idea # 2 : Why sorted subarrays help ?
Recap of ‘Merge’ from Mergesort
X ,Y : Sorted arrays of length n/2 Merge exposes split inversions :
Z : Output array of length n

IT
i, j = 1 𝟏 𝟑 𝟓 𝟐 𝟒 𝟔
for k=1 to n
if X(i) < Y(j) i j
D(k) = X(i)
i++ 2
else
D(k) = Y(j)
12
j++
 Key Idea # 2 : Why sorted subarrays help ?
Recap of ‘Merge’ from Mergesort
X ,Y : Sorted arrays of length n/2 Merge exposes split inversions :
Z : Output array of length n
i, j = 1 𝟏 𝟑 𝟓 𝟐 𝟒 𝟔
for k=1 to n
if X(i) < Y(j) i j
D(k) = X(i)
i++
else
I 23
D(k) = Y(j)
j++
 Key Idea # 2 : Why sorted subarrays help ?
Recap of ‘Merge’ from Mergesort
X ,Y : Sorted arrays of length n/2 Merge exposes split inversions :
Z : Output array of length n

I
i, j = 1 𝟏 𝟑 𝟓 𝟐 𝟒 𝟔
for k=1 to n
if X(i) < Y(j) i j
D(k) = X(i)

If
i++
else
D(k) = Y(j)
I 234
j++
 Key Idea # 2 : Why sorted subarrays help ?
Recap of ‘Merge’ from Mergesort
X ,Y : Sorted arrays of length n/2 Merge exposes split inversions :
Z : Output array of length n
i, j = 1 𝟏 𝟑 𝟓 𝟐 𝟒 𝟔
for k=1 to n
if X(i) < Y(j) i j
D(k) = X(i)
i++
else
D(k) = Y(j)
I 23456
j++
 Key Idea # 2 : Why sorted subarrays help ?
Recap of ‘Merge’ from Mergesort
X ,Y : Sorted arrays of length n/2 Merge exposes split inversions :
Z : Output array of length n
i, j = 1 𝟏 𝟑 𝟓 𝟐 𝟒 𝟔
for k=1 to n
if X(i) < Y(j) i j
D(k) = X(i)
i++
else
D(k) = Y(j)
j++
 A General Claim
Claim. The split inversions involving an element y in the
subarray Y is precisely n/2-i+1, where i is position of the
first pointer when y is copied to D

it itE
Proof:

x i Xli Y Y 4
Da For all elements
bet i I
Do u u u
Xli x 721 y sick
 Key Idea # 2 : Why sorted subarrays help ?
CountAndMerge
X ,Y : Sorted arrays of length n/2
Z : Output array of length n, count = 0
i, j = 1 Return count
for k=1 to n
if X(i) < Y(j)
D(k) = X(i)
i++
else
D(k) = Y(j)
j++ a
count = count + (n/2 – i + 1)
 Algorithm :
SortAndCountInv (array A, length n)
If n==1 return 0 T IA
X = A[1,2,…n/2] ,Y = A[n/2+1, …n ]
d
(B, x) = SortAndCountInv (X, n/2) E
s (C, y) = SortAndCountInv (Y, n/2)
(D, z) = CountAndMerge (X,Y,a n)
t
Return x+y+z D BC
Matrix Multiplication : Divide and Conquer
Matrix Multiplication
Square Matrices
n xn n x m n xn

𝑋 𝑌 = 𝑍
 Matrix Multiplication

If
i
𝑋 𝑌 = 𝑍

dot
𝑍𝑖𝑗 ∶ The product of ith row of X and jth column of Y
k
Zig 2 i n
Yay Cns
algorithm
 Matrix Multiplication

𝑋 𝑌 = 𝑍

𝑍𝑖𝑗 ∶ The product of ith row of X and jth column of Y

 Matrix Multiplication

𝑋 𝑌 = 𝑍

What is the best runtime we can hope for ?

0cm
 Matrix Multiplication

𝑋 𝑌 = 𝑍

A naïve iterative algorithm :

Cns
Divide and Conquer Idea

𝐴 𝐵 𝐸 𝐹
𝐶 𝐷 𝐺 𝐻

d 1 Check
A Et BE A Ft BH
f
C Et DG CF 1 DH
 Recursive algorithm I

StepIII Recursively
8 products
compute the

7Do the additions

Runtime 1 n 8 T M2 to ne 043
Master Thu ba D 2
1 n
95
1 n n
 Recursive Algorithm II : Strassen (1969)
Step 1 : Recursively Compute 7 cleverly chosen
products.

Step II : Perform additions and subtraction with total

runtime 𝑂(𝑛2 )

1 a 7 TAKI 10cal
M i m2 8704
Applying O
 Recursive Algorithm II : Strassen (1969)
Step 1 : Recursively Compute 7 cleverly chosen
products.

Step II : Perform additions and subtraction with total

runtime 𝑂(𝑛2 )

Question :What is the runtime of this algorithm

 The Cleverly Chosen Products X A

Pi A CF H Pz A By H
Pz Ct D E pg D G E
Y
E II
ATD Et H1 PG B D GTA
Ps
Pz A c Etf

I
 Matrix Multiplication : Trivia
Strassen : 𝑂(𝑛2.8074 )
W
At least 15 research papers till December 2020 on this topic !

Current best runtime ~ 2.37286 Strassen et at

(Josh Alman, Virginia Williams – ’21) y
Not practical ! Lytvyn
Can we :
1. Get the exponent down to 2 ?
2. Design ‘practical’ algorithms which are faster than Strassen ?
 O