DP Optimizations

Dynamic
Programming
2
DP
Optimizations
Convex Hull Dynamic Programming 2

Trick
Construction COMP4128 Programming Challenges
Application
Examples
Divide and
Conquer
Optimization
Divide and
Conquer School of Computer Science and Engineering
Framework
Proving UNSW Australia
monotonicity of
opt
Modifications
Table of Contents 2
Dynamic
Programming
2
1 DP Optimizations
DP
Optimizations
Convex Hull 2 Convex Hull Trick

Trick
Construction Construction
Application
Examples Application
Divide and Examples
Conquer
Optimization
Divide and
Conquer
Framework 3 Divide and Conquer Optimization
Proving
monotonicity of
opt
Divide and Conquer Framework
Modifications Proving monotonicity of opt
Modifications
DP Optimizations Overview 3
Dynamic
Programming
2
Sometimes when you’re doing DP your solution will be too

DP
Optimizations slow but your recurrence is very structured.
Convex Hull
Trick You’ve seen this before with ranges and range trees.
Construction
Application This lecture will show a few more cases of structured
Examples
Divide and
recurrences where there are well known techniques for
Conquer
Optimization
speeding them up.
Divide and
Conquer Again, this does not help if your state space is too large.
Framework
Proving
monotonicity of
This should not really change your overall approach to DP
opt
Modifications
but you should keep an eye out for recurrences that can be
sped up.
Table of Contents 4
Dynamic
Programming
2
1 DP Optimizations
DP
Optimizations

Trick
Application
Divide and Examples
Conquer
Optimization
Divide and
Conquer
Proving
monotonicity of
opt
Modifications
Convex Hull Trick Motivation 5
Dynamic
Programming
2
DP
Optimizations
Convex Hull
Trick One of the easiest to apply and most useful optimizations.
Construction
Application
Examples
Handy when your recurrence is formed by linear functions.
Divide and
Conquer
Though once you get good at spotting these, more things
Optimization look like a linear function than you might expect!
Divide and
Conquer
Framework
Proving
monotonicity of
opt
Modifications
Dynamic
Programming
2
General setting is:

DP
Optimizations
You are doing a 1D DP. Let’s say you are calculating
Convex Hull
dp[N] in order of increasing i.
Trick You have an array m[N]. Think of these as gradients. The
Construction
Application array m[N] is in decreasing order.
Examples
You have an array p[N]. Think of these as the positions of
Divide and
Conquer the points for which you are calculating dp[N].
Optimization
Divide and
Conquer
You have some base case value dp[0] and the recurrence is:
Framework
Proving
monotonicity of
opt dp[i] = min(dp[j] + m[j] ∗ p[i])
Modifications j<i
Dynamic
Programming
2
DP
Optimizations Take a look at the recurrence:
Convex Hull
Trick
Construction
dp[i] = min(dp[j] + m[j] ∗ p[i])
Application
j<i
Examples
Divide and
Conquer
Hopefully you see an O(N 2 ) solution.
Optimization
Divide and
What do those terms in the min look like?
Conquer
Framework
Proving
Equations for lines with dp[j] as the y-intercept and m[j]
monotonicity of
opt as the slope.
Modifications
Dynamic
Programming Recurrence:
2
j<i
DP
Optimizations
Suppose so far we have:
Convex Hull
Trick
dp[0] = −2, m[0] = 2
Construction dp[1] = −1, m[1] = 1
Application
Examples dp[2] = 5, m[2] = −1
Divide and
Conquer
And we want to calculate dp[3] (leave p[3] unfixed for
Optimization
Divide and
now).
Conquer
Framework
Then dp[i] is the minimum of a set of lines at the x
Proving
monotonicity of coordinate p[3] where our lines are:
y = m[0]x + dp[0] = 2x − 2
opt
Modifications
y = m[1]x + dp[1] = x − 1
y = m[2]x + dp[2] = −x + 5
So far we have done this in O(N) for each dp[i]
calculation. We will see how to speed this up.
Dynamic
Programming
2
To speed up this recurrence, we will build a data structure
DP that exactly supports the operations we need.
Optimizations
It will export 2 methods:
Convex Hull
Trick void add(int m, int b): Add a line l = mx + b.
Construction
Application Requirement: m is strictly less than the gradient of all
Examples
lines in the data structure already.
Divide and
Conquer int query(q): Over all lines {li = mi x + bi } that we
Optimization have added so far, return mini (mi q + bi ).
Divide and
Conquer
Framework
Proving
We will have add run in O(1) amortized and query run in
monotonicity of
opt O(log n) where n is the number of lines added so far.
Modifications
Afterwards we will apply this data struture as a black box

to some DP problems.
CHT Data Structure Construction 10
Dynamic
Programming
2
DP
Optimizations To get the right complexity, we need some observations
Convex Hull
Trick
first regarding the set of lines we are querying.
Construction
Application
For concreteness, let us return to our earlier example.
Examples Suppose we have:
Divide and
Conquer
b[0] = −2, m[0] = 2
Optimization b[1] = −1, m[1] = 1
b[2] = 5, m[2] = −1
Divide and
Conquer
Framework
Proving
monotonicity of
opt
(where m[i] are the gradients and b[i] are the y-intercepts)
Modifications
Dynamic
Programming
2
DP
Optimizations
Convex Hull
Trick
Construction
Application
Examples
Divide and
Conquer
Optimization
Divide and
Conquer
Framework
Proving
monotonicity of
opt
Modifications
1
https://www.geogebra.org/graphing
Dynamic
Programming
2
DP
Optimizations
Convex Hull
Trick
Remember our aim is to query the minimum over these 3
Construction
Application
lines at the x coordinate q.
Examples
What could this possibly be?
Divide and
Conquer
Optimization
Let’s emphasize what the minimum value is at each x
Divide and
Conquer
coordinate.
Framework
Proving
monotonicity of
opt
Modifications
Dynamic
Programming
2
DP
Optimizations
Convex Hull
Trick
Construction
Application
Examples
Divide and
Conquer
Optimization
Divide and
Conquer
Framework
Proving
monotonicity of
opt
Modifications
2
https://www.geogebra.org/graphing
Dynamic
Programming
2 Key Observation: This is the upper convex hull of the
areas below the lines.
DP
Optimizations This means each line is optimal for a contiguous range of
Convex Hull x (possibly empty) and as we move left to right, the
Trick
Construction gradient of the optimal line never increases.
Application
Examples Our goal is to maintain the convex hull by maintaining the
Divide and
Conquer
set of line segments that the convex hull is made out of.
Optimization
Divide and We will say a line l is dominant at x-coordinate x if it is
Conquer
Framework
Proving
the line that forms the convex hull at x. This is the line
monotonicity of
opt that gives us our minimum in the equation
Modifications
query (q) = min(mi · q + bi )

i
Dynamic
Programming
2
For this, it suffices to store the lines that make up the

DP
Optimizations convex hull in left to right order. (same as in decreasing
Convex Hull gradient order).
Trick
Construction
Application
Note: Importantly, this does not contain all the lines. It
Examples
omits any line that is never in the upper convex hull.
Divide and
Conquer
Optimization
Given this data, we can calculate the range of x at which
Divide and
Conquer
each line l is dominant. The segment is the range between
Framework
Proving
the intersection of l with the line before it, and after it, in
monotonicity of
opt the convex hull.
Modifications
We keep the lines in a vector.

Dynamic
Programming
2
DP
Optimizations struct line { long long m , b ; };
double i n t e r s e c t ( line a , line b ) {
Convex Hull return ( double ) ( b . b - a . b ) / ( a . m - b . m ) ;
Trick }
Construction // I n v a r i a n t : cht [ i ]. m is in d e c r e a s i n g order .
Application vector < line > cht ;
Examples /*
* The i n t e r s e c t i o n points are
Divide and * i n t e r s e c t ( cht [0] , cht [1]) , i n t e r s e c t ( cht [1] , cht [2]) , ...
Conquer * Line i is optimal in the range
Optimization * [ i n t e r s e c t ( cht [i -1] , cht [ i ]) , i n t e r s e c t ( cht [ i ] , cht [ i +1]) ]
Divide and * where for i = 0 , the first point is - infinity ,
Conquer
Framework * and for i = N -1 , the last point is i n f i n i t y .
Proving */
monotonicity of
opt
Modifications
Dynamic
Programming
2
DP
Optimizations
We keep the lines in a vector.
Convex Hull
Trick Recall the 2 methods our data structure is meant to
Construction
Application support are:
Examples
What’s the line on the upper convex hull at x = q?
Divide and
Conquer (Equivalently, what is mini (mi · q + bi ) over all the lines)
Optimization
Divide and
Add the line y = m[i] ∗ x + dp[i].
Conquer
Framework
Proving We handle the former with binary search.
monotonicity of
opt
Modifications
CHT Query Implementation 18
Dynamic
Programming
2
Our aim is to find the line that is dominant at x = q.
DP
Suppose our lines are ordered in decreasing gradient order.
Optimizations Then recall the range at which line li is dominant is
Convex Hull
Trick
Construction [intersect(li−1 , li ), intersect(li , li+1 )]
Application
Examples
Divide and where for i = 0, the left term is −∞ and for the last line,
Conquer
Optimization the right term is +∞.
Divide and
Conquer
Framework
So to find the i for which the segment contains q, it
Proving
monotonicity of suffices to find the minimum i such that
opt
Modifications
intersect(li , li+1 ) ≥ q
CHT Query Implementation 19
Dynamic
Programming struct line { long long m , b ; };
2 double i n t e r s e c t ( line a , line b ) {
return ( double ) ( b . b - a . b ) / ( a . m - b . m ) ;
}
// I n v a r i a n t : cht [ i ]. m is in d e c r e a s i n g order .
DP vector < line > cht ;
Optimizations
/* Recall that the range the ith line is do m i n a n t in is :
Convex Hull
* [ i n t e r s e c t ( cht [i -1] , cht [ i ]) , in t e r s e c t ( cht [ i ] , cht [ i +1]) ]
Trick
* We want to find the line that is do m i n a n t at x .
Construction * To do this , we note that the se q u e n c e ( i n t e r s e c t ( cht [ i ] , cht [ i +1]) )
Application
* is m o n o t o n i c a l l y i n c r e a s i n g in i .
Examples
* Hence we can binary search for the minimum i such that
Divide and * i n t e r s e c t ( cht [ i ] , cht [ i +1]) >= x
Conquer */
Optimization long long query ( long long x ) {
Divide and int lo = 0; int hi = cht . size () -2;
Conquer // Find largest idx such that x <= i n t e r s e c t ( cht [ idx ] , cht [ idx +1])
Framework // If this doesn ’t exist then idx should be cht . size () -1.
Proving int idx = cht . size () -1;
monotonicity of
opt while ( lo <= hi ) {
Modifications int mid = ( lo + hi ) /2;
if ( i n t e r s e c t ( cht [ mid ] , cht [ mid +1]) >= x ) {
idx = mid ; hi = mid -1;
} else { lo = mid +1; }
}
return cht [ idx ]. m * x + cht [ idx ]. b ;
}
CHT Add Implementation 20
Dynamic
Programming
2
DP
Optimizations
To add a line we crucially use the fact that m[N] is in
Convex Hull
Trick decreasing order.
Construction
Application So the new line has to go on the end of our convex hull
Examples
Divide and
(recall the convex hull is sorted in non-increasing order of
Conquer
Optimization
gradients).
Divide and
Conquer
Framework
However, this may cause some lines to disappear from the
Proving
monotonicity of
convex hull.
opt
Modifications
Dynamic
Programming
2
For example, consider adding the line y = −2x + 2 to the
earlier example:
DP
Optimizations
Convex Hull
Trick
Construction
Application
Examples
Divide and
Conquer
Optimization
Divide and
Conquer
Framework
Proving
monotonicity of
opt
Modifications
What is the new convex hull?

Dynamic
Programming
2
So some of the lines may become useless and we need to

DP remove them.
Optimizations
Convex Hull When does a line l become useless? When the line we just
Trick
Construction
added covers the entire range l is dominant in.
Application
Examples Observation: The useless lines are always at the end of
Divide and
Conquer
the convex hull.
Optimization
Divide and
So we just need to keep popping the last line of the
Conquer
Framework convex hull as long as it is useless.
Proving
monotonicity of
opt How do we check if the last line is useless?
Modifications
For this, it helps to draw pictures and move your new line
l around.
Dynamic
Programming
2
You will hopefully observe some variant of the following:

DP Letting cht[N − 1] be the last line, it is useless if:
Optimizations
intersect(cht[N − 1], l) ≤ intersect(cht[N − 2], cht[N − 1])

Convex Hull
Trick
Construction
Application
Examples Recall the range that cht[N − 1] is dominant in is
Divide and
Conquer (intersect(cht[N − 2], cht[N − 1]), ∞).
Optimization
Divide and The above essentially says l is better than cht[N − 1] for
Conquer
Framework
Proving
this entire range.
monotonicity of
opt
Modifications
Another way of phrasing this is that the intersections
(intersect(cht[i], cht[i + 1])) need to be kept in increasing
order.
CHT Full Implementation 24
Dynamic
struct line { long long m , b ; };
Programming
2
}
// I n v a r i a n t : cht [ i ]. m is in d e c r e a s i n g order .
DP vector < line > cht ;
Optimizations
void add ( line l ) {
Convex Hull auto n = cht . size () ;
Trick while ( n >= 2 &&
Construction i n t e r s e c t ( cht [n -1] , cht [n -2]) >= i n t e r s e c t ( cht [n -1] , l ) ) {
Application cht . pop_bac k () ;
Examples n = cht . size () ;
}
Divide and cht . p u s h _ b a c k ( l ) ;
Conquer }
Optimization
Divide and long long query ( long long x ) {
Conquer
Framework int lo = 0; int hi = cht . size () -2;
Proving // Find largest idx such that x <= i n t e r s e c t ( cht [ idx ] , cht [ idx +1])
monotonicity of int idx = cht . size () -1;
opt
while ( lo <= hi ) {
Modifications
int mid = ( lo + hi ) /2;
} else { lo = mid +1; }
}
}
Dynamic
Programming
2
DP
Optimizations
Complexity? O(1) amortized per add. O(log n) per
Convex Hull
Trick query.
Construction
Application But do remember, we did assume the gradients are in
Examples
Divide and
decreasing order. It is non-trivial to remove this
Conquer
Optimization
assumption.
Divide and
Conquer In certain special cases, we can actually get a better
Framework
Proving
monotonicity of
complexity!
opt
Modifications
CHT Data Structure Modifications 26
Dynamic
Programming
2
DP One common case, each query we make is at least the

Optimizations
previous query we make. (formally, say our code calls
Convex Hull
Trick query with query (q1 ), query (q2 ), . . . , query (qQ ). Then
q1 ≤ q2 ≤ . . . ≤ qQ )
Construction
Application
Examples
Divide and In this case, the index of the dominant line for each qi
Conquer
Optimization only increases.
Divide and
Conquer
Framework
So we keep track of the index of the dominant line.
Proving
monotonicity of Whenever we query, we check if the current dominant line
opt
Modifications is still the dominant line for the new query point.
Dynamic
Programming If cp is the current dominant line, this amounts to
2
checking if
DP
Optimizations
p[i] ≤ intersect(cht[cp], cht[cp + 1])
Convex Hull
Trick While this does not hold, increase cp.
Construction
Application The above had an omission. There is a special case. Since
Examples
the number of lines in the convex hull can decrease, cp
Divide and
Conquer may be out of bounds. To fix this, after each update we
Optimization
Divide and just need to update cp to point to the last line if it is out
Conquer
Framework
Proving
of bounds.
monotonicity of
opt
Modifications
If you want to rigorously check this, the invariant you are
maintaining is
intersect(cht[cp − 1], cht[cp]) ≤ p[i]
Dynamic
Programming
2
struct line { long long m , b ; };
DP }
Optimizations // I n v a r i a n t : cht [ i ]. m is in d e c r e a s i n g order .
vector < line > cht ;
Convex Hull int cp ;
Trick
Construction void add ( line l ) {
Application auto n = cht . size () ;
Examples while ( n >= 2 &&
i n t e r s e c t ( cht [n -1] , cht [n -2]) >= i n t e r s e c t ( cht [n -1] , l ) ) {
Divide and cht . pop_bac k () ;
Conquer n = cht . size () ;
Optimization }
Conquer
Framework cp = min ( cp , ( int ) cht . size () -1) ;
Proving }
monotonicity of
opt
long long query ( long long x ) {
Modifications
while ( cp +1 != cht . size () &&
i n t e r s e c t ( cht [ cp ] , cht [ cp +1]) < x ) cp ++;
return cht [ cp ]. m * x + cht [ cp ]. b ;
}
Dynamic
Programming
2
DP
Optimizations
Convex Hull
Trick
Construction
Application Complexity? Now updates and queries are both O(1)
Examples
Divide and
amortized.
Conquer
Optimization
Divide and
Conquer
Framework
Proving
monotonicity of
opt
Modifications
More Modifications 30
Dynamic
Programming
2
DP
Optimizations
Convex Hull Currently query returns the minimum over all lines. We
Trick
Construction
can also instead return the maximum. If so, our invariant
Application
Examples
is that our gradients should be increasing. We are now
Divide and calculating the lower convex hull of the area above all lines.
Conquer
Optimization But we can actually use the exact same code! (literally no
Divide and
Conquer
Framework modifications needed)
Proving
monotonicity of
opt
Modifications
Dynamic
Programming
2
DP
Optimizations
Not hard to adjust for gradients non-increasing (instead of
Convex Hull
Trick decreasing). Just one extra special case.
Construction
Application Can also have no conditions on the gradients. We still
Examples
Divide and
keep lines in sorted gradient order. However, we now need
Conquer
Optimization
to keep lines in a set since insertions are arbitrary. Not
Divide and
Conquer
that common mostly because it is tedious to get right.
Framework
Proving
monotonicity of
Alternatively we can also use a ”Li Chao tree”.
opt
Modifications
Dynamic
Programming Another useful modification is when everything (gradients
2
and query points) is in integers.
DP
Currently we use doubles to compute the intersection
Optimizations
points. This gives us the precise ranges at which we
Convex Hull
Trick
dominate (up to precision errors)
Construction
Application
[intersect(cht[i − 1], cht[i]), intersect(cht[i], cht[i + 1])].
Examples
Doubles are not precise though even for the range of a
Divide and
Conquer long long. Usually not problematic for competitions since
Optimization
Divide and numbers generally only go up to 109 but you have to keep
Conquer
Framework
Proving
it in mind if you need to query higher.
monotonicity of
opt If we only care about integer coordinates of x then we can
Modifications
round these down and work entirely in integers:
(⌊intersect(cht[i −1], cht[i])⌋, ⌊intersect(cht[i], cht[i +1])⌋]

Dynamic
Programming
2 But if we only care about integer coordinates of x then we
can round these down:
DP
Optimizations
Convex Hull
(⌊intersect(cht[i −1], cht[i])⌋, ⌊intersect(cht[i], cht[i +1])⌋]
Trick
Construction
Application Just be careful. If you want to do this, make sure it is
Examples
clear to you why the above is exclusive-inclusive.
Divide and
Conquer
Optimization
In integers, whether your inequalities are strict or not
Divide and
Conquer
actually matters. When you compare with an intersection
Framework
Proving point, mentally check if it agrees with the above ranges.
monotonicity of
opt
Modifications If we only care about positive integers, we can omit the
floor and just use regular integer division. But if you care
about negatives, note that integer division isn’t floor, it
rounds towards 0.
CHT without floats 34
Dynamic struct line { long long m , b ; };

Programming long long fl oo rd iv ( long long a , long long b ) {
2 return a / b - ( a % b && (( a <0) ^ (b <0) ) ) ;
}
long long i n t e r s e c t ( line a , line b ) {
floordiv ( b . b - a .b , a . m - b . m ) ;
DP // for PO SI TI VE ints : can do :
Optimizations // return ( b . b - a . b ) / ( a . m - b . m ) ;
}
Convex Hull
vector < line > cht ;
Trick
void add ( line l ) {
Construction
auto n = cht . size () ;
Application
while ( n >= 2 &&
Examples
i n t e r s e c t ( cht [n -1] , cht [n -2]) >= i n t e r s e c t ( cht [n -1] , l ) ) {
Divide and cht . pop_bac k () ;
Conquer n = cht . size () ;
Optimization }
Conquer }
Framework long long query ( long long x ) {
Proving int lo = 0; int hi = cht . size () -2;
monotonicity of
opt int idx = cht . size () -1;
Modifications while ( lo <= hi ) {
int mid = ( lo + hi ) /2;
// NOTE : It is c r i t i c a l here that this is >= not >.
} else { lo = mid +1; }
}
}
Convex Hull Trick Application 35
Dynamic
Programming
2
The above construction gives us exactly the data structure
we needed.
DP
Optimizations As a reminder, it supports 2 methods:
Convex Hull
Trick
void add(int m, int b): Add a line l = mx + b.
Construction Requirement: m is strictly less than the gradient of all
Application
Examples lines in the data structure already.
Divide and Complexity: O(1).
Conquer
Optimization int query(q): Over all lines {li = mi x + bi } that we
Divide and
Conquer
have added so far, return
Framework
Proving
monotonicity of
opt
min mi q + bi
Modifications
i
Complexity: O(log n) in general, we can make it O(1) if

queries are given in non-decreasing order.
Dynamic
Programming
2
DP
Optimizations There are 2 knobs we can tweak for our data structure.
Convex Hull If queries are in non-decreasing order, we can replace our
Trick
Construction
query code with an O(1) routine.
Application
Examples
If everything is in integers, we can use integer division.
Divide and
Conquer
However, aside from these, we can essentially use it as a
Optimization black box!
Divide and
Conquer
Framework So the construction is good to know, but application wise,
Proving
monotonicity of
opt you can just copy paste it in if need be.
Modifications
Dynamic
Programming
2
DP
Optimizations
Let us return to our earlier DP problem. We had the
Convex Hull
recurrence:
Trick

Construction
Application
Examples j<i
Divide and
Conquer
Optimization where we assumed m[j] is in decreasing order.
Divide and
Conquer
Framework
Since m[j] is decreasing, we fulfil the one requirement of
Proving
monotonicity of our CHT data structure. Hence we can directly apply
opt
Modifications convex hull trick as a black box.
Convex Hull Trick 38
Dynamic
Programming
2
DP int main () {
Optimizations // Base case :
dp [0] = b a s e C a s e C o s t ;
Convex Hull line l ;
Trick l . m = m [0];
Construction l . b = dp [0];
Application add ( l ) ;
Examples
for ( int i = 1; i < N ; i ++) {
Divide and dp [ i ] = query ( p [ i ]) ;
Conquer line l ;
Optimization l . m = m [ i ];
Divide and l . b = dp [ i ];
Conquer add ( l ) ;
Framework
Proving }
monotonicity of return 0;
opt }
Modifications
Convex Hull Trick 39
Dynamic
Programming
2
DP
Optimizations
Convex Hull
Trick Complexity? O(N log N).
Construction
Application
Examples
This is a significant improvement from O(N 2 )!
Divide and
Conquer
Note, we could even get O(N) if our p[i] are
Optimization non-decreasing.
Divide and
Conquer
Framework
Proving
monotonicity of
opt
Modifications
Key Takeaways 40
Dynamic
Programming
2 DPs with recurrences in the form
DP
Optimizations
j<i
Convex Hull
Trick can be done in O(N log N).
Construction
Application This is done by creating a data structure for CHT which
Examples
supports:
Divide and
Conquer void add(int m, int b): Add a line l = mx + b.
Optimization
Divide and Requirement: m is strictly less than the gradient of all
Conquer
Framework lines in the data structure already.
int query(q): Over all lines {li = mi x + bi } that we
Proving
monotonicity of
opt
Modifications have added so far, return mini (mi q + bi ).
You can treat this data structure as a black box. However
it is good to at least know how to tweak it using the 2
knobs mentioned above.
In Practice 41
Dynamic
Programming
2
In practice, the difficulty of CHT comes from:
DP
Recognizing it is useful. Often you just have to write out
Optimizations the recurrences. Be suspect whenever the recurrence is
Convex Hull given by a formula.
Trick
Construction
Figuring out how to calculate the gradients and y
Application
Examples
intercepts. You will often have to juggle around terms in
Divide and
the recurrence to make it work. This comes mostly with
Conquer practice.
Optimization
Divide and Essentially, the recurrence
Conquer
Framework
Proving
monotonicity of dp[i] = min(dp[j] + m[j] ∗ p[i])
opt j<i
Modifications
tells us that we need the gradient to be a function of only

j and the query point to be a function of only i.
Example: Covered Walkway 42
Dynamic
Programming
2
Problem Statement: I have N points on a walkway I
DP need to cover. To cover the walkway from point x to
Optimizations
point y inclusive costs C + (x − y )2 where C is a fixed,
Convex Hull
Trick given constant. Note that you can cover a single point
Construction
Application
with cost C . What is the minimum cost needed to cover
Examples
all the points?
Divide and
Conquer
Optimization
Input Format: First line 2 integers,
Divide and
Conquer
N, C , 1 ≤ N ≤ 106 , 1 ≤ C ≤ 109 . The next N lines
Framework
Proving contain the points in increasing order. All points are in the
monotonicity of
opt
Modifications
range [1, 109 ].
Source: 2012 University of Chicago Invitational
Programming Contest.
Dynamic
Programming
2
Hope you can see a O(N 2 ) DP.
DP
Calculate dp[N], where dp[i] is the min cost to cover
Optimizations exactly the points up to the ith.
Convex Hull
Trick But the recurrence here is a very nice formula. Let us
Construction
Application unpack it.
Examples
Divide and
Conquer dp[j] = min dp[i − 1] + C + (x[j] − x[i])2
Optimization i<j
= min dp[i − 1] + C + x[i]2 + x[j]2 − 2 ∗ x[i] ∗ x[j]

Divide and
Conquer
Framework
Proving
i<j
monotonicity of
opt
Modifications
Focus on the last term. What does this look like?
A linear function (if you squint hard enough)! So we
should try to fit this into our CHT framework.
Dynamic
Programming
2
Recurrence:
DP
Optimizations dp[j] = min dp[i − 1] + C + x[i]2 + x[j]2 − 2 ∗ x[i] ∗ x[j]
Convex Hull
i<j
Trick
Construction
Application
If you recall, we were earlier looking at recurrences of the
Examples
form mini<j dp[i] + m[i] ∗ p[j].
Divide and
Conquer
Optimization
What should m[i] and p[j] be?
Divide and
Conquer We should have m[i] = −2x[i], p[j] = x[j] (since we need
Framework
Proving
monotonicity of
the gradient to be determined by i and the query point by
opt
Modifications
j).
Note that our gradients are decreasing which we need for
our CHT!
Dynamic
Programming
2
Recurrence:
DP
Optimizations dp[j] = min dp[i − 1] + C + x[i]2 + x[j]2 − 2 ∗ x[i] ∗ x[j]
i<j
Convex Hull
Trick
Construction
Application
We also need to modify the y-intercept of the line, it isn’t
Examples
just dp[i] anymore. What should it be?
Divide and
Conquer Key: It needs to include all terms dependent on i. So it
Optimization
Divide and needs to include dp[i − 1] and x[i]2 . Why?
Conquer
Framework
Proving But it can’t include x[j]2 for obvious reasons. So we
monotonicity of
opt
Modifications
should add the x[j]2 part when we calculate dp[j].
You can choose whether to add C when you calculate
dp[j] or whether to add C to the y-intercept.
Dynamic
Programming
2
DP Recurrence:
Optimizations
dp[j] = min dp[i − 1] + C + x[i]2 + x[j]2 − 2 ∗ x[i] ∗ x[j]

Convex Hull
Trick
Construction i<j
Application
Examples
Divide and
So for i we will add a line where:
Conquer
Optimization
y-intercept is: dp[i − 1] + x[i]2 .
Divide and
Conquer
Gradient is: −2x[i].
Framework
Proving
monotonicity of
And to calculate dp[j], we query our CHT with point x[j].
Define r := query (x[j]). Then dp[j] = r + C + x[j]2 .
opt
Modifications
Dynamic
/* Insert CHT code here : You can use the version with only
Programming
* positi ve integer queries and non - d e c r e a s i n g queries */
2
void add ( line l ) ;
long long query ( long long x ) ;
DP const int MAXN = 1000000;

Optimizations int N ;
long long x [ MAXN +1];
Convex Hull long long C ;
Trick
Construction long long dp [ MAXN +1];
Application
Examples int main () {
scanf ( " % d % lld " , &N , & C ) ;
Divide and for ( int i = 0; i < N ; i ++) {
Conquer scanf ( " % lld " , & x [ i ]) ;
Optimization }
Divide and for ( int i = 0; i < N ; i ++) {
Conquer
Framework // Compare to f or mu la s written in slides
Proving dp [ i ] = query ( x [ i ]) + C + x [ i ]* x [ i ];
monotonicity of line l ;
opt
l . m = -2* x [ i ];
Modifications
// Base case is i == 0 , dp [i -1] = 0.
l . b = ( i == 0 ? 0 : dp [i -1]) + x [ i ]* x [ i ];
add ( l ) ;
}
// Again , c a l c u l a t i n g dp [N -1] using the same formula as above .
printf ( " % lld \ n " , dp [N -1]) ;
}
Dynamic
Programming
2
DP
Optimizations
Complexity? We note our query points, p[j] = x[j] are
Convex Hull
Trick increasing in j. So we can use the O(1) amortized CHT.
Construction
Application Then the complexity is just O(N).
Examples
Divide and
Since our query points are integers, we can also use the
Conquer
Optimization
version of CHT with no doubles. Using just integer
Divide and
Conquer
division doesn’t change the complexity but in practice is a
Framework
Proving large speed up (I get a 3 times speedup locally).
monotonicity of
opt
Modifications
Dynamic
Programming
2
DP
This is how many CHT problems go.
Optimizations First you come up with a normal DP that is too slow. This
Convex Hull
Trick
step is the same as in the DP lecture.
Construction Then you note the recurrence is a nice formula. Write it
Application
Examples out.
Divide and Now, split up the recurrence as in this example. Figure out
Conquer
Optimization
which part corresponds to the slope and query point and
Divide and
Conquer
how to split up the constant into the y intercept and the
Framework
Proving
part added when you calculate dp[j].
monotonicity of
opt Usually this is just breaking up the terms depending on if
Modifications
they depend on j or i.
Example: Mini Commando 50
Dynamic
Programming
2
Problem Statement: I have N soldiers in a line. The ith
DP soldier has effectiveness xi . I want to partition the soldiers
Optimizations
into squadrons. Each squadron consists of a subsegment
Convex Hull
Trick of contiguous soldiers. Say these soldiers have total
Construction
Application effectiveness S. Then the effectiveness of the squadron is
Examples
AS 2 + C where A < 0 and C are given constants.
Divide and
Conquer Maximize sum of the effectiveness of the squadrons.
Optimization
Divide and
Conquer
Input Format: First line 2 integers, N, A, C .
1 ≤ N ≤ 106 , −5 < A < −1, |C | ≤ 104 . Next line contains
Framework
Proving
monotonicity of
opt
Modifications
N integers, the effectiveness of the soldiers in order. Each
effectiveness satisfies 1 ≤ xi ≤ 100.
Source: APIO 2010.
Dynamic
Programming
2
DP
Optimizations Sample Input:
Convex Hull
Trick 4
Construction
Application -1 -20
Examples
2 2 3 4
Divide and
Conquer
Optimization Sample Output: -101
Divide and
Conquer
Framework
Explanation: Split the soldiers into groups of
Proving
monotonicity of {1, 2}, {3}, {4}, with effectiveness {−36, −29, −36}.
opt
Modifications
Dynamic
Programming
2
Hope you can see a O(N 2 ) DP.

DP
Optimizations
Calculate dp[N], where dp[i] is the min cost to partition
Convex Hull
Trick exactly the first i soldiers.
Construction
Application Where to go next?
Examples
Divide and Look at the recursion!

Conquer
Optimization
Divide and
Conquer
dp[j] = max dp[i − 1] + C + A(xi + xi+1 + . . . + xj )2
Framework i<j
Proving
monotonicity of
opt
Modifications
This is kind of messy but should remind you a bit of the
last example.
Dynamic
Programming
2
Useful trick: We want to clean up the sum in the square.
DP It is often easier to work with cumulative arrays than sums.
Optimizations
Convex Hull
Define S[N] to be the cumulative sum,
Trick S[i] := (x1 + . . . + xi ).
Construction
Application
Examples
Then let’s rewrite the recurrence.
Divide and
Conquer
Optimization
dp[j] = max dp[i − 1] + C + A(S[j] − S[i − 1])2
i<j
Divide and
Conquer
Framework
Proving
= max dp[i − 1] + C + A · S[i − 1]2 + A · S[j]2
monotonicity of i<j
opt
Modifications − 2A · S[i − 1] · S[j]
Much cleaner IMO.

Dynamic
Programming
2
DP
Optimizations
Convex Hull
Trick
Construction
Application
dp[j] = max dp[i − 1] + C + A · S[i − 1]2 + A · S[j]2 − 2A · S[i − 1] · S[j]
i<j
Examples
Divide and
Conquer What does this look like?
Optimization
Divide and
Conquer
The key is the last term. If you unpack the important
Framework
Proving parts, a line!
monotonicity of
opt
Modifications
Dynamic
Programming
2
DP dp[j] = max dp[i − 1] + C + A · S[i − 1]2 + A · S[j]2 − 2A · S[i − 1] · S[j]

Optimizations i<j
Convex Hull
Trick Again, main question is to convert this into
Construction
Application
Examples
b[i] + m[i] ∗ p[j] form. Let us do the latter part first.
Divide and What is m[i]?
Conquer
Optimization
Divide and
−2A · S[i − 1]. So p[j] = S[j]. Gradient is increasing (since
Conquer
Framework A is negative) which is correct for maximization problems.
Proving
monotonicity of
opt How about the constant? Which parts form b[i] and
Modifications
which parts should we add directly to dp[j]?
Again, we need b[i] to include dp[i − 1] + A · S[i − 1]2 .
This time around we will also include C in b[i].
Dynamic
Programming
2
DP
Optimizations
Convex Hull
Trick dp[j] = max dp[i − 1] + C + A · S[i − 1]2 + A · S[j]2 − 2A · S[i − 1] · S[j]
Construction i<j
Application
Examples
Divide and
So for i we will add a line where:
Conquer y-intercept is: dp[i − 1] + A · S[i − 1]2 + C .
Optimization
Divide and Gradient is: −2A · S[i − 1].
Conquer
Framework
Proving
monotonicity of
And to calculate dp[j], we query our CHT with point S[j].
opt
Modifications Define r := query (S[j]). Then dp[j] = r + A · S[j]2 .
Dynamic
Programming /* Insert CHT code here : You can use the version with only
2 * positi ve integer queries and non - d e c r e a s i n g queries */
void add ( line l ) ;
long long query ( long long x ) ;
DP const int MAXN = 1000000;

Optimizations int N ;
long long a, c;
Convex Hull
long long S [ MAXN +1];
Trick
long long dp [ MAXN +1];
Construction
Application
int main () {
Examples
scanf ( " % d " , & N ) ;
Divide and scanf ( " % lld % lld % lld " , &a , & c ) ;
Conquer for ( int i = 1; i <= N ; i ++) {
Optimization long long x ; scanf ( " % lld " , & x ) ;
Divide and S [ i ] = S [i -1] + x ;
Conquer }
Framework for ( int i = 1; i <= N ; i ++) {
Proving line l ;
monotonicity of
opt l . m = -2* a * S [i -1];
Modifications l . b = dp [i -1] + a * S [i -1]* S [i -1] + c ;
add ( l ) ;
dp [ i ] = query ( S [ i ]) + a * S [ i ]* S [ i ];
}
printf ( " % lld \ n " , dp [ N ]) ;
return 0;
}
Dynamic
Programming
2 This time around we want to maximize the value.
However, our code for CHT works for both min (with
DP decreasing gradients) and max (with increasing gradients)
Optimizations
without modification.
Convex Hull
Trick
Construction
Complexity? We note our query points, p[j] = S[j] are
Application
Examples
increasing in j. So we can use the O(1) amortized CHT.
Divide and Then the complexity is just O(N).
Conquer
Optimization One should be a bit cautious here of overflows. Here S[i]
Divide and
Conquer
Framework
can go up to 108 , but we take S[i]2 , and dp[i] can go up
Proving
monotonicity of
to around 1013 . This is okay with long longs.
opt
Modifications Our y intercepts go up to around 1016 . So we’re going to
lose precision if we use doubles for our intercepts. I think
this is okay given our query points only go up to 108 (it
gets AC) but honestly I’m not sure.
Table of Contents 59
Dynamic
Programming
2
1 DP Optimizations
DP
Optimizations

Trick
Application
Divide and Examples
Conquer
Optimization
Divide and
Conquer
Proving
monotonicity of
opt
Modifications
Divide and Conquer Optimisation 60
Dynamic
Programming
2
DP
Optimizations
Convex Hull
Trick
Construction Here the structure comes in the choices we make during
Application
Examples the DP.
Divide and
Conquer The most common setting is a 2D DP.
Optimization
Divide and
Conquer
Framework
Proving
monotonicity of
opt
Modifications
Example Problem: Noisy Kids 61
Dynamic
Programming
2
DP
Problem Statement: You have N kids in a row. Each
Optimizations has a noisiness si > 0. You want to partition the kids into
Convex Hull
Trick
K rooms. Each room must contain an contiguous segment
Construction
Application
of the kids. The noisiness of a room containing kids [i, j]
Examples is (si + si+1 + . . . sj )2 (because kids...).
Divide and
Conquer What is the minimum total noisiness that is attainable?
Optimization
Divide and Input Format: First line 2 integers, N, K .
Conquer
Framework
Proving
1 ≤ K ≤ min(100, N), 1 ≤ N ≤ 104 . Next line contains N
monotonicity of
opt integers, the noisiness of the kids in order. Each noisiness
Modifications
is in the range [1, 1000].
Dynamic
Programming
2
Hope you can see an O(N 2 K ) DP.
DP Our state will be dp[k][i], the minimum noisiness to
Optimizations
partition the first i kids into k groups.
Convex Hull
Trick
Construction
We will proceed in increasing k and increasing i order.
Application
Examples
Recurrence?
Divide and
Conquer
Optimization dp[k][i] = min dp[k − 1][j] + (sj+1 + . . . + si )2
Divide and j<i
Conquer
Framework
Proving
monotonicity of
Naively this recurrence is O(N 2 ), there are O(N) values of
opt
Modifications j and the sum takes O(N) to calculate. How do we speed
up this recurrence to O(N)?
Cumulative sum!
Dynamic # include < bits / stdc ++. h >

Programming using namespace std ;
2
const long long INF = 1 e17 ;
const int MAXN = 10000;
const int MAXK = 100;
DP
int N , K ;
Optimizations
// 1 - indexed . S [ i ] = sum ( s_1 + ... + s_i )
Convex Hull long long S [ MAXN +1];
Trick long long dp [ MAXK +1][ MAXN +1];
Construction
int main () {
Application
Examples
scanf ( " % d % d " , &N , & K ) ;
for ( int i = 1; i <= N ; i ++) {
Divide and long long s ; scanf ( " % lld " , & s ) ;
Conquer S [ i ] = S [i -1] + s ;
Optimization }
Divide and for ( int i = 1; i <= N ; i ++) dp [0][ i ] = INF ;
Conquer for ( int k = 1; k <= K ; k ++) {
Framework
for ( int j = 1; j <= N ; j ++) {
Proving
monotonicity of dp [ k ][ j ] = INF ;
opt for ( int t = 0; t < j ; t ++) {
Modifications dp [ k ][ j ] = min ( dp [ k ][ j ] ,
dp [k -1][ t ] + ( S [ j ] - S [ t ]) *( S [ j ] - S [ t ]) ) ;
}
}
}
printf ( " % lld \ n " , dp [ K ][ N ]) ;
return 0;
}
Dynamic
Programming Let us take a look at the recurrence again.
2
dp[k][i] = min dp[k − 1][j] + (sj+1 + . . . + si )2

DP j<i
Optimizations
Convex Hull Let opt[k][i] be the value of j at which the minimum is

Trick
Construction
attained in the above recurrence. (if there’s multiple, any
Application
Examples will suffice)
Divide and
Conquer
Key Claim:
Optimization opt[k][i] ≤ opt[k][i + 1]
Divide and
Conquer
Framework
Proving
Proof later. Intuitively, one expects increasing i to move
monotonicity of
opt our groups to the right. If opt[k][i + 1] < opt[k][i] then
Modifications
the kth group for i + 1 encompasses the kth group for i.
This seems wrong because the cost of squaring increases
quicker the larger the group gets.
More formally, the key is x 2 is convex.
Dynamic
Programming
2
This is the setting we work in. We have a 2D dp with

DP recurrence
Optimizations
dp[k][i] = min dp[k − 1][j] + Cost(j + 1, i)

Convex Hull
Trick
Construction j<i
Application
Examples
Divide and
We define the array opt[k][i] to be any value of j at which
Conquer
Optimization
the above attains a minimum.
Divide and
Conquer
Framework
And we know (through some voodoo or maths) that:
Proving
monotonicity of
opt
Modifications
opt[k][i] ≤ opt[k][i + 1]
We now see how to speed this up.

Dynamic
Programming
2
The structure of the following slides:

DP
Optimizations In the next few slides, I assume the setting just mentioned.
Convex Hull In particular, I assume that opt[k][i] ≤ opt[k][i + 1] for all
Trick
Construction k, i.
Application
Examples
Under this setting, I will explain the general procedure for
Divide and
Conquer optimising the DP from O(N 2 K ) to O(NK log N).
Optimization
Divide and
Conquer
I will demonstrate the procedure on our example problem.
I will then talk about when opt[k][i] ≤ opt[k][i + 1] holds.
Framework
Proving
monotonicity of
opt
Modifications
As a special case, we will show it holds for our example
problem.
Divide and Conquer Framework 67
Dynamic
Programming
2
Suppose opt[k][i] ≤ opt[k][i + 1].

DP
Optimizations
The earlier recurrence for dp[k][i] was O(N) because we
Convex Hull
Trick had to try every j < i.
But if we know prev := opt[k][i − 1] then we only need to
Construction
Application
try every j ∈ [prev , i).

Examples
Divide and
Conquer
Optimization This doesn’t lead to a speed up just yet. But for example,
Divide and
Conquer if we further know nxt := opt[k][i + 1] then we only have
to try every j ∈ [prev , nxt].
Framework
Proving
monotonicity of
opt
Modifications This is how we will proceed. We will constrain the range
we have to search on both sides.
Dynamic
Programming
2
DP First we calculate dp[k][N/2] and opt[k][N/2]. For

Optimizations
convenience, define ok,N/2 := opt[k][N/2].
Convex Hull
Trick
Construction
Now, for all i < N/2 we know ok,i < ok,N/2 and for all
Application
Examples
i > N/2 we know ok,i > ok,N/2 . So on the left we have to
Divide and search the range [1, ok,N/2 ] and on the right [ok,N/2 , N].
Conquer
Optimization Note: this partitions the range between the 2 halves.
Divide and
Conquer
Framework We now divide and conquer by repeating this procedure in
Proving
monotonicity of
opt
both halves, except only searching the above range in each
Modifications
of the halves.
Dynamic
Programming
2
Suppose we have k fixed and N = 7. Then we calculate
DP dp[k][n] in top to bottom order in the following tree.
Optimizations
Convex Hull
Each node will store its index i and ok,i in the form o:ok,i .
Trick
Construction
Each node will also the range in which it searched for its
Application
Examples
recurrence like [x, y].
Divide and
Conquer
Optimization 4
Divide and
Conquer
Framework
Proving
monotonicity of
2 6
opt
Modifications
1 3 5 7
Dynamic
Programming
2
DP
First, we have to search the range [1, 7] to find dp[k][4]
Optimizations and ok,4 . Suppose ok,4 = 3:
Convex Hull
Trick
Construction
Application
Examples 4 o:3 [1, 7]
Divide and
Conquer
Optimization 2 6
Divide and
Conquer
Framework
Proving
1 3 5 7
monotonicity of
opt
Modifications
Dynamic
Programming
2
Next we calculate dp[k][i] for the 2 children. We should

DP
Optimizations only search for j in the ranges that make sense. Suppose
Convex Hull ok,2 = 1 and ok,6 = 4.
Trick
Construction
Application
Examples
4 o:3 [1, 7]
Divide and
Conquer
Optimization
Divide and
Conquer
2 o:1 [1, 3] 6 o:4 [3, 7]
Framework
Proving
monotonicity of
opt 1 3 5 7
Modifications
Dynamic
Programming
2
Finally we calculate dp[k][i] for the last layer, again only

DP
Optimizations
searching in the ranges that make sense. I am going to
Convex Hull
omit o:x for this layer because it does not matter.
Trick
Construction
Application
Examples 4 o:3 [1, 7]
Divide and
Conquer
Optimization
Divide and
Conquer
Framework
2 o:1 [1, 3] 6 o:4 [3, 7]
Proving
monotonicity of
opt
Modifications
1 [1, 1] 3 [1, 3] 5 [3, 4] 7 [4, 7]
Dynamic
Programming
2
What can you say about the search ranges of each layer of
DP the tree?
Optimizations
Convex Hull
Trick 4 o:3 [1, 7]
Construction
Application
Examples
Divide and
Conquer 2 o:1 [1, 3] 6 o:4 [3, 7]
Optimization
Divide and
Conquer
Framework 1 [1, 1] 3 [1, 3] 5 [3, 4] 7 [4, 7]
Proving
monotonicity of
opt
Modifications
Besides overlaps at endpoints, they form a partition of
[1, 7].
Dynamic
Programming
2
Under this procedure, what is the cost to calculate
DP dp[k][2]?
Optimizations
Convex Hull
Trick 4 o:3 [1, 7]
Construction
Application
Examples
Divide and
Conquer 2 o:1 [1, 3] 6 o:4 [3, 7]
Optimization
Divide and
Conquer
Framework 1 [1, 1] 3 [1, 3] 5 [3, 4] 7 [4, 7]
Proving
monotonicity of
opt
Modifications
The size of the range we have to search in for index 2. In
this case it is |[1, 3]| = 3.
Dynamic
Programming
2
What is the total cost to calculate all the dp[k][i]?
DP
Optimizations
Convex Hull
4 o:3 [1, 7]
Trick
Construction
Application
Examples
2 o:1 [1, 3] 6 o:4 [3, 7]
Divide and
Conquer
Optimization
Divide and
1 [1, 1] 3 [1, 3] 5 [3, 4] 7 [4, 7]
Conquer
Framework
Proving
monotonicity of
opt
Modifications
Sum of the sizes of the ranges. The sum of the sizes for
each layer is O(N), there are O(log N) layers. Therefore
O(N log N) overall.
Dynamic
Programming
2
DP
Optimizations Hence by calculating our DP in this divide and conquer
Convex Hull order, we get a speed up of O(N 2 K ) to O(NK log N).
Trick
Construction
Application
Implementation wise, we DFS instead of doing it layer by
Examples layer.
Divide and
Conquer Our code is similar to range trees. During our DFS, we
Optimization
Divide and keep track of a range [cL, cR) and our DFS will calculate
Conquer
Framework
Proving
dp[k][i] for all i ∈ [cL, cR). We also keep the search range
monotonicity of
opt (the range in the tree diagrams).
Modifications
Dynamic
Programming // Insert your cost fun ct io n here :
2 long long Cost ( int i , int j ) ;
// Search range for our DP is in the range [ qL , qR ]

DP // In this branch , c a l c u l a t i n g dp [ k ][ cL ,.. cR )
Optimizations void dnc ( int qL , int qR , int cL , int cR , int k ) {
if ( cR <= cL ) return ;
Convex Hull int bestpos ( -1) ;
Trick int mid = ( cL + cR ) / 2;
Construction dp [ k ][ mid ] = INF ; // assume this is a m i n i m i s a t i o n problem
Application for ( int i = qL ; i <= min ( qR , mid -1) ; i ++) {
Examples // D e p e n d i n g on d e f i n i t i o n of Cost ,
// add Cost ( i +1 , mid ) or Cost (i , mid )
Divide and long long newcost = dp [k -1][ i ] + Cost ( i +1 , mid ) ;
Conquer if ( newcost < dp [ k ][ mid ]) {
Optimization dp [ k ][ mid ] = newcost ;
Divide and bestpos = i ;
Conquer }
Framework
Proving }
monotonicity of // Split our range in 2.
opt // In the left , the search range is [ qL , bestpos ]
Modifications // and we c a l c u l a t e dp [ k ][ cL , mid )
// In the right , the search range is [ bestpos , qR ]
// and we c a l c u l a t e dp [ k ][ mid +1 , cR )
dnc ( qL , bestpos , cL , mid , k ) ;
dnc ( bestpos , qR , mid +1 , cR , k ) ;
}
Dynamic
Programming
2
DP
Optimizations
Convex Hull
Trick
Let us return to our original example problem.
Construction
Application What is the cost function here? Letting S[N] be the
Examples
cumulative array, Cost(j + 1, i) = (S[i] − S[j])2 .
Divide and
Conquer
Optimization
We can substitute this directly into our divide and conquer
Divide and
Conquer
template.
Framework
Proving
monotonicity of
opt
Modifications

2
const long long INF = 1 e17 ;
const int MAXN = 10000;
const int MAXK = 100;
DP int N , K ;
Optimizations long long S [ MAXN +1]; // c u m u l a t i v e array
long long dp [ MAXK +1][ MAXN +1];
Convex Hull
Trick
// Cost of segment (i , j ]
Construction
long long Cost ( int i , int j ) {
Application
return ( S [ j ] - S [ i ]) *( S [ j ] - S [ i ]) ;
Examples
}
Divide and // Search range : [ qL , qR ] , c a l c u l a t i n g dp [ k ][ cL .. cR )
Conquer void dnc ( int qL , int qR , int cL , int cR , int k ) {
Optimization if ( cR <= cL ) return ;
Divide and int bestpos ;
Conquer int mid = ( cL + cR ) / 2;
Framework dp [ k ][ mid ] = INF ;
Proving for ( int i = qL ; i <= min ( qR , mid -1) ; i ++) {
monotonicity of
opt // Cost expects (] , so we use Cost (i , mid ) .
Modifications long long newcost = dp [k -1][ i ] + Cost (i , mid ) ;
if ( newcost < dp [ k ][ mid ]) {
dp [ k ][ mid ] = newcost ;
bestpos = i ;
}
}
}
Dynamic
Programming
2
DP
Optimizations int main () {
scanf ( " % d % d " , &N , & K ) ;
Convex Hull for ( int i = 1; i <= N ; i ++) {
Trick long long c ; scanf ( " % lld " , & c ) ;
Construction S [ i ] = S [i -1] + c ;
Application }
Examples // For K = 0 , dp [0][0] = 0 is the base case .
for ( int i = 1; i <= N ; i ++) dp [0][ i ] = INF ;
Divide and // Just call dnc for k from 1 to N
Conquer // Make sure you get the initial [ qL , qR ] and [ cL , cR ) correct .
Optimization for ( int k = 1; k <= K ; k ++) dnc (0 , N , 1 , N +1 , k ) ;
Divide and printf ( " % lld \ n " , dp [ K ][ N ]) ;
Conquer
Framework return 0;
Proving }
monotonicity of
opt
Modifications
Dynamic
Programming
2
DP
Optimizations
Convex Hull
Trick
Complexity? O(NK log N). Here N = 104 , K = 100 we
Construction
Application
get ≈ 14 · 106 which is fine for a second.
Examples
But we left out a detail, how do we know that opt[k][i] is
Divide and
Conquer non-decreasing for a fixed k?
Optimization
Divide and
Conquer
This is a special case of a more general phenomenon.
Framework
Proving
monotonicity of
opt
Modifications
Proof of Monotonicty of opt 82
Dynamic
Programming
2
DP
Optimizations
Convex Hull
Trick This part is quite technical. At the end I’ve included a key
Construction
Application
Examples
takeaways slide.
Divide and The more important part to get is the intuition. Feel free
Conquer
Optimization to skip verifying any of the maths equations.
Divide and
Conquer
Framework
Proving
monotonicity of
opt
Modifications
Dynamic
Programming For concreteness sakes, let Cost(i, j) := (S[j] − S[i])2 .
2
I claim Cost(i, j) satisfies the key feature, if
a < b < c < d:
DP
Optimizations
Cost(a, d) − Cost(b, d) ≥ Cost(a, c) − Cost(b, c)
Convex Hull
Trick
Construction
What does this say? Suppose we started with looking at
Application
Examples
the ranges [b, c] and [b, d] and we define A := Cost(b, c)
Divide and and B := Cost(b, d).
Conquer
Optimization Then it says that adding an extra item to the interval
Divide and
Conquer [b, d] increases the cost more than adding the same item
Framework
Proving
monotonicity of
to the interval [b, c].
opt
Modifications
But when our function is squaring, this is immediate. This
just says that (B + c)2 − B 2 ≥ (A + c)2 − A2 if
B > A ≥ 0.
This follows from x 2 being convex (its derivative is an
increasing function).
Dynamic
Programming
2
DP
Optimizations Now, let Cost can be any function that satisfies:
Convex Hull
Cost(a, d) − Cost(b, d) ≥ Cost(a, c) − Cost(b, c)

Trick
Construction
Application
Examples
Divide and for all a < b < c < d.

Conquer
Optimization To prove opt[k][i] is increasing in i, we proceed with the
Divide and
Conquer
Framework
standard swapping argument.
Proving
monotonicity of
opt
Suppose for a contradiction that opt[k][i + 1] < opt[k][i].
Modifications
Dynamic
Programming Then on the one hand by definition of ok,i+1 :
2
dp[k − 1][ok,i+1 ] + Cost(ok,i+1 , i + 1) <
DP dp[k − 1][ok,i ] + Cost(ok,i , i + 1)
Optimizations
Convex Hull or equivalently:

Trick
Construction
Application
Cost(ok,i+1 , i + 1) − Cost(ok,i , i + 1) <
dp[k − 1][ok,i ] − dp[k − 1][ok,i+1 ]
Examples
Divide and
Conquer
Optimization But the opposite situation holds for ok,i :
Divide and
Conquer
Framework dp[k − 1][ok,i ] + Cost(ok,i , i) <
Proving
monotonicity of
opt dp[k − 1][ok,i+1 ] + Cost(ok,i+1 , i)
Modifications
or equivalently:
dp[k − 1][ok,i ] − dp[k − 1][ok,i+1 ] <
Cost(ok,i+1 , i) − Cost(ok,i , i)
Dynamic
Programming
2
DP
Optimizations
Convex Hull
Composing these two inequalities, we get
Trick
Construction
Application
Cost(ok,i+1 , i + 1) − Cost(ok,i , i + 1) <
Examples
Divide and
Cost(ok,i+1 , i) − Cost(ok,i , i)
Conquer
Optimization
Divide and exactly contradicting our inequality on Cost as
Conquer
Framework
Proving
ok,i+1 < ok,i < i < i + 1.
monotonicity of
opt
Modifications
Quadrangle Inequality 87
Dynamic
Programming
2 This inequality
DP Cost(a, d) − Cost(b, d) ≥ Cost(a, c) − Cost(b, c)

Optimizations
Convex Hull
Trick
for all a < b < c < d is called the quadrangle inequality.
Construction
Application You should think of it as saying the larger your segment
Examples
already is, the more costly it is to put items into it.
Divide and
Conquer
Optimization
So intuitively, one might expect this to imply monotonicity
Divide and
Conquer of opt since moving the right endpoint from i to i + 1
Framework
Proving should disincentivize the last segment growing any larger
monotonicity of
opt
Modifications
to the left.
Which is what the above proof showed. Hence satisfying
the quadrangle inequality is sufficient for opt to be
monotonic, and hence to apply D&C Optimisation.
Examples of good Cost functions 88
Dynamic
Programming
Some examples:
2
Our example involved Cost(i, j) = F (si+1 + . . . + sj )
where F (x) := x 2 . If all our {si } are positive then Cost
DP
Optimizations satisfies the quadrangle inequality whenever F is convex.
Convex Hull So F (x) = x 3 , x 1.5 , x log x all work.
Trick
Construction Also a lot of CHT! (Cost(i, j) = bi + mi · pj where mi is
Application
Examples non-increasing and pj non-decreasing). Conversely, a
Divide and decent number of D&C problems can be solved with CHT
Conquer
Optimization after clever rewriting.
Divide and
Conquer
Framework By induction, it suffices that we can prove it for one step:
Proving
monotonicity of
opt
Modifications
Cost(b−1, c+1)−Cost(b, c+1) ≥ Cost(b−1, c)−Cost(b, c)
Often Cost(i, j) is defined as some function operating on
the multiset of {si , si+1 , . . . , sj }. For these, proving a
single step is often easy.
E.g: Cost(i, j) = |{(a, b) | sa + sb = 0 ∧ a, b ∈ [i, j]}|.
Key Takeaways 89
Dynamic
Programming Key Takeaways:
2
To show opt is monotone, we almost always instead show
DP Cost satisfies the quadrangle inequality.
Optimizations
You don’t need to know why Cost satisfying the
Convex Hull
Trick quadrangle inequality implies monotonicity of opt. But it is
Construction
Application worth knowing roughly how to check the inequality holds.
Examples
Divide and Intuitively, it just says the larger a range already is, the
Conquer
Optimization more Cost increases more upon adding another item.
Divide and
Conquer
Framework
Many Cost functions satisfy the quadrangle inequality, in
Proving
monotonicity of particular convex functions applied to the sum of the
opt
Modifications range.
If Cost satisfies the quadrangle inequality or we suspect it
does, then we can apply the earlier Divide and Conquer
framework directly just by modifying the Cost function.
Modifications 90
Dynamic
Programming
2
For maximisation problems, we want the opposite

DP
Optimizations
quadrangle inequality
Convex Hull
Trick Cost(a, d) − Cost(b, d) ≤ Cost(a, c) − Cost(b, c)
Construction
Application
Examples that is, it is harder to increase the cost of larger segments.
Divide and
Conquer
Some examples:
Optimization Concave functions (e.g: F (x) := x 0.5 , F (x) = log x).
Divide and
Conquer
Framework
Again, it suffices to prove it for a single step.
Proving
monotonicity of
opt Often the annoying part is calculating Cost(i, j) quickly.
Modifications
E.g: when Cost(i, j) = |{(i, j) | si + sj = 0}|. For D&C it
is too slow to calculate it in O(j − i).
Aside: Knuths Optimisation 91
Dynamic
Programming
2
DP
Optimizations
The quadrangle inequality further gives that
Convex Hull
Trick
Construction opt[k − 1][i] ≤ opt[k][i] ≤ opt[k][i + 1]
Application
Examples
Divide and Using this and doing the DP in increasing k order, then
Conquer
Optimization decreasing i order, gives a running time of O((N + K )N).
Divide and
Conquer
Framework
This is sometimes called ”Knuth’s Optimisation”. (though
Proving
monotonicity of
I feel this term is overloaded)
opt
Modifications
Example: Arranging Heaps 92
Dynamic
Programming
2
DP
Optimizations
Problem Statement: There are N mines on a river, the
Convex Hull
ith is at position Xi and produces Wi units of coal.
Trick To collect coal, K facilities will be built on the river. The
Construction
Application
Examples
river only flows downstream so coal from position i can be
Divide and
delivered to position j iff j > i. To deliver w units of coal
Conquer
Optimization
from position X to Y costs w · (Y − X ) units of fuel.
Divide and
Conquer
If we optimally position the facilities, what is the minimum
Framework
Proving amount of fuel necessary to collect all the coal?
monotonicity of
opt
Modifications Source: 2012 Latin America Regionals.
Dynamic
Programming
2
Input Format: First line, 2 integers N, K .

DP
Optimizations
1 ≤ K < N ≤ 2000. Next N lines describe the mines.
Convex Hull
Each line contains 2 integers Xi , Wi with
Trick
Construction
1 ≤ Xi , Wi ≤ 106 . The mines are in strictly increasing
Application
Examples
order of Xi .
Divide and Sample Input:
Conquer
Optimization 3 2
Divide and
Conquer
Framework 11 3
Proving
monotonicity of
opt
12 2
Modifications 14 1
Sample Output: 3
Dynamic
Programming
2
As always, start simple. What order to process in and
what state?
DP
Optimizations
The standard ones for 2D DP. State is dp[k][i], minimum
Convex Hull
Trick cost to collect coal exactly from the first i mines using k
Construction
Application
facilities.
Examples
Divide and
Order is increasing k then increasing i. Increasing i then k
Conquer
Optimization
works too.
Divide and
Conquer Recurrence? Has to be something about building facilities.
Framework
Proving
monotonicity of Easy Observation: We should only build facilities at
opt
Modifications mines. By definition, in the state dp[k][i], we must build a
facility at mine i. So what is the choice?
Choice is where the previous facility was built.
Dynamic
Programming Recurrence:
2
dp[k][i] = min dp[k − 1][j] + Cost(j, i)
j<i
DP
Optimizations What is Cost(j, i) here? (in words)
Convex Hull
Trick
It is the total cost to deliver coal from mines j + 1, . . . , i
Construction to a facility at mine i.
Application
Examples Formula?
Divide and
Conquer
Optimization ∑
i
Divide and
Conquer Wt · (Xi − Xt )
Framework
Proving t=j+1
monotonicity of
Exercise: Make this run in O(N 2 K ) (Hint: Either rewrite

opt
Modifications
the equation or do a backwards sweep to calculate the

above min).
O(N 2 K ) ≈ 8bil. Too slow (at least without extreme micro
optimization).
Dynamic
Programming How to speed this up? Note this is a standard 2D DP
2
where we are making a choice per state, but making this
choice is introducing an O(N) cost. And we are blowing
DP
Optimizations out due to this O(N) cost not the state space.
Convex Hull
Trick
So we should look at whether this choice has a nice
Construction
Application
monotone structure. I.e: is opt[k][i] ≤ opt[k][i + 1].
Examples
To check this, we should check the quadrangle inequality,
Divide and
Conquer or the one step version:
Optimization
Cost(b−1, c+1)−Cost(b, c+1) ≥ Cost(b−1, c)−Cost(b, c)

Divide and
Conquer
Framework
Proving
monotonicity of
opt Important: What does your intuition tell you?
Modifications
My intuition tells me this seems likely. Adding another
mine to a larger segment should incur more cost relative
to adding it to a smaller segment. In the former, the coal
will have to be shipped further.
Dynamic
Programming
2
We can also give a quick proof this holds:
DP
Optimizations Cost(b−1, c+1)−Cost(b, c+1) ≥ Cost(b−1, c)−Cost(b, c)
Convex Hull
Trick
Construction
Recall
Application ∑
i
Examples
Cost(j, i) := Wt · (Xi − Xt )
Divide and
Conquer t=j+1
Optimization
Divide and
Conquer Since most terms cancel, LHS is just
Wb · (X [c + 1] − X [b]) while RHS is just
Framework
Proving
monotonicity of
opt
Modifications
Wb · (X [c] − X [b]).
Therefore quadrangle inequality is satisfied and we can
apply D&C optimization.
Dynamic
Programming The final ingredient is we need to calculate Cost(j, i) in
2
O(1) where
DP
Optimizations ∑
i
Convex Hull
Cost(j, i) := Wt · (Xi − Xt )
Trick t=j+1
Construction
Application
Examples The difficulty is that Xi varies so we can’t precompute this.
Divide and
Conquer Standard trick: rewrite this to isolate out the Xi part:
Optimization
   
∑ ∑
Divide and
Conquer i i
Cost(j, i) :=  Wt  Xi −  Wt Xt 
Framework
Proving
monotonicity of
opt
Modifications
t=j+1 t=j+1
How do we calculate this in O(1) now?

Cumulative sums over the array W [N] and the array
{Wt Xt }N
t=1 .

2 const long long INF = 1 e18 ;
const int MAXN = 1000 , MAXK = 1000;
int N , K ;
long long X [ MAXN +1] , W [ MAXN +1] , sXW [ MAXN +1] , sW [ MAXN +1];
DP
long long dp [ MAXK +1][ MAXN +1];
Optimizations
Convex Hull // Cost of moving mines (i , j ] to mine j .

Trick long long Cost ( int i , int j ) {
return ( sW [ j ] - sW [ i ]) * X [ j ] - ( sXW [ j ] - sXW [ i ]) ;
Construction
}
Application
Examples
// Best choice for our DP is in the range [ qL , qR ]
// C a l c u l a t i n g dp [ k ][ cL ,.. cR )
Divide and void dnc ( int qL , int qR , int cL , int cR , int k ) {
Conquer if ( cR <= cL ) return ;
Optimization int bestpos ( -1) ;
Divide and int mid = ( cL + cR ) / 2;
Conquer dp [ k ][ mid ] = INF ;
Framework
for ( int i = qL ; i <= min ( qR , mid -1) ; i ++) {
Proving
monotonicity of // We use Cost (i , mid ) since Cost (i , j ) is the cost of (i , j ]
opt long long newcost = dp [k -1][ i ] + Cost (i , mid ) ;
Modifications if ( newcost < dp [ k ][ mid ]) {
dp [ k ][ mid ] = newcost ;
bestpos = i ;
}
}
}
Dynamic
Programming
2
DP
Optimizations
int main () {
Convex Hull scanf ( " % d % d " , &N , & K ) ;
Trick for ( int i = 1; i <= N ; i ++) {
Construction scanf ( " % lld % lld " , & X [ i ] , & W [ i ]) ;
Application sXW [ i ] = sXW [i -1] + X [ i ] * W [ i ];
Examples sW [ i ] = sW [i -1] + W [ i ];
}
Divide and for ( int i = 1; i <= N ; i ++) dp [0][ i ] = INF ;
Conquer for ( int k = 1; k <= K ; k ++) dnc (0 , N , 1 , N +1 , k ) ;
Optimization printf ( " % lld \ n " , dp [ K ][ N ]) ;
Divide and return 0;
Conquer }
Framework
Proving
monotonicity of
opt
Modifications
Dynamic
Programming
2
Complexity? O(NK log N) ≈ 44mil. Likely fine for a
second on a relatively modern machine.
DP
Optimizations Common in many problems of this kind.
Convex Hull Start with the obvious DP but the recurrence is too slow.
Trick
Construction Look at the structure there. Since you are making choices,
Application
Examples check if these choices might be monotone. You want your
Divide and intuition to play a strong role here.
Conquer
Optimization
To apply D&C, want Cost to be around O(1) amortized.
Divide and
Conquer
For simple examples, this usually just involves rewriting the
Framework
Proving
equation until you can apply basic data structures. For
monotonicity of
opt more complicated examples, you may need to be more
Modifications
careful with how you persist state in your D&C.
Aside: with some rewriting, not hard to do this with CHT
either.

DP Optimizations

Uploaded by

Copyright:

Available Formats

You might also like

DP Optimizations

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

DP Optimizations

Uploaded by

Copyright:

Available Formats

Dynamic

Convex Hull Dynamic Programming 2

Convex Hull 2 Convex Hull Trick

Sometimes when you’re doing DP your solution will be too

Convex Hull 2 Convex Hull Trick

General setting is:

Afterwards we will apply this data struture as a black box

query (q) = min(mi · q + bi )

For this, it suﬃces to store the lines that make up the

We keep the lines in a vector.

What is the new convex hull?

So some of the lines may become useless and we need to

You will hopefully observe some variant of the following:

intersect(cht[N − 1], l) ≤ intersect(cht[N − 2], cht[N − 1])

DP One common case, each query we make is at least the

(⌊intersect(cht[i −1], cht[i])⌋, ⌊intersect(cht[i], cht[i +1])⌋]

Dynamic struct line { long long m , b ; };

Complexity: O(log n) in general, we can make it O(1) if

dp[i] = min(dp[j] + m[j] ∗ p[i])

tells us that we need the gradient to be a function of only

= min dp[i − 1] + C + x[i]2 + x[j]2 − 2 ∗ x[i] ∗ x[j]

dp[j] = min dp[i − 1] + C + x[i]2 + x[j]2 − 2 ∗ x[i] ∗ x[j]

DP const int MAXN = 1000000;

Hope you can see a O(N 2 ) DP.

Divide and Look at the recursion!

Much cleaner IMO.

DP dp[j] = max dp[i − 1] + C + A · S[i − 1]2 + A · S[j]2 − 2A · S[i − 1] · S[j]

DP const int MAXN = 1000000;

Convex Hull 2 Convex Hull Trick

Dynamic # include < bits / stdc ++. h >

dp[k][i] = min dp[k − 1][j] + (sj+1 + . . . + si )2

Convex Hull Let opt[k][i] be the value of j at which the minimum is

This is the setting we work in. We have a 2D dp with

dp[k][i] = min dp[k − 1][j] + Cost(j + 1, i)

We now see how to speed this up.

The structure of the following slides:

Suppose opt[k][i] ≤ opt[k][i + 1].

try every j ∈ [prev , i).

DP First we calculate dp[k][N/2] and opt[k][N/2]. For

Next we calculate dp[k][i] for the 2 children. We should

Finally we calculate dp[k][i] for the last layer, again only

// Search range for our DP is in the range [ qL , qR ]

Dynamic # include < bits / stdc ++. h >

Cost(a, d) − Cost(b, d) ≥ Cost(a, c) − Cost(b, c)

Divide and for all a < b < c < d.

Convex Hull or equivalently:

DP Cost(a, d) − Cost(b, d) ≥ Cost(a, c) − Cost(b, c)

For maximisation problems, we want the opposite

Input Format: First line, 2 integers N, K .

Exercise: Make this run in O(N 2 K ) (Hint: Either rewrite

the equation or do a backwards sweep to calculate the

Cost(b−1, c+1)−Cost(b, c+1) ≥ Cost(b−1, c)−Cost(b, c)

How do we calculate this in O(1) now?

Dynamic # include < bits / stdc ++. h >

Convex Hull // Cost of moving mines (i , j ] to mine j .

You might also like