Grid Search in Matlab

You might also like

Download as pdf or txt
Download as pdf or txt
You are on page 1of 15

Grid Search in MATLAB

Andrii Parkhomenko

Universitat Autònoma de Barcelona and Barcelona GSE

Spring 2017

Andrii Parkhomenko (UAB & Barcelona GSE) Grid Search in MATLAB 0 / 14


Grid Search

Problem:
max f (x)
x∈Rn

Approach. Build a grid: a finite set of points in the domain of f :

G = {(x11 , ..., xn1 ), ..., (x1m1 , ..., xnmn )}

where mi is the number of gridpoints in the dimension i

Andrii Parkhomenko (UAB & Barcelona GSE) Grid Search in MATLAB 1 / 14


Grid Search

Example 1
Problem:
max f (x) = x1 − 0.2x12 + x2 − 0.3x22
x∈R2
Construct an equally-spaced grid:
G = {(x1 , x2 )|x1 ∈ {0, 1, ..., 5}, x2 ∈ {0, 1, ..., 5}}

Andrii Parkhomenko (UAB & Barcelona GSE) Grid Search in MATLAB 2 / 14


Grid Search

In MATLAB:

Andrii Parkhomenko (UAB & Barcelona GSE) Grid Search in MATLAB 3 / 14


Grid Search

Grid is too coarse:


Algorithm finds x = (3, 2)
True maximum: x = (2.5, 1.667)

Solutions:
Finer grid
Non-equally spaced grid

Andrii Parkhomenko (UAB & Barcelona GSE) Grid Search in MATLAB 4 / 14


Grid Search

Construct non-equally spaced grid:

G = {(h(x1 ), h(x2 ))|x1 ∈ {0, 1, ..., 5}, x2 ∈ {0, 1, ..., 5}}

where h(xi ) = ln(3xi + 1)

Andrii Parkhomenko (UAB & Barcelona GSE) Grid Search in MATLAB 5 / 14


Grid Search

In MATLAB:

Andrii Parkhomenko (UAB & Barcelona GSE) Grid Search in MATLAB 6 / 14


Grid Search

This is a better grid for our purposes:


Solution algorithm finds x = (2.565, 1.996)
True maximum: x = (2.5, 1.667)

Non-equally spaced grids are also useful when you don’t know the
neighborhood in which to look for maximizer x

Andrii Parkhomenko (UAB & Barcelona GSE) Grid Search in MATLAB 7 / 14


Grid Search

“Curse of Dimensionality”

Typically we search on x ∈ Rn
When n is even moderately large, grid search requires too many
iterations:
I if n = 5, mi = 100 for i = 1, ..., 5, then we have to perform
1005 = 10, 000, 000, 000 iterations!

Solutions

Adaptive (refined) grid search


Don’t use grid search (see notes from previous class)

Andrii Parkhomenko (UAB & Barcelona GSE) Grid Search in MATLAB 8 / 14


Adaptive Grid Search

Idea:
Start with a coarse grid
Refine as you approach to the solution

Andrii Parkhomenko (UAB & Barcelona GSE) Grid Search in MATLAB 9 / 14


Adaptive Grid Search

Example 2
Problem:
max f (x) = x1 − 0.2x12 + x2 − 0.3x22
x∈R2+

1 Start with:
G 0 = {(x1 , x2 )|x1 ∈ {0, 1, ..., 5}, x2 ∈ {0, 1, ..., 5}}
2 Obtain interim solution: x = (3, 2)
3 Refine the grid around the interim solution by halving the length of
the search interval:

G 1 = (x1 , x2 )|x1 ∈ {1.75, 2.25, 2.75, 3.25, 3.75, 4.25},




x2 ∈ {0.75, 1.25, 1.75, 2.25, 2.75, 3.25}
4 Repeat steps 2-3 until convergence
Andrii Parkhomenko (UAB & Barcelona GSE) Grid Search in MATLAB 10 / 14
Adaptive Grid Search

In MATLAB:

Andrii Parkhomenko (UAB & Barcelona GSE) Grid Search in MATLAB 11 / 14


Adaptive Grid Search

After 4 refinements:
Solution algorithm finds x = (2.531, 1.656)
True maximum: x = (2.5, 1.667)

This level of precision requires about 37,000 iterations with simple grid
search, but only 144 with adaptive grid search!

Andrii Parkhomenko (UAB & Barcelona GSE) Grid Search in MATLAB 12 / 14


Adaptive Grid Search

Adaptive grid search faces the same problems as most other numerical
optimization methods when
objective function is not concave, discontinuous or non-differentiable
local optima exist
(see notes from previous class)

Andrii Parkhomenko (UAB & Barcelona GSE) Grid Search in MATLAB 13 / 14


Adaptive Grid Search

Further reading

Carroll, Christopher D. “The Method of Endogenous Gridpoints for


Solving Dynamic Stochastic Optimization Problems” (2006) Economics
Letters 91(3), 312-320

Fella, Giulio. “A Generalized Endogenous Grid Method for Non-Smooth


and Non-Concave Problems” (2014) Review of Economic Dynamics 17,
329-344

Andrii Parkhomenko (UAB & Barcelona GSE) Grid Search in MATLAB 14 / 14

You might also like