Workshop 7

Stochastic Optimization Using

Markov Chain Monte Carlo
Disclaimer: The United States Army Corps of Engineers has granted access to the information in this
model for instructional purposes only. Do not copy, forward, or release the information
without United States Army Corps of Engineers approval.

You will be working with a simple basin model for the Mahoning Creek watershed above Punxsutawney,
PA. Your objective is to use stochastic optimization via Markov Chain Monte Carlo to explore plausible
parameter sets for the Mahoning Creek sub-basin, with the goal of improving estimating parameters
affecting peak discharge from an event simulation.

Pre-Workshop Thought Questions (no more than 15 minutes total)

Take a quick look over the basin and meteorologic models included with the provided HMS project.
Consider the following questions which will help you perform your optimization trials:

1. Given the model formulation (selection of hydrologic modeling methods), which parameters
would you consider first for optimization?

2. What means might you use to estimate reasonable starting points and search ranges for these
parameters?

1
W – Stochastic Optimization/Karlovits
 3. In your own words, what does it mean to say that an MCMC simulation has converged to the
target distribution?

4. How does the search range for a parameter affect the speed with which an MCMC simulation
might converge?

Tasks:

1. Determine appropriate adjustments to the basin parameters for the Recession Baseflow
parameters for the April 1994 event so that the simulation initial conditions are properly
configured
2. Run the “Apr 94” simulation run to take a look at the current state of the basin model. You
should see that the current basin model is not calibrated. Hint: a simplex search on the
parameters defining initial conditions (Initial Discharge, Recession Constant) could provide a
good starting point, but be careful of the objective function date window. Alternatively, consider
running a simplex search for all seven parameters (two infiltration, two transform and three
baseflow) so you can estimate starting values for all three baseflow parameters which take into

2
W – Stochastic Optimization/Karlovits
 account possible offsets by the other methods. You will need starting values for the loss and
transform parameters anyway.
3. After estimating new baseflow parameter(s), create a copy of the Punxsutawney basin model
called “April 1994” and edit the Mahoning Cr sub-basin baseflow parameters to reflect the
update.
4. Update the “Specified Gage Wt” meteorologic model to use the new basin model.
5. Configure a new optimization trial called “MCMC_April1994” using the new April 1994 basin
model and the “Specified Gage Wt” meteorologic model for the time window 11Apr1994 01:00
– 17Apr1994 00:00. This is done by selecting the Optimization Trial Manager found under the
Compute tab, as shown in Figure 1.

Figure 1. A reminder of where to create new optimization trials.

6. Choose the Markov Chain Monte Carlo search method. Use a Pool Size of 6 (this will create
six separate Markov chains), an initial sample size of 100 (this will determine the initial
conditions for each chain), 2,000 minimum iterations, 50,000 maximum iterations, a
Convergence (Gelman-Rubin) statistic of 1.2 (which sets how much variation between chains

3
W – Stochastic Optimization/Karlovits
 indicates convergence) and a small equilibrium sample of 50 (which controls the sample size
drawn after convergence.)1
7. Optimize the Storage Coefficient, Time of Concentration, Constant Rate, and Initial Loss.
8. Add these parameters by right-clicking on the name of the Optimization Trial on the Compute
tab and selecting “Add Parameter”, as shown in Figure 2.

Figure 1. Right click on the Optimization Trial name on the Compute tab to add a parameter.

9. Choose a sensible starting value for the search for each of the four parameters, considering what
strategies you might employ to achieve this. The model is not calibrated so the provided starting
values (see the results for the “Apr 94” Simulation Run) may not be entirely reasonable. More
importantly, choose a reasonable minimum and maximum value for each parameter, keeping
in mind the trade-off between the space that the algorithm needs to search and the time it takes to
converge. Hint: a well-constrained Simplex search with the right objective function can be fairly
speedy and very informative. If you box out a parameter’s mode with the bounds, you’ll find that
the algorithm winds up with a result right on the edge of your boundary. On the other hand, if you
set the bounds too wide, the chains will travel very far apart and convergence can be very slow.
10. There is only one objective function compatible with the MCMC search method (Sum of Squared
Residuals), but set the objective function evaluation window to the same as the simulation
window (11Apr1994 01:00 – 17Apr1994 00:00).

1
Why these values? Mostly for speed. A recommended range for pool size is n to 2n where n is the number of
parameters. More chains usually help prevent false convergence. An initial sample of 100 is fast but should do an
passable job of characterizing the parameter space for finding initial conditions. 2,000 minimum iterations is
actually fairly low – HMS won’t start assessing convergence until it has at least that many, but we will want to see
the evolution of the convergence statistic. It is also the minimum allowable value in HMS. This model formulation
should be converged by 50,000 iterations – if not then the settings for the MCMC search need to be changed. A
Gelman-Rubin convergence statistic of 1.0 is ideal and a model can only approach that value in the limit; generally
1.2 is a relaxed but OK level to use for general practice.

4
W – Stochastic Optimization/Karlovits
 11. Run the MCMC simulation – this will take a little while because HMS is performing literally
thousands of simulations. So what is going on while HMS is performing all of these
simulations? Read on while HMS is working away on your model.

Stochastic optimization is attempting to “walk” across a very wide range of

combinations of parameters towards the most likely combinations of parameters.
Every step of the walk is a position in n-dimensional space (where n is the number of
parameters you chose to optimize.) With each step (iteration) of the algorithm, each
chain proposes to take a take a step away from its current position in order to find a
parameter set which improves the calibration. In likelihood terms, this means the
parameter set is more likely to have generated the observed data. These steps
between positions can be quite small in order to keep the algorithm from jumping too
far and inadvertently skipping over the mode (region of highest likelihood) of the joint
probability distribution of the parameters. Sometimes the algorithm determines that
the current position of the chain is likely to be better than the proposed movement,
and does not move at all, which is called “rejecting the proposal.” The algorithm will
move away from a “bad” position with very high probability. When a position is “very
good”, the algorithm is very unlikely to move away from it, resulting in multiple
simulations in a row using the same parameters! This is a necessary condition of the
MCMC algorithm, whose intention is to visit combinations of parameters with a
frequency that is proportional to how likely they are to have generated the observed
data (the so-called posterior distribution.)

The initial conditions are spread out across the parameter space to ensure that when
the chains meet at a region of the space that each chain considers likely, that each
has explored a large enough amount of the space to get a good picture of it. False
convergence occurs when one or more chains get stuck in either different regions of
the parameters space than each other, or a single region which is not the true mode
of the joint distribution of the parameters. We improve our estimate of whether or
not the algorithm has converged by using more chains and a lower convergence
threshold for the Gelman-Rubin statistic, but both of these choices increases the
number of simulations necessary to get to the state of equilibrium.

If you set your parameter search ranges well and provided good initial estimates, your simulation
should converge in 20-30 minutes or less. Let’s dive into the results and see what the search output
is showing us.

5
W – Stochastic Optimization/Karlovits
12. First, switch to the “Results” tab and expand the “MCMC_April1994” Optimization Trial.
13. Click on the Objective Function to open the plot. How do you interpret this plot? Hint: the
value for “Average Likelihood” isn’t as important as the general trend, which is measured as an
average across the six chains you ran.

14. Next, open the Convergence Statistic plot. At what point did HMS consider the MCMC
simulation “converged”?

15. Next, open the Parameter 1 (Clark UH Storage Coefficient) trace plot. What is the trend in the
central tendency and variation of each of the six chains as the simulation evolves?

16. Open the Parameter 2 (Clark UH Time of Concentration) trace plot. Do you notice the same
trend in the state of the chains for this parameter? What does this trend indicate?

17. After looking at the trace plots for all four of the parameters, which do you think converged the
fastest? What characteristics of the trace plot lead you to think that?

6
W – Stochastic Optimization/Karlovits
 18. Then, switch back to the Convergence Statistic plot. Check your conclusion about which
parameters you thought converged fastest.

Key Question: Although you optimized the four hydrologic process parameters, what other aspects of a
hydrologic model (that we didn’t touch at all in this workshop) may prevent a “perfect” optimization or
calibration?

7
W – Stochastic Optimization/Karlovits