Professional Documents
Culture Documents
NELDER - MEAD - The Nelder-Mead Optimization Algorithm
NELDER - MEAD - The Nelder-Mead Optimization Algorithm
Usage:
x_opt = nelder_mead ( simplex , f , flag ) where simplex is a matrix which contains a list of distinct points that serve as initial guesses for the solution. If the dimension of the space is M , then the matrix must contain exactly M+1 points. For instance, for a 2D space, you supply 3 points. Each row of the matrix contains one point; for a 2D space, this means that simplex would be a 3x2 matrix. f is the "function handle"; that is, either a quoted expression for the function, or the name of an M-file that defines the function, preceded by an "@" sign; flag is an optional argument; if present, and set to 1, it will cause the program to display a graphical image of the contours and solution procedure. Note that this option only makes sense for problems in 2D, that is, with N=2. x_opt is the program's estimate for the minimizer of the function; Very simple functions can be input as a quoted string. Thus, one could specify the f argument as '(x(1)-2*x(2)+7)^2'; However, for more complicated functions it makes sense to prepare an M-file that defines the function. For this same example, a suitable M-file would be:
f u n c t i o nf=e x a m p l e(x) f=(x ( 1 )-2*x ( 2 )+7) ^ 2 ;
If this information was stored in an M-file called example.m, then one might invoke the optimization program with a command like
x _ o p t=n e l d e r _ m e a d(x _ i n i t ,@ e x a m p l e ,0)
people.sc.fsu.edu/~jburkardt/m_src/nelder_mead/nelder_mead.html 1/5
11/12/13
MATLAB's built in command fminsearch minimizes a scalar function of several variables using the NelderMead algorithm.
Licensing:
The computer code and data files described and made available on this web page are distributed under the GNU LGPL license.
Languages:
NELDER_MEAD is available in a MATLAB version.
Author:
Jeff Borggaard, Mathematics Department, Virginia Tech.
Reference:
1. Evelyn Beale, On an Iterative Method for Finding a Local Minimum of a Function of More than One Variable, Technical Report 25, Statistical Techniques Research Group, Princeton University, 1958. 2. Richard Brent, Algorithms for Minimization without Derivatives, Dover, 2002, ISBN: 0-486-41998-3, LC: QA402.5.B74.
people.sc.fsu.edu/~jburkardt/m_src/nelder_mead/nelder_mead.html 2/5
11/12/13
3. David Himmelblau, Applied Nonlinear Programming, McGraw Hill, 1972, ISBN13: 978-0070289215, LC: T57.8.H55. 4. Jeffrey Lagarias, James Reeds, Margaret Wright, Paul Wright, Convergence properties of the Nelder-Mead simplex method in low dimensions, SIAM Journal on Optimization, Volume 9, Number 1, 1998, pages 112-147. 5. Ken McKinnon, Convergence of the Nelder-Mead simplex method to a nonstationary point, SIAM Journal on Optimization, Volume 9, Number 1, 1998, pages 148-158. 6. Zbigniew Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs, Third Edition, Springer, 1996, ISBN: 3-540-60676-9, LC: QA76.618.M53. 7. John Nelder, Roger Mead, A simplex method for function minimization, Computer Journal, Volume 7, Number 4, January 1965, pages 308-313. 8. Michael Powell, An Iterative Method for Finding Stationary Values of a Function of Several Variables, Computer Journal, Volume 5, 1962, pages 147-151. 9. William Press, Brian Flannery, Saul Teukolsky, William Vetterling, Numerical Recipes in FORTRAN: The Art of Scientific Computing, Second Edition, Cambridge University Press, 1992, ISBN: 0-521-43064-X, LC: QA297.N866. 10. Howard Rosenbrock, An Automatic Method for Finding the Greatest or Least Value of a Function, Computer Journal, Volume 3, 1960, pages 175-184.
Source Code:
nelder_mead.m, the Nelder-Mead optimization algorithm.
11/12/13
nelder_mead_test03.m, uses the extended Rosenbrock function. nelder_mead_test04.m, uses the Goldstein-Price function. nelder_mead_test05.m, uses the Beale function. nelder_mead_test06.m, uses the Powell function. nelder_mead_test07.m, uses the Local function. nelder_mead_test08.m, uses the McKinnon function. BEALE is the Beale function, for which N = 2. beale.m, defines the Beale function. BOHACH1 is the Bohachevsky function #1, for which N=2. bohach1.m, defines the Bohachevsky function #1. BOHACH2 is the Bohachevsky function #2, for which N=2. bohach2.m, defines the Bohachevsky function #2. EXTENDED_ROSENBROCK is the "extended" Rosenbrock function. This version of the Rosenbrock function allows the spatial dimension to be arbitrary, except that it must be even. extended_rosenbrock.m, defines the extended Rosenbrock function. GOLDSTEIN_PRICE is the Goldstein-Price polynomial, for which N=2. goldstein_price.m, defines the Goldstein-Price polynomial. HIMMELBLAU is the Himmelblau function:
f ( x )=( x ( 1 ) ^ 2+x ( 2 )-1 1 ) ^ 2+( x ( 1 )+x ( 2 ) ^ 2-7 ) ^ 2
which has four global minima. himmelblau.m, defines the Himmelblau function. LOCAL is a badly scaled function with a local minimum, for which N=2. local.m, defines the "local" function. MCKINNON is the McKinnon function, for which N=2. This function can cause problems for the NelderMead optimization algorithm. mckinnon.m, defines the McKinnon function. POWELL is the Powell singular quartic function, for which N = 4. powell.m, defines the Powell function. ROSENBROCK is the Rosenbrock "banana" function. The contour lines form a nested set of "banana", which can make convergence very slow:
f ( x )=(1-x ( 1 )) ^ 2+1 0 0*(x ( 2 )-x ( 1 )*x ( 1 )) ^ 2
people.sc.fsu.edu/~jburkardt/m_src/nelder_mead/nelder_mead.html 4/5
11/12/13
rosenbrock.m, defines the Rosenbrock function. You can go up one level to the MATLAB source codes. Last revised on 06 September 2010.
people.sc.fsu.edu/~jburkardt/m_src/nelder_mead/nelder_mead.html
5/5