ENCH 421 – Dr.

Hector De la Hoz Siegler

Experimental Data Analysis – Least Squares

What it is? A regression analysis method to find the “best” values for the model parameters.

Linear least squares

min L = eT e = (yˆ − Mβ ) (yˆ − Mβ )

T
e
β

∂L
= 0 = −2M T (yˆ − Mβ ) = −2M T yˆ + 2M T Mβ
∂β

M T Mβ = M T yˆ ⇒ [
β = M T M M T yˆ ]−1

Calculations can be done by hand, with a calculator, in Excel, Matlab, or Polymath.

Provides an optimal solution if the original model was linear.

Non-linear least squares

For non-linear models, there is no close algebraic solution to estimate parameter values. Solution requires
an iterative (trial and error) approach.

For a non-linear model the error function is defined as:

e 2 = ∑ ( yi − f ( xi , β1 , β 2 ,) )
2

Newton method:

 β1   β1  H is the Hessian matrix, 2nd derivatives of L w.r.t. model parameters

   
 β 2  =  β 2  − H ∇L
−1
∇L is the gradient of L (vector of all 1st derivatives)
    
 i +1   i

 ∂ (e2 )
2 2
∂ 2 ( e2 )
∂β1∂β 2 
2
(
 ∂∂(βe )   ∑ − 2 yi − f ( xi , β1 , β 2 ,) ∂β1
∂f
)
 ∂2 β1 2   2   i 
( )
1

H ( L) =  ∂∂β2(∂eβ1) ∂ 2 ( e2 )
∂β 2 2
 ∇L =  ∂∂(βe2 )  = ∑ − 2 yi − f ( xi , β1 , β 2 ,) ∂∂βf 2
      i 
 
     


Levenberg – Marquardt method

−1
 β1   β2   Σ ∂∂βf ∂f
Σ ∂∂βf1 ∂f

     ∂f 1 ∂β1 ∂β 2 
 β 2  =  β 2  − α1 ∇ − α 2
∂f ∂f ∂f
 Σ ∂β1 ∂β 2 Σ ∂β 2 ∂β 2  ∇
      
 i +1   i   

This method avoids the costly calculation of the Hessian matrix. Implemented in Matlab as lsqnonlin

1
ENCH 421 – Dr. Hector De la Hoz Siegler

EXCEL SOLVER - HOW TO LOAD OR START SOLVER

In Excel for Windows, if you don't see the Solver command or the Analysis group on the Data tab, you need to
load the Solver add-in.

1. Click the File tab, click Options, and then click the Add-ins category.

2. In the Manage box, click Excel Add-ins, and then click Go.

3. In the Add-ins available box, select the Solver Add-in check box. If you don't see this name in the
list, click theBrowse... button and navigate to the folder containing Solver.xlam. Then click OK.

4. Now on the Data tab, in the Analysis group, you should see the Solver command.

In Excel for Macintosh, if you don't see the Solver command on the Tools menu, you need to load the Solver
add-in.

1. Click the Tools menu, then click the Add-ins command.

2. In the Add-ins available box, select the Solver.xlam check box. If you don't see this name in the list,
click theSelect... button and navigate to the folder containing Solver.xlam. Then click OK.

3. Now on the Tools menu, you should see the Solver command.

In Excel Online for Office 365, Excel Web App for SharePoint, or Excel on iPad, use the Solver add-in. If you
don't see the Solver Task Pane in your worksheet, you need to insert the Solver add-in:

1. Open the workbook where you want to use Solver. Click the Insert tab, then click the My Add-
ins button.

2. If Solver appears in the Recently Used Add-ins dropdown list, select it there, and skip to step 4.

3. Select See All... from the dropdown menu. In the Office Add-ins dialog, find and select Solver under
My Add-ins or My Organization.
If this is your first time using the Solver add-in, click Store, and look in the Data Analytics category.
Click to see the Solver add-in listing in the Office Store.

4. The Solver Task Pane should appear. Click File Save As, and save to your Office 365 or SharePoint
document library, or OneDrive or OneDrive for Business folder.

5. Now when you open your workbook, the Solver Task Pane should appear automatically.