A brief review of Linear Algebra and Linear Programming Models

Developed by James Orlin

1
MIT and James Orlin 2003

Review of Linear Algebra

  

Some elementary facts about vectors and matrices. The Gauss-Jordan method for solving systems of equations. Bases and basic solutions and pivoting.

2
MIT and James Orlin 2003

Elementary Facts about Vectors

v ! ?v1 v2 v3 v4 A is called a row vector.
v1 v vt ! 2 The transpose of v is a column vector. v3 v4

w ! ?w1

w2

w3

w4 A is another row vector.

The inner product of vectors w and v is given by: v $ w ! v1w1  v 2 w2  v3 w3

MIT and James Orlin 2003

 v4 w4
3

Matrix Multiplication
A! (aij ) B ! (bij ) C ! (cij ) ! AvB

Suppose that A has n columns and B has n rows.

cij ! k !1aikbkj
n
4
MIT and James Orlin 2003

Multiplying Matrices
Let C = (cij) = AB. Then cij is the inner product of row i of A and column j of B.
a11 a12 A ! a21 a22 a31 a32 a13 b11 b12 a23 B ! b21 b22 a33 b31 b32 b13 b23 b33

For example, what is c23?

c23 ! a21b13  a22 b23  a23b33

5
MIT and James Orlin 2003

Multiplying Matrices
Let C = (cij) = A v B. Then each column of C is obtained by adding multiples of columns of A.
1 2 3 100 123 4 5 6 v 10 ! 456 7 8 9 1 789
1 2 3 ! 100 4  10 5  6 7 8 9
MIT and James Orlin 2003

Similarly, each row of C is obtained by adding multiples of rows of B.

6

Elementary Facts about Solving Equations

Solve for Ax = b, where

x1 1 2 4 x ! x A! 2 2 1 1 x3
2v3 3v1

0 b! 6
2v1

x1  2 x2  4 x3 ! 0 2 x1  x2  x3 ! 6
Find a linear combination of the columns of A that equals b.

1 2 4 0 2 x1  1 x2  1 x3 ! 6

Solving a System of Equations

x1 x2 x3 x4

1 2 -1

2 1 1

4 -1 2

1 -1 2

= = =

0 6 -3

To solve a system of equations, use Gauss-Jordan elimination.

8
MIT and James Orlin 2003

To solve the system of equations:

x1 x2 2 1 1 x3 4 -1 2 x4 1 -1 2 0 6 -3

1 2 -1

= = =

Go to the animation
9
MIT and James Orlin 2003

The fundamental operation: pivoting

x1 x2 x3 x4

a11 a21 a31

a12 a22 a32

a13 a23 a33

a14= = a24 = a34

b1 b2 b3

Pivot on a23
10
MIT and James Orlin 2003

Pivot on a23
x1 x2 x3 x4

Da11 =a11 a13(a21/a23)

a11 Da11

a12

a0 13 a1 23 0 a33

a14= = a24/a23 a24 = a34

b1 b2b2 23 /a b3

a21/a23 a22/a23 a21 a22 a31 a32

What will be the next coefficient of b1? a32? of aij for i {2? MIT and James Orlin 2003

11

Jordan Canonical Form for an m x n matrix

x1 x2 x3 x4

1 0 0

0 1 0

0 0 1

-1 1 0

= = =

2 1 -1

There are m columns that have been transformed into unit vectors, one for each row. The variables in these columns are called basic. The remaining variable(s) are called non-basic.
MIT and James Orlin 2003

12

Jordan Canonical Form for an m x n matrix

x1 x2 x3 x4

1 0 0

0 1 0

0 0 1

-1 1 0

= = =

2 1 -1

In the basic solution, the non-basic variables are set to 0. The basic solution is:
MIT and James Orlin 2003

x1 = 2, x2 = 1, x3 = -1, x4 = 0
13

There is an easily determined solution for every choice of non-basic variables.

x1 x2 x3 x4

1 0 0

0 1 0

0 0 1

-1 1 0

= = =

2 4 -2 1 -1 3

We get a feasible solution to the system of equations for every choice of values for non-basic variables. If we set x4 = 2, what solution do we get? If we set x4 = (, what solution do we get? 14
MIT and James Orlin 2003

Another Jordan Canonical Form for the same system of equations

x1 x2 x3 x4

0 1 0

1 1 0

-2 0 0 3 -3 1

-1 1 0 1 0

= = =

1 4 -2 3 -1 3

What are the basicvariables? What is the basic solution?

15
MIT and James Orlin 2003

Applications


A Financial Model
an investment model over multiple time periods goal: optimize the total revenues at the end of the time horizon

16
MIT and James Orlin 2003

A Financial Problem
Sarah has $1.1 million to invest in five different projects for her firm.  Goal: maximize the amount of money that is available at the beginning of 2006.


(Returns on investments are on the next slide).

At most $500,000 in any investment, except for CDs.  Can invest in CDs, at 5% per year.


17
MIT and James Orlin 2003

Return on investments (undiscounted dollars)

A Jan. 2003 Jan. 2004 Jan. 2005 Jan. 2006 -1 .4 .8 B -1 .4 .8 C -1 1.2 D -1 1.5 E -1 1.2
18
MIT and James Orlin 2003

Formulate Sarahs problem as an LP

Payback for A: for every dollar invested in January of 2003, Sarah receives $.40 in January of 2004 and $.80 in January of 2005.  FORMULATION.


STEP 1. Choose the decision variables Let xA denote the amount in millions of dollars invested in A. Define xB, xC, xD, and xE similarly. Let x3 denote the amount put in a CD in 2003. (Define x4 and x5 similarly)

19
MIT and James Orlin 2003

Formulating the model

 

With your partner formulate the LP model. Step 2. Formulate the objective function
put the objective function in words first. E.G. we are minimizing cost or maximizing utility

Step 3. Formulate the constraints

Put the constraints in words first

20
MIT and James Orlin 2003

The model
Max z= Subject to .8 xB + 1.5 xD + 1.2 xE + 1.05 x5 xA + xC + xD + x3 = 1.1 (Year 2003) (Year 2004) (Year 2005)

.4 xA - xB + 1.2 xC + 1.05 x3 - x4 = 0 .8 xA + .4 xB - xE + 1.05 x4 - x5 = 0 0 e xj e .5 for j = A, B, C, D, E, 3, 4, 5

Excel Solution
21
MIT and James Orlin 2003

FAQ. Do the units matter? How does one choose the units?


The units do not matter so long as one is careful to use units correctly. It would be possible to have xA be in $ millions and for xB to be in dollars. But some choices of units are more natural than others, and easier to use and to communicate.

22
MIT and James Orlin 2003

Addressing Problems in Practice



Understanding the model:

know the purpose of the model vary numbers and assess the impact (sensitivity analysis) discuss ways of enhancing the model

Financial Problems in practice are much larger, involving lots of potential investments, and more time periods. Data is often stored in a database. Other (to be addressed soon)
23

MIT and James Orlin 2003

Understanding the model



How might management use this model in practice? Lets vary some of the data and see if we can obtain insights:
How will the return vary with the initial investment? How will the return vary with the CD rate of return? Use SolverTable or usual sensitivity analysis Excel Solution
24

MIT and James Orlin 2003

Scaling up: dealing with large problems

  

  

n investments, over m time periods. Total budget B to invest in period 1. The payback of $1 of investment j in period i is pij. If investment j starts in period i, then pij = -1, indicating that $1 is invested in that period. Everything is reinvested. Maximum of uj in investment j. Maximize the total return in period m.

Work with your partner on formulating the generalization. Let xj be the amount put into investment j
25
MIT and James Orlin 2003

The model
Maximize z ! subject to

n j !1 n j !1

n j !1

pmj x j

p1 j x j !  B pij x j ! 0 for i = 2 to m-1

0 e x j e u j for j = 1 to n
26
MIT and James Orlin 2003

Enrichments of the model



Finance concentrators: have we made assumptions that you would like to challenge? Can we deal with a more realistic model?

27
MIT and James Orlin 2003

Summary


Gauss-Jordan solving of equations and other background in linear algebra A financial problem Note: Modeling in practice is an art form. It requires finding the right simplifications of reality for a given situation.
28

MIT and James Orlin 2003