Tech-Pro Consultants

Under-Determined, OverDetermined, Ill-conditioned Linear Systems

August 2005 Dr. K.S.Ravichandran

Linear Systems

A non-singular matrix A can always be factorized in to A=LU where L is lower triangular and U is strictly upper triangular with unit entries on the diagonal. The procedure for triangulation breaks down for a singular matrix because at some stage of the procedure, a non-zero pivot does not exist.

Reference :
1.

Lecture notes on optimization, Prof. Baldick, The University of Texas at Austin

Tech-Pro Consultants

Linear Systems Ill-Conditioning

Example:Consider Ax = b,

Det[A] = - , 0 and the matrix A is nearly singular.

Using infinite precision arithmetic, we can solve Ax = b for any vector b = [b1 , b2]t. The solution is

Tech-Pro Consultants

Linear Systems - Perturbations

Example: Take b = [1 , 1]t. . Then ||b || = 2 and the solution x* = [1 , 0]t , || x* || = 1 Perturb the RHS: Take b = [1 , 1]t.+ [ , 0]t = b + b Then ||b || = and the solution x** = x*+ x* = [1 , 0]t + [0 , /]t , || x** || = {1+ (/)2} || x* || =|/|. The relative change in solution is || x* || / || x* || =|/|. The relative change in b is || b || / || b || =|| / 2.

Tech-Pro Consultants

Linear Systems Ill-Conditioning

Ratio of the relative change is =|/| / (|| / 2) = 2/ || The change in the solution vector is independent of the change in the forcing vector and is large if is small. We say that the matrix is ill-conditioned.

Tech-Pro Consultants

Linear Systems Ill-Conditioning

Perturb the Matrix : Take (A+ A)x = b. Let A = Then ||A || = and the solution. With b = [1 , 1]t x** = x*+ x* = [1 , 0]t + [0 , -/]t , || x* || =|/|. The relative change in solution is || x* || / || x* || =|/|. The relative change in A is || A || / || A || =|| / 2. Ratio of the relative change is =|/| / (|| / 2) = 2/ || The change in the solution vector is independent of the change in the matrix and is large if is small. Thus the matrix is ill-conditioned.

Tech-Pro Consultants

Analysis Condition Number

Condition Number of a Matrix : Denote the L2 norm of a vector x by ||x|| = <x,x> = xtx

The L2 norm of a matrix A is given by Max ||Ax=y|| for all x s.t. ||x|| = 1.
Typically the L2 norm of a matrix is computed as

||A||2 = { aij2}1/2 where the aij are the elements of A

The condition number is now defined as C = ||A||2 ||A-1||2

A matrix for which C is large is said to be ill-conditioned

Tech-Pro Consultants

Analysis Main Result

Suppose A is a non-singular matrix , and consider Ax = b, Ax = b + b and (A + A)x = b. We then have the following bounds: (i) If x*+ x* solves Ax = b + b we have || x*|| / || x*|| ||A||2 ||A-1||2 ||b|| / ||b|| = C ||b|| / ||b|| (ii) If x*+ x* solves (A + A)x = b we have || x*|| / || x* + x*|| C ||A|| / ||A|| where Ax* = b In either case, the amplification to the solution is large if the condition number of the system is large.

Tech-Pro Consultants