Copper Slides

Fast Algorithms for the Geostatistical approach for solving
Linear Inverse Problems based on Hierarchical matrices

Arvind Saibaba1
Peter K. Kitanidis2,1
1
Institute for Computational and Mathematical Engineering

2
Department of Civil and Environmental Engineering.
Stanford University
March 30, 2012
Arvind Saibaba Peter K. Kitanidis
Geostatistical approach to Inverse problems
1 / 21
Outline
Motivation
Bayesian approach to Inverse problems
Hierarchical matrices
Solving the system
Contaminant Source Identification
2 / 21
Contaminant Source Identification
Figure: Stockie SIAM Review 2011
Figure: Flath et al. SISC 2011

3 / 21
Random Field
Model unknowns as a Gaussian random field
E [s] = X
E [(s X)(s X)T ] = Q
Figure: Three realizations of a Gaussian random field with exponential covariance
Storage and computational costs for Qij = (xi , xj ) i, j = 1, . . . , m high.

Examples of (, ) : Matern family, Gaussian, Exponential.
4 / 21
Bayesian viewpoint
Consider the measurement equation

y = h(s) + v
v N (0, R)
where,
y
s
h(s)
:=
:=
:=
:=
observations or measurements - given.

model parameters, we want to estimate.
parameter-to-observation map - given.
(unknown) drift coefficients
Using Bayes rule, the posterior pdf is

p(s, |y)
p(y|s, )p(s, )

1
1
exp ks XkQ1 ky HskR1
2
2
5 / 21
Best Estimate
Maximum a posteriori estimate:

1
1
arg min
ks XkQ1 + ky HskR1
s,
2
2
Solution:

HQHT + R
(HX)T
HX
0

= 0
+ QHT
s = X
Operation costs:
n
m
p
:
:
:
number of measurements 103

number of unknowns 105
number of drift coefficients 1
Construction: O(m2 n + mn + mnp)
Solving: O(n + p)3
6 / 21
Fast summation algorithms
Various schemes for computing

(Qv)i =
N
X
(xi , xj )vj
i, j = 1, . . . , N
j=1
FFT based algorithms

Restricted to regular grids.
O(N log N) complexity.
7 / 21

(Qv)i =
N
X
(xi , xj )vj
i, j = 1, . . . , N
j=1

Fast Multipole Method - Greengard and Rokhlin.

N-body problem, Boundary Element Method, Radial basis interpolation.
O(N log N) or O(N) complexity.
Several black-box versions exist.
7 / 21

(Qv)i =
N
X
(xi , xj )vj
i, j = 1, . . . , N
j=1

Fast Multipole Method - Greengard and Rokhlin.

N-body problem, Boundary Element Method, Radial basis interpolation.
Several black-box versions exist.
Hierarchical matrices - Hackbusch and co-workers.

Boundary Element Method, preconditioners for sparse pde systems.
Other matrix operations possible - adding, multiplying, inversion,
factorizations.
7 / 21
H-matrix formulation: An Intuitive Explanation.
1
, i = 1, . . . , N
Consider : [0, 1]2 R, for xi , yi = (i 1) N1
(x, y ) = exp(|x y |)
Figure: blockwise rank- = 106 , N = M = 256
8 / 21
H-matrix formulation
Key features of Hierarchical matrices:
A hierarchical separation of space.
An acceptable tolerance that is specified.
Low-rank approximation of admissible sub-blocks.
Valid for asymptotically smooth kernels.
Definition
A cluster pair (, ) is considered admissible if
min{diam(X ), diam(X )} dist(X , X )
Definition
A kernel is called asymptotically smooth, if there exist constants c1as , c2as and a
real number g 0 such that for all multi-indices Nd0 it holds that

y K (x, y) c1as p!(c2as )p (|x y|)g p , p = ||
9 / 21
H-matrix
Storage and Multiplication is almost linear complexity.

Low rank approximation computed by Adaptive Cross Approximation.
Well suited for Krylov Subspace methods (eg. GMRES, CG).
3
10
HierMatrix
Direct
2
time (sec)
10
10
10
10
10
10
10
10
10
Figure: left: A typical H-matrix rank structure and right: Time for matrix vector
product for exponential covariance function = 106
10 / 21
Iterative solver
Krylov subspace methods for solving Ax = b, at the i-th iteration satisfy
ri span{r0 , Ar0 , A2 r0 , . . . , Ai1 r0 } = (A)r0 , Pi
where, ri = b Axi , is the residual at the i-th iteration.
Minimal residual methods such as MINRES or GMRES try to compute a
polynomial such that
kri k = min k(A)r0 k
Pi
We apply it to the system

HQHT + R
(HX)T
|
{z
A
HX
0

= 0
A is not constructed explicitly, rely only on matrix vector products. For eg.

z }| {

HQHT + R x = H Q |{z}
HT x +Rx
| {z }
Construction: O(p+k 2 m log m)

Matrix-vector product: O(km log m+2+np+n)

11 / 21
Eigenvalue decay of Covariance kernel (, )
Consider the integral eigenvalue problem

Z
(x, y)(y)dy = (x)
D
Smoother the kernel is, the faster {m } 0.

If D Rd and if the kernel is
piecewise H r
piecewise smooth
piecewise analytic
m c1 mr /d
m c2 mr for any r
>0
m c3 exp c4 m1/d
We expect the eigenvalues of the Qij = (xi , xj ) to behave similarly.
Schwab and Todor (2006).

12 / 21
Steps to compute Preconditioner

= HQHT + R = R1/2 (R1/2 HQHT R1/2 + I)R1/2
Compute low-rank representation using Thick-restart Lanczos algorithm
Q Vr r VTr
Form the matrix
M = R1/2 HVr 1/2

r
Compute the singular value decomposition (SVD) of the matrix M

M = UVT
Use Sherman-Morrison-Woodbury update to compute the inverse of

i2
(MMT + I)1 = I UDr UT
Dr = diag
1 + i2
1 as
Compute the approximate inverse of , which we denote by
1 = R1 R1/2 UDr UT R1/2
13 / 21
Steps to compute Preconditioner - computational complexity
Assumptions: Hx has complexity and R is diagonal.

1
2
3
4
5
Q Vr r VTr
1/2
M = R1/2 HVr r
T
M = UV
(MMT + I)1 = I UDr UT
1 = R1 R1/2 UDr UT R1/2
O(rkm log m)
O(r + r )
O(nr 2 )
O(r )
O(nr )
1 x only costs O(nr ).

Finally,
14 / 21
Why does it work?
Theorem (Bauer-Fike)
Let A be a diagonalizable matrix, and V be the non-singular eigenvector matrix
such that A = VV1 . If is an eigenvalue of A + A, then an eigenvalue
(A) exists such that
| | cond(V)kAk
1 , we have
Applying this result to the matrix A =
1 )| r kQkkHk2 k
1 k
|1 (
gives us an explicit bound on the spectral radius of the matrix.
15 / 21
Application: Contaminant Source Identification
Forward problem
u
v.u + D2 u
t
u
u
[0, T ]
0,
u0 ,
{t = 0}
D [0, T ]
Measurement Operator
h(s) = Hs = |{z}
H
Sensors
A1
|{z}
Forward Propagation
T
|{z}
Prolongation
16 / 21
Reconstruction
Figure: Reconstruction of a Gaussian initial condition 2 exp(30kx xc k) which is

centered at xc = (0.25, 0.5). The unknowns are discretized on a 100 100 grid in
space and 20 time steps, in the domain with L = T = 1. The measurements are
collected in a 10 10 10 grid, i.e. nm = 100, nt = 10. (left) reconstructed field and
(right) true field. The relative error in the reconstruction was 0.063.
17 / 21
Results: with preconditioner
All tests performed with

K (x, y) = (1 + r + (r )2 /3) exp(r )
Sensors
88
10 10
12 12
Unknowns
100 100
200 200
300 300
100 100
200 200
300 300
100 100
200 200
300 300
Iterations
30
32
33
38
39
41
136
196
200
r = kx yk
Rel. Err
0.0941
0.0949
0.0953
0.0669
0.0675
0.0679
0.0495
0.0503
0.0500
Table: The performance of the iterative scheme for the contaminant source
identification problem. In each case the number of time measurements were nt = 10,
with L = T = 1 and t = 0.05. For the preconditioner, we used r = 100.
18 / 21
Results: with preconditioner
All tests performed with

K (x, y) = (1 + r + (r )2 /3) exp(r )
r
129
201
278
355
Iterations
300
196
96
41
r
2.45 105
4.76 106
1.75 106
7.09 107
r = kx yk
Rel. err.
0.0148
0.0146
0.0145
0.0144
Table: Performance of iterative scheme with increasing r for grid size 100 100 and
number of sensors 25 25, so that number of measurements are 6, 250. . indicates
that it reached the maximum number of iterations 300, without converging to the
desired solver tolerance.
19 / 21
Spectrum of preconditioned matrix
10
10
m=400
m=1600
m=10000
m=400
m=1600
m=10000
2
10
| |
# | 1| > x
10
10
10
10
10
10
10
10
10
6
10
10
10
x = | 1|
10
10
Figure: (left) Eigenvalues of the preconditioned operator (right) Plot of

#{i : |i 1| > x} against x for all eigenvalues. For these plots, we assumed the
number of observations to be 10 10 10, the number of unknowns varied from
20 20 to 100 100.
20 / 21
Conclusions
Our contributions
A scalable matrix-free approach to solving linear inverse problems.
A preconditioner that is expensive to compute but performs well.
Quantifying uncertainty - generating conditional realizations.
Unconditional realizations using Chebyshev matrix polynomials.
Acknowledgement The authors were supported by NSF Award 0934596,

Subsurface Imaging and Uncertainty Quantification.
21 / 21

Copper Slides

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Copper Slides

Uploaded by

Copyright:

Available Formats

Fast Algorithms for the Geostatistical approach for solving

Linear Inverse Problems based on Hierarchical matrices

Institute for Computational and Mathematical Engineering

March 30, 2012

Arvind Saibaba Peter K. Kitanidis

Geostatistical approach to Inverse problems

Bayesian approach to Inverse problems

Solving the system

Contaminant Source Identification

Arvind Saibaba Peter K. Kitanidis

Geostatistical approach to Inverse problems

Contaminant Source Identification

Figure: Stockie SIAM Review 2011

Figure: Flath et al. SISC 2011

Geostatistical approach to Inverse problems

E [(s X)(s X)T ] = Q

Figure: Three realizations of a Gaussian random field with exponential covariance

Storage and computational costs for Qij = (xi , xj ) i, j = 1, . . . , m high.

Arvind Saibaba Peter K. Kitanidis

Geostatistical approach to Inverse problems

Consider the measurement equation

observations or measurements - given.

Using Bayes rule, the posterior pdf is

Arvind Saibaba Peter K. Kitanidis

Geostatistical approach to Inverse problems

Maximum a posteriori estimate:

number of measurements 103

Construction: O(m2 n + mn + mnp)

Arvind Saibaba Peter K. Kitanidis

Solving: O(n + p)3

Geostatistical approach to Inverse problems

Fast summation algorithms

Various schemes for computing

FFT based algorithms

Arvind Saibaba Peter K. Kitanidis

Geostatistical approach to Inverse problems

Fast summation algorithms

Various schemes for computing

FFT based algorithms

Fast Multipole Method - Greengard and Rokhlin.

Arvind Saibaba Peter K. Kitanidis

Geostatistical approach to Inverse problems

Fast summation algorithms

Various schemes for computing

FFT based algorithms

Fast Multipole Method - Greengard and Rokhlin.

Hierarchical matrices - Hackbusch and co-workers.

Arvind Saibaba Peter K. Kitanidis

Geostatistical approach to Inverse problems

H-matrix formulation: An Intuitive Explanation.

Figure: blockwise rank- = 106 , N = M = 256

Arvind Saibaba Peter K. Kitanidis

Geostatistical approach to Inverse problems

Geostatistical approach to Inverse problems

Storage and Multiplication is almost linear complexity.

Arvind Saibaba Peter K. Kitanidis

Geostatistical approach to Inverse problems

We apply it to the system

Construction: O(p+k 2 m log m)

Matrix-vector product: O(km log m+2+np+n)

Eigenvalue decay of Covariance kernel (, )

Consider the integral eigenvalue problem

Smoother the kernel is, the faster {m } 0.