2.

Over- and Under-Determined Problems

N>M

usually no model will match all of the data due to errors on the data due to inconsistency between the model parameterization and reality compute the data misfit:
M

d i = ei g i m = ei gij m j
j =1

d = e Gm

minimize a norm of the data misfit, where any norm could be used define an objective function and minimize its value

= d
M 2 = ei gij m j i=1 j =1 N

if the l2 norm is chosen ==> least squares method

= d 22

GEOS 5004: Analytical Geosciences

Hole: 2.1

Inversion: Over- and Under-Determined Problems

least-squares method for minimizing an objective function by choosing the correct parameters: - take the partial derivative of the objective function with respect to each parameter - set the partial derivatives to equal zero results in M equations (one for each derivative) in the M unknowns
N M = 0 = 2 ei gij m j gik m k i=1 j =1

k = 1, 2,..., M

M ij j ik

g m g
i=1 j =1

= ei gik
i=1 N

k = 1,2,..., M k = 1,2,..., M
N N T ik

N T

(g )
j =1 i=1

T g m = g ( ) ki ei ij j ki i=1
T

( G G) m = G
T

( G G) = ( g )
ij k =1

g kj = g ki g kj
i=1

m = ( G T G) G T e matrix that needs to be inverted is square, of size MxM , and symmetric the
if none of the eigenvalues are zero

the least squares misfit solution:

the matrix can be inverted eigenvalue decomposition can be used to find the solution
Hole: 2.2 Inversion: Over- and Under-Determined Problems

GEOS 5004: Analytical Geosciences

least-squares best-fitting straight line

y = + x

e = Gm

y1 1 x1 M = M M yN 1 x N
1 N 2 xi 1 N i=1 1 T G G = ( ) N 2 2 x Nx i x i=1
N N N

N GT G = N x i i=1

x i 1 i=1 = N N xi 2 x i=1
N

x N 1 xi 2 N i=1

x 1

N y yi G T e = Ni=1 = N N 1 x y x y i i i i N i=1 i=1

y x i 2 x x i yi

x y Nxy
i i

i=1 N

i=1

i=1 N 2 x i Nx 2 i=1

2 x i Nx 2 i=1

these are the familiar expressions from linear regression

under-determined the number of data is less than the number of model parameters

N<M

an infinite number of models match the data solution is non-unique the data kernels do not sample all directions in the M -dimensional model space models that fit the data can have any length in the unsampled directions minimize a norm of the model size, where any norm could be used define an objective function and minimize its value

= m
this can only be solved by adding the constraints that the model must fit the data if the l2 norm is chosen ==> l2 smallest model
M M

e = Gm

= m2

2 = m m = m j
j =1

subject to:

ei gij m j = 0
j =1

i = 1,2,..., N

such a system is solved by the Lagrange multiplier method, adding extra unknown parameters (called Lagrange multipliers) i (dont confuse these with eigenvalues, despite same letter!)

M N = m j + 2 i ei gij m j j =1 i=1 j =1 M 2

GEOS 5004: Analytical Geosciences

Hole: 2.4

Inversion: Over- and Under-Determined Problems

smallest model - take the partial derivative of the objective function with respect to each parameter - set the partial derivatives to equal zero results in M+N equations (one for each derivative) in the M+N unknowns differentiating with respect to the i results in the original data equations: e = Gm

mj

and

= 0 = 2 m k 2 i gik m k i=1
N

k = 1,2,..., M

mk = (g
i=1

ki

k = 1,2,..., M
m = GT

substitute into the data equations

= (GG ) e substitute back into the previous equation e= GG T

T 1

M T ik kj

(GG ) = g ( g )
T ij k =1

= gik g jk
k =1

m = G T (GG T ) e
the matrix (called the Gram matrix) that is to be inverted is square, of size NxN , & symmetric if none of the eigenvalues are zero the matrix can be inverted eigenvalue decomposition can be used to find the solution

GEOS 5004: Analytical Geosciences Hole: 2.5 Inversion: Over- and Under-Determined Problems

smallest model example: in 2D space, acquire only one datum

g1 = [2 3] 2 m1 + 3m2 = 7

e1 = 7

e1 = g1 m

in a plot of m2 vs. m1 , this is the formula for a straight line any point on that straight line is a satisfactory match to the data the vector g1 is perpendicular to this line, while [3,2] is parallel the line model is the one that is closest (in the l2 sense) to the origin [0,0] , which will be a the smallest distance perpendicular to the line (parallel to the data kernel) for the example above, this is the point

7 2 m2 = + m1 3 3

7 2 m= 13 3 7 2 3 m = + 13 3 2

all other models that fit the data equal the smallest model plus a distance parallel the line for the example above, all models that fit the data are:

example: in 3D space, acquire only two data each datum equation gives the formula for a plane in 3D space the data kernel vectors are perpendicular to these planes the intersection of these two planes defines a line in 3D space, along which all models match the data; this line is perpendicular to the two data kernels
GEOS 5004: Analytical Geosciences Hole: 2.6 Inversion: Over- and Under-Determined Problems

smallest model
N

m=G
where the

m k = ( g T ) ki i
i=1

were the Lagrange multipliers, a set of unknown constants

rearranging (and replacing

with

ai

so as to minimize confusion with eigenvalues)

m = aG

m k = ai gik
i=1

m = ai g i
i=1

the smallest model is a linear combination of the data kernel vectors

models that fit the data can have any length in the directions perpendicular to the data kernels because we have not measured the model length in those directions => the smallest model is chosen to have zero length in those directions philosophically: if we know nothing, assume literally nothing all models that fit the data can be expressed as
M

m = m smallest +

a n
j j = N +1

where the nj are a set of vectors perpendicular to each other and to the data kernels any model represented by the term at right is called an annihilator because it produces data that equal zero; these models point into null space, which is unsampled by the data kernels

GEOS 5004: Analytical Geosciences Hole: 2.7 Inversion: Over- and Under-Determined Problems

closest model (smallest model deviation) but physically, does it make sense to make the model small as close to zero as possible? create reference model m0 that we believe looks like reality m0 does not match the data, but has the size/shape/character we expect for example, based upon knowledge from outside our experiment for example, based upon a hypothesis we would like to test models that match the data can be expressed as

m = m 0 + m
substitute this into the data equation

G(m 0 + m) = e

Gm = (e Gm 0 ) Gm = d 0

where d0 is the calculated misfit of the model m0 this is the exact same system of linear equations, with the data and model replaced by a model deviation (or perturbation) and a corresponding data misfit use the smallest model technique to find the smallest model deviation m => find the closest model to the reference model by adding this smallest deviation to the reference model by selecting different reference models, we can explore the set of models that fit the data (or do not)!
GEOS 5004: Analytical Geosciences Hole: 2.8 Inversion: Over- and Under-Determined Problems