1 Maximum Likelihood Estimation of Model

Parameters
(a) For scenario 2, find out which anchor has exponentially distributed measurements.
To find the used distribution, different methods can be considered. The most simple
but also less reliable way is to use the properties of the distributions. Considering
our N = 2000 measurements be can be relatively sure that if the data is gaussian
distributed the mean of our data is close to the actual distance between the anchors
and pref since this is the expected value. In case the difference of the mean and
the real distance is greater than an estimated  we can be sure that an other
distribution was used with an expected value other than the mean. In our case
this would be the exponential distribution. Another way to use the properties
would be to look for negative differences of the data and the actual distance.
Since the exponential distribution is only defined for positive values negative ones
would show that the data is gaussian. These methods are less reliable since there
are cases in which this separation does not work like expected valuegaussian =
expected valueexponential or larger measurement errors.
A better way to evaluate the used distribution is a statistical hypothesis test. In
this case we create a null hypothesis with assumes that the given data is gaussi-
an distributed. For this assumption the python package scipy.stats provides the
shapiro() function which uses the Shapiro-Wilk test as testing function and re-
turns the probability that the null hypothesis is correct. If this value is greater
than the chosen statistical significance value (like 0.05) we can assume that the
data was gaussian distributed.
In contrast to those analytical methods there are also graphical options. By plot-
ting the data set as histograms the distribution can be identified in case those
differentiate enough. Since the gaussian and exponential PDF are very different
from each other this separation would work. By fitting known distributions on the
data even more similar sets can be set apart.
The implementation in HW 4_skeleton.py is based on the first method presented,
since the use of graphical methods is less convenient in the further processing and
the use of copious python function is not explicit permitted.

(b) Analytically derive the maximum likelihood solution for the exponential distribu-
tion.
The calculation of the max. likelihood function was done with the use of the log-
likelihood. Additionally the limit of the product was changed to 1 till N instead of
0 till N-1 since it does not make any difference and to make sure that this 1 step
does not get lost.

1
 Gaussian
N
Y

θM L = argmax p dn (ai , p) |p
θ n=1
(d˜n (ai ,p)−d(ai ,p))2
N
!
X 1 −
2σ 2
= argmax log p e i
θ n=1 2πσi2

Set the first derivative

 to zero to receive
 the max. likelihood estimator. Here we
˜
substitute x = dn (ai , p) − d (ai , p) .

N 2
!!
X 1 − 2σx 2 1 1
0= e i − 2
+ 4
x2
n=1
2π σM L,i σM L,i

By dividing by the exponential function and multiplying π we can simplify the

equation. Also the sum can be split in two.

N N
X 1 X 1
0= − 2
+ x2
n=1
σM L,i σ4
n=1 M L,i
N
1 1 X
2
N= 4
x2
σM L,i σM L,i n=1
N
2 1 X
2
σM L,i = x
N n=1
N 
1 X 2
= d˜n (ai , p) − d (ai , p)
N n=1

2
 Exponential
N
Y

θM L = argmax p dn (ai , p) |p
θ n=1
N  
˜
log λi e−λi (dn (ai ,p)−d(ai ,p))
X
= argmax
θ n=1

Set the first derivative

 to zero to receive
 the max. likelihood estimator. Here we
˜
substitute x = dn (ai , p) − d (ai , p) .

N  
X 1
0= (1 − λM L,i · x)
n=1
λM L,i

Multiplying by λM L,i and splitting the sum.

N
X N
X
0= 1− λM L,i · x
n=1 n=1
N
X
N = λM L,i x
n=1
N
λM L,i = PN
n=1 x
N
=P  
N ˜n (ai , p) − d (ai , p)
d
n=1

(c) Estimate the parameters for all anchors, i.e., estimate σi2 and λi in all 3 scenarios
using the maximum likelihood method.
Table 1 shows the parameters estimated with the max. likelihood function. Co-
lumn Dist. shows the distribution used where G. stands for gaussian and E. for
exponential. If the distribution is gaussian the parameter σi2 takes effect otherwise
λi .

3
 Tabelle 1: Estimated parameters.
Anchor 0 Anchor 1 Anchor 2 Anchor 3
Dist. σi2 /λi Dist. σi2 /λi Dist. σi2 /λi Dist. σi2 /λi

Scenario 1 G. 0.0909 G. 0.0884 G. 0.0871 G. 0.0925

Scenario 2 E. 1.1210 G. 0.0837 G. 0.0933 G. 0.0899
Scenario 3 E. 1.080 E. 1.0888 E. 1.1109 E. 1.1372

2 Estimation of the Position

2.1 Least-Squares Estimation of the Position

(a) Show analytically that for scenario 1 (all anchors are Gaussian), the least-squares
estimator of the position is equivalent to the maximum likelihood estimator.
It is to prove that:

NΛ 
Y 
p̂M L (n) = argmax ˜
p dn (ai , p) |p
p
i=1
N
X A  2
= argmin d˜n (ai , p) − d (ai , p) = p̂LS (n)
p
i=1

 
For p̂M L (n) we can introduce the gaussian distribution for p d˜n (ai , p) |p and
apply the logarithmic function.

(d˜n (ai ,p)−d(ai ,p))2

NΛ
!!
X 1 −
2σ 2
p̂M L (n) = argmax log p e i
p
i=1 2πσi2
   2 
NΛ
! ˜n (ai , p) − d (ai , p)
d
X 1
= argmax log + −
  
2
p
2 2σ

p
i=1 2πσi i

4
 This equation can be simplified by eliminating those parts which are independent
of p. Therefore the logarithmic part and the denominator get dropped.

NΛ  
X 2 
p̂M L (n) = argmax ˜
− dn (ai , p) − d (ai , p)
p
i=1

Since maximizing a negative parameter means to minimize its inverse we can write:

NΛ 
X 2
p̂M L (n) = argmin ˜
dn (ai , p) − d (ai , p) = p̂LS (n)
p
i=1

2.2 Gauss-Newton Algorithm for Position Estimation

(a) Implement the Gauss-Newton algorithm to find the least-squares estimate for the
position. Therefore, write a function least_squares_GN(), which takes the (2 x NA )
anchor positions, the (2 x 1) initial position, the (NA x 1) distance measurements,
the maximum number of iterations, and the chosen tolerance as input. You may
choose suitable values for the tolerance and the maximum number of iterations on
your own. The output is the estimated position.
The Gauss–Newton algorithm is a method to solve non-linear minimization pro-
blems by looking for the least square. It works similar to the multidimensional
gradient decedent but with the advantage of a much faster convergence. For the
implementation the number of max. iterations was chosen as 1000 and the tole-
rance as 0.0001. Additionally a note to the implementation of the Jacobi matrix
is stated below. This derivation is mathematically not correct but should the idea
behind the implementation.

 
∂ d˜n (ai , p) − d (ai , p)
[J(p)]i,1 =
∂x
∂ ((p̃n − panchor ) − (pstart − panchor ))
=
∂x
(p̃n − pstart )
≈
∆d

Where ∆d is the euclidean distance between the measured position of p̃ and the
iterative pstart . Since p̃n is unknown we can not compute the matrix and further

5
 decrease ∆d to bring pstart closer to the real position. Therefore we can try to
maximize d (ai , p) to get closer to the measured distance by using the negative
Jacobi.

 
∂ d˜n (ai , p) − ∆d (ai , p)
[J(p)]i,1 = −
∂x
∂ ((p̃n − panchor ) − (p̃n − pstart ))
=−
∂x
(pstart − panchor ) (pstart − panchor )
≈− =−
dstart−anchor d

Figure 1 shows a sketch of the different distances and points to make this more
clear.

Abbildung 1: Sketch of the various points and distances.

(b) Run your algorithm for each of the N = 2000 independent measurements; choose
the initial guess p̃0 randomly according to a uniform distribution within the square
spanned by the anchor point.
The algorithm was executed 2000 times via the function position_estimation_least_squares,
where the inital guess p̃0 was chosen randomly with the numpy method ran-
dom.uniform according to a uniform distribution, where the borders are declared
within the distribution parameters.

6
 Concerning the mean and variance of the position estimation error, table 2 shows
the absolute values for these error parameters.

Tabelle 2: Error parameters for every scenario.

Error Parameter Scenario 1 Scenario 2 Scenario 3 Scenario 2 mod.

Mean 0.177192600943 0.398958606393 0.77193087035 0.251690644008

Variance 0.018036941189 0.183028031365 0.663392106699 0.0432104232242

After regarding the error specifications, we would like to take a look at the dis-
tribution of the estimated positions. To do this, we plotted a scatter plot of the
estimated positions an added the contour of a gaussian distribution over the scatter
plot.
Figures 2 to 4 show the very scatter plots with overlapped gaussian contours for
each scenario.

Abbildung 2: Scatter plot of the estimated positions for scenario 1.

7
 Abbildung 3: Scatter plot of the estimated positions for scenario 2.

Abbildung 4: Scatter plot of the estimated positions for scenario 3.

As we can observe, the estimated positions follow the gaussian contours, therefore
we can assume that the estimated positions look gaussian.
When we take a look at the cumulative distribution functions of the different
scenarios as shown in figures 5 to 7, we can observe, that the probability of large
estimation errors is low for the gaussian distribution and higher for the exponential
distribution.

8
Abbildung 5: Cumulative distribution function for scenario 1.

Abbildung 6: Cumulative distribution function for scenario 2.

9
 Abbildung 7: Cumulative distribution function for scenario 3.

For further information please take a look at the python file attatched.
(c) Consider scenario 2: the least-squares estimator is equivalent to the maximum
likelihood estimator for Gaussian error distributions. Therefore, neglect the anchor
with the exponentially distributed measurements! Compare your results to before!
What can you observe (Gaussianity of the estimated positions, probability of large
errors,...)?
When we regard Scenario 2 (Measurement with one anchor following the exponen-
tial model and the other anchors following the Gaussian models) and neglect the
anchor with the exponentially distributed measurements, we can see that the per-
formance is better for the second case without the exponential anchor. From table
2 we can see that both the mean and variance of the position estimation errors is
lower for modified Scenario 2 than original Scenario 2. When the estimated positi-
ons are plotted, you can see that the modified Scenario 2 has a better gaussianity
compared to the original Scenario We can also observe that the probability of large
errors is larger for original Scenario 2 than for modified Scenario 2.

2.3 Gauss-Newton Algorithm for Position Estimation

2.3.1 Single Measurement

(a) Evaluate the joint likelihood p(d̃0 (p)|p) over a 2-dimensional grid with a resolution
of 0.05. Confine the evaluation to the square region enclosed by the anchors.

10
For calculation the joint likelihood, we simply have to multiply the single likelihood
functions for each anchor on a 0.05 grid going from x = [-5, 5] and y = [-5, 5].
Finding the maximum with a gradient ascent using a arbitrary starting point within
the evaluation region might be difficult, since the gradient is very discontinuous
and reaches very high values near to infinity. T he maximum is not exactly at the
true position.
However, Plot 8 shows the 3D-Plot of the joint likelihood in a grid of 0.05.

Abbildung 8: 3D-Plot of the joint likelihood.

11