Euclidean Distance Reconstruction from Partial

Distance Information

Jiaming Xu
Department of Electrical and Computer Engineering
jmx@mail.utexas.edu

Yuxin Chen
Department of Electrical and Computer Engineering
yxchen@mail.utexas.edu

1 Introduction
Euclidean distance matrix completion problem aims at reconstructing the low dimensional geometry
structure of nodes given only a few pairwise distances between nodes [1]. This problem arises in
many applications including networks and machine learning where much information of points is
laking. For instance, in sensor networks, due to the constraints of energy and communication radius,
sensor nodes can only measure the pairwise distances accurately between their nearby nodes. As a
consequence, we can only form a partial Euclidean distance matrix. Now, an interesting question is:
given this partial Euclidean distance matrix, can we reconstruct or estimate the Euclidean distance
matrix exactly? Many works are based on heuristic algorithms without rigorous mathematical justi-
fication. However, recent landmark work in low-rank matrix reconstruction [2][3][4] provides some
new directions on attacking this problem. One of the central message of these works is the feasibil-
ity of exact matrix reconstruction by solving a tractable semidefinite programming problem, with a
few entries at hand. We notice that the Euclidean distance matrix is extremely low rank considering
the fact that nodes are located in low-dimensional space (typically of dimensions two or three). In
particular, the rank of the Euclidean distance matrix is at most d + 2, where d is the dimension of
the coordinates of node locations. This fact motivates us to apply the low rank matrix completion
theory to the Euclidean distance matrix completion problem.
In this paper, we are interested in two efficient algorithms. Singular value projection (SVP) al-
gorithm, first proposed in [5], has shown to be a fast and easily-implemented algorithm for rank
minimization problem for affine constraints. For many linear operators that satisfy restricted isom-
etry properties, this algorithm has been proved to converge to true matrix for appropriately chosen
step size. Moreover, this iterated algorithm is robust to noise, i.e. it exhibits a strong geometric
convergence rate even for noisy measurements. All these aspects make SVP a natural candidate
for Euclidean matrix completion problem. Another interesting algorithm for matrix completion is
singular value thresholding (SVT). Instead of directly solving the rank-minimization problem, SVT
attempts to solve the convex relaxation of the original problem, i.e. nuclear norm minimization. Al-
though nuclear norm minimization can be formulated as a semidefinite program that is known to be
solvable within polynomial time, SVT provides an efficient algorithm for solving this special prob-
lem and supports much larger input dimensions and higher convergence rate. For suitable stepsize
and thresholding constant, SVT will converge arbitrarily close to the target matrix, which makes it
useful in Euclidean distance estimation problems.
Some prior works are given as follows. Fazel et. al. [6] use the logarithm of the determinant as a
smooth approximation for rank, and attempts to obtain a sequence of trace minimization problems
from local minimization. Reference [7] provides the first online algorithm to get low-rank approxi-
mation based on the Multiplicative Weights Update framework, but it does not guarantee to get the
exact minimum-rank solution. [4][2] are the first nontrivial work to show potential exact recovery

1
from some observation of random projections or a random subset of entries. However, this requires
the true matrix to obey restricted isometry property [8, 4] or strong incoherence property[2, 3],
which are not typically satisfied for Euclidean distance matrices. Reference [9] provides a scalable
heuristic approach to estimate network distance like round-trip propagation based on geometric co-
ordinates. This is not exactly the minimum rank problem for Euclidean distance matrix, but provides
some connection between network distance and geometric space.

2 Formulation

Assume that there are n points with coordinates x1 , x2 , . . . , xn Rd in the d-dimensional space.
Consider a n n Euclidean distance matrix D = [dij ], where dij denotes the Euclidean distance
between xi and xj . We are given partial distance information between some pairs of nodes, i.e. we
know the distance dij if (i, j) is contained in a subset of [n] [n]. Here, all distance information
at hand are expressed by d , which is a || 1 vector. For any pair of nodes of coordinates
xi , xj Rd , we can have:
dij = hxi xj , xi xj i
= kxi k2 + kxj k2 2xTi xj
  
 I I xi
= xTi xTj

I I xj
= Vec(X)T (ij I) Vec(X) (1)
= ij , XT X


(2)
 T
Here, Vec(X) = xT1 , xT2 , , xTn Rnd , ij = (ei ej )(ei ej )T , signifies the Kronecker
product.
This motivates us to consider G := XT X. It follows from (2) that:
hij , Gi = dij
which yields:
D(G) = diag(G)1T + 1diag(G)T 2G (3)
Therefore, all n n Euclidean distance matrices form a convex cone, which can be expressed as
follows:
EDMn = D(G)|G Sn+

(4)

Without loss of generality, we can assume that the geometric center of x1 , , xn is the origin,
which means:
X1=0 G1 = 0 (5)
Combining this fact with (3), we can get:
 
1 1 T T

1 T T
G= D D11 + 11 D + 2 11 D11 (6)
2 n n
Let V := I n1 11T , then:
1
G = VDV (7)
2
Therefore, we have the following sufficient and necessary condition for EDM:

n VDV Sn+
D EDM (8)
D Snh
where Shn is the set of n n symmetric hollow matrices with main diagonal entries being 0.

2
Now, let us consider the EDM completion problem. This can be formulated as the following low
rank matrix completion problem:
minimize rankVXV (9)
s.t. X, ei eTj = dij


(i, j)
VXV  0
X Snh
For A Sn , using the self-adjointness property of the main-diagonal linear operator, we get:
D E
T
hD, Ai = diag(G)1T + 1diag(G) 2G, A = 2 hG, diag (A1) Ai (10)

It follows from that (7), (9) can be rewritten as follows:

min imizerank G (11)
s.t. G, diag ei eTj 1 + ej eTi 1 ei eTj + ej eTi




= dij (i, j)
G0
Since G is symmetric and positive semidefinite, its nuclear norm is equal to trace. Therefore, we
can consider the convex relaxation for (11) as follows:
minimize trace G
s.t. G, diag ei eTj 1 + ej eTi 1 ei eTj + ej eTi




= dij (i, j) (12)
G0
This is also equivalent to the following program:
minimize kXk2F (13)
s.t. G, diag ei eTj 1 + ej eTi 1 ei eTj + ej eTi




= dij (i, j)
G0
G = XT X
After recovering G, we can immediately obtain D from (3).

3 Singular Value Projection

SVP algorithm was first proposed by Meka et. al. [5] as an efficient algorithm for guaranteed rank
minimization. Following [5], the main step of SVP algorithm for EDM reconstruction can be given
as follows:
(
Yt = Xt A (A (X t ) d )
t+1 (14)
X = PSdn (Yt )
+

where the operator PSdn (Yt ) is the projection of Yt onto the subspace of symmetric positive
+
semidefinite matrices with rank at most d. Here we implement this operation as follows:
X t+1 Ud (d )+ Ud (15)

where Yt is projected to the eigenspace associated with the largest d eigenvalues to generate Xt+1 ,
besides, these d eigenvalues should be guaranteed as non-negative value. This is because Yt is
always a symmetric matrix. Interestingly, when the step size is relatively small and X t is sym-
metric positive semidefinite, Yt is simply obtained from a small perturbation of Xt which means
all negative eigenvalues of Yt is remarkably small. In this case, projecting Yt onto the d largest
singular-vector space is almost the same as projecting onto the d largest eigenspace. Moreover, let
hAk , Xi = dik jk . Rewrite X as a vector Vec (X), then we can easily get a sparse vector Vec (Ak )
T
such that Vec (Ak ) Vec (X) = dik jk . Here, (Vec (Ak ))i = 1 for the elements associated with

3
Xik ik and Xjk jk , and (Vec (Ak ))i = 1 for the elements associated with Xik jk and Xjk ik , and 0
otherwise. Therefore, T
Vec (A1 )
Vec (A2 )T
A (X) = Vec (X) (16)


T
Vec A||
T
Vec (A1 )
Vec (A2 )T
A (b) = Mat b (17)


T
Vec A||
where
 Mat() rearrange the n n
vector  to a n n matrix. We notice that
Vec (A1 ) Vec (A2 ) Vec A|| is a sparse matrix with at most 4 || elements, which
can be used to accelerate computation.
Theorem 1. Suppose that obeys 0 < < 2/L2 (A ). If rank (G) = d and G can be exactly
recovered by (11), then Xt will converge to X , where X = G is the optimizer of (11).

Proof. Let Y be the optimal solution (in fact, it can be seen that Y = X ). Proceeding the
iteration as (14), we can have:
Y F = Xt+1 A A X t+1 d Y + A (A (X ) d ) F
t+1


Y
= Xt+1 Y + A A (X ) A X t+1



F
t+1
By noticing that X = PSdn t
(Y ) and X = PSdn (Y ) = Y , we get:
+ +

2
Y F
t+1
Y
2


= PSdn+ Yt Y + A A X Xt+1


F

d t



2
t+1
= PSn+ Y Y + A A X X
F

2
D
E
2
= PSn+ Y Y 2 PSn+ Y Y , A A X Xt+1 + 2 A A X Xt+1 F

d t d t


F

Moreover, the iteration step yields:

D
E

PSdn+ Yt Y , A A X Xt+1 = Xt+1 X , A A X Xt+1



= A Xt+1 X , A X Xt+1





2
= A Xt+1 X F

Combining the above equations, we get:

2
2

2
2
Y F = PSdn+ Yt Y 2 A Xt+1 X F + 2 A A X Xt+1 F
t+1
Y
F
2
2
PSn+ Yt Y + L2 (A ) 2 A Xt+1 X F
d


F
2


PSdn+ Yt Y

F

This means that Yt+1 will converge to a limiting point Y. Similarly, Xt+1 will converge to X.
They satisfy:  
Y = X A AX d = X
  (18)
X = P dn Y
S +

Therefore, we can see:    

X = PSdn+ Y = PSdn+ X (19)

4
which means X is a positive semidefinite matrix of rank d. Therefore, X = X since G is recover-
able.

Corollary 1. Suppose that obeys 0 < < 2. If the true matrix is recoverable by rank minimization
problem, then Xt will converge to the true matrix in SVP algorithm.

Proof. The proof proceeds with similar spirit by noticing that P = P and L (P ) 1.

4 Singular Value Thresholding

4.1 Algorithm

Singular value thresholding [10] is an efficient algorithm for nuclear norm minimization in matrix
completion problem that can guarantee convergence to the true matrix for appropriately chosen step
size. In order to adapt SVD to EDM reconstruction, we need to modify the algorithm according to
specific convex constraints. Here, we are primarily considering the following optimization problem:

1 2
minimize kXkF + kXk
2

s.t. X, diag ei eTj 1 + ej eTi 1 ei eTj + ej eTi


= dij (i, j) (20)
X0
where (20) reverts to (12) when . The Lagrangian function of this optimization problem can
be written as:
1 2
L (X, y, Z) = kXkF + kXk + hy, d A (X)i hZ, Xi (21)
2
2
where Z  0. Define f (X) = 12 kXkF + kXk , and define gX (y, Z) = L (X, y, Z), then we
have: 
y gX (y, Z) = d A (X)
(22)
Z gX (y, Z) = X
where  
1 2
X = arg min kXkF + kXk + hy, d A (X)i hZ, Xi (23)
X 2
This motivates the following iteration process:
n 
t1 o
t 1 2
t1
X = arg minX 2 kXkF + kXk + y , d A (X) Z , X

yt = yt1 + t (d A (Xt )) (24)

t

t1 t
Z = Z t X +
where ()+ signifies the projection onto symmetric positive semidefinite space. Besides, we notice
that:  
1 2
Xt = arg min kXkF + kXk + A yt1 , X Zt1 , X




(25)
X 2
2
Let h (X) = 12 kXkF + kXk A yt1 + Zt1 , X , then its subgradient:




h (X) = X A yt1 Zt1 + UV + W : U W = 0, WV = 0, kWk 1 (26)



where U and V denote the left and right singular vector matrices of X. Proceeding similar spirit as
matrix completion problem, we can see that:
Xt = D A yt1 + Zt1



(27)
where D denotes the singular value shrinkage operator. Therefore, the SVD algorithm can be
summarized as:

t t1



X = D A y
+ Zt1
yt = yt1 + t (d A (Xt )) (28)
Zt = Zt1 t Xt


+

5
4.2 Convergence Analysis

Lets summarize a lemma in [10] . This lemma establishes the strong convexity of objective function
f , which is the key lemma for proving convergence.
Lemma 1. Let M f (X) and M 0 f (X 0 ). Then we can get

hM M 0 , X X 0 i kX X 0 k2F

Proof. See proof in [10].

Lemma 2. Let (X , y , Z ) be a primal-dual optimal pair, then for each > 0, Z obeys
Z = [Z X ]+ . (29)

Proof. Since Z is dual optimal, we can see:

L (X , y , Z ) L (X , y , Z) Z  0 (30)
Substitute this to the expression of L () we can get:
hZ , X i hZ, X i
hZ Z , X i 0
hZ Z , Z X Z i 0 Z  0, 0
This follows that Z is the projection of Z X onto positive semidefinite cone. Since the
projection of an arbitrary matrix Z onto positive semidefinite cone is given by Z+ , hence
Z = [Z X ]+ . (31)

Theorem 2. Suppose that the step size satisfies 0 < inf k sup k < 2 L2 (A) + 1 , where
L (A) is the Lipshitz constant such that
kA (X1 ) A (X2 )k L (A) kX1 X2 kF (32)

for all X1 and X2 . Then, the sequence Xk converges to the unique solution of (20).

Proof. Suppose (X , y , Z ) is primal-dual optimal, then we claim that for all X,

k
M , X Xk yk1 , A (X) A Xk + Zk1 , X + Xk = 0




(33)

hM , X X i hy , A (X) A (X )i + hZ , X + X i = 0 (34)

for some Mk f Xk and M f (X ). We justify this claim
by proving one of two

inequalities. For
the first inequality, Xk minimizes L X, yk1 , Zk1 over all X, therefore,
Mk f Xk such that
Mk A yk1 Zk1 = 0


(35)
M , X X A yk1 + Zk1 , X Xk = Mk A yk1 Zk1 , X X = 0

k 





which justifies the claim.

Now, we sum up (33) and (34), and set X in (33) to be X and X in (34) to be Xk . This gives:

M k M ? , X X ? + y k1 y ? , A(X ? X k ) + Z k1 Z ? , X ? X k = 0





It follows from Lemma 1 that

k1
y ? , A(X ? X k ) + Z k1 Z ? , X ? X k = M k M ? , X X ? kX k X ? k2F



y
By Lemma 2, we have:

6
 ky k y ? k2 = y k1 + k d A Xk y ? k (d A (X? )) 2

= y k1 y ? + k (A (X? ) A Xk 2

kZ k Z ? kF = [Z k1 k X k ]+ [Z ? k X ? ]+ F

Z k1 Z ? + k (X ? X k ) F

Hence, we have:

ky k y ? k22 + kZ k Z ? k2F
=ky k1 y ? k22 + kZ k1 Z ? k2F + 2k y k1 y ? , A(X ? X k )



+ 2k Z k1 Z ? , X ? X k + k2 A (X? ) A Xk 2F + k2 k(X ? X k )k2F



let k = k , and refer to, we have:

ky k y ? k22 + kZ k Z ? k2F
ky k1 y ? k22 + kZ k1 Z ? k2F 2k kX k X ? k2F + k2 (L (A) 2 + 1)kX k X ? k2F
2
By assumption: k L2 (A)+1 , we have 2k + k2 (L(A)2 + 1) > for all k 1 and some 0.
Then,

ky k y ? k22 + kZ k Z ? k2F
ky k1 y ? k22 + kZ k1 Z ? k2F kX k X ? k2F
Two properties can be observed from this:

(a) The sequence ky k y ? k22 + kZ k Z ? k2F is nonincreasing and therefore converges to a limit
point.
(b) As a consequence, kX k X ? k2F 0 as k .
The proof is complete.

5 Experimental Results and Discussion

This section provides the implementation details of SVP and SVT algorithm such as the selection
of step size k . This section also introduces the experimental results which demonstrates the perfor-
mance of SVP and SVT algorithm for EDM completion.

5.1 Implementation details of SVP and SVT algorithm

The main parameters of SVP and SVT algorithm are step size, the stopping criterion, the maximum
number of iterations , the oversampling parameter and the sampling density .

5.1.1 Step size

Selection of step size is a key problem in implementation of algorithm. If the step size is too large,
the algorithm may diverge, however, if it is too small, the algorithm will converge very slowly. In our
SVP algorithm, for simplicity, we choose the step size to be constant over iterations. By Theorem 1,
convergence of SVP algorithm is guaranteed provided that 0 < k < 2/L2 (A ) and by Theorem 2,

convergence of SVT algorithm is guaranteed provided that 0 < inf k sup k < 2 L2 (A) + 1
. In our experiments , we choose the step size k = 0.5 np . Simulation results show that this choice is
robust to the change of matrix size, rank and sampling density.

7
5.1.2 Stopping criterion
Here, we describe the stopping criterion for SVP and SVT algorithm. We suggest stopping the
algorithm when

kA (A (X t ) d ) kF
< EP S
kd k2

where EP S is the fixed tolerance. In our experiments, we choose EP S = 1 102 .

5.1.3 Maximum number of iterations

Here, in our experiments , we choose the maximum number of iterations M axiter = 2000.

5.1.4 Oversampling parameter and sampling density

In our experiments, we varied the oversampling parameter osp and sampling density spd to see the
performance of SVP algorithm. Oversampling parameter is defined as:

m
osp =
df

where df is the degrees of freedom of EDM. Because EDM is always symmetric, so we have the
eigenvalue decomposition of EDM. Suppose the EDM has rank r, then the degrees of freedom of
EDM is the sum of number of nonzero eigenvalues which is r and degrees of freedom of eigenvectors
which is (n + n 1 + n 2 + n r + 1). Therefore, for EDM of rank r, df = 21 r(2n + 1 r).
m is the number of sampled entries of EDM.
Similarly, the sampling density parameter spd is defined as:

m
spd =
n2

5.2 Experiment results

Our implementation is in Matlab and all the computational results were obtained on the laptop
computer with a 2 GHz CPU and 1 GB of memory. In our simulations, we generate N points of
dimension d uniformly distributed over interval [0, 1] and then form the Gram matrix G of rank d and
EDM D. The set of observed entries is sampled uniformly at random among all sets of cardinality
of m.
Our computational results are displayed in following figures. First observation is that SVP algorithm
outperforms SVT algorithm both in time efficiency and accurate. The reason is twofold. First,
SVP algorithm exploits the prior information of rank and converges faster than SVT algorithm.
Second, SVP algorithm minimizes the rank, while SVT algorithm approximately minimizes the
nuclear norm. The rank minimization is more accurate than the nuclear norm minimization.
Second observation is that with increasing matrix size, the oversampling factor osp increases while
sampling density spd drops. This coincides the theoretical analysis. However, there still exists gap
between the theoretic sampling limit nd log n. Also, for SVP algorithm, with rank increasing, the
computation time and the minimal sampling density increases.

6 Future Work
We have shown that both SVP and SVT algorithms under partial distance constraints can converge
to the minimizer of associated minimization program. However, when these program can exactly
recover the true distance matrix is unknown. In order to guarantee exact recovery, we need to
investigate the minimum requirement for the number of observations, or the specific structure of
coordinate distribution. This is nontrivial since most Euclidean distance matrices dont obey the

8
 Minimum Sampling Density Minimum Oversampling Factor
40

30

Min OSP
Min SPD 0.1
20

10
0.05
0
0 500 1000 0 500 1000
size n size n

Figure 1: The relation of minimum oversampling factor and minimum sampling density with the
size n is plotted using SVP algorithm. Here, all true distance matrices are of rank 2.

Computation Time for SVP

1000

800
Time taken (/sec)

600

400

200

0
0 200 400 600 800 1000 1200
size n

Figure 2: The time required for SVP algorithm to converge with the size n. Here, all true distance
matrices are of rank 2.

Computation Time for SVP Minimum Sampling Factor vs. Rank for SVP
300 0.25

280 0.2
Time taken (/sec)

Min spd

260 0.15

240 0.1

220 0.05

200 0
0 5 10 0 5 10
rank r rank r

Figure 3: The time required for SVP algorithm to converge and minimum sampling density with the
size n.

9
 Minimum Sampling Density Minimum Oversampling Factor
30

0.25
25

0.2 20

Min OSP
Min SPD

15
0.15
10
0.1
5

0.05 0
0 200 400 600 0 200 400 600
size n size n

Figure 4: The relation of minimum oversampling factor and minimum sampling density with the
size n is plotted using SVT algorithm. Here, all true distance matrices are of rank 2.

Computation Time for SVT

1200

1000
Time taken (/sec)

800

600

400

200

0
0 100 200 300 400 500 600
size n

Figure 5: The time required for SVT algorithm to converge with the size n. Here, all true distance
matrices are of rank 2.

10
strong incoherence properties, which play vital roles in most landmark works on matrix completion.
Therefore, how to exploit the structure of Euclidean distance matrices for better reconstruction is
significant for this distance estimation problem.

References
[1] J. Dattorro, Convex Optimization and Euclidean Distance Geometry, 2005.
[2] E. J. Candes and B. Recht, Exact matrix completion via convex optimization, to appear in
Foundations of Computational Mathematics.
[3] E. J. Candes and T. Tao, The power of convex relaxation: Near-optimal matrix completion,
to appear in IEEE Transactions on Information Theory.
[4] M. F. Benjamin Recht and P. A. Parrilo, Guaranteed minimum rank solutions of matrix equa-
tions via nuclear norm minimization, Submitted to SIAM Review, 2007.
[5] R. Meka, P. Jain, and I. S. Dhillon, Guaranteed rank minimization via singular value projec-
tion, 2009. [Online]. Available: http://www.citebase.org/abstract?id=oai:arXiv.org:0909.5457
[6] H. H. Maryam Fazel and S. P. Boyd, Log-det heuristic for matrix rank minimization with
applications to hankel and euclidean distance matrices, American Control Conference, 2003.
[7] R. Meka, P. Jain, C. Caramanis, and I. S. Dhillon, Rank minimization via online learning,
ICML 08: Proceedings of the 25th international conference on Machine learning, pp. 656
663, 2008.
[8] E. Candes and T. Tao, Decoding by linear programming, IEEE Transactions on Information
Theory, vol. 51, no. 12, pp. 42034215, Dec. 2005.
[9] T. Ng and H. Zhang, Predicting internet network distance with coordinates-based ap-
proaches, Twenty-First Annual Joint Conference of the IEEE Computer and Communications
Societies. Proceedings. IEEE, vol. 1, pp. 170179 vol.1, 2002.
[10] J.-F. Cai, E. J. Candes, and Z. Shen, A singular value thresh-
olding algorithm for matrix completion, 2008. [Online]. Available:
http://www.citebase.org/abstract?id=oai:arXiv.org:0810.3286

11