Professional Documents
Culture Documents
Random Projection and Its Applications
Random Projection and Its Applications
net/publication/320857039
CITATION READS
1 734
1 author:
Mahmoud Nabil
North Carolina Agricultural and Technical State University
82 PUBLICATIONS 1,507 CITATIONS
SEE PROFILE
All content following this page was uploaded by Mahmoud Nabil on 28 February 2019.
Abstract—Random Projection is a foundational research topic Another task which frequently involves random projection
that connects a bunch of machine learning algorithms under a when the data dimensionality is high, is nearest neighbor
similar mathematical basis. It is used to reduce the dimensionality search where the target is to return a group of data points
of the dataset by projecting the data points efficiently to a
which are closely related to a given query. One can argue here
arXiv:1710.03163v1 [cs.LG] 9 Oct 2017
Xd d
X
V ar(ui ·v) = V ar( uij vj ) = vj2 V ar(uij ) = |v| = 1
j=1 j=1
2
Lemma 2. For random matrix R ∈ Rp×q where all entries V ar(uT v − xT y) ≤
ri,j are independent and identically chosen from gaussian k
distribution with zero mean and σr2 variance then It can be seen that, the error goes down as k increases. This
implies that at high dimension space, the technique works
E(RT R) = pσr2 I, and E(RRT ) = qσr2 I better.
For privacy analysis. two types of attacks are considered
Proof. lets proof the first inequality. Assume ǫi,j is the entry
1) The adversary tries to retrieve the exact values of the
from RT R then
projected matrix X or Y, the authors proved that when
p
m ≥ 2k − 1, even if matrix R is disclosed the original
X matrices can not be retrieved.
ǫi,j = ri,t rt,j
2) The adversary tries to estimates matrix X or Y, if the
t=1
p distribution of R is known, if the adversary generates R̂
according to the known distribution then
X
E(ǫi,j ) = E( ri,t rt,j )
t=1 1 1 1 1
p √ RˆT u = √ RˆT √ Rx = ǫ̂x
X
k σˆr k σˆr kσr kσ r
= E(ri,t rt,j )
t=1 the estimation of any data element from the vector x is
p
X given by
= E(ri,t )E(rt,j )
t=1 1 X
(P
p
x̂i = ǫˆi,t xt
E(ri,t )E(rt,j ) i 6= j k σˆr σr t
= Ppt=1 2
t=1 E(rt,i ) i=j The expectation and the variance can be calculated as
(
0 i 6= j
= E(x̂i ) = 0
pσr2 i=j
1X 2
Lemma 3. for any two data sets X ∈ Rm1 ×n and Y ∈ V ar(xˆi ) = x
Rm2 ×n , and let random matrix R ∈ Rp×q where all entries k t t
ri,j are independent and identically chosen from unknown So the adversary can only get a null vector centered
distribution with zero mean and σr2 variance, also let U = around the zero.
√ 1 RX,V = √ 1 RY , then
kσr kσ r The authors considered three applications on their paper all of
them relies on the dot product estimation namely: distance
E(U T V ) = X T Y
estimation, k-mean clustering, and linear perceptron. As a
The above results enables the following statistical measure- result, the random projection-based multiplicative perturbation
ments (distance, angle, correlation) to be applied to the hidden technique keeps both the statistical properties and the confi-
data knowing the original vectors are normalized dentiality of the data.
sX V. C ONCLUSION
dist(x, y) = (xi − yi )2 In this paper, we explained the random projection and the
i mathematical foundation behind it. In addition, we explained
some related applications such as compressed sensing which
sX X X p
= x2i + yi2 − 2 xi yi = 2 − 2xT y made a breakthrough in the traditional communication the-
i i i
orems where a very low sampling rate can be used while
xT y the signal construction is achievable. Also, we explained the
cosθ = = xT y matrix completion problem that is a basis for many data
|x|.|y|
mining tasks such as recommendation systems and image in-
ρx,y = xT y painting algorithms.
Thus, the number of attributes of the data can be reduced by
random projection and the statistical dependencies among the R EFERENCES
observations will be maintained. It is worth to mention that, [1] G. H. Golub and C. F. Van Loan, Matrix computations. JHU Press,
given only the projected data U or V , original data can not 2012, vol. 3.
be retrieved as the number of possible solutions are infinite. [2] E. Bingham and H. Mannila, “Random projection in dimensionality
reduction: applications to image and text data,” in Proceedings of
For error analysis, it can easily be proven that the mean the seventh ACM SIGKDD international conference on Knowledge
difference and the variance difference between the projected discovery and data mining. ACM, 2001, pp. 245–250.
and the original data are given as [3] K. Liu, H. Kargupta, and J. Ryan, “Random projection-based multiplica-
tive data perturbation for privacy preserving distributed data mining,”
IEEE Transactions on knowledge and Data Engineering, vol. 18, no. 1,
E(uT v − xT y) = 0 pp. 92–106, 2006.
6
http://arxiv.org/ps/1710.03163v1
This figure "compressed-sensing.jpg" is available in "jpg" format from:
http://arxiv.org/ps/1710.03163v1
This figure "matrixcompletion.png" is available in "png" format from:
http://arxiv.org/ps/1710.03163v1