Maximal entropy random walk

Maximal entropy random walk (MERW) is a popular type of biased random walk on a graph, in which transition
probabilities are chosen accordingly to the principle of maximum entropy, which says that the probability distribution which
best represents the current state of knowledge is the one with largest entropy. While standard random walk chooses for
every vertex uniform probability distribution among its outgoing edges, locally maximizing entropy rate, MERW maximizes
it globally (average entropy production) by assuming uniform probability distribution among all paths in a given graph.

MERW is used in various fields of science. A direct application is choosing probabilities to maximize transmission rate
through a constrained channel, analogously to Fibonacci coding. Its properties also made it useful for example in analysis of
complex networks,[1] like link prediction,[2] community detection,[3] robust transport over networks[4] and centrality
measures.[5] Also in image analysis, for example for detecting visual saliency regions,[6] object localization,[7] tampering
detection[8] or tractography problem.[9]

Additionally, it recreates some properties of quantum mechanics, suggesting a way to repair the discrepancy between
diffusion models and quantum predictions, like Anderson localization.[10]

Basic model
Consider a graph with
vertices, defined by an
adjacency matrix
:
if there is an edge from
vertex to , 0 otherwise.
For simplicity assume it is
an undirected graph,
which corresponds to a
symmetric ; however,
MERW can also be
generalized for directed
and weighted graphs (for
example Boltzmann
distribution among paths Left: basic concept of the generic random walk (GRW) and maximal entropy random walk (MERW)
instead of uniform). Right: example of their evolution on the same inhomogeneous 2D lattice with cyclic boundary
conditions – probability density after 10, 100 and 1000 steps while starting from the same vertex.
We would like to choose a The small boxes represent defects: all vertices but the marked ones have additional self-loop
random walk as a Markov (edge to itself). For regular lattices (no defects), GRW and MERW are identical. While defects do
process on this graph: for not strongly affect the local beha­vior, they lead to a completely different global stationary
every vertex and its probability here. While GRW (and based on it standard diffusion) leads to nearly uniform stationary
outgoing edge to , choose density, MERW has strong localization property, imprisoning the walkers in entropic wells in
probability of the analogy to electrons in defected lattice of semi-conductor.
walker randomly using this
edge after visiting .
Formally, find a stochastic matrix (containing the transition probabilities of a Markov chain) such that

for all and

for all .

Assuming this graph is connected and not periodic, ergodic theory says that evolution of this stochastic process leads to
some stationary probability distribution such that .
Using Shannon entropy for every vertex and averaging over probability of visiting this vertex (to be able to use its entropy),
we get the following formula for average entropy production (entropy rate) of the stochastic process:

This definition turns out to be equivalent to the asymptotic average entropy (per length) of the probability distribution in the
space of paths for this stochastic process.

In the standard random walk, referred to here as generic random walk (GRW), we naturally choose that each outgoing edge
is equally probable:

For a symmetric it leads to a stationary probability distribution with

It locally maximizes entropy production (uncertainty) for every vertex, but usually leads to a suboptimal averaged global
entropy rate .

MERW chooses the stochastic matrix which maximizes , or equivalently assumes uniform probability distribution
among all paths in a given graph. Its formula is obtained by first calculating the dominant eigenvalue and corresponding
eigenvector of the adjacency matrix, i.e. the largest with corresponding such that . Then
stochastic matrix and stationary probability distribution are given by

for which every possible path of length from the -th to -th vertex has probability

Its entropy rate is and the stationary probability distribution is

In contrast to GRW, the MERW transition probabilities generally depend on the structure of the entire graph (are nonlocal).
Hence, they should not be imagined as directly applied by the walker – if random-looking decisions are made based on the
local situation, like a person would make, the GRW approach is more appropriate. MERW is based on the principle of
maximum entropy, making it the safest assumption when we don't have any additional knowledge about the system. For
example, it would be appropriate for modelling our knowledge about an object performing some complex dynamics – not
necessarily random, like a particle.

Sketch of derivation

Assume for simplicity that the considered graph is indirected, connected and aperiodic, allowing to conclude from the
Perron–Frobenius theorem that the dominant eigenvector is unique. Hence can be asymptotically ( )
approximated by (or in bra–ket notation).
MERW requires uniform distribution along paths. The number of paths with length and vertex in the center is

hence for all ,

Analogously calculating probability distribution for two succeeding vertices, one obtains that the probability of being at the
-th vertex and next at the -th vertex is

Dividing by the probability of being at the -th vertex, i.e. , gives for the conditional probability of the -th vertex
being next after the -th vertex

Weighted MERW: Boltzmann path ensemble

We have assumed that for MERW corresponding to uniform ensemble among paths. However, the above
derivation works for real nonnegative . Parametrizing and asking for probability of length path
, we get:

As in Boltzmann distribution of paths for energy defined as sum of over given path. For example it allows to calculate
probability distribution of patterns in Ising model.

Examples
Let us first look at a simple nontrivial situation: Fibonacci coding, where we want to transmit a message as a sequence of 0s
and 1s, but not using two successive 1s: after a 1 there has to be a 0. To maximize the amount of information transmitted in
such sequence, we should assume uniform probability distribution in the space of all possible sequences fulfilling this
constraint. To practically use such long sequences, after 1 we have to use 0, but there remains a freedom of choosing the
probability of 0 after 0. Let us denote this probability by , then entropy coding would allow encoding a message using this
chosen probability distribution. The stationary probability distribution of symbols for a given turns out to be
. Hence, entropy production is ,
which is maximized for , known as the golden ratio. In contrast, standard random walk would
choose suboptimal . While choosing larger reduces the amount of information produced after 0, it also reduces
frequency of 1, after which we cannot write any information.

A more complex example is the defected one-dimensional cyclic lattice: let say 1000 nodes connected in a ring, for which
all nodes but the defects have a self-loop (edge to itself). In standard random walk (GRW) the stationary probability
distribution would have defect probability being 2/3 of probability of the non-defect vertices – there is nearly no
localization, also analogously for standard diffusion, which is
infinitesimal limit of GRW. For MERW we have to first find the
dominant eigenvector of the adjacency matrix – maximizing in:

for all positions , where for defects, 0 otherwise.

Substituting and multiplying the equation by −1 we get:

Left: choosing the optimal probability after symbol 0 in

where is minimized now, becoming the analog of
energy. The formula inside the bracket is discrete Laplace operator,
making this equation a discrete analogue of stationary Schrodinger
equation. As in quantum mechanics, MERW predicts that the
probability distribution should lead exactly to the one of quantum
ground state: with its strongly localized density (in
contrast to standard diffusion). Taking the infinitesimal limit, we can
get standard continuous stationary (time-independent) Schrodinger
equation ( for ) here.[11]
Fibonacci coding. Right: one-dimensional defected
lattice and its stationary density for length 1000 cycle
(it has three defects). While in standard random walk
the stationary density is proportional to degree of a
vertex, leading to 3/2 difference here, in MERW
density is nearly completely localized in the largest
defect-free region, analogous to the ground state
predicted by quantum mechanics.

See also
Principle of maximum entropy
Eigenvector centrality
Markov chain
Anderson localization

References
1. Sinatra, Roberta; Gómez-Gardeñes, Jesús; Lambiotte, Renaud; Nicosia, Vincenzo; Latora, Vito (2011).
"Maximal-entropy random walks in complex networks with limited information" (http://www.robertasinatra.co
m/pdf/sinatra_maximal_entropy.pdf) (PDF). Physical Review E. 83 (3): 030103. arXiv:1007.4936 (https://arx
iv.org/abs/1007.4936). Bibcode:2011PhRvE..83c0103S (https://ui.adsabs.harvard.edu/abs/2011PhRvE..83
c0103S). doi:10.1103/PhysRevE.83.030103 (https://doi.org/10.1103%2FPhysRevE.83.030103).
ISSN 1539-3755 (https://www.worldcat.org/issn/1539-3755). PMID 21517435 (https://pubmed.ncbi.nlm.nih.
gov/21517435). S2CID 6984660 (https://api.semanticscholar.org/CorpusID:6984660).
2. Li, Rong-Hua; Yu, Jeffrey Xu; Liu, Jianquan (2011). Link prediction: the power of maximal entropy random
walk (https://web.archive.org/web/20170212090812/https://pdfs.semanticscholar.org/f185/bc2499be95b43
12169a7e722bac570c2d509.pdf) (PDF). Association for Computing Machinery Conference on Information
and Knowledge Management (http://www.cikm2011.org/). p. 1147. doi:10.1145/2063576.2063741 (https://d
oi.org/10.1145%2F2063576.2063741). S2CID 15309519 (https://api.semanticscholar.org/CorpusID:15309
519). Archived from the original (https://pdfs.semanticscholar.org/f185/bc2499be95b4312169a7e722bac57
0c2d509.pdf) (PDF) on 12 February 2017.
3. Ochab, J.K.; Burda, Z. (2013). "Maximal entropy random walk in community detection". The European
Physical Journal Special Topics. 216 (1): 73–81. arXiv:1208.3688 (https://arxiv.org/abs/1208.3688).
Bibcode:2013EPJST.216...73O (https://ui.adsabs.harvard.edu/abs/2013EPJST.216...73O).
doi:10.1140/epjst/e2013-01730-6 (https://doi.org/10.1140%2Fepjst%2Fe2013-01730-6). ISSN 1951-6355
(https://www.worldcat.org/issn/1951-6355). S2CID 56409069 (https://api.semanticscholar.org/CorpusID:564
09069).
4. Chen, Y.; Georgiou, T.T.; Pavon, M.; Tannenbaum, A. (2016). "Robust transport over networks" (https://www.
ncbi.nlm.nih.gov/pmc/articles/PMC5600536). IEEE Transactions on Automatic Control. 62 (9): 4675–4682.
arXiv:1603.08129 (https://arxiv.org/abs/1603.08129). Bibcode:2016arXiv160308129C (https://ui.adsabs.har
vard.edu/abs/2016arXiv160308129C). doi:10.1109/TAC.2016.2626796 (https://doi.org/10.1109%2FTAC.2
016.2626796). PMC 5600536 (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5600536). PMID 28924302
(https://pubmed.ncbi.nlm.nih.gov/28924302).
 5. Delvenne, Jean-Charles; Libert, Anne-Sophie (2011). "Centrality measures and thermodynamic formalism
for complex networks". Physical Review E. 83 (4): 046117. arXiv:0710.3972 (https://arxiv.org/abs/0710.397
2). Bibcode:2011PhRvE..83d6117D (https://ui.adsabs.harvard.edu/abs/2011PhRvE..83d6117D).
doi:10.1103/PhysRevE.83.046117 (https://doi.org/10.1103%2FPhysRevE.83.046117). ISSN 1539-3755 (ht
tps://www.worldcat.org/issn/1539-3755). PMID 21599250 (https://pubmed.ncbi.nlm.nih.gov/21599250).
S2CID 25816198 (https://api.semanticscholar.org/CorpusID:25816198).
6. Jin-Gang Yu; Ji Zhao; Jinwen Tian; Yihua Tan (2014). "Maximal Entropy Random Walk for Region-Based
Visual Saliency". IEEE Transactions on Cybernetics. Institute of Electrical and Electronics Engineers
(IEEE). 44 (9): 1661–1672. doi:10.1109/tcyb.2013.2292054 (https://doi.org/10.1109%2Ftcyb.2013.229205
4). ISSN 2168-2267 (https://www.worldcat.org/issn/2168-2267). PMID 25137693 (https://pubmed.ncbi.nlm.
nih.gov/25137693). S2CID 20962642 (https://api.semanticscholar.org/CorpusID:20962642).
7. L. Wang, J. Zhao, X. Hu, J. Lu, Weakly supervised object localization via maximal entropy random walk (htt
ps://ieeexplore.ieee.org/abstract/document/6678551/), ICIP, 2014.
8. Korus, Pawel; Huang, Jiwu (2016). "Improved Tampering Localization in Digital Image Forensics Based on
Maximal Entropy Random Walk". IEEE Signal Processing Letters. Institute of Electrical and Electronics
Engineers (IEEE). 23 (1): 169–173. Bibcode:2016ISPL...23..169K (https://ui.adsabs.harvard.edu/abs/2016I
SPL...23..169K). doi:10.1109/lsp.2015.2507598 (https://doi.org/10.1109%2Flsp.2015.2507598).
ISSN 1070-9908 (https://www.worldcat.org/issn/1070-9908). S2CID 16305991 (https://api.semanticscholar.
org/CorpusID:16305991).
9. Galinsky, Vitaly L.; Frank, Lawrence R. (2015). "Simultaneous Multi-Scale Diffusion Estimation and
Tractography Guided by Entropy Spectrum Pathways" (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC441
7445). IEEE Transactions on Medical Imaging. Institute of Electrical and Electronics Engineers (IEEE). 34
(5): 1177–1193. doi:10.1109/tmi.2014.2380812 (https://doi.org/10.1109%2Ftmi.2014.2380812).
ISSN 0278-0062 (https://www.worldcat.org/issn/0278-0062). PMC 4417445 (https://www.ncbi.nlm.nih.gov/p
mc/articles/PMC4417445). PMID 25532167 (https://pubmed.ncbi.nlm.nih.gov/25532167).
10. Burda, Z.; Duda, J.; Luck, J. M.; Waclaw, B. (23 April 2009). "Localization of the Maximal Entropy Random
Walk". Physical Review Letters. 102 (16): 160602. arXiv:0810.4113 (https://arxiv.org/abs/0810.4113).
Bibcode:2009PhRvL.102p0602B (https://ui.adsabs.harvard.edu/abs/2009PhRvL.102p0602B).
doi:10.1103/physrevlett.102.160602 (https://doi.org/10.1103%2Fphysrevlett.102.160602). ISSN 0031-9007
(https://www.worldcat.org/issn/0031-9007). PMID 19518691 (https://pubmed.ncbi.nlm.nih.gov/19518691).
S2CID 32134048 (https://api.semanticscholar.org/CorpusID:32134048).
11. J. Duda, Extended Maximal Entropy Random Walk (http://www.fais.uj.edu.pl/documents/41628/d63bc0b7-c
b71-4eba-8a5a-d974256fd065), PhD Thesis, 2012.

External links
Gábor Simonyi, Y. Lin, Z. Zhang, "Mean first-passage time for maximal-entropy random walks in complex
networks" (http://www.nature.com/articles/srep05365). Scientific Reports, 2014.
Electron Conductance Models Using Maximal Entropy Random Walks (http://demonstrations.wolfram.com/
ElectronConductanceModelsUsingMaximalEntropyRandomWalks/) Wolfram Demonstration Project

Retrieved from "https://en.wikipedia.org/w/index.php?title=Maximal_entropy_random_walk&oldid=1163638579"