Professional Documents
Culture Documents
Full download Dimensions of Uncertainty in Communication Engineering Ezio Biglieri file pdf all chapter on 2024
Full download Dimensions of Uncertainty in Communication Engineering Ezio Biglieri file pdf all chapter on 2024
https://ebookmass.com/product/uncertainty-quantification-in-
multiscale-materials-modeling-mcdowell/
https://ebookmass.com/product/gate-2019-electronics-and-
communication-engineering-trishna-knowledge-systems/
https://ebookmass.com/product/aesthetic-dimensions-of-modern-
philosophy-andrew-bowie/
https://ebookmass.com/product/new-dimensions-in-production-and-
utilization-of-hydrogen-1st-edition-sonil-nanda/
Work, Retire, Repeat: The Uncertainty of Retirement in
the New Economy Ghilarducci
https://ebookmass.com/product/work-retire-repeat-the-uncertainty-
of-retirement-in-the-new-economy-ghilarducci/
https://ebookmass.com/product/communication-in-everyday-life-a-
survey-of-communication-3rd-edition-ebook-pdf/
https://ebookmass.com/product/tahriib-journeys-into-the-unknown-
an-ethnography-of-uncertainty-in-migration-anja-simonsen/
https://ebookmass.com/product/visions-of-the-buddha-creative-
dimensions-of-early-buddhist-scripture-eviatar-shulman/
https://ebookmass.com/product/the-importance-of-work-in-an-age-
of-uncertainty-the-eroding-work-experience-in-america-david-l-
blustein/
Dimensions of Uncertainty in
Communication Engineering
This page intentionally left blank
Dimensions of
Uncertainty in
Communication
Engineering
Ezio Biglieri
Academic Press is an imprint of Elsevier
125 London Wall, London EC2Y 5AS, United Kingdom
525 B Street, Suite 1650, San Diego, CA 92101, United States
50 Hampshire Street, 5th Floor, Cambridge, MA 02139, United States
The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, United Kingdom
Copyright © 2022 Elsevier Inc. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or by any means, electronic or
mechanical, including photocopying, recording, or any information storage and retrieval system, without
permission in writing from the publisher. Details on how to seek permission, further information about
the Publisher’s permissions policies and our arrangements with organizations such as the Copyright
Clearance Center and the Copyright Licensing Agency, can be found at our website:
www.elsevier.com/permissions.
This book and the individual contributions contained in it are protected under copyright by the Publisher
(other than as may be noted herein).
Notices
Knowledge and best practice in this field are constantly changing. As new research and experience
broaden our understanding, changes in research methods, professional practices, or medical treatment
may become necessary.
Practitioners and researchers must always rely on their own experience and knowledge in evaluating and
using any information, methods, compounds, or experiments described herein. In using such
information or methods they should be mindful of their own safety and the safety of others, including
parties for whom they have a professional responsibility.
To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume
any liability for any injury and/or damage to persons or property as a matter of products liability,
negligence or otherwise, or from any use or operation of any methods, products, instructions, or ideas
contained in the material herein.
ISBN: 978-0-323-99275-6
Preface xi
Notations and symbols xv
Acronyms and abbreviations xvii
1 Model selection 1
1.1 Parametric models 1
1.2 Wireless channel models 2
1.2.1 Rayleigh fading 2
1.2.2 Rice fading 3
1.2.3 Nakagami-m fading 3
1.3 Parameter estimation 4
1.3.1 A word of caution 8
1.4 Differential entropy and Kullback–Leibler divergence 9
1.4.1 Differential entropy 9
1.4.2 Kullback–Leibler divergence 10
1.5 KKT optimality conditions 13
1.6 Choosing the best model: the maximum-entropy principle 15
1.6.1 Maximizing entropy with order-statistics constraints 19
1.6.2 Spherically invariant processes 20
1.7 Choosing the best model in a set: Akaike Information Criterion 22
1.8 Choosing the best model in a set: minimum description length
criterion 24
Sources and parerga 24
3 Moment bounds 49
3.1 Some classical results 49
3.2 The general moment-bound problem 52
3.3 Calculation of moments 54
3.3.1 Using multinomial expansions 55
3.3.2 Using moment-generating functions 55
3.3.3 Using recursive computations 56
3.4 Moments of unimodal pdfs 57
3.4.1 Moment transfer 58
3.5 When is a sequence a valid moment sequence? 59
3.5.1 Probability measures represented by moments 60
3.6 Moments in a parallelepiped 63
3.7 Geometric bounds 64
3.7.1 Two-dimensional case 67
3.7.2 Application to spectrum sensing 69
3.8 Quadrature-rule approximations and bounds 73
3.8.1 The algebraic moment problem 73
3.8.2 Čebyšev form of orthogonal polynomials 75
3.8.3 Recursive generation of orthogonal polynomials 78
3.8.4 Numerical generation of orthogonal polynomials 79
3.9 Čebyšev systems and principal representations 82
3.9.1 Principal and canonical representations as quadrature rules 86
3.9.2 Moment bounds and quadrature rules 90
3.9.3 Bounds on CDFs and quadrature rules 92
3.9.4 Bounds on Laplace–Stieltjes transform 93
3.9.5 Extension to nondifferentiable functions: the method of
contact polynomials 95
3.10 Moment problems as semidefinite programs 98
3.10.1 Moment problem in a parallelepiped 99
3.10.2 Generalized Čebyšev bounds 100
3.11 Multidimensional moment bounds and approximations 101
3.11.1 Multidimensional moment bounds 102
Sources and parerga 110
Bibliography 253
Index 269
Preface
As we know, there are known knowns; there are things we know we know.
We also know there are unknowns;
that is to say, we know there are some things we do not know.
But there are also unknown unknowns—the ones we don’t know we don’t know.
Donald Rumsfeld, 2002
randomness. More recent research involving wireless channels has realized that the
impact of epistemic uncertainty on performance analysis could not be neglected.
The notion of distinguishing epistemic from aleatory uncertainty is far from new.
Original probability theory was rooted in the analysis of games of chance (Gero-
lamo Cardano, Blaise Pascal, and Pierre De Fermat), and hence was based on purely
aleatory uncertainty. Later, Pascal presented his famous wager, observing that, since
either God exists or not, one can either wager for God or against Him, with widely
different outcomes. This was a decision made under purely epistemic uncertainty, due
to the missing knowledge about the actual existence of God.
Purely aleatory uncertainty assumes full confidence in the probability distribution
model chosen, and implies “decisions under risk.” Purely epistemic uncertainty is nat-
urally measured by confidence, or by subjective knowledge [88, p. 9]. In [263], the
distinction between epistemic and aleatory uncertainty is made from the viewpoint
of financial investing, where an agent is expected to simultaneously seek maximum
expected returns and minimum return variability. Here, distinct investment strategies
reflect the relative weights assigned to epistemic and aleatory uncertainty. Since epis-
temic uncertainty results from incomplete knowledge, viewing the market’s nature as
purely epistemic will make an investor try to reduce uncertainty by seeking the max-
imum available amount of information before making investment choices. Investors
who view the market as purely aleatory, and hence inherently unpredictable, will try
to reduce variability using strategies like asset diversification (see also our discussion
of decisions under risk and agent attitudes in Chapter 7). More generally, one might
find himself in a situation where aleatory and epistemic uncertainty coexist. Consider
the example of an urn A containing 50 blue balls and 50 red balls, and an urn B con-
taining 100 red and blue balls in an unknown proportion. Consider a bet in which an
agent draws a ball from one of the urns, and receives a prize if the ball is red. Draws
from urn A are affected by pure aleatory uncertainty, while draws from urn B are
affected by a mixture of aleatory and epistemic uncertainty.
Mathematical tools useful for dealing with epistemic uncertainty are the subject of the
present monograph. My leitmotiv here is that one should (i) avoid performance anal-
yses relying on unwarranted statistical assumptions, and at the same time (ii) evaluate
the cost of the inaccuracy of a physical model. Citing verbatim from [81, p. 112],
“it is better to have a correct analysis that honestly distinguishes between variability
and incertitude than an analysis that depends on unjustified assumptions and wishful
thinking. (· · · ) If the price of a correct assessment is broad uncertainty as a recognition
or admission of limitations in our scientific knowledge, then we must pay that price.”
The “law of decreasing credibility,” formulated by C.F. Manski [166, p. 1] states
that “the credibility of inference decreases with the strength of the assumptions main-
tained,” and brings explicitly into the argument the reliability of conclusions, or per-
formance evaluations, derived from assumptions lacking a strong support. One should
Preface xiii
keep in mind Bertrand Russell’s quip about unjustified assumptions: “The method of
‘postulating’ what we want has many advantages; they are the same as the advantages
of theft over honest toil” [215, p. 71], and the celebrated phrase used by Isaac Newton
in his essay “General Scholium” and appended to the second (1713) edition of the
Principia: “Hypotheses non fingo” (Latin for “I do not feign hypotheses,” or “I con-
trive no hypotheses”) [185, p. 589]. For example, Chapter 6 will describe how wrong
assumptions about the dependence structure of random variables may heavily affect
the accuracy of system performance. Thus, one should accept that, if calculations are
known to be affected by epistemic uncertainty, at least an analysis of the sensitivity
of the final results to the inaccurate assumptions is called for. One way of proceeding,
as followed here, is to derive bounds to system performance that reflect the amount
and the quality of epistemic uncertainty. Wide bounds should guide the search for
ways to reduce uncertainty, while tight bounds indicate that the influence of epistemic
uncertainty is limited.
Citing verbatim from [143, p. 9], “Our choice of uncertainty theory for dealing
with each given problem should be determined solely by the nature of the problem.
The chosen theory should allow us to express fully our ignorance and, at the same
time, it should not allow us to ignore any available information.” Since theories of
epistemic uncertainty were developed across different fields, they took various forms
and resulted into a fragmented terminology. Several mathematical structures are avail-
able: these include interval analysis, possibility theory, imprecise probability theory,
random-set theory, fuzzy measure theory [278], Dempster–Shafer theory [229], and
evidence theory [114, p. 605] (relations among these theories, and some equivalence
results, are discussed, for example, in [8,163,207,272]). While a few of these tech-
niques will be studied, it is beyond the scope of this book to pursue an extensive
comparison of their merits. One related topic that will not be covered in this book is
Robust Statistics, which carries an intimate relationship with uncertainty theories. As
observed in [119, p. 1], simplifications are often justified by appealing to a sort of
continuity principle, which assumes that a minor error in mathematical models causes
a small error in performance prediction. Robust Statistics theory collects procedures
aimed at reducing the sensitivity of a system behavior to errors caused by epistemic
uncertainty, like modeling errors.
Although my selection of material reflects its focus on communication engineering,
the techniques I describe are also of interest in broader contexts involving statistics and
computational probability theory. The reader will realize that those techniques reflect
research work done in disparate areas, sometimes having very little in common. Given
the importance to other applications that many topics covered by this book have, ex-
tensive commonalities with areas like transportation theory and financial mathematics
are often pointed at through a number of references to research done in those areas.
Each chapter is devoted to a topic which could be easily expanded to book length.
To avoid writing an omnium-gatherum, only the essential aspects of each topic are
highlighted, sometimes at the price of loss of rigor. Inevitable omissions and meager
coverage of certain important topics, caused by space constraints, are partly com-
pensated by the generous use of bibliographical references to previous related work,
which can be used for deepening the treatment, widening the range of its applications,
xiv Preface
and giving the reader a view of how different contributions are related both in time and
context. Some references, placed in the “Further reading” section of bibliography, are
not cited in the text but could be useful for additional reading. The “Sources and Par-
erga” section at the end of each chapter is intended as a warehouse storing the pieces
that were left out of the construction of the main text.
Audience
The material here does not demand any great mathematical knowledge. It only as-
sumes a firm grasp of the fundamental concepts of probability, information, and
communication theories as presented in senior-level college courses. Elementary no-
tions of optimization theory may be useful to follow some arguments in a few sections
of Chapters 1 to 3. Although readers would probably benefit the most by reading this
book from cover to cover, I tried to keep each chapter standing on its own, so that
individual chapters could be read separately, with no need to work through the entire
text. The reader will realize that, due to the variety of topics covered, the introduction
of proofs of each individual statement would require an unreasonable expansion of the
mathematical background needed to appreciate their details. Thus, although I loathe
(especially in textbooks) the expression “it can be proved that . . . ,” I could not avoid
using it often, lest the size of the treatment was expanded beyond control. In exchange,
I provide abundant references to papers and journals where the interested reader can
find the details that were omitted, or delve deeper (after all, as Joseph Conrad wrote
in 1897 in a letter to Cunninghame Graham, “One writes only half the book; the other
half is with the reader”).
This book is meant for researchers and scientists working in communication engi-
neering and for graduate students at any level. It may also be useful as a supplementary
textbook in courses devoted to communication engineering or information theory.
Tabula gratulatoria
I have been developing the material included in this book for about a decade. Much
of it includes work I have done along the years with several coauthors, from whom
I have learned a good deal about the topics covered here. Among them I want to
thank Nabil Alrajeh, Daniele Angelosante, Emanuele Grossi, I-Wei Lai, Marco Lops,
Giorgio Taricco, Emanuele Viterbo, Cheng-An Yang, and especially Kung Yao. I also
thank Anthony Ephremides and Sergio Verdú for a friendship rooted in scholarship
and truth.
Ezio Biglieri
Universitat Pompeu Fabra,
Barcelona, Spain
December 2021
Notations and symbols
1. The set of real numbers is denoted by R; R+ and N denote the sets of nonnegative real numbers and
nonnegative integers, respectively; R denotes the extended real line R ∪ {−∞, ∞}.
2. log denotes the natural logarithm.
3. denotes the real part of a complex number.
4. var(X) denotes the variance of the random variable X, and cov(X, Y ) the covariance of the pair of
random variables X, Y .
5. If A is a set, Ac denotes its complement and |A| its cardinality; 1A denotes its indicator (or char-
acteristic) function, viz., 1A (x) = 1 if x ∈ A, and 1A (x) = 0 if x ∈ / A; 2A denotes the power set of
A, i.e., the family of all subsets of A. If A and B are subsets of X, A − B denotes their difference
{x ∈ X | x ∈ A, x ∈ / B}.
6. The notation [a, b] denotes a closed interval, and (a, b) an open interval; I denotes the interval [0, 1].
7. The symbol ∼ denotes asymptotic equality, ≈ approximate equality (in numerical value), and equality
by definition.
8. Lower-case boldface symbols denote vectors, and capital boldface symbols denote matrices. In particu-
lar, I denotes the identity matrix, and 0 the matrix or vector, all of whose entries are zero. The superscript
T denotes the transpose of a vector or matrix; A 0 means that matrix A is nonnegative definite.
9. The support X of a random variable X is a set such that X ∈ X with probability one; X is also called
the support set of the probability measure, of the probability density function, and of the cumulative
distribution function of X.
10. We write X ∼ N(μ, σ 2 ) to indicate that the random variable X has a Gaussian distribution with mean
μ and variance σ 2 , X ∼ N(μ, R) to indicate that X is a Gaussian random vector with mean μ and
covariance matrix R, with Nc indicating that X is a complex random variable. We write X ∼ U[a, b]
to indicate that X has a uniform distribution with support set [a, b], and X ∼ G(α, β) to indicate that X
has a Gamma distribution with parameters α, β.
11. The symbol P denotes probability measure, and E expectation.
12. The cumulative distribution function associated with P is denoted by F , and the probability density
function, assumed to exist, is denoted by f . If convenient, when we refer to the probability measure of
a random variable X, we specify its cumulative distribution function and probability density function
writing FX (x) and fX (x), respectively. The function F ← denotes the quasi-inverse of the cumulative
distribution function F .
13. δi,j denotes the Kronecker delta function, taking value 1 if i = j and 0 otherwise; u(x) denotes the
unit-step function, taking value 1 if x > 0, and 0 otherwise; sgn(x) denotes the signum function, taking
value −1 if x < 0, 0 if x = 0, and 1 if x > 0.
14. X denotes the empirical mean of the random variable X.
15. xn = O[g(n)] means that a positive constant M exists such that |xn | M · |g(n)| for all sufficiently
large n.
This page intentionally left blank
Acronyms and abbreviations
MADAME SHUM. — Do, do, l’enfant do… Bon, le voilà parti… (Elle
pousse un profond soupir.) Oui, dors, pauvre enfant, fils d’une mère
infortunée et d’un père anonyme quant à la profession…
MOI, à part. — Vieille folle !
MARY. — Maman, ne dites plus de mal de Frédéric, il m’adore.
MADAME SHUM, avec ironie. — Ah, c’est juste !… Il vous a donné un
beau châle hier ; mais avec quel argent l’a-t-il acheté, ce châle ?
voilà la question… Qui est-il ? Que fait-il ?… Plaise à Dieu que vous
n’ayez pas épousé un assassin !… Mary, j’en ai l’intime conviction,
votre mari est un affreux bandit.
I
JE COUPE, ATOUT ET ATOUT
Cinqpoints était avocat, bien qu’il n’eût jamais plaidé une cause
ni parcouru le plus mince dossier. Il attendait avec patience que les
whigs, alors au pouvoir, voulussent bien créer à son intention
quelque bonne petite sinécure. Son père avait débuté sur la scène
politique dans un rôle de libéral enragé ; mais, depuis lors, il avait
toujours suffi d’une crise ministérielle pour lui faire changer
d’opinion. N’étant pas riche, lord Crabs se voyait forcé de voter
tantôt blanc, tantôt noir, afin de pouvoir soutenir la dignité de son
rang et obtenir des places lucratives pour messieurs ses fils.
— Il n’est pas facile, remarquait plaisamment cet aimable
vieillard, d’être bon pair, lorsqu’on a beaucoup d’enfants et
beaucoup de dettes.
Le bruit courait que le comte de Crabs nous servait une pension
de dix mille francs par an. C’était fort généreux de la part d’un
homme qui, tant de fois déjà, avait fait à sa famille le sacrifice de ses
opinions politiques ; seulement, j’ai tout lieu de croire que mon
maître, bien qu’il fût trop bon fils pour démentir cette rumeur, ne
touchait que bien rarement la rente paternelle. Cependant il ne
manquait jamais d’argent ; car les gens comme il faut ont mille
manières de subvenir à leurs dépenses dont la vile multitude ne se
doute pas.
On voyait dans son salon une longue pancarte où les noms de
ses ancêtres se lisaient en lettres rouges sur les branches d’un
chêne planté dans le ventre d’un homme d’armes. Il appelait cela
son arbre généalogique. Je ne sais pas au juste ce que c’est que cet
arbre, n’en ayant jamais vu qu’en peinture ; mais je soupçonne fort
que c’est là ce qui lui permettait de vivre comme il faisait. S’il ne se
fût pas appelé l’Honorable Hector-Percy Cinqpoints, peut-être
l’aurait-on pris pour un simple escroc, car il jouait beaucoup et ne
perdait que lorsqu’il voulait bien s’en donner la peine. Pour un
homme de basse extraction une pareille profession est fort
dangereuse ; mais, lorsqu’un véritable gentilhomme consent à
l’embrasser, il ne saurait manquer d’y gagner beaucoup d’argent. Il
est vrai que le plus habile ne tarde pas à y laisser sa réputation, et
alors l’état ne rapporte plus que de maigres profits, assaisonnés de
soufflets et de condamnations infamantes.
Mon maître n’en était pas là. Jusqu’à ce jour, il avait eu le talent
de plumer ses victimes sans les faire crier. Sachant combien les
oiseaux de Thémis sont coriaces, il cultivait aussi peu la
connaissance des hommes de loi que celle du code ; mais, afin
d’ajouter à sa respectability en ayant l’air de s’occuper de sa
profession, il habitait le quartier des avocats, et daignait parfois
mettre la main sur quelque pigeon roturier qui s’aventurait dans son
dangereux voisinage.
De ce nombre fut le pauvre Thomas Dakins, Esq. [4] , étudiant en
droit, récemment installé dans la maison que nous habitions, et dont
Cinqpoints ne tarda pas à convoiter le plumage argenté. Ce jeune
imprudent eût mieux fait de ne jamais venir au monde que de planer
sous les serres d’un oiseleur aussi impitoyable ; car il fut bientôt
complétement ruiné, grâce aux efforts combinés de mon maître et
du sieur Richard Blewitt, dont le nom, gravé sur une plaque de
cuivre, se lisait sur la porte d’un appartement voisin du nôtre.
[4] Le titre d’esquire, écuyer, affecté dans l’origine aux
aspirants chevaliers et plus tard à certains propriétaires
fonciers, se donne aujourd’hui à tout Anglais vivant de
ses rentes ou exerçant une profession libérale ; en un
mot, à celui que les paysans nomment un monsieur.
Cette désignation s’écrit en abrégé à la suite du nom.
(Note du traducteur.)