CHAPTER -1

BASIC INEQUALITIES:
Outline
 INTRODUCTION
 CHEBYSHEV INEQUALITIES
 MARKOV INEQUALITY
 CHERNOFF BOUND
 THE LAWS OF LARGE NUMBERS:
 Histograms
 Central Limit Theorem

2
 INTRODUCTION

 The variance expresses the variability of the random variable about the mean
value. If this variability is zero, then the random variable will be no longer
random but a constant.

3
 In many problems in communications we are interested in finding bounds on
certain performance measures. The idea in using a bound is to substitute a
simpler expression for something more complicated, thus gaining some insight
into the problem at the expense of accuracy.

 CHEBYSHEV INEQUALITIES

4
5
  MARKOV INEQUALITY

Proof:

 CHERNOFF BOUND

There are many different forms of Chernoff bounds, We will start with the statement of
the bound for the simple case of a sum of independent Bernoulli trials, i.e. the case in
which each random variable only takes the values 0 or 1. For example, this corresponds to
the case of tossing unfair coins, each with its own probability of heads, and counting the
total number of heads.

6
  THE LAWS OF LARGE NUMBERS:

The average of many measurements of the same unknown quantity tends to give a
better estimate than a single measurement. Intuitively, this is because the random
error of each measurement cancels out in the average. For this, intuition precise in
two ways: law of large numbers (LoLN) and the Central Limit Theorem (CLT).

7
 Weak Law of Large Numbers (WLLN)

8
The weak law of large numbers

9
  Strong Law of Large Numbers

The strong law of large numbers states that this convergence is not only in probability
but also with probability 1.

 Histograms

10
11
 Central Limit Theorem (CLT)

CLT states that if we sum a sufficient number of random variables, the resulting probability is
a Gaussian.

12
13
14