- Hello again, everyone.

In the last video we talked about conditional probability

and what we would like to discuss now is independence.

So intuitively speaking, we want to set two events

E and F independent, which we denote using this symbol here,

if the occurrence of one of them does not change

the probability of the other occurring.

Now a little more formally, we would like to say

that E and F are independent if the probability

of F given E, namely the probability that F happens

given that E has happened is the same as P of F,

it doesn't change from not knowing that E happened.

Now visually, we can interpret it as follows.

So first of all, P of F is the fraction of omega that is F.

That's our definition of the probability of an event,

it's what fraction of times out of all of omega happening,

F will happen,

and P of F given E is according to our definition

that we have in the class video

is the probability of the intersection of E of F

divided by the probability of E.

So looking at the Venn Diagram,

P of F is this red area here

relative to the area of the whole space

and P of F given E
is the probability of this intersection here,

the probability of this pink space here

relative to the area of all of E here.

So that's what P of F given E is

and now if the events are independent,

if P of F is equal to P of F given E,

that means that the fraction of time

that F happens out of the whole sample space, omega,

is the same as the fraction here

of the intersection given all of E,

so that's visually what independence means for us,

the fraction that F is the whole space

is the same as the fraction of the intersection out of E.

So know that if E happens, the probability

that F will also happen is this fraction,

and it is the same as the original fraction

of F out of all of omega.

So that is what independence means for us.

Our definition that we have just said

said that F and E are independent

if P of F is equal to P of F given E

which was by definition

was P of E in the intersection F

divided by P of E.

It's a good definition and it conveys our intuition,

but there are two small issues with it.

One is that it's asymmetric,

we want to say that E and F are independent,

whereas this definition is not symmetric,

it talks about P of F given E,

not P of E given F, so that's one issue.

A second one is that P of F given E

is not defined when P of E is zero

because we would be dividing by zero,

so it doesn't apply to events which probability is zero.

So we want to resolve these two small issues

and formally we'll say that E and F are independent

if P of E intersection F is P of E times P of F

so it's essentially the same definition as we had,

it's just that we move P of E to the numerator here

so we get the P of E times F

is equal P of E intersection F

and if that happens we say that E and F

are statistically independent or independent.

This you can easily see that it applies

when the probability of E or the probability of F is zero.

In fact, it will say that if the probability

of E or F is zero, then they are independent

which makes sense because for example,

if E has zero probability,

that means that it will never happen

and I if told you that F happened,

then still E will not happen

because it never happens,

so the probability of E doesn't change.

It was zero and it will remain zero.

And likewise if P of F is zero,

so it applies when the probabilities are zero

and also it's symmetric.

You can see that this definition is symmetric

in E and F so if F is independent of E,

then E is independent of F

and that's the way that you would like it to be.

And it also applies the earlier definition

because if this is true

and P of E is non-zero,

then you can just normalize by P of E

and you'll get the original definition.

It's just slightly more general.

So let's consider dependent

and independent events for a die.

So we'll look at three events.

First, that the number we get is a prime

and that means we got a value of two, three, and five.

Happens probability three over six which is one half,

or the event where the value we get is odd,

namely one, three, five.

Again, the probability one half.

And the third event is that the number we got is a square,

namely one or four and that happens probability six,

which is one third.

We would like to see which of these events

are dependent and which of the pairs are independent.

So to do that, we need to consider

the probability of the intersection

and the product of the probabilities

so let's first look at prime and odd.

So the intersection of prime and odd

is the set three and five,

those are the only numbers that are both prime and odd

between one and six and their probability

is two over six, which is one third.

So the probability of the intersection

of prime and odd is one third.

On the other hand,

the product of the probabilities of prime and odd.

Well, prime has probability one half

because their are three numbers

and odd has has probability one half because again,

there are three numbers so the product

for those probabilities is going to be

one half times one half, which is a quarter

and because one quarter is not equal to one third,

these two events are dependent.

Similarly, if we look at prime and square,

then their intersection is the empty set,

and their probability is therefore zero.

Not empty set, but zero.

On the other hand, the product of the probabilities

is one half times one third, which is one sixth,

so one sixth is different from zero

so these two events are independent

and if we look at the last pair, odd and square,

then their intersection is one

because the only number between one and six

that is both odd and a perfect square is one,

so the probability of the intersection

is therefore one sixth,

while if we look at the product of the probabilities,

the probability of odd is one half

and the probability of a square is one third,

the product is one sixth and that tells us

that these two probabilities,

the probability of the intersection

and the product of probabilities are the same

and therefore odd and square are independent.

Now that doesn't mean that odd is somehow related

to square or not, it just means that given

the probabilities we give to odds and square,

if I told you that the outcome is odd,

it will not change your probability

that the value is a square,

it's still going to be one third.

And if I told you that the value was square,

namely one of four, it is not going to change

your estimate of the probability that it is odd,

it was half before because it was one, three, or five

out of six values and now we know that it's one of four

because the square and the probability it's odd

is again, one half.

Finally, let's look at one more example, coins.

Let's look again, three events.

First of all, H one, the first coins is heads.

Namely get get H heads and then something else.

That happens probability one half.

Second event is H two, namely that the second coin is heads.

And the other two we don't care.

That again, happens probability one half.

And the third event is that we get exactly

two heads in a row, namely we either get

H H tails, or tails H H

and that happens probability two over eight,

which is a quarter.

And now we want to see which of these three events

are dependent or independent,

so if we look at H one intersection H two, that's the event

that both the first and second coin are heads.

That happens probability one quarter

and if we look at the product of H one and H two,

the probabilities are half times half which is a quarter,

so we get that the probability of the intersection

is the product of the probabilities

and therefore the events are independent.

If we look at H two intersection with H H,

with two heads in a row,

that's exactly the event that we get

H heads tails or tails heads heads,

so the probability of that event is a quarter

and we see that the product of the probability

is H two has probability one half

and two Hs in a row is a quarter,

so the product is one eighth, so these are dependent.

Namely, if I told you that H two happens,

that changes your estimate

about whether you'll get two heads in a row

because before the probability

of H heads was one quarter and now

it's going to be one half as you can see.

And finally, let's look at H one,

namely the first is heads and then HH.

The intersection of these two events

is just heads heads tails because we need to have

H the first time and then two heads in a row exactly,

so it's H H tails.

That probability is one eighth,

it's one outcome out of eight

and if we look at the probability of H one

of the first heads which is half

times the probability of two heads in a row,

which is one quarter,

we get one eighth and so these values are equal

and therefore these two events are independent.

So what we can see is that these events

to be dependent or independent

is a statistical phenomenon.

We're just looking at the probabilities

and seeing whether the probability change or not

and that's our measure of when

probabilities are dependent or independent.

With that, we're going to finish this slide

and we talked about independence

and next time we'll talk about base rule.

End of transcript. Skip to the start.

POLL

Of 10 students in a classroom, 4 are taking only history, 3 are taking only math, and 3 are taking both
history and math. If you select a student uniformly at random, the events of the student taking history
and the student taking math are:

Independent

Dependent