Multivariate Distributions

Limit Theorems

1-1

Limit Theorems
Central Limit Theorem describes the (asymptotic) behaviour of
sample mean

X1 , X2 , . . . , Xn , i.i.d with Xi ( , )

n(x ) Np (0, )
for

n .

The CLT can be easily applied for testing.

Normal distribution plays a central role in statistics.

MVA: HumboldtUniversitt zu Berlin

Asymptotic Distribution, N=5

0.4 0.4

Asymptotic Distribution, N=35

Estimated and Normal Density 0.1 0.2 0.3

-3

-2

-1 0 1 1000 Random. Samples

Estimated and Normal Density 0.1 0.2 0.3

-2 0 1000 Random. Samples

The CLT for Bernoulli distributed random variables. Sample size

n = 5 (left) and n = 35 (right).

The CLT in the two-dimensional case. Sample size

n = 85 (right).

n = 5 (left) and

Limit Theorems

1-4

a consistent estimator of is asymptotically normal:

P .

2 (x ) n Np (0, p )

Condence interval for (univariate) mean

Xi N ( ,

2)

N (0, 1)

MVA: HumboldtUniversitt zu Berlin

Limit Theorems

1-5

Dene

u1

/2 as the 1

Then we get the

C1

/2 quantile of the N (0, 1) distribution. following 1 condence interval: ] [ = x u1 /2 , x + u1 /2

P ( C1 ) 1

for

n .

MVA: HumboldtUniversitt zu Berlin

EDF and CDF, n=100 1 edf(x), cdf(x) 0 0.5

-2

-1

0 x

The standard normal cdf and the empirical distribution function for

n = 100.

EDF and CDF, n=1000 1 edf(x), cdf(x) 0 0.5

-2 x

The standard normal cdf and the empirical distribution function for

n = 1000

Limit Theorems

1-8

Bootstrap condence intervals

Empirical distribution function
edf

Xi F Xi Fn x = mean of bootstrap sample

{
sup

Fn = n1

n i =1 I (xi x )

n(x x )

<u

n(x )

<u

)}

a.s.

Construction of Condence Intervals possible! The unknown distribution of

x .

can be approximated by the known distribution of

MVA: HumboldtUniversitt zu Berlin

EDF and 2 bootstrap EDFs, n=100

1 0 edfs{1..3}(x) 0.5

-2

-1

0 x

The cdf

Fn

and two bootstrap cdf `s

. Fn

Limit Theorems

1-10

Transformation of Statistics
If

n(t ) Np (0, ) and if f = (f1 , . . . , fq ) : p q

are real valued functions which are dierentiable at

p ,

then

f (t ) is asymptotically normal with mean f ( ) and covariance

, i.e.,

where

n{f (t ) f ( )} Nq (0,

for

n ,

( =

fj ti

(t )

t=

(p q )

matrix of all partial derivatives.

MVA: HumboldtUniversitt zu Berlin

Limit Theorems
Suppose

1-11

p = 2,

{Xi }n i =1 ( , );

( ) 0 = ,
0

( =

1 0.5

0.5 1

) .

We have by CLT for

n(x ) N (0, ).

(
The distribution of

f = (f1 , f2 )

x2 1 x2 x 1 + 3x 2

)
? This means to consider

with

2 f1 (x1 , x2 ) = x1 x2 ,

f2 (x1 , x2 ) = x1 + 3x2 ,

q = 2.

MVA: HumboldtUniversitt zu Berlin

Limit Theorems
Then

f( )=

(0)
0

1-12
and

= (dij ),

dij =

fj xi

x=

2 1

1 3

) =

1 3

) .

x =0

We have the covariance

0 1

1
3

) (

This yields

1 2

1 2
1

) (

1 3

) =

7 2

7 2
13

) .

x2 1 x2 x 1 + 3x 2

(( ) ( 0 N2 ,

7 2

7 2
13

)) .

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-1

Heavy-Tailed Distributions

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-2

Heavy-Tailed Distributions
Distribution Comparison

0.4

2f

1f

1f

2f

0.3

Gaussy Cauchy

0.0 6

0.1

Y 0.2

0 X

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-3

Student's t-distribution
PDF of tdistribution CDF of tdistribution

0.4

0.3

t3 t6 t30

0.8

1.0

t3 t6 t30

0.6

Y 0.2

Y 4 2 0 X 2 4 0.0 4 0.2 0.4

0.0

0.1

0 X

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-4

Student's t-distribution
Let

be a normally distributed random variable with mean

and

variance

2 , and

2 2 has a be the random variable such that Y /

chi-square distribution with and

degrees of freedom. Assume that

are independent, then

X n t = Y
def

is distributed as Student's

with

degrees of freedom.

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-5

Student's t-distribution
The

t -distribution has the following density function

( ) ( ) n +1 1 2 n+ 2 x2 ft (x ; n) = ( ) 1 + n n n 2

where

n is the number of degrees of freedom, < x < , and

( ) =
0

is the gamma function, e.g.,

1 x

e dx .

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-6

Multivariate t-distribution
If

2 Np ( , ) and n respectively, and X n/Y = t , then the pdf of t is given by

and are independent and distributed as

ft (t ; n, , )
= (n/2)np/2 p/2

{(n + p )/2}
1/2 {
1

}(n+p)/2 1 +n (t ) 1 (t )

The distribution of

is the noncentral

t -distribution with n
.

degrees

of freedom and the noncentrality parameter

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-7

Laplace Distribution
PDF of Laplace distribution CDF of Laplace distribution

0.5

0.4

0.8

1.0

L1 L1.5 L2

L1 L1.5 L2

0.3

Y 0.2 0.1 0.0 6 4 2 0 X 2 4 6 0.0 6 0.2 0.4

0.6

0 X

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-8

Laplace Distribution
The Laplace distribution with mean the pdf and scale parameter 1 2
x

has

fLaplace (x ; , ) =
and the cdf 1{ 1 2

FLaplace (x ; , ) =
where

+ sign(x )(1 e

} ) ,

sign

is sign function.

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-9

Laplace Distribution
With mean 0 and distribution

= 1,

we obtain the standard Laplace

f (x ) = F (x ) =

e x
{2

ex
2

e x 1 2

for x < 0 for x 0

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-10

Multivariate Laplace Distribution

Let

G be the pdf and cdf of a d -dimensional Gaussian distribution Nd (0, ), the pdf and cdf of a multivariate Laplace
and distribution can be written as

fMLaplaced (x ; m, ) = FMLaplaced (x , m, ) =

0
0

g (z 2 x z 2 m)z 2 e z dz
1 1

G (z 2 x z 2 m)e z dz
1 1

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-11

Multivariate Laplace Distribution

the pdf can also be described as 2

fMLaplaced (x ; m, ) =

d 1 (2 ) 2 2

ex

m(

x 1 x 2 + m 1 m

K
where

) (2 + m 1 m)(x 1 x )

2d 2 and

K (x ) is the modied Bessel function of the

third kind

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-12

Cauchy Distribution
PDF of Cauchy distribution CDF of Cauchy distribution

0.4

0.3

C1 C2

0.8

C1 C1.5 C2

0.6

C1.5

Y 0.2

Y 6 4 2 0 X 2 4 6 0.2 0.4 6

0.0

0.1

0 X

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-13

Cauchy Distribution
Example 1 A gangster has just robbed a bank. As he runs to a point

meters

away from the wall of the bank, a policeman reaches the crime scene. The robber turns back and starts to shoot but he is such a poor shooter that the angle of his re is uniformly distributed. The bullets hit the wall at distance distribution of

x , the random variable where the bullet hits the wall,

(from the centre). Obviously the

is of vital knowledge to the policeman in order to identify the location of the gangster.

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-14

Cauchy Distribution
S

Wall of Bank

Robber a

Bullet Line

Policeman

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-15

Cauchy Distribution
Since is uniformly distributed:

f( )=
and tan

I ( [ /2, /2])

= =

x s
arctan 1

(x )

2 s 1 + (x s)

dx

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-16

Cauchy Distribution
d

For a small interval

, the probability is given by 1

f ( )d

= =

d
1

dx 2 + (x s)

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-17

Cauchy Distribution
with

2
1

1 1

x 2 dx = s 1 + ( s )
= =

arctan

( x )}

{ 1
2 1

)}
2

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-18

Cauchy Distribution

So the pdf of

can be written as:

f (x ) =

2 + (x s)

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-19

Cauchy Distribution
The general formula for the pdf and cdf of the Cauchy distribution is

fCauchy (x ; m, s ) = FCauchy (x ; m, s ) =
where

s
1 2

m 2 + ( x s )(
arctan

x m s

and

are location and scale parameter respectively.

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-20

Cauchy Distribution
The case in the above example where

m = 0 and s = 1 is called the

1

standard Cauchy distribution with pdf and cdf as following,

fCauchy (x ) = FCauchy (x ; m, s ) =
1 2

(1 + x 2 ) +

arctan(

x)

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-21

Mixture Model
Mixture modelling concerns modelling a statistical distribution by a mixture (or weighted sum) of dierent distributions. For many choices of component density functions, the mixture model can approximate any continuous density to arbitrary accuracy, provided that the number of component density functions is suciently large and the parameters of the model are chosen correctly.

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-22

Mixture Model
The pdf of a mixture distribution consists of be written as:

n distributions and can

f (x ) =
under the constraints: 0

L l =1

wl pl (x )
1 1

wl L wl = l =1

where

pl (x )dx =

1 is a

pl (x ) is the pdf of the l 'th component density and wl

weight.

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-23

Mixture Model

The pdf for a Gaussian mixture is:

fGM (x ) =

L l =1

wl
2

(x

l) 2 2 l

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-24

Mixture Model

For a Gaussian mixture consisting of Gaussian distributions with mean 0, this can be simplied to:

fGM (x ) =

L l =1

wl
2

2 x2 2

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-25

Mixture Model
Pdf of a Gaussian mixture and Gaussian Cdf of a Gaussian mixture and Gaussian

0.4

0.3

0.8

1.0

Gaussian Mixture Gaussian

Gaussian Mixture Gaussian

Y 0.2

Y 6 4 2 0 X 2 4 6 0.0 6 0.2 0.4

0.0

0.1

0.6

0 X

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-26

Multivariate Mixture Model

A multivariate mixture model comprises multivariate distributions, e.g. the pdf of a multivariate Gaussian distribution can be written as

f (x ) =

wl

l =1 2 l

1 2

e 2 (x
1

1 l ) (x l )

MVA: HumboldtUniversitt zu Berlin

Heavy-Tailed Distributions

2-27

Heavy Tailed Distributions

Distribution comparison Tail comparison

0.020

0.5

q q q q q

0.4

0.015

q q q q q q q q
q q

Laplace NIG Cauchy Gaussian

0.3

q q q q q q q q q

NIG Laplace Cauchy Gaussian

q q q q q q q

Y 0.010

q q q q q q q

q q

0.2

q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q

q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q

0.1

0 X

0.000

0.0

0.005 5.0

4.8

4.6 X

4.4

4.2

4.0

MVA: HumboldtUniversitt zu Berlin

Copulae

Copulae

3-1

Example

we pay 200 EUR for the chance to win 1000 EUR, if DAX returns decrease by 2%

PDAX (rDAX

0.02) = FDAX (0.02) = 0.2

we pay 200 EUR for the chance to win 1000 EUR, if DJ returns decrease by 1%

PDJ (rDJ 0.01) = FDJ (0.01) = 0.2

MVA: HumboldtUniversitt zu Berlin

Copulae

3-2

Example

we get 1000 EUR if DAX and DJ indices decrease simultaneously by 2% and 1% respectively. how much are we ready to pay in this case?

P{(rDAX

0.02) (rDJ 0.01)}

= FDAX ,DJ (0.02, 0.01) = C {FDAX (0.02), FDJ (0.01)} = C (0.2, 0.2).
MVA: HumboldtUniversitt zu Berlin

Copulae

3-3

Correlation
Gaussian
4 4

Gumbel

q q q q q q q q q q q q q q qq q qq q q qq q q qq q q q q q q qq q q q q q qqq q q q q q q qq q q q q q q q q q q q q qq q q q qq q q q q q q q qqq q q q q q q qq q q q qq q q qq q q q qq q q q qq qq q q q qqq q q q q q q q q q q q q q qq q q q q qq q qq q q q q qq q q q q q q qq qq qq q q q q q q qq qq qqq q qq qq q q q q qq q q q q q q q qq q q q q q q q qq qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q qq q q q q q q q q q q qqq q q qq q qq q q q q q qqq q q q q q q q qqq q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q qq q q q q q q qq q q q qq q qq qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q qq q qq q q qq q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q qq q q q qqq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q qq qq q qq qq q q q q q q q q q qq q q qq qq q q q q q q qq q q qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q qq q q q qq q qq q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q qq q qq q q q q q q qq q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qqq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q qq qq q q q q q q q q qqq q q q q q qq q qq q q q q q qq q q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q qq q q q q q q q q qq q q q q q qq q q qq qq q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q qqq q q q q q q q q qq q q q q q qq q q q q q q q q qq q q q qq q qqq q q q q q q qq q q q qqq q qq q q q q qq q q qq qq q q qq q q q q q q qq q q q q qq q qq q q q q q q q q qq qqq qqq q q q qq q qq qq q qq q q qq q q q q q qq q q q q qq q qq q qq qq q q q q q q qq qq q q q q q q q q q q q q qq q q q q q q q qq qq q q q q q q q qq q q qq q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q

q q q q

q q q q

q q q qq q q qq q q q q q q q q q q q q qq q q qq q q q qq q q qq q q q q qq qq q q q q q q q q q q q q q q q q qqq qq q qq qq qq q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q qq q q q qq q q q q qq q q q q q q qq q q qq q q q qq q q q q q q q q q q q q qq q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q qq q qq q q q q q q q q q q q q q q qq qq q q q qq q qq q qq q q q q q q q q qq q q q q q q q qq q qq q qq q q q q q qq qq q q q q q q q qq q q q q q q q qq q q qq q q q qq q q q q qq q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q qq q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq qq q qq q q q q q qq q q q q q q q q q q q q q qq q q q q q qq q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q qq qq q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q qq q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q qq q q q q q q q q q q q q qq qq qq q q q q q q qq q q qq q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q qqq q q q q q q q q qq qq q q q q qq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq qq q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q qq q qq q q q q q q q q q q q qqq q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q qq q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q qq q q q q qq q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q qqq q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q qq q qq q qq q q q q qq q q qq q q q qq q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q q qq q q qq q q qq q q q q q q qq q q q q q q q q qq q q q q q q qq q q q q q q q q q q qq q q q q qq q q q q q q q q qq q q q q qq q q q q q q q q qq q q q qq q qq q q q qq q q q q qq q q q q q qq qq q q q q q q q q q qq qq q qq q q q q q qq q q q q q q q q q q q qq q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q qqq q q qq q q q q q q q q q q q q qq qq q q q qqq q qq q q qq q q q q q q q q q q q q q q q qq q q q q q

qq

Figure 1: Scatterplots for two distribution with

= 0.4

same marginal distributions same linear correlation coecient

MVA: HumboldtUniversitt zu Berlin

Copulae

3-4

Extreme, synchronized rises and falls in nancial markets occur infrequently but they do occur. The problem with the models is that they did not assign a high enough chance of occurrence to the scenario in which many things go wrong at the same time - the perfect storm scenario

(Business Week, September 1998)

MVA: HumboldtUniversitt zu Berlin

Copulae

3-5

Correlation
0.15
q

4
q q q

4
q q

1. 19.10.1987 Black Monday

0.10

q q q q

q q q

2
q

3 4
q

0.10

q q q

q q q q qq q q q q qq qq q q q q q q q q q qq q q qq q q q q q q q q q q qq q q q q q q q q q q q q qq q qq q qq q qq q qq qq q q q q qq qq q q qq qq q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q qq q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q qq q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q qq qq qq q qq q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqqq q q q q q q q q q q q q q q q q qq q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q qq q qq q q q q q qq q q q q q qq q q qq q q q q q q q qq q q q q q q q q q qq q q qq q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qq q qq q q qq q q q qq q q qqq q q qq q q q qq q q q q q q qq q q q q q q q q q

q q q

4
q q

0.05

q q q q q q

2. 16.10.1989 Berlin Wall 3. 19.08.1991 Kremlin 4. 17.03.2008, 19.09.2008,

siemens

0.00

0.05

10.10.2008, 13.10.2008,

0.15

4
q

4
q

15.10.2008, 29.10.2008
0.00 0.05 0.10

0.15

0.10

0.05

Crisis

bmw

MVA: HumboldtUniversitt zu Berlin

Copulae

3-6

Copula
For a distribution function exists a copula

C : [0, 1]d [0, 1],

with marginals such that

FX1 , . . . , FXd , there

F (x1 , . . . , xd ) = C{FX1 (x1 ), . . . , FXd (xd )}.

MVA: HumboldtUniversitt zu Berlin

Copulae

3-7

A little bit of history

1940s:

Wassilij Hoeding studies properties of multivariate Abe Sklar)

distributions 1959: The word copula appears for the rst time ( 1999: Introduced to nancial applications (

Alexander McNeil, Daniel Straumann in RISK Magazine)

2000: Paper by

Paul Embrechts,

David Li in Journal of Derivatives on

application of copulae to CDO 2006: Several insurance companies, banks and other nancial institutions apply copulae as a risk management tool

MVA: HumboldtUniversitt zu Berlin

Copulae

3-8

Applications
Practical Use: 1. medicine (Vandenhende (2003)) 2. hydrology (Genest and Favre (2006)) 3. biometrics (Wang and Wells (2000, JASA), Chen and Fan (2006, CanJoS)) 4. economics

portfolio selection (Patton (2004, JoFE), Xu (2004, PhD thesis), Hennessy and Lapan (2002, MathFin)) time series (Chen and Fan (2006a, 2006b, JoE), Fermanian and Scaillet (2003, JoR), Lee and Long (2005, JoE)) risk management (Junker and May (2002, EJ), Breyman et. al. (2003, QF))

MVA: HumboldtUniversitt zu Berlin

Copulae

3-9

Applications

Bourdeau-Brien (2007) covers 871 publications

MVA: HumboldtUniversitt zu Berlin

Copulae

3-10

Special Copulas
Theorem Let C be a copula. Then for every (u1 , u2 ) [0, 1]2
max ( 1

u + u2 1, 0) C (u1 , u2 ) min(u1 , u2 ),

where bounds are called lower and upper Frchet-Hdings bounds. When they are copulas they represent perfect negative and positive dependence respectively.
The simplest copula is product copula

(u1 , u2 ) = u1 u2
characterize the case of independence.

MVA: HumboldtUniversitt zu Berlin

Copulae

3-11

Copula Classes
1. elliptical

implied by well-known multivariate df 's (Normal, t ), derived through Sklar's theorem do not have closed form expressions and are restricted to have radial symmetry

2. Archimedean

C (u1 , u2 ) =

{ (u1 ) + (u2 )}

allow for a great variety of dependence structures closed form expressions several useful methods for multivariate extension not derived from mv df 's using Sklar's theorem

MVA: HumboldtUniversitt zu Berlin

Copulae

3-12

Copula Examples 1
Gaussian copula

G (u , u ) 1 2

= {1 (u1 ), 1 (u2 )}
1 1 (u1 ) (u2 )

=

2

1 1

{
2
exp

(s 2 2 st + t 2 ) 2) 2(1

}
ds dt ,

Gaussian copula contains the dependence structure

normal marginal distribution + Gaussian copula = multivariate

normal distributions

non-normal marginal distribution + Gaussian copula =

meta-Gaussian distributions allows to generate joint symmetric dependence, but no tail dependence

MVA: HumboldtUniversitt zu Berlin

Copulae

3-13

Copula Examples 2
Gumbel copula

Gu (u , u ) C 1 2

{ [ ] } 1/ 1/ = exp ( log u1 ) + ( log u2 ) .

for for for

>1 =1

allows to generate dependence in the upper tail reduces to the product copula obtain Frchet-Hoeding upper bound

C (u1 , u2 ) min(u1 , u2 )
MVA: HumboldtUniversitt zu Berlin

Copulae

3-14

Copula Examples 3
Clayton copula

Cl (u , u ) 1 2

max

u1

)] 1 + u2 1, 0

dependence becomes maximal when independence is achieved when

=0

the distribution tends to the lower Frchet-Hoeding bound when

allows to generate asymmetric dependence and lower tail dependence, but no upper tail dependence

MVA: HumboldtUniversitt zu Berlin

Copulae
Normal Copula Gumbel Copula Clayton Copula

3-15

x
0.12 0.08
0.04

0.2

0.1

0.2

0.0

MVA: HumboldtUniversitt zu Berlin

z
0.0 2

z
0.02

z
0.0 2
0.04

0.0

0.06 0.1
0.14
0.1 4

0.1

0.08

0.08
0.14

y
0.1

0.18

0.18
0.2

x
0.12
0.06
0.04

0.1

x
2 0.1

0.1

6
2 0.0

0.1
2

0.0

0.0

0.0

z
0.0 2

z
0.0 2

z
0.0 2

2 0.0

0.04
0.06

0.04
0.06

0.08

0.1
0.14

0.12

0.1

0.16

0.14

6 0.1
0.12

0.14

x
0.12
0.08
0.0 2
0.0 2

6 0.1

0.08
0.0

0.1
0.06

0.0

0.0

0.04
2

Copulae

3-16

Dependencies, Linear Correlation

(X1 , X2 ) =

Cov (X1 , X2 ) . Var (X1 )Var (X2 )

Sensitive to outliers Measures the 'average dependence' between

X1

and

X2

Invariant under strictly increasing linear transformations May be misleading in situations where multivariate df is not elliptical

MVA: HumboldtUniversitt zu Berlin

Copulae

3-17

Dependencies, Kendall's tau

Denition If F is continuous bivariate cdf and let (X1 , X2 ), (X1 , X2 ) be independent random pairs with distribution F . Then Kendall's
is

tau

= P {(X1 X1 )(X2 X2 ) > 0} P {(X1 X1 )(X2 X2 ) < 0}

MVA: HumboldtUniversitt zu Berlin

Copulae

3-18

Dependencies, Kendall's tau

Less sensitive to outliers Measures the 'average dependence' between X and Y Invariant under strictly increasing transformations Depends only on the copula of For elliptical copulae:

(X1 , X2 ) ( ) (X1 , X2 ) = sin 2

MVA: HumboldtUniversitt zu Berlin

Copulae

3-19

Dependencies, Spearmans's rho

Denition If F is a continuous bivariate cumulative distribution function with marginal F1 and F2 and let (X1 , X2 ) F . Then Spearmans's rho is a correlation between F1 (X1 ) and F2 (X2 )
=

Cov {F1 (X1 ), F2 (X2 )} . Var {F1 (X1 )}Var {F2 (X2 )}

MVA: HumboldtUniversitt zu Berlin

Copulae

3-20

Dependencies, Kendall's tau

Less sensitive to outliers Measures the 'average dependence' between

X1

and

X2

Invariant under strictly increasing transformations Depends only on the copula of For elliptical copulae:

(X1 , X2 ) ( ) (X1 , X2 ) = 2 sin 6

MVA: HumboldtUniversitt zu Berlin

Copulae
= = = 0.892, 0.956, 0.996 = = = 0.659, 0.888, 0.982

3-21

q q q q q q q q

10

1.0

0.5

q q q q q qq q q

0.0

q q q qq q q qqq q q qq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q

y 2
q qq q q q q q qqq q q q q q q q q q q q q q q q q q qq q q q q q qq q q qq q q q q q q q q q q qqq q q q q q q q q q qq q q qq q q q q q q q q q qq q q q q q q q qq q q q q q q q

q q q qq q q q q q q q qq qq qq q qq q q q qq q q q q

0.5

1.0

q q

1.0

0.5

0.0 x

0.5

1.0

4 1.0

0.5

0.0 x

0.5

1.0

MVA: HumboldtUniversitt zu Berlin

Copulae

3-22

Dependencies, Examples
Gaussian copula

= =
where

6 2

arcsin arcsin

is a linear correlation coecient.

Gumbel copula

no closed form, 1

MVA: HumboldtUniversitt zu Berlin

Copulae

3-23

Multivariate Copula Denition

Denition
The copula is a multivariate distribution with all univariate margins being U 0 1 .

( , )

Theorem (Sklar, 1959)

Let X1 , . . . , Xk be random variables with marginal distribution functions

: [0, 1]k [0, 1] x1 , . . . , xk = [, ]

k -dimensional copula C F (x1 , . . . , xk )

F1 , . . . , Fk and joint distribution function F . Then there exists a such that

= C {F1 (x1 ), . . . , Fk (xk )}

(1)

MVA: HumboldtUniversitt zu Berlin

Copulae

3-24

Copula Density
Several theorems provides existence of derivatives of copulas, having them copula density is dened as

c (u1 , . . . , uk ) =

n C (u1 , . . . , uk ) . u1 . . . uk

Joint density function based on copula

c f (x1 , . . . , xk ) = c {F1 (x1 ), . . . , Fk (xk )} f1 (x1 ) . . . fk (xk ),

where 1 (), . . . ,

fk () are marginal density functions.

MVA: HumboldtUniversitt zu Berlin

Copulae

3-25

Special Copulas
Theorem
Let C be a copula. Then for every

(u1 , . . . , uk ) [0, 1]k

max

( k
i =1

)
ui

+ 1 k, 0

C (u1 , . . . , uk ) min(u1 , . . . , uk ),

where bounds are called

lower and upper Frchet-Hdings bounds.

When they are copulas they represent perfect negative and positive dependence respectively.
The simplest copula is product copula

(u1 , . . . , uk ) =
characterize the case of independence.

k i =1

ui

MVA: HumboldtUniversitt zu Berlin

Simulation

4-1

Simulation
Frees and Valdez, (1998, NAAJ), Whelan, (2004, QF), Marshal and Olkin, (1988, JASA)

Conditional inversion method:

= C (u1 , . . . , uk ), Ci = C (u1 , . . . , ui , 1, . . . , 1) and Ck = C (u1 , . . . , uk ). Conditional distribution of Ui is given

Let C

by

Ci (ui u1 , . . . , ui 1 )

= =

P {Ui

ui U1 = u1 . . . Ui 1 = ui 1 } i 1 Ci (u1 , . . . , ui ) i 1 Ci 1 (u1 , . . . , ui 1 ) / u 1 . . . u i 1 u 1 . . . u i 1

Generate i.r.v. v1 Set u1

, . . . , vk U (0, 1)

= v1

ui

MVA: HumboldtUniversitt zu Berlin

1 = Ck (vi u1 , . . . , ui 1 ) i = 2, k

Estimation

5-1

Estimations: Empirical Copula

i , . . . , x i ) be the order statistics if i -th stock and (x( 1) (T ) i , . . . , r i ) corresponding rank statistics such that x i i = x i (r1 t (rt ) T all i = 1, . . . , d . Any function
Let

for

(t

,...,

T d td ) 1 i = I{r ti } T T t =1 i =1 t

is an empirical copula

MVA: HumboldtUniversitt zu Berlin

Estimation

5-2

Estimation: bivariate case

based on Kendall's estimator

n=
where

Pn

P 1, n(n 1) n
1 1 n

is the number of concordant pairs.

For Gumbel copula

n =

based on Spearman's

estimator

n 2 2 i =1 (Ri R ) (Si S ) = , n n n 2 2 i =1 (Ri R ) i =1 (Si S )

where

(Ri , Si ) i = 1, n

are pairs of ranks.

For Gaussian Copula

MVA: HumboldtUniversitt zu Berlin

n = 2 sin

Estimation

5-3

Copula Estimation
The distribution of X

= (X1 , . . . , Xd )

with marginals F j x

X ( j, j)

= 1, . . . , d

is given by

FX (x1 , . . . , xd )
and its density is given by

= C {FX1 (x1 ,

1 ), . . . , FXd (xd ,

d ); }

f (x1 , . . . , xd , 1 , . . . ,

d , ) = c {FX1 (x1 ,

1 ), . . . , FXd (xd ,

d ); }

d j =1

fj (xj ,

j)

MVA: HumboldtUniversitt zu Berlin

Estimation

5-4

Copula Estimation
For a sample of observations

= ( 1 , . . . , d ; ) d +1

{xt } t =1

and

the likelihood function is

L( ; x1 , . . . , xT ) =
T

T t =1

f (x1,t , . . . , xd ,t ; 1 , . . . , d ; )

and the corresponding log-likelihood function

( ; x1 , . . . , xT ) = +

t =1 T d t =1 j =1

log

c {FX1 (x1,t , 1 ), . . . , FXd (xd ,t , d ); }

log

fj (xj ,t , j )

MVA: HumboldtUniversitt zu Berlin

Estimation

5-5

Full Maximum Likelihood (FML)

FML estimates vector of parameters in one step through

FML = arg max( ) FML = (1 , . . . , d , )

the estimates

solve

(/ 1 , . . . , / d , / ) = 0

Drawback:with an increasing dimension the algorithm becomes too burdensome computationally

MVA: HumboldtUniversitt zu Berlin

Estimation

5-6

Inference for Margins (IFM)

estimate parameters

from the marginal distributions:

j = arg max

{ T
t =1

}
log f

j (xj ,t ; j )

estimate the dependence parameter

by minimizing the pseudo

log-likelihood function

( ; 1 , . . . , d ) =

T t =1

log c

{FX1 (x1,t ; 1 ), . . . , FXd (xd ,t ; d ); }

the estimates

IFM = (1 , . . . , d , )

solve

(/ 1 , . . . , / d , / ) = 0
MVA: HumboldtUniversitt zu Berlin

Estimation

5-7

Canonical Maximum Likelihood (CML)

CML maximizes the

empirical marginal distributions

( ) = CML =
where

pseuso log-likelihood function with

X (x1,t ), . . . , F X (xd ,t ); } c {F 1 d )

T t =1

log

arg max(

X (x ) = F j

T + 1 t =1

I { Xj , t

x}

Advantage: no assumptions about the parametric form of the

MVA:marginal HumboldtUniversitt zu Berlin distributions

Estimation

5-8

(X1 , X2 ) C Gu , with = 1.5 and 2) = 2) = F1 = F2 = ( 1 , 1 ( 2, 2

estimate

(0, 1)

std. error 0.00998 0.00690 0.00991 0.00684 0.01327

1 2 1 2 2 2

0.00365 1.00553 -0.00106 0.99779 1.49632

MVA: HumboldtUniversitt zu Berlin

Estimation

5-9

Attractive Features

A copula describes how the marginals are tied together in the joint distribution The joint df is decomposed into the marginal dfs and a copula The marginal dfs and the copula can be modelled and estimated separately, independent of each other Given a copula, we can obtain many multivariate distributions by selecting dierent marginal dfs The copula is invariant under increasing and continuous transformations

MVA: HumboldtUniversitt zu Berlin