Multivariate Distributions

Univariate Distributions

2-1

Univariate Case
Let

x1 , . . . , xn be realizations of the random variable X X F , where F is unknown xi xi

by are returns of the asset for one rm at the day ti are numbers of sold albums

Example

David Bowie at day ti

The Man Who Sold the World

What is a good approximation of F ?

traditional or modern approach
MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-2

DAX (Pt )
1000 2000 3000 4000 5000 6000 7000 8000 1986

Pt

1988

1990

1992

1994

1996

1998 date

2000

2002

2004

2006 2008

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-3

Histogram of DAX
6e04 Density 0e+00 1e04 2e04 3e04 4e04 5e04

2000

4000 P

6000

8000

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-4
t t

DAX returns (rt = log PP1 )

10 rt 10 1986 5 0 5

1988

1990

1992

1994

1996

1998 date

2000

2002

2004

2006 2008

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-5

Histogram of DAX returns

0.5 Density 0.0 6 0.1 0.2 0.3 0.4

0 r

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-6

Traditional approach:

F0  known distribution parameters of F0 are estimated from the sample x1 , . . . , xn

Estimator, Tail Exponent Estimation, etc.

F0 = N(, 2 ) (, ), here = x , 2 = s 2 F0 = St (, , , 2 ) (, , , ) are estimated by Hull

check the appropriateness of

F0 by a test (KS type)

vs

H0 : F
if test conrm

= F0

H1 : F

= F0

F0 , use F0

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-7

Fit of the Normal distribution to DAX returns ( = 0.0002113130, 2 = 0.0002001865)

0.5 Density 0.0 6 0.1 0.2 0.3 0.4

0 R

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-8

Modern approach: calculate the edf

F (x ) =

n i =1

I { Xi x } ,

or the nonparametric kernel smoother n 1 K fh (x ) = nh i =1 name Uniform Epanechnikov Gaussian

3 4 (1 1 2

x Xi h K (u ) u
1 2 I{| 2 )I{|

exp

u | 1} u | 1} 1 2 2 u

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-9

x = (0.475, 1.553, 0.434, 1.019, 0.395)

0.6

Kernel smoothing with UNI kernel

0.4

0.2

0.0

2.5

2.0

1.5

1.0

0.5

0.0

0.5

1.0

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-10

x = (0.475, 1.553, 0.434, 1.019, 0.395)

0.6

Kernel smoothing with EPA kernel

0.4

0.2

0.0

2.5

2.0

1.5

1.0

0.5

0.0

0.5

1.0

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-11

x = (0.475, 1.553, 0.434, 1.019, 0.395)

0.6

Kernel smoothing with GAU kernel

0.4

0.2

0.0

2.5

2.0

1.5

1.0

0.5

0.0

0.5

1.0

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-12

The estimated density of DAX returns

0.5 Histogram Epanechnikov Normal tVert., 5 df

Density

0.0 6

0.1

0.2

0.3

0.4

0 R

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-13

Multivariate Case
{x1i , . . . , xdi }i =1,...,n is the realization of the vector (X1 , . . . , Xd ) F, where F is unknown.

Example
{x1i , . . . , xdi }i =1,...,n are returns of the at day ti

assets in the portfolio

(x1i , x2i ) are numbers of sold albums The Man Who Sold The World by David Bowie and singles I Saved The World Today by Eurythmics at day ti
MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-14

Multivariate Case

What is a good approximation of F ?

traditional or modern approach
Very exible approximation to F is challenging in high dimension due to curse of dimensionality.
MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-15

Multivariate Distributions
Rp (multivariate) distribution function is
Random vector

F (x ) = P (X F (x ) =

x ) = P (X1 x1 , X2 x2 , . . . , Xp xp )

f (x ) denotes density of X , i.e.

x

f (u )du
b a

f (u ) du = 1
(a, b)} =

P {X

f (x )dx

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-16

= (X1 , X2 ) ,

X1 Rk X2 Rpk

marginal density of X1 is

fX1 (x1 ) =

f (x1 , x2 )dx2

conditional density of X2 (conditioned on X1 = x1 )

fX2 |X1 =x1 (x2 ) = f (x1 , x2 )/fX1 (x1 )

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-17

Example

f (x1 , x2 ) =

1 2 1

3 2 2

0 x1 , x2 1, otherwise.

f (x1 , x2 ) is a density since f (x1 , x2 )dx1 x2 =

1 2
2 x1 1

+
0

3 2

2 x2

=
0

1 3 + = 1. 4 4

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-18

The marginal densities

fX1 (x1 ) fX2 (x2 )

= =

f (x1 , x2 )dx2 = f (x1 , x2 )dx1 =

3 1 2 1+ 2 2 1 3 2 1+ 4

1 0 0 1

3 1 x1 + x2 2 2 1 3 x1 + x2 2 2

dx2 = x1 + dx1 = x2 +
3 1 2 1+ 2 2 3 1 2 2+ 4

1 2 3 2

3 ; 4 1 4

The conditional densities

f (x2 | x1 ) =

x x

and

f (x1 | x2 ) =

x x

These conditional pdf's are nonlinear in joint pdf has a simple (linear) structure.
MVA: HumboldtUniversitt zu Berlin

x1 and x2 although the

Univariate Distributions

2-19

Denition of independence

X1 , X2 are independent i f (x ) = f (x1 , x2 ) = fX1 (x1 )fX2 (x2 ).

Two random variables may have identical marginals but dierent joint distribution.

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-20

Example

f (x1 , x2 ) = 1, f (x1 , x2 ) = 1+(2x1 1)(2x2 1),

1 0

0 < x1 , x2 < 1, 0 < x1 ,

x2 < 1,

1 1.

fX1 (x1 ) = 1, fX2 (x2 ) = 1.

2 x2 ]1 1 + (2x1 1)(2x2 1)dx2 = 1 + (2x1 1)[x2 0 = 1.

MVA: HumboldtUniversitt zu Berlin

swiss bank notes

0.5 0.6 0.4 density 0.3 density 0.2 0.1
7.5 8 8.5 9 9.5 10 10.5

swiss bank notes

0
8

0.1

0.2

0.3

0.4

0.5

10

11

lower inner frame (X4)

upper inner frame (X5)

Univariate estimates of the density of the bank notes.

X4 (left) and X5 (right) of

MVAdenbank2.xpl

swiss bank notes

swiss bank notes

2 1

2 1

Product of univariate density estimates (left) and joint density estimate (right) for

X4 (left) and X5 of the bank notes.

MVAdenbank3.xpl

Univariate Distributions

2-23

Summary: Distributions
The cumulative distribution function (cdf) is

F (x ) = P (X

< x ).

If a probability density function (pdf) x F (x ) = f (u )du . Let

exists then

= (X1 , X2 ) be partitioned in subvectors X1 and X2 with joint cdf F . Then FX1 (x1 ) = P (X1 x1 ) is the marginal cdf of X1 . The marginal pdf of X1 is

fX1 (x1 ) =

f (x1 , x2 )dx2 .

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-24

Summary: Distributions
Dierent joint pdf's may have the same marginal pdf's. The conditional pdf of X2 given X1 = x1 is f (x , x ) f (x2 | x1 ) = 1 2 fX1 (x1 ) Two random variables X1 , X2 are called independent i

f (x1 , x2 ) = fX1 (x1 )fX2 (x2 ). f (x2 | x1 ) = fX2 (x2 ).

This is equivalent to

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-25

Moments and Characteristic Functions

EX
Rp denotes the p -dimensional vector of expected values of the random vector X

EX

EX1 EXp
. . .

xf (x )dx =

x1 f (x )dx xp f (x )dx
. . .

= .

The properties of expected value follow from the properties of the integral:

E (X + Y ) = EX + EY

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

If

and

2-26

are independent then

E (XY
=

)=

xy f (x , y )dxdy y f (y )dy = EXEY

xf (x )dx

Denition of variance matrix ()

= Var (X ) = E (X )(X )
We say that random vector

X X

has a distribution with the vector of

expected values and the covariance matrix ,

(, )

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-27

Properties of the Covariance Matrix

Elements of are variances and covariances of the components of the random vector

X:

= (X X )
i j

X X
i i

= Cov(Xi , Xj ) =

X X
Computational formula:

Var (Xi )

= E (XX ) Variance matrix is positive semidenite: 0 (variance a a of any linear combination

MVA: HumboldtUniversitt zu Berlin

a X

cannot be -ve).

Univariate Distributions

2-28

Properties of Variances and Covariances

Var (a X ) Var (AX + b) Cov(X + Y , Z ) Var (X + Y ) Cov(AX , B Y )

a Var (X ) a =

= A = = =

Var (X ) A Cov(X , Z ) + Cov(Y , Z ) Var (X ) + Cov(X , Y ) + Cov(Y , X ) + Var (Y ) A Cov(X , Y ) B .

i ,j

ai aj X X
i

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-29
1 0 3 x1 0 1

=
1

x1 f (x1 , x2 )dx1 dx2 =

3 1 1 x1 + dx1 = 2 4 2 0 1 3 4+9 13 + = = , 6 8 24 24

x1
+
3 4

3 1 x1 + x2 2 2
2 x1 1 0

dx1 dx2

= =

x1

1 0

=
1

x2 f (x1 , x2 )dx1 dx2 =

1 3 1 x2 + x2 dx2 = 4 2 4 0 1 1 1+4 5 + = = 8 2 8 8

1 0 2 x2 0

x2

1 3 x1 + x2 2 2
3 x2 1 0

dx1 dx2

= =

3 + 2 0 2

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-30

Covariance Matrix
X1 X1 = = =
1 2

EX12 2 1
1 1 0

with
1 3 x1 + x2 2 2
3 x1 1

EX12

3 + 4 0 4

0 4 1 1

2 x1

dx1 dx2
3 8

=
0

X2 X2

= = =

EX22 2 2
1 1 0

with
1 3 x1 + x2 2 2
4 x2 1

EX22

1 4

3 + 3 0 2

0 3 1 2

2 x2

dx1 dx2
11 24

=
0

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-31

X1 X2

E (X1 X2 ) 1 2
1 1 0

with

E (X1 X2 )

= = =
1 6

0 0

x1 x2

1 3 x1 + x2 2 2

dx1 dx2

1 3 2 x2 + x2 6 4
1

dx2
1 0

2 x2

+
0

3 4

3 x2

1 = . 3

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-32

=
0.0815 0.0052 0.0052 0.0677

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-33

Conditional Expectations
= (X1 , X2 ) Conditional expectation of X2 , given
Random vector

X1 = x1 :

E (X2 | x1 ) =
and conditional expectation of

x2 f (x2 | x1 ) dx2 X1 , given X2 = x2 : x1 f (x1 | x2 ) dx1

E (X1 | x2 ) =
The conditional expectation

E (X2 | x1 ) is a function of x1 .

Typical example of this setup is simple linear regression, where

E (Y | X

= x ) = x .

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-34

Error term in approximation:

U = X2 E (X2 | X1 )
(1) (2)

E (U ) = 0 E (X2 |X1 ) is the best approximation of X2 by a function h(X1 ) of X1 in the sense of mean squared error (MSE) when MSE (h) = E [{X2 h(X1 )} {X2 h(X1 )}] and h : Rk Rpk .

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-35

Summary: Moments
X is = xf (x ) dx , the covariance matrix = Var (X ) = E (X )(X ) . We denote X (, ). Expectations are linear, i.e., E (X + Y ) = EX + EY . If X , Y are independent then E (XY ) = EXEY .
The expectation of a random vector

MVA: HumboldtUniversitt zu Berlin

Univariate Distributions

2-36

Summary: Moments
The covariance between two random vectors

X , Y is XY = Cov(X , Y ) = E (X EX )(Y EY ) = E (XY ) EXEY . If X , Y are independent then Cov(X , Y ) = 0. The Conditional Expectation E (X2 |X1 ) is the MSE best approximation of X2 by a function of X1 .

MVA: HumboldtUniversitt zu Berlin