Functional Linear Regression

Statistical Models

So far we have focussed on exploratory data analysis

Smoothing
Functional covariance
Functional PCA
Now we wish to examine predictive relationships → generalization
of linear models.
X
yi = α + βj xij + i
Functional Linear Regression

Functional Linear Regression

yi = α + xi β + i
Three different scenarios
Functional covariate, scalar response
Scalar covariate, functional response
Functional covariate, functional response
We will deal with each in turn.
Functional Linear Regression

Models for Scalar Responses

Temperature curves shifted by total annual precipitation:

We want to relate annual precipitation to the shape of the

temperature profile (see handout or text).
Functional Linear Regression

A First Idea

We observe yi , xi (t)
Choose t1 , . . . , tk
Then we set

X
yi = α + βj xi (tj ) + i
= α + xi β + 

And do linear regression.

But how many t1 , . . . , tk and which ones?
Functional Linear Regression

In the Limit

If we let t1 , . . . get increasingly dense

X
yi = α + βj xi (tj ) + i = α + xi β + 
becomes
Z
yi = α + β(t)xi (t)dt + i

Minimize squared error:

X Z 2
β(t) = argmin yi − α − β(t)xi (t)dt
Functional Linear Regression

Identification

Problem:
In linear regression, we must have fewer covariates than
observations.
if I have yi , xi (t), there are infinitely many covariates.
Z
yi = α + β(t)xi (t)dt + i

I can always make the i = 0

Functional Linear Regression

Smoothing

We want to insist that β(t) is smooth.

Fit by penalized squared error

n 
X Z 2 Z
PENSSEλ (β) = yi − α − β(t)xi (t)dt +λ [Lβ(t)]2 dt
i=1

Very much like smoothing.

Still need to represent β(t) – use a basis expansion
X
β(t) = ci φi (t)
Functional Linear Regression

Choosing A Basis
Smoothing problem

n 
X Z 2 Z
PENSSEλ (β) = yi − α − β(t)xi (t)dt +λ [Lβ(t)]2 dt
i=1

has a unique minimizer over all functions for which

PENSSEλ (β) < ∞.

No explicit representation is available.

However, using a rich enough basis will approximate it well.
Usually, “rich enough” isn’t too rich.
If xi (t) are represented by a basis, using the same basis often
works well.
Functional Linear Regression

Calculation
Z Z 
yi = α + β(t)xi (t)dt + i = α + Φ(t)xi (t)dt c + i

= α + xi c + i
so
 
α
y=Z +
c
and
 −1
[α̂ ĉT ]T = Z T Z + λR ZTy
Then
Z  
α̂
ŷ = β̂(t)xi (t)dt = Z = Sy
ĉ
Functional Linear Regression

Confidence Intervals
Following from smoothing methods we have that
  −1 −1
α̂  
Var = σe2 Z T Z + λR Z T Z Z T Z + λR
ĉ
Assuming independent

i ∼ N(0, σe2 )
Estimate

σ̂e2 = SSE /(n − df ), df = trace(S)

And confidence intervals are
q
Φ(t)ĉ ± 2 Φ(t)T Var[ĉ]Φ(t)
Functional Linear Regression

Multivariate Functional Linear Regression

What if there are scalar covariates z and multiple functional

covariates x1 (t), . . . , xK (t)?

k Z
X
yi = α + zi γ + βj (t)xij (t)dt + i
j=1

Then the penalized sum of squares is

 2
n
X k Z
X K
X Z
yi − α − zi γ + βj (t)xij (t)dt  + λj [Lj βj (t)]2 dt
i=1 j=1 j=1
Functional Linear Regression

Summary

Functional linear regression: move from summation to

integration
Identifiability – use a smoothing penalty (remarkable similarity
to smoothing)
Cross validation for smoothing penalty choice
Confidence intervals based on residual errors
Can generalized to multiple and mixed covariates