Motivation & Overview References

CH5350: Applied Time-Series Analysis

Arun K. Tangirala

Department of Chemical Engineering, IIT Madras

Introductory Talk
Arun K. Tangirala Applied Time-Series Analysis 1
Motivation & Overview References

Time-Series

A time-series simple refers to an ordered collection of data (usually in time)

I e.g., yearly wages, annual production, daily temperature, hourly satellite images
I Measurements could be a function of other dimensions (e.g., frequency, space)
I Data may be collected at regular or irregular intervals
I Many variables could be recorded simultaneously (multivariate data)

Arun K. Tangirala Applied Time-Series Analysis 2

Motivation & Overview References

Examples
10

24 9

8
22
7
20
Temperature

Amplitude
Time-series can exhibit di↵erent features
18 5

4
16
3
14
2

12 1

0
I Non-stationarity (e.g., trend, random
500 1000 1500 2000 2500 0 5 10 15 20 25 30
Time Time (days)

walk)
(a) Temperature (b) Wind speed
I Oscillatory (periodicity)
9000 0.6

I Seasonality
8000 0.5

7000 0.4
Swiss Market Index

6000 0.3
Amplitude

I Non-linearity
5000 0.2

4000 0.1

3000 0

2000 −0.1

1000 −0.2
1991 1992 1993 1994 1995 1996 1997 1998 1999 0 2 4 6 8 10
Time (years) Time (sec)

(c) Swiss SMI (d) ECG

Arun K. Tangirala Applied Time-Series Analysis 3
Motivation & Overview References

Examples of time-series
More examples Time-Series Plots Power Spectra (normalized)
!"#$%&'()*+)(,-.#&(
! ()%*+,-%&%,-%,.*/01,.+.+%2*3.+4%5*67%/0%,89*:7*2&18%;*17%*<,:=,*12*
-+4%,2+:,2>

10
10

! $+?,1@2*81,*A%*B#-+4%,2+:,1@5*'#C5*"#C*:7*4:7%>

9
9

I1@P%*:0.&0.*67:4*1*8:,.7:@* I1@P%*:0.&0.*3&:=%7;*12*1* LQLH(RSHTRCDSURTE

8
8

@::&*12*1*60,8.+:,*:6*.+4% 60,8.+:,*:6*67%/0%,89* 3LHU;

7
7
Variables

Variables

6
6

5
5

4
4

I:@1.+@+.9*:6*JK$L*H:4&:2+.%* '#C*D41?%*:6*E172* "#C*$1.%@@+.%*D41?%*:6*

D,-%M*67:4*BNOO#'!!B :,*F0,%*!G5*'!!" H1,1-+1,*=%1.)%7*8:,-+.+:,
3
3

2
2
1

0 500 1000 1500 2000 5e-04 5e-03 5e-02 5e-01

Time (Samples) Frequency (normalized)

E:,.)2
!"#$%&#'!!" !!"#$%!&'!()%*+#),)-!.//0-!1!23!" "

Time-series could be multivariate, multidimensional and also transformed into

Arun K Tangirala (IIT Madras) Applied Time-Series Analysis July-Nov, 2007 3 / 16

another domain
Arun K. Tangirala Applied Time-Series Analysis 4
Motivation & Overview References

Systematic analysis of time-series data

Arun K. Tangirala Applied Time-Series Analysis 5

Motivation & Overview References

Applications

Arun K. Tangirala Applied Time-Series Analysis 6

Motivation & Overview References

Challenges in time-series analysis

Randomness & Uncertainty

1. Lack of a precise mathematical function to describe the process of interest (leading

to a probabilistic framework)
2. To be able to draw inferences on the ensemble from a single realization
3. Estimation of “unknowns” from (uncertain) observations / knowns.

Arun K. Tangirala Applied Time-Series Analysis 7

Motivation & Overview References

Sources of uncertainty and randomness

1. Uncertainty in process knowledge:

I Process characteristics are seldom known accurately and/or completely. e.g.,
atmospheric process, roll of a die, grade of a student, etc.
I From a prediction point of view, therefore, only outcomes with chances can be
stated at best.

Arun K. Tangirala Applied Time-Series Analysis 8

Motivation & Overview References

Uncertainty and randomness . . . contd.

2. Uncertainty in measurements:
I Every sensor introduces its characteristics into measurements. Hence, in reality
Observation = Truth + Perception (of the sensor)
I Sensor characteristics are usually not fully understood and therefore have to be
treated as random variables
3. Unknown causes: In several situations, the causal variables are unknown and/or
known, but cannot be measured or quantified.

Arun K. Tangirala Applied Time-Series Analysis 9

Motivation & Overview References

The “realization” challenge

A time-series is an ordered collection of random variables. Therefore, a single time-series

is one of the many possible combinations.

A single time-series is said to be a realization of the random process.

The collection of all possible realizations is said to be the ensemble.

Arun K. Tangirala Applied Time-Series Analysis 10

Motivation & Overview References

Example: Liquid level measurement

Consider measuring liquid level in a storage tank. Neglecting all other losses, the level is
constant. A single time-series (record) is a consequence of using one sensor to observe
the process. Readings from three di↵erent sensors are shown (true value shown in red).

Sensor 1 Sensor 2 Sensor 3

Arun K. Tangirala Applied Time-Series Analysis 11

Motivation & Overview References

The “realization” challenge . . . contd.

In practice, we have only a single realization. The challenge is to be able to
infer the truth from this single realization.

Two questions that repeatedly arise:

1. How good are the estimates (of model parameters, statistical properties, etc.) from
a single realization?
2. Is a single estimate sufficient?
Example: The average reading from sensor 1 is 3.005 and from sensor 3, is 2.996. What
can we say about the true level?

Arun K. Tangirala Applied Time-Series Analysis 12

Motivation & Overview References

A unifying question

Under what conditions can the inferences drawn from a single record of data be
meaningful and useful?

1. Stationarity: Invariance property of the process w.r.t. time (or space).

2. Ergodicity: Ability to replace ensemble averages with time (or spatial) averages.

Arun K. Tangirala Applied Time-Series Analysis 13

Motivation & Overview References

Approaches in TSA
Approaches depend on objective:

1. Forecasting: The most prevalent objective of TSA - applies to every field.

I Tests for within-series and across-series predictability - auto-correlation and
cross-correlation functions.
I Develop di↵erence equation models relating present to past - “auto-regressive”
models.
I Imagine the series to be a result of an unpredictable shock wave passing through a
filter - “moving average” models.
I Build mixed models that can also include other e↵ects such as trends, random walk
non-stationarities, etc. - (seasonal) ARIMA models.

Arun K. Tangirala Applied Time-Series Analysis 14

Motivation & Overview References

Approaches in TSA . . . contd.

2. Detection of periodicities: A common goal of many engineering, meteorological,

astronomical and biomedical applications.
I Frequency-domain or spectral representations - extensive use of Fourier
transforms (other transforms are also used in advanced analysis)
I Notion of a “random periodic process”

It is necessary to distinguish between a

periodic deterministic and periodic random process

Arun K. Tangirala Applied Time-Series Analysis 15

Motivation & Overview References

Periodic processes
I Periodic deterministic process
I Accurately predictable after one period.
e,g., ideal spring-mass system without any damping/friction and excited with an
impulse
I Periodic random process (harmonic processes):
I The process has an underlying periodicity coupled with some randomness
I Randomness could be in amplitude and/or in phase (if the signal is viewed as a sum
of sine waves)
I The power spectrum / power spectral density is a powerful concept that
allows us to study both these classes in a single framework.

Arun K. Tangirala Applied Time-Series Analysis 16

Motivation & Overview References

Estimation

Estimators are at the heart of TSA. They produce estimates of “unobserved” or

“hidden” quantities / variables from observations.

We shall learn how to:

I Characterize “goodness” of estimators

I Estimate statistical properties / parameters / signals
I Report estimation results and test hypotheses (on random processes)

Arun K. Tangirala Applied Time-Series Analysis 17

Motivation & Overview References

Scope of this course

Course deals with largely basic and a few advanced concepts. The objective is to equip
the learner with foundations of time-series analysis and estimation.

I Linear random processes

I Stationary and non-stationary processes.
I Mostly univariate and to a lesser extent, bivariate analysis
I Time-domain predictive models (ARMA, ARIMA and SARIMA models)
I Frequency-domain (spectral) analysis (deterministic and stochastic)
I Estimation theory (MoM, LS, MLE and Bayesian estimators and their properties)

Arun K. Tangirala Applied Time-Series Analysis 18

Motivation & Overview References

A few remarks

I Emphasis is on concepts rather than on the rigour.

I Idea is to introduce and illustrate ideas first through examples.
I Student is expected to take this first-level treatment to a higher level by self-inquiry.
I TSA requires both theory and skill, i.e., it is both a science and an art.
I A computational tool is indispensable to a good understanding and practice.

Arun K. Tangirala Applied Time-Series Analysis 19

Motivation & Overview References

R: Software for TSA

Rr is an integrated software package for data manipulation, calculation and graphical
display

I Powerful graphical (plotting) and statistical analysis tools.

I An expression language and like many other languages, case sensitive
I R provides the user with a very wide range of data structures (objects)
I Symbolic, vector, array, expressions, functions, lists, data frames, factors, . . .
I Each such data object can have attributes, names, dimensions, classes, etc.
I Writing user-defined and specialized add-on packages is easy
I Use of RStudior makes the use of R easy and useR-friendly!
Arun K. Tangirala Applied Time-Series Analysis 20
Motivation & Overview References

Bibliography

Box, G. E., G. M. Jenkins, and G. C. Reinsel (2008). Time Series Analysis: Forecasting and
Control. New York, USA: John Wiley & Sons, Inc.
Brockwell, P. (2002). Introduction to Time-Series and Forecasting. New York, USA: Springer-
Verlag.
Hamilton, J. D. (1994). Time Series Analysis. Princeton, NJ, USA: Princeton University Press.
Priestley, M. B. (1981). Spectral Analysis and Time Series. London, UK: Academic Press.
Shumway, R. and D. Sto↵er (2006). Time Series Analysis and its Applications. New York, USA:
Springer-Verlag.
Tangirala, A. K. (2014). Principles of System Identification: Theory and Practice. Boca Raton,
FL, USA: CRC Press, Taylor & Francis Group.

Arun K. Tangirala Applied Time-Series Analysis 21