Restriction on memory: Ergodicity

We have estimators of moments using time average under the

stationary assumption
Are these consistent?
The idea of ergodicity is related to the concept of convergence in
q
probability of (1/T ) T t=1 Yt (time average) to E (Yt ) for a stationary
process
A stationary process will be ergodic for the mean if the autocovariance
“j drops to zero sufficiently quickly as j becomes large (memory
persistence less)

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 37

Ergodicity

Definition: Ergodic for the mean

A covariance stationary process {Yt }Œ
t=≠Œ is said to be ergodic for the
q
mean if (1/T ) T Y
t=1 t converges in probability to µ as T æ Œ.

Definition: Ergodic for autocovariance

A stationary process is ergodic for autocovariance if
T
1 ÿ p
(Yt ≠ µ)(Yt≠j ≠ µ) æ “j
T ≠ j t=j+1

for all j.

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 38

Condition for ergodicity for the mean
Ergodicity allows us to use time average as a consistent estimator of
mean of a covariance stationary process
When autocovariance “j drops sufficiently quickly as j increases, the
process is called weakly dependent
Dependence between far away observations must be sufficiently small
Note: stationarity does not mean the process is completely
independent over time.
The sufficient condition for mean ergodicity is absolute summability of
“j .
Œ
ÿ
|“j | < Œ
j=0

For many applications, stationarity and ergodicity turns out to amount

to the same requirements (but not necessarily)
Example of a non-ergodic stationary process? [think of i th computer
generating random series Yti = µ(i) + Át where µ(i) are Át are two
independent white noise]
Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 39
Exercise

Check whether the following processes are stationary

I White Noise
I Moving Average: vt = 13 (wt≠1 + wt + wt+1 )
I Random walk process (both with and without drift)
I yt = – + —t + xt where xt is stationary
The last process has stationary behaviour around a linear trend.
Sometimes it is called trend stationarity

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 40

Scatter plot of lag data

Figure 8: Scatter plot of SOI - one month apart (left) and six months apart(right).
Estimated correlation in box.

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 41

Confidence band of Sample ACF

The formula for standard error of the sample ACF of a stationary series
becomes complicated
The sample ACF fl̂j of an i.i.d. white noise process {Yt }T
t=1 (with finite
fourth moments) is approximately Ô normally distributed with zero mean
and standard deviation ‡flˆj = 1/ T as T æ Œ
Ô
We use this result to find confidence band as ±2/ T for a white noise
sequence
Many statistical modelling reduces the series to a white noise process
through several transformations
Then plotted ACF of the residuals should lie roughly within this band
This gives a rough method of assessing autocorrelation

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 42

Exercise

Find theoretical ACF for the following series

yt = 3 + xt ≠ .6xt≠1

where I
+1 with probability 0.5,
xt =
≠1 with probability 0.5

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 43

Compare theoretical ACF with sample ACFs

We calculate sample ACF of data generated from the above process for
two sample sizes: {Y1t }10 100
t=1 and {Y2t }t=1
What difference do you observe when the sample sizes are different?
##
## Autocorrelations of series y1 , by lag
##
## 0 1 2 3 4
## 1.000 -0.673 0.425 -0.312 -0.003
##
## Autocorrelations of series y2 , by lag
##
## 0 1 2 3 4
## 1.000 -0.455 -0.005 -0.007 0.000

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 44

Plot of ACFs

Figure 9: Comparison of simulated ACF plots for different sample sizes(T=10 and
T=100)

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 45

Two series and their dependence
Sometimes, we have more than one stationary series and we are
interested in cross series relationship
We can define, in a very similar way, joint stationarity and cross
correlation function
Definition: Joint stationarity
Two time series {Xt } and {Yt } are jointly stationary if individually they are
stationary and the cross-covariance function “XY (j) = Cov (Xt+j , Yt ) is a
function only of the lag j

Definition: Cross-correlation function

Cross-correlation of two jointly stationary time series Xt and Yt is defined as

“XY (j)
flXY (j) = 
“X (0)“Y (0)

Note that usually it is not symmetric about zero,

i.e. flXY (h)
Sourabh B Paul
”= fl (≠h) Time Series Econometrics
(IIT Delhi) XY I Semester 2024-25 46
Exercise

Are the following two series jointly stationary?

Xt = wt + wt≠1 and Yt = wt ≠ wt≠1

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 47

Correlation of two series

We can plot sample joint correlation to analyse relationship between

two series
Example: SOI and the number of new fish (Recruitment)
Autocorrelation and cross-correlation functions (ACFs and CCF) for
these two series help us to understand the joint behaviour
ACFs depicts periodicities (12 months)

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 48

Sample CCF

Figure 10: Sample ACFs of the SOI series (top) and of the Recruitment series
(middle), and the sample CCF of the two series (bottom)

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 49

Confidence band of CCF plot

Sample CCF is approximately

Ô (for large sample) normal with mean zero
and variance 1/ T as long as at least one series is independent white
noise
We can construct 95% CI around the CCF
Often the series are not independent white noise
In that case sample CCF and confidence band would give wrong hint
about cross correlation
We convert one series to make it more like a white noise (prewhitening)
and plot CCF and Confidence Interval

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 50

Sample CCF of uncorrelated series

Figure 11: Top panel: two independent series, Middle panel: Sample ACF,
Bottom panel: Sample CCF of the series (left) and the sample CCF of the first
series with the prewhitened second series (right)

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 51

Multiple linear regression in a time series I

Can explore further cross relation between two series?

Classical linear regression is still very useful in time series context for
exploratory analysis or preprocessing nonstationary data
Some outcome variable (yt ) has linear (in parameter) relationship with
input variables (x1t , x2t , . . . , xkt ).

yt = X— + ut

X may include lagged values (xt≠1 , xt≠2 , etc.) as well

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 52

OLS in the context of TS data

The data is no longer a random sample - need to revise the underlying

assumptions for unbiasedness and consistency of OLS
To apply OLS either we assume fixed regressors or the error term in
any given period is uncorrelated with the explanatory variables in all
time periods
A sufficient condition for unbiasedness is that the conditional
expectation of error given all past, present and future x is zero, i.e.

E (ut |X) = 0, t = 1, 2, . . . , T

The above assumption is required for unbiasedness (strict exogeneity)

A milder version (contemporaneous exogeneity) is sufficient for
consistency (large sample property)

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 53

Contemporaneous and strict exogeneity of regressors

In time series model we assume strict exogeneity:

E (ut | x0 , x1 , ...xT ) = 0
Error at time t uncorrelated with all regressors in all time periods.
This assumption ensures unbiasedness of OLS
We assume two more assumptions for efficiency: Var (ut | X) = ‡ 2
Corr (ut , us | X) = 0, ’t ”= s

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 54

Different Models I
To explore cross series relationships, we have different models
Static model:

yt = —0 + —1 zt + ut , t = 1, 2, ..T

Linear trend model

yt = —0 + —1 t + ut

Finite Distributed Lag Models

yt = –0 + ”0 xt + ”1 xt≠1 + ”2 xt≠2 + ut

It could be a combination of trend and lags

We shall first explore techniques of univariate time series modelling and
later, more sophisticated multivariate time series modelling
Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 55
Home work

Before we start building the models of TS, we must have basic knowledge of
OLS
Review OLS
You should be familiar with the following:
I Derivation of OLS estimator,
I ANOVA (different sum of squares and their interpretations),
I R 2 , Adjusted R 2 ,
I degrees of freedom,
I standard error, p-value, t-stat, F-stat,
I hypothesis testing

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 56

 Section 3

R primer for TS

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 57

Time Series Data Types in R

Explicitly define structure of data for calculations specific in time series

context: lags, trend, seasonality, differencing, etc.
Most important distinction: whether the data is equispaced or irregular
Convenient way to deal with equispaced time series in R is to store
them as ts object

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 58

Import data I

Create a data in Excel: First column “year” (fill from 1981 to 2000),
second column “value” (fill with any numbers).
You can give any name to the data columns but remember it.
Save the excel file as “csv” (note the location).
Open R Studio. Start a new R script (File > New File > R Script))
Write the following code (change the csv file path and data column)

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 59

Import data II

production <- [Link]([Link](

"~/Dropbox/TimeSeriesClass/data/[Link]",
header = TRUE))
# Line starts with hash (#) is a comment.
# R will not execute the line
# Explanations:
# We first import the csv file and create
# a data frame. Then assign a name to it
# header=TRUE means, the first row of
# the csv file has the names of the variables
# Right hand side of "<-" sign is
# doing two things: first importing
# the csv file and then making a
# data frame (special way of storing data in R)

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 60

Import data III

# Left hand side of <- is assigning a

# name to the imported data ("production").
# You can give any name. Now data is
# in R memory. We refer it by "production".
# It is not saved in your computer though.

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 61

Import data IV

[Link] <- ts(production$value,

start = c(1981),
frequency = 1)
# Here we declare that the "value"
# variable (column) of the production
# data frame (production$value)
# is a time series, which
# starts at 1981 and frequency 1
# means it is annual data.
# We give a new name ("[Link]").
# You can give any name,
# or replace the original "production"
# data frame by assigning the same name

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 62

Plot

plot([Link], ylab="Production(in Rs crore)",

main="Production of Wheat")
# Here we plot the data

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 63

Plot

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 64

Import another data

# We specify the path of data folder

# in a variable.
# So that we do not have to type again and again
datafolder <- "/Users/SBP/Dropbox/TimeSeriesClass/data/"
barium <- [Link]([Link](
[Link](datafolder,
"[Link]")))
[Link] <- ts(barium$chnimp,
start = c(1978,2),
frequency = 12)

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 65

Plot

plot([Link], ylab="Import",
main="Import of Barium
Chloride from China")

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 66

Plot

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 67

Irregular Time Series in R I

Irregular time series is important in finance (daily stock data)

Two important packages deal with this: xts and zoo
intdef <- [Link]([Link](
paste(dropboxpath,
"/CEP2017/R_example/[Link]",
sep = "")))
#Variable "year" is the time variable
intdef$year

## [1] 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958
## [16] 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973
## [31] 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988
## [46] 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 68

Irregular Time Series in R II

# Define zoo object

#[Link]("zoo", repo = getOption("repos"))
library("zoo")
[Link] <- zoo(intdef,
[Link] = intdef$year)
plot([Link]$i3,
main="Interest rate", ylab = "Interest rate")

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 69

Irregular Time Series in R III

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 70

Downloads publicly available data I

pdfetch package downloads publicly available data. You can also use
quantmod.
In this example we download yahoo finance data

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 71

Downloads publicly available data II
#[Link]("pdfetch", getOption("repos"))
#library("pdfetch")
# Check [Link] for ticker symbols
# Fileds: "open", "high", "low", "close", "volume", "adjclose"
#tickers <- c("ˆIXIC", "ˆGSPC","AAPL", "ˆNSEI")
#yahoo_data <- pdfetch_YAHOO(tickers,
# fields="adjclose",
# from="2016-01-01")
# If end date is not given, then by default till today
# [Link]()
# nrow(yahoo_data)
# head(yahoo_data)
# tail(yahoo_data)
library(quantmod)
yahoo_data <- getSymbols("AAPL",
from="2006-01-01",
src="yahoo",
Sourabh B Paul (IIT Delhi) [Link]=FALSE)
Time Series Econometrics I Semester 2024-25 72
The dynlm package

Package dynlm gives convenient ways to include several time series

operator directly into the model
L(x): Variable x lagged by one time unit, xt≠1
L(x, k): Variable x lagged by k time unit, xt≠k (The order could be
vector like (0:3))
d(x): First difference xt ≠ xt≠1
trend(x): Linear time trend
season(x): Seasonal effects

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 73

Static model: Effect of inflation and deficit on
interest rate I

# Linear regression of static model

fit.i3 <- lm(i3 ~ inf+def, data=intdef)
summary(fit.i3)

term estimate [Link] statistic [Link]

(Intercept) 1.7332658 0.4319670 4.012496 0.0001898
inf 0.6058659 0.0821348 7.376481 0.0000000
def 0.5130579 0.1183841 4.333843 0.0000657

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 74

Linear trend model I

Load chicken price data

chickenprice <- [Link](
[Link]([Link](
datafolder, "[Link]"),
colClasses = c("NULL", NA)))
chickenprice <- ts(chickenprice,
start = c(2001,8),
frequency = 12)

Fit a linear trend model

# Trend model of Price of chicken
fit1 <- lm(chickenprice ~ time(chickenprice),
[Link]=NULL)
# Display the summary of the fit
summary(fit1)

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 75

Linear trend model II
##
## Call:
## lm(formula = chickenprice ~ time(chickenprice), [Link] =
##
## Residuals:
## Min 1Q Median 3Q Max
## -8.7411 -3.4730 0.8251 2.7738 11.5804
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -7.131e+03 1.624e+02 -43.91 <2e-16 *
## time(chickenprice) 3.592e+00 8.084e-02 44.43 <2e-16 *
## ---
## Signif. codes: 0 *** 0.001 ** 0.01 * 0.05 . 0.1
##
## Residual standard error: 4.696 on 178 degrees of freedom

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 76

Linear trend model III

## Multiple R-squared: 0.9173, Adjusted R-squared: 0.9168

## F-statistic: 1974 on 1 and 178 DF, p-value: < 2.2e-16

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 77

Fit linear trend of interest rate

We fit linear trend of interest rate

intdef <- [Link]([Link](
[Link](datafolder, "[Link]")))
fit2 <- lm(i3 ~ year, data=intdef,
[Link]=NULL)

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 78

Plot trends I

# Plot two trends side by side

par(mfrow=c(1,2))
plot(chickenprice, ylab="Price of chicken")
abline(fit1)
plot(intdef$year, intdef$i3, ylab="Interest rate")
abline(fit2) # add the fitted line

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 79

Plot trends II

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 80

Example: Water level in a lake

Level of Lake Huron 1875–1972 (Brockwell and Davis)

lake level appears to decline at a roughly linear rate
We fit a trend model using OLS

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 81

Fit a linear trend model

lake <- [Link]([Link](

[Link](datafolder, "[Link]"), header = FALSE))
lake <- ts(lake,
start = c(1875),
frequency = 1)
# Trend model
[Link] <- lm(lake ~ time(lake),
[Link]=NULL)

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 82

Plot the trend
plot(lake, type="l", ylab="Water level")
abline([Link])

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 83

Detrended data - residual plot
plot([Link]$residuals, type="o", ylab = "Residuals")
abline(a=0, b=0)

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 84

What do you observe from the residual plot?

Absence of any discernible trend

Smoothness of the plot implies some form of dependence among the
observations
We estimate the Sample ACF to find dependence

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 85

Analysing residuals

We now see whether residuals have some smoothness left

We check autocorrelation (gives smoothness) of the residuals

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 86

Calculating ACF

acf([Link]$residuals, [Link]=6, plot=FALSE)

##
## Autocorrelations of series [Link]$residuals , by lag
##
## 0 1 2 3 4 5 6
## 1.000 0.762 0.464 0.261 0.140 0.080 0.034

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 87

Plot of ACF
acf([Link]$residuals, [Link]=6, plot=TRUE,
main="Autocorrelation plot of detrended data")
legend( topright , legend="ACF of residuals")

Figure 12: ACF plots of detrended lake water level

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 88

Cross correlation

R has nice command to find cross correlation of two series.

Import the SOI and Recruitment of fish data.
Plot the raw data.
Calculate ACF and CCF.

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 89

Import the data I

[Link] <- [Link]([Link](

"~/Dropbox/TimeSeriesClass/data/[Link]",
sep = ","))
[Link] <- [Link]([Link](
"~/Dropbox/TimeSeriesClass/data/[Link]",
sep = ","))
#Variable "index" is the time variable
head([Link], 2)

## index value
## 1 1950 Jan 0.377
## 2 1950 Feb 0.246

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 90

Import the data II

# Define zoo object

[Link] <- zoo([Link]$value,
[Link] = [Link](
[Link]$index, format="%Y %b"))
[Link] <- zoo([Link]$value,
[Link] = [Link](
[Link]$index, format="%Y %b"))

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 91

Plot SOI and recruitment I
par(mfrow = c(2,1)) # set up the graphics
plot([Link], ylab="", xlab="",
main="Southern Oscillation Index")
plot([Link], ylab="", xlab="",
main="Recruitment(number of new fish)")

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 92

Plot SOI and recruitment II

Figure 13: Monthly SOI and Recruitment (estimated new fish), 1950-87

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 93

Sample CCF
ccf([Link], [Link], 48,
main="SOI(t+j) vs Recruitment(t)",
ylab="CCF")

Figure 14: Sample CCF of the SOI and Recruitment series

Sourabh B Paul (IIT Delhi) Time Series Econometrics I Semester 2024-25 94