Applied Econometrics
Introduction to Time Series
Roman Horvth
Lecture 3
Contents
Stationarity
What it is and what it is for
Some basic time series models
Autoregressive (AR)
Moving average (MA)
Consequences of non-stationarity (spurious
regression)
Testing for (non)-stationarity
Dickey-Fuller test
Augmented Dickey-Fuller test
2
(Weak) Stationarity
Xt is stationary if:
the series fluctuates around a constant long run
mean
Xt has finite variance which is not dependent upon time
Covariance between two values of Xt depends only on the
difference apart in time (e.g. covariance between Xt and Xt-1 is
the same as for Xt-8 and Xt-9)
E(Xt) =
(mean is constant in t)
Var(Xt) = 2
(variance is constant in t)
Cov(Xt ,Xt+k) = (k) (covariance is constant in t)
If data not stationary, spurious regression problem
Examples of Times Series Models
AR autoregressive models
Xt = + *Xt-1 + ut is called AR(1) process
Xt = + 1*Xt-1 + 2*Xt-2 +.+k*Xt-k + ut is AR(k) process
MA moving average models
Xt = + ut + 2*ut-1 is called MA(1) process
Xt = + ut + 2*ut-1 +.+k*ut-k .is called MA(k) process
If you combine AR and MA process, you get
ARMA process
E.g. ARMA (1,1) is Xt = +*Xt-1+ut+ 2 ut-1
4
Is MA and AR process stationary?
Compute mean, variance and covariance and
check if it depends on time
For AR process, you may easily derive that the
process is stationary if < 1
For MA (1) process, mean is , variance is ut2
*(1+22) and covariance cov(Xt , Xt-k) is either 0
if k>1 or ut2*2 ,so it does not depend on time
(MA(k) is stationary process)
5
Example of Stationary Time Series
White noise process:
Xt = ut
ut ~ IID(0, 2 )
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
Another Example of Stationary Time Series
Xt = 0.5*Xt-1 + ut
ut ~ IID(0, 2 )
Stationary without drift
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
7
Example of Non-stationary Time Series
Yt = + *t + ut , where t is time trend
Take the expected value E(Yt ) = + *t ,
clearly the mean depends on time and the series
is non-stationary
Non-stationary time series
In contrast a non-stationary time series has at least
one of the following characteristics:
Does not have a long run mean which the series
returns
Variance is dependent upon time and goes to
infinity as the sample period approaches infinity
Correlogram does not die out - long memory
Example of Non-stationary Time Series
2003 Q1
2002 Q2
2001 Q3
2000 Q4
2000 Q1
1999 Q2
1998 Q3
1997 Q4
1997 Q1
1996 Q2
1995 Q3
1994 Q4
1994 Q1
1993 Q2
1992 Q3
120
100
80
60
40
20
0
UK GDP Level
The level of GDP is not constant; the mean increases over
10
time.
Non-stationary time series correlogram
UK GDP (Yt)
1.0
ACF-Y
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
For non-stationary series the Autocorrelation Function (ACF)
declines towards zero at a slow rate as k increases.
11
Possible solutions of non-stationarity
2003 Q1
2002 Q2
2001 Q3
2000 Q4
2000 Q1
1999 Q2
1998 Q3
1997 Q4
1997 Q1
1996 Q2
1995 Q3
1994 Q4
1994 Q1
1993 Q2
1.4
1.2
1
0.8
0.6
0.4
0.2
0
1992 Q3
Some transformation = first differenece, logarithm, second
difference ...
First difference of UK GDP (Yt = Yt - Yt-1) is stationary:
- growth rate is reasonably constant through time
- variance is also reasonably constant through time
12
Stationary time series - correlogram
UK GDP Growth ( Yt)
1.00
ACF-DY
0.75
0.50
0.25
0.00
-0.25
-0.50
-0.75
ACF decline towards zero as k increases
Decline of ACF is rapid for stationary series
13
Non-stationary Time Series Continued
Random Walk
Xt = Xt-1 + ut , where ut ~ IID(0, 2 )
Mean is constant in t: E(Xt) = E(Xt-1)
X1 = X0 + u1
(take initial value X0)
X2 = X1 + u2 = (X0 + u1 ) + u2
Xt = X0 + u1 + u2 ++ ut (take expectations)
E(Xt) = E(X0 + u1 + u2 ++ ut) = E(X0) = constant
Variance is not constant in t:
Var(Xt) = Var(X0) + Var(u1) ++ Var(ut)
= 0 + 2 ++ 2 = t 2
14
Random walk
Xt = Xt-1 + ut
ut ~ IID(0, 2 )
2.5
2
1.5
1
0.5
0
15
Relationship between stationary and nonstationary process
AR(1) process: Xt = + Xt-1 + ut ;(ut ~ IID(0, 2 ))
< 1stationary process - process forgets
past
= 1non-stationary process - process does
not forget past
= 0 without drift
0 with drift
MA process is always stationary
16
Summary on basic time series processes
AR (k) process
MA(k) process
ARMA (p,l) process
If you k-th difference the data, then you have
ARIMA (p,k,l) estimation of ARIMA models
is a subject of next lecture
17
Spurious Regression
( spurious correlation)
Problem that time-series data usually includes trend
Result:
Spurious correlation (variables with similar trends
are correlated)
Spurious regression (independent variable with
similar trend looks as dependent = strong statistical
relationship)
coefficient significant (high adjusted-R2, large
t-statistics) ... even if unrelated in economic
terms
18
How to avoid spurious regression:
3 approaches to non-stationarity
1.
Include a time trend as an independent variable (oldfashioned)
yt = c + 1xt + 2t + ut .....(t = 1,2, ..., T)
2.
1st difference the data if variables I(1); 2nd difference if
I(2)
= converts non-stationary variables into stationary
variables
Problems:
theory often about levels
detrending loss of information
3.
Cointegration + ECM
= Long-run relationship + short-run adjustment
19
How do we identify non-stationary
processes?
(A) Informal methods:
- Plot time series
- Correlogram
(B) Formal methods:
- Statistical test for stationarity
- Dickey-Fuller tests.
20
Informal Procedures to identify non-stationary
processes
(a) Constant mean?
12
RW2
10
0
0
50
100
150
200
250
300
350
400
450
500
(b) Constant variance?
var
200
150
100
50
0
-50
-100
-150
21
-200
0
50
100
150
200
250
300
350
400
450
500
Informal Procedures to identify non-stationary
processes
Diagnostic test Correlogram for stationary process (dies out
rapidly, series has no memory)
0.50
whitenoise
0.25
0.00
-0.25
0
1.0
50
100
150
200
250
300
350
400
450
500
ACF-whitenoise
0.5
0.0
-0.5
10
22
Informal Procedures to identify non-stationary
processes
Diagnostic test Correlogram for a random walk (does not die out,
high autocorrelation for large values of k)
12.5
randomwalk
10.0
7.5
5.0
2.5
0.0
0
1.00
50
100
150
200
250
300
350
400
450
500
ACF-randomwalk
0.75
0.50
0.25
10
23
Dickey-Fuller Test
Test based on Yt = Yt-1 + ut
- DF test to determine whether =1
Yes unit root non-stationary
No no unit root
Dynamic model:
Subtract Yt-1 ... Yt - Yt-1 = (-1)Yt-1 + ut
Reparameterise: Yt = Yt-1 + ut
where = (-1)
Test =0 equivalent to test =1
24
Augmented Dickey-Fuller Test
Augment dynamic model Yt = Yt-1 + ut:
1) Constant or drift term (0)
Yt = 0 + Yt-1 + ut
2) Time trend (T)
Yt = 0 + T + Yt-1 + ut
3) Lagged values of the dependent variable
Yt = 0 + T + Yt-1 + 1Yt-1 + 2Yt-2 + ... + ut
Find the right specification, trade-off parsimony vs. white
noise in residual
25
Critical values
DF/ADF use t- and F-statistics but critical values are
not standard
Problems:
distributions of these statistics are non-standard
special tables of critical values (derived from
numerical simulations)
Usual t- and F-tests not valid in presence of unit
roots
26
