1 Linear Probability Model (LPM)

The LPM represents a regression model where the dependent variable is

binary. The model is given by:

Yi = β1 + β2 Xi + ui (1)

where Yi takes values 1 or 0.

Properties of LPM
1. E(Yi |Xi ) = β1 + β2 Xi , which is the conditional probability P (Yi =
1|Xi ).
2. Disadvantages of LPM:
• Non-normality of the error term ui .
• Heteroskedasticity in ui .
• Predicted probabilities can fall outside the range [0,1].
• R2 may not be a meaningful measure of fit.

2 Logit Model
The logit model uses the logistic distribution function for modeling binary
outcomes:
1
Pi = , Zi = β1 + β2 Xi (2)
1 + e−Zi
The log of the odds ratio is linear:
 
Pi
Li = ln = Zi = β1 + β2 Xi (3)
1 − Pi

Properties of Logit Model

1. Probabilities are bounded between 0 and 1.
2. Nonlinear in probabilities but linear in log-odds.
3. Slope coefficient β2 represents the change in log-odds for a unit change
in X.
4. Logit model accommodates multiple regressors.

1
 Form of Y Form of X Interpretation
Y X When X changes by 1 unit, Y will change by β1 units.
ln Y ln X When X changes by 1%, Y will change by β1 % (elasticity).
Y ln X When X changes by 1%, Y will change by β1 /100 units.
ln Y X When X changes by 1 unit, Y will change by (β1 × 100)%.

Table 1: Interpretation of Regression Coefficients

3 Probit Model
The probit model assumes a cumulative normal distribution for binary out-
comes:
P (Y = 1|X) = Φ(β1 + β2 X) (4)
where Φ is the cumulative distribution function (CDF) of the standard nor-
mal distribution: Z X
1 t2
Φ(X) = √ e− 2 dt (5)
−∞ 2π

Properties of Probit Model

1. Probabilities are bounded between 0 and 1.

2. Nonlinear relationship between X and probabilities.

3. Typically used when normality of the underlying distribution is as-

sumed.

Summary of Time Series Models and Their

Properties
White Noise (WN)
• Definition: A purely random process with constant mean and variance
and no autocorrelation.

• Equation: ut ∼ IIDN(0, σ 2 ).

• Properties:

– Mean is constant over time.

– Variance is constant over time.

2
 – ACF is zero at all lags except lag 0.

Model LPM Logit Probit

Direction + Significance Coefficients indicate Coefficients indicate direc- Coefficients indicate direc-
direction and signifi- tion and significance, but tion and significance, but
cance of X’s effect on not directly the marginal ef- not directly the marginal ef-
P (Y = 1|X). fect on P (Y = 1|X). fect.
Marginal Effect Each coefficient repre- The marginal effect must be The marginal effect must be
sents the marginal ef- computed since coefficients computed using the cumula-
fect of X on P (Y = show the effect on ln(odds): tive standard normal distri-
1|X). bution.
ln(odds) = Z

odds ratio = eZ

Table 2: Comparison of LPM, Logit, and Probit Models

Autoregressive (AR) Model

• Definition: The AR model expresses the current value of a time series
as a linear function of its past values and a stochastic error term.

• Equation: Yt = ϕ1 Yt−1 + ϕ2 Yt−2 + . . . + ϕp Yt−p + ut .

• Properties:

– Stationarity depends on the roots of the characteristic equation

being outside the unit circle.
– The autocorrelation function (ACF) decays gradually.
– The partial autocorrelation function (PACF) cuts off after lag p.

Moving Average (MA) Model

• Definition: The MA model expresses the current value of a time series
as a linear function of past error terms.

• Equation: Yt = ut + θ1 ut−1 + θ2 ut−2 + . . . + θq ut−q .

• Properties:

– Always stationary.
– The ACF cuts off after lag q.
– The PACF decays gradually.

3
Autoregressive Moving Average (ARMA) Model
• Definition: Combines the AR and MA models to explain a time series
using both past values and past error terms.

• Equation: Yt = ϕ1 Yt−1 + . . . + ϕp Yt−p + ut + θ1 ut−1 + . . . + θq ut−q .

• Properties:

– Stationarity depends on the AR part.

– ACF and PACF both exhibit mixed patterns (do not abruptly cut
off).
– Suitable for stationary time series.

Vector Autoregressive (VAR) Model

• Definition: A generalization of the AR model to multiple time series,
where each variable is explained by its own lags and the lags of other
variables.

• Equation: Yt = Φ1 Yt−1 + . . . + Φp Yt−p + ut .

• Properties:

– Captures the dynamic relationship between multiple time series.

– Requires stationarity of all variables.
– ACF and PACF are calculated for each variable in the system.

Notes on Stochastic Processes

Stochastic Processes
A stochastic process is a collection of random variables ordered in time.

Stationary Stochastic Process

A process is stationary if:

• Mean and variance are constant over time.

• Covariance depends only on the lag (distance) between two time peri-
ods, not the actual time.

4
Key Properties:
• Mean Reversion: The series tends to return to its mean over time.
• Constant Fluctuations: Variance remains stable, indicating consis-
tent amplitude of fluctuations.
Importance of Stationarity:
• Nonstationary series are specific to the observed time period and un-
suitable for generalization or forecasting.

White Noise Process

A process is purely random (white noise) if:
• Mean = 0
• Variance = σ 2 (constant)
• Serially uncorrelated: ut ∼ IIDN (0, σ 2 ), meaning the terms are inde-
pendently and identically distributed with a normal distribution.

Random Walk Models

Random Walk Without Drift:
• Equation: Yt = Yt−1 + ut , where ut is white noise.
• Properties:
– Mean: E(Yt ) = Y0
– Variance: Var(Yt ) = tσ 2 (depends on time → nonstationary).
– Persistence of Shocks: Random shocks accumulate and do not
dissipate.
– First Difference: ∆Yt = ut , which is stationary.
Random Walk With Drift:
• Equation: Yt = δ + Yt−1 + ut
• δ: Drift parameter, representing a deterministic trend.
• Properties:
– Mean: E(Yt ) = Y0 + t · δ
– Variance: Var(Yt ) = tσ 2 (nonstationary).
– First Difference: ∆Yt = δ + ut , which is stationary.

5
Key Concepts
• Random walks (with or without drift) are nonstationary processes.
• Random walks exhibit stochastic trends.

Types of Trends
Deterministic Trend:
• Predictable and constant over time.
• Equation: Yt = β1 + β2 t + ut
• Subtracting the trend (β1 + β2 t) results in a stationary series. This
process is called detrending.
Stochastic Trend:
• Unpredictable and nonstationary.
• Found in random walks with or without drift.

Integrated Processes
A stochastic process requiring differencing d times to achieve stationarity is
said to be integrated of order d, denoted as I(d).
• Example:
– I(0): Stationary time series.
– I(1): Requires first differencing.
– I(2): Requires second differencing.

Autocorrelation Function (ACF) and Correlogram

Autocorrelation Coefficient ρk :
• Measures correlation between observations separated by k lags.
• Formula: ρk = Covariance at lag k
Variance

• Range: −1 ≤ ρk ≤ 1.
Correlogram:
• A plot of ρk against k (lags).
• High and slowly decaying ρk : Indicates nonstationarity.

6
Testing Autocorrelation Significance
Box–Pierce Q Statistic:

• Tests whether all autocorrelations up to lag m are zero.

• Approximation: Q ∼ χ2 (m).

Ljung–Box (LB) Statistic:

• Variant of Q statistic with better small-sample properties.

• LB ∼ χ2 (m).

Notes on Unit Root Test and Stationarity

Unit Root Test
• Definition: The unit root test checks if a time series is stationary or
nonstationary.

• Key Concept: Start with the equation:

Yt = ρYt−1 + ut where − 1 ≤ ρ ≤ 1

If ρ = 1, the equation becomes a random walk model without drift,

which is nonstationary.

• Reformulation: Subtract Yt−1 from both sides:

Yt − Yt−1 = (ρ − 1)Yt−1 + ut

Let δ = (ρ − 1), then:

∆Yt = δYt−1 + ut

– Null Hypothesis (H0 ): δ = 0 (time series has a unit root and is

nonstationary).
– Alternative Hypothesis (H1 ): δ < 0 (time series is stationary).

7
Dickey-Fuller (DF) Test
• If H0 is true (δ = 0), the t-statistic for the coefficient of Yt−1 in the
regression follows the τ -statistic distribution (critical values available
in specialized tables).

• Variations of the DF Test:

1. Random walk without drift:

∆Yt = δYt−1 + ut

2. Random walk with drift:

∆Yt = β1 + δYt−1 + ut

3. Random walk with drift and trend:

∆Yt = β1 + β2 t + δYt−1 + ut

Augmented Dickey-Fuller (ADF) Test

• Accounts for serial correlation in the error term by adding lagged dif-
ferences of Yt :
m
X
∆Yt = β1 + β2 t + δYt−1 + αi ∆Yt−i + ϵt
i=1

• The number of lagged terms (m) is determined empirically to ensure ϵt

is white noise.

• Null Hypothesis (H0 ): δ = 0 (unit root present).

F-Test for Joint Significance

• Test if both β1 = β2 = 0 (i.e., random walk without drift or trend):

1. Compare the restricted model (without intercept or trend) with

the unrestricted model.
2. Use specialized F-distribution critical values for inference.

8
The F-statistic is given by the equation:
SSR
Explained Variation per Degree of Freedom k−1
F = = SSE
(6)
Unexplained Variation per Degree of Freedom n−k

where:

• SSR: Sum of Squares for Regression

• SSE: Sum of Squares for Error

• k: Number of parameters in the model

• n: Number of observations

Difference-Stationary Processes (DSP)

• A time series with a unit root can be made stationary by taking first
differences:
∆Yt = Yt − Yt−1

Trend-Stationary Processes (TSP)

• A TSP is stationary around a deterministic trend.

• To make a TSP stationary:

1. Regress Yt on time (t).

2. Use the residuals, which are stationary.

Notes on Cointegration and Error Correction

Mechanism
Cointegration
If two time series X and Y are nonstationary, and we can find a linear
combination of them that is stationary, then we can run a regression, and it
will not be spurious.

9
Engle–Granger (EG) or Augmented Engle–Granger (AEG) Test
• The DF or ADF unit root tests can be applied by estimating a regres-
sion of the form:
Yt = β1 + β2 Xt + ut ,
obtaining the residuals ut , and applying the DF or ADF tests to ut .

• Since the residuals are based on the estimated cointegrating parameter

β2 , the standard DF and ADF critical values are not quite appropriate.
Engle and Granger have provided critical values for these tests.

• Modern software packages often include these critical values in their

outputs.

Cointegrating Regression Durbin–Watson (CRDW) Test

• A quicker method to test for cointegration is the CRDW test.

• Critical values for this test were provided by Sargan and Bhargava.

Error Correction Mechanism (ECM)

• If two variables, such as P CE (Personal Consumption Expenditure)
and P DI (Personal Disposable Income), are cointegrated, there exists
a long-term equilibrium relationship between them.

• In the short run, disequilibrium may occur. The error term in the
cointegrating equation can be treated as the equilibrium error.

• The ECM, first introduced by Sargan and later popularized by Engle

and Granger, corrects for disequilibrium:

∆Yt = α(Yt−1 − β1 − β2 Xt−1 ) + γ∆Xt + ϵt .

• The Granger representation theorem states that if two variables are

cointegrated, their relationship can be expressed as an ECM.

Vector Autoregressive (VAR) Methodology

• The VAR methodology resembles simultaneous-equation modeling but
with differences:

10
 – Each endogenous variable is explained by its own lagged values
and the lagged values of all other endogenous variables in the
model.
– Typically, there are no exogenous variables in the model.

Summary and Conclusions

1. Regression analysis based on time series data implicitly assumes that
the underlying time series are stationary. The classical t tests, F tests,
etc., are based on this assumption.

2. In practice, most economic time series are nonstationary.

3. A stochastic process is said to be weakly stationary if its mean, vari-

ance, and autocovariances are constant over time (i.e., they are time-
invariant).

4. At the informal level, weak stationarity can be tested by the correlo-

gram of a time series, which is a graph of autocorrelation at various
lags. For stationary time series, the correlogram tapers off quickly,
whereas for nonstationary time series it dies off gradually. For a purely
random series, the autocorrelations at all lags 1 and greater are zero.

5. At the formal level, stationarity can be checked by finding out if the

time series contains a unit root. The Dickey–Fuller (DF) and aug-
mented Dickey–Fuller (ADF) tests can be used for this purpose.

6. An economic time series can be trend stationary (TS) or difference

stationary (DS). A TS time series has a deterministic trend, whereas
a DS time series has a variable, or stochastic, trend. The common
practice of including the time or trend variable in a regression model
to detrend the data is justifiable only for TS time series. The DF and
ADF tests can be applied to determine whether a time series is TS or
DS.

7. Regression of one time series variable on one or more time series vari-
ables often can give nonsensical or spurious results. This phenomenon
is known as spurious regression. One way to guard against it is to find
out if the time series are cointegrated.

8. Cointegration means that despite being individually nonstationary, a

linear combination of two or more time series can be stationary. The

11
 Engle–Granger (EG), Augmented Engle–Granger (AEG), and Cointe-
grating Regression Durbin–Watson (CRDW) tests can be used to find
out if two or more time series are cointegrated.

9. Cointegration of two (or more) time series suggests that there is a long-
run, or equilibrium, relationship between them.

10. The error correction mechanism (ECM) developed by Engle and Granger
is a means of reconciling the short-run behavior of an economic variable
with its long-run behavior.

11. The field of time series econometrics is evolving. The established results
and tests are in some cases tentative, and a lot more work remains. An
important question that needs an answer is why some economic time
series are stationary and some are nonstationary.

12