Chapter 2: Fundamental Concepts Partial autocorrelation function (PACF)

Time series and stochastic processes

Terminology:

Difference between Autocorrelation and Partial Autocorrelation

Think of Autocorrelation (ACF) and Partial Autocorrelation (PACF) as ways to measure how past
values of a time series influence the present.
Stochastic process:
● Autocorrelation (ACF) – Total Influence
A stochastic process is a mathematical model that describes the evolution of a system or a phenomenon
ACF tells you how much a past value (lag) affects the current value, including indirect effects.
over time in a probabilistic manner. It is a collection of random variables indexed by time, where a
probability distribution determines the values of the variables at each time point. In simpler terms, it is
7 Imagine a chain reaction: If yesterday’s weather affects today, and the day before yesterday
influenced yesterday, then the day before yesterday also indirectly affects today. ACF captures
a sequence of random events or observations that unfold over time.
both direct and indirect relationships.
Autocorrelation (ACF) and Partial Autocorrelation Function (PACF)
Autocorrelation (ACF) and partial autocorrelation (PACF) are statistical measures that help analyze the ACF is a statistical test that is commonly used to assess the presence of seasonality in TS.
relationship between a time series and its lagged values. ● Partial Autocorrelation (PACF) – Direct Influence
In R Programming Language, the acf() and pacf() functions can be used to compute and visualize PACF tells you only the direct effect of a past value on the present, ignoring indirect influences.
autocorrelation and partial autocorrelation, respectively. 7 It removes the "middleman." If we want to know how the weather three days ago affects
Autocorrelation Function (ACF) today without yesterday’s influence, PACF isolates that direct relationship.

Key Difference:
● ACF shows how past values influence the present, including indirect effects.
● PACF isolates the direct impact of a specific lag by removing intermediate influences.

Some (named) stochastic processes:

White noise (residuals) process:
A white noise process in time series is a sequence of completely random variables with no pattern. The
variables are independent and identically distributed (iid). It’s like flipping a fair coin or rolling a fair
die—each outcome is independent of the previous ones.
Key Characteristics of White Noise: (L.I.N.E)
Definition: for a set of time series data Y1, Y,2, …, YT, we define the sample autocorrelation function, at lag 1. Zero Mean: The average value is around zero.
k, by: 2. Constant Variance: The spread of values stays the same over time
3. No Autocorrelation: Past values do not influence future values—each observation is completely
random.

The Autocorrelation function of white noise:

Where Y bar is the sample mean of Y1, Y,2, …, YT pk= { 1, k=0
Rk is a point estimate of the true population autocorrelation pk 0, k=1,2,..
Random walk process: Moving average(MA) process
A random walk process is a time series where each new value is just the previous value plus some In time series analysis, a Moving Average (MA) process is a fundamental stochastic model used
random change. to describe time-dependent data. It expresses the current value of a time series as a linear
Y1 = e 1 combination of past white noise (error) terms. (measures trend when it is linear)
Y2 = e 1 + e 2
Suppose that {et} is a zero mean white process with Var(et)=std².
… 1
Define, 𝑌𝑡 = (𝑒𝑡 + 𝑒𝑡−1 + 𝑒𝑡−2), that is a running(moving) average of the white noise process
Yn= e 1 + e 2 + … + en 3

Mathematically, it's written as: (averaged across the most recent 3 times)
Yt = Yt-1 + ϵt for t > 1 Autocorrelation function
where:
● Yt is the value at time t.
● Yt-1 is the value at the previous time step.
● ϵt is a random number (white noise).

Ps: a random walk is similar to a "suite algébrique" (arithmetic sequence), but instead of adding a fixed Autoregessive (AR) process
r, we add a random value ϵt, which changes at every step. An Autoregressive (AR) process is a fundamental time series model where the current value depends
linearly on its past values and a random noise term.
The Autocorrelation function of a random walk process: Suppose that {et} is a zero mean white process with Var(et)=std².
Consider the stochastic process defined by: Yt = 0.75 Yt-1 + et
that is,Yt is directly related to the (downweighted) previous value of the process Yt-1 and the random
- Note that when k is closer to 0, the autocorrelation Corr(Yt,Yt−k) is closer to 1. That is, two error et (a "shock" or "innovation" that occurs at time t). This is called an autoregressive model.
observations Yt and Yt−k close together in time are likely to be close together, especially when t and Autoregression means "regression on itself." Essentially, we can envision "regressing" Yt on Yt-1.

(t−k) are both large (later on in the series). Stationarity

A time series is stationary if its statistical properties (such as mean and variance) do not change over
- On the other hand, when k is far away from 0 (that is, for two points Yt and Yt−k far apart in time), the
time. This makes it easier to analyze and forecast.
autocorrelation is closer to 0.
Characteristics of a Stationary Time Series
Uses of random walk process:
A time series is considered stationary if:
Random walk processes are used in various fields to model systems that evolve in unpredictable or
1. There is no trend (the mean remains constant over time).
random ways. Here's how it’s applied:
2. The variance remains stable (fluctuations do not grow or shrink over time).
1. Stock prices: Stock prices often exhibit random movement, where the future price is influenced by
3. There are no periodic patterns (seasonal effects have been removed).
random factors like market news, trends, and investor sentiment. A random walk suggests that the
Mathematical Definition :
next price is a random fluctuation around the current price, making future price predictions difficult.
The stochastic process {Yt : t = 0, 1, 2, ..., n} is said to be stationary (or weakly stationary) if:
2. Movements of molecules in gases and liquids: In physics, molecules in gases or liquids move in
1. The mean function μt = E(Yt ) is constant throughout time; i.e., μt is free of t.
random directions due to collisions with other molecules. This random motion is modeled by a
2. The covariance between any two points depends only on the time gap between them, not on their
random walk, helping explain diffusion, where particles spread out over time.
actual position in time. i.e., depends only on k (not on t).
3. Animal locations: In ecology, random walk processes are used to model animal movements,
especially in the absence of clear patterns. For example, an animal may randomly change direction Difference between Stationary and Non-Stationary time series
when foraging or exploring its environment, and its location over time can be modeled as a
Feature Stationary Time Series Non-Stationary Time Series
random walk.
In each case, the key idea is that the system's future state is based on a random step from its current Mean Constant over time Changes over time

state, without any deterministic trend. Variance Remains the same May increase or decrease
Seasonality/Trends Removed or absent Present
Autocovariance Depends only on lag Changes over time
Stationary vs. Non-Stationary The Ljung-Box test
The Ljung-Box test is a statistical test used to check for the presence of autocorrelation and
Process Type Stationary?
seasonality in a time series. It helps determine whether the residuals (errors) of a model are white noise
White noise (random fluctuations) Yes
(i.e., random and uncorrelated).
Random walk (e.g., stock prices) No
Moving average (MA) process Yes Pearson Correlation Coefficient in Time Series Analysis
Autoregressive (AR) process Depends on the coefficients The Pearson correlation coefficient (r) measures the strength and direction of a linear relationship
between two variables. In time series analysis, it is often used to assess the relationship between two
time-dependent variables.
Time SeriesTransforming Non-Stationary Data
If a time series is non-stationary, we can often make it stationary by: Granger Causality Test
● Differencing: Subtracting the previous value from the current value, i.e., ∇Yt = Yt−Yt-1 , for t = 1, 2, ...., T. The Granger Causality Test is a statistical hypothesis test used to determine whether one time series can
● Log transformation: Taking the natural log of values to stabilize variance. predict another. It does not imply direct causality in the traditional sense, but rather whether past values
● Removing trends and seasonality using techniques like decomposition. of one time series help forecast future values of another.

Testing stationarity: Box-Cox Transformation

Visual tests for stationarity The Box-Cox transformation is a statistical technique used to stabilize variance and make a time series
Visual inspection can serve as an intuitive first step in determining stationarity. more normally distributed, which is often a requirement for many forecasting models.
● Plot the time series and check for trends or changes in variance.
● If the series has an upward or downward trend or increasing fluctuations, it is likely non-stationary. Forecasting Horizon in Time Series Analysis
The forecasting horizon refers to the number of time steps into the future for which predictions are made
in a time series model. It defines how far ahead we want to predict based on historical data.

Box Plot Method in Time Series Analysis

A box plot (or box-and-whisker plot) is a visualization tool used to detect outliers or anomalies,
analyze seasonal variations, and understand the distribution of a time series dataset over time.

Pearson Correlation Coefficient (PPC) in Time Series Analysis

The Pearson correlation coefficient (r) measures the strength and direction of a linear relationship
between two variables. In time series analysis, it is often used to assess the relationship between two
time-dependent variables.
Statistical tests for stationarity
The Augmented Dickey-Fuller (ADF) test: - When the trend is exponential, the moving averages are to be computed using Geometric mean

Null Hypothesis (H₀): The time series is non-stationary. - Method of least squares: The most commonly used mathematical method for measuring the trend.

Alternative Hypothesis (H₁): The time series is stationary. - Moving averages smooth out the time series
Explanation:
The Kwiatkowski-Phillips-Schmidt-Shin (KPSS):
Moving averages are used in time series analysis to smooth out short-term fluctuations and
(useful for data sets that might not fit standard distributions)
highlight long-term trends or patterns. They help in reducing noise, making it easier to identify the
Null Hypothesis (H₀): The time series is stationary.
underlying trend.
Alternative Hypothesis (H₁): The time series is not stationary.
- The straight line is fitted to the time series when the movements in the time series are: Linear
Why Does Stationarity Matter?
Stationarity is important because most time series models (like ARMA, ARIMA) assume that the data is
stationary.
If the series is not stationary, predictions may be unreliable.
By ensuring stationarity, we can build more accurate and interpretable models.