ARMA TIME SERIES MODEL

Time series analysis is a crucial aspect of data science, particularly when dealing with data that is collected over time. One of the fundamental models used in time series analysis is the ARMA (Autoregressive Moving Average) model. This article will delve into the ARMA model, its components, how it works, and its applications.

Understanding ARMA Model

The ARMA model is a combination of two simpler models: the Autoregressive (AR) model and the Moving Average (MA) model. The ARMA model is used to describe time series data that is stationary, meaning its statistical properties do not change over time.

• Autoregressive (AR) Model: This model uses the dependency between an observation and a number of lagged observations (previous time points). It is denoted as AR(p), where p is the number of lagged observations included.
• Moving Average (MA) Model: This model uses the dependency between an observation and a residual error from a moving average model applied to lagged observations. It is denoted as MA(q), where ?q is the number of lagged forecast errors included.

The ARMA model combines these two approaches and is denoted as ARMA(p, q), where p is the order of the autoregressive part and q is the order of the moving average part.

1. ARMA Components: Autoregressive (AR)

The Autoregressive (AR) part of the ARMA model uses the relationship between an observation and a number of lagged (previous) observations to predict future values. Imagine, that you are attempting to forecast the temperature for tomorrow by using the data from the last several days. The AR portion makes the assumption that the current temperature and the temperatures from earlier days are connected. For instance suppose we write the temperature of today as T_t and the temperatures of the last two days as T_{t-1} and T_{t-2}, an AR(2) model (since it uses two lagged values) can be written as:

T_{t}=c+\phi_{1}T_{t-1}+\phi_{2}T_{t-2}+e_{t}

Where:

• c is a constant.
• \phi_{1} and \phi_{2} are coefficients that determine the influence of the past temperatures.
• e_{t} is the error term (random noise).

2. ARMA Components: Moving Average (MA)

The Moving Average (MA) part of the ARMA model uses the dependency between an observation and a residual error from a moving average model applied to lagged observations. Continuing with our temperature example, the MA part assumes that today's temperature is also influenced by the errors made in predicting previous days' temperatures. If we denote today's error as e_t and the errors of the last two days as e_{t-1} and e_{t-2} an MA(2) model can be written as:

T_{t}=c+e_t+\theta_{1}e_{t-1}+\theta_{2}e_{t-2}

Where:

• c is a constant.
• \phi_{1} and \phi_{2} are coefficients that determine the influence of the past temperatures.

Mathematical Representation of ARMA Model

The ARMA model is a combination of both AR and MA components. An ARMA(p, q) model, where p is the number of lagged observations (AR part) and q is the number of lagged forecast errors (MA part), is represented as:

T_t=c+\Sigma^{p}_{i=1}\phi_{i}T_{t-i}+\Sigma^{q}_{j=1}\theta_{j}e_{t-j}+e_t

How to Determine the Orders p and q in ARMA Model?

Determining the appropriate values for p and q is crucial for building an effective ARMA model. This can be done using the following methods:

1. Partial Autocorrelation Function (PACF):
   • PACF is used to determine the order p of the AR model. It measures the correlation between observations at different lags, excluding the influence of intermediate lags.
   • The order p is determined by the lag at which the PACF plot cuts off.
2. Autocorrelation Function (ACF):
   • ACF is used to determine the order q of the MA model. It measures the correlation between observations at different lags.
   • The order q is determined by the lag at which the ACF plot cuts off.

Implementing ARMA Model in Python

Python provides several libraries for implementing ARMA models, such as statsmodels and pandas. Here is a basic example of how to implement an ARMA model in Python:

Step 1: Import Libraries

We use the same libraries as in the previous example for consistency.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from statsmodels.tsa.arima.model import ARIMA

%matplotlib inline

Step 2: Load an Dataset

For this example, we'll use the monthly airline passengers dataset, which records the number of passengers flying each month from 1949 to 1960. This dataset is available online and can be loaded directly using its URL.

• We load the dataset from a URL directly into a pandas DataFrame.
• We set the Month column as the index and parse the dates.
• We plot the dataset to visualize the trend and seasonality.

# Load the dataset
url = 'https://raw.githubusercontent.com/jbrownlee/Datasets/master/airline-passengers.csv'
airline_data = pd.read_csv(url, index_col='Month', parse_dates=True)

# Plot the dataset
plt.figure(figsize=(10, 5))
plt.plot(airline_data)
plt.title('Monthly Airline Passengers')
plt.xlabel('Date')
plt.ylabel('Number of Passengers')
plt.show()

Output:

Step 3: Fit the ARMA Model

To fit an ARMA model, the time series data should be stationary. We first check for stationarity and, if necessary, difference the data to make it stationary. We use the Augmented Dickey-Fuller (ADF) test to check for stationarity. The ADF test provides a statistic and a p-value. If the p-value is less than 0.05, the series is considered stationary. Since the p-value is greater than 0.05, we difference the data to make it stationary.

from statsmodels.tsa.stattools import adfuller

# Check for stationarity
result = adfuller(airline_data['Passengers'])
print('ADF Statistic:', result[0])
print('p-value:', result[1])

# Since the p-value is > 0.05, the data is not stationary. We need to difference it.
airline_data_diff = airline_data.diff().dropna()

# Check for stationarity again
result = adfuller(airline_data_diff['Passengers'])
print('ADF Statistic:', result[0])
print('p-value:', result[1])

# Plot the differenced data
plt.figure(figsize=(10, 5))
plt.plot(airline_data_diff)
plt.title('Differenced Monthly Airline Passengers')
plt.xlabel('Date')
plt.ylabel('Number of Passengers')
plt.show()

Output:

ADF Statistic: 0.8153688792060498
p-value: 0.991880243437641
ADF Statistic: -2.8292668241700047
p-value: 0.05421329028382478

Step 4: Fit the ARMA Model on Differenced Data

Now that the data is stationary, we can fit the ARMA model. We create an ARIMA model with the order (1, 0, 1) and fit it to the differenced data and print the model summary to understand its parameters and performance.

# Fit the ARMA(1, 1) model
model = ARIMA(airline_data_diff, order=(1, 0, 1))
model_fit = model.fit()

# Print the model summary
print(model_fit.summary())

Output:

==============================================================================
Dep. Variable:             Passengers   No. Observations:                  143
Model:                 ARIMA(1, 0, 1)   Log Likelihood                -694.061
Date:                Thu, 06 Jun 2024   AIC                           1396.122
Time:                        15:08:35   BIC                           1407.973
Sample:                    02-01-1949   HQIC                          1400.937
                         - 12-01-1960                                         
Covariance Type:                  opg                                         
==============================================================================
                 coef    std err          z      P>|z|      [0.025      0.975]
------------------------------------------------------------------------------
const          2.4507      3.441      0.712      0.476      -4.293       9.195
ar.L1         -0.4767      0.128     -3.735      0.000      -0.727      -0.227
ma.L1          0.8645      0.080     10.743      0.000       0.707       1.022
sigma2       958.5228    107.063      8.953      0.000     748.683    1168.363
===================================================================================
Ljung-Box (L1) (Q):                   0.22   Jarque-Bera (JB):                 2.17
Prob(Q):                              0.64   Prob(JB):                         0.34
Heteroskedasticity (H):               7.01   Skew:                            -0.21
Prob(H) (two-sided):                  0.00   Kurtosis:                         3.43
===================================================================================

Warnings:
[1] Covariance matrix calculated using the outer product of gradients (complex-step).
/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: No frequency information was provided, so inferred frequency MS will be used.
  self._init_dates(dates, freq)
/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: No frequency information was provided, so inferred frequency MS will be used.
  self._init_dates(dates, freq)
/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning: No frequency information was provided, so inferred frequency MS will be used.
  self._init_dates(dates, freq)

Step 5: Make Predictions

Finally, we use the fitted model to make future predictions.

• We specify the start and end points for our predictions.
• We use the predict function to generate predictions.
• We plot the differenced original series and the predictions to visualize the model's performance.

# Make predictions
start = len(airline_data_diff)
end = start + 20
predictions = model_fit.predict(start=start, end=end)

# Plot the results
plt.figure(figsize=(10, 5))
plt.plot(airline_data_diff, label='Differenced Original Series')
plt.plot(predictions, label='Predictions', color='red')
plt.legend()
plt.title('ARMA Model Predictions on Airline Data')
plt.xlabel('Date')
plt.ylabel('Number of Passengers')
plt.show()

Output:

These comprehensive instructions will help you learn how to build, fit and use ARMA models for time series analysis on both artificial and actual data. The graphics make it easier for us to understand the data and model performance.

Application and Use Cases of ARMA Model

For predicting and evaluating time series data the ARMA model is extensively utilized in many different domains. A few typical uses are as follows:

• Economics: Predicting stock prices, exchange rates, and economic indicators.
• Weather Forecasting: Analyzing temperature, rainfall, and other meteorological data.
• Sales Forecasting: Predicting future sales based on past sales data.
• Engineering: Monitoring and controlling industrial processes.
• Inventory management: Forecasting future demand for products.
• Epidemiology: Predicting the spread of diseases.

Advantages and Disadvantages of ARMA Model

Conclusion

The ARMA model is a powerful tool for time series analysis, helping us predict future values based on past trends. It offers a thorough method for deciphering patterns and generating forecasts by merging the moving average and autoregressive components. Even though it has drawbacks, its ease of use and potency make it a useful technique in a variety of sectors.

We have deconstructed the ARMA model in this easy-to-read introduction for beginners. Always keep in mind that improving forecasts requires balancing historical values and mistakes. Next time you encounter time series data, think ARMA !