Introduction to time
series and
stationarity
A RIMA MODELS IN P YTH ON
James Fulton
Climate informatics researcher
Motivation
Time series are everywhere
Science
Technology
Business
Finance
Policy
ARIMA MODELS IN PYTHON
Course content
You will learn
Structure of ARIMA models
How to t ARIMA model
How to optimize the model
How to make forecasts
How to calculate uncertainty in predictions
ARIMA MODELS IN PYTHON
Loading and plotting
import pandas as pd
import matplotlib as plt
df = pd.read_csv('time_series.csv', index_col='date', parse_dates=True)
date values
2019-03-11 5.734193
2019-03-12 6.288708
2019-03-13 5.205788
2019-03-14 3.176578
ARIMA MODELS IN PYTHON
Trend
fig, ax = plt.subplots()
df.plot(ax=ax)
plt.show()
ARIMA MODELS IN PYTHON
Seasonality
ARIMA MODELS IN PYTHON
Cyclicality
ARIMA MODELS IN PYTHON
White noise
White noise series has uncorrelated values
Heads, heads, heads, tails, heads, tails, ...
0.1, -0.3, 0.8, 0.4, -0.5, 0.9, ...
ARIMA MODELS IN PYTHON
Stationarity
Stationary Not stationary
Trend stationary: Trend is zero
ARIMA MODELS IN PYTHON
Stationarity
Stationary Not stationary
Trend stationary: Trend is zero
Variance is constant
ARIMA MODELS IN PYTHON
Stationarity
Stationary Not stationary
Trend stationary: Trend is zero
Variance is constant
Autocorrelation is constant
ARIMA MODELS IN PYTHON
Train-test split
# Train data - all data up to the end of 2018
df_train = df.loc[:'2018']
# Test data - all data from 2019 onwards
df_test = df.loc['2019':]
ARIMA MODELS IN PYTHON
Let's Practice!
A RIMA MODELS IN P YTH ON
Making time series
stationary
A RIMA MODELS IN P YTH ON
James Fulton
Climate informatics researcher
Overview
Statistical tests for stationarity
Making a dataset stationary
ARIMA MODELS IN PYTHON
The augmented Dicky-Fuller test
Tests for trend non-stationarity
Null hypothesis is time series is non-stationary
ARIMA MODELS IN PYTHON
Applying the adfuller test
from statsmodels.tsa.stattools import adfuller
results = adfuller(df['close'])
ARIMA MODELS IN PYTHON
Interpreting the test result
print(results)
(-1.34, 0.60, 23, 1235, {'1%': -3.435, '5%': -2.913, '10%': -2.568}, 10782.87)
0th element is test statistic (-1.34)
More negative means more likely to be stationary
1st element is p-value: (0.60)
If p-value is small → reject null hypothesis. Reject non-stationary.
4th element is the critical test statistics
ARIMA MODELS IN PYTHON
Interpreting the test result
print(results)
(-1.34, 0.60, 23, 1235, {'1%': -3.435, '5%': -2.863, '10%': -2.568}, 10782.87)
0th element is test statistic (-1.34)
More negative means more likely to be stationary
1st element is p-value: (0.60)
If p-value is small → reject null hypothesis. Reject non-stationary.
4th element is the critical test statistics
1 https://www.statsmodels.org/dev/generated/statsmodels.tsa.stattools.adfuller.html
ARIMA MODELS IN PYTHON
The value of plotting
Plotting time series can stop you making wrong assumptions
ARIMA MODELS IN PYTHON
The value of plotting
ARIMA MODELS IN PYTHON
Making a time series stationary
ARIMA MODELS IN PYTHON
Taking the difference
Difference: Δyt = yt − yt−1
ARIMA MODELS IN PYTHON
Taking the difference
df_stationary = df.diff()
city_population
date
1969-09-30 NaN
1970-03-31 -0.116156
1970-09-30 0.050850
1971-03-31 -0.153261
1971-09-30 0.108389
ARIMA MODELS IN PYTHON
Taking the difference
df_stationary = df.diff().dropna()
city_population
date
1970-03-31 -0.116156
1970-09-30 0.050850
1971-03-31 -0.153261
1971-09-30 0.108389
1972-03-31 -0.029569
ARIMA MODELS IN PYTHON
Taking the difference
ARIMA MODELS IN PYTHON
Other transforms
Examples of other transforms
Take the log
np.log(df)
Take the square root
np.sqrt(df)
Take the proportional change
df.shift(1)/df
ARIMA MODELS IN PYTHON
Let's practice!
A RIMA MODELS IN P YTH ON
Intro to AR, MA and
ARMA models
A RIMA MODELS IN P YTH ON
James Fulton
Climate informatics researcher
AR models
Autoregressive (AR) model
AR(1) model :
yt = a1 yt−1 + ϵt
ARIMA MODELS IN PYTHON
AR models
Autoregressive (AR) model
AR(1) model :
yt = a1 yt−1 + ϵt
AR(2) model :
yt = a1 yt−1 + a2 yt−2 + ϵt
AR(p) model :
yt = a1 yt−1 + a2 yt−2 + ... + ap yt−p + ϵt
ARIMA MODELS IN PYTHON
MA models
Moving average (MA) model
MA(1) model :
yt = m1 ϵt−1 + ϵt
MA(2) model :
yt = m1 ϵt−1 + m2 ϵt−2 + ϵt
MA(q) model :
yt = m1 ϵt−1 + m2 ϵt−2 + ... + mq ϵt−q + ϵt
ARIMA MODELS IN PYTHON
ARMA models
Autoregressive moving-average (ARMA) model
ARMA = AR + MA
ARMA(1,1) model :
yt = a1 yt−1 + m1 ϵt−1 + ϵt
ARMA(p, q)
p is order of AR part
q is order of MA part
ARIMA MODELS IN PYTHON
Creating ARMA data
yt = a1 yt−1 + m1 ϵt−1 + ϵt
ARIMA MODELS IN PYTHON
Creating ARMA data
yt = 0.5yt−1 + 0.2ϵt−1 + ϵt
from statsmodels.tsa.arima_process import arma_generate_sample
ar_coefs = [1, -0.5]
ma_coefs = [1, 0.2]
y = arma_generate_sample(ar_coefs, ma_coefs, nsample=100, sigma=0.5)
ARIMA MODELS IN PYTHON
Creating ARMA data
yt = 0.5yt−1 + 0.2ϵt−1 + ϵt
ARIMA MODELS IN PYTHON
Fitting and ARMA model
from statsmodels.tsa.arima_model import ARMA
# Instantiate model object
model = ARMA(y, order=(1,1))
# Fit model
results = model.fit()
ARIMA MODELS IN PYTHON
Let's practice!
A RIMA MODELS IN P YTH ON
