Time Series Analysis and Forecasting

Math 667
Al Nosedal

Department of Mathematics
Indiana University of Pennsylvania

Time Series Analysis and Forecasting p. 1/11

Introduction

Many decision-making applications depend on a forecast of

some quantity. Here are a couple of examples.
When a service organization, such as a fast-food
restaurant, plans its staffing over some time period, it
must forecast the customer demand as a function of
time.

Time Series Analysis and Forecasting p. 2/11

Introduction

Many decision-making applications depend on a forecast of

some quantity. Here are a couple of examples.
When a service organization, such as a fast-food
restaurant, plans its staffing over some time period, it
must forecast the customer demand as a function of
time.
When a company plans its ordering or production
schedule for a product it sells to the public, it must
forecast the customer demand for this product so that it
can stock appropriate quantities-neither too much nor
too little.

Time Series Analysis and Forecasting p. 2/11

Forecasting Methods: An Overview

There are many forecasting methods available, these

methods can generally be divided into three groups:

1. Judgemental methods.

Time Series Analysis and Forecasting p. 3/11

Forecasting Methods: An Overview

There are many forecasting methods available, these

methods can generally be divided into three groups:

1. Judgemental methods.
2. Extrapolation (or Time Series) methods, and

Time Series Analysis and Forecasting p. 3/11

Forecasting Methods: An Overview

There are many forecasting methods available, these

methods can generally be divided into three groups:

1. Judgemental methods.
2. Extrapolation (or Time Series) methods, and
3. Econometric (or causal) methods.

Time Series Analysis and Forecasting p. 3/11

Extrapolation Methods

Extrapolation methods are quantitative methods that use

past data of a time series variable-and nothing else, except
possibly time itself-to forecast future values of the variable.
The idea is that we can use past movements of a variable,
such as a company sales to forecast its future values.

Time Series Analysis and Forecasting p. 4/11

Extrapolation Methods

There are many extrapolation methods available, including

trend-based regression, moving averages, and
autoregression models. All these extrapolation methods
search for patterns in the historical series and then
extrapolate these patterns into the future.

Time Series Analysis and Forecasting p. 5/11

Random Series

The simplest time series model is the random model. A

random model can be written as

(1) Y (t) = + (t)

Here, is a constant, the average of the Y (t)0 s, and (t)

is the residual (or error) term. We assume that the
residuals have mean 0, variance 2 , and are
probabilistically independent of one another.

Time Series Analysis and Forecasting p. 6/11

Random series (Graph)

2
1
Y(t)

0
1

0 10 20 30 40

Observation number

Time Series Analysis and Forecasting p. 7/11

Random Series (1)

There are two situations where random time series occur.

The first is when the original time series is random. For
example, when studying the time pattern of diameters of
individual parts from a manufacturing process, we might
discover that the successive diameters behave like a
random time series.

Time Series Analysis and Forecasting p. 8/11

Random Series (1)

There are two situations where random time series occur.

The first is when the original time series is random. For
example, when studying the time pattern of diameters of
individual parts from a manufacturing process, we might
discover that the successive diameters behave like a
random time series.

Time Series Analysis and Forecasting p. 8/11

Random Series (1)

There are two situations where random time series occur.

The first is when the original time series is random. For
example, when studying the time pattern of diameters of
individual parts from a manufacturing process, we might
discover that the successive diameters behave like a
random time series.

Time Series Analysis and Forecasting p. 8/11

Random Series (2)

The second situation where a random series occurs is

more common. This is when we fit a model to a time
series to obtain an equation of the form

(2) Y (t) = f itted part + residual.

Although the fitted part varies from model to model, its

essential feature is that it describes any underlying time
series pattern in the original data. The residual is then
whatever is left, and we hope that the series of residuals
is random with mean = 0. The fitted part is used to
make forecasts, and the residuals, often called noise,
are forecasted to be zero.

Time Series Analysis and Forecasting p. 9/11

Random Series (2)

The second situation where a random series occurs is

more common. This is when we fit a model to a time
series to obtain an equation of the form

(3) Y (t) = f itted part + residual.

Although the fitted part varies from model to model, its

essential feature is that it describes any underlying time
series pattern in the original data. The residual is then
whatever is left, and we hope that the series of residuals
is random with mean = 0. The fitted part is used to
make forecasts, and the residuals, often called noise,
are forecasted to be zero.

Time Series Analysis and Forecasting p. 9/11

Runs test

For each observation Y (t) we associate a 1 if Y (t) y and 0

if Y (t) < y. A run is a consecutive sequence of 0s or 1s. Let
T be the number of observations, let Ta above the mean,
and let Tb the number below the mean. Also let R be the
observed number of runs. Then it can be show that for a
random series

T + 2Ta Tb
(4) E(R) =
T
s
2Ta Tb (2Ta Tb T )
(5) Stdev(R) =
T 2 (T 1)
When T > 20, the distribution of R is roughly Normal.
Time Series Analysis and Forecasting p. 10/11
Example: Runs test

Suppose that the successive observations are 87, 69, 53,

57, 94, 81, 44, 68, and 77, with mean Y = 70. It is possible
that this series is random. Does the runs test support this
conjecture?

Time Series Analysis and Forecasting p. 11/11

Example: Runs test (solution)

The preceding sequence has five runs: 1; 0 0 0; 1 1; 0 0;

and 1. Then, we have T = 9, Ta = 4, Tb = 5 and R = 5. Under
a randomness hypothesis,

(6) E(R) = 5.44

(7) Stdev(R) = 1.38

R E(R)
(8) Z= = 0.32
Stdev(R)

Time Series Analysis and Forecasting p. 12/11

Exercise

Write a function in R that performs a run test. My version of

this function will be posted on our website tomorrow.

Time Series Analysis and Forecasting p. 13/11

Autocorrelation (1)

Recall that the successive observations in a random series

are probabilistically independent of one another. Many time
series violate this property and are instead autocorrelated.
For example, in the most common form of autocorrelation,
positive autocorrelation, large observations tend to follow
large observations, and small observations tend to follow
small observations. In this case the runs test is likely to pick
it up. Another way to check for the same nonrandomness
property is to calculate the autocorrelations of the time
series.

Time Series Analysis and Forecasting p. 14/11

Autocorrelation (2)

To understand autocorrelation it is first necessary to

understand what it means to lag a time series.
Y (t) = Yt =(3,4,5,6,7,8,9,10,11,12)
Y lagged 1 = Y (t 1) = Yt1 =(*,3,4,5,6,7,8,9,10,11)

Time Series Analysis and Forecasting p. 15/11

Autocovariance Function

The autocovariance function is defined as follows:

(9) (k) = E(Y (t) )(Y (t + k) ), k = 0, 1, 2, ...

where Y (t) represents the values of the series, is the
mean of the series and k is the lag at which the
autocovariance is being considered.

Time Series Analysis and Forecasting p. 16/11

Autocorrelation Function

A standardized version of (k) is the autocorrelation

function, defined as
(k)
(10) =
(0)

Time Series Analysis and Forecasting p. 17/11

Estimator of the autocovariance function

The autocovariance function at lag k can be estimated by

nk
1X
(11) (k) = (Y (t) Y )(Y (t + k) Y )
n
t=1

Time Series Analysis and Forecasting p. 18/11

Estimator of the autocorrelation function

The autocorrelation function estimate at lag k is simply

(k)
(12) (k) =
(0)

Time Series Analysis and Forecasting p. 19/11

Example

Find the autocorrelation at lag 1 for the following time series:

Y (t) = (3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
Answer.
(1) = 0.7

Time Series Analysis and Forecasting p. 20/11

Finding the autocorrelation at lag 1

y<-3:12

### function to find autocorrelation when

###lag=1
my.auto<-function(x){
n<-length(x)
denom<-(x-mean(x))%*%(x-mean(x))/n
num<-(x[-n]-mean(x))%*%(x[-1]-mean(x))/n
result<-num/denom

return(result)
}
my.auto(y)

Time Series Analysis and Forecasting p. 21/11

An easier way of finding autocorrelations

Using R, we can easily find the autocorrelation at any lag k.

y<-3:12

auto.lag1<-acf(y,lag=1)$acf[2]

auto.lag1

Time Series Analysis and Forecasting p. 22/11

Graph of autocorrelations

A graph of the lags against the corresponding

autocorrelations is called a correlogram. The following lines
of code can be used to make a correlogram in R.
### autocorrelation function for
###a random series

y<-rnorm(25)

acf(y,lag=8,main=Random Series N(0,1))

Time Series Analysis and Forecasting p. 23/11

Correlogram (Graph)

Random Series N(0,1)

0.8
0.4
ACF

0.0
0.4

0 2 4 6 8

Lag

Time Series Analysis and Forecasting p. 24/11

Example: Stereo Retailers

Monthly sales for a chain of stereo retailers are listed in the

file stereo.txt. They cover the period from the beginning of
1995 to the end of 1998, during which there was no upward
or downward trend in sales and no clear seasonal peaks or
valleys. This behavior is apparent in the time series chart of
sales shown in our next slide. It is possible that this series is
random. Do autocorrelations support this conclusion?

Time Series Analysis and Forecasting p. 25/11

Time Series Plot (stereo.txt)

250
200
sales

150

0 10 20 30 40

1:N

Time Series Analysis and Forecasting p. 26/11

Time Series Plot (stereo.txt)

250
200
new.sales

150

1995 1996 1997 1998 1999

Time

Time Series Analysis and Forecasting p. 27/11

Correlogram (stereo.txt)

Stereo Sales

1.0
0.6
ACF

0.2
0.2

0 2 4 6 8 10 12

Lag

Time Series Analysis and Forecasting p. 28/11

How large is a "large" autocorrelation?

Suppose that Y1 , ..., YT are independent and identically

distributed random variables with arbitrary mean. Then it
can be shown that
1
(13) E(k ) '
T

1
(14) V ar(k ) '
T
Thus, having plotted the correlogram, we can plot

approximate 95% confidence limits at1/T 2/ T , which
are often further approximated to 2/ T .

Time Series Analysis and Forecasting p. 29/11

Conclusion (Stereo Sales)
p
In this case, V (k ) = 1/ 48. If the series is truly random,
then only an occasional autocorrelation should be larger
than two standard errors in magnitude. Therefore, any
autocorrelation that is larger than two standard errors in
magnitude is worth our attention. The only "large"
autocorrelation for the sales data is the first, or lag 1. It
seems that the pattern of sales is not completely random.

Time Series Analysis and Forecasting p. 30/11

Example: Demand

The dollar demand for a certain class of parts at a local

retail store has been recorded for 82 consecutive days.
(See the file demand.txt) A time series plot of these
demands appears in the next slide. The store manager
wants to forecast future demands. In particular, he wants to
know whether is any significant time pattern to the historical
demands or whether the series is essentially random.

Time Series Analysis and Forecasting p. 31/11

Time Series Plot of Demand for Parts

Time Series Plot of Demand for Parts

350
300
Daily demand

250
200
150

0 20 40 60 80

Time

Time Series Analysis and Forecasting p. 32/11

Solution

A visual inspection of the time series graph in the previous

slide shows that demands vary randomly around the sample
mean of $247.54 (shown as the horizontal centerline). The
variance appears to be constant through time, and there are
no obvious time series patterns. To check formally whether
this apparent randomness holds, we perform the runs test
and calculate the first 10 autocorrelations. (The associated
correlogram appears in the next slide). The p-value for the
runs test is relatively large 0.118 and none of the
autocorrelations is significantly large. These findings are
consistent with randomness.

Time Series Analysis and Forecasting p. 33/11

Graph

Correlogram for Demand Data

0.2 0.4 0.6 0.8 1.0

ACF

0.2

0 2 4 6 8 10

Lag

Time Series Analysis and Forecasting p. 34/11

Conclusion

For all practical purposes there is no time series pattern to

these demand data. It is as if every days demand is an
independent draw from a distribution with mean $247.54
and standard deviation $47.78. Therefore, the manager
might as well forecast that demand for any day in the future
will be $247.54. If he does so, about 95% of his forecasts
should be within two standard deviations (about $95) of the
actual demands.

Time Series Analysis and Forecasting p. 35/11

A simulation

It is often helpful to study a time series model by simulation.

This enables the main features of the model to be observed
in plots, so that when historical data exhibit similar features
the model may be selected as a potential candidate. The
following commands can be used to simulate random walk
data for y.

set.seed(2) #we do this to get

#the same random numbers
y<-e<-rnorm(1000)
for (t in 2:1000){ y[t] <- y[t-1] + e[t]}
plot(y, type=l,col=blue)

Time Series Analysis and Forecasting p. 36/11

Simulation (plot)

60
40
y

20
0

0 200 400 600 800 1000

Index

Time Series Analysis and Forecasting p. 37/11

Autocorrelations for Random Walk

Autocorrelations for Random Walk

1.0
0.8
0.6
ACF

0.4
0.2
0.0

0 5 10 15 20 25 30

Lag

Time Series Analysis and Forecasting p. 38/11

Random Walk Model

Random series are sometimes building blocks for other time

series models. The model we now discuss, the random walk
model, is an example of this. In a random walk model the
series itself is not random. However, its differences - that is,
the changes from one period to the next - are random. This
type of behavior is typical of stock price data. For example,
the graph in the next slide shows monthly Dow Jones
averages from January 1988 through March 1992. (See the
file dow.txt)

Time Series Analysis and Forecasting p. 39/11

Time series chart of closing Dow average

Time Series Plot of Dow Jones Index

3200
closing Dow avg

2800
2400
2000

1988 1989 1990 1991 1992

Time

Time Series Analysis and Forecasting p. 40/11

More about Dow Jones

This series is not random, as can be seen from its gradual

upward trend (Although the runs test and autocorrelations
are not shown here, they confirm that the series is not
random.)
If we were standing in March 1992 and were asked to
forecast the Dow Jones average for the next few months, it
is intuitive that we would not use the average of the historical
values as our forecast. This forecast would probably be too
low because the series has an upward trend. Instead, we
would base our forecast on the most recent observation.
This is exactly what the random walk model does.

Time Series Analysis and Forecasting p. 41/11

Random walk (equation)

An equation for the random walk model is

(15) Yt = Yt1 + + t

where is a constant and t is a random series with mean 0

and some standard deviation . If we let DYt = Yt Yt1 , the
change in the series from time t to time t 1, then we can
write the random walk model as

(16) DYt = + t

This implies that the differences form a random series with

mean and standard deviation .

Time Series Analysis and Forecasting p. 42/11

Random walk (cont.)

An estimate of is the average of the differences, labeled

YD , and an estimate of is the sample standard deviation of
the differences , labeled sD . In words, a series that behaves
according to this random walk model has random
differences, and the series tends to trend upward (if > 0) or
downward (if < 0) by an amount each period. If we are
standing in period t and want to make a forecast Ft+1 of Yt+1 ,
then a reasonable forecast is

(17) Ft+1 = Yt + YD

That is, we add the estimated trend to the current

observation to forecast the next observation.

Time Series Analysis and Forecasting p. 43/11

Example: Dow Jones revisited

Given the monthly Dow Jones data in the file dow.txt, check
that it satisfies the assumptions of a random walk, and use
the random walk model to forecast the value for April 1992.

Time Series Analysis and Forecasting p. 44/11

Solution

We have already seen that the Dow Jones series itself is not
random, due to the upward trend, so we form the differences
diff.dow<-diff(index)
ts.diff<-ts(diff.dow)
plot(ts.diff,col=blue,
ylab=first difference, main=Time Series
Plot of Differences)
abline(h=mean(diff.dow),col=red)

Time Series Analysis and Forecasting p. 45/11

Time Series Plot of Dow Differences

Time Series Plot of Differences

100 200
first difference

0
200

0 10 20 30 40 50

Time

Time Series Analysis and Forecasting p. 46/11

Dow Differences (cont.)

It appears to be a much more random series, varying

around the mean difference 26. The runs test shows that
there is absolutely no evidence of nonrandom differences.
runs(diff.dow)
$T [1] 50
$Ta [1] 26
$Tb [1] 24
$R [1] 26
$Z [1] 0.01144965
$E(R) [1] 25.96
$P value [1] 0.9908647

Time Series Analysis and Forecasting p. 47/11

Dow Differences (cont.)

Similarly, the autocorrelations are all small except for a

random "blip" at lag 11. Because there is probably no
reason to believe that values 11 months apart really are
related, we would tend to ignore this autocorrelation.

Time Series Analysis and Forecasting p. 48/11

Correlogram for Dow Differences

Autocorrelations for Dow Differences

1.0
0.6
ACF

0.2
0.2

0 2 4 6 8 10 12

Lag

Time Series Analysis and Forecasting p. 49/11

Forecast of April 1992

Assuming the random walk model is adequate, the forecast

of April 1992 made in March 1992 is the observed March
value, 3247.42, plus the mean difference, 26, or 3273.42. A
measure of the forecast accuracy is provided by the
standard deviation, sD =84.65, of the differences. Provided
that our assumptions hold, we can be 95 % confident that
our forecast is off by no more than 2sD , or about 170.

Time Series Analysis and Forecasting p. 50/11

Exercise

Write a function in R that gives you a one-step-ahead

forecast for a Random Walk Model.

Time Series Analysis and Forecasting p. 51/11

One solution

forecast<-function(y){

diff<-diff(y)
y.diff.bar<-mean(diff) #average difference
last<-length(y) #last observation
F.next<-y[last]+y.diff.bar
new.y<-c(y,F.next)

list(Y(t)=y,Y(t+1)=new.y,F(t+1)=F.next)

Time Series Analysis and Forecasting p. 52/11

Exercise

Using your one-step-ahead forecast function, write another

function in R that computes forecast for times:
t+1,t+2,...,t+N, where t represents the length of your original
time series.

Time Series Analysis and Forecasting p. 53/11

One solution

forecast.N<-function(y,N){

original<-y #original time series

for (i in 1:N){
new<-forecast(original)$Y(t+1)
original<-new
}

list(Y(t)=y,N=N,Y(t+N)=original)

Time Series Analysis and Forecasting p. 54/11

Forecasts for Dow Jones Avg

Dow Jones Avg with forecasts

Dow Jones average

3000
2500
2000

0 10 20 30 40 50

Time

Time Series Analysis and Forecasting p. 55/11

Another simulation

set.seed(1)
y <- e <- rnorm(100)
for (t in 2:100) {
y[t]<-0.7*y[t-1] + e[t]}
plot(y,type=l,col=blue)
acf(y)

Time Series Analysis and Forecasting p. 56/11

Autoregression Models

A regression-based extrapolation method is to regress the

current value of the time series on past (lagged) values.
This is called autoregression, where the "auto" means that
the explanatory variables in the equation are lagged values
of the response variable, so that we are regressing the
response variable on lagged versions of itself. Some trial
and error is generally required to see how many lags are
useful in the regression equation. The following exercise
illustrates the procedure.

Time Series Analysis and Forecasting p. 57/11

Exercise

Plot your time series

Make a correlogram
Fit an autoregressive model of order p (p suggested by
your correlogram)
Determine if your model is adequate
Fit another model if necessary

Time Series Analysis and Forecasting p. 58/11

Exercise: Hammers

A retailer has recorded its weekly sales of hammers (units

purchased) for the past 42 weeks. (See the file hammers.txt)
How useful is autoregression for modeling these data and
how would it be used for forecasting?

Time Series Analysis and Forecasting p. 59/11

Exercise: Hammers (cont.)

Plot your time series

Make a correlogram
Fit an autoregressive model of order p (p suggested by
your correlogram)
Determine if your model is adequate
Fit another model if necessary

Time Series Analysis and Forecasting p. 60/11

Regression-Based Trend Models

Many time series follow a long-term trend except for random

variation. This trend can be upward or downward. A
straightforward way to model this trend is to estimate a
regression equation for Yt , using time t as the single
explanatory variable. In this "section" we will discuss the two
most frequently used trend models, linear trend and
exponential trend.

Time Series Analysis and Forecasting p. 61/11

Linear Trend

A linear trend means that the time series variable changes

by a constant amount each time period. The relevant
equation is

(18) Yt = + t + t
where, as in previous regression equations, is the
intercept, is the slope, and t is an error term.

Time Series Analysis and Forecasting p. 62/11

Exercise: Reebok

The file reebok.txt includes quarterly sales data for Reebok

from the first quarter 1986 through second quarter 1996.
Sales increase from $ 174.52 million in the first quarter to
$ 817.57 million in the final quarter. How well does a linear
trend fit these data? Are the residuals from this fit random?

Time Series Analysis and Forecasting p. 63/11

Exercise: Reebok (cont.)

Plot your time series

Fit a linear regression model and interpret your results
Find the residuals
Determine if the residuals are random

Time Series Analysis and Forecasting p. 64/11

Exponential Trend

In contrast to a linear trend, an exponential trend is

appropriate when the time series changes by a constant
percentage (as opposed to a constant dollar amount) each
period. Then the appropriate regression equation is

(19) Yt = c exp(bt)ut

where c and b are constants, and ut represents a

multiplicative error term. By taking logarithms of both sides,
and letting a = ln(c) and t = ln(ut ), we obtain a linear
equation that can be estimated by the usual linear
regression method. However, note that the response
variable is now the logarithm of Yt :

(20) ln(Yt ) = a + bt + t
Time Series Analysis and Forecasting p. 65/11
Exercise: Intel

The file intel.txt contains quarterly sales data for the chip
manufacturing firm Intel from the beginning of 1986 through
the second quarter of 1996. Each sales value is expressed
in millions of dollars. Check that an exponential trend fits
these sales data fairly well. Then estimate the relationship
and interpret it.

Time Series Analysis and Forecasting p. 66/11

Exercise: Intel (cont)

Plot your time series

If your original time series shows an exponential trend,
apply ln to the series
Use the transformed series to estimate the relationship
Express the estimated relationship in the original scale
Find an estimate of the standard deviation

Time Series Analysis and Forecasting p. 67/11

Moving Averages

Perhaps the simplest and one of the most frequently used

extrapolation methods is the method of moving averages. To
implement the moving averages method, we first choose a
span, the number of terms in each moving average. Lets
say the data are monthly and we choose a span of 6
months. Then the forecast of next months value is the
average of the most recent 6 months values. For example,
we average January-June to forecast July, we average
February-July to forecast August, and so on. This procedure
is the reason for the term moving averages.

Time Series Analysis and Forecasting p. 68/11

Exercise (toy example)

Suppose that Y (t) = [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15].

Plot your time series

Write a function in R that finds a one-step-ahead moving
average forecast for a span=2
Write a function in R that does a moving average
"smoothing" to a time series

Time Series Analysis and Forecasting p. 69/11

Exercise: Dow Jones revisited

We again look at the Dow Jones monthly data from January

1988 through March 1992. (See the file dow.txt). How well
do moving averages track this series when the span is 3
months; when the span is 12 months?

Time Series Analysis and Forecasting p. 70/11

Exponential Smoothing

There are two possible criticisms of the moving averages

method. First, it puts equal weight on each value in a typical
moving average when making a forecast. Many people
would argue that if next months forecast is to be based on
the previous 12 months observations, then more weight
ought to be placed on the more recent observations. The
second criticism is that moving averages method requires a
lot of data storage. This is particularly true for companies
that routinely make forecasts of hundreds or even thousands
of items. If 12-month moving averages are used for 1000
items, then 12000 values are needed for next months
forecasts. This may or may not be a concern considering
todays relatively inexpensive computer storage capabilities.
Time Series Analysis and Forecasting p. 71/11
Exponential Smoothing (cont.)

Exponential smoothing is a method that addresses both of

these criticisms. It bases its forecasts on a weighted average
of past observations, with more weight put on the more
recent observations, and it requires very little data storage.
There are many versions of exponential smoothing. The
simplest is called simple exponential smoothing. It is
relevant when there is no pronounced trend or seasonality in
the series. If there is a trend but no seasonality, then Holts
method is applicable. If, in addition, there is seasonality,
then Winters method can be used.

Time Series Analysis and Forecasting p. 72/11

Simple Exponential Smoothing

We now examine simple exponential smoothing in some

detail. We first introduce two new terms. Every exponential
model has at least one smoothing constant, which is always
between 0 and 1. Simple exponential smoothing has a
single smoothing constant denoted by . The second new
term is Lt , called the level of the series at time t. This value
is not observable but can only be estimated. Essentially, it is
where we think the series would be at time t if there were no
random noise.

Time Series Analysis and Forecasting p. 73/11

Simple Exponential Smoothing (cont.)

Then the simple exponential smoothing method is defined

by the following two equations, where Ft+k is the forecast of
Yt+k made at time t:

(21) Lt = Yt + (1 )Lt1

(22) Ft+k = Lt

Time Series Analysis and Forecasting p. 74/11

Example:Exxon

The file exxon.txt contains data on quarterly sales (in

millions of dollars) for the period from 1986 through the
second quarter of 1996. Does a simple exponential
smoothing model track these data well? How do the
forecasts depend on the smoothing constant ?

Time Series Analysis and Forecasting p. 75/11

Example: Exxon (cont.)

Simple Exponential Smoothing with = 0.01

HoltWinters filtering

30000
Observed / Fitted

25000
20000

obs
forecasts

1986 1990 1994 1998

Time
Time Series Analysis and Forecasting p. 76/11
Example: Exxon (R code)

exxon<-read.table(file="exxon.txt",
header=TRUE)
sales<-exxon$Sales
new.sales<-ts(sales,start=c(1986,1),
end=c(1996,2),freq=4)
mod1<-HoltWinters(new.sales,alpha=0.1,
beta=FALSE,gamma=FALSE)
plot(mod1,xlim=c(1986,1998))
lines(predict(mod1,n.ahead=6),col="red")
legend(1992,20000,c("obs","forecasts"),
col=c("black","red"),lty=c(1,1),bty="n")

Time Series Analysis and Forecasting p. 77/11

Exxon (changing )

The following lines of code produce a "movie" that illustrates

the effect of changing .
n<-100
for (i in 1:n){
mod1<-HoltWinters(new.sales,
alpha=(1/n)*i,beta=FALSE,gamma=FALSE)
plot(mod1,xlim=c(1986,1997))
Sys.sleep(0.3)

Time Series Analysis and Forecasting p. 78/11

Choosing .

What value of should we use? There is no universally

accepted answer to this question. Some practitioners
recommend always using a value around 0.1 or 0.2. Others
recommend experimenting with different values of until a
measure such as the Mean Square Error (MSE) is
minimized.
R can find this optimal value of as follows:
mod2<-HoltWinters(new.sales,
beta=FALSE,gamma=FALSE)

Time Series Analysis and Forecasting p. 79/11

Holts Model for Trend

The simple exponential smoothing model generally works

well if there is no obvious trend in the series. But if there is a
trend, then this method consistently lags behind it. Holts
method rectifies this by dealing with trend explicitly. In
addition to the level of the series, Lt , Holts method includes
a trend term, Tt , and a corresponding smoothing constant .
The interpretation of Lt is exactly as before.

Time Series Analysis and Forecasting p. 80/11

Holts Model for Trend (cont.)

The interpretation of Tt is that it represents an estimate of

the change in the series from one period to the next. The
equations for Holts model are as follows:

(23) Lt = Yt + (1 )(Lt1 + Tt1 )

(24) Tt = (Lt Lt1 ) + (1 )Tt1

(25) Ft+k = Lt + kTt

Time Series Analysis and Forecasting p. 81/11

Example: Dow Jones revisited

We return to the Dow Jones data (see file dow.txt). Again,

these are average monthly closing prices from January 1988
through March 1992. Recall that there is a definite upward
trend in this series. In this example we investigate whether
simple exponential smoothing can capture the upward trend.
Then we see whether Holts exponential smoothing method
can make an improvement.

Time Series Analysis and Forecasting p. 82/11

Example: Dow Jones (cont.)

Simple Exponential Smoothing with optimal

HoltWinters filtering

3500
Observed / Fitted

3000
2500
2000

1988 1990 1992 1994

Time
Time Series Analysis and Forecasting p. 83/11
Example: Dow Jones (R code)

dow<-read.table(file="dow.txt",header=TRUE)
index<-dow$Dow
DJI<-ts(index,start=c(1988,1),end=c(1992,3),
freq=12)
mod1<-HoltWinters(DJI,beta=FALSE,gamma=FALSE)

### predictions
plot(mod1,xlim=c(1988,1994),
ylim=c(1800,3700))
preds<-predict(mod1,n.ahead=12)
lines(preds,col="red")

Time Series Analysis and Forecasting p. 84/11

Dow Jones (Holts Model)

Holts Model with optimal and .

HoltWinters filtering

3500
Observed / Fitted

3000
2500
2000

1988 1990 1992 1994

Time
Time Series Analysis and Forecasting p. 85/11
R code (Holts Model)

mod2<-HoltWinters(DJI,gamma=FALSE)
#Fitting Holts model

### predictions

plot(mod2,xlim=c(1988,1994),
ylim=c(1800,3700))
preds<-predict(mod2,n.ahead=12)
lines(preds,col=red)

Time Series Analysis and Forecasting p. 86/11

Winters Model

So far we have said practically nothing about seasonality.

Seasonality is defined as the consistent month-to-month (or
quarter-to-quarter) differences that occur each year. For
example, there is seasonality in beer sales - high in the
summer months, lower in other months.
How do we know whether there is seasonality in a time
series? The easiest way is to check whether a plot of the
time series has a regular pattern of ups and /or downs in
particular months or quarters.

Time Series Analysis and Forecasting p. 87/11

Winters Model (cont.)

Winters exponential smoothing model is very similar to

Holts model- it again has a level and a trend terms and
corresponding smoothing constants and - but it also has
seasonal indexes and a corresponding smoothing constant
. This new smoothing constant controls how quickly the
method reacts to perceived changes in the pattern of
seasonality.

Time Series Analysis and Forecasting p. 88/11

Example: Coke

The data in the coke.txt file represent quarterly sales (in

millions of dollars) for Coca Cola from quarter 1 of 1986
through quarter 2 of 1996. As we might expect, there has
been an upward trend in sales during this period, and there
is also a fairly regular seasonal pattern, as shown in the next
slide. Sales in the warmer quarters, 2 and 3, are
consistently higher than in colder quarters, 1 and 4. How
well can Winters method track this upward trend and
seasonal pattern?

Time Series Analysis and Forecasting p. 89/11

Coke sales with forecasts

HoltWinters filtering

5000
Observed / Fitted

3000
1000

1986 1990 1994 1998

Time

Time Series Analysis and Forecasting p. 90/11

R code

mod.coke<-HoltWinters(sales)
# Holt-Winters model

### predictions

plot(mod.coke,ylim=c(1000,6000),
xlim=c(1986,1998))

preds<-predict(mod.coke,n.ahead=6)

lines(preds,col=red)

Time Series Analysis and Forecasting p. 91/11

ARIMA-Models

Forecasting based on ARIMA (autoregressive integrated

moving averages) models, commonly known as the
Box-Jenkins approach, comprises the following stages:

Model Identification
Parameter estimation
Diagnostic checking
Iteration

These stages are repeated until a "suitable" model for the

given data has been identified.

Time Series Analysis and Forecasting p. 92/11

Identification

Various methods can be useful for identification:

Time Series Plot
ACF / Correlogram
PACF (Partial Autocorrelation Function)
Test for White Noise

Time Series Analysis and Forecasting p. 93/11

Important Processes

White Noise
Moving Average Process (MA)
Autoregressive Process (AR)

Time Series Analysis and Forecasting p. 94/11

White Noise

White Noise. A stationary time series for which Y (t) and

Y (t + k) are uncorrelated, i.e., E[Y (t) ][Y (t + k) ] = 0
for all integers k 6= 0, is called white noise. Such process is
sometimes loosely termed a "purely random process".

Time Series Analysis and Forecasting p. 95/11

Moving Average Process

Moving Average Process. A moving average process of

order q, denoted M A(q), is defined by the equation

(26) Y (t) = + 0 t + 1 t1 + ... + q tq

where t is a white noise process.

Time Series Analysis and Forecasting p. 96/11

Autoregressive Process

Consider a time series Y (t) that satisfies the difference

equation

(27) Y (t) = 1 Y (t 1) + 2 Y (t 2) + ... + p Y (t p) + t

where t is a white noise process with zero mean and finite

variance 2 . The time series Y (t) is called an autoregressive
process of order p and is denoted AR(p).

Time Series Analysis and Forecasting p. 97/11

Analysis of ACF and PACF

A first step in analyzing time series is to examine the

autocorrelations (ACF ) and partial autocorrelations (P ACF ).
R provides the functions acf () and pacf () for computing and
plotting ACF and PACF. The order of "pure" AR and M A
processes can be identified from the ACF and P ACF .

Time Series Analysis and Forecasting p. 98/11

Simulating AR(1) and MA(1)

#simulation of an AR(1) process

sim.ar<-arima.sim(list(ar=c(0.7)),n=1000)

#simulation of a MA(1) process

sim.ma<-arima.sim(list(ma=c(0.6)),n=1000)

Time Series Analysis and Forecasting p. 99/11

ACF and PACF for our simulations

# setting for a "matrix" of plots

par(mfrow=c(2,1))

## AR process

acf(sim.ar,main="ACF of AR(1) process")

pacf(sim.ar, main="PACF of AR(1) process")

Time Series Analysis and Forecasting p. 100/11

ACF and PACF for our simulations

# setting for a "matrix" of plots

par(mfrow=c(2,1))

## MA process

acf(sim.ma,main="ACF of MA(1) process")

pacf(sim.ma, main="PACF of MA(1) process")

Time Series Analysis and Forecasting p. 101/11

Simulating AR(2) and MA(2)

#simulation of an AR(2) process

sim.ar<-arima.sim(list(ar=c(0.4,0.4)),n=1000)

#simulation of a MA(2) process

sim.ma<-arima.sim(list(ma=c(0.6,-0.4)),n=1000)

Time Series Analysis and Forecasting p. 102/11

ACF and PACF for our simulations

# setting for a "matrix" of plots

par(mfrow=c(2,1))

## AR process

acf(sim.ar,main="ACF of AR(2) process")

pacf(sim.ar, main="PACF of AR(2) process")

Time Series Analysis and Forecasting p. 103/11

ACF and PACF for our simulations

# setting for a "matrix" of plots

par(mfrow=c(2,1))

## MA process

acf(sim.ma,main="ACF of MA(2) process")

pacf(sim.ma, main="PACF of MA(2) process")

Time Series Analysis and Forecasting p. 104/11

Simulating AR(3) and MA(3)

#simulation of an AR(3) process

sim.ar<-arima.sim(list(ar=c(0.4,0.3,0.2)),
n=1000)

#simulation of a MA(3) process

sim.ma<-arima.sim(list(ma=c(0.6,-0.4,0.5)),
n=1000)

Time Series Analysis and Forecasting p. 105/11

ACF and PACF for our simulations

# setting for a "matrix" of plots

par(mfrow=c(2,1))

## AR process

acf(sim.ar,main="ACF of AR(3) process")

pacf(sim.ar, main="PACF of AR(3) process")

Time Series Analysis and Forecasting p. 106/11

ACF and PACF for our simulations

# setting for a "matrix" of plots

par(mfrow=c(2,1))

## MA process

acf(sim.ma,main="ACF of MA(3) process")

pacf(sim.ma, main="PACF of MA(3) process")

Time Series Analysis and Forecasting p. 107/11

Important conclusions from simulations

We know that for the moving average model M A(q) the

theoretical ACF has the property that h = 0 for h > q , so
we can expect the sample ACF rh 0 when h > q .
If the process Y (t) is really AR(p), then there is no
dependence of Y (t) on Y (t p 1), Y (t p 2), ... once
Y (t 1), ..., Y (t p) have been taken into account. Thus
we might expect that h 0 for h > p, i.e. that the P ACF
cuts off beyond p.

Time Series Analysis and Forecasting p. 108/11

Behavior of Theoretical ACF and PACF

M A(q): ACF cuts off after lag q .

M A(q): P ACF shows exponential decay and/or
dampened sinusoid.
AR(p): ACF shows exponential decay and/or dampened
sinusoid.
AR(p): P ACF cuts off after lag q .
ARM A(p, q):ACF shows exponential decay and/or
dampened sinusoid.
ARM A(p, q):P ACF shows exponential decay and/or
dampened sinusoid.

Time Series Analysis and Forecasting p. 109/11

Fitting

Suppose that the observed series has, if necessary, been

differenced to reduce it to stationarity and that we wish to fit
an ARM A(p, q) model with known p and q to the stationary
series. Fitting the model then amounts to estimating the
p + q + 1 parameters, the coefficients for the AR(p) process,
the coefficients for the M A(q) process and 2 .
The three main methods of estimation are (Gaussian)
maximum likelihood, least squares and solving the
Yule-Walker equations. (We are not going to talk about
these methods in detail, Sorry!)

Time Series Analysis and Forecasting p. 110/11

Diagnostics

Having estimated parameters we need to check whether the

model fits, and, if it doesnt, detect where it is going wrong.
Specific checks could include:

Time Plot of Residuals. A plot of residuals, et , against t

should not show any structure. Similarly for a plot of et
against Y (t).
ACF and PACF of Residuals. The hope is that these will
show the White Noise pattern: approximately
independent, mean zero, normal values, each with
variance n1 .

Time Series Analysis and Forecasting p. 111/11

Diagnostics (cont.)

The Box-Pierce (and Ljung-Box) test examines the Null

of independently distributed residuals. Its derived from
the idea that the residuals of a "correctly specified"
model are independently distributed. If the residuals are
not, then they come from a miss-specified model.

Time Series Analysis and Forecasting p. 112/11

Summary of R commands

arima.sim() was used to simulate ARIM A(p, d, q) models

acf () plots the Autocorrelation Function
pacf () plots the Partial Autocorrelation Function
arima(data, order = c(p, d, q)) this function can be used to
estimate the parameters of an ARIM A(p, d, q) model
tsdiag() produces a diagnostic plot of fitted time series
model
predict() can be used for predicting future values of the
levels under the model

Time Series Analysis and Forecasting p. 113/11

Exercises

Fit an ARIMA model to ts1.txt

Fit an ARIMA model to ts2.txt
Fit an ARIMA model to ts3.txt
Fit an ARIMA model to ts4.txt

Time Series Analysis and Forecasting p. 114/11

THANK YOU!!!

Time Series Analysis and Forecasting p. 115/11