Type author name/s here

Dougherty

Introduction to Econometrics,
5th edition
Chapter heading
Chapter 3: Multiple Regression
Analysis

© Christopher Dougherty, 2016. All rights reserved.

 PROPERTIES OF THE MULTIPLE REGRESSION COEFFICIENTS

ASSUMPTIONS FOR MODEL A

A.1 The model is linear in parameters and correctly specified.

Y   1   2 X 2  ...   k X k  u

A.2 There does not exist an exact linear relationship among the regressors
in the sample.

A.3 The disturbance term has zero expectation

A.4 The disturbance term is homoscedastic

A.5 The values of the disturbance term have independent distributions

A.6 The disturbance term has a normal distribution.

Moving from the simple to the multiple regression model, we start by restating the
regression model assumptions.

1
 PROPERTIES OF THE MULTIPLE REGRESSION COEFFICIENTS

ASSUMPTIONS FOR MODEL A

A.1 The model is linear in parameters and correctly specified.

Y   1   2 X 2  ...   k X k  u

A.2 There does not exist an exact linear relationship among the regressors
in the sample.

A.3 The disturbance term has zero expectation

A.4 The disturbance term is homoscedastic

A.5 The values of the disturbance term have independent distributions

A.6 The disturbance term has a normal distribution.

Only A.2 is different. Previously it stated that there must be some variation in the X
variable. We will explain the difference in one of the following slideshows.

2
 PROPERTIES OF THE MULTIPLE REGRESSION COEFFICIENTS

ASSUMPTIONS FOR MODEL A

A.1 The model is linear in parameters and correctly specified.

Y   1   2 X 2  ...   k X k  u

A.2 There does not exist an exact linear relationship among the regressors
in the sample.

A.3 The disturbance term has zero expectation

A.4 The disturbance term is homoscedastic

A.5 The values of the disturbance term have independent distributions

A.6 The disturbance term has a normal distribution.

Provided that the regression model assumptions are valid, the OLS estimators in the
multiple regression model are unbiased and efficient, as in the simple regression model.

3
 PROPERTIES OF THE MULTIPLE REGRESSION COEFFICIENTS

True model Fitted model

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3

 X X 
 2i 2 i  3i 3
 Y  Y   X  X  2

   X 3 i  X 3   Yi  Y    X 2 i  X 2   X 3 i  X 3 
ˆ
2 
  X 2i  X 2    X 3i  X 3      X 2i  X 2   X 3i  X 3  
2 2 2

We will not attempt to prove efficiency. We will however outline a proof of unbiasedness.

4
 PROPERTIES OF THE MULTIPLE REGRESSION COEFFICIENTS

True model Fitted model

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3

 X X 
 2i 2 i  3i 3
 Y  Y   X  X  2

   X 3 i  X 3   Yi  Y    X 2 i  X 2   X 3 i  X 3 
ˆ
2 
  X 2i  X 2    X 3i  X 3      X 2i  X 2   X 3i  X 3  
2 2 2

Yi  Y    1   2 X 2 i   3 X 3 i  ui     1   2 X 2   3 X 3  u 
  2  X 2 i  X 2    3  X 3 i  X 3   ui  u

The first step, as always, is to substitute for Y from the true relationship. The Y ingredients
of ̂ 2 are actually in the form of Yi minus its mean, so it is convenient to obtain an
expression for this.
5
 PROPERTIES OF THE MULTIPLE REGRESSION COEFFICIENTS

True model Fitted model

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3

 X X 
 2i 2 i  3i 3
 Y  Y   X  X  2

   X 3 i  X 3   Yi  Y    X 2 i  X 2   X 3 i  X 3 
ˆ
2 
  X 2i  X 2    X 3i  X 3      X 2i  X 2   X 3i  X 3  
2 2 2

Yi  Y    1   2 X 2 i   3 X 3 i  ui     1   2 X 2   3 X 3  u 
  2  X 2 i  X 2    3  X 3 i  X 3   ui  u

ˆ2   2   ai*2 ui

After substituting, and simplifying, we find that ̂ 2 can be decomposed into the true value b2
plus a weighted linear combination of the values of the disturbance term in the sample.

6
 PROPERTIES OF THE MULTIPLE REGRESSION COEFFICIENTS

True model Fitted model

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3

 X X 
 2i 2 i  3i 3
 Y  Y   X  X  2

   X 3 i  X 3   Yi  Y    X 2 i  X 2   X 3 i  X 3 
ˆ
2 
  X 2i  X 2    X 3i  X 3      X 2i  X 2   X 3i  X 3  
2 2 2

Yi  Y    1   2 X 2 i   3 X 3 i  ui     1   2 X 2   3 X 3  u 
  2  X 2 i  X 2    3  X 3 i  X 3   ui  u

ˆ2   2   ai*2 ui

This is what we found in the simple regression model. The difference is that the expression
for the weights, which depend on all the values of X2 and X3 in the sample, is considerably
more complicated.
7
 PROPERTIES OF THE MULTIPLE REGRESSION COEFFICIENTS

True model Fitted model

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3

 X X 
 2i 2 i  3i 3
 Y  Y   X  X  2

   X 3 i  X 3   Yi  Y    X 2 i  X 2   X 3 i  X 3 
ˆ
2 
  X 2i  X 2    X 3i  X 3      X 2i  X 2   X 3i  X 3  
2 2 2

Yi  Y    1   2 X 2 i   3 X 3 i  ui     1   2 X 2   3 X 3  u 
  2  X 2 i  X 2    3  X 3 i  X 3   ui  u

ˆ2   2   ai*2 ui
E  ˆ2    2  E   ai*2 ui    2   E  ai*2 ui    2   ai*2 E  ui    2

Having reached this point, proving unbiasedness is easy. Taking expectations, b2 is

unaffected, being a constant. The expectation of a sum is equal to the sum of expectations.

8
 PROPERTIES OF THE MULTIPLE REGRESSION COEFFICIENTS

True model Fitted model

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3

 X X 
 2i 2 i  3i 3
 Y  Y   X  X  2

   X 3 i  X 3   Yi  Y    X 2 i  X 2   X 3 i  X 3 
ˆ
2 
  X 2i  X 2    X 3i  X 3      X 2i  X 2   X 3i  X 3  
2 2 2

Yi  Y    1   2 X 2 i   3 X 3 i  ui     1   2 X 2   3 X 3  u 
  2  X 2 i  X 2    3  X 3 i  X 3   ui  u

ˆ2   2   ai*2 ui
E  ˆ2    2  E   ai*2 ui    2   E  ai*2 ui    2   ai*2 E  ui    2

The a* terms are nonstochastic since they depend only on the values of X2 and X3, and
these are assumed to be nonstochastic. Hence the a* terms may be taken out of the
expectations as factors.
9
 PROPERTIES OF THE MULTIPLE REGRESSION COEFFICIENTS

True model Fitted model

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3

 X X 
 2i 2 i  3i 3
 Y  Y   X  X  2

   X 3 i  X 3   Yi  Y    X 2 i  X 2   X 3 i  X 3 
ˆ
2 
  X 2i  X 2    X 3i  X 3      X 2i  X 2   X 3i  X 3  
2 2 2

Yi  Y    1   2 X 2 i   3 X 3 i  ui     1   2 X 2   3 X 3  u 
  2  X 2 i  X 2    3  X 3 i  X 3   ui  u

ˆ2   2   ai*2 ui
E  ˆ2    2  E   ai*2 ui    2   E  ai*2 ui    2   ai*2 E  ui    2

By Assumption A.3, E(ui) = 0 for all i. Hence E(ˆ2 ) is equal to b2 and so ̂ 2 is an unbiased
estimator. Similarly ̂ 3 is an unbiased estimator of b3.

10
 PROPERTIES OF THE MULTIPLE REGRESSION COEFFICIENTS

True model Fitted model

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3

ˆ1  Y  ˆ2 X 2  ˆ3 X 3

Finally we will show that ̂ 1 is an unbiased estimator of b1. This is quite simple, so you
should attempt to do this yourself, before looking at the rest of this sequence.

11
 PROPERTIES OF THE MULTIPLE REGRESSION COEFFICIENTS

True model Fitted model

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3

ˆ1  Y  ˆ2 X 2  ˆ3 X 3

   1   2 X 2   3 X 3  u   ˆ2 X 2  ˆ3 X 3

First substitute for the sample mean of Y.

12
 PROPERTIES OF THE MULTIPLE REGRESSION COEFFICIENTS

True model Fitted model

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3

ˆ1  Y  ˆ2 X 2  ˆ3 X 3

   1   2 X 2   3 X 3  u   ˆ2 X 2  ˆ3 X 3

E  ˆ1    1   2 X 2   3 X 3  E  u   X 2 E  ˆ2   X 3 E  ˆ3 

 1   2 X 2   3 X 3  X 2  2  X 3  3
 1

Now take expectations. The first three terms are nonstochastic, so they are unaffected by
taking expectations.

13
 PROPERTIES OF THE MULTIPLE REGRESSION COEFFICIENTS

True model Fitted model

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3

ˆ1  Y  ˆ2 X 2  ˆ3 X 3

   1   2 X 2   3 X 3  u   ˆ2 X 2  ˆ3 X 3

E  ˆ1    1   2 X 2   3 X 3  E  u   X 2 E  ˆ2   X 3 E  ˆ3 

 1   2 X 2   3 X 3  X 2  2  X 3  3
 1

The expected value of the mean of the disturbance term is zero since E(u) is zero in each
observation. We have just shown that E(̂ 2 ) is equal to b2 and that E(̂ 3 ) is equal to b3.

14
 PROPERTIES OF THE MULTIPLE REGRESSION COEFFICIENTS

True model Fitted model

Y  1   2 X 2   3 X 3  u Ŷ  ˆ1  ˆ2 X 2  ˆ3 X 3

ˆ1  Y  ˆ2 X 2  ˆ3 X 3

   1   2 X 2   3 X 3  u   ˆ2 X 2  ˆ3 X 3

E  ˆ1    1   2 X 2   3 X 3  E  u   X 2 E  ˆ2   X 3 E  ˆ3 

 1   2 X 2   3 X 3  X 2  2  X 3  3
 1

Hence ̂ 1 is an unbiased estimator of b1.

15
Copyright Christopher Dougherty 2016.

These slideshows may be downloaded by anyone, anywhere for personal use.

Subject to respect for copyright and, where appropriate, attribution, they may be
used as a resource for teaching an econometrics course. There is no need to
refer to the author.

The content of this slideshow comes from Section 3.3 of C. Dougherty,

Introduction to Econometrics, fifth edition 2016, Oxford University Press.
Additional (free) resources for both students and instructors may be
downloaded from the OUP Online Resource Centre
www.oxfordtextbooks.co.uk/orc/dougherty5e/.

Individuals studying econometrics on their own who feel that they might benefit
from participation in a formal course should consider the London School of
Economics summer school course
EC212 Introduction to Econometrics
http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx
or the University of London International Programmes distance learning course
EC2020 Elements of Econometrics
www.londoninternational.ac.uk/lse.

2016.04.28