Multicollinearity 2023

Multicollinearity
1
Review the assumptions
 Assumption of OLS
 The regression model is linear in parameters
 Values X is fixed in repeated sampling
 Expected value of the random error (disturbance)=0 for each i
 Homoscedasticity or equal variance of the error term
 No autocorrelations between the error terms (disturbnaces)
 Zero covariance between the error term εi and the Xi
 The number of observations must be greater than the number of
parameters to be estimated
 Variability in X values
 The regression model is correctly specified
 There is no perfect multcollinearity
2
Outline
• Nature of Multicollinearity.
• Sources of Multicollinearity
• Implications of Multicollinearity.
• How to detect Multicollinearity?
• Remedial Measures for
Multicollinearity.
3
Nature of Multicollinearity
What’s Multicollinearity?
It measures the existence of a perfect or exact linear

relationship among some or all explanatory variable
of a regression model.
4
Hala Abou-Ali (FEPS) - Econometrics I April 25, 2011 5
Estimation in the presence of PERFECT
Multicollinearity?
 Well you cannot, remember X’X is singular and you cannot
get its inverse.
Rank(X´X) = Rank(X) this rank will be less than number of columns in
(X´X)& (X) or less than the number of parameters, We can not
estimate the parameters
If the matrix X is not in a full rank, its determinant would be zero.
we cannot get the inverse of the matrix, hence we cannot estimate the
parameters (ie. 𝛽𝑠 will not exist).
7
Sources of Multicollinearity
• the most important source of multicollinearity is the existing of a

common trend especially in time series data, may be that the
regressors included in the model share a common trend, that is,
they all increase or decrease over time.

Implications of Multicollinearity (Imperfect-Multicollinearity)
• The Coefficients remain BLUE.

• OLS estimators are BLUE but with large variances or Large
standard errors:
• so, the estimated coefficients are not precise
• although OLS estimates have large variance, these
variances are still the lowest ones among the other
estimation methods, so they are still efficient).
• Large Confidence Intervals
(The probability of not rejecting the null hypothesis increases ).
• The value of the t statistic decreases
8
Implications of Multicollinearity: Practical Consequences
1. Large Variance and Covariance of the estimated

coefficients
9
• Variance Inflation Factor (VIF):

• Measures the speed with which the variance and
covariance increase in case of multicollinearity.
• It shows how the variance of an estimation is inflated by
the presence of multicollinearity.
13
• The Variance and the VIF:

• The variances of the estimated parameters are directly
proportional to VIF.
14
Implication
Implicationsof Multicollinearity: Practical Consequences
of Multicollinearity: Practical Consequences
• In a model of (K-1) variables
• Where R2j is from the regression of xj on the remaining K-2

regressors.
15
• We can also use the Tolerance
18
Implications
Implication of Multicollinearity:
Practical Cnsequences
2. Wider Confidence Intervals
• As the standard errors are large.
• The confidence intervals are wider.
• The probability of do not rejecting the null hypothesis

increases.
19
Implication
3. Insignificant t ratios
• As the standard errors increases with high collinearity.
• t value decreases.
• One will increasingly accept the null hypothesis that

the relevant true parameter in the population is zero.
20
Implication
4. High R2 but few significant t ratio
• One or more coefficient are not significant according to

t ratios.
• But R2 may be so high.
• Based on F test, we reject the null hypothesis
21
Remarks!
• 1- Multicollinearity is a sample phenomenon:
Even if the X variables are not linearly related in the population, they
may be related in a particular sample.
• 2- The same effect of imperfect multicollinearity can be obtained in
case of very small number of observations ( near micronumerosity)
𝜎2
In simple regression: 𝑣𝑎𝑟 𝛽2 = σ
(𝑋 𝑖 −𝑋ത )2
• When sample size is small, (𝑋 ′ 𝑋) or σ(𝑋 − 𝑋ത)2 becomes small, and as it is in the
𝑖
denominator, 𝑣𝑎𝑟 𝛽 would be large.
• Exact micronumerosity arises when n, the sample size is less than the
number of parameters to be estimated (𝑛 < 𝑛𝑜. 𝑜𝑓 𝛽𝑠 ), in this case we
cannot estimate the parameters (cannot get unique estimate of each)
• 3- The same effect of multicollinearity when the independent
variables have very small variances. In this case the value (𝑋 ′ 𝑋) or
σ(𝑋𝑖 − 𝑋)ത 2 becomes small and as it is in the denominator, var(β ̂ )
would be large.
22
How
Howtotodetect Multicollinearity
detect Multicollinearity
How to detect Multicllinearity?

1. High R² but few significant t ratios.
As there is contradiction between the significance of
the whole model and the significance of the individual
coefficients.
2- High pair wise correlation among regressors:
• If it is higher than 0.8.
• if the pair-wise or zero-order correlation coefﬁcient
between two regressors is high, say, in excess of 0.8,
then multicollinearity is a serious problem.
• It is a sufficient but not a necessary condition. in models
involving more than two explanatory variables
23
How
Howtotodetect Multicollinearity
detect Multicollinearity
3- Auxiliary Regressions
• Consist of regression each Xj on the others Xs and get the R²

of these regressions.
• Compute the F statistic: If F calculated > F tabulated, so the
Xi is correlated with the other variables.
• Klein’s Rule (1962): if the R² obtained from one auxiliary
regression is greater than the overall R² of the original model,
so multicollinearity is a problem!
• 4. Tolerance and VIF.
24
Remedy
Remedy Measure for Multicollinearity
Measures for Multicollinearity
Remedy Measure for Multicollinearity

1. Do Nothing!
• multicollinearity is essentially a data deﬁciency problem and
some times we have no choice over the data we have available
for empirical analysis.
2. Using the valid linear restriction between the parameters
• For example, in the Cobb–Douglas–type production function if

one expects constant returns to scale to prevail, then, in which
case we could regress the output-labor ratio on the capital-
labor ratio. If there is collinearity between labor and capital, as
generally is the case in most sample data, such a transformation
may reduce or eliminate the collinearity problem but we should
to test for the validity of such restrictions.
27
3. Transformation of variables
• The most used transformation in practice is the ratio
transformation.
4. Dropping variables and Specification bias.
In situation of extreme multicollinearity dropping the
highly collinear variable will often make the other X
variable statistically significant. but take care of the
misspecification problem which arises due to excluding
relevant variables according to the theory and this
problem leads to biased estimates.
5. Additional or new data.
29
3. Transformation of variables
• The most commonly used transformation in practice is the
ratio transformation.
4. Dropping variables and Specification bias.
In situation of extreme multicollinearity dropping the
highly collinear variable will often make the other X
variable statistically significant. but take care of the
misspecification problem which arises due to excluding
relevant variables according to the theory and this
problem leads to biased estimates.
5. Additional or new data.
32

Multicollinearity 2023

Uploaded by

Copyright:

Available Formats

Multicollinearity 2023

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Multicollinearity 2023

Uploaded by

Copyright:

Available Formats

Multicollinearity

It measures the existence of a perfect or exact linear

If the matrix X is not in a full rank, its determinant would be zero.

• the most important source of multicollinearity is the existing of a

Hala Abou-Ali (FEPS) - Econometrics I April 25, 2011 7

• The Coefficients remain BLUE.

1. Large Variance and Covariance of the estimated

• Variance Inflation Factor (VIF):

• The Variance and the VIF:

• In a model of (K-1) variables

• Where R2j is from the regression of xj on the remaining K-2

• We can also use the Tolerance

2. Wider Confidence Intervals

• As the standard errors are large.

• The confidence intervals are wider.

• The probability of do not rejecting the null hypothesis

• As the standard errors increases with high collinearity.

• One will increasingly accept the null hypothesis that

4. High R2 but few significant t ratio

• One or more coefficient are not significant according to

• But R2 may be so high.

• Based on F test, we reject the null hypothesis

How to detect Multicllinearity?

• Consist of regression each Xj on the others Xs and get the R²

• 4. Tolerance and VIF.

Remedy Measure for Multicollinearity

2. Using the valid linear restriction between the parameters

• For example, in the Cobb–Douglas–type production function if

5. Additional or new data.

5. Additional or new data.

You might also like