GUJARATI D, PORTER D,

2008: Basic
EconometricS 5th
EDITION SUMMARY OF
CHAPTER 3-5
Kelompok 2 Finance 4A
Two-Variable regression
model: the problem of CHAPTER 3
estimation
 Method of ordinary
least square (ols)
● First, we use SRF which is
observable:
Yi = ˆβ1 + ˆβ2Xi + ui
= Yˆi + ˆui
● û1i = (Yi −Yˆ1i)

● The OLS Method provides β1 and

β2 that give the smallest possible
value of the sum square error.
 ORDINARY LEAST SQUARE
1. The OLS estimators are expressed solely in terms of the observable (i.e., sample) quantities (i.e., X and Y).
Therefore, they can be easily computed.

2. They are point estimators; that is, given the sample, each estimator will provide only a single (point) value of
the relevant population parameter.

3. Once the OLS estimates are obtained from the sample data, the sample regression line can be easily
obtained.

● Ӯ = ˆ β1 + ˆ β2¯X, where ¯X = mean

● The mean value of the estimated Y = Yˆi is equal to the

mean value of the actual Y.

● The mean value of the residuals ˆui is zero.

● The residuals ˆu i are uncorrelated with the predicted Yi.

● The residuals ˆu i are uncorrelated with Xi ; that is, sum of

ˆui Xi = 0.
The Classical Linear
Regression Model
(CLRM)
 cLASSICAL lINEAR REGRESSION MODEL

Assumptions:

1. Linear Regression Model: The regression model is linear in the parameters, though it
may or may not be linear in the variables.

Yi = β1 + β2 Xi + ui

2. Fixed X Values or X Values Independent of the Error Term: Values taken by the
regressor X may be considered fixed in repeated samples or they may be sampled along
with the dependent variable Y (the case of stochastic regressor).

3. Zero Mean Value of Disturbance ui: Given the value of Xi, the mean, or expected, value
of the random disturbance term ui is zero.

4. Homoscedasticity or Constant Variance of ui: The variance of the error, or disturbance,

term is the same regardless of the value of X.
 cLASSICAL lINEAR REGRESSION MODEL

Assumptions:

5. No Autocorrelation between the Disturbances: Given any two X values, Xi and Xj

(i≠ j), the correlation between any two ui and uj (i≠j) is zero. In short, the
observations are sampled independently.

6. The Number of Observations n Must Be Greater than the Number of

Parameters to Be Estimated: Alternatively, the number of observations must be
greater than the number of explanatory variables.

7. The Nature of X Variables: The X values in a given sample must not all be the
same

Before we turn to this theorem, which provides the theoretical justification for
the popularity of OLS, we first need to consider the precision or standard
errors of the least-squares estimates.

Patterns of correlation among the disturbances.

(a) positive serial correlation; (b) negative serial
correlation; (c) zero correlation.
 Precision or Standard Errors of
Least-Squares Estimates
● In statistics the precision of an estimate is measured by its standard error
(se).

where ˆσ2 is the OLS estimator of the true but unknown σ2 and where the
expression n − 2 is known as the number of degrees of freedom (df), Σû2i
being the sum of the residuals squared or the residual sum of squares
(RSS).

● ˆσ is known as the standard error of estimate or the standard error of

the regression (se) and it represents the (conditional) variance of both ui
and Yi
 Precision or Standard Errors of
Least-Squares Estimates

Features of the variances (standard errors) of ˆβ1 and ˆβ2:

1. The variance of ˆβ2 is directly proportional to σ2 but inversely

proportional to Σx2i .

2. The variance of ˆβ1 is directly proportional to σ2 and Σx2i but

inversely proportional to Σx2i and the sample size n.

3. Since ˆβ1 and ˆβ2 are estimators, they will not only vary from sample
to sample but in a given sample they are likely to be dependent on
each other, this dependence being measured by the covariance
between them.
Properties of
least-squares
estimators: The
gauss-markov theorem
 Gauss-markov theorem

As explained in Appendix A, an estimator, say the OLS estimator ˆ β2, is said to be a best linear
unbiased
estimator (BLUE) of β2 if the following hold:

1. It is linear, that is, a linear function of a random variable, such as the dependent variable
Y in the regression model.
2. It is unbiased, that is, its average or expected value, E( ˆ β2), is equal to the true value, β2.
3. It has minimum variance in the class of all such linear unbiased estimators; an unbiased
estimator with the least variance is known as an efficient estimator.
Gauss-markov theorem
The Coefficient of
Determination r2: A
Measure of
“Goodness of Fit”
 Goodness of fit

The coefficient of determination r2 (two-variable case) or R2

(multiple regression) is a summary measure that tells how well the sample regression
line fits
the data.

Before we show how r2 is computed, let us consider a heuristic explanation of r2 in

terms of a graphical device, known as the Venn diagram, or the Ballentine, as shown
in Figure 3.8.

In this figure, circle Y represents

variation variation in the dependent
variable Y
Circle X represents variation in the
explanatory variable X

The overlap of the two circles results in r2

increases , and r2 decreases when no
overlap
 Goodness of fit

There are several ways to compute r2

03

01 02
04
EXAMPLE
 EXAMPLE

Regression line : Yˆi = −0.0144 + 0.7240Xi

 Summary of chapter 3

● OLS method doesn’t use Population Regression Function instead it utilizes the Sample
chosen which are observable.
● The OLS method provides the smallest value of sum square error.
● The basic framework of regression analysis is the CLRM. The CLRM is based on a set of
assumptions.
● Based on these assumptions, the least-squares estimators take on certain properties
summarized in the Gauss–Markov theorem, which states that in the class of linear
unbiased estimators, the least-squares estimators have minimum variance. In short,
they are BLUE
● The precision of OLS estimators is measured by their standard errors.
● The overall goodness of fit of the regression model is measured by the coefficient of
determination, r 2.
 CHAPTER 4

04. Classical Normal

Linear Regression Model
(CNLRM)
 CLASSICAL NORMAL LINEAR
REGRESSION MODEL

● The classical theory of statistical inference consists of two branches, namely,

estimation and hypothesis testing. We have thus far covered the topic of
estimation.

● Under the assumptions of the CLRM, we were able to show that the estimators
of these parameters, βˆ1, βˆ2, and σˆ2, Note that, since these are estimators,
their values will change from sample to sample, they are random variables.

● In regression analysis our objective is not only to estimate the SRF, but also to
use it to draw inferences about the PRF. We need to find out their probability
distributions, otherwise, we will not be able to relate them to their true values.
 The Probability Distribution of
Disturbances
● Consider βˆ2. As showed in Appendix 3A.2

● βˆ2 = ∑ kiYi (4.1.1)

● Where ki= xi/ ∑xi2. But since the X’s are assumed fixed, Eq. (4.1.1) shows that βˆ2is a linear function of Yi,
which is random by assumption. But since Yi = β1 + β2Xi + ui, we can write (4.1.1) as

● βˆ2 = ∑ ki(β1 + β2Xi + ui) (4.1.2)

● Because ki, the betas, and Xi are all fixed, βˆ2 is ultimately a linear function of ui,which is random by
assumption. Therefore, the probability distribution of βˆ2 (and also of βˆ1) will depend on the
assumption made about the probability distribution of u

● OLS does not make any assumption about the probabilistic nature of ui. This void can be filled if we are
willing to assume that the u’s follow some probability distribution.

● For reasons to be explained shortly, in the regression context it is usually assumed that the u’s follow the
normal distribution.
 The Probability Distribution of
Disturbances

Recall that error disturbances :

1. Have zero expectations
2. Are Uncorrelated
3. Have a constant variance

In the regression context, it it usually assumed that the u’s

follow the normal distribution
 The Probability Distribution of Disturbances

● We have studied studied the expected expected value and

variance of the OLS estimators

● In order to do inference inference, we need to know the full

sampling distribution of the estimator

● To make this sampling sampling distribution distribution tractable

tractable, we now assume that the unobserved error term (u) is
normally normally distributed distributed in the population.

● This is often referred to as the normality assumption.

 Normality assumption

● We continue continue to make the assumptions assumptions

introduced introduced in the previous lecture (linear regression, no
perfect collinearity, zero conditional mean, homoskedasticity
homoskedasticity, …).

● And we add the following:

○ Normality – The population error u is independent of the
explanatory variables x1, x2,…,xk, and is normally distributed
with zero mean and variance σ2: u ~ Normal(0, σ2)
 NORMALITY ASSUMPTION -
Why are we assuming normality?

● Answer: It implies implies that the OLS estimator

estimator follows follows a normal distribution too. And
this makes it straightforward to do inference.

● Under the CLM assumptions (1‐7), conditional on the

sample values of the independent variables,
 NORMALITY ASSUMPTION
● The result that :

● Implies that :

○ In words, this says that the deviation between the estimated value and the true
parameter value, divided by the standard standard deviation deviation of the estimator
estimator, is normally normally distributed with mean zero and variance equal to 1.

○ The assumptions 1—7 are call de the cl il ass cal linear model (CLM) assumptions

○ One immediate implication of the CLM assumptions assumptions is that, conditional

conditional on the explanatory variables, the dependent variable y has a normal
distribution distribution with constant constant variance
 NORMALITY ASSUMPTION -
How to justify normality ?

● Central Central limit theorem theorem (CLT): the residual residual u is the
sum of many different factors; and by the CLT the sum of many random
variables is normally distributed

● This argument is not without weaknesses (e.g. doesn’t hold if u is not

additive).

● Whether normality holds in a particular application is an empirical matter

– which can be investigated

● Sometimes using a transformation – e.g. taking the log – yields a

distribution that is closer to normal.
 Properties of ols estimators under the normality
assumption

With ui follow the normal distribution, OLS estimators have the following properties:
1. They are unbiased.
2. They have minimum variance. Combined with 1., this means that they are
minimum-variance unbiased, or efficient estimators.
3. They have consistency; that is, as the sample size increases indefinitely, the
estimators converge to their true population values.
■ βˆ1(being a linear function of ui) is normally distributed with
● Mean: E(βˆ1) = β1 (4.3.1)
● var (βˆ1): σ2βˆ1 = (∑ X2i/n ∑ x2i)σ2 (4.3.2)
Or more compactly,
■ βˆ1 ∼ N (β1, σ2β ˆ1)
Then by the properties of the normal distribution the variable Z, which is defined as:
■ Z = (βˆ1 − β1 )/ σβˆ1(4.3.3)
 Properties of ols estimators under the
normality assumption
Follows the standard normal distribution, that is, a normal distribution with zero mean and unit
( = 1) variance, or

● Z ∼ N(0, 1)

βˆ2(being a linear function of ui) is normally distributed with

● Mean: E(βˆ2) = β2 (4.3.4)

● var (βˆ2): σ2βˆ2 =σ2/ ∑ x2i (4.3.5)

Or, more compactly,

● βˆ2 ∼ N(β2, σ2βˆ2)

Then, as in (4.3.3),

● Z = (βˆ2 − β2 )/σβˆ2 (4.3.6)

also follows the standard normal distribution.

Geometrically, the probability distributions of βˆ1 and βˆ2 are shown in Figure 4.1.
Figure 4.1
 Properties of ols estimators under the
normality assumption

● (n− 2)( ˆσ2/σ 2) is distributed as the χ2 (chi-square) distribution with (n − 2)df.

● (βˆ1, βˆ2) are distributed independently of σˆ2.

● βˆ1 and βˆ2 have minimum variance in the entire class of unbiased
estimators, whether linear or not. This result, due to Rao, is very powerful
because, unlike the Gauss–Markov theorem, it is not restricted to the class of
linear estimators only. Therefore, we can say that the least-squares estimators
are best unbiased estimators (BUE); that is, they have minimum variance in
the entire class of unbiased estimators.
 Summary of chapter 4

● The classical normal linear regression model (CNLRM) model differs from the classical
linear regression model (CLRM). The CLRM does not require any assumption about the
probability distribution of ui ; it only requires that the mean value of ui is zero and its
variance is a finite constant.

● The theoretical justification for the normality assumption is the central limit theorem.

● Without the normality assumption, under the other assumptions discussed in Chapter
3, the Gauss–Markov theorem showed that the OLS estimators are BLUE.

● With the additional assumption of normality, the OLS estimators are not only best
unbiased estimators (BUE) but also follow well-known probability distributions.

● In this text, we will largely rely on the OLS method for practical reasons: (a) Compared
to ML, the OLS is easy to apply; (b) the ML and OLS estimators of β1 and β2 are
identical (which is true of multiple regressions too); and (c) even in moderately large
samples the OLS and ML estimators of σ2 do not differ vastly
Interval estimation
CHAPTER 5
and hypothesis CHAPTER 5
testing
 Interval estimation

● Because of sampling fluctuations, a single estimate is likely to differ from the

true value, although in repeated sampling its mean value is expected to be
equal to the true value.
● Now in statistics, the reliability of a point estimator is measured by its standard
error. Therefore, instead of relying on the point estimate alone, we may
construct an interval around the point estimator
●

●
 Aspects of interval estimation

● The probability of constructing an interval that contains

● Since the Confidence Interval is random, the probability
statements attached to it should be understood in the
long-run sense, that is, repeated sampling.
● If a specific sample and obtain a specific numerical value
of , the interval is no longer random, it is fixed
Confidence intervals for regression coefficient β1 and β2
Confidence intervals for regression coefficient β1 and β2
 Confidence intervals for regression coefficient β1 and β2

Confidence interval
● coefficient t table (Std. Error)

β2
0.724097 2.201(0.069581)
0.724097 0.1540
0.570097 β2 0.878097

Same for β1
-1.8871 β1 1.8583

Regression from table 2.6

Confidence interval for σ2
 Hypothesis testing

● The problem of statistical hypothesis testing may be stated simply as

follows: “Is a given observation or finding compatible with some
stated hypothesis or not?”
● the stated hypothesis is known as the null hypothesis and is denoted
by the symbol H0.
● The null hypothesis is usually tested against an alternative hypothesis
(also known as maintained hypothesis) denoted by H
● The theory of hypothesis testing is concerned with developing rules or
procedures for deciding whether to reject or not reject the null
hypothesis.
● There are two mutually complementary approaches for devising such
rules, namely, confidence interval and test of significance.
 Hypothesis Testing

Two Sided or Two Tail Test One Sided or One Tail Test

● Two-sided alternative hypothesis reflects the fact that we do not have a strong a priori
or theoretical expectation about the direction in which the alternative hypothesis
should move from the null hypothesis

● One-sided alternative hypothesis reflects the fact that we have a strong a priori or
theoretical expectation (or expectations based on some previous empirical work) that
the alternative hypothesis is one-sided or unidirectional rather than two-sided
 Hypothesis Testing : The Confidence Interval Approach

Two Sided or Two Tail Test One Sided or One Tail Test

Decision rule:
Construct 100(1-)% confidence interval for 2 . If the 2
under H0 falls within this confidence interval, do not reject
H0 , but if falls outside this interval, reject H0 .

● In statistics, when we reject the null hypothesis, we

say that our finding is statistically significant. On the
other hand, when we do not reject the null
hypothesis, we say that our finding is not statistically
significant
 Hypothesis Testing : The test of significance Approach

● Used to verify the truth or falsity of null hypothesis

● Testing the significance of regression coefficient: the
t-test

df = n-2 (for T table)

● In the language of significance tests, a

statistic is said to be statistically
significant if the value of the test
statistic lies in the critical region.
 Hypothesis Testing : The test of significance Approach

Decision rule :

Notes:
● β*2 is the hypothesized numerical value of β2.
● |t | means the absolute value of t. tα or tα/2 means the critical t
value at the α or α/2 level of significance.
● df: degrees of freedom, (n − 2) for the two-variable model, (n − 3)
for the three-variable model, and so on
 Testing the Significance of σ2: The χ2 Test

Decision Rules:

Note: σ2 0 is the value of σ2 under the null hypothesis.

The first subscript on χ2 in the last column is the level of significance, and the second subscript is the
degrees of freedom. These are critical chi-square values. Note that df is (n − 2) for the two-variable
regression model, (n − 3) for the three-variable regression model, and so on.
 Hypothesis Testing: Some Practical Aspects

● The Meaning of “Accepting” or “Rejecting” a Hypothesis

● The “Zero” Null Hypothesis and the “2-t” Rule of Thumb
● Forming the Null and Alternative Hypotheses
● Choosing α, the Level of Significance
● The Exact Level of Significance: The p Value
● Statistical Significance versus Practical Significance
 Regression analysis and analysis of variance

TSS = ESS + RSS

TSS = Total Sum of Squares / ANOVA df=n-1
ESS = Explained Sum of Squares
RSS = Residual Sum of Squares df=n-2

F ratio provides a test statistic to test the null hypothesis that true β2 is zero. All that needs to be done is
to compute the F ratio and compare it with the critical F value obtained from the F tables at the chosen
level of significance, or obtain the p value of the computed F statistic.
 Evaluating the result of regression analysis

Evaluate the model

● the signs of the estimated coefficient in accordance with
theoretical or prior expectation?
● Statistically significance? Check using t-test, F-test
● Explain variation? Using r^2
● Satisfies the assumption of CNLRM? Check normality
disturbance term ui
 Normality test: the Jarque-Bera (JB) test

H0: Residuals are normally distributed

H1: Residuals are not normally distributed

where S=skewness and K=kurtosis. For normal

distribution, S=0 and K=3

The JB statistic follows the 2 with 2 df

Decision Rules:
Reject H0 if P-value < 5% (α)
 Summary of chapter 5

● Estimation and hypothesis ● Significance test use a test

testing are parts of the two main statistic and examine its sampling
branches of classical statistics distribution under the null
● Consist of confidence Interval hypothesis. (Normal, t, F, or
and test of significance chi-squares)
● In choosing the p value,
● Underlying the confidence
investigator has to bear in mind
interval approach is the
the probabilities of committing
concept of interval
Type I errors and Type II errors
estimation.
● It is better to quote the p value of t
● Confidence interval often statistics
stated in percent (90 or 95).
If the null hypothesis value ● Normality test is used to
lies in the confidence find out whether the error
interval, the hypothesis is terms follow the normal
not rejected. distributions.
THANKS
Does anyone have any questions?

CREDITS: This presentation template was created by

Slidesgo, including icons by Flaticon, and
infographics & images by Freepik and illustrations
by Stories
AWESOME
WORDS
 Alternative RESOURCES
Find more illustrations like these on Stories by Freepik
Money ICONS
 Stories by Freepik
Create your Story with our illustrated concepts. Choose the style you like the most, edit its colors, pick
the background and layers you want to show and bring them to life with the animator panel! It will boost
your presentation. Check out how it works.

Pana Amico Bro Rafiki

 Use our editable graphic resources...
You can easily resize these resources, keeping the quality. To change the color, just ungroup the resource and
click on the object you want to change. Then, click on the paint bucket and select the color you want. Don’t
forget to group the resource again when you’re done.
 ...and our sets of editable icons
You can resize these icons, keeping the quality.
You can change the stroke and fill color; just select the icon and click on the paint bucket/pen.
In Google Slides, you can also use Flaticon’s extension, allowing you to customize and add even more icons.
Educational Icons Medical Icons
Business Icons Teamwork Icons
Help & Support Icons Avatar Icons
Creative Process Icons Performing Arts Icons
Nature Icons
SEO & Marketing Icons