Multiple Linear Regression (MLR)

Definition, Formula, and Example

What Is Multiple Linear Regression (MLR)?
Multiple linear regression (MLR), also known
simply as multiple regression, is a statistical
technique that uses several explanatory
variables to predict the outcome of a
response variable. The goal of MLR is to
model the linear relationship between the
explanatory (independent) variables and
response (dependent) variables. In essence,
multiple regression is the extension of
ordinary least-squares
(OLS) regression because it involves more
than one explanatory variable.
KEY TAKEAWAYS
 Multiple linear regression (MLR) is a
statistical technique that uses several
explanatory variables to predict the
outcome of a response variable.
 It is also known as multiple regression,
 Multiple regression is an extension of


linear (OLS) regression that uses just one

explanatory variable.
 MLR is used extensively in econometrics

and financial inference.

 Multiple regressions are used to make

forecasts, explain relationships between

financial variables, and test existing
theories.
Multiple Linear Regression
A statistical technique that is used to predict the outcome of a variable
based on the value of two or more variables

Written by Sebastian Taylor

Over 2 million + professionals use CFI to learn accounting, financial
analysis, modeling and more. Unlock the essentials of corporate finance
with our free resources and get an exclusive sneak peek at the first
module of each course.Start FreeStart Free

What is Multiple Linear Regression?

Multiple linear regression refers to a statistical technique that is used to
predict the outcome of a variable based on the value of two or more
variables. It is sometimes known simply as multiple regression, and it is
an extension of linear regression. The variable that we want to predict is
known as the dependent variable, while the variables we use to predict
the value of the dependent variable are known as independent or
explanatory variables.
Figure 1: Multiple linear regression model predictions for individual observations (Source)

Summary
 Multiple linear regression refers to
a statistical technique that uses
two or more independent
variables to predict the outcome
of a dependent variable.
 The technique enables analysts to
determine the variation of the
model and the relative
contribution of each independent
variable in the total variance.
 Multiple regression can take two
forms, i.e., linear regression and
non-linear regression.
Multiple Linear Regression Formula

Where:
 yi is the dependent or predicted variable
 β0 is the y-intercept, i.e., the value of y

when both xi and x2 are 0.

 β1 and β2 are the regression coefficients

representing the change in y relative to a

one-unit change in xi1 and xi2,
respectively.
 βp is the slope coefficient for each

independent variable
 ϵ is the model’s random error (residual)

term.
Understanding Multiple Linear
Regression
Simple linear regression enables
statisticians to predict the value of
one variable using the available
information about another variable.
Linear regression attempts to
establish the relationship between
the two variables along a straight
line.
Multiple regression is a type of
regression where the dependent
variable shows a linear relationship
with two or more independent
variables. It can also be non-linear,
where the dependent
and independent variables do not
follow a straight line.
Both linear and non-linear regression
track a particular response using two
or more variables graphically.
However, non-linear regression is
usually difficult to execute since it is
created from assumptions derived
from trial and error.
Assumptions of Multiple Linear
Regression
Multiple linear regression is based on
the following assumptions:
1. A linear relationship between
the dependent and independent
variables
The first assumption of multiple
linear regression is that there is a
linear relationship between the
dependent variable and each of the
independent variables. The best way
to check the linear relationships is to
create scatterplots and then visually
inspect the scatterplots for linearity.
If the relationship displayed in the
scatterplot is not linear, then the
analyst will need to run a non-linear
regression or transform the data
using statistical software, such as
SPSS.
2. The independent variables are
not highly correlated with each
other
The data should not show
multicollinearity, which occurs when
the independent variables
(explanatory variables) are highly
correlated. When independent
variables show multicollinearity,
there will be problems figuring out
the specific variable that contributes
to the variance in the dependent
variable. The best method to test for
the assumption is the Variance
Inflation Factor method.
3. The variance of the residuals
is constant
Multiple linear regression assumes
that the amount of error in the
residuals is similar at each point of
the linear model. This scenario is
known as homoscedasticity. When
analyzing the data, the analyst
should plot the standardized
residuals against the predicted
values to determine if the points are
distributed fairly across all the
values of independent variables. To
test the assumption, the data can be
plotted on a scatterplot or by using
statistical software to produce a
scatterplot that includes the entire
model.
4. Independence of observation
The model assumes that the
observations should be independent
of one another. Simply put, the
model assumes that the values of
residuals are independent. To test
for this assumption, we use the
Durbin Watson statistic.
The test will show values from 0 to 4,
where a value of 0 to 2 shows
positive autocorrelation, and values
from 2 to 4 show negative
autocorrelation. The mid-point, i.e., a
value of 2, shows that there is no
autocorrelation.
5. Multivariate normality
Multivariate normality occurs when
residuals are normally distributed. To
test this assumption, look at how the
values of residuals are distributed. It
can also be tested using two main
methods, i.e., a histogram with a
superimposed normal curve or the
Normal Probability Plot method.

Formula and Calculation of Multiple Linear

Regression (MLR)
yi=β0+β1xi1+β2xi2+...+βpxip+ϵwhere, for i=n
observations:yi=dependent variablexi=explan
atory variablesβ0=y-intercept (constant
term)βp=slope coefficients for each explanat
ory variableϵ=the model’s error term (also kn
own as the residuals)yi=β0+β1xi1+β2xi2+...
+βpxip+ϵwhere, for i=n observations:yi
=dependent variablexi=explanatory
variablesβ0=y-intercept (constant term)βp
=slope coefficients for each explanatory varia
bleϵ=the model’s error term (also known as t
he residuals)1
What Multiple Linear Regression (MLR) Can
Tell You
Simple linear regression is a function that
allows an analyst or statistician to make
predictions about one variable based on the
information that is known about another
variable. Linear regression can only be used
when one has two continuous variables—an
independent variable and a dependent
variable. The independent variable is the
parameter that is used to calculate the
dependent variable or outcome. A multiple
regression model extends to several
explanatory variables.
The MLR model is based on the following
assumptions:
 There is a linear relationship between the
dependent variables and the independent
variables
  The independent variables are not too
highly correlated with each other
 yi observations are selected
independently and randomly from the
population
 Residuals should be normally
distributed with a mean of 0
and variance σ2
MLR assumes there is a linear relationship
between the dependent and independent
variables, that the independent variables are
not highly correlated, and that the variance of
the residuals is constant.2
The coefficient of determination (R-squared)
is a statistical metric that is used to measure
how much of the variation in outcome can be
explained by the variation in the independent
variables. R2 always increases as more
predictors are added to the MLR model, even
though the predictors may not be related to
the outcome variable.
R2 by itself can't thus be used to identify
which predictors should be included in a
model and which should be excluded. R 2 can
only be between 0 and 1, where 0 indicates
that the outcome cannot be predicted by any
of the independent variables and 1 indicates
that the outcome can be predicted without
error from the independent variables.
When interpreting the results of multiple
regression, beta coefficients are valid while
holding all other variables constant ("all else
equal"). The output from a multiple
regression can be displayed horizontally as an
equation, or vertically in table form.
Example of How to Use Multiple Linear
Regression (MLR)
As an example, an analyst may want to know
how the movement of the market affects the
price of ExxonMobil (XOM). In this case, the
linear equation will have the value of the S&P
500 index as the independent variable, or
predictor, and the price of XOM as the
dependent variable.
In reality, multiple factors predict the
outcome of an event. The price movement of
ExxonMobil, for example, depends on more
than just the performance of the overall
market. Other predictors such as the price of
oil, interest rates, and the price movement of
oil futures can affect the price of Exon Mobil
(XOM) and the stock prices of other oil
companies. To understand a relationship in
which more than two variables are present,
MLR is used.
MLR is used to determine a mathematical
relationship among several random
variables.3 In other terms, MLR examines
how multiple independent variables are
related to one dependent variable. Once each
of the independent factors has been
determined to predict the dependent
variable, the information on the multiple
variables can be used to create an accurate
prediction on the level of effect they have on
the outcome variable. The model creates a
relationship in the form of a straight line
(linear) that best approximates all the
individual data points.
Referring to the MLR equation above, in our
example:
 yi = dependent variable—the price of
XOM
 xi1 = interest rates
 xi2 = oil price
 xi3 = value of S&P 500 index
 xi4= price of oil futures
 B0 = y-intercept at time zero
 B1 = regression coefficient that measures
a unit change in the dependent variable
when xi1 changes—the change in XOM
price when interest rates change
 B2 = coefficient value that measures a unit
change in the dependent variable when
xi2 changes—the change in XOM price
when oil prices change
The least-squares estimates—B 0, B1, B2…Bp—
are usually computed by statistical software.
As many variables can be included in the
regression model in which each independent
variable is differentiated with a number—1,2,
3, 4...p.
Multiple regression can also be non-linear, in
which case the dependent and independent
variables would not follow a straight line.
The multiple regression model allows an
analyst to predict an outcome based on
information provided on multiple explanatory
variables. Still, the model is not always
perfectly accurate as each data point can
differ slightly from the outcome predicted by
the model. The residual value, E, which is the
difference between the actual outcome and
the predicted outcome, is included in the
model to account for such slight variations.
We ran our XOM price regression model
through a statistics computation software. It
returned this output:
An analyst would interpret this output to
mean if other variables are held constant, the
price of XOM will increase by 7.8% if the price
of oil in the markets increases by 1%. The
model also shows that the price of XOM will
decrease by 1.5% following a 1% rise in
interest rates. R2 indicates that 86.5% of the
variations in the stock price of Exxon Mobil
can be explained by changes in the interest
rate, oil price, oil futures, and S&P 500 index.
The Difference Between Linear and Multiple
Regression
Ordinary linear squares (OLS) regression
compares the response of a dependent
variable given a change in some explanatory
variables. However, a dependent variable is
rarely explained by only one variable. In this
case, an analyst uses multiple regression,
which attempts to explain a dependent
variable using more than one independent
variable.
Multiple regressions can be linear and
nonlinear. MLRs are based on the assumption
that there is a linear relationship between
both the dependent and independent
variables. It also assumes no major
correlation between the independent
variables.
What Makes a Multiple Regression Multiple?
A multiple regression considers the effect of
more than one explanatory variable on some
outcome of interest. It evaluates the relative
effect of these explanatory, or independent,
variables on the dependent variable when
holding all the other variables in the model
constant.
Why Would One Use a Multiple Regression
Over a Simple OLS Regression?
A dependent variable is rarely explained by
only one variable. In such cases, an analyst
uses multiple regression, which attempts to
explain a dependent variable using more than
one independent variable. The model,
however, assumes that there are no major
correlations between the independent
variables.
Difference Between Linear Regression and Multiple
Regression: Linear Regression vs Multiple Regression
Linear (Simple)
Parameter Multiple Regression
Regression
Models the relationship Models the relationship between
Definition between one dependent and one dependent and two or more
one independent variable. independent variables.
Y = C0 + C1X1 + C2X2 + C3X3 + …..
Equation Y = C0 + C1X + e
+ CnXn + e
It is simpler to deal with one More complex due to multiple
Complexity
relationship. relationships.
Suitable when there is one Suitable when multiple factors
Use Cases
clear predictor. affect the outcome.
Same as linear regression, with
Linearity, Independence,
Assumptions the added concern of
Homoscedasticity, Normality
multicollinearity.
Typically visualized with a Requires 3D or multi-dimensional
Visualization 2D scatter plot and a line of space, often represented using
best fit. partial regression plots.
Higher, especially if too many
Risk of Lower, as it deals with only
predictors are used without
Overfitting one predictor.
adequate data.
A primary concern; having
Multicollinearity Not applicable, as there’s correlated predictors can affect the
Concern only one predictor. model’s accuracy and
interpretation.
Basic research, simple Complex research, multifactorial
Applications predictions, understanding a predictions, studying interrelated
singular relationship. systems.
Goodness-of-Fit

What Is Goodness-of-Fit?
The term goodness-of-fit refers to a statistical
test that determines how well sample data
fits a distribution from a population with
a normal distribution. Put simply, it
hypothesizes whether a sample is skewed or
represents the data you would expect to find
in the actual population.
Goodness-of-fit establishes the discrepancy
between the observed values and those
expected of the model in a normal
distribution case. There are multiple methods
to determine goodness-of-fit, including the
chi-square.
KEY TAKEAWAYS
  A goodness-of-fit is a statistical test that
tries to determine whether a set of
observed values match those expected
under the applicable model.
 They can show you whether your sample
data fit an expected set of data from a
population with normal distribution.
 There are multiple types of goodness-of-
fit tests, but the most common is the chi-
square test.
 The chi-square test determines if a
relationship exists between categorical
data.
 The Kolmogorov-Smirnov test determines
whether a sample comes from a specific
distribution of a population.
Understanding Goodness-of-Fit
Goodness-of-fit tests are statistical methods
that make inferences about observed values.
For instance, you can determine whether a
sample group is truly representative of the
entire population. As such, they determine
how actual values are related to the
predicted values in a model. When used in
decision-making, goodness-of-fit tests make it
easier to predict trends and patterns in the
future.
As noted above, there are several types of
goodness-of-fit tests. They include the chi-
square test, which is the most common, as
well as the Kolmogorov-Smirnov test, and the
Shapiro-Wilk test. The tests are normally
conducted using computer software. But
statisticians can do these tests using formulas
that are tailored to the specific type of test.
To conduct the test, you need a certain
variable, along with an assumption of how it
is distributed. You also need a data set with
clear and explicit values, such as:
 The observed values, which are derived
from the actual data set
 The expected values, which are taken
from the assumptions made
 The total number of categories in the set
Goodness-of-fit tests are commonly used to
test for the normality of residuals or to
determine whether two samples are
gathered from identical distributions.
Establishing an Alpha Level
In order to interpret a goodness-of-fit test,
it's important for statisticians to establish an
alpha level, such as the p-value for the chi-
square test. The p-value refers to the
probability of getting results close to
extremes of the observed results. This
assumes that the null hypothesis is correct. A
null hypothesis asserts there is no
relationship that exists between variables,
and the alternative hypothesis assumes that a
relationship exists.
Instead, the frequency of the observed values
is measured and subsequently used with the
expected values and the degrees of
freedom to calculate chi-square. If the result
is lower than alpha, the null hypothesis is
invalid, indicating a relationship exists
between the variables.
Types of Goodness-of-Fit Tests
Chi-Square Test
χ2=∑i=1k(Oi−Ei)2/Eiχ2=i=1∑k(Oi−Ei)2/Ei
The chi-square test, which is also known as
the chi-square test for independence, is an
inferential statistics method that tests the
validity of a claim made about a population
based on a random sample.
Used exclusively for data that is separated
into classes (bins), it requires a sufficient
sample size to produce accurate results. But it
doesn't indicate the type or intensity of the
relationship. For instance, it does not
conclude whether the relationship is positive
or negative.
To calculate a chi-square goodness-of-fit, set
the desired alpha level of significance. So if
your confidence level is 95% (or 0.95), then
the alpha is 0.05. Next, identify the
categorical variables to test, then define
hypothesis statements about the
relationships between them.1
Variables must be mutually exclusive in order
to qualify for the chi-square test for
independence. And the chi goodness-of-fit
test should not be used for data that is
continuous.
Kolmogorov-Smirnov (K-S) Test
D=max⁡1≤i≤N(F(Yi)
−i−1N,iN−F(Yi))D=1≤i≤Nmax(F(Yi)−Ni−1,Ni
−F(Yi))
Named after Russian mathematicians
Andrey Kolmogorov and Nikolai Smirnov, the
Kolmogorov-Smirnov (K-S) test is a statistical
method that determines whether a sample is
from a specific distribution within a
population.
This test, which is recommended for
large samples (e.g., over 2000), is non-
parametric. That means it does not rely on
any distribution to be valid. The goal is to
prove the null hypothesis, which is the
sample of the normal distribution.
Like chi-square, it uses a null and alternative
hypothesis and an alpha level of significance.
Null indicates that the data follow a specific
distribution within the population, and
alternative indicates that the data did not
follow a specific distribution within the
population. The alpha is used to determine
the critical value used in the test. But unlike
the chi-square test, the Kolmogorov-Smirnov
test applies to continuous distributions.
The calculated test statistic is often denoted
as D. It determines whether the null
hypothesis is accepted or rejected. If D is
greater than the critical value at alpha, the
null hypothesis is rejected. If D is less than
the critical value, the null hypothesis is
accepted.2
The Anderson-Darling (A-D) Test
S=∑i=1N(2i−1)N[ln⁡F(Yi)
+ln⁡(1−F(YN+1−i))]S=∑i=1NN(2i−1)[lnF(Yi)
+ln(1−F(YN+1−i))]
The Anderson-Darling (A-D) test is a variation
on the K-S test, but gives more weight to the
tails of the distribution. The K-S test is more
sensitive to differences that may occur closer
to the center of the distribution, while the A-
D test is more sensitive to variations
observed in the tails.3 Because tail risk and
the idea of "fatty tails" is prevalent in
financial markets, the A-D test can give more
power in financial analyses.
Like the K-S test, the A-D test produces a
statistic, denoted as A2, which can be
compared against the null hypothesis.
Shapiro-Wilk (S-W) Test
W=(∑i=1nai(x(i))2∑i=1n(xi−xˉ)2, W=∑i=1n(xi
−xˉ)2(∑i=1nai(x(i))2,
The Shapiro-Wilk (S-W) test determines if a
sample follows a normal distribution. The test
only checks for normality when using a
sample with one variable of continuous data
and is recommended for small sample sizes
up to 2000.
The Shapiro-Wilk test uses a probability plot
called the QQ Plot, which displays two sets of
quantiles on the y-axis that are arranged from
smallest to largest. If each quantile came
from the same distribution, the series of plots
are linear.
The QQ Plot is used to estimate the variance.
Using QQ Plot variance along with the
estimated variance of the population, one
can determine if the sample belongs to a
normal distribution. If the quotient of both
variances equals or is close to 1, the null
hypothesis can be accepted. If considerably
lower than 1, it can be rejected.
Just like the tests mentioned above, this one
uses alpha and forms two hypotheses: null
and alternative. The null hypothesis states
that the sample comes from the normal
distribution, whereas the alternative
hypothesis states that the sample does not
come from the normal distribution.4
Other Goodness-of-Fit Tests
Aside from the more common types of tests
mentioned above, there are numerous other
goodness-of-fit tests an analyst can use:
 The Bayesian information criterion
(BIC) is a statistical measure used for
model selection among a finite set of
models. The BIC is a goodness-of-fit test
that balances the complexity of a model
with its goodness-of-fit to the data.
 The Cramer-von Mises criterion (CVM) is
a goodness-of-fit test that is used to
assess how well a set of observed data fits
a hypothesized probability distribution.
Often used in economics, engineering, or
finance, it is based on the cumulative
distribution function of the observed data
and the hypothesized distribution.
 The Akaike information criterion (AIC) is
a measure of the relative quality of a
statistical model for a given set of data,
and it provides a trade-off between the
goodness-of-fit of the model and its
complexity. It's based on information
theory and measures the amount of
information lost by a model when it is
used to approximate the true underlying
distribution of the data.
 The Hosmer-Lemeshow test compares
the expected frequencies of a binary
outcome with the observed frequencies
of that outcome in different groups or
intervals. The groups are typically formed
by dividing the predicted probabilities of
the outcome into ten groups or bins.
 Kuiper's test is similar to the Kolmogorov-
Smirnov test, but it is more sensitive to
differences in the tails of the distribution.
 Moran's I test or Moran's Index is a
statistical test used to assess spatial
autocorrelation in data. Spatial
 autocorrelation is a measure of the
degree to which observations of a
variable are similar or dissimilar in space.
A very general rule of thumb is to require that
every group within a goodness-of-fit test
have at least five data points. This ensures
that sufficient information is fed into the test
to determine the distribution.
Importance of Goodness-of-Fit Tests
Goodness-of-fit tests are important in
statistics for many reasons. First, they provide
a way to assess how well a statistical model
fits a set of observed data. The main
importance of running a goodness-of-fit test
is to determine whether the observed data
are consistent with the assumed statistical
model. By extension, a goodness-of-fit test
may be useful in choosing between different
models which may better fit the data.
Goodness-of-fit tests can also help to identify
outliers or market abnormalities that may be
affecting the fit of the model. Outliers can
have a large impact on the model fit and may
need to be removed or dealt with separately.
Sometimes, outliers are not easily identifiable
until they have been integrated into an
analytical model.
Goodness-of-fit tests can also provide
information about the variability of the data
and the estimated parameters of the model.
This information can be useful for making
predictions and understanding the behavior
of the system being modeled. Based on the
data being fed into the model, it may be
necessary to refine the model specific to the
dataset being tested, the residuals being
calculated, and the p-value for potentially
extreme data.
Goodness-of-Fit Test vs. Independence Test
Goodness-of-fit test and independence test
are both statistical tests used to assess the
relationship between variables; therefore, it
may be easy to confuse the two. However,
each are designed to answer different
questions.
A goodness-of-fit test is used to evaluate how
well a set of observed data fits a particular
probability distribution. On the other hand,
an independence test is used to assess the
relationship between two variables. It is used
to test whether there is any association
between two variables. The primary purpose
of an independence test is to see whether a
change in one variable is related to a change
in another variable.
An independence test is typically used when
the research question is focused on
understanding the relationship between two
variables and whether they are related or
independent. In many cases, an
independence test is pointed towards two
specific variables (i.e. does smoking cause
lung cancer?). On the other hand, a
goodness-of-fit test is used on an entire set of
observed data to evaluate the
appropriateness of a specific model.
Goodness-of-Fit Example
Here's a hypothetical example to show how
the goodness-of-fit test works.
Suppose a small community gym operates
under the assumption that the highest
attendance is on Mondays, Tuesdays, and
Saturdays, average attendance on
Wednesdays, and Thursdays, and lowest
attendance on Fridays and Sundays. Based on
these assumptions, the gym employs a
certain number of staff members each day to
check in members, clean facilities, offer
training services, and teach classes.
But the gym isn't performing well financially
and the owner wants to know if these
attendance assumptions and staffing levels
are correct. The owner decides to count the
number of gym attendees each day for six
weeks. They can then compare the gym's
assumed attendance with its observed
attendance using a chi-square goodness-of-fit
test for example.
Now that they have the new data, they can
determine how to best manage the gym and
improve profitability.
What Does Goodness-of-Fit Mean?
Goodness-of-Fit is a statistical hypothesis test
used to see how closely observed data
mirrors expected data. Goodness-of-Fit tests
can help determine if a sample follows a
normal distribution, if categorical variables
are related, or if random samples are from
the same distribution.
Why Is Goodness-of-Fit Important?
Goodness-of-Fit tests help determine if
observed data aligns with what is expected.
Decisions can be made based on the outcome
of the hypothesis test conducted. For
example, a retailer wants to know what
product offering appeals to young people.
The retailer surveys a random sample of old
and young people to identify which product is
preferred. Using chi-square, they identify
that, with 95% confidence, a relationship
exists between product A and young people.
Based on these results, it could be
determined that this sample represents the
population of young adults. Retail marketers
can use this to reform their campaigns.
What Is Goodness-of-Fit in the Chi-Square
Test?
The chi-square test whether relationships
exist between categorical variables and
whether the sample represents the whole. It
estimates how closely the observed data
mirrors the expected data, or how well they
fit.
How Do You Do the Goodness-of-Fit Test?
The Goodness-of-FIt test consists of different
testing methods. The goal of the test will help
determine which method to use. For
example, if the goal is to test normality on a
relatively small sample, the Shapiro-Wilk test
may be suitable. If wanting to determine
whether a sample came from a specific
distribution within a population, the
Kolmogorov-Smirnov test will be used. Each
test uses its own unique formula. However,
they have commonalities, such as a null
hypothesis and level of significance.
The Bottom Line
Goodness-of-fit tests determine how well
sample data fit what is expected of a
population. From the sample data, an
observed value is gathered and compared to
the calculated expected value using a
discrepancy measure. There are different
goodness-of-fit hypothesis tests available
depending on what outcome you're seeking.
Choosing the right goodness-of-fit test largely
depends on what you want to know about a
sample and how large the sample is. For
example, if wanting to know if observed
values for categorical data match the
expected values for categorical data, use chi-
square. If wanting to know if a small sample
follows a normal distribution, the Shapiro-
Wilk test might be advantageous. There are
many tests available to determine goodness-
of-fit.

R-Squared vs. Adjusted R-Squared: What's the

Difference?
R-Squared vs. Adjusted R-Squared: An
Overview
R-squared and adjusted R-squared enable
investors to measure the performance of a
mutual fund against that of a benchmark.
Investors may also use them to calculate the
performance of their portfolio against a given
benchmark.
In the world of investing, R-squared is
expressed as a percentage between 0 and
100, with 100 signaling perfect
correlation and zero no correlation at all. The
figure does not indicate how well a particular
group of securities is performing. It only
measures how closely the returns align with
those of the measured benchmark. It is also
backwards-looking—it is not a predictor of
future results.
Adjusted R-squared can provide a more
precise view of that correlation by also taking
into account how many independent
variables are added to a particular model
against which the stock index is measured.
This is done because such additions of
independent variables usually increase the
reliability of that model—meaning, for
investors, the correlation with the index.
KEY TAKEAWAYS
 R-squared and the adjusted R-squared
both help investors measure the
correlation between a mutual fund or
portfolio with a stock index.
  Adjusted R-squared, a modified version of
R-squared, adds precision and reliability
by considering the impact of additional
independent variables that tend to skew
the results of R-squared measurements.
 The predicted R-squared, unlike the
adjusted R-squared, is used to indicate
how well a regression model predicts
responses for new observations.
 One misconception about regression
analysis is that a low R-squared value is
always a bad thing.
R-Squared
R-squared (R2) is a statistical measure that
represents the proportion of the variance for
a dependent variable that's explained by an
independent variable or variables in
a regression model. R-squared explains to
what extent the variance of one variable
explains the variance of the second
variable. So, if the R2 of a model is 0.50, then
approximately half of the observed variation
can be explained by the model's inputs.
An R-squared result of 70 to 100 indicates
that a given portfolio closely tracks the stock
index in question, while a score between 0
and 40 indicates a very low correlation with
the index.1 Higher R-squared values also
indicate the reliability of beta readings.
Beta measures the volatility of a security or a
portfolio.
While R-squared can return a figure that
indicates a level of correlation with an index,
it has certain limitations when it comes to
measuring the impact of independent
variables on the correlation. This is where
adjusted R-squared is useful in measuring
correlation.
Adjusted R-Squared
Adjusted R-squared is a modified version of
R-squared that has been adjusted for the
number of predictors in the model. The
adjusted R-squared increases when the new
term improves the model more than would
be expected by chance. It decreases when a
predictor improves the model by less than
expected. Typically, the adjusted R-squared is
positive, not negative. It is always lower than
the R-squared.
Adding more independent variables or
predictors to a regression model tends to
increase the R-squared value, which tempts
makers of the model to add even more
variables. This is called overfitting and can
return an unwarranted high R-squared value.
Adjusted R-squared is used to determine how
reliable the correlation is and how much it is
determined by the addition of independent
variables.2
In a portfolio model that has more
independent variables, adjusted R-squared
will help determine how much of the
correlation with the index is due to the
addition of those variables. The adjusted R-
squared compensates for the addition of
variables and only increases if the new
predictor enhances the model above what
would be obtained by probability. Conversely,
it will decrease when a predictor improves
the model less than what is predicted by
chance.
Key Differences
The most obvious difference between
adjusted R-squared and R-squared is simply
that adjusted R-squared considers and tests
different independent variables against the
stock index and R-squared does not. Because
of this, many investment professionals prefer
using adjusted R-squared because it has the
potential to be more accurate. Furthermore,
investors can gain additional information
about what is affecting a stock by testing
various independent variables using the
adjusted R-squared model.
R-squared, on the other hand, does have its
limitations. One of the most essential limits
to using this model is that R-squared cannot
be used to determine whether or not the
coefficient estimates and predictions are
biased. Furthermore, in multiple linear
regression, the R-squared cannot tell us
which regression variable is more important
than the other.
Adjusted R-Squared vs. Predicted R-Squared
The predicted R-squared, unlike the adjusted
R-squared, is used to indicate how well a
regression model predicts responses for new
observations. So where the adjusted R-
squared can provide an accurate model that
fits the current data, the predicted R-squared
determines how likely it is that this model will
be accurate for future data.
R-Squared vs. Adjusted R-Squared Examples
When you are analyzing a situation in which
there is a guarantee of little to no bias, using
R-squared to calculate the relationship
between two variables is perfectly useful.
However, when investigating the relationship
between say, the performance of a single
stock and the rest of the S&P500, it is
important to use adjusted R-squared to
determine any inconsistencies in the
correlation.
If an investor is looking for an index fund that
closely tracks the S&P500, they will want to
test different independent variables against
the stock index such as the industry, the
assets under management, how long the
stock has been available on the market, and
so on to ensure they have the most accurate
figure of the correlation.
Special Considerations
R-Squared and Goodness-of-Fit
The basic idea of regression analysis is that if
the deviations between the observed values
and the predicted values of the linear model
are small, the model has well-fit
data. Goodness-of-fit is a mathematical
model that helps to explain and account for
the difference between this observed data
and the predicted data. In other words,
goodness-of-fit is a statistical hypothesis test
to see how well sample data fit a distribution
from a population with a normal distribution.
Low R-Squared vs. High R-Squared Value
One misconception about regression analysis
is that a low R-squared value is always a bad
thing. This is not so. For example, some data
sets or fields of study have an inherently
greater amount of unexplained variation. In
this case, R-squared values are naturally
going to be lower. Investigators can make
useful conclusions about the data even with a
low R-squared value.
In a different case, such as in investing, a high
R-squared value—typically between 85% and
100%—indicates the stock or fund's
performance moves relatively in line with the
index. This is very useful information to
investors, thus a higher R-squared value is
necessary for a successful project.
What Is the Difference Between R-Squared
and Adjusted R-Squared?
The most vital difference between adjusted
R-squared and R-squared is simply that
adjusted R-squared considers and tests
different independent variables against the
model and R-squared does not.
Which Is Better, R-Squared or Adjusted R-
Squared?
Many investors prefer adjusted R-squared
because adjusted R-squared can provide a
more precise view of the correlation by also
taking into account how many independent
variables are added to a particular model
against which the stock index is measured.
Should I Use Adjusted R-Squared or R-
Squared?
Using adjusted R-squared over R-squared
may be favored because of its ability to make
a more accurate view of the correlation
between one variable and another. Adjusted
R-squared does this by taking into account
how many independent variables are added
to a particular model against which the stock
index is measured.
What Is an Acceptable R-Squared Value?
Many people believe there is a magic number
when determining an R-squared value that
marks the sign of a valid study; however, this
is not so. Because some data sets are
inherently set up to have more unexpected
variations than others, obtaining a high R-
squared value is not always realistic.
However, in some instances an R-squared
value between 70-90% is ideal.3
The Bottom Line
R-squared and adjusted R-squared enable
investors to measure the performance of a
mutual fund against that of a benchmark.
Many investors have found success using
adjusted R-squared over R-squared because
of its ability to make a more accurate view of
the correlation between one variable and
another.
F Test
F test is a statistical test that is used in
hypothesis testing to check whether the
variances of two populations or two samples
are equal or not. In an f test, the data follows
an f distribution. This test uses the f statistic
to compare two variances by dividing them.
An f test can either be one-tailed or two-
tailed depending upon the parameters of the
problem.
The f value obtained after conducting an f test
is used to perform the one-way ANOVA
(analysis of variance) test. In this article, we
will learn more about an f test, the f statistic,
its critical value, formula and how to conduct
an f test for hypothesis testing.
What is F Test in Statistics?
F test is statistics is a test that is performed
on an f distribution. A two-tailed f test is used
to check whether the variances of the two
given samples (or populations) are equal or
not. However, if an f test checks whether one
population variance is either greater than or
lesser than the other, it becomes a one-tailed
hypothesis f test.
F Test Definition
F test can be defined as a test that uses the f
test statistic to check whether the variances
of two samples (or populations) are equal to
the same value. To conduct an f test, the
population should follow an f distribution and
the samples must be independent events. On
conducting the hypothesis test, if the results
of the f test are statistically significant then
the null hypothesis can be rejected otherwise
it cannot be rejected.
F Test Formula
The f test is used to check the equality of
variances using hypothesis testing. The f test
formula for different hypothesis tests is given
as follows:
Left Tailed Test:
Null Hypothesis: H0H0 : σ21=σ22σ12=σ22
Alternate
Hypothesis: H1H1 : σ21<σ22σ12<σ22
Decision Criteria: If the f statistic < f critical
value then reject the null hypothesis
Right Tailed test:
Null Hypothesis: H0H0 : σ21=σ22σ12=σ22
Alternate
Hypothesis: H1H1 : σ21>σ22σ12>σ22
Decision Criteria: If the f test statistic > f test
critical value then reject the null hypothesis
Two Tailed test:
Null Hypothesis: H0H0 : σ21=σ22σ12=σ22
Alternate
Hypothesis: H1H1 : σ21≠σ22σ12≠σ22
Decision Criteria: If the f test statistic > f test
critical value then the null hypothesis is
rejected
F Statistic
The f test statistic or simply the f statistic is a
value that is compared with the critical value
to check if the null hypothesis should be
rejected or not. The f test statistic formula is
given below:
F statistic for large samples: F
= σ21σ22σ12σ22, where σ21σ12 is the
variance of the first population and σ22σ22 is
the variance of the second population.
F statistic for small samples: F
= s21s22s12s22, where s21s12 is the variance
of the first sample and s22s22 is the variance
of the second sample.
The selection criteria for
the σ21σ12 and σ22σ22 for an f statistic is
given below:
 For a right-tailed and a two-tailed f test,

the variance with the greater value will

be in the numerator. Thus, the sample
 corresponding to σ21σ12 will become
the first sample. The smaller value
variance will be the denominator and
belongs to the second sample.
 For a left-tailed test, the smallest

variance becomes the numerator

(sample 1) and the highest variance
goes in the denominator (sample 2).
F Test Critical Value
A critical value is a point that a test statistic is
compared to in order to decide whether to
reject or not to reject the null hypothesis.
Graphically, the critical value divides a
distribution into the acceptance and rejection
regions. If the test statistic falls in the
rejection region then the null hypothesis can
be rejected otherwise it cannot be rejected.
The steps to find the f test critical value at a
specific alpha level (or significance level), αα,
are as follows:
 Find the degrees of freedom of the first

sample. This is done by subtracting 1

 from the first sample size. Thus, x
= n1−1n1−1.
 Determine the degrees of freedom of

the second sample by subtracting 1

from the sample size. This given y
= n2−1n2−1.
 If it is a right-tailed test then αα is the

significance level. For a left-tailed test 1

- αα is the alpha level. However, if it is a
two-tailed test then the significance
level is given by αα / 2.
 The F table is used to find the critical

value at the required alpha level.

 The intersection of the x column and the

y row in the f table will give the f test

critical value.
ANOVA F Test
The one-way ANOVA is an example of an f
test. ANOVA stands for analysis of variance. It
is used to check the variability of group
means and the associated variability in
observations within that group. The F test
statistic is used to conduct the ANOVA test.
The hypothesis is given as follows:
H0H0: The means of all groups are equal.
H1H1: The means of all groups are not equal.
Test Statistic: F = explained variance /
unexplained variance
Decision rule: If F > F critical value then reject
the null hypothesis.
To determine the critical value of an ANOVA f
test the degrees of freedom are given
by df1df1 = K - 1 and df1df1 = N - K, where N
is the overall sample size and K is the number
of groups.
F Test vs T-Test
F test and t-test are different types of
statistical tests used for hypothesis testing
depending on the distribution followed by the
population data. The table given below
outlines the differences between the F test
and the t-test.
F Test T-Test

An F test is a test The T-test is used

statistic used to when the sample
check the size is small (n < 30)
equality of and the population
variances of two standard deviation
populations is not known.

The data follows a

The data follows
Student t-
an F distribution
distribution

The t-test statistic

for 1 sample is
given by t
= ¯¯¯x−μs√nx¯−μsn,
The F test
where ¯¯¯xx¯ is the
statistic is given
sample mean, μμ is
as F
the population
= σ21σ22σ12σ22
mean, s is the
sample standard
deviation and n is
the sample size.
 F Test T-Test

The f test is used It is used for

for variances. testing means.
Important Notes on F Test
 The f test is a statistical test that is

conducted on an F distribution in order

to check the equality of variances of two
populations.
 The f test formula for the test statistic is

given by F = σ21σ22σ12σ22.
 The f critical value is a cut-off value that

is used to check whether the null

hypothesis can be rejected or not.
 A one-way ANOVA is an example of an f

test that is used to check the variability

of group means and the associated
variability in the group observations.