Multiple Correlation Coefficient PDF

Multiple Correlation Coefficient
Herv Abdi1
1 Overview
The multiple correlation coefficient generalizes the standard coefficient of correlation. It is used in multiple regression analysis to
assess the quality of the prediction of the dependent variable. It
corresponds to the squared correlation between the predicted and
the actual values of the dependent variable. It can also be interpreted as the proportion of the variance of the dependent variable
explained by the independent variables. When the independent
variables (used for predicting the dependent variable) are pairwise
orthogonal, the multiple correlation coefficient is equal to the sum
of the squared coefficients of correlation between each independent variable and the dependent variable. This relation does not
hold when the independent variables are not orthogonal. The significance of a multiple coefficient of correlation can be assessed
with an F ratio. The magnitude of the multiple coefficient of correlation tends to overestimate the magnitude of the population correlation, but it is possible to correct for this overestimation. Strictly
speaking we should refer to this coefficient as the squared multiple correlation coefficient, but current usage seems to ignore the
1
In: Neil Salkind (Ed.) (2007). Encyclopedia of Measurement and Statistics.

Thousand Oaks (CA): Sage.
Address correspondence to: Herv Abdi
Program in Cognition and Neurosciences, MS: Gr.4.1,
The University of Texas at Dallas,
Richardson, TX 750830688, USA
E-mail: herve@utdallas.edu http://www.utd.edu/herve
Herv Abdi: Multiple Correlation Coefficient
adjective squared, probably because mostly its squared value is

considered.
2 Multiple Regression framework

In linear multiple regression analysis, the goal is to predict, knowing the measurements collected on N subjects, a dependent variable Y from a set of J independent variables denoted
{X 1 , . . . , X j , . . . , X J } .
(1)
We denote by X the N (J +1) augmented matrix collecting the data

for the independent variables (this matrix is called augmented because the first column is composed only of ones), and by y the N 1
vector of observations for the dependent variable. These two matrices have the following structure.
1 x 1,1
..
..
.
.
X = 1 x n,1
..
..
.
.
1 x N ,1
..
.
..
.
x 1, j
.. . .
.
.
x n, j
.. . .
.
.
xN , j
x 1,J
..
.
x n,J
..
.
x N ,J
y1
..
.
y
and y =
n
..
.
yN
(2)
The predicted values of the dependent variable Yb are collected

in a vector denoted b
y and are obtained as:
b
y = Xb
with
T
XT y .
b= X X
(3)
The regression sum of squares is obtained as

SS regression = bT XT y
1 T 2
(1 y)
N
(4)
(with 1T being a row vector of 1s conformable with y).

The total sum of squares is obtained as
SS total = yT y
2
1 T 2
(1 y) .
N
(5)
The residual (or error) sum of squares is obtained as

SS error = yT y bT XT y .
(6)
The quality of the prediction is evaluated by computing the

multiple coefficient of correlation denoted R Y2 .1,...,J . This coefficient is equal to the squared coefficient of correlation between the
dependent variable (Y ) and the predicted dependent variable (Yb ).
An alternative way of computing the multiple coefficient of correlation is to divide the regression sum of squares by the total sum
of squares. This shows that R Y2 .1,...,J can also be interpreted as the
proportion of variance of the dependent variable explained by the
independent variables. With this interpretation, the multiple coefficient of correlation is computed as
R Y2 .1,...,J =
SS regression
SS regression + SS error
SS regression
SS total
(7)
2.1 Significance test

In order to assess the significance of a given R Y2 .1,...,J , we can compute an F ratio as
F=
R Y2 .1,...,J
1 R Y2 .1,...,J
N J 1
.
J
(8)
Under the usual assumptions of normality of the error and of independence of the error and the scores, this F ratio is distributed
under the null hypothesis as a Fisher distribution with 1 = J and
2 = N J 1 degrees of freedom.
2.2 Estimating the population correlation:

shrunken and adjusted R
Just like its bivariate counterpart r , the multiple coefficient of correlation is a descriptive statistic which always overestimates the
population correlation. This problem is similar to the problem
of the estimation of the variance of a population from a sample.
3
Table 1: A set of data. The dependent variable Y is to be predicted

from two orthogonal predictors X 1 and X 2 (data from Abdi et al.,
2002). These data are the results of an hypothetical experiment on

retroactive interference and learning. Y is the number of sentences
remembered from a set of sentences learned, X 1 is the number of
learning trials, and X 2 is the number of interpolated lists learned.
Number of
interpolated lists (T )
Number of
learning trials (X )
2
4
8
35
39
40
52
61
73
21
31
34
42
58
66
6
8
18
26
46
52
In order to obtain a better estimate of the population, the value

R Y2 .1,...,J needs to be corrected. The corrected value of R Y2 .1,...,J goes
under different names: corrected R, shrunken R, or adjusted R (there are some subtle differences between these different appellations, but we will ignore them here) and we denote it by ReY2 .1,...,J .
There are several correction formulas available, the one most often used estimates the value of the population correlation as
N 1
2
2
.
(9)
ReY .1,...,J = 1 1 R Y .1,...,J
N J 1
3 Example 1:
Multiple correlation coefficient
with orthogonal predictors
When the independent variables are pairwise orthogonal, the importance of each of them in the regression is assessed by computing the squared coefficient of correlation between each of the independent variables and the dependent variable. The sum of these
squared coefficients of correlation is equal to the multiple coefficient of correlation. We illustrate this case with the data from Table 1. In this example, the dependent variable (Y ) is the number or
sentences recalled by participants who learned a list of unrelated
sentences. The first independent variable or first predictor, X 1 is
the number of trials used to learn the list. It takes the values 2,
4, and 8. It is expected that recall will increase as a function of the
number of trials. The second independent variable, X 2 is the number of additional interpolated lists that the participants are asked
to learned. It takes the values 2, 4, and 8. As a consequence of
retroactive inhibition, it is expected that recall will decrease as a
function of the number of interpolated lists learned.
Using Equation 3, we found that Yb can be obtained from X 1
and X 2 as
Yb = 30 + 6 X 1 4 X 2 .
(10)
Using these data and Equations 4 and 5, we find that
SS regression = 5824,
SS total = 6214,
and SS error = 390 .
(11)
This gives the following value for the multiple coefficient of correlation:
SS regression 5824
=
= .9372 .
(12)
R Y2 .1,...,J =
SS total
6214
In order to decide if this value of R Y2 .1,...,J is large enough to be considered significant, we compute an F ratio equal to
F=
R Y2 .1,...,J
1 R Y2 .1,...,J
N J 1
.9372
15
=
= 111.93 .
J
1 .9372 2
5
(13)
Such a value of F is significant at all the usual alpha levels, and

therefore we can reject the null hypothesis.
Because X 1 and X 2 are orthogonal to each other (i.e., their correlation is equal to 0), the multiple coefficient of correlation is equal
to the sum of the squared coefficients of correlation between the
independent variables and the dependent variable:
R Y2 .1,...,J = .9372 = r Y2 ,1 + r Y2 ,2 = .6488 + .2884 .
(14)
A better estimate of the population value of the multiple coefficient of correlation can obtained as
N 1
17
2
2
e
= 1(1.9372) = .9289 .
R Y .1,...,J = 1 1 R Y .1,...,J
N J 1
15
(15)
4 Example 2:
Multiple correlation coefficient
with non-orthogonal predictors
When the independent variables are correlated, the multiple coefficient of correlation is not equal to the sum of the squared correlation coefficients between the dependent variable and the independent variables. In fact, such a strategy would overestimate the
contribution of each variable because the variance that they share
would be counted several times.
For example, consider the data given in Table 2 where the dependent variable is to be predicted from the independent variables
X 1 and X 2 . The prediction of the dependent variable (using Equation 3) is found to be equal to
Yb = 1.67 + X 1 + 9.50X 2 ;
(16)
this gives a multiple coefficient of correlation of R Y2 .1,...,J = .9866.

The coefficient of correlation between X 1 and X 2 is equal to r X 1 .X 2 =
.7500, between X 1 and Y is equal to r Y .1 = .8028, and between X 2
and Y is equal to r Y .2 = .9890. It can easily be checked that the
6
Table 2: A set of data. The dependent variable Y is to be predicted

from two correlated (i.e., non-orthogonal) predictors: X 1 and X 2
(data from Abdi et al., 2002). Y is the number of digits a child can
remember for a short time (the memory span"), X 1 is the age of
the child, and X 2 is the speech rate of the child (how many words
the child can pronounce in a given time). Six children were tested.
Y (Memory span)
X 1 (age)
X 2 (Speech rate)
14
4
1
23
4
2
30
7
2
50
7
4
39
10
3
67
10
6
multiple coefficient of correlation is not equal to the sum of the

squared coefficients of correlation between the independent variables and the dependent variables:
R Y2 .1,...,J = .9866 6= r Y2 .1 + r Y2 .2 = .665 + .9780 = 1.6225 .
(17)
Using the data from Table 2 along with Equations 4 and 5, we

find that
SS regression = 1822.00,
SS total = 1846.83,
and SS error = 24.83 .

(18)
This gives the following value for the multiple coefficient of correlation:
SS regression 1822.00
=
= .9866 .
(19)
R Y2 .1,...,J =
SS total
1846.83
In order to decide if this value of R Y2 .1,...,J is large enough to be considered significant, we compute an F ratio equal to
F=
R Y2 .1,...,J
1 R Y2 .1,...,J
N J 1
.9866
3
=
= 110.50 .
J
1 .9866 2
(20)
Such a value of F is significant at all the usual alpha levels, and

therefore we can reject the null hypothesis.
A better estimate of the population value of the multiple coefficient of correlation can obtained as
N 1
5
2
2
e
R Y .1,...,J = 1 1 R Y .1,...,J
= 1 (1 .9866) = .9776 .
N J 1
2
(21)
7
References
[1] Abdi, H., Dowling, W.J., Valentin, D., Edelman, B., & Posamentier M. (2002). Experimental Design and research methods. Unpublished manuscript. Richardson: The University of
Texas at Dallas, Program in Cognition.
[2] Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences (2nd edition). Hillsdale (NJ): Erlbaum.
[3] Darlington, R.B. (1990). Regression and linear models. NewYork:
McGraw-Hill.
[4] Pedhazur, E.J. (1997). Multiple regression in behavioral research. (3rd edition) New York: Holt, Rinehart and Winston,
Inc.

Multiple Correlation Coefficient PDF

Uploaded by

Copyright:

Available Formats

Multiple Correlation Coefficient PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Multiple Correlation Coefficient PDF

Uploaded by

Copyright:

Available Formats

Multiple Correlation Coefficient

In: Neil Salkind (Ed.) (2007). Encyclopedia of Measurement and Statistics.

Herv Abdi: Multiple Correlation Coefficient

adjective squared, probably because mostly its squared value is

2 Multiple Regression framework

We denote by X the N (J +1) augmented matrix collecting the data

The predicted values of the dependent variable Yb are collected

The regression sum of squares is obtained as

(with 1T being a row vector of 1s conformable with y).

Herv Abdi: Multiple Correlation Coefficient

The residual (or error) sum of squares is obtained as

The quality of the prediction is evaluated by computing the

2.1 Significance test

2.2 Estimating the population correlation:

Herv Abdi: Multiple Correlation Coefficient

Table 1: A set of data. The dependent variable Y is to be predicted

2002). These data are the results of an hypothetical experiment on

In order to obtain a better estimate of the population, the value

Herv Abdi: Multiple Correlation Coefficient

and SS error = 390 .

Herv Abdi: Multiple Correlation Coefficient

Such a value of F is significant at all the usual alpha levels, and

this gives a multiple coefficient of correlation of R Y2 .1,...,J = .9866.

Herv Abdi: Multiple Correlation Coefficient

Table 2: A set of data. The dependent variable Y is to be predicted

multiple coefficient of correlation is not equal to the sum of the

Using the data from Table 2 along with Equations 4 and 5, we

and SS error = 24.83 .

Such a value of F is significant at all the usual alpha levels, and

Herv Abdi: Multiple Correlation Coefficient

You might also like