Anova

Ph. D.
in Materials Engineering Statistical methods
A. F-test for the comparison of means (ANOVA)

Typical problem
A material is subject to r different kinds of treatment
treatments =⇒ 1, 2, ... , r
and some measurements of a certain property x are performed
on each treated material sample
We assume for simplicity that the number of measurements is
the same — c — for all the r treatments
c measured values of x
i = 1, . . . , r
on the material subject =⇒ xij
j = 1, . . . , c
to the treatment i
Question
Are the mean values of the quantity x significantly different for
the various treatments?
Or are they essentially equal, so that different treatments do
not modify the property x in a significant way?
Stefano Siboni 1
Ph. D. in Materials Engineering Statistical methods
The model
Set of independent random variables with two indices
xij i = 1, . . . , r j = 1, . . . , c
i specifies the group (treatment)
j identifies an element of the i-th group

(value measured on a sample subject to the treatment i)
For all i = 1, . . . , r and j = 1, . . . , c the random variable xij is

normal of mean µi and variance σ 2
Remark
All the groups are characterized by the same variance σ 2 ,
but they could have different means µi
Hypothesis to be tested
H0 : µi = µ ∀ i = 1, . . . , r (equal means)
H1 : µh 6= µk for at least one h and one k in 1, . . . , r

(at least two unequal means)
Method
1-factor analysis of variance

(ANalysis Of VAriance → ANOVA)
Stefano Siboni 2
ANOVA
For a set of data arranged in r groups of c data each, the
1-factor analysis of variance consists in the interpretation (or
“explanation”) of data spread by distinguishing:
• a contribution due to the choice of the group (row)
• an intrinsic contribution of the single groups
The data are classified according to an appropriate experimen-

tal condition, or “factor”
In the present example, the only factor which classifies the

various data is the kind of treatment
The factor admits r different “levels”, each corresponding to

a group of data measured in the same conditions
In this example, each level corresponds to a particular kind

of treatment and is accordingly associate to a group of data
in the same row
The ANOVA we apply is the 1-factor ANOVA

(also known as one-way ANOVA)
Stefano Siboni 3
It may happen that the data are classified by two or more

experimental conditions varied in a controlled way
For instance, the treatment could consist in the action of a

chemical reagent, out of a certain set, at various temperatures
kind of
datum ⇐⇒ + temperature
chemical reagent
In this case each datum will be identified by a pair of indices

and we will use a 2-factor ANOVA without replications
(or two-way ANOVA without replications)
The 1st factor corresponds to the kind of chemical reagent

of the given set, the associate levels are the different reagents
The 2nd factor is the temperature,

the corresponding levels are the various temperatures used
Another case is when for any combination reagent-temperature

many replications of the measurements are available, form-
ing a homogeneous group of data
kind of
group ⇐⇒ + temperature
chemical reagent
In such a case each datum will be labeled by a triplet of

indices and a 2-factor ANOVA with replications will be
applied (or two-way ANOVA with replications)
Stefano Siboni 4
Further, the treatment could be specified by the kind of reagent,

the temperature and the time duration
kind of
+ temperature
datum ⇐⇒ chemical reagent
+ time
Each experimental datum will correspond to a combination of

the three conditions and be identified by a triplet of indices
We will apply a 3-factor ANOVA without replications
(or three-way ANOVA without replications)
The 1st factor corresponds to the kind of chemical reagent

of the given set, the associate levels are the different reagents
The 2nd factor consists in the temperature,

the corresponding levels are the various temperatures used
The 3rd factor is the time duration of the treatment,

the associate levels are the various time intervals considered
And so forth...
Remark:
Only 1-factor and 2-factor ANOVA are commonly applied
Stefano Siboni 5
General mathematical formulation of ANOVA
• 1-factor ANOVA
when the factor is related to the first index
X
xij = µ + αi + εij αi = 0 εij i.i.d. N (0, σ)
i
or to the second index

X
xij = µ + βj + εij βj = 0 εij i.i.d. N (0, σ)
j
• 2-factor ANOVA without interaction

X X
xij = µ + αi + βj + εij αi = 0 βj = 0
i j
εij i.i.d. N (0, σ)
• 2-factor ANOVA with interaction

X X
xijk = µ + αi + βj + αβij + εijk αi = 0 βj = 0
i j
X X
αβij = 0 ∀ j αβij = 0 ∀ i εijk i.i.d. N (0, σ)
i j
• 3-factor ANOVA without interaction
xijk = µ + αi + βj + γk + εijk εijk i.i.d. N (0, σ)

X X X
αi = 0 βj = 0 γk = 0
i j k
and so forth...
Stefano Siboni 6
A.1 1-FACTOR ANOVA (one-way ANOVA)

A.1.1 Basic definitions of 1-factor ANOVA
Mean of data
the mean of all the data
r c
1 XX 1 X
x = xij = xij
rc i=1 j=1 rc i,j
Mean of a group
the mean of the data in a row, i.e. within a group
c
1X
xi· = xij
c j=1
Deviation
difference between the value of a variable and the mean of data
xij − x
Deviation “explained” by the factor related to index i

the part of the deviation which can be attributed to the differ-
ent means of the various groups, as estimated by the relative
sample mean
xi· − x
“Unexplained” deviation
the remaining component of the deviation, which describes the
spread of the data within the group (i.e., within the row)
xij − x − (xi· − x) = xij − xi·
Stefano Siboni 7
Total variation
the sum of (deviation) squares
X
SSt = (xij − x)2
i,j
Variation “explained” by the factor related to index i

the sum of squares of deviations explained by the factor
X r
X
2
SS1 = (xi· − x) = c (xi· − x)2
i,j i=1
It expresses the data variation due to the fact that the groups
(rows) have unequal means
Also known as variation “between” the groups
Unexplained variation
the sum of squares of unexplained deviations
X
SSu = (xij − xi· )2
i,j
It describes the data spread within each group (row), due to

purely random phenomena and not to belonging to one sample
or another
It is independent on having the samples the same mean or not

Also known as variation “within” the groups
Stefano Siboni 8
A.1.2 Statistical model of 1-factor ANOVA
Model:
xij = µ + αi + εij i = 1, . . . , r j = 1, . . . , c
with: X
αi = 0
i
and
εij i.i.d. random variables N (0, σ) ∀ i, j
We make the strong assumption that σ is the same for all

the variables
The purely algebraic identity

X
(xij − x)2 = total variation SSt
i,j
X
= (xi· − x)2 explained variation SS1
i,j
X
+ (xij − xi· )2 unexplained variation SSu
i,j
holds independently on the correctness of the model
Stefano Siboni 9
Moreover, if the model is correct we have the X 2 RVs:

1 X 2
(x ij − x − αi ) =
σ 2 i,j
D1 1 E
= (εij )(I − P1 P2 ) (εij ) = X 2 rc−1

σ σ
1 X 2
(x i· − x − α i ) =
σ 2 i,j
D1 1 E
= (εij )P2 (I − P1 ) (εij ) = X 2 r−1

σ σ
1 X
2
(x ij − x i· ) =
σ 2 i,j
D1 1 E
= (εij )(I − P2 ) (εij ) = X 2 r(c−1)

σ σ
with X 2 r−1 and X 2 r(c−1) stochastically independent
List of symbols:
• (vij ) = (v11 , v12 , . . . , vrc ) ∈ Rrc (arbitrary vector of Rrc )
• P1 (vij ) = (v ·j ) (operator “mean over the 1st index”)
• P2 (vij ) = (v i· ) (operator “mean over the 2nd index”)
• α(vij ) = (αvij ) ∀ α ∈ R, ∀ (vij ) ∈ Rrc

(uij ) + (vij ) = (uij + vij ) ∀ (uij ), (vij ) ∈ Rrc
X
• h(vij )|(uij )i = vij uij ∀ (vij ), (uij ) ∈ Rrc
ij
(usual inner product in Rrc )
Stefano Siboni 10
“Projector” P1 : mean over the 1st index (rows)

 
v11 . . . v1c
(vij ) =  . . . . . .  ∈ Rrc
vr1 . . . vrc

P1 y

 
v ·1 . . . v ·c
P1 (vij ) =  . . . . . .  ∈ V1 ⊂ Rrc
v ·1 . . . v ·c
V1 : subspace of (vij ) ∈ Rrc whose “rows” are all equal
“Complement projector to P1 ”
I − P1 : (vij ) ∈ Rrc −−→ (I − P1 )(vij ) ∈ V2
V2 : subspace of (vij ) ∈ Rrc whose “columns” have zero mean
Symmetry and idempotency

h(vij )|P1 (uij )i = hP1 (vij )|(uij )i ∀ (uij ),
h(vij )|(I − P1 )(uij )i = h(I − P1 )(vij )|(uij )i (vij ) ∈ Rrc
P12 = P1 (I − P1 )2 = I − P1
=⇒ X 2 RVs by characterization theorem (rank = d.o.f.)
Vanishing product
P1 (I − P1 ) = 0
=⇒ stochastically independent RVs by Craig’s theorem
Stefano Siboni 11
“Projector” P2 : mean over the 2nd index (columns)

 
v11 . . . v1c
(vij ) =  . . . . . .  ∈ Rrc
vr1 . . . vrc

P2 y

 
v 1· . . . v 1·
P2 (vij ) =  . . . . . .  ∈ V3 ⊂ Rrc
v r· . . . v r·
V3 : subspace of (vij ) ∈ Rrc whose “columns” are all equal
“Complement projector to P2 ”
I − P2 : (vij ) ∈ Rrc −−→ (I − P2 )(vij ) ∈ V4
V4 : subspace of (vij ) ∈ Rrc whose “rows” have zero mean
Symmetry and idempotency

h(vij )|P2 (uij )i = hP2 (vij )|(uij )i ∀ (uij ),
h(vij )|(I − P2 )(uij )i = h(I − P2 )(vij )|(uij )i (vij ) ∈ Rrc
P22 = P2 (I − P2 )2 = I − P2
=⇒ X 2 RVs by characterization theorem (rank = d.o.f.)
Vanishing product
P2 (I − P2 ) = 0
=⇒ stochastically independent RVs by Craig’s theorem
Stefano Siboni 12
The projector operators P1 and P2 commute:
P1 P 2 = P2 P1
w
w

The operators of Rrc :
I − P1 P 2
P2 (I − P1 )
I − P2
are idempotent =⇒ X 2 RVs (rank = d.o.f.)
The product of the operators P2 (I − P1 ) and I − P2 vanishes:
P2 (I − P1 ) (I − P2 ) = 0
=⇒ stochastically independent RVs
These ideas can be used to prove the fundamental theorem

of 1-factor ANOVA
Stefano Siboni 13
Fundamental theorem (1-factor ANOVA)

For any set of data {xij } the total variation is the sum of
the variation explained by the factor related to i and of the
unexplained one
SSt = SS1 + SSu
Moreover, provided that the ANOVA model is correct,
we have that:
(i) if αi = 0 ∀ i, SSt /σ 2 is a X 2 variable with rc − 1 d.o.f.
(ii) if αi = 0 ∀ i, SS1 /σ 2 is a X 2 variable with r − 1 d.o.f.
(iii) in any case SSu /σ 2 is a X 2 variable with r(c − 1) d.o.f.
(iv) if αi = 0 ∀ i, SS1 /σ 2 and SSu /σ 2 are stoch. independent
Remarks
• if αi = 0 ∀ i, a balance on the number of d.o.f. is satisfied
rc − 1 = (r − 1) + r(c − 1)
• the sums of squares divided by the respective number of d.o.f.

are defined as variances
· SSt /(rc − 1) total variance
· SS1 /(r − 1) variance explained by the factor considered
· SSu /(rc − r) unexplained variance
• if αi = 0 ∀ i, the ratio of variances:

SS1 r(c − 1) SS1 σ 2 r(c − 1)
F = = 2
r − 1 SSu σ (r − 1) SSu
follows a Fisher distribution with (r − 1, rc − r) d.o.f.
Stefano Siboni 14
Test for the dependence of the means on the group

Coming back to the model, if the null hypothesis H0 is satisfied
H0 : µi = µ ∀ i = 1, . . . , r
i.e.
H0 : αi = 0 ∀ i = 1, . . . , r ,
then:
· the ratio F is a Fisher RV with (r − 1, rc − r) d.o.f.
· the explained variance SS1 /(r −1) is likely close to zero
· the unexplained variance SSu /r(c − 1) can be
assumed of the order of σ 2
In contrast, if H0 is false (so that at least one of the means

µi turns out to be different from the others) we expect that
SS1 > 2
σ
r−1 ∼
whereas the unexplained variance is however of the order of σ 2
F is taken as the test variable

and the rejection region of H0 is of the form
{F ≥ Fcritical = F[1−α](r−1,r(c−1)) }
with significance level α

(H0 is rejected whenever F turns out to be large enough)
Stefano Siboni 15
Remark. Unbalanced one-way ANOVA

for groups with unequal numbers of data
It may happen that the number of data in the various groups
is not the same (unbalanced one-way ANOVA)
The ANOVA data table cannot be regarded as a r × c matrix,
but it takes the form:
x11 x12 . . . . . . . . . x1c1
x21 x22 . . . . . . x2c2
··· ··· ··· ··· ···
xr1 xr2 . . . . . . . . . . . . xrcr
with unequal numbers of data c1 , . . . , cr for the r groups
One-way ANOVA applies as well, but paying attention that,

when H0 holds true:
Pr
• SSt /σ 2 is a X 2 variable with i=1 i − 1 d.o.f.
c
• SS1 /σ 2 is a X 2 variable with r − 1 d.o.f.
2 2
Pr
• SSu /σ is a X variable with i=1 ci − r d.o.f.
• SS1 /σ 2 and SSu /σ 2 are stochastically independent
As a consequence, the rejection region for H0 becomes:

F ≥ F[α](r−1,n−r)
Pr
where n = i=1 ci is the whole number of data
The Excel and Maple tools do not require that the number
of data is the same for all groups, except the matrix imple-
mentation of Maple’s command OneW ayAN OV A
In MATLAB the data matrix may contain some NaN entries,
allowing thereby unbalanced one-way ANOVA as well
Stefano Siboni 16
MATLAB implementation of 1-factor ANOVA

It is provided by the function anova1:
[p,table,stats] = anova1(x,group)
where:
x is a matrix whose columns contain the elements of the

different groups (the observations)
If some entries of x hold NaN, they are ignored
This enables unbalanced 1-factor ANOVA (groups with
different numbers of data)
group is a cell array of strings, one for each column of x, whose

entries are the names of the groups, e.g.:
{’gname1’ ’gname2’ ’gname3’}
{’gname1’,’gname2’,’gname3’}
or a character array, with one row for each column of x:
[’gname1’;’gname2’;’gname3’]
Omitted, or set to [ ] or {}, to specify no group name
p is the p-value
(probability of exceeding the calculated F statistic)
table is a cell array where the ANOVA table is returned
stats is a structure array that can be used to perform a further

multiple comparison test (function multcompare)
Stefano Siboni 17
Alternatively:
x is a vector containing the elements of all the groups, in

any order
group is a cell array of strings with one string for each entry of x,
in the appropriate order, the strings labelling the groups
Elements of x corresponding to the same label belongs to
the same group. E.g., if:
x= [ 2 4 1 7 5 6 ]
group = { ’A’ ’B’ ’B’ ’A’ ’B’ ’A’ }
the groups are A{2,7,6} and B{4,1,5}
Possible NaN entries in x or empty strings (”) in group cause

the corresponding data to be neglected in the analysis, e.g., if:
x= [ NaN 4 1 7 5 6]
group = { ’A’ ’B’ ’B’ ’A’ ” ’A’ }
ANOVA is performed on groups A{7,6} and B{4,1}
This form of x and choice is particularly suitable to perform

unbalanced 1-factor ANOVA (groups consisting of different
numbers of data)
Stefano Siboni 18
Running the function anova1 yields also two pictures:

(a) an ANOVA table, that looks as follows:
(b) a Box Plot of data for all the groups:
Stefano Siboni 19
Properly, the box plot has nothing to do with ANOVA:

it is an alternative way to assess if group means are equal
On each box:
− the central mark is the median
− the (lower and upper) edges of the box are the 1st and
the 3rd quartiles, q1 and q3 respectively
− the whiskers extend to the most extreme data points
within the chosen maximum whisker length
− points outside the maximum whisker length are considered
outliers, and plotted individually (red cross)
− the notch denotes the 95% CI of the median
Stefano Siboni 20
The maximum whisker length is expressed as w(q3 − q1 ),

by the interquartile range q3 − q1 (IQR) and a parameter
w whose default value is 1.5
The default of 1.5 corresponds to approximately ±2.7σ and

99.3% coverage if the data are normally distributed
Points are drawn as outliers if they are smaller than

q1 − w(q3 − q1 ) or larger than q3 + w(q3 − q1 )
Each plotted whisker extends to the most extreme data
value that is not an outlier
Stefano Siboni 21
The endpoints of the 95% CI for the median are the extremes
of the notch, estimated in the following way:
q3 − q1 q3 − q1
q2 − 1.57 √ q2 + 1.57 √
n n
where q2 is the median (2nd quartile) and n the number of data

without any NaN value
When the sample size is small, notches may extend beyond

the end of the box, leading to a picture like this:
Two medians are significantly different at the 5% significance

level if their intervals do not overlap
Notches in the boxplot provide thereby a test on group medians

different from the F -test for means in the ANOVA table
Stefano Siboni 22
Example. One-way ANOVA

The data of the following table are assumed to be drawn from 3
independent normal populations (5 data per each group) with
the same variance and a mean possibly dependent on the group:
group 1 51.7 53.0 52.0 51.8 51.0

group 2 52.1 52.3 52.9 53.6 51.1
group 3 52.8 51.8 52.3 52.8 51.8
Check at a significance level of 5% if the mean actually depends

on the group.
Solution
The table consists of r = 3 rows (one per group), each of which
containing c = 5 data. The data can be denoted by xij , with
i = 1, 2, 3 and j = 1, 2, 3, 4, 5. The levels of the unique factor
correspond to the 3 different groups. If σ 2 is the common vari-
ance, µi = µ + αi is the model for the mean of each group, and
all the sample data are assumed to be drawn from independent
normal random variables, ANOVA theory states that:
3 X 5
X (xij − x − αi )2
is a X 2 RV with rc − 1 = 14 d.o.f.
i=1 j=1
σ2
3 X
5
X (xi· − x − αi )2
is a X 2 RV with r − 1 = 2 d.o.f.
i=1 j=1
σ2
3 X 5
X (xij − xi· )2
is a X 2 RV with r(c − 1) = 12 d.o.f.
i=1 j=1
σ2
and the last two RVs are stochastically independent.
Stefano Siboni 23
For each group i = 1, 2, 3 we compute the sample mean and

variation, as shown in the table below:
c
X
group i mean xi· variation (xij − xi· )2
j=1
1 51.9 2.08
2 52.4 3.48
3 52.3 1.00
and the sum of variations over all the groups provides the un-
explained variation, or variation within groups:
r X
X c
SSu = (xij − xi· )2 = 2.08 + 3.48 + 1.00 = 6.56
i=1 j=1
The mean of data holds instead:

r c
1 XX
x = xij = 52.2
rc i=1 j=1
so that the deviations explained by the factor become
group i xi· − x
1 −0.3
2 0.2
3 0.1
and the variation explained by the factor is thereby

r X
X c
(xi· − x)2 = 5 · (−0.3)2 + 0.22 + 0.12 = 0.70

SS1 =
i=1 j=1
Stefano Siboni 24
As for the deviations xij − x of the data we finally have
group 1 −0.5 0.8 −0.2 −0.4 −1.2

group 2 −0.1 0.1 0.7 1.4 −1.1
group 3 0.6 −0.4 0.1 0.6 −0.4
along with the squares
group 1 0.25 0.64 0.04 0.16 1.44

group 2 0.01 0.01 0.49 1.96 1.21
group 3 0.36 0.16 0.01 0.36 0.16
and the total variation can be calculated as:

r X
X c
SSt = (xij − x)2 = 7.26 .
i=1 j=1
Notice that the total variation equals the sum of explained

(between groups) and unexplained (within groups) variations:
SSt = 7.26 = 0.70 + 6.56 = SS1 + SSu
The variance explained by the factor holds then
SS1 0.70
= = 0.35
r−1 2
whereas the unexplained variance is
SSu 6.56
= = 0.546667
r(c − 1) 3·4
Stefano Siboni 25
The ratio of the two variances above provides the value of the
test variable:
SS1 r(c − 1) 0.35

F = = = 0.64024
r − 1 SSu 0.546667
which, if the mean µ does not depend on the factor, obeys a

Fisher probability distribution with ( r − 1, r(c − 1) ) = (2, 12)
d.o.f.
At a significance level of 5% the rejection region becomes:
{F ≥ F[1−α](r−1,r(c−1)) } = {F ≥ F[0.95](2,12) }
i.e.
{F ≥ 3.8852938}
and since the calculated value of F does not fall within such
a rejection region, we conclude that the null hypothesis:
H0 : the mean is independent on the group (factor)
must be accepted.
There is no statistical evidence that the means of the three

statistical populations, from which the three groups of data
have been drawn, are really different.
All the above calculations can be effectively performed in an

Excel workbook by the “Anova: Single Factor” Data Anal-
ysis tool.
Stefano Siboni 26
Calculation by Excel
The application of the Excel tool to the previous data, at a
significance level α = 5% results in the following ANOVA table:
where:
· the r = 3 groups are denoted by “Row1, Row2, Row3”
· “Count” is the number of data per group (c = 5)
· “Sum” stands for the sum of data in a row
· “Average” denotes the sample mean of data in a row
· “V ariance” is the sample variance in a row
· “Source of variation” refers to the kind of variation:
− SS1 (Between Groups)
− SSu (Within Groups)
− SSt (Total)
· “SS”, “df ” and “M S” denote the corresponding values of
variation, d.o.f. and variance, respectively
· F shows the calculated value of the F -test statistic
· F crit is the lower limit F[0.95](2,12) of the critical region
Stefano Siboni 27
Notice that Excel provides also the coefficient “P -value”, i.e.,

the probability that, the null hypothesis being true, the Fisher
random variable F of the test takes by chance a value larger
than or equal to the observed one; formally:
+∞
Z
P -value = p(2,12) (F ) dF
0.6402439
where p(2,12)(F ) stands for the Fisher probability distribution

with (2, 12) d.o.f.
It is apparent from the definition that P -value is larger than

the significance level α of the test if and only if the calculated
value of F is smaller than the critical value F[α],(r−1,r(c−1)) :
P -value > α ⇐⇒ F < F[α],(r−1,r(c−1))
and in that case H0 must be accepted, since F belongs to
the acceptance region
Using P -value is an alternative way to perform the test!
For the present problem we have

P -value = 0.544256 > 0.05 = α
and thereby conclude, as before, that H0 cannot be rejected
The same calculation can be carried out also by the Maple

command OneWayANOVA.
Stefano Siboni 28
Calculation by Maple
One-way ANOVA is also implemented by the Maple Statistics
package
The data groups are defined as column vectors,
by the commands:
G1 := V ector([51.7, 53.0, 52.0, 51.8, 51.0]);
G2 := V ector([52.1, 52.3, 52.9, 53.6, 51.1]);
G3 := V ector([52.8, 51.8, 52.3, 52.8, 51.8]);
and the command OneW ayAN OV A is then applied to the list
of column vectors to obtain the desired result:
OneW ayAN OV A([G1, G2, G3]);
The output always appears as a 3 × 5 matrix:
2 .700000000 .350000000 .640243902 .5442561085
12 6.56000000 .546666667 N U LL N U LL
14 7.26000000 N U LL N U LL N U LL
The informations are the same displayed in the second section

of the Excel ANOVA report, except for:
· the position of columns SS and df , shown here in reversed
order
· the lack of the last column, F crit, not calculated
· the use of the label N U LL for the lacking entries
Alternative implementations are available, by using matri-

ces instead of column vectors, but only when the number
of data is the same for all groups!
Stefano Siboni 29
Calculation by MATLAB
In MATLAB the 1-factor ANOVA test can be easily performed
by defining a cell array of the group names:
groups = {’group 1’ ’group 2’ ’group 3’};
and the corresponding matrix of data:
x=[
51.7 53.0 52.0 51.8 51.0; . . . % data of group 1
52.1 52.3 52.9 53.6 51.1; . . . % data of group 2
52.8 51.8 52.3 52.8 51.8; . . . % data of group 3
];
to be transposed (one group per column)! The command:
[p,table,stats] = anova1(x’,groups)
yields then the ANOVA table:
which is precisely the same as before and leads to the same

conclusions (H0 cannot be rejected)
Stefano Siboni 30
The result agrees with the associated box plot:
which shows that the confidence intervals of the medians for

the three groups overlap, so that such medians are probably
equal
(remember that, due to the assumption of normal data, the

medians of each group coincide with the corresponding means)
Stefano Siboni 31
A.1.3 CI for the difference of two means

In the conditions assumed for the model — same variance σ 2
and means µi for the various groups — it is often important to
determine the confidence intervals of the mean differences
We do not want simply to know whether the means µi are all

equal or not, but also how different two means may be
The CI for the difference of two means µi and µh , i 6= h, can

be calculated by using the Student’s t distribution
The approach is the same that defines the unpaired t-test for
the comparison of the means of two independent normal
populations of equal variance
(by using groups i and h, with the same number c of data)
At a confidence level 1 − α, such a CI can then be written as:

r
1 1
xi· − xh· ± t[1− α2 ](c+c−2) sih +
c c
where s2ih is the pooled variance of the two groups:
Xc c
1 X
s2ih = (xia − xi· )2 + (xhb − xh· )2
c + c − 2 a=1
b=1
Indeed, the RV:
xi· − xh· − (µi − µh )
t = r
1 1
sih +
c c
follows exactly a Student’s distribution with c + c − 2 d.o.f.
Stefano Siboni 32
A.1.4 CIs for all the differences of two means

The previous CIs are generally not independent!
To obtain CIs valid simultaneously for all the differences, at

a confidence level 1 − α, we can apply the following result:
Scheffé theorem
For any r-uple of real coefficients (C1 , C2 , . . . , Cr ) with zero
mean
X r
Ci = 0
i=1
and for any given 1 − α ∈ (0, 1), the inequality
r r
X 2
SS u r − 1 X
Ck2

Ci (xi· − µi ) ≤ F[1−α](r−1,rc−r)

i=1
r(c − 1) c
k=1
is verified with probability 1 − α
We deduce the confidence interval,

Pr with confidence level 1 − α,
for the linear combination i=1 Ci µi :
s v
r u r
X q SSu u r − 1 X
Ci xi· ± F[1−α](r−1,rc−r) t Ck2
i=1
r(c − 1) c
k=1
In particular, for the difference of any two means µi and

µh the CI is
s r
q SSu r−1
xi· − xh· ± F[1−α](r−1,rc−r) 2
r(c − 1) c
with a confidence level 1 − α, but independently on the
choice of i and h
Stefano Siboni 33
MATLAB calculation of the CIs

for all the differences of two means
The calculation of the CIs for all the differences of two means
is also known as multiple comparison test
In MATLAB the test is implemented by the function
multcompare
The function acts on the structure array (stats) generated by
the function anova1 as a third output value:
and it has the following form:
c = multcompare(stats,’name’,value,. . .)
The main parameters in the form of a ’name’/value pair are:
’alpha’ any α ∈ (0, 1) such that 1 − α is the confidence level
for the CIs of all the differences of means
(α = 0.05 by default)
’display’ one of the strings ’on’ (default) or ’off’, to show an
interactive graph of all the estimates of means,
with comparison intervals around them
’ctype’ the method used to define CIs for the comparisons:
’hsd’ Tukey’s honestly significant difference
criterion (default)
’scheffe’ Scheffé method
’bonferroni’ Bonferroni-corrected t critical values
’lsd’ Tukey’s least significant difference
’dunn-sidak’ Dunn-Sidák method
Stefano Siboni 34
The multiple comparison test based on Scheffé theorem

with significance level α = 0.10 is thus obtained by:
c = multcompare(stats,’ctype’,’scheffe’,’alpha’,0.10)
The output c is a matrix with one row for each pair of groups,
consisting of:
◦ the number of one group (i)
◦ the number of another group (j > i)
◦ the lower limit of the (1 − α)-CI for µi − µj
◦ the sample estimate to the difference of means µi − µj
◦ the upper limit of the (1 − α)-CI for µi − µj
For n groups, the matrix c has precisely n(n − 1)/2 rows,

whereas the number of columns is always 5
The interactive graph of all the estimates of means shows

for each group:
◦ the estimate to the mean
◦ the comparison interval for that mean
(a horizontal segment centred at the estimate of the mean)
The width of the comparison intervals is half the width

of all the CIs for the differences of the means
Hence, if two comparison intervals overlap
the corresponding means are not significantly different
If this is not the case, the difference is significant
Stefano Siboni 35
Let ∆x be the common half-length of the (1 − α)-CI for the

difference of any two means, µi − µh . Then, equivalently:
µi and µh are equal with significance level α
m
the hypothesized difference 0 belongs to the (1 − α)-CI of µi − µh
0 ∈ [xi· − xh· − ∆x, xi· − xh· + ∆x]
xi· − xh· − ∆x ≤ 0 ≤ xi· − xh· + ∆x
−∆x ≤ xi· − xh· ≤ ∆x
m

xi· − xh· ≤ ∆x
m
the comparison intervals of µi and µh overlap
Stefano Siboni 36
Another example. One-way ANOVA

Four non-destructive techniques are checked that allow to
measure the Mg concentration in a sample. By repeating the
analysis 4 times for each technique, the data below are found:
Method Mg concentration (ppm)
A 76.42 78.62 80.40 78.20
B 80.41 82.26 81.15 79.20
C 74.20 72.68 78.84 80.32
D 86.20 86.04 84.36 80.68
where the techniques are conventionally named A, B, C and D.

Does the sample suggest, at a significance level of 5%, that the
four techniques are equivalent?
Solution
The unique factor is the method of analysis, with r = 4 levels
each corresponding to a technique A, B, C, or D. Each group
of measurements obtained by the same technique contains an
equal number c = 4 of data. Data can be denoted by xij ,
with i, j = 1, 2, 3, 4, the first index referring to the group, the
second specifying the datum. With the usual assumptions and
notations, the ANOVA model states that:
4 X 4
X (xij − x − αi )2
2
is a X 2 RV with rc − 1 = 15 d.o.f.
i=1 j=1
σ
4 X
4
X (xi· − x − αi )2
is a X 2 RV with r − 1 = 3 d.o.f.
i=1 j=1
σ2
4 X 4
X (xij − xi· )2
is a X 2 RV with r(c − 1) = 12 d.o.f.
i=1 j=1
σ2
Stefano Siboni 37
and that the last two RVs are stochastically independent.

The Excel Data Analysis tool provides the ANOVA table:
where we find:
− the explained variation (“between groups”)
Xr X c
SS1 = (xi· − x)2 = 135.762475
i=1 j=1
− the unexplained variation (“within groups”)

Xr X c
SSu = (xij − xi· )2 = 72.6605
i=1 j=1
− the explained variance

SS1 135.762475
= = 45.254158
r−1 3
− the unexplained variance
SSu 72.6605
= = 6.0550417
r(c − 1) 4·3
Stefano Siboni 38
The test variable F holds thereby:
SS1 r(c − 1) 45.254158

F = = = 7.473798
r − 1 SSu 6.0550417
while the rejection region, at a significance level α = 5%, writes:
{F ≥ F[1−α](r−1,r(c−1)) } = {F ≥ F[0.95](3,12) }
that is
{F ≥ 3.4902948}
As the calculated F is larger than F crit and thus located

within the rejection region, the null hypothesis:
H0 : the mean is independent on the technique
must be rejected.
Equivalently, the same conclusion stems from the very low
P -value:
P -value = 0.0044065 < 0.05 = α
Finally, the collected data suggest that the methods of mea-

surement are not all equivalent, since the means of the mea-
surements are expected to depend on the technique
In other words, at least one of the techniques must introduce

some systematic error in the measurement of magnesium
concentration
Stefano Siboni 39
Remark
To have an idea of how different are the performances of the
various techniques, we can calculate the CI for all the mean
differences, for instance at a confidence level 1 − α = 66%
Let µ1 , µ2 , µ3 , µ4 be the means related to the techniques A,
B, C, D (i.e., the Mg concentrations theoretically detected by
the various methods on the same sample)
First method: Student’s distribution

For any i, h = 1, 2, 3, 4, i 6= h, the CI for the difference µi − µh ,
at confidence level α, is given by Student’s formula
r
1 1
xi· − xh· ± t[1− α2 ](c+c−2) sih +
c c
in terms of the pooled variance s2ih of the two groups i and h:
X c c
1 X
s2ih = (xia − xi· )2 + (xhb − xh· )2 =
c + c − 2 a=1
b=1
c c
1 X 1 X
= (xia − xi· )2 + (xhb − xh· )2
2(c − 1) a=1 2(c − 1)
b=1
For the i-th group, the sample variance
c
1 X
(xia − xi· )2
c − 1 a=1
is at the last column of the Summary in the Excel ANOVA
table.
Therefore, s2ih equals the arithmetic mean of the sample vari-
ances of groups i and h
Finally, the last but one column of the Summary provides the
sample mean xi· .
Stefano Siboni 40
The estimates to the mean differences (centers of the CIs) and

the pooled variances are easily calculated:
Estimates to mean differences, xi· − xh·

A B C D
A −2.35 1.90 −5.91
B 2.35 4.24 −3.57
C −1.90 −4.24 −7.81
D 5.91 3.57 7.81
Pooled variances s2ih

A B C D
A 2.161083 7.993067 4.625400
B 2.161083 7.484683 4.117017
C 7.993067 7.484683 9.949000
D 4.625400 4.117017 9.949000
Since for α = 0.34 and c = 4 there holds

r
1 1
t[1− α2 ](2c−2) = t[0.83](6) = 1.036324 + = 0.707107
c c
we finally deduce the half-width of the CIs:
Half-width of the CI for the mean differences

A B C D
A 1.08 2.07 1.58
B 1.08 2.00 1.49
C 2.07 2.00 2.31
D 1.58 1.49 2.31
Stefano Siboni 41
Second method: Scheffé theorem

For any i and h as before, the CI of the difference µi − µh can
be written by Scheffé theorem as
s r
q SSu r−1
xi· − xh· ± F[1−α](r−1,rc−r) 2
r(c − 1) c
with a confidence level 1 − α and independently on the

choice of i and h (simultaneous CIs)
Of course, the centers of the CIs are the same as before
For α = 0.34, r = 4, c = 4, and recalling the value of the

unexplained variance, we obtain:
F[1−α](r−1,rc−r) = F[0.66](3,12) = 1.234799734

SSu
= 6.055041667
r(c − 1)
r−1
2 = 1.5
c
and the common half-width of all the CIs for the mean dif-
ferences becomes
s r
q SSu r−1
F[1−α](r−1,rc−r) 2 = 3.348902172 ' 3.35
r(c − 1) c
at a 66%-confidence level.
Notice that the latter value is significantly larger than those

estimated by Student’s formula: this was the price to pay for
the uniformity of the estimate, at the same confidence level
Stefano Siboni 42
A.2 2-factor ANOVA without interaction

The set of random variables with two indices
xij i = 1, . . . , r j = 1, . . . , c
describes the measured values of a quantity x concerning a

material treated by a certain chemical reagent at a given
temperature
− i identifies the chemical reagent
− j specifies the treatment temperature
Question
Have the chemical reagent or the treatment temperature a
significant effect on the property x of the material?
Model
For any i = 1, . . . , r and j = 1, . . . , c the random variable xij
is normal with mean µij and variance σ 2
Remark
The mean of the variable xij may depend on both the indices
i and j (on both the chemical reagent and the temperature),
while the variance σ 2 is the same
Hypothesis to be tested
H0 : means µij independent on the index i (or j)
H1 : means µij significantly dependent on the index i (or j)
Stefano Siboni 43

without interaction
Mean of data
r c
1 XX 1 X
x = xij = xij
rc i=1 j=1 rc i,j
Mean over the index j

c
1X
xi· = xij
c j=1
Mean over the index i

r
1X
x·j = xij
r i=1
Deviation
xij − x
xi· − x
Deviation “explained” by the factor related to index j
x·j − x
xij − x − (xi· − x) − (x·j − x) = xij − xi· − x·j + x
Stefano Siboni 44
Total variation
X
SSt = (xij − x)2
i,j
Variation explained by the factor related to index i
X r
X
SS1 = (xi· − x)2 = c (xi· − x)2
i,j i=1
Variation explained by the factor related to index j
X c
X
2
SS2 = (x·j − x) = r (x·j − x)2
i,j j=1
X
SSu = (xij − xi· − x·j + x)2
i,j
Stefano Siboni 45

without interaction
Model:
xij = µ + αi + βj +εij i = 1, . . . , r j = 1, . . . , c
| {z }
µij
with: X X
αi = 0 βj = 0
i j
and
εij i.i.d. random variables N (0, σ) ∀ i, j
σ is assumed to be the same for all the variables!
The purely algebraic identity

X
(xij − x)2 total variation SSt
i,j
X
= (xi· − x)2 variation explained by the 1st factor, SS1
i,j
X
+ (x·j − x)2 variation explained by the 2nd factor, SS2
i,j
X
+ (xij − x·j − xi· + x)2 unexplained variation SSu
i,j
is independent on the correctness of the model
Stefano Siboni 46
But if the model is correct we also have the X 2 RVs:
1 X
2
(xij − x − αi − βj )2 =
σ i,j
D1 1 E
= (εij )(I − P1 P2 ) (εij ) = X 2 rc−1

σ σ
1 X
2
(xi· − x − αi )2 =
σ i,j
D1 1 E
= (εij )P2 (I − P1 ) (εij ) = X 2 r−1

σ σ
1 X 2
(x ·j − x − β j ) =
σ 2 i,j
D1 1 E
= (εij )(I − P2 )P1 (εij ) = X 2 c−1

σ σ
1 X
2
(xij − xi· − x·j + x)2 =
σ i,j
D1 1 E
= (εij )(I − P2 )(I − P1 ) (εij ) = X 2 (r−1)(c−1)

σ σ
with
X 2 r−1 X 2 c−1 X 2 (r−1)(c−1)
stochastically independent
The latter result leads to the fundamental theorem of 2-

factor ANOVA without interaction
Stefano Siboni 47
Fundamental theorem (2-factor ANOVA without int.)
For any set {xij } the total variation coincides with the sum of
− the variation explained by the factor related to i
− the variation explained by the factor associated to j
− the unexplained variation
SSt = SS1 + SS2 + SSu
Moreover, provided that the ANOVA model is correct,
we have that:
(i) if αi = 0 ∀ i and βj = 0 ∀ j:
SSt /σ 2 is a X 2 variable with rc − 1 d.o.f.
(ii) if αi = 0 ∀ i:
SS1 /σ 2 is a X 2 variable with r − 1 d.o.f.
(iii) if βj = 0 ∀ j:
SS2 /σ 2 is a X 2 variable with c − 1 d.o.f.
(iv) in any case:

SSu /σ 2 is a X 2 variable with (r − 1)(c − 1) d.o.f.
Finally:
− in case (ii) the RVs SS1 /σ 2 and SSu /σ 2
are stochastically independent
− in case (iii) the RVs SS2 /σ 2 and SSu /σ 2

Stefano Siboni 48
Remarks
• for αi = 0 ∀ i and βj = 0 ∀ j the number of d.o.f. is balanced
rc − 1 = (r − 1) + (c − 1) + (r − 1)(c − 1)
• we define the variances

SSt /(rc − 1) total variance
SS1 /(r − 1) variance explained by the factor related to i
SS2 /(c − 1) variance explained by the factor related to j
SSu /(r − 1)(c − 1) unexplained variance
• if αi = 0 ∀ i, the ratio of the variance explained by the factor

i to the unexplained one
SS1 (r − 1)(c − 1) SS1 σ 2 (r − 1)(c − 1)

F = = 2
r−1 SSu σ (r − 1) SSu
follows a Fisher distribution with (r − 1, (r − 1)(c − 1)) d.o.f.
• if βj = 0 ∀ j, the ratio of the variance explained by the factor

j to the unexplained one
SS2 (r − 1)(c − 1) SS2 σ 2 (r − 1)(c − 1)

F = = 2
c−1 SSu σ (c − 1) SSu
is a Fisher random variable with (c − 1, (r − 1)(c − 1)) d.o.f.
Stefano Siboni 49
Test for the dependence of the means on the index i
H0 : means µij independent on the index i

(αi = 0 ∀ i = 1, . . . , r)
H1 : means µij appreciably dependent on the index i
(αi ’s not all vanishing)
Test variable:
SS1 (r − 1)(c − 1) SS1
F = = (c − 1)
r−1 SSu SSu
which, for H0 true, is a Fisher RV with (r − 1, (r − 1)(c − 1))
d.o.f.
If H0 is true, there typically holds
SS1 SSu
' σ2 ⇐⇒ F small
r−1 (r − 1)(c − 1)
whereas in the opposite case we expect that

SS1 > SSu
∼ ' σ2 ⇐⇒ F large
r−1 (r − 1)(c − 1)
We define then an acceptance region for H0 , of the form:
F < F[1−α](r−1,(r−1)(c−1))
and a corresponding rejection region:
F ≥ F[1−α](r−1,(r−1)(c−1))
both with given significance level α ∈ (0, 1)
Stefano Siboni 50
Test for the dependence of the means on the index j
H0 : means µij independent on the index j

(βj = 0 ∀ j = 1, . . . , c)
H1 : means µij appreciably dependent on the index j
(βj ’s not all vanishing)
Test variable:
SS2 (r − 1)(c − 1) SS2
F = = (r − 1)
c−1 SSu SSu
which, for H0 true, is a Fisher RV with (c − 1, (r − 1)(c − 1))
d.o.f.
SS2 SSu
c−1 (r − 1)(c − 1)
while in the opposite case it seems reasonable to assume that

SS2 > SSu
c−1 (r − 1)(c − 1)
F < F[1−α](c−1,(r−1)(c−1))
along with a corresponding rejection region:
F ≥ F[1−α](c−1,(r−1)(c−1))
both with given significance level α ∈ (0, 1)
Stefano Siboni 51

without interaction
It is provided by the function anova2:
[p,table,stats] = anova2(x,displayopt)
where:
x is the matrix of data:

- columns correspond to the different levels of one factor
- rows correspond to the different levels of the other factor
- no NaN value is allowed (balanced ANOVA only)
displayopt is one of the strings ’on’ (default) or ’off ’

and serves to enable the ANOVA table display
(ANOVA table on a separate figure)
p is a row vector with two p-values:

- the 1st for the null hypothesis that µij does not depend
on the column factor;
- the 2nd for the null hypothesis that µij does not depend
on the row factor
table is a 5 × 6 cell array where the ANOVA table is returned

Stefano Siboni 52
Example. Two-way ANOVA without interaction

15 samples of a polymer cut from the same, homogeneous sheet
are aged by 10-hour exposition to 3 different types of UV lamp,
and subsequent immersion in 5 different solutions for 2 hours.
After rinsing in water and drying in nitrogen flow, the rough-
ness of the 15 samples has been measured. The table below
shows the results (in µm):
sol. 1 sol. 2 sol. 3 sol. 4 sol. 5

lamp 1 51.7 53.0 52.0 51.8 51.0
lamp 2 52.1 52.3 52.9 53.6 51.1
lamp 3 52.8 51.8 52.3 52.8 51.8
Check, with a significance level of 5%, if the sample roughness

is affected (a) by the kind of UV lamp, or (b) by the choice of
the solution.
Solution
The table consists of r = 3 rows, each corresponding to a dif-
ferent model of lamp, and of c = 5 columns, each related to one
kind of solution. As usual, we can denote the data by xij , with
i = 1, 2, 3 and j = 1, . . . , 5. The first factor is the kind of lamp,
and it admits r = 3 levels; the second factor is the solution
employed, with c = 5 possible levels. As each entry of the ta-
ble corresponds to a unique combination of the two factors, no
measurement repetition is available in the same experimental
conditions and we can only apply a two-way ANOVA without
interaction: the data are assumed to be normal, independent
and with same unknown variance σ 2 , while the mean may only
have the form µij = µ + αi + βj ∀ i = 1..3, j = 1..5, with
P3 P5
i=1 αi = 0 and j=1 βj = 0.
Stefano Siboni 53
ANOVA theory proves that, if the model is correct, the RVs:

r X c
2
X (xij − x − αi − βj )2
X rc−1 =
i=1 j=1
σ2
r X
c
2
X (xi· − x − αi )2
X r−1 =
i=1 j=1
σ2
r X c
2
X (x·j − x − βj )2
X c−1 =
i=1 j=1
σ2
r X c
2
X (xij − xi· − x·j + x)2
X (r−1)(c−1) =
i=1 j=1
σ2
follow X 2 distributions with:
rc − 1 r−1 c−1 (r − 1)(c − 1)
d.o.f., respectively, and that moreover the last three variables

are stochastically independent.
Checking that the roughness is not affected by the model of
lamp is equivalent to check the null hypothesis:
H0 : αi = 0 ∀ i = 1, 2, 3
against the alternative that this is not true.

Similarly, verifying that the final roughness is the same regard-
less of the kind of solution used corresponds to check the null
hypothesis:
H0 : βj = 0 ∀ j = 1, . . . , 5
versus the alternative hypothesis that this is not the case.
Stefano Siboni 54
To check the two hypotheses we must compute:
− the variance explained by the 1st factor (lamp)

r c
SS1 1 XX
= (xi· − x)2 = 0.35
r−1 r − 1 i=1 j=1
− the variance explained by the 2nd factor (solution)

r c
SS2 1 XX
= (x·j − x)2 = 0.8716667
c−1 c − 1 i=1 j=1

SSu
=
(r − 1)(c − 1)
r X c
1 X
= (xij − x·j − xx· + x)2 = 0.3841667
(r − 1)(c − 1) i=1 j=1
Independence on the 1st factor (lamp)

The independence on the type of lamp is checked by calculating
the ratio of the variance explained by the 1st factor to the
unexplained variance:
SS1 (r − 1)(c − 1) 0.35

F = = = 0.9110629
r−1 SSu 0.3841667
and considering the rejection region:
{F ≥ F[1−α](r−1,(r−1)(c−1)) }
Stefano Siboni 55
which for r = 3, c = 5 and α = 0.05 reduces to
{F ≥ F[0.95](2,8) = 4.4589701}
Since F = 0.9110629 < 4.4589701 = F[0.95](2,8) , we conclude

that H0 cannot be rejected and therefore the choice of
the UV lamp probably does not affect roughness in a
significant way.
Independence on the 2nd factor (solution)

The independence on the kind of solution can be verified by
using the ratio of the variance explained by the 2nd factor to
the unexplained variance:
SS2 (r − 1)(c − 1) 0.87166667

F = = = 2.2689805
c−1 SSu 0.3841667
The rejection region is now:
{F ≥ F[1−α](c−1,(r−1)(c−1)) }
and for r = 3, c = 5, α = 0.05 becomes
{F ≥ F[0.95](4,8) = 3.8378534}
As before, the calculated test statistic is smaller than the crit-

ical value, so that the null hypothesis cannot be refused.
Probably, roughness does not depend on the choice of
the solution either.
We thereby conclude that the sample roughness is likely af-

fected neither by the model of lamp nor by the kind of solution
employed for the ageing treatment
Stefano Siboni 56
The previous calculations can be easily performed by means
of the “Anova: Two-Factor Without Replication” Data
Analysis tool. For α = 5%, we obtain the ANOVA table below:
where:
· the r = 3 levels of the 1st factor are denoted as “Rows”
· the c = 5 levels of the 2nd factor are named “Columns”
· “Count” stands for the number of data in a Row/Column
· “Average” is the sample mean within a Row/Column
· “V ariance” is the sample variance within a Row/Column
· “Source of variation” specifies the kind of variation:
· SS1 (Rows) - explained by the 1st factor
· SS2 (Columns) - explained by the 2nd factor
· SSu (Error) - unexplained
· SSt (Total) - total
· “SS”, “df ” and “M S” denote the corresponding values of
variation, d.o.f. and variance, respectively
Stefano Siboni 57
· the column “F ” shows the calculated values of the F -test

statistic for the independence on the 1st factor (Rows) and
on the 2nd factor (Columns)
· the column “F crit” collects the corresponding lower limits
of the rejection region, for the given α
As in the one-way case, Excel allows an alternative implemen-

tation of the tests by computing the “P -value” of each F -test
statistic
Remind that by definition the P -value is the probability that
the Fisher random variable takes by chance a value larger than
or equal to the observed one
In the present case the P -value of the first test statistic is
thereby:
+∞
Z
P -value = p(2,8) (F ) dF = 0.440086429
0.911062907
as easily checked by the function F DIST (0, 911062907; 2; 8).

Similarly, for the second test statistic we have:
+∞
Z
P -value = p(4,8) (F ) dF = 0.150596383
2.268980477
i.e., the same result of F DIST (2, 268980477; 4; 8).

The p-value is anyway larger than the significance level α
P -value > α = 0.05
so that the null hypotheses cannot be rejected.
Stefano Siboni 58
In MATLAB the 2-factor ANOVA test without interactions can
be performed by defining the matrix of data:
x = [ ...
51.7 53.0 52.0 51.8 51.0 ; . . .
52.1 52.3 52.9 53.6 51.1 ; . . .
52.8 51.8 52.3 52.8 51.8 ];
where the 1st factor is the kind of solution (columns), and

the 2nd factor the model of UV lamp (rows)
Both the equivalent commands:
[p,table,stats] = anova2(x)
[p,table,stats] = anova2(x,1,’on’) % reps=1
yield then the ANOVA table:
which is precisely the same as before, except that the critical

values of the test statistics are not calculated.
The test must be performed on the p-values,
also returned in the output row vector p
Stefano Siboni 59
Another example. Two-way ANOVA without inter.

3 different cleaners are tested with 4 different kinds of deter-
gent. The following table shows the effectiveness of the clean-
ing, as measured on a suitable scale:
cleaner C1 C2 C3
detergent
D1 53 50 59
D2 54 54 60
D3 56 58 62
D4 50 45 57
Check, with a significance level of 5%, the hypothesis that the

final result is not affected: (a) by the choice of the detergent,
and (b) by the cleaner employed.
Solution
The 1st factor is represented by the kind of detergent, with
r = 4 different leves (D1, D2, D3, D4), while the 2nd factor
corresponds to the model of cleaner, with c = 3 levels (C1,
C2, C3). For each combination detergent-cleaner we have pre-
cisely one measurement, so we can apply ANOVA without in-
teractions. With the usual assumptions and notations, we can
introduce the X 2 random variables:
r X c
X (xij − x − αi − βj )2
X 2 rc−1 =
i=1 j=1
σ2
Stefano Siboni 60
c
r X
X (xi· − x − αi )2
X 2 r−1 =
i=1 j=1
σ2
r X c
2
X (x·j − x − βj )2
X c−1 =
i=1 j=1
σ2
r X c
2
X (xij − xi· − x·j + x)2
X (r−1)(c−1) =
i=1 j=1
σ2
with rc − 1 = 11, r − 1 = 3, c − 1 = 2 and (r − 1)(c − 1) = 6

d.o.f., respectively. Moreover, we known that X 2 r−1 , X 2 c−1
and X 2 (r−1)(c−1) are stochastically independent. The Excel
Data Analysis tool provides the two-way ANOVA table below:
where we find:
· the variance explained by the 1st factor (Rows/detergent):
r c
SS1 1 XX
= (xi· − x)2 = 34.111111
r−1 r − 1 i=1 j=1
Stefano Siboni 61
· the variance explained by the 2nd factor (Columns/cleaner):

r c
SS2 1 XX
= (x·j − x)2 = 67.583333
c−1 c − 1 i=1 j=1
· the unexplained variance (Error):
SSu
=
(r − 1)(c − 1)
r X c
1 X
= (xij − x·j − xx· + x)2 = 3.6944444
(r − 1)(c − 1) i=1 j=1
To check whether the type of detergent may affect the result

we need the ratio of the variance explained by the 1st factor to
the unexplained one:
SS1 (r − 1)(c − 1) 34.111111

F = = = 9.2330827
r−1 SSu 3.6944444
along with the critical value:
F[1−α](r−1,(r−1)(c−1)) = F[0.95](3,6) = 4.7570627
Since F > F[0.95](3,6) , with a significance level of 5% we must

reject the null hypothesis that the 1st factor does not affect
the cleaning quality. The kind of detergent is probably
relevant to the final result of the cleaning process.
The same conclusion is suggested by the P -value:
P -value = 0.011490324 < 0.05 = α
Stefano Siboni 62
As for the possible effect of the cleaner, we consider the ratio

of the variance explained by the 2nd factor to the unexplained
variance:
SS2 (r − 1)(c − 1) 67.583333

F = = = 18.293233
c−1 SSu 3.6944444
and the critical value:
F[1−α](c−1,(r−1)(c−1)) = F[0.95](2,6) = 5.1432529
Since the rejection region is of the form:
{F ≥ F[1−α](c−1,(r−1)(c−1)) } = {F ≥ F[0.95](2,6) } ,
at the 5% significance level the inequality F > F[0.95](2,6) makes

us to reject the null hypothesis that the cleaning result is inde-
pendent on the choice of the cleaner. The sample suggests that
the quality of the cleaning significanlty depends on the
cleaner used
The same conclusion is supported by the calculated P -value:
P -value = 0.00279666 < 0.05 = α
Finally, the result of the ANOVA is that the quality of the

cleaning process is likely affected by both the type of detergent
and the choice of the cleaner.
Stefano Siboni 63
Remark. CIs for the coefficients αi and βj

In the previous example we may also wonder, for instance:
(c) which is the increment of the expected cleaning “score”
when detergent D1 is used instead of D2, D3 or D4;
(d) which is the improvement of the cleaning performance if
cleaner C1 is used instead of C2 or C3.
Two-way ANOVA may help us to answer this kind of questions.
Problem (c) is equivalent to obtain an estimate of the coeffi-
cient α1 of our ANOVA model without interaction; similarly,
question (d) can be answered by a satisfactory estimate of the
coefficient β1 .
More generally, it is possible to determine appropriate CIs for
all the coefficients αi and βj .
This can be done because, if the ANOVA model is correct:
− ∀ i = 1, . . . , r the RV
p xi· − x − αi
ti = rc(c − 1) √
SSu
obeys a Student’s distribution with (r − 1)(c − 1) d.o.f.,

so that a CI for the coefficient αi is given by:
s
SSu
xi· − x ± t[1− α2 ]((r−1)(c−1))
rc(c − 1)
at a confidence level α;
Stefano Siboni 64
− ∀ j = 1, . . . , c the RV
0
p x·j − x − βj
tj = rc(r − 1) √
SSu
follows a Student’s distribution with (r − 1)(c − 1) d.o.f.,
and a CI for the coefficient writes thereby:
s
SSu
x·j − x ± t[1− α2 ]((r−1)(c−1))
rc(r − 1)
with the same confidence level α as before.
Coming back to the example, and assuming 1 − α = 66% as a

confidence level, we obtain for α1 :
r
SSu
x1· − x ± t[0.83](6) =
24
r
22.16667
= 54.0 − 54.83333 ± 1.03632 =
24
= − 0.83333 ± 0.99596
and for β1 :
r
SSu
x·1 − x ± t[0.83](6) =
36
r
22.16667
= 53.25 − 54.83333 ± 1.03632 =
36
= − 1.58333 ± 0.81319
No wonder the precision is so small, owing to the small size
of the sample (12 data in all). In spite of that, the results
suggest that the use of detergent D1 or cleaner C1 makes the
cleaning performances worse in comparison with the other
detergents and cleaners.
Stefano Siboni 65
A.3 2-factor ANOVA with interaction

The set of random variables with three indices
xijk i = 1, . . . , r j = 1, . . . , c, k = 1, . . . , s
describes the measured values of a quantity x concerning a

material treated by a certain chemical reagent at a given tem-
perature:
− i identifies the chemical reagent
− j specifies the treatment temperature
− k denotes a repeated measurement for given i and j
In this case the measurement is repeated s times for each

combination of levels i and j
Question
Have the chemical reagent or the treatment temperature
a significant effect on the property x of the material?
In there any interaction between the effects of the reagent
and those of the temperature?
Model
For any i = 1, . . . , r, j = 1, . . . , c and k = 1, . . . , s the random
variable xijk is normal with mean µij and variance σ 2
Remark
The mean of the variable xijk may depend on the indices
i and j (i.e., on the chemical reagent and the temperature),
while the variance σ 2 is the same
Stefano Siboni 66

with interaction
Mean of data
r c s
1 XXX 1 X
x = xijk = xijk
rcs i=1 j=1 rcs
k=1 ijk
Mean over the index k

s
1X
xij· = xijk
s
k=1
Mean over the indices j and k

c s
1 XX
xi·· = xijk
cs j=1
k=1
Mean over the indices i and k

r s
1 XX
x·j· = xijk
rs i=1
k=1
Stefano Siboni 67
Deviation
xijk − x

(actually, by the additive contribution of the factor)
xi·· − x
Deviation “explained” by the factor related to index j

x·j· − x
Deviation “explained” by the i-j interaction
xij· − x − (xi·· − x) − (x·j· − x) =

= xij· − xi·· − x·j· + x
xijk − x − (xi·· − x) − (x·j· − x) − (xij· − xi·· − x·j· + x) =

= xijk − xij·
Stefano Siboni 68
Total variation
X
SSt = (xijk − x)2
ijk
Variation explained by the factor related to index i

X
SS1 = (xi·· − x)2
ijk
Variation explained by the factor related to index j

X
SS2 = (x·j· − x)2
ijk
Variation explained by the i-j interaction

X
SS12 = (xij· − xi·· − x·j· + x)2
ijk
X
SSu = (xijk − xij· )2
ijk
Stefano Siboni 69

with interaction
Model:
µij
z }| {
xijk = µ + αi + βj + αβij + εijk
i = 1, . . . , r j = 1, . . . , c k = 1, . . . , s
with:
X X
αi = 0 βj = 0
i j
X X
αβij = 0 ∀j αβij = 0 ∀i
i j
and
εijk i.i.d. random variables N (0, σ) ∀ i, j, k
The purely algebraic identity:

X
(xijk − x)2 total variation SSt
ijk
X
= (xi·· − x)2 var.on explained by the 1st factor, SS1
ijk
X
+ (x·j· − x)2 var.on explained by the 2nd factor, SS2
ijk
X var.on explained by the interaction
+ (xij· − xi·· − x·j· + x)2
between the factors 1 and 2, SS12
ijk
X
+ (xijk − xij· )2 unexplained variation SSu
ijk
holds also when the model is incorrect
Stefano Siboni 70
But if the model is correct we can introduce the X 2 RVs

1 X 2
(x ijk − x − α i − β j − αβ ij ) =
σ2
ijk
D1 1 E
= (εijk )(I − P3 P2 P1 ) (εijk ) = X 2 rcs−1

σ σ
1 X 2
(x i·· − x − αi ) =
σ2
ijk
D1 1 E
= (εijk )P3 P2 (I − P1 ) (εijk ) = X 2 r−1

σ σ
1 X
2
(x·j· − x − βj )2 =
σ
ijk
D1 1 E
= (εijk )P3 (I − P2 )P1 (εijk ) = X 2 c−1

σ σ
1 X
2
(xij· − xi·· − x·j· + x − αβij )2 =
σ
ijk
D1 1 E
= (εijk )P3 (I − P2 )(I − P1 ) (εijk ) = X 2 (r−1)(c−1)

σ σ
1 X
2
(xijk − xij· )2 =
σ
ijk
D1 1 E
= (εijk )(I − P3 ) (εijk ) = X 2 rc(s−1)

σ σ
with X 2 r−1 X 2 c−1 X 2 (r−1)(c−1) X 2 rc(s−1)
stochastically independent
(P3 denotes the operator “mean over the 3rd index”)
Stefano Siboni 71
Fundamental theorem (2-factor ANOVA with int.)
For any set {xijk } the total variation is the sum of

− the variation explained by the factor related to i
− the variation explained by the factor associated to j
− the variation explained by the interaction of the 2 factors
− the unexplained variation
SSt = SS1 + SS2 + SS12 + SSu
Moreover, provided that the ANOVA model holds true,

we have that:
(i) if αi = 0 ∀ i, βj = 0 ∀ j and αβij = 0 ∀ i, j:

SSt /σ 2 is a X 2 variable with rcs − 1 d.o.f.
(ii) if αi = 0 ∀ i:
SS1 /σ 2 is a X 2 variable with r − 1 d.o.f.
(iii) if βj = 0 ∀ j:
SS2 /σ 2 is a X 2 variable with c − 1 d.o.f.
(iv) if αβij = 0 ∀ i, j:
SS12 /σ 2 is a X 2 variable with (r − 1)(c − 1) d.o.f.
(v) in any case:

SSu /σ 2 is a X 2 variable with rc(s − 1) d.o.f.
Stefano Siboni 72
Finally:
− in case (ii) the RVs SS1 /σ 2 and SSu /σ 2

− in case (iii) the RVs SS2 /σ 2 and SSu /σ 2

− in case (iv) the RVs SS12 /σ 2 and SSu /σ 2

Stefano Siboni 73
Remarks
• for αi = 0 ∀ i, βj = 0 ∀ j and αβij = 0 ∀ i, j

the number of d.o.f. is balanced
rcs − 1 = (r − 1) + (c − 1) + (r − 1)(c − 1) + rc(s − 1)
• we define the variances

SSt /(rcs − 1) total variance
SS1 /(r − 1) variance explained by the factor related to i
SS2 /(c − 1) variance explained by the factor related to j
SS12 /(r − 1)(c − 1) variance explained by the interactions
SSu /rc(s − 1) unexplained variance
Stefano Siboni 74
• if αi = 0 ∀ i, the ratio of the variance explained by the factor

i to the unexplained one
SS1 rc(s − 1) SS1 σ 2 rc(s − 1)

F = = 2
r − 1 SSu σ (r − 1) SSu
follows a Fisher distribution with (r − 1, rc(s − 1)) d.o.f.
• if βj = 0 ∀ j, the ratio of the variance explained by the factor

j to the unexplained one
SS2 rc(s − 1) SS2 σ 2 rc(s − 1)

F = = 2
c − 1 SSu σ (c − 1) SSu
is a Fisher random variable with (c − 1, rc(s − 1)) d.o.f.
• if αβij = 0 ∀ i, j, the ratio of the variance explained by the

interaction to the unexplained one
SS12 rc(s − 1) SS12 σ 2 rc(s − 1)

F = = 2
(r − 1)(c − 1) SSu σ (r − 1)(c − 1) SSu
is a Fisher random variable with ((r − 1)(c − 1), rc(s − 1)) d.o.f.
Stefano Siboni 75
Test for the dependence of the means on the index i
H0 : means µij mainly independent on the index i

(precisely, αi = 0 ∀ i = 1, . . . , r)
(αi ’s not all vanishing)
Test variable:
SS1 rc(s − 1)
F =
r − 1 SSu
which, for H0 true, is a Fisher RV with (r − 1, rc(s − 1)) d.o.f.
SS1 SSu
r−1 rc(s − 1)
SS1 > SSu

r−1 rc(s − 1)
F < F[1−α](r−1,rc(s−1))
F ≥ F[1−α](r−1,rc(s−1))
both with a significance level α ∈ (0, 1)
Stefano Siboni 76
Test for the dependence of the means on the index j
H0 : means µij mainly independent on the index j

(precisely, βj = 0 ∀ j = 1, . . . , c)
H1 : means µij appreciably dependent on the index j
(βj ’s not all vanishing)
Test variable:
SS2 rc(s − 1)
F =
c − 1 SSu
which, for H0 true, is a Fisher RV with (c − 1, rc(s − 1)) d.o.f.
SS2 SSu
c−1 rc(s − 1)
SS2 > SSu

c−1 rc(s − 1)
F < F[1−α](c−1,rc(s−1))
F ≥ F[1−α](c−1,rc(s−1))
Stefano Siboni 77
Test for the occurrence of interactions on the means
H0 : no significant interaction term on the means µij

(αβij = 0 ∀ i = 1, . . . , r , ∀ j = 1, . . . , c)
(αβij ’s not all vanishing)
Test variable:
SS12 rc(s − 1)
F =
(r − 1)(c − 1) SSu
a Fisher RV with ((r − 1)(c − 1), rc(s − 1)) d.o.f., for H0 true
SS12 SSu
(r − 1)(c − 1) rc(s − 1)
SS12 SSu
>
(r − 1)(c − 1) rc(s − 1)
F < F[1−α]((r−1)(c−1),rc(s−1))
F ≥ F[1−α]((r−1)(c−1),rc(s−1))
Stefano Siboni 78

with interaction
It consists in a different construct of the function anova2:
[p,table,stats] = anova2(x,reps,displayopt)
where:
reps is the number of replications

(the same for all the level combinations of the two factors)
x is the matrix of data:

- each column corresponds to one level of the 1st factor
- each block of reps consecutive rows corresponds
to one level of the 2nd factor
- no NaN value is allowed (balanced ANOVA only)
Here is an example of a matrix x with 2 levels for the column
factor A, 3 levels for the row factor B and 2 replications:
Stefano Siboni 79
displayopt is one of the strings ’on’ (default) or ’off’

and serves to enable the ANOVA table display
(ANOVA table on a separate figure)
p is a row vector with three p-values:

- the 1st for the null hypothesis that µij does not depend
on the column factor;
- the 2nd for the null hypothesis that µij does not depend
on the row factor
- the 3rd for the null hypothesis that µij does not depend
on the interaction of the two factors
table is a 6 × 6 cell array where the ANOVA table is returned

Stefano Siboni 80
Example. Two-way ANOVA with interaction

A researcher wants to compare the ultimate tensile strength of
plywood panels produced by 3 kinds of timber and 3 different
types of adhesive. To this end, he prepares 5 samples for each
possible combination timber-glue and performs a stress test on
each sample, obtaining the results shown below:
glue GL1 GL2 GL3

timber
196 208 247 214 216 235 258 250 264
poplar
216 221 240 252 248 272
216 228 240 215 217 235 246 247 261
birch
224 236 219 241 250 255
230 242 232 212 218 216 255 251 261
beech
244 228 224 222 258 247
We want to check, at a significance level of 5%, the hypothesis:

(a) that the effects of timber and glue are simply additive;
(b) that the choice of the timber does not affect the ultimate
tensile strength of the panel;
(c) that the type of adhesive is irrelevant to the ultimate ten-
sile strength of the product.
Solution
The 1st factor consists in the kind of timber, with r = 3 possible
levels (poplar, birch, beech). The 2nd factor is the type of
adhesive, still with c = 3 levels (denoted by GL1, GL2, GL3).
For each combination timber-glue the same number s = 5 of
repeated measurements has been performed. It is then possible
Stefano Siboni 81
to apply an ANOVA with interaction. The 2-factor dependent

means are written as µij = µ + αi + βj + αβij , while the data
are denoted as xijk , with i = 1 . . . r, j = 1 . . . c and k = 1 . . . s.
If the ANOVA model is correct we known that the RVs:
r X c Xs
X (xijk − x − αi − βj − αβij )2
2
= X 2 rcs−1
i=1 j=1
σ
k=1
r X
c X
s
X (xi·· − x − αi )2
= X 2 r−1
i=1 j=1 k=1
σ2
c X
r X s
X (x·j· − x − βj )2
= X 2 c−1
i=1 j=1 k=1
σ2
r X
c X
s
X (xij· − xi·· − x·j· + x − αβij )2
= X 2 (r−1)(c−1)
i=1 j=1 k=1
σ2
r X c X s
X (xijk − xij· )2 2
= X rc(s−1)
i=1 j=1 k=1
σ2
obey X 2 distributions with:
rcs − 1 r−1 c−1 (r − 1)(c − 1) rc(s − 1)
d.o.f., respectively, and moreover that the RVs:
X 2 r−1 X 2 c−1 X 2 (r−1)(c−1) X 2 rc(s−1)
Stefano Siboni 82
Checking if the possible effects of the kind of wood and glue

are purely additive is equivalent to check the null hypothesis:
H0 : αβij = 0 ∀ i = 1, 2, 3 , ∀ j = 1, 2, 3
versus the alternative that this does not hold true.
Similarly, testing that the ultimate tensile strength does not

depend significantly on the kind of wood is equivalent to
check the null hypothesis:
H0 : αi = 0 ∀ i = 1, 2, 3
against the alternative that this is not the case.
Finally, the possible independence of ultimate tensile strength

on the kind of adhesive is checked by the null hypothesis:
H0 : βj = 0 ∀ j = 1, 2, 3
versus the alternative that at least one coefficient βj is nonzero.
Stefano Siboni 83
To perform the task we must calculate:
− the variance explained by the 1st factor (wood)
SS1 1 X
= (xi·· − x)2 = 1.755555
r−1 r−1
ijk
− the variance explained by the 2nd factor (glue)
SS2 1 X
= (x·j· − x)2 = 4145.0889
c−1 c−1
ijk
− the variance explained by the interaction between factors
SS12
=
(r − 1)(c − 1)
1 X
= (xij· − xi·· − x·j· + x)2 = 330.98889
(r − 1)(c − 1)
ijk
SSu 1 X
= (xijk − xij· )2 = 121.8222
rc(s − 1) rc(s − 1)
ijk
Stefano Siboni 84
(a) Lack of interactions

We need to calculate the ratio of the variance explained by the
interaction between the factors to the unexplained variance:
SS12 rc(s − 1) 330.98889

F = = = 2.71698
(r − 1)(c − 1) SSu 121.8222
and consider the rejection region:
{F ≥ F[1−α]((r−1)(c−1),rc(s−1)) }
which for r = 3, c = 3, s = 5 and α = 0.05 becomes:
{F ≥ F[0.95](4,36) = 2.63353}
As F = 2.71698 > 2.63353 = F[0.95](4,36) , we must reject

the null hypothesis that the interactions between the two
factors are completely irrelevant to the tensile strength.
(b) Independence on the 1st factor (wood)

The ratio of the variance explained by the 1st factor to the
unexplained variance is:
SS1 rc(s − 1) 1.755555

F = = = 0.01441
r − 1 SSu 121.8222
while the rejection region becomes:
{F ≥ F[1−α](r−1,rc(s−1)) = F[0.95](2,36) = 3.25945}
and we conclude thereby that the type of timber probably

does not affect the tensile strength of the plywood panels.
Stefano Siboni 85
(c) Independence on the 2nd factor (adhesive)

The independence on the choice of the adhesive can be tested
by computing the ratio of the variance explained by the 2nd
factor to the unexplained variance:
SS2 rc(s − 1) 4145.0889

F = = = 34.02572
c − 1 SSu 121.8222
The rejection region takes the form:
{F ≥ F[1−α](c−1,rc(s−1)) }
and for r = 3, c = 3, s = 5, α = 0.05 becomes:
{F ≥ F[0.95](2,36) = 3.25945}
so that the null hypothesis should be certainly rejected. We

must conclude that the choice of the adhesive significantly
affects the ultimate tensile strength of the product.
Finally, at a significance level of 5% ANOVA suggests that:

◦ a slight interaction occurs between the two factors;
◦ nevertheless, the tensile strength does not significantly de-
pend on the type of wood;
◦ the tensile stregth is mainly affected by the kind of glue.
Stefano Siboni 86
All the previous calculations can be easily performed by using
the Excel “Anova: Two-Factor With Replication” Data
Analysis tool, which provides the ANOVA table below:
In the Summary section, for each of the r = 3 levels of the

1st factor (wood) a table is given where the usual parameters
“Count”, “Average” and “V ariance” are provided for each of
the c = 3 levels of the 2nd factor (glue)
Stefano Siboni 87
In the ANOVA Section:

· “Source of variation” classifies the kind of variation:
− SS1 (Sample) - explained by the 1st factor
− SS2 (Columns) - explained by the 2nd factor
− SS12 (Interactions) - explained by the interaction
− SSu (Within) - unexplained
− SSt (Total) - total
· “SS”, “df ” and “M S” stand for the related values of vari-
ation, d.o.f. and variance, respectively
· the column “F ” collects the calculated values of the F-test
statistic for the independence on the 1st factor (Sample),
the independence on the 2nd factor (Columns) and the
lack of interaction (Interaction)
· the column “F crit” shows the corresponding lower limits
of the rejection region, for the assigned α
The column “P -value” provides an alternative implementation
of the F -test in the three cases, by computing the probability
of exceeding by chance the calculated value of the F statistic:
+∞
Z
· P -value = p(2,36) (F ) dF = 0.985698 > 0.05
0.01441
+∞
Z
· P -value = p(2,36) (F ) dF = 5.04637 10−9 < 0.05
34.02572
+∞
Z
· P -value = p(4,36) (F ) dF = 0.04482 < 0.05
2.71698
Stefano Siboni 88
In MATLAB the 2-factor ANOVA test with interactions is car-
ried out by defining the matrix of data:
x = [ ...
196 214 258 ; ...
208 216 250 ; ...
247 235 264 ; ...
216 240 248 ; ...
221 252 272 ; ...
216 215 246 ; ...
228 217 247 ; ...
240 235 261 ; ...
224 219 250 ; ...
236 241 255 ; ...
230 212 255 ; ...
242 218 251 ; ...
232 216 261 ; ...
244 224 258 ; ...
228 222 247 ...
];
The levels of the 1st factor are associate to columns, so that

each column contains data related to one kind of glue
The levels of the 2nd factor correspond to blocks of 5 rows

each, in such a way that each block of 5 rows collects data
pertaining to a kind of timber
Replicated data for each combination of levels are accordingly

stored in 5 × 1 submatrices
Stefano Siboni 89
In both the equivalent commands:
[p,table,stats] = anova2(x,5,’on’)
[p,table,stats] = anova2(x,5)
the second argument reps = 5 is mandatory (default = 1)
Either of them yields then the ANOVA table:
which is precisely the same as before, except that the critical

values of the test statistics are not calculated.
The test must be performed on the p-values,

also returned in the output row vector p
Stefano Siboni 90
Another example. Two-way ANOVA with interaction

We want to check 3 kinds of gasoline in combination with 3
different additives. Each gasoline-additive pair is tested with 4
equal vehicles, using 5 gallons of fuel each time. The distances
(in miles) covered by the vehicles are listed in the table below:
additive Add1 Add2 Add3

gasoline
126.2 124.8 130.4 131.6 127.0 126.6
Gas1
125.3 127.0 132.5 128.6 129.4 130.1
127.2 126.6 142.1 132.6 129.5 142.6
Gas2
125.8 128.4 128.5 131.2 140.5 138.7
127.1 128.3 132.3 134.1 125.2 123.3
Gas3
125.1 124.9 130.6 133.6 122.6 120.9
At a significance level of 5%:

(a) can we assert that all the gasolines yield analogous re-
sults?
(b) can we deem that the various additives have different ef-
fects on the fuel performances?
(c) can we conclude that an interaction effect occurs?
Solution
The 1st factor corresponds to the kind of gasoline, with the
r = 3 different levels Gas1, Gas2, Gas3. The 2nd factor consists
in the type of additive, with c = 3 different levels (Add1, Add2,
Add3). For each gasoline-additive pair we have s = 4 repeated
measurements, and thereby we can apply a two-way ANOVA
with interactions.
Stefano Siboni 91
For r = 3, c = 3 and s = 4, the Excel Data Analysis tool leads

to the ANOVA table below:
where we can find:

· the variance explained by the 1st factor (Sample/gasoline):
r c s
SS1 1 XXX
= (xi·· − x)2 = 102.58861
r−1 r − 1 i=1 j=1
k=1
Stefano Siboni 92
· the variance explained by the 2nd factor (Columns/additive):

r c s
SS2 1 XXX
= (x·j· − x)2 = 106.65194
r−1 c − 1 i=1 j=1
k=1
· the variance explained by the interaction of the two factors

(Interaction):
SS12
=
(r − 1)(c − 1)
r X c Xs
1 X
= (xij· − x·j· − xi·· + x)2 = 66.4677
(r − 1)(c − 1) i=1 j=1
k=1
· the unexplained variance (Within):

r X c Xs
SSu 1 X
= (xijk − xij· )2 = 9.4040741
rc(s − 1) rc(s − 1) i=1 j=1
k=1
(a) To check if the kind of gasoline affects the result, the

ANOVA table provides the ratio of the variance explained by
the 1st factor to the unexplained one:
SS1 rc(s − 1) 102.58861
F = = = 10.908954
r − 1 SSu 9.4040741
along with the related critical value:
F[1−α](r−1,rc(s−1)) = F[0.95](2,27) = 3.354131
As F > F[0.95](2,27) , at the significance level of 5% we can
conclude that probably the kind of gasoline affects the
performances of the vehicle.
The same conclusion is suggested by the calculated P -value:
P -value = 0.00033697 < 0.05 = α
Stefano Siboni 93
(b) To test the possible relevance of the additive, we need

the ratio of the variance explained by the 2nd factor to the
unexplained variance:
SS2 rc(s − 1) 106.65194
F = = = 11.34104
c − 1 SSu 9.4040741
and the corresponding critical value:
F[1−α](c−1,rc(s−1)) = F[0.95](2,27) = 3.354131
and we deduce that the kind of additive is relevant to the
vehicle performance, due to the inequality F > F[0.95](2,27) .
This result is consistent with the P -value:
P -value = 0.0002659 < 0.05 = α
(c) As for the possible interactions, we find in the ANOVA

table the ratio of the variance explained by the interaction to
the unexplained one:
SS12 rc(s − 1) 66.4677
F = = = 7.067977
(r − 1)(c − 1) SSu 9.4040741
and the related critical value:
F[1−α]((r−1)(c−1),rc(s−1)) = F[0.95](4,27) = 2.72777
The condition F > F[0.95](4,27) leads to the conclusion that
probably the effect of interaction between the kind of
gasoline and the choice of the additive is significant
and must be taken into due account.
The same result stems from the P -value:
P -value = 0.00049833 < 0.05 = α
Stefano Siboni 94
A.4 MATLAB implementation of the general

n-factor ANOVA
The general n-factor ANOVA is implemented by the function
anovan
The same function is also used to implement the 2-factor
ANOVA with interaction if the number of replications is
not the same for all level combinations (unbalanced ANOVA)
The full statement of the function is as follows:
[p,table,stats,terms] = anovan(y,group,’name’,value,. . .)
where:
y is the vector of data, possibly with NaN values
group is a cell array, each cell of which contains a list of levels

classifying the data in y with respect to one of the factors;
each list must have the same number of elements as y;
the number of cells is the number of factors n;
the cell order in the array corresponds to the factor order
p is a column vector collecting the p-values for the null

hypotheses of the performed ANOVA model tests
table is a cell array where the ANOVA table is returned

terms is a matrix encoding the main and interaction terms used

in the ANOVA computations
Stefano Siboni 95
The structure of the matrix terms is illustrated below for the

case of the 3-factor ANOVA, with factors A, B and C:
Matrix row ANOVA term

[1 0 0] main term A
[0 1 0] main term B
[0 0 1] main term C
[1 1 0] interaction term AB
[1 0 1] interaction term AC
[0 1 1] interaction term BC
[1 1 1] interaction term ABC
The main ’name’/value pairs are:
(•) ’alpha’, a number α ∈ (0, 1) that defines the significance

level of the F-tests (default is α = 0.05)
(•) ’display’, ’on’ displays the ANOVA table (default),

’off’ omits the display
(•) ’model’, to specify the kind of ANOVA model to be used:

’linear’ to consider only the n main effects (default)
’interaction’ to consider the n main effects
and the n(n-1)/2 two-factor interactions
’full’ to consider the main effects
and the interactions at all levels
a positive integer k≤n to calculate all the interaction
effects through order k
a matrix of term definitions having the same format as the
previous matrix terms
Stefano Siboni 96
(•) ’varnames’, a character matrix or a cell array of strings

specifying names of grouping variables (factors), one per
grouping variable. If not specified, the default labels ’X1’,
’X2’, ’X3’, ..., ’XN’ are used
Remember that:
in the presence of n factors with no replications of data,

the interactions can be calculated and tested up to order n-1
in the case of n factors with replications for at least one

combination of levels of all the factors, interactions can
be calculated and checked up to order n
when ’full’ model is specified, anovan calculates the order n

interaction also in the case where no replications are available,
but the ANOVA tests fail because the unexplained (residual)
variance vanishes and the F ratios are thereby singular
Stefano Siboni 97
Table of contents
A. F-test for the comparison of means (ANOVA) . . . . . . 1
A.1 1-factor ANOVA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
A.1.1 Basic definitions of 1-factor ANOVA . . . . . . . . . . . . . . . . 7
A.1.2 Statistical model of 1-factor ANOVA . . . . . . . . . . . . . . . 9
A.1.3 CI for the difference of two means . . . . . . . . . . . . . . . . . . 32
A.1.4 CIs for all the differences of two means . . . . . . . . . . . . . 33
A.2 2-factor ANOVA without interaction . . . . . . . . . . . . . . . . 43
without interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
without interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
A.3 2-factor ANOVA with interaction . . . . . . . . . . . . . . . . . . . 66
A.3.1 Basic definitions of 2-factor ANOVA with interaction 67
A.3.2 Statistical model of 2-factor ANOVA with interaction 70
A.4 MATLAB implementation of the general
n-factor ANOVA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
Stefano Siboni -1

Anova

Uploaded by

Copyright:

Available Formats

Anova

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Anova

Uploaded by

Copyright:

Available Formats

Ph. D.

in Materials Engineering Statistical methods

A. F-test for the comparison of means (ANOVA)

i specifies the group (treatment)

j identifies an element of the i-th group

For all i = 1, . . . , r and j = 1, . . . , c the random variable xij is

H1 : µh 6= µk for at least one h and one k in 1, . . . , r

1-factor analysis of variance

The data are classified according to an appropriate experimen-

In the present example, the only factor which classifies the

The factor admits r different “levels”, each corresponding to

In this example, each level corresponds to a particular kind

The ANOVA we apply is the 1-factor ANOVA

It may happen that the data are classified by two or more

For instance, the treatment could consist in the action of a

In this case each datum will be identified by a pair of indices

The 1st factor corresponds to the kind of chemical reagent

The 2nd factor is the temperature,

Another case is when for any combination reagent-temperature

In such a case each datum will be labeled by a triplet of

Further, the treatment could be specified by the kind of reagent,

Each experimental datum will correspond to a combination of

The 1st factor corresponds to the kind of chemical reagent

The 2nd factor consists in the temperature,

The 3rd factor is the time duration of the treatment,

General mathematical formulation of ANOVA

or to the second index

• 2-factor ANOVA without interaction

εij i.i.d. N (0, σ)

• 2-factor ANOVA with interaction

• 3-factor ANOVA without interaction

xijk = µ + αi + βj + γk + εijk εijk i.i.d. N (0, σ)

A.1 1-FACTOR ANOVA (one-way ANOVA)

Deviation “explained” by the factor related to index i

Variation “explained” by the factor related to index i

It describes the data spread within each group (row), due to

It is independent on having the samples the same mean or not

A.1.2 Statistical model of 1-factor ANOVA

We make the strong assumption that σ is the same for all

The purely algebraic identity

holds independently on the correctness of the model

Moreover, if the model is correct we have the X 2 RVs:

• P1 (vij ) = (v ·j ) (operator “mean over the 1st index”)

• P2 (vij ) = (v i· ) (operator “mean over the 2nd index”)

• α(vij ) = (αvij ) ∀ α ∈ R, ∀ (vij ) ∈ Rrc

“Projector” P1 : mean over the 1st index (rows)

V1 : subspace of (vij ) ∈ Rrc whose “rows” are all equal

Symmetry and idempotency

“Projector” P2 : mean over the 2nd index (columns)

V3 : subspace of (vij ) ∈ Rrc whose “columns” are all equal

Symmetry and idempotency

The projector operators P1 and P2 commute:

The operators of Rrc :

are idempotent =⇒ X 2 RVs (rank = d.o.f.)

The product of the operators P2 (I − P1 ) and I − P2 vanishes:

=⇒ stochastically independent RVs

These ideas can be used to prove the fundamental theorem

Fundamental theorem (1-factor ANOVA)

• the sums of squares divided by the respective number of d.o.f.