Lesson 10 – Tests of Differences

Between Two Groups (Part 2)

.
The t-test for Dependent Groups

• Studies involving “before-and-

after” tests
• What is compared is performance
before and after a treatment is
administered
Problem

To motivate citizens to conserve gasoline, the government is

considering mounting a nationwide conservation campaign.
However, before doing so on a national level, it decides to conduct
an experiment to evaluate the effectiveness of the campaign. For
the experiment, the conservation campaign is conducted in a small
but representative geographical area. Twelve families are randomly
selected from the area, and the amount of gasoline they use is
monitored for 1 month before the advertising campaign and for 1
month after the campaign. The following data are collected:
Data
Problem Analysis

• In this type of study, a single group is used (12 randomly selected

families). The group is then exposed to a treatment (i.e., an
energy conservation campaign). The researcher is interested in
whether the conservation campaign (specifically an advertising
campaign) is effective in modifying the respondents’ energy
consumption (measured in terms of gasoline use on a particular
month).
• Notice that our problem does not mention any directional terms
(such as increase, enhance, improve, etc.). This suggests that a
two-tailed (non-directional) hypothesis testing procedure is to
be employed.
•
Solution

Step 1: Write the Problem

Problem: Does the energy conservation campaign affect gasoline

consumption among families?

Can you think of other ways of writing the problem? Please type
your answers in the chatbox.
Solution

Step 2: Write the Hypothesis

Null Hypothesis: H0: Before = After

or
H0: Before - After = 0

Alternate Hypothesis: H1: Before  After

or
H1: Before - After  0
Solution

Step 2: Write the Hypothesis

In statement form, thus:

Null Hypothesis: The energy conservation campaign does not affect

gasoline consumption among families.

Alternative Hypothesis: The energy conservation campaign affects

gasoline consumption among families.
Solution

Step 3: Set criteria for decision making

 The null hypothesis will be tested at .05 level of significance (two-
tailed)

Step 4: Collect data and compute statistics

 The statistic to be used here is the t-test for dependent groups (or the
dependent t-test). First we layout our data in a table (see next slide):
Solution
 Note the additional columns (under Difference).
Column D is simply the difference between
each “before” and “after” score, while D2 is the
squared value of this score. Check table values
first to see that you are on track.
• Note here as well that it is possible to get a
negative value for D (i.e., when the “before”
score is lower than the “after” score). Since
we are doing a two-tailed test, we do not
reverse the groupings to be assured of a
positive t-value obtained.
• Notice that we also computed the mean value
of D, which is D/N. The mean value of D in
this case is 2.917.
• There is, likewise, a D2 value. Review the
table data again before we proceed.
Solution

• First, we compute the sum From here, we compute the t-value

of squares for D. using this formula:

(Note: the answer is actually 2.907)

Solution

• Computing the Degrees of Freedom

In a dependent t-test, the formula for Degrees of Freedom is DF = N – 1
(not N – 2 because we only have one group). This, our DF in this problem
is DF = 12 – 1 = 11. Referring now to our table of critical values of t, our
tcrit = 2.201.
Solution

Step 5: We now make our decision:

 Since the obtained t-value (2.91) is greater than the critical t-value
(2.201) at .05 level of significance, the null hypothesis is rejected.
Therefore, the energy conservation campaign affected gasoline
consumption among families.

In APA format:

 “The energy conservation campaign affected gasoline consumption

among families, t = 2.91, p<.05two-tailed.”
Lesson 11 – Introduction to
Analysis of Variance
Looking Back

• Lesson 10
• Comparing two independent
groups (each receiving a different
treatment) to check significant
mean differences between
groups.
• Comparing “before and after”
measures after exposing a single
group to a treatment.
Looking Forward

• What is there are more than two

groups (i.e., three or more), which
means there are more than two
treatment conditions. What to do
then?
Context

• Suppose you are interested in determining

the effects of different types of
processing on students’ memory
retention.
• Participants in this experiment were
shown lists of words and asked to answer
questions about each word.
• The questions were designed to require
different levels of processing, from
superficial to deep.
Context: Experimental Conditions

• Physical (i.e., processing based on the physical characteristics of

each printed word. “Is it printed in capital letters or lowercase
letters?”)
• Sound (i.e., processing based on the sound of each word. “Does it
rhyme with ‘boat’?”)
• Meaning (i.e., processing based on the meaning of each word.
“Does it have the same meaning as ‘attractive’?”)
• Self (i.e., processing based on the participants’ understanding of
each word and relating its meaning to themselves. “Does this word
describe you?”)
Context: Experimental Procedure

• Sample size: 40
• Random assignment to four groups
(n=10/group)
• Participants in each group were
shown lists of words, and asked to
answer questions about each word
• Note: the participants were NOT
trying to memorize the words; they
were simply reading through the list
and answering questions
Context: Results

• As you can see in Figure 12.1, deeper

processing resulted in better
memory.
• The more they processed and
understood the words, the better
they recalled the words on the test.
Preliminary Questions

1. What is our independent variable?

2. How many levels are there (of the independent variable?)
3. What is our dependent variable?
4. What is the main difference between the design of this study and
the design of the study in Lesson 10?
5. What problem can you propose for this study?
What to do with more than two groups?

• Since there are four (4) groups in this

experiment, there will obviously be four
(4) means, and the statistical procedure
would require the researcher to
compare these means to see whether
the mean differences are statistically
significant.
• How to do it?
• Do pairwise comparisons six times!
Introducing ANOVA

• Analysis of variance (ANOVA) is a hypothesis-testing procedure

that is used to evaluate mean differences between more than
two treatments (or populations). As with all inferential
procedures, ANOVA uses sample data as the basis for drawing
general conclusions about populations.
• It may appear that ANOVA and t tests are simply two different
ways of doing exactly the same job: testing for mean differences.
In some respects, this is true—both tests use sample data to test
hypotheses about population means.
Introducing ANOVA

• ANOVA has a tremendous advantage over t tests.

• Specifically, t tests are limited to situations in which there are
only two treatments to compare.
• The major advantage of ANOVA is that it can be used to compare
more than two treatments.
• Thus, ANOVA provides researchers with much greater flexibility in
designing experiments and interpreting results.
Introducing ANOVA
Introducing ANOVA

• Figure 12.2 shows a typical research situation for which ANOVA

would be used.
• Note that the study involves three samples representing three
populations.
• The goal of the analysis is to determine whether the mean
differences observed among the samples provide enough evidence
to conclude that there are mean differences among the three
populations.
Introducing ANOVA

• Specifically, we must decide between two interpretations:

• There really are no differences between the populations (or treatments).
The observed differences between the sample means are caused by
random, unsystematic factors (sampling error) that differentiate one
sample from another.
• The populations (or treatments) really do have different means, and these
population mean differences are responsible for causing systematic
differences between the sample means.
• You should recognize that these two interpretations correspond to
the two hypotheses (null and alternative) that are part of the
general hypothesis-testing procedure.
Terminologies in ANOVA

• Independent Variable (also called

Factor)
• There are more than two levels of
the Independent Variable
• Dependent Variable
Example

• You are doing a study that examines

driving performance under three different
telephone conditions:
• Driving with no phone
• Driving while talking on a hands-free phone
• Driving while talking on a hand-held phone

• Independent Variable?
• Levels?
• Dependent Variable?
One Way Analysis of Variance

• Since there is only one

independent variable in the study
being examined, this statistical
procedure is more properly called
a “One-Way (or Single Factor)
Analysis of Variance”
Statistical Hypotheses for One-Way ANOVA

• Suppose that a researcher examined driving performance under

three different telephone conditions: no phone, a hands-free
phone, and a hand-held phone.
• Three samples of participants are selected, one sample for each
treatment condition.
• The purpose of the study is to determine whether using a
telephone while driving affects driving performance.
• In statistical terms, we want to decide between two hypotheses:
the null hypothesis (H0), which states that the “using a telephone”
condition has no effect, and the alternative hypothesis (H1), which
states that the “using a telephone” condition affects driving.
Statistical Hypotheses for One-Way ANOVA

H0: 1 = 2 = 3
• H0: Using a telephone does not affect
driving performance.
H1: not H0
• H1: Using a telephone affects driving
performance.
Limitations of the One-Way ANOVA

• It can determine whether significant differences exist between

groups.
• Does not determine which pairs of means are significantly
different.
• Need for a Post-Hoc test (Scheffe Test)
Computational Procedures

The process for performing a one-way analysis of variance requires

that we produce the following Sums of Squares (SS):
• SSW or Sum of Squares Within
• SSB or Sum of Squares Between
• SST or Total Sum of Squares

The following formula will be used to generate these sums of

squares:
• SSW = X12 + X22 + X32 – [(X1)2/n1 + (X2)2/n2 + [(X3)2/n3]
• SSB = [(X1)2/n1 + (X2)2/n2 + [(X3)2/n3] – (X1 + X2 + X3)2/N
• SST = X12 + X22 + X32 - (X1 + X2 + X3)2/N
Sample Problem

Suppose you are interested in determining whether certain

situations produce differing amounts of stress. You know the amount
of the hormone corticosterone circulating in the blood is a good
measure of how stressed a person is. You randomly assign 15
students into three groups of 5 each. The students in Group 1 have
their corticosterone levels measured immediately after returning
from vacations. The students in Group 2 have their corticosterone
levels measured after they have been in class for a week. The
students in Group 3 are measured immediately before final exam
week. All measurements are taken at the same time of day. Scores
are in milligrams of corticosterone per 100 milliliters of blood.
Terms

Independent Variable: Type of

Situation
• Levels: Level 1: Vacation
Level 2: Class
Level 3: Final Exam
Dependent Variable: Amount of Stress
Problem

• Does the type of situation affect stress level?

• (If we go directly with what is given, we may also write something
like this: Do certain situations produce differing amounts of
stress?)
Hypotheses and Criteria for Decision Making

Statistical Hypotheses: H0: 1 = 2 = 3

H1: not H0

Research Hypotheses: H0: The type of situation does not affect

stress level.
H1: The type of situation affects stress
level.
Set Criterion at .05 (Level of Significance)
Collect Data and Compute Statistics
Solving for Sums of Squares
Solving for Sums of Squares
Solving for Degrees of Freedom

In a One-Way ANOVA, we need to generate three degrees of freedom

(i.e., DFBetween, DFWithin, and DFTotal) using the following formulae:

• DFBetween = k – 1 (where k is the number of groups)

• DFWithin = N – k (where N is the total number of respondents)
• DFTotal = N - 1
Solving for Degrees of Freedom

Using our data then: DFBetween =k–1

=3–1=2
DFWithin =N–k
= 15 – 3 = 12
DFTotal =N–1
= 15 – 1 = 14
Creating the Summary Table for 1-Way ANOVA
Creating the Summary Table for 1-Way ANOVA

Notes:
1. To compute MSB (Mean Squares Between) = SSB/DFBetween
2. To compute MSW (Mean Squares Within) = SSW/DFWithin
3. To compute Fobtained (One Way ANOVA) = MSB/MSW
4. To determine Fcrit, we consult our table appendices for ANOVA. Use
DFB as your Numerator DF, and DFW as your denominator DF. In this
case, since our DFBetween or Numerator DF is 2, and our DFWithin or
Denominator DF is 12, our Fcrit at .05 level of significance is 3.88
(Refer to table below. Lightface (.05) and Boldface (.01). We always
use the values in Lightface.
Measuring Effect Size

• The effect size shows the extent to which the independent

variable affected the dependent variable.
• To provide an indication of how large the effect actually is,
researchers should report a measure of effect size in addition to
the measure of significance.
• The formula for effect size in a One-Way ANOVA is as follows:
SSB/SST. Our effect size, then, is 203.333/257.333 = 0.79. The
symbol for effect size is 2 (or eta squared).
Interpreting Effect Size

The effect size tells us the extent to which the dependent variable
is caused by the independent variable. In our study, therefore, it
can be concluded that 0.79 (or 79%) of stress levels is attributed to
the type of situation. The remaining 0.21 (or 21%) of stress is caused
by other factors, including chance or error.
Make a Decision

• Since the obtained F value is greater than the critical value for F
at .05 level of significance, the null hypothesis is rejected.
Therefore, the type of situation affects stress level.
APA Format

The means and standard deviations are presented in Table 1. The

analysis of variance indicates that there are significant differences
in the stress levels among the respondents when exposed to
different types of situations, suggesting that the type of situation
affects stress levels, F (2,12) = 22.593, p < .05., 2 = 0.79.
Performing Post-Hoc Tests
Scheffe Test
Conclusions (after Scheffe Test)

• The three F values obtained following the Scheffe Test will then be compared
with an “adjusted Fcrit”, which is essentially our existing Fcrit multiplied by
(number of groups – 1). For our example, then, our adjusted Fcrit is 3.88 x (3-1)
= 7.76. Each of the F values obtained using the Scheffe Test will now be
compared with 7.76.

F1-2 = 8.889 > 7.76 (mean 1 and mean 2 are significantly different)
F1-3 = 45 > 7.76 (mean 1 and mean 3 are significantly different)
F2-3 = 13.889 > 7.76 (mean 2 and mean 3 are significantly different)

• What are our conclusions? Since all the F values obtained exceeded the adjusted
F , we can now conclude that all pairs of means are significantly different.