COMPUTER PRACTICAL

COMPUTER ANALYSIS OF CONTINUOUS VARIABLES;

CONFIDENCE INTERVALS, T-TESTS AND ANALYSIS OF VARIANCE

Introduction

We learned how to calculate confidence intervals for the mean of a continuous variable, and how
to compare means between two groups. We also learned how to compare two means when they
are based on separate measurements for the same person (or same unit of observation, more
generally). All the calculation in those sessions were carried out on paper using a calculator.

In the first part of this session, we will learn how to carry out the same tasks in Stata..

An alternative way of testing statistically the difference between the means of a variable between
two groups is analysis of variance. We shall introduce this technique. Analysis of variance can
also be used to compare the means of several groups and we shall cover this. It is a powerful
technique that has still more applications; we shall not cover these in this session, but will indicate
some of these other applications.

One advantage of computer statistical analysis is that it is much easier to look at the distributions
of variables. This is always recommended as a preliminary to further work on the variables.

Thus in this session , where a response variable to be a continuous variable, will shall cover

* The overall distribution of the response

* The distribution of the response in subgroups defined by some other variables

* How to make comparisons between these subgroups

Reminder

Read in the data from the file babnew and open a log file. Type the command:

summ bweight, detail

produces a detailed description of the distribution of bweight. This includes the mean and
standard deviation, the median, some percentiles, the four smallest and largest values and the
number of non-missing observations. It is useful to accompany this summary with a histogram of
the distribution of bweight which we also looked at in the Introduction session, by typing

hist bweight, bin(12) freq

and

hist bweight, bin(12) freq norm

(The choice of 12 bins - that is, with birthweight divided into 12 groups - is arbitrary but seems to
work OK). The normal distribution, with the same mean and standard deviation as that observed
in the dataset, is superimposed.

1
Try repeating this command, leaving the number of bins at the default (that is by omitting the
option “bin”.

Look at the distribution of gestation age, in the same way. What would you say about this
distribution?

Neither birthweight nor gestational age have truly symmetric distributions. Furthermore, the
distribution of gestational age suggests a sizeable number of premature births. Type in a suitable
command to get the histogram for the distribution of birthweight for babies of more than 32 weeks
gestation; does it appear to be closer to a Normal distribution?

Confidence Intervals

The confidence interval for a mean is produced thus:-

ci bweight

One can ask for confidence intervals for several variables at a time:-

ci bweight gestwks matage

One can also vary the confidence level with the keyword level. For example:-

ci bweight gestwks matage, level(99)

These confidence intervals are based on the t-distribution, and strictly this assumes Normality in
the data. The fact that the distributions are somewhat skewed has been noted. However, this
effect is not very large, and the sample size is quite large, and one can trust these confidence
intervals on account of the central limit theorem (which we recall states that the distribution of a
mean is closer to the normal distribution than the original data, and that this effect grows stronger
with increasing numbers of subjects).

Distribution in subgroups

In the practical of Associations between two Categorical Variables the relationship between
birthweight and hypertension was analysed by grouping the values of birthweight into two
categories, low and normal, and studying the percentage of low birthweight babies for each
category of hypertension. Here, we shall work with the original birthweight measurements and
start by looking at the mean birthweight for each sex. These means can be obtained by typing

sort sex
tab sex, summarize(bweight)

1. What are your tentative conclusions?

Generally speaking, it only makes sense to compare means for distributions which are roughly
normal, or at least symmetric, with the same spread of values (that is, approximately the same
SD). How far “roughly normal” matters depends on the number of subjects, because of the central
limit theorem). The distributions in the two groups are quite similar to the overall distribution (see
above), and can be inspected using the commands

2
histogram bweight, by(sex) normal percent bin(12)
graph box bweight, by(sex)

2. Do you think the distributions are roughly normal with a similar spread of values? Are the
sample sizes still reasonably large in the two groups?

Comparing means with a given value and comparing two groups

We already learn how to use of the z-test and the t-test for getting the statistical significance of the
difference between two separate groups of subjects. Both assume the distribution of the response
variable is roughly normal in each of the groups being compared, and we have looked at this. The
t-test also assumes that the standard deviation of the response variable is the same in each group,
and takes account of the fact that its value must be estimated, while the z-test allows for different
standard deviations but assumes them to be known. The t-test was recommended for small
samples where the additional uncertainty due to estimating the common standard deviation in
groups could be appreciable.

In Stata, there is only one command for these tests, ttest. When one is comparing two groups, it
has options that allow one to let the standard deviations on the two groups be different, or to
require them to be the same. In e ach case, Stata uses the same formulae as shown to calculate the
standard error of the difference of the means. It always uses the t-distribution to go from the test
statistic

difference between the means

SE of the difference

to the p-value. 1 Recall that when the number of subjects, and therefore the degrees of freedom,
are large enough, p-values under the normal and the t-distributions are virtually the same.

We start with a one-sample test. This is a test of the hypothesis that the true mean of a variable in
a data set is equal to a selected value. The command

ttest bweight=3300

will test whether the true mean birthweight of BAB babies is equal to 3300g, the England and
Wales national average for normal births.

The output from this command can at first be rather puzzling. At the end is a section that reads

Ho: mean(bweight) = 3300

Ha: mean < 3300 Ha: mean != 3300 Ha: mean > 3300
t = -6.6269 t = -6.6269 t = -6.6269
P < t = 0.0000 P > |t| = 0.0000 P > t = 1.0000

1
When the standard deviations are allowed to be different, there a special formula to adjust the degrees of
freedom which ensures the test is appropriate in this situation

3
The null hypothesis is that the mean (in the population) is 3300. The p-value needed is the one
corresponding to the alternative that they are different in either direction; this is the central one in
the second section of the text above, and is given as P=0.0000 2 , though we would write this as
P<0.0001

3. What do you conclude from this test?

A two-sample test is a test of the null hypothesis that the true mean outcome is the same in each of
two groups. To test whether the mean of the birthweight of babies is the same for boys and girls,
type

ttest bweight, by(sex)

4. What do you conclude about the difference in birthweight between boys and girls?

This version of the test assumes that the standard deviations of the response in the two samples are
the same “in the population” (or roughly the same in the data). This assumption can be tested
statistically by typing

sdtest bweight, by(sex)

If the assumption of equal standard deviations is considered inappropriate, use this version of the
command

ttest bweight, by(sex) unequal welch

and this allows for unequal standard deviations in calculating the standard error of the difference.3
(An alternative is to carry out a non-parametric or distribution-free test; see topic 23).

5. Was it reasonable to use the t-test with equal; standard deviations to compare birthweights
between boys and girls?

The analysis of variance for two groups

Try this command:

oneway bweight sex, tabulate

It produces the following output:

Sex of | Summary of Birthweight

infant | Mean Std. Dev. Freq.
------------+------------------------------------
male | 3211.2791 665.97976 326
female | 3044.127 628.66035 315
------------+------------------------------------
2
The other alternatives relate to one-tailed tests, and we generally do not recommended these.
3
The extra option Welch asks for a particular way of making the degrees of freedom adjustment. If you
leave it out, you get another one. It probably makes little difference which you use!

4
 Total | 3129.1373 652.78265 641

Analysis of Variance
Source SS df MS F Prob > F
------------------------------------------------------------------------
Between groups 4476041.4 1 4476041.4 10.66 0.0012
Within groups 268244081 639 419787.293
------------------------------------------------------------------------
Total 272720122 640 426125.19

Bartlett's test for equal variances: chi2(1) = 1.0593 Prob>chi2 = 0.303

This output gives almost all the results of the analysis we have done using ttest and sdtest. But
some of it is presented in a rather different way.

The first part of the output is not, however, particularly different. It shows the mean birthweight
for males and female babies, with standard deviations and numbers. This corresponds to the
results from the earlier tabulations of means.

The first part does not appear unless you type the option tabulate. The second part always appears
and needs more explanation; it is the analysis of variance itself.

Look first at the extreme right hand side. This shows a p-value, which is the same as that fro the t-
test earlier. It is the test of the same null hypothesis, that the male and female birthweights are the
same in the population. It is based on the F value and this is related to t; see below.

Moving to the left, there is a column headed MS. This stands for Mean Square. Look at the
“within groups” mean square. The values of this is 419787.293. The square root of this is 647.9.
This is the within group standard deviation, written as σ in the notes to topic 6, where the t-test
was introduced. In the notes to that session, σ is given by this formula:

√ ( n1 − 1 ) s21 + ( n2 − 1) s22
( n 1 + n 2 − 2)

The Mean Square within groups is thus the squared within-group standard deviation, which is also
called the within-group variance. It is measure of how far the individual babies’ birthweights
deviate from the mean male birthweight for boy babies, and how far they deviate from the mean
female birthweight for girl babies.

The male and female mean birthweights, although also vary from the mean birthweight for all the
babies together, both girls and boys. The measure of this variation is given by the between groups
mean square, which is 4476041.4 . It is a squared deviation, although it is based on only two
values, or a variance, in the same way as the within group mean square. It is specially calculated
to be comparable, as follows.

Suppose that there is no true difference between boy and girl mean birthweights. If this is true,
then the girl and boy means in our data will be different only because sample means vary about
underlying populations means. If that is true, then the between-sexes (between-group) mean
square will on average be the same as the within group mean square and any tendency to be larger
will be due to chance. This is the situation under the null hypothesis.

5
The measure used to asses if the between group mean square is statistically significantly greater
then the within group mean square is their ratio. This is called F. Under the null hypothesis, (and
assuming normally distributed data) is has a theoretical basis from which the p-value in the last
column can be calculated.

(The first column, SS, gives quantities called Sums of Squares and need concern us. These are
needed to get to the Mean Squares. The second column gives degrees of freedom. Between groups,
this is one because on is comparing two groups. Within groups, it is the same as the degrees of
freedom of the t-test).

The square root of F is the square of T from the t-test, 3.26.

The last line of the output is a statistical test of equality of the standard deviations in the two
groups. As such, it does the same job as the earlier command sdtest although in a different way. It
leads to the same conclusion, of no evidence of any difference. This is assumed for the F-test to
be viable.

Analysis of variance for several groups

The above analysis of variance has produced the same conclusion as the t-test. You may well ask
why go to all this extra detail, when the t-test does the same thing? For just 2 groups, this is so.
But suppose that we have more than two groups to compare. For binary outcomes, we met this
situation in Topic __, when the 2xc chi-squared test was presented. For a continuous outcome, as
here, we can extend the one-way analysis of variance.

For instance, the variable matagegp which we created in the practical of Introduction to Computer
Analysis has four levels. Suppose we wanted to compare, and test statistically, the mean
birthweights in these four groups. We could use t-tests to compare each group with each other
group (1 with 2, 1 with 3, 1 with 4, 2 with 3, 2 with 4, 3 with 4) but this would mean performing
six tests and this rapidly becomes cumbersome as the number of groups increases. Another
problem is that the results from large numbers of tests become difficult to interpret, and are not
independent of each other.

The one-way analysis of variance allows us to test the overall null hypothesis that the means in
each group are equal. For example, to compare the mean birthweight by maternal age group, type

oneway bweight matagegp, tabulate

The output is similar to the one we saw with two groups, male and female. There are now frou
means with standard deviations, of course. The analysis of variance now shows three degrees of
freedom, for the comparisons between four groups.

What is the basis of the null hypothesis in testing the comparison of means between more than
two groups? It is that the outcome differs between the groups no more than would be expected by
chance.

6. What is the null hypothesis being tested in the analysis of variance, and what is the result of the
test?

Analysis of Variance in other situations

6
The two examples above are both of one-way analyses of variance (hence the command oneway).
They compare two or more groups.

It is also possible use analysis of variance to investigate an outcome in relation to two different
factors; an example would be to investigate birthweight in relation to the child’s sex and the
gestational age in the same analysis. This leads to a two-way analysis of variance.

One-way analysis of variance also features in regression analysis, which is a topic later in this
course.

There are further more complex situations where analysis of variance is used. However, both
two-way analysis and these other applications are beyond you scope in this course.