Interval estimation: Part 2

(Module 4)
Statistics (MAST20005) & Elements of Statistics (MAST90058)
Semester 2, 2018

Contents
1 Confidence intervals 1
1.1 Less common scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 General techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 Choice of confidence level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.5 Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Prediction intervals 7

3 Sample size determination 8

Aims of this module

• Explain some less common scenarios where confidence intervals are used
• Describe some general aspects of confidence intervals
• Introduce prediction intervals, an interval estimator in the context of predicting a future value
• Explain how to calculate the sample size required for a study

1 Confidence intervals

1.1 Less common scenarios

Less common scenarios: overview

• One-sided CIs
• CIs based on discrete statistics

One-sided confidence intervals

We can construct one-sided confidence intervals, e.g. just an upper or lower bound.
For example, if we sample from N(µ, σ 2 ) with known σ:
 
X̄ − µ
Pr √ <c =1−α
σ/ n

where c = Φ−1 (1 − α). Rearranging gives

 
σ
Pr X̄ − c √ < µ = 1 − α
n

1
and therefore a one-sided 100 · (1 − α)% confidence interval for µ is
 
σ
x̄ − c √ ,∞ .
n

Remarks
• The main thing to remember is to start with a one-sided probability statement about the pivot.
• In this example, we obtained a lower bound.
• To get an upper bound, start with an inequality in the other direction.
• Other scenarios are analogous. For example, if σ is unknown then replace σ with s and let c be a quantile from
tn−1 rather than N (0, 1).
• Since we only need one tail probability, we don’t need to separate α into two parts. That’s why we use the 1 − α
quantile here rather than the 1 − α/2 quantile.

Example (one-sided interval)

A winemaker requires a minimum concentration of 10 g/L of sugar in the grapes used to make a certain wine. In
a sample of 30 units she finds an average concentration of 11.9 g/L and a standard deviation of 0.96. Is that high
enough?
She calculates a 95% lower bound (one-sided CI) as follows:
s 0.96
x̄ − c √ = 11.9 − 1.699 × √ = 11.60
n 30
where c = 1.699 is the 0.95 quantile from t29 .
On that basis, she is confident that the average sugar content is adequately high.

Example using R

Recall the butterfat example from the previous module. Now re-doing using one-sided CIs. . .
> t.test(butterfat,
+ conf.level = 0.90,
+ alternative = "less")

...

90 percent confidence interval:

-Inf 534.146
> t.test(butterfat,
+ conf.level = 0.90,
+ alternative = "greater")

...

90 percent confidence interval:

480.854 Inf

Confidence intervals based on discrete statistics*

Our starting point has been probability intervals like:

Pr (a(θ) < T < b(θ)) = 0.95

What if T is discrete? For example, T ∼ Bi(n, θ)

Limitation: a() and b() can only take specific (discrete) values.
⇒ Cannot guarantee an exact probability (confidence level).

2
⇒ Inversion is messy.
Usually aim for something close, with ‘at least’ probability. For example,

Pr (a(θ) 6 T 6 b(θ)) > 0.95

where:
• a(θ) is the largest value of x such that Pr(x 6 T | θ) > 0.975
• b(θ) is the smallest value of x such that Pr(T 6 x | θ) > 0.975
How do we invert these?
For an observed value tobs (of T ), we have:
• c is such that Pr(tobs 6 T | θ = c) = 0.025
• d is such that Pr(T 6 tobs | θ = d) = 0.025
Then, the ‘at least’ 95% confidence interval is (c, d).

1.2 General techniques

CIs from MLEs

Maximum likelihood estimators have many convenient properties. We will cover some of the theory later in the
semester. For now, it is useful to know the following. . .
Let,
∂ 2 ln L
V (θ) = −
∂θ2
This is known as the observed information function. It can be used to estimate the standard deviation of the MLE:
1
se(θ̂) = q
V (θ̂)

Moreover, the MLE is asymptotically unbiased and asymptotically normally distributed.

Therefore, for large sample sizes, we can construct approximate CIs using:
c
θ̂ ± q
V (θ̂)

where c = Φ−1 (1 − α/2).

Example (approximate CI from MLE)

Sampling (iid) from: X ∼ Exp(θ). Previously we found that θ̂ = X̄ and

P
∂ ln L n xi
=− + 2 .
∂θ θ θ
Differentiate once more,
∂ 2 ln L
P
n 2 xi
= 2−
∂θ2 θ θ3
and so we have,
 P − 12
n 2 xi
se(θ̂) = − +
θ̂2 θ̂3
and an approximate 95% confidence interval is given by θ̂ ± 1.96 se(θ̂).

3
Review of general methods for constructing CIs

Methods:
• Invert a probability interval based on a known sampling distribution (use a pivot)
• Use the asymptotic MLE result
Common approximations:
• Normality (based on the CLT or the asymptotic MLE)
• Substitute parameter estimates into the expression for the standard deviation of the estimator

1.3 Properties

CIs are random intervals

Recall: the CI estimator is a random interval.

A CI consists of two statistics: the lower bound and the upper bound of the interval. They both have sampling
distributions.
The random elements are therefore the endpoints, not the parameter:

Pr(L < θ < U) = 0.95

Contrast this with a probability statement for a statistic:

Pr(l < T < u) = 0.95

Coverage

The coverage or coverage probability of a confidence interval (estimator) is the probability it contains the true value of
the parameter,
C = Pr(L < θ < U )

Usually this is equal to the confidence level, which is also known as the nominal coverage probability.

4
However, due to various approximations we use, the actual coverage achieved may vary from the confidence level.

Example: Bernoulli sampling, n = 25, Quadratic approximation CI

1.00
0.95
Coverage

0.90
0.85

0.0 0.2 0.4 0.6 0.8 1.0

More detail about the quadratic approximation will be shown in the tutorials and lab classes.

1.4 Choice of confidence level

Choice of confidence level

This is somewhat arbitrary.

If very high:
• More likely to capture the true value.
• Impractically wide: won’t act as a useful guide for showing plausible values based on the data.
• It will place too much emphasis on tails of the sampling distribution, which aren’t actually all that likely.
If very low:
• More ‘useful’ in the sense of being more selective about the possible values of the parameter.
• This comes at the expense of the loss of ‘confidence’, i.e. not as certain about whether the true value is captured
inside the interval.

Choice of confidence level: some guidelines

• 95% is a very common convention. If you follow this, it will rarely be questioned. Others may be expecting
this, so always be clear if you deviate from it.
• 90% can also be a reasonable choice.
• 50% is sometimes useful, due to easy interpretation. A good use case: plotting a large number of overlapping
intervals, to reduce visual clutter.
• The choice can vary by application, and you may even use different choices for the same problem (e.g. 50%
for a particular plot, but 95% when reporting a headline result in text).
• Whatever you choose, remember that the true value is never guaranteed to be inside the interval. There is
always a chance it will be outside.

5
1.5 Interpretation

Explaining CIs

The probability associated with a CI (i.e. the confidence level) relates to the sampling procedure. In particular, it
refers to hypothetical repeated samples.
Once a specific sample is observed and a CI is calculated, the confidence level cannot be interpreted probabilistically
in the context of the specific data at hand.
It is incorrect to say things like:
• This CI has a 95% chance of including the true value
• We can be ‘95% confident’ that this CI includes the true value
Don’t do it!
The probability only has a meaning when considering potential replications of the whole sampling and estimation
procedure.
We can only say something like:
• If we were to repeat this experiment, then 95% of the time the CI we calculate will cover the true value.
(This is a bit of a mouthful...)
In practice:
• If you are reporting results to people who know what they are, you can just state that the “95% confidence
interval is. . . ”
• If people want to know what this means, use an intuitive notion like, “it is the set of plausible values of the
parameter that are consistent with the data”. (Note: this is not actually true in general, but will be accurate
enough for all of the examples we cover this semester.)
• If you need to actually explain what a CI is precisely, you need to explain it in terms of repeated sampling. (No
shortcuts!)

Communicating results: general tips

• Describe the extent of your uncertainty
• Emphasise a range a plausible values
• Phrase results in terms of the degree of evidence (e.g. ‘strong/modest/weak evidence of. . . ’)

1.6 Summary

Confidence intervals: summary

• Interval estimates are the most common way to quantify uncertainty.
• Confidence intervals are the most common type of interval estimate.
• Confidence intervals are straightforward to construct if we know or can approximate the sampling distribution
of the statistic and can construct a pivot.
• We have looked at some well known (and widely used) examples for means, variances and proportions.
• We can derive CIs, whether exact or approximate, for a variety of scenarios, and have techniques for constructing
them in general.
• 95% CIs are the most common convention.

6
2 Prediction intervals
Prediction intervals

Suppose we want to estimate the value of a future observation, rather than a parameter of the distribution. We usually
call this prediction rather than ‘estimation’.
We have available data that arose from the same probability distribution. Can we use this to come up with an interval
estimate?
Yes. Easiest to see with an example. . .

Example (prediction interval)

Random sample (iid): X1 , . . . , Xn on X ∼ N(µ, 1)

Let X ∗ be a future observation on X, independent of those currently observed.
By independence, we have:
1
X̄ ∼ N(µ, )
n
X ∗ ∼ N(µ, 1)
1
X̄ − X ∗ ∼ N(0, 1 + )
n

Therefore we can write, r r !

1 ∗ 1
Pr −1.96 1 + < X̄ − X < 1.96 1 + = 0.95
n n
r r !
1 ∗ 1
Pr X̄ − 1.96 1 + < X < X̄ + 1.96 1 + = 0.95
n n

From this we get a 95% prediction interval for X,

r
1
x̄ ± 1.96 1+
n
Compare this with the 95% confidence interval for µ,
r
1
x̄ ± 1.96
n

Remarks
• The prediction interval for X is much wider than the confidence interval for µ.
• As n → ∞, the width of the confidence interval shrinks to zero, but the width of the prediction interval tends
to the width of the corresponding population probability interval (µ ± 1.96).
• This makes sense: we get complete certainty about µ, but each observation on X has inherent variability (in
this case, a variance of 1).
• In the prediction interval estimator, all quantities are random variables:

Pr(L < X < U) = 0.95

7
3 Sample size determination
Sample size determination: overview

We are planning a study. How much data do we need?

It depends on how much precision is required, often measured by the desired width of a confidence interval.
We will go through two estimation scenarios:
• Means
• Proportions

Example: sample size for means

Random sample (iid): X1 , . . . , Xn ∼ N(µ, 152 )

Want a 95% confidence interval of width 2 (i.e. x̄ ± 1).
The confidence interval will be given by x̄ ± 1.96 √15n .
So we need,
15
1.96 √ = 1
n
which gives √
n = 29.4, or n ≈ 864.36
and so for our study we need sample size of at least 865.

Sample size for means

Confidence interval of the form:

σ
x̄ ± c √ = x̄ ± 
n
where c = Φ−1 (1 − α/2).
For a prespecified , we have:
σ  cσ 2
 = c√ , or n =
n 

Example 2: sample size for means

A researcher plans to select a sample of first-grade girls in order to estimate their mean height µ. The sample is
required to be large enough to get an estimate to within 0.5 cm. From previous studies we know σ ≈ 2.8 cm.
 cσ 2  1.96 × 2.8 2
n= = = 120.47
 0.5
The researcher selects 121 girls.

Sample size for proportions

Confidence interval is of the form: r

p̂(1 − p̂)
p̂ ± c
n
−1
where c = Φ (1 − α/2). In order to have p̂ ±  for a given , we need:
r
p̂(1 − p̂) c2 p̂(1 − p̂)
=c , or n =
n 2
Can use a preliminary estimate of p̂ if this is available. Otherwise, note that p̂(1 − p̂) 6 14 , which means we can use
n = c2 /(42 ) as a conservative choice.

8
Example: Sample size for proportions

The unemployment rate has been 8% for a while. A researcher wishes to take new sample to estimate it and wants to
be ‘very certain’, by using a 99% CI, that the new estimate is within 0.001 of true proportion.

c2 p̂(1 − p̂) 2.5762 × 0.08 × (1 − 0.08)

n= = ≈ 488394
2 0.0012
At this stage the researcher panics and says they don’t really need to be that sure!
Try again. . . a 98% CI and a difference of 0.01 gives n = 3982, which is more practical, although possibly still a bit
large.