Hypothesis Testing
Hypothesis Testing
Hypothesis Testing
By: Amare M
amaremu7@gmail.com
January 2024
Dessie, Ethiopia
Objectives
• Define hypothesis testing
Hypothesis Testing 2
Introduction to hypothesis testing
• The second area of statistical inference is hypothesis testing
• In hypothesis testing, a specific statement or hypothesis is
generated about a population parameter, and sample
statistics are used to assess the likelihood that the hypothesis
is true
• This statement or hypothesis is based on available
information and the investigator’s belief about the parameter
Hypothesis Testing 3
Introduction to hypothesis testing …
• The process of hypothesis testing involves setting up two
competing hypotheses:
o One reflects no difference, no association, or no effect
(called the null hypothesis) and
o The other reflects the investigator’s belief (called the
research or alternative hypothesis)
• We select a random sample (or multiple samples when
there are more comparison groups) and generate summary
statistics
Hypothesis Testing 4
Introduction to hypothesis testing …
Hypothesis Testing 5
The statistical ideas used in the tests of hypothesis
share the same routs with those used in
confidence intervals
Hypothesis testing is a way of organizing and
presenting evidence that helps us reach a decision
Although a confidence interval and tests of
hypothesis can be used to reach the same
conclusion; their emphasis are different
Hypothesis Testing 6
Introduction to hypothesis testing …
• The techniques for Hypothesis testing depends on;
o The appropriate classification of the outcome/dependent variable
(the key study variable) as continuous or dichotomous
o The number of comparison groups in the investigation
o Eg. Two comparison groups
Independent Men Vs Women or participants
assigned to receive a new drug or placebo in a clinical
trial
Dependent (matched/paired ) pre- and post-
assessment on the same participants.
• These issues dictate the appropriate hypothesis testing technique
Hypothesis Testing 7
Remember on estimation!!!
Techniques for one and two samples
• We discussed estimation for a specific parameter
• (e.g., the mean µ or proportion P of a population),
• for differences
• for ratios
• [(e.g., the relative risk RR = p1/p2 and
Hypothesis Testing 8
Introduction to hypothesis testing …
• In hypothesis testing
• One sample , two sample sand more than two samples
• In estimation
• One sample or two samples
Hypothesis Testing 9
Hypothesis testing techniques
Number of samples Outcome variable Test statistic
(groups)
Estimation 10
The level of significance
• The level of significance denoted α
• The level of significance (α) is the probability that we reject the null
hypothesis (in favor of the alternative) when it is actually true
Hypothesis Testing 11
Decision rule for the test
• Reject H0 if the calculated value is greater
than the tabulated value
• Reject H0 if the p.value is <0.05
Hypothesis Testing 12
Decision rule for the test …
Hypothesis Testing 13
Decision rule for the test …
Upper-tailed test (right tailed)
• The decision rule has investigators reject Ho if the test statistic
is greater than or equal to the critical value
Lower-tailed test (left tailed)
• The decision rule has investigators reject Ho if the test statistic
is less than or equal to the critical value
Two-tailed test
• The decision rule has investigators reject Ho if the test statistic
is extreme-either greater than or equal to an upper critical
value or less than or equal to an lower critical value
Hypothesis Testing 14
Test statistic
• If the test statistic follows the standard normal distribution (z),
then the decision rule is based on the standard normal
distribution
• If the test statistic follows the t distribution, then the decision
rule is based on the t distribution
• The appropriate critical value is selected from the t
distribution again depending on the specific alternative
hypothesis and the level of significance
Hypothesis Testing 15
Test statistic …
Hypothesis Testing 16
Steps in the hypothesis testing
1. Identify the null hypothesis H0 and the alternate hypothesis
HA .
2. Choose α. The value should be small, usually less than 10%.
It is important to consider the consequences of both types
of errors.
3. Select the test statistic and determine its value from the
sample data. This value is called the observed value of the
test statistic.
4. Compare the observed value of the statistic to the critical
value obtained for the chosen α.
5. Make a decision
Hypothesis Testing 17
Test with one sample, continuous outcome
• Hypothesis testing applications with a continuous outcome
variable in a single population are performed according to the
five-step procedure outlined earlier.
• A key component is setting up the null and research hypotheses.
• The objective is to compare the mean in a single population (µ) to a
known mean(µ0).
• The known value is generally derived from another study or
report—for example, a study in a similar but not identical
population or a study performed some years ago.
• The latter is called a historical control (“control” here referring to
the fact that the historical study is the comparator).
• It is important in setting up the hypotheses in a one-sample test
that the mean specified in the null hypothesis is a fair and
reasonable comparator
Hypothesis Testing 18
Test with one sample, continuous outcome…
• In one-sample tests for a continuous outcome, we set up our
hypotheses against an appropriate comparator.
• We select a sample and compute descriptive statistics
• Specifically, we compute the sample size (n), the sample
mean , and the sample standard deviation (s).
• We then determine the appropriate test statistic for the
hypothesis test.
• The formulas for test statistics depend on the sample size and
are given in Table below.
• Appropriate use of the t distribution assumes that the
outcome of interest is approximately normally distribute
Hypothesis Testing 19
Test with one sample, continuous outcome…
• Test statistics for testing H0 : µ=µ0
Hypothesis Testing 20
Test with one sample, continuous outcome…
• Example: The National Center for Health Statistics report indicated that in
2002, Americans paid an average of $3302 per year on health care and
prescription drugs.
• An investigator hypothesizes that in 2005, expenditures are lower
primarily due to the availability of generic drugs.
• To test the hypothesis, a sample of 100 Americans is selected and their
expenditures on health care and prescription drugs in 2005 are measured.
• The sample data are summarized as follows: n=100, X bar= $3190,
and s=$890.
• Is there statistical evidence of a reduction in expenditures on health care
and prescription drugs in 2005?
• Is the sample mean of $3190 evidence of a true reduction in the mean or
is it within chance fluctuation? We run the test using the five-step
approach
Hypothesis Testing 21
Test with one sample, continuous outcome…
• Step 1: set up the hypotheses and determine the level of significance
Hypothesis Testing 22
Test with one sample, continuous outcome…
• Step 4: Compute the test statistic
We now substitute the sample data into the formula for the test statistic
identified in Step 2:
• Step 5: Conclusion
We do not reject H0 because -1.26 > -1.645.
We do not have statistically significant evidence at α=0.05 to show that the
mean expenditures on health care and prescription drugs are lower in 2005
than the mean of $3302 reported in 2002.
Hypothesis Testing 23
Test with one sample, continuous outcome…
• Recall that when we fail to reject H0 in a test of hypothesis
that either the null hypothesis is true (the mean expenditures
in 2005 are the same as those in 2002 and equal to $3302) or
we are committing a Type II error (we fail to reject H0 when
in fact it is false).
• In summarizing this test, we conclude that we do not have
sufficient evidence to reject H0.
• We do not conclude that H0 is true because there may be a
moderate to high probability that we are committing a Type II
error.
• It is possible that the sample size is not large enough to detect
a difference in mean expenditures
Hypothesis Testing 24
Test with one sample, continuous outcome…
Assumptions of one sample test
• Independent of observation
• Continuous outcome
Hypothesis Testing 25
Test with one sample, dichotomous outcome
• Hypothesis testing applications with a dichotomous outcome
variable in a single population are also performed according
to the five-step procedure.
• Similar to tests for means described in test with one sample
continuous outcome; a key component is setting up the
null and research hypotheses.
• The objective is to compare the proportion of successes in a
single population to a known proportion (P0).
• That known proportion is generally derived from another
study or report and is sometimes called a historical control.
• It is important in setting up the hypotheses in a one sample
test that the proportion specified in the null hypothesis is a
fair and reasonable comparator.
Hypothesis Testing 26
Test with one sample, dichotomous outcome…
• In one-sample tests for a dichotomous outcome, we set up
our hypotheses against an appropriate comparator.
• We select a sample and compute descriptive statistics on the
sample data using the techniques described in elsewhere .
• Specifically, we compute the sample size (n) and the sample
proportion (pˆ), which is computed by taking the ratio of the
number of successes to the sample size, pˆ = x/n.
• We then determine the appropriate test statistic (Step 2) for
the hypothesis test.
• The formula for the test statistic is given in Table below.
Hypothesis Testing 27
Test with one sample, dichotomous outcome…
Hypothesis Testing 28
Test with one sample, dichotomous outcome…
• The preceding formula is appropriate for large samples,
defined when the smaller of nP0 and n(1-P0) is at least 5.
• This is similar, but not identical, to the condition required for
appropriate use of the confidence interval formula for a
population proportion specified previously—i.e., min[npˆ,
n(1-pˆ)]≥5.
• Here we use the proportion specified in the null
hypothesis (p0) as the true proportion of successes rather
than the sample proportion (pˆ).
• If we fail to satisfy the condition, then alternative procedures
called exact methods must be used to test the hypothesis
about the population proportion.
Hypothesis Testing 29
Test with one sample, dichotomous outcome…
• Example: The NCHS report indicated that in 2002 the
prevalence of cigarette smoking among American adults was
21.1%. Data on prevalent smoking in n=3536 participants who
attended the seventh examination of the offspring in the
Framingham Heart Study. Suppose we want to assess whether
the prevalence of smoking is lower in the Framingham
offspring sample due to the focus on cardiovascular health in
that community. Data from the Framingham Offspring Study
were summarized in elsewhere and indicated that 482/
3536=13.6% of the respondents were currently smoking at
the time of the exam. Is there evidence of a statistically lower
prevalence of smoking in the Framingham Offspring Study as
compared to the prevalence among all Americans?
Hypothesis Testing 30
Test with one sample, dichotomous outcome…
• Step 1: set up the hypotheses and determine the level of
significance
Hypothesis Testing 31
Test with one sample, dichotomous outcome…
• Step 3: Set up the decision rule
This is a lower-tailed test, using a z statistic and a 5% level of significance.
The appropriate critical value can be found in z-Table and the decision rule is
• Step 5: Conclusion
We reject H0 because -10.93 < -1.645.
We have statistically significant evidence at α=0.05 to show that the prevalence of smoking in
the Framingham Offspring is lower than the national prevalence (21.1%). The p-value is
<0.0001
Hypothesis Testing 32
Tests with two independent samples, continuous outcome
Hypothesis Testing 33
Tests with two independent samples, continuous outcome…
Hypothesis Testing 34
Tests with two independent samples, continuous outcome…
Hypothesis Testing 35
Tests with two independent samples, continuous outcome…
Hypothesis Testing 36
Tests with two independent samples, continuous outcome…
Hypothesis Testing 37
Tests with two independent samples, continuous outcome…
Hypothesis Testing 38
Tests with two independent samples, continuous outcome…
Hypothesis Testing 39
Tests with two independent samples, continuous outcome…
Hypothesis Testing 40
Tests with two independent samples, continuous outcome…
Hypothesis Testing 41
Tests with two independent samples, continuous outcome…
Hypothesis Testing 42
Tests with two independent samples, continuous outcome…
Hypothesis Testing 43
Tests with two independent samples, continuous outcome…
Hypothesis Testing 44
Tests with two independent samples, continuous outcome…
• Step 5: Conclusion
We reject H0 because 2.66 > 1.960.
We have statistically significant evidence at a=0.05 to show that
there is a difference in mean systolic blood pressures between
men and women. The p-value can be found in z-table and is equal to p ,
0.010.
Hypothesis Testing 45
Tests with two independent samples, continuous outcome…
Hypothesis Testing 46
Tests with two independent samples, continuous outcome…
Hypothesis Testing 47
Tests with two independent samples, continuous outcome…
Hypothesis Testing 48
Tests with two independent samples, continuous outcome…
Hypothesis Testing 49
Tests with two independent samples, continuous outcome…
Hypothesis Testing 50
Tests with matched samples, continuous outcome
• An alternative study design to that described in the previous
section (to compare two groups with respect to their mean
scores on a continuous outcome) is one based on matched or
paired samples.
• The two comparison groups are said to be dependent
(matched or paired) and the data can arise from a single
sample of participants where each participant is measured
twice, possibly before and after an intervention, or from two
samples that are matched or paired on one or more specific
characteristics (e.g., siblings).
• When the samples are dependent, we focus on difference
scores in each participant or between members of a pair, and
the test of hypothesis is based on the mean difference, µd.
• The null hypothesis again reflects “no difference” and is
stated as µd=0.
Hypothesis Testing 51
Tests with matched samples, continuous outcome…
• Note that there are some instances where it is of
interest to test whether there is a difference of a
particular magnitude (e.g., µd=5) but in most instances,
the null hypothesis reflects no difference (i.e., µd=0).
• The appropriate formula for the test of hypothesis
depends on the sample size.
• The formulas are shown in Table below and are identical
to those we presented for hypothesis testing with one
sample and a continuous outcome presented in tests
with one sample, continuous outcome Section, except
here we focus on difference scores.
Hypothesis Testing 52
Tests with matched samples, continuous outcome…
Hypothesis Testing 53
Tests with matched samples, continuous outcome…
Hypothesis Testing 54
Tests with matched samples, continuous outcome…
• Step 1: set up the hypotheses and determine the level
of significance
Hypothesis Testing 55
Tests with matched samples, continuous outcome…
• Step 3: Set up the decision rule.
This is a two-tailed test, using a t statistic and a 5% level of significance. The
appropriate critical value can be found in t-table with degrees of freedom, df,
defined as df=n-1=15-1=14. The critical value is 2.145 and the decision rule is
Hypothesis Testing 56
Tests with matched samples, continuous outcome…
• Step 5: Conclusion
We do not reject H0 because -2.145 < -1.60 < 2.145.
We do not have statistically significant evidence at a=0.05 to
show that there is a difference in systolic blood pressures over time.
In the CI estimation Example, we estimated a confidence interval and
were 95% confident that the mean difference in systolic blood
pressures between Examination 6 and Examination 7
(approximately 4 years apart) was between -12.4 and 1.8.
Because the null value of the confidence interval for the mean
difference is 0, we concluded that there was no statistically significant
difference in blood pressures over time because the confidence
interval for the mean difference included 0. The test of hypothesis
gives the same result.
Hypothesis Testing 57
Tests with matched samples, continuous outcome…
Hypothesis Testing 58
Tests with two independent samples, dichotomous outcome
Hypothesis Testing 59
Tests with two independent samples, dichotomous outcome
Hypothesis Testing 60
Tests with two independent samples, dichotomous outcome…
Hypothesis Testing 61
Tests with two independent samples, dichotomous outcome…
Hypothesis Testing 62
Tests with two independent samples, dichotomous outcome…
Hypothesis Testing 63
Tests with two independent samples, dichotomous outcome…
Hypothesis Testing 64
Tests with two independent samples, dichotomous outcome…
Hypothesis Testing 65
Tests with two independent samples, dichotomous outcome…
Hypothesis Testing 66
Tests with two independent samples, dichotomous outcome…
Hypothesis Testing 67
Tests with two independent samples, dichotomous outcome…
Estimation 68
Tests with two independent samples, dichotomous outcome…
Hypothesis Testing 69
Tests with two independent samples, dichotomous outcome…
Step 1: set up the hypotheses and determine the level of significance
Hypothesis Testing 70
Tests with two independent samples, dichotomous outcome…
Hypothesis Testing 71
Tests with two independent samples, dichotomous outcome…
• Step 5: Conclusion
We do not reject H0 because -1.960 < 0.927 <1.960.
We don't have statistically significant evidence at a=0.05 to show that there is
a difference in prevalent CVD between smokers and nonsmokers
Hypothesis Testing 72
Tests with two independent samples, dichotomous outcome…
Hypothesis Testing 73
Types of errors in hypothesis testing
• When we reject or accept the Ho, we commit errors.
• Two types of errors are committed.
– Type I Error (serious)
– Type II Error
Hypothesis Testing 74
• Type I error (α: false positive)---level of significance
Probability of incorrectly rejecting a true null hypothesis
• Type II error (β: false negative)
Probability of incorrectly failing to reject a false null
hypothesis
• Power (1-β: true positive)
Probability of correctly rejecting a false null hypothesis
• True negative (1-α)--- level of confidence
Probability of correctly failing to reject a true null
hypothesis
Hypothesis Testing 75
Hypothesis Testing 76
• You decide to get tested for COVID-19 based on mild symptoms. There are
two errors that could potentially occur:
• Type I error (false positive): the test result says you have coronavirus, but
you actually don’t.
• Type II error (false negative): the test result says you don’t have
coronavirus, but you actually do.
• Type I error =p(rejecting Ho/Ho is true) = α
• p(fail to reject Ho/Ho is True)= Confidence level/size of the test = (1-α)
correct decision
• Type II error= p(rejecting Ho/Ho is true) Power=(1-β)= P(rejecting Ho/Ho
false) correct decision
Hypothesis Testing 77
Thanks!
Introduction 78