Hypothesis Testing

Download as pdf or txt
Download as pdf or txt
You are on page 1of 78

Hypothesis testing

By: Amare M
amaremu7@gmail.com

January 2024
Dessie, Ethiopia
Objectives
• Define hypothesis testing

• Appreciate the concepts of null hypothesis and alternative


hypothesis

• Differentiate between type I error and type II error

• Test hypothesis for single mean or proportion and double


means or proportions

• Explain the meaning and application of statistical significance

Hypothesis Testing 2
Introduction to hypothesis testing
• The second area of statistical inference is hypothesis testing
• In hypothesis testing, a specific statement or hypothesis is
generated about a population parameter, and sample
statistics are used to assess the likelihood that the hypothesis
is true
• This statement or hypothesis is based on available
information and the investigator’s belief about the parameter

Hypothesis Testing 3
Introduction to hypothesis testing …
• The process of hypothesis testing involves setting up two
competing hypotheses:
o One reflects no difference, no association, or no effect
(called the null hypothesis) and
o The other reflects the investigator’s belief (called the
research or alternative hypothesis)
• We select a random sample (or multiple samples when
there are more comparison groups) and generate summary
statistics

Hypothesis Testing 4
Introduction to hypothesis testing …

• We then assess the likelihood that the sample data


support the research or alternative hypothesis

• Similar to estimation, the process of hypothesis


testing is based on probability theory and the Central
Limit Theorem

Hypothesis Testing 5
 The statistical ideas used in the tests of hypothesis
share the same routs with those used in
confidence intervals
 Hypothesis testing is a way of organizing and
presenting evidence that helps us reach a decision
 Although a confidence interval and tests of
hypothesis can be used to reach the same
conclusion; their emphasis are different
Hypothesis Testing 6
Introduction to hypothesis testing …
• The techniques for Hypothesis testing depends on;
o The appropriate classification of the outcome/dependent variable
(the key study variable) as continuous or dichotomous
o The number of comparison groups in the investigation
o Eg. Two comparison groups
 Independent Men Vs Women or participants
assigned to receive a new drug or placebo in a clinical
trial
 Dependent (matched/paired ) pre- and post-
assessment on the same participants.
• These issues dictate the appropriate hypothesis testing technique

Hypothesis Testing 7
Remember on estimation!!!
Techniques for one and two samples
• We discussed estimation for a specific parameter
• (e.g., the mean µ or proportion P of a population),

• for differences

• (e.g., difference in means µ1 – µ2, the risk difference p1 – p2) and

• for ratios
• [(e.g., the relative risk RR = p1/p2 and

• odds ratio OR = (p1 /(1 – p1))/(p2 /(1 – p2))]

Hypothesis Testing 8
Introduction to hypothesis testing …

• In hypothesis testing
• One sample , two sample sand more than two samples

• In estimation
• One sample or two samples

• Not intuitive when there are more than two groups

Hypothesis Testing 9
Hypothesis testing techniques
Number of samples Outcome variable Test statistic
(groups)

One sample Continuous One sample T-test

Two independent samples Continuous Independent two


sample T-test
Two dependent/matched/ paired samples Continuous Paired T-test
More than two independent samples Continuous ANOVA test

One sample Dichotomous Z-test

Two independent samples Dichotomous Z-test

More than two independent samples Dichotomous


One sample Categorical or ordinal (more
than 2 responses/categories)
More than two independent samples Categorical or ordinal

Estimation 10
The level of significance
• The level of significance denoted α
• The level of significance (α) is the probability that we reject the null
hypothesis (in favor of the alternative) when it is actually true

• The usual value for α is 0.05 or 5%


• The typical values for α are 0.01, 0.05, and 0.10 with α=0.05 the
most commonly used value
• α=false positive (Type I error)

Hypothesis Testing 11
Decision rule for the test
• Reject H0 if the calculated value is greater
than the tabulated value
• Reject H0 if the p.value is <0.05

Hypothesis Testing 12
Decision rule for the test …

• The decision rule for a specific test depends on three factors:

• The research hypothesis (upper-tailed, lower-tailed and


two-tailed test is proposed),

• The test statistic; z, t,X2,… and

• The level of significance (α); 0.01, 0.05 or 0.10

Hypothesis Testing 13
Decision rule for the test …
Upper-tailed test (right tailed)
• The decision rule has investigators reject Ho if the test statistic
is greater than or equal to the critical value
Lower-tailed test (left tailed)
• The decision rule has investigators reject Ho if the test statistic
is less than or equal to the critical value
Two-tailed test
• The decision rule has investigators reject Ho if the test statistic
is extreme-either greater than or equal to an upper critical
value or less than or equal to an lower critical value

Hypothesis Testing 14
Test statistic
• If the test statistic follows the standard normal distribution (z),
then the decision rule is based on the standard normal
distribution
• If the test statistic follows the t distribution, then the decision
rule is based on the t distribution
• The appropriate critical value is selected from the t
distribution again depending on the specific alternative
hypothesis and the level of significance

Hypothesis Testing 15
Test statistic …

Hypothesis Testing 16
Steps in the hypothesis testing
1. Identify the null hypothesis H0 and the alternate hypothesis
HA .
2. Choose α. The value should be small, usually less than 10%.
It is important to consider the consequences of both types
of errors.
3. Select the test statistic and determine its value from the
sample data. This value is called the observed value of the
test statistic.
4. Compare the observed value of the statistic to the critical
value obtained for the chosen α.
5. Make a decision

Hypothesis Testing 17
Test with one sample, continuous outcome
• Hypothesis testing applications with a continuous outcome
variable in a single population are performed according to the
five-step procedure outlined earlier.
• A key component is setting up the null and research hypotheses.
• The objective is to compare the mean in a single population (µ) to a
known mean(µ0).
• The known value is generally derived from another study or
report—for example, a study in a similar but not identical
population or a study performed some years ago.
• The latter is called a historical control (“control” here referring to
the fact that the historical study is the comparator).
• It is important in setting up the hypotheses in a one-sample test
that the mean specified in the null hypothesis is a fair and
reasonable comparator

Hypothesis Testing 18
Test with one sample, continuous outcome…
• In one-sample tests for a continuous outcome, we set up our
hypotheses against an appropriate comparator.
• We select a sample and compute descriptive statistics
• Specifically, we compute the sample size (n), the sample
mean , and the sample standard deviation (s).
• We then determine the appropriate test statistic for the
hypothesis test.
• The formulas for test statistics depend on the sample size and
are given in Table below.
• Appropriate use of the t distribution assumes that the
outcome of interest is approximately normally distribute

Hypothesis Testing 19
Test with one sample, continuous outcome…
• Test statistics for testing H0 : µ=µ0

Hypothesis Testing 20
Test with one sample, continuous outcome…
• Example: The National Center for Health Statistics report indicated that in
2002, Americans paid an average of $3302 per year on health care and
prescription drugs.
• An investigator hypothesizes that in 2005, expenditures are lower
primarily due to the availability of generic drugs.
• To test the hypothesis, a sample of 100 Americans is selected and their
expenditures on health care and prescription drugs in 2005 are measured.
• The sample data are summarized as follows: n=100, X bar= $3190,
and s=$890.
• Is there statistical evidence of a reduction in expenditures on health care
and prescription drugs in 2005?
• Is the sample mean of $3190 evidence of a true reduction in the mean or
is it within chance fluctuation? We run the test using the five-step
approach
Hypothesis Testing 21
Test with one sample, continuous outcome…
• Step 1: set up the hypotheses and determine the level of significance

The research hypothesis is that expenditures have decreased, and therefore a


lower-tailed test is used.
• Step 2: Select the appropriate test statistic
Because the sample size is large (n>30), the appropriate test statistic is

• Step 3: Set up the decision rule.


This is a lower-tailed test, using a z statistic and a 5% level of significance.
The appropriate critical value can be found in z-Table and the decision rule is

Hypothesis Testing 22
Test with one sample, continuous outcome…
• Step 4: Compute the test statistic
We now substitute the sample data into the formula for the test statistic
identified in Step 2:

• Step 5: Conclusion
We do not reject H0 because -1.26 > -1.645.
We do not have statistically significant evidence at α=0.05 to show that the
mean expenditures on health care and prescription drugs are lower in 2005
than the mean of $3302 reported in 2002.

Hypothesis Testing 23
Test with one sample, continuous outcome…
• Recall that when we fail to reject H0 in a test of hypothesis
that either the null hypothesis is true (the mean expenditures
in 2005 are the same as those in 2002 and equal to $3302) or
we are committing a Type II error (we fail to reject H0 when
in fact it is false).
• In summarizing this test, we conclude that we do not have
sufficient evidence to reject H0.
• We do not conclude that H0 is true because there may be a
moderate to high probability that we are committing a Type II
error.
• It is possible that the sample size is not large enough to detect
a difference in mean expenditures

Hypothesis Testing 24
Test with one sample, continuous outcome…
Assumptions of one sample test
• Independent of observation
• Continuous outcome

Hypothesis Testing 25
Test with one sample, dichotomous outcome
• Hypothesis testing applications with a dichotomous outcome
variable in a single population are also performed according
to the five-step procedure.
• Similar to tests for means described in test with one sample
continuous outcome; a key component is setting up the
null and research hypotheses.
• The objective is to compare the proportion of successes in a
single population to a known proportion (P0).
• That known proportion is generally derived from another
study or report and is sometimes called a historical control.
• It is important in setting up the hypotheses in a one sample
test that the proportion specified in the null hypothesis is a
fair and reasonable comparator.

Hypothesis Testing 26
Test with one sample, dichotomous outcome…
• In one-sample tests for a dichotomous outcome, we set up
our hypotheses against an appropriate comparator.
• We select a sample and compute descriptive statistics on the
sample data using the techniques described in elsewhere .
• Specifically, we compute the sample size (n) and the sample
proportion (pˆ), which is computed by taking the ratio of the
number of successes to the sample size, pˆ = x/n.
• We then determine the appropriate test statistic (Step 2) for
the hypothesis test.
• The formula for the test statistic is given in Table below.

Hypothesis Testing 27
Test with one sample, dichotomous outcome…

• Test statistics for testing H0 : P=P0

Hypothesis Testing 28
Test with one sample, dichotomous outcome…
• The preceding formula is appropriate for large samples,
defined when the smaller of nP0 and n(1-P0) is at least 5.
• This is similar, but not identical, to the condition required for
appropriate use of the confidence interval formula for a
population proportion specified previously—i.e., min[npˆ,
n(1-pˆ)]≥5.
• Here we use the proportion specified in the null
hypothesis (p0) as the true proportion of successes rather
than the sample proportion (pˆ).
• If we fail to satisfy the condition, then alternative procedures
called exact methods must be used to test the hypothesis
about the population proportion.

Hypothesis Testing 29
Test with one sample, dichotomous outcome…
• Example: The NCHS report indicated that in 2002 the
prevalence of cigarette smoking among American adults was
21.1%. Data on prevalent smoking in n=3536 participants who
attended the seventh examination of the offspring in the
Framingham Heart Study. Suppose we want to assess whether
the prevalence of smoking is lower in the Framingham
offspring sample due to the focus on cardiovascular health in
that community. Data from the Framingham Offspring Study
were summarized in elsewhere and indicated that 482/
3536=13.6% of the respondents were currently smoking at
the time of the exam. Is there evidence of a statistically lower
prevalence of smoking in the Framingham Offspring Study as
compared to the prevalence among all Americans?

Hypothesis Testing 30
Test with one sample, dichotomous outcome…
• Step 1: set up the hypotheses and determine the level of
significance

• Step 2: Select the appropriate test statistic


We must first check that the sample size is adequate.
Specifically, we need to check min[nP0, n(1-P0)]
=min[3536(0.211), 3536(1- 0.211)] = min(746, 2790) =746
The sample size is more than adequate so the following formula
can be used:

Hypothesis Testing 31
Test with one sample, dichotomous outcome…
• Step 3: Set up the decision rule
This is a lower-tailed test, using a z statistic and a 5% level of significance.
The appropriate critical value can be found in z-Table and the decision rule is

• Step 4: Compute the test statistic


We now substitute the sample data into the formula for the test statistic identified in Step 2:

• Step 5: Conclusion
We reject H0 because -10.93 < -1.645.
We have statistically significant evidence at α=0.05 to show that the prevalence of smoking in
the Framingham Offspring is lower than the national prevalence (21.1%). The p-value is
<0.0001

Hypothesis Testing 32
Tests with two independent samples, continuous outcome

• There are many applications where it is of interest to compare


two independent groups with respect to their mean scores
on a continuous outcome.
• In Estimation session, we presented techniques to estimate
the difference in means.
• Here we again compare means between groups, but rather
than generating an estimate of the difference, we test
whether the observed difference (increase, decrease, or
difference) is statistically significant or not

Hypothesis Testing 33
Tests with two independent samples, continuous outcome…

• In this section, we discuss the comparison of means when the


two comparison groups are independent or physically
separate.
• The two groups might be determined by a particular attribute
(e.g., sex, history of cardiovascular disease) or might be set up
by the investigator (e.g., participants assigned to receive an
experimental drug or placebo).
• The first step in the analysis involves computing descriptive
statistics on each of the two samples using the
techniques described elsewhere previously.
• Specifically, we compute the sample size, mean, and standard
deviation in each sample and we denote these summary
statistics as follows:

Hypothesis Testing 34
Tests with two independent samples, continuous outcome…

• The designation of Sample 1 and Sample 2 is essentially


arbitrary.
• In a clinical trial setting, the convention is to call the
treatment Group 1 and the control Group 2.
• However, when comparing men and women, either group can
be 1 or 2.
• In the two independent samples application with a
continuous outcome, the parameter of interest in the test of
hypothesis is the difference in population means, µ1-µ2
• The null hypothesis is always that there is no difference
between groups with respect to means, i.e., H0:µ1-µ2=0.

Hypothesis Testing 35
Tests with two independent samples, continuous outcome…

• The null hypothesis can also be written as H0:µ1=µ2


• In the research hypothesis, an investigator can hypothesize
that the first mean is larger than the second (H1:µ1>µ2), that
the first mean is smaller than the second (H1:µ1<µ2), or that
the means are different (H1:µ1≠µ2).
• The three different alternatives represent upper-, lower-, and
two-tailed tests, respectively.
• The next Table contains the formulas for test statistics for the
difference in population means.

Hypothesis Testing 36
Tests with two independent samples, continuous outcome…

• Test statistics for testing H0:µ1=µ2

• In the formulas in the above table, are the means of the


outcome in the independent samples, and Sp is the pooled estimate of
the common standard deviation (again assuming that the variances in
the populations are similar) computed as the weighted average of the
standard deviations in the samples,

Hypothesis Testing 37
Tests with two independent samples, continuous outcome…

• Because we are assuming equal variances between groups,


we pool the information on variability (sample variances) to
generate an estimate of the variability in the population.
• As a guideline, if the ratio of the sample variances is between
0.5 and 2, the assumption of equality of population variances
is taken to be appropriate. (Note that because Sp is a
weighted average of the standard deviations in the sample, Sp
is always between s1 and s2.)

Hypothesis Testing 38
Tests with two independent samples, continuous outcome…

• Example: The data presented in elsewhere on n=3539


participants who attended the seventh examination of the
offspring in the Framingham Heart Study and constructed a
95% confidence interval for the difference in mean systolic
blood pressures between men and women. The table below
contains summary statistics on the characteristics measured
in men and women. Suppose we now wish to assess whether
there is a statistically significant difference in mean systolic
blood pressures between men and women using a 5% level of
significance.

Hypothesis Testing 39
Tests with two independent samples, continuous outcome…

• Summary statistics in Men and Women

Hypothesis Testing 40
Tests with two independent samples, continuous outcome…

• Step 1: set up the hypotheses and determine the level


of significance

• Step 2: Select the appropriate test statistic


Because both samples are large (n1≥30 and n2≥30), we use the z test statistic
as opposed to t. Before implementing the formula, we first check
whether the assumption of equality of population variances is
reasonable.

Hypothesis Testing 41
Tests with two independent samples, continuous outcome…

• The guideline suggests investigating the ratio of the sample


variances, S12/s22
• Suppose we call the men Group 1 and the women Group 2.
• Again, this is arbitrary; it needs to be noted only when
interpreting the results.
• The ratio of the sample variances is 17.52/20.12=0.76,
which falls in between 0.5 and 2, suggesting that the
assumption of equality of population variances is reasonable.
The appropriate test statistic is

Hypothesis Testing 42
Tests with two independent samples, continuous outcome…

• Step 3: Set up the decision rule.


This is a two-tailed test, using a z statistic and a 5% level of significance.
The appropriate critical values can be found in the z-table and the decision
rule is

• Step 4: Compute the test statistic


• We now substitute the sample data into the formula for the test statistic
identified in Step 2. Before substituting, we first compute Sp, the pooled
estimate of the common standard deviation.

Hypothesis Testing 43
Tests with two independent samples, continuous outcome…

• Notice that the pooled estimate of the common standard


deviation, Sp, falls between the standard deviations in
the comparison groups (i.e., 17.5 and 20.1). Sp is slightly
closer in value to the standard deviation in women (20.1)
as there are slightly more women in the sample.
• Recall that Sp is a weighted average of the standard
deviations in the comparison groups, weighted by the
respective sample sizes. We now calculate the test
statistic,

Hypothesis Testing 44
Tests with two independent samples, continuous outcome…

• Step 5: Conclusion
We reject H0 because 2.66 > 1.960.
We have statistically significant evidence at a=0.05 to show that
there is a difference in mean systolic blood pressures between
men and women. The p-value can be found in z-table and is equal to p ,
0.010.

Hypothesis Testing 45
Tests with two independent samples, continuous outcome…

In the above example, we find that there is a statistically significant difference


in mean systolic blood pressures between men and women at p , 0.010.
Notice that there is a very small difference in the sample means (128.2-
126.5=1.7 units) but this difference is beyond what would be expected by
chance.
The large sample sizes in this example are driving the statistical significance.
In example elsewhere, we computed a 95% confidence interval for the
difference in mean systolic blood pressures as 1.7±1.26, or (0.44, 2.96).
The confidence interval provides an assessment of the magnitude of the
difference between means, whereas the test of hypothesis and p-value
provides an assessment of the statistical significance of the difference.
From the confidence interval in example elsewhere, we see that the
difference in means is significant at the 5% level of significance because the
95% confidence interval does not include the null value of 0. The formal test
is needed to compute the exact statistical significance of the difference or the
p-value

Hypothesis Testing 46
Tests with two independent samples, continuous outcome…

Assumptions of two independent samples test


The variance of the dependent variable in the two
populations are equal

The dependent variable is normally distributed within


each population

The data are independent (scores of one participant


are not related systematically to the scores of the
others)

Hypothesis Testing 47
Tests with two independent samples, continuous outcome…

Hypothesis Testing 48
Tests with two independent samples, continuous outcome…

Hypothesis Testing 49
Tests with two independent samples, continuous outcome…

Hypothesis Testing 50
Tests with matched samples, continuous outcome
• An alternative study design to that described in the previous
section (to compare two groups with respect to their mean
scores on a continuous outcome) is one based on matched or
paired samples.
• The two comparison groups are said to be dependent
(matched or paired) and the data can arise from a single
sample of participants where each participant is measured
twice, possibly before and after an intervention, or from two
samples that are matched or paired on one or more specific
characteristics (e.g., siblings).
• When the samples are dependent, we focus on difference
scores in each participant or between members of a pair, and
the test of hypothesis is based on the mean difference, µd.
• The null hypothesis again reflects “no difference” and is
stated as µd=0.

Hypothesis Testing 51
Tests with matched samples, continuous outcome…
• Note that there are some instances where it is of
interest to test whether there is a difference of a
particular magnitude (e.g., µd=5) but in most instances,
the null hypothesis reflects no difference (i.e., µd=0).
• The appropriate formula for the test of hypothesis
depends on the sample size.
• The formulas are shown in Table below and are identical
to those we presented for hypothesis testing with one
sample and a continuous outcome presented in tests
with one sample, continuous outcome Section, except
here we focus on difference scores.

Hypothesis Testing 52
Tests with matched samples, continuous outcome…

• Test statistics for testing H0: µd=0

Hypothesis Testing 53
Tests with matched samples, continuous outcome…

• Example: we compared systolic blood pressures measured at the


sixth and seventh examinations (approximately 4 years apart) of the
Framingham Offspring Study in a subsample of n=15 randomly
selected participants. The data are shown in Example elsewhere
previously, where we generated a 95% confidence interval for the
mean difference in systolic blood pressures over a 4-year period.
Using the same data, we now test whether there is a statistically
significant difference in systolic blood pressures over 4 years using
the five-step approach.

Hypothesis Testing 54
Tests with matched samples, continuous outcome…
• Step 1: set up the hypotheses and determine the level
of significance

• Step 2: Select the appropriate test statistic


Because the sample is small (n<30), we use the t test statistic,

Hypothesis Testing 55
Tests with matched samples, continuous outcome…
• Step 3: Set up the decision rule.
This is a two-tailed test, using a t statistic and a 5% level of significance. The
appropriate critical value can be found in t-table with degrees of freedom, df,
defined as df=n-1=15-1=14. The critical value is 2.145 and the decision rule is

• Step 4: Compute the test statistic


• We now substitute the sample data into the formula for the test statistic
identified in Step 2. In the CI estimation Example, we had: n=15,
and
• The test statistic is

Hypothesis Testing 56
Tests with matched samples, continuous outcome…
• Step 5: Conclusion
We do not reject H0 because -2.145 < -1.60 < 2.145.
We do not have statistically significant evidence at a=0.05 to
show that there is a difference in systolic blood pressures over time.
In the CI estimation Example, we estimated a confidence interval and
were 95% confident that the mean difference in systolic blood
pressures between Examination 6 and Examination 7
(approximately 4 years apart) was between -12.4 and 1.8.
Because the null value of the confidence interval for the mean
difference is 0, we concluded that there was no statistically significant
difference in blood pressures over time because the confidence
interval for the mean difference included 0. The test of hypothesis
gives the same result.

Hypothesis Testing 57
Tests with matched samples, continuous outcome…

• Assumptions of paired two samples test


• Continuous outcome

Hypothesis Testing 58
Tests with two independent samples, dichotomous outcome

• Techniques presented in tests with independent two samples


(continuous outcome) section are appropriate when there are
two independent comparison groups and the outcome of
interest is continuous (e.g., blood pressure, total cholesterol,
weight loss).
• Here we consider the situation where there are two
independent comparison groups and the outcome of interest
is dichotomous (e.g., success/failure).
• The goal of the analysis is to compare proportions of
successes between the two groups.

Hypothesis Testing 59
Tests with two independent samples, dichotomous outcome

• The relevant sample data are the sample sizes in


each comparison group (n1 and n2) and the
sample proportions (pˆ1 and pˆ2), which are
computed by taking the ratios of the numbers of
successes to the sample sizes in each group

Hypothesis Testing 60
Tests with two independent samples, dichotomous outcome…

• In tests of hypothesis comparing proportions between two


independent groups, one test is performed and it can be
interpreted relative to a risk difference
• As a reminder, the risk difference is computed by taking the
difference in proportions between comparison groups
• When performing tests of hypothesis for the risk difference,
the convention is to label the exposed or treated Group 1 and
the unexposed or control Group 2.
• For example, suppose a study is designed to assess whether
there is a significant difference in proportions in two
independent comparison groups. The test of interest is

Hypothesis Testing 61
Tests with two independent samples, dichotomous outcome…

• The following are the hypotheses for testing for a difference


in proportions using the risk difference (RD), as measures of
effect. First, the previous hypotheses are equivalent to

• Which are, by definition, equal to

Hypothesis Testing 62
Tests with two independent samples, dichotomous outcome…

• Suppose a test is performed to test

and the test rejects H0 at α=0.05.


• Based on this test, we can conclude that there is
significant evidence (α=0.05) of a difference in
proportions, significant evidence that the risk
difference is not 0.

Hypothesis Testing 63
Tests with two independent samples, dichotomous outcome…

• The risk difference is analogous to the difference in means


when the outcome is continuous.
• The test for the difference in means was described elsewhere
previously
• Here the parameter of interest is the difference in
proportions in the population, RD=P1-P2, and the null value for
the risk difference is 0.
• In a test of hypothesis for the risk difference, the null
hypothesis is always H0: RD=0, H1: P1≠P2, which is equivalent
to H1: RD ≠0.

Hypothesis Testing 64
Tests with two independent samples, dichotomous outcome…

• Test statistic for testing H0: P1=P2

• Note that pˆ1 and pˆ2 are the proportions of successes in


Groups 1 and 2, respectively.
• pˆ is the overall proportion of successes, which is
computed as

Hypothesis Testing 65
Tests with two independent samples, dichotomous outcome…

• The preceding formula is appropriate for large samples,


defined as at least five successes (npˆ) and at least five
failures [n(1-pˆ)] in each of the two samples.
• If there are fewer than five successes or failures in either
comparison group, then alternative procedures called
exact methods must be used to test whether there is a
difference in population proportions.

Hypothesis Testing 66
Tests with two independent samples, dichotomous outcome…

• Example: In the CI estimation example, we analyzed data from


n=3799 participants who attended the fifth examination of
the offspring in the Framingham Heart Study. The outcome
of interest was prevalent CVD and we compared prevalent
CVD between participants who were and were not currently
smoking cigarettes at the time of the fifth examination of
the Framingham Offspring Stud. The data are shown in
the Table below. We now use the data to test if the
prevalence of CVD is significantly different in smokers as
compared to nonsmokers.

Hypothesis Testing 67
Tests with two independent samples, dichotomous outcome…

• Prevalent CVD in Smokers and Nonsmokers

Estimation 68
Tests with two independent samples, dichotomous outcome…

• The prevalence of CVD (or proportion of participants


with prevalent CVD) among nonsmokers is 298/
3055=0.0975, and the prevalence of CVD among
current smokers is 81 / 744=0.1089. Here smoking
status defines the comparison groups, and we call
the current smokers Group 1 (exposed) and the
nonsmokers (unexposed) Group 2. The test of
hypothesis is conducted on the following slides using
the five-step approach.

Hypothesis Testing 69
Tests with two independent samples, dichotomous outcome…
Step 1: set up the hypotheses and determine the level of significance

Step 2: Select the appropriate test statistic


• We must first check that the sample size is adequate.
• Specifically, we need to ensure that we have at least five successes
and five failures in each comparison group.
• In this example, we have more than enough successes (cases of
prevalent CVD) and failures (persons free of CVD) in each
comparison group.
• The sample size is more than adequate, so the following formula
can be used:

Hypothesis Testing 70
Tests with two independent samples, dichotomous outcome…

• Step 3: Set up the decision rule.


This is a two-tailed test, using a z statistic and a 5% level of significance.
The appropriate critical value can be found in the Z-table and the decision rule
is

• Step 4: Compute the test statistic


• We now substitute the sample data into the formula for the test statistic
identified in Step 2. We first compute the overall proportion of successes:

Hypothesis Testing 71
Tests with two independent samples, dichotomous outcome…

• We now substitute to compute the test statistic,

• Step 5: Conclusion
We do not reject H0 because -1.960 < 0.927 <1.960.
We don't have statistically significant evidence at a=0.05 to show that there is
a difference in prevalent CVD between smokers and nonsmokers

Hypothesis Testing 72
Tests with two independent samples, dichotomous outcome…

• In the CI estimation example, we estimated the 95%


confidence interval for the difference in prevalent CVD (or risk
difference) between smokers and nonsmokers as 0.0114
±0.0247, or between -0.0133 and 0.0361.
• Because the 95% confidence interval for the risk difference
included 0, we could not conclude that there was a
statistically significant difference in prevalent CVD between
smokers and nonsmokers.
• This is consistent with the test of hypothesis result.

Hypothesis Testing 73
Types of errors in hypothesis testing
• When we reject or accept the Ho, we commit errors.
• Two types of errors are committed.
– Type I Error (serious)
– Type II Error

Hypothesis Testing 74
• Type I error (α: false positive)---level of significance
 Probability of incorrectly rejecting a true null hypothesis
• Type II error (β: false negative)
 Probability of incorrectly failing to reject a false null
hypothesis
• Power (1-β: true positive)
 Probability of correctly rejecting a false null hypothesis
• True negative (1-α)--- level of confidence
 Probability of correctly failing to reject a true null
hypothesis

Hypothesis Testing 75
Hypothesis Testing 76
• You decide to get tested for COVID-19 based on mild symptoms. There are
two errors that could potentially occur:
• Type I error (false positive): the test result says you have coronavirus, but
you actually don’t.
• Type II error (false negative): the test result says you don’t have
coronavirus, but you actually do.
• Type I error =p(rejecting Ho/Ho is true) = α
• p(fail to reject Ho/Ho is True)= Confidence level/size of the test = (1-α)
correct decision
• Type II error= p(rejecting Ho/Ho is true) Power=(1-β)= P(rejecting Ho/Ho
false) correct decision

Hypothesis Testing 77
Thanks!

Introduction 78

You might also like