Bicycle helmet effectiveness in preventing injury and death

Bicycle Helmet Effectiveness in Preventing Injury and Death Lloyd F. Novick, MD, MPH, Martha Wojtowycz, PhD, Cynthia B. Morrow, MD, MPH, Sally M. Sutphen, MSc, MPH Abstract: This case— bicycle helmet effectiveness—is one of a series of teaching cases in the Case-Based Series in Population-Oriented Prevention (C-POP). It has been developed for use in medical school and residency prevention curricula. The complete set of cases is presented in this supplement to the American Journal of Preventive Medicine. This case examines the cost-effectiveness of three interventions to increase utilization of bicycle helmets to avert head injuries in individuals aged 18 years and under in Onondaga Count NY. Students are initially presented with data on head injuries, hospitalization, and death related to bicycle use. They then appraise a published study on the effectiveness of bicycle helmets in averting head injury. Finally, students work in groups to determine the cost-effectiveness of each intervention by calculating implementation costs and the specific number of head injuries averted associated with intervention. The three interventions are legislative, school, and community-based campaigns to increase helmet use. Students are provided with budget estimates and assumptions needed to complete the exercise. Cost-effectiveness analysis, cost-benefit analysis, and related concepts are discussed, including provider versus societal perspectives and importance of sensitivity analysis. (Am J Prev Med 2003;24(4S):143–149) © 2003 American Journal of Preventive Medicine Recommended Reading: ● critically appraise a published clinical study, ● Study Design, Prevention Effectiveness: A Guide to Decision Analysis and Economic Evaluation. Edited by AC Haddix, SM Teutsch, PA Shaffer, DO Dunet; New York: Oxford University Press, 1996. ● Anonymous. Injury Control Recommendation: Bicycle Helmets. MMWR 1995; 44 (RR-1): 1–18. ● Gaspoz JM, Coxson PG, Williams LW, Kuntz KM, Hunink MM, Goldman L. Cost effectiveness of aspirin, clopidogrel, or both for the secondary prevention of coronary heart disease. N Engl J Med 2002; 346 (23):1800 – 6. ● Thompson RS, Rivara FP, Thompson DC. A case– control study of the effectiveness of bicycle safety helmets. N Engl J Med 1989; 320 (21): 1361–7. ● critically appraise strengths and weakness of different study designs, Objectives: At the end of the case, the student will be able to: ● define cost-effectiveness and how it is measured, ● review cost-effectiveness analysis examples from the medical literature, ● interpret trends from data from a State Health Department, From SUNY–Upstate Medical University, Syracuse, New York Address correspondence and reprint requests to: Lloyd F. Novick, MD, MPH, Preventive Medicine Program, Department of Medicine, SUNY–Upstate Medical University, 714 Irving Avenue, Syracuse NY 13210. E-mail: PMP@upstate.edu. ● calculate and apply cost-effectiveness principles, and ● Apply economic evaluation concepts. Case Note: Due to the complexities of some of the sections in this case, it is helpful, although not necessary, to teach this case with someone who has training in cost-effectiveness analysis. The preceptors’ guide contains comprehensive answers if a co-facilitator is not available. Section A Cost-Effectiveness Teaching Note: Students should complete Section A prior to class. Cost-effectiveness plays a critical role in determining the best course of action for the management of health problems both in clinical and in population medicine. Refer to the first two recommended readings when addressing these questions. Question 1. Define cost-effectiveness. Question 2. How is cost-effectiveness calculated and what outcome measures are commonly used? Am J Prev Med 2003;24(4S) © 2003 American Journal of Preventive Medicine • Published by Elsevier Inc. 0749-3797/03/$–see front matter doi:10.1016/S0749-3797(03)00042-4 143 Table 1. Deaths due to bicycle injuries by age and gender Age (years) Males: Frequency (rate)* 1996 Population Females: Frequency (rate)* 1996 Population Total: (rate)* 1996 Population 0–4 5–9 10–14 15–19 20–24 25–44 45–64 ⬎65 Total 0 (0) 4 (0.58) 4 (0.63) 6 (1.01) 4 (0.65) 17 (0.57) 9 (0.49) 2 (0.21) 46 (0.51) 671,564 686,178 630,136 596,126 611,686 2,973,953 1,823,532 943,640 8,936,815 0 (0) 0 (0) 0 (0) 1 (0.18) 0 (0) 2 (0.07) 0 (0) 1 (0.07) 4 (0.04) 643,473 652,821 600,153 570,697 602,435 3,009,727 2,022,921 1,467,358 9,569,585 0 (0) 4 (0.30) 4 (0.33) 7 (0.60) 4 (0.33) 19 (0.32) 9 (0.23) 3 (0.12) 50 (0.27) 1,315,037 1,338,999 1,230,289 1,166,823 1,214,121 5,983,680 3,846,453 2,410,998 18,506,998 *Rate: Frequency/ 1996 Estimated Population ⫻ 100,000. Source: New York State Department of Health, Bureau of Injury Prevention and Biometrics. Section B Analysis of Available Data: Effectiveness of Bicycle Helmets in Preventing Morbidity and Mortality As a consultant to the local legislature, you are asked to determine the best means of reducing morbidity and mortality associated with bicycle riding in your county. To provide advice regarding this issue, you need to be able to interpret the available data. Local data on morbidity is not available because of the lack of uniform reporting of such injuries. In regard to mortality data, the number of fatalities associated with bicycle use in a community of this size is too small to be useful for analysis. Fortunately, the New York State Department of Health (NYSDOH) is able to provide you with the following information. (Refer to Table 1). Question 1. Comment on the differences in bicycle injury mortality by age and gender as well as on the interaction between age and gender. Question 2. What are possible explanations for these differences? Question 3. How would this information help you formulate prevention strategies for your community? The NYSDOH is also able to provide you with the following graphs on overall bicycle-related morbidity and mortality rates, as well as information specific to traumatic brain injury or death due to bicycle use for the period 1991–1996. (Refer to Figures 1– 4). Question 4. What are your hypotheses with respect to the trends in rates for death, hospitalization, and traumatic brain injuries associated with bicycle use during these years? Figure 1. Deaths due to bicycle-related injuries. Rate per 100,000 by age group. New York state residents, 1991–1996. The rate of deaths due to bicycle-related injuries in children aged 0 –13 years has been declining sharply since 1991, with a mortality rate in 1996 that is almost one third the mortality rate in 1991. Data source: NYSDOH. Prepared by the OCHD, Bureau of Surveillance and Statistics. 144 American Journal of Preventive Medicine, Volume 24, Number 4S Figure 2. Hospitalizations due to bicycle-related injuries. Rate per 100,000 by age group. New York state residents, 1991–1996. Deaths due to bicycle-related traumatic brain injuries have been declining for persons of all ages since 1991. Data source: NYSDOH. Prepared by the OCHD, Bureau of Surveillance and Statistics. Question 5. What are some of the limitations of the data that have been presented? Thompson RS, et al. A Case–Control Study of the Effectiveness of Bicycle Safety Helmets; NEJM May 1989. Section C Effectiveness of Bicycle Helmet Use: An Appraisal of Scientific Evidence Question 1. Why did the author choose to do a case– control study to determine cost-effectiveness of helmet use? Could he have done a randomized control study? A prospective cohort study? What are the major limitations of these study designs in this situation? In addition to demographic information provided, you need more knowledge about the effectiveness of bicycle helmets before you present your official recommendations to the local health advisory board. You review Question 2. Identify biases associated with case– control studies, including selection of cases and controls. Figure 3. Deaths due to bicycle-related traumatic brain injuries. Rate per 100,000 by age group. New York state residents, 1991–1996. *While the rate of hospitalizations due to bicycle-related injuries in children aged 0 –13 has been declining since 1990, the rate of deaths due to bicycle-related injuries in persons aged 14 and over has remained relatively stable. Data source: NYSDOH. Prepared by the OCHD, Bureau of Surveillance and Statistics. Am J Prev Med 2003;24(4S) 145 Figure 4. Hospitalizations due to bicycle-related traumatic brain injuries. Rate per 100,000 by age group. New York state residents, 1991–1996. *While the rate of hospitalizations due to bicycle-related injuries in children aged 0 –13 has been declining since 1990, the rate of deaths due to bicycle-related injuries in persons aged 14 and over has remained relatively stable. Data source: NYSDOH. Prepared by the OCHD, Bureau of Surveillance and Statistics. Question 3. Comment on the comparability between “cases” and “controls.” Question 4. What information provided by this study regarding effectiveness of bicycle helmets is generalizable? Question 5. Discuss how you would develop and implement a study to determine use of bicycle helmets by age, gender, and location in your county. Discuss sampling and measurement issues. Question 6. What are some of the factors that would influence the effectiveness of bicycle helmets in preventing injuries and death at a population level? — Limited public education (publicity/media) to increase awareness of helmet law; — Enforcement of law ● Provision of helmets: No helmets are provided under this option. Target population is expected to purchase helmets. Community option: The local health department is responsible for a comprehensive program to educate the entire community about the risks of bicycle injuries and the benefits of helmet use. The health department will also provide helmets at cost to indigent children. Section D Development of Preventive Programs Utilizing a Cost-Effectiveness Approach ● Target population (all county residents): 450,000 ● Program costs to be considered: — Health education (publicity/media) of bicycle injuries and helmet use — Distribution of helmets at cost to all indigent children You now have demographic information about bicyclerelated injuries and deaths as well as scientific evidence to support the effectiveness of bicycle helmets in reducing bicycle-related morbidity and mortality. You determine that three feasible options for preventive programs are aimed to increase helmet use in your county. The options are: Legislative option: This option involves efforts to educate the public about the passage of a new law that requires helmet use for all individuals 18 years old or younger. It also requires enforcement of this new law. ● Provision of helmets: County provides helmets at cost for indigent children. Based on the most recent census data, the number of indigent children is 20% of all children less than 18 years old (125,000 ⫻ 20%⫽ 25,000). — The health department will buy helmets for 25,000 children at $10 per helmet — The health department will sell helmets to parents/guardians of 20,000 children at $10 per helmet (assuming that not all helmets will be sold) ● Target population (all residents aged ⱕ18 years): 125,000 ● Program costs to be considered: School option: The school board and the health department are responsible for educating school-aged children about the risks of bicycle injuries and the 146 American Journal of Preventive Medicine, Volume 24, Number 4S benefits of helmet use. The health department will also provide helmets at cost to indigent children. ● Target population (all school-aged children): 84,000 ● Program costs to be considered: — Classroom education of helmet use aimed at school-aged children. Educational efforts will also be made to parents of the target population. — Distribution of helmets at cost to all indigent children. ● Provision of helmets: County provides helmets for indigent children at cost. Based on the most recent census data, the number of indigent children is 20% of all school-aged children less than 18 years old (84,000 ⫻ 20% ⫽ 16,800). — The health department will buy helmets for 16,800 children at $10 per helmet — The health department will sell helmets to the parents/guardians of 13,500 children at $10 per helmet Calculating Cost-Effectiveness: Teaching Note: This section is taught with students divided into at least three groups, one for each option. The groups are given 15 minutes to construct their budget and to calculate the number of head injuries averted. They must budget sufficient resources to realistically accomplish the goals set out by their option but they cannot bankrupt the county, the health department or the school district. Each option entails different budget costs associated with it. You are asked to determine which option is the most cost effective. For each of the options, you need to use the following formula: Cost of Option Cost-Effectiveness ⫽ Number of Head Injuries Averted Both the numerator and the denominator need to be calculated. To determine the total cost of each option, you will need to use your judgment to determine how much will be spent on personnel costs and how much will be used on the education campaign. For personnel costs, depending on the option, the cost of health educators, of the staff responsible for organizing and distributing helmets, and of officers for enforcement of the law will need to be considered. Guidelines for the estimated costs are provided in Table 2. Question 1: What is the total cost of your option? The following formula can be used to determine the number of head injuries averted: Number of head injuries averted ⫽ (Change in helmet use) ⫻ (Number of bicyclists in the target population) ⫻ (National bicycle-related head injury rate) ⫻ (Efficacy rate of helmet use) Table 2. Cost estimates for budget calculation Program component: Cost: Helmets Health education staff Helmet program staff Public Information Campaign Develop one television spot Pay for one television spot Public service television spot Develop and pay for one radio spot Brochures Enforcement $10 cost; $25 retail $40,000/employee/year $30,000/employee/year $10,000 $2000 Free- $250 $350 $2,500 for 10,000 brochures $50,000 per year To simplify calculations, certain assumptions about helmet use must be made. Some of these assumptions may be optimistic. For this exercise, it is assumed that all people in the target population are potential bicyclists. Data from the health department indicate that baseline helmet use is approximately 20%. It is assumed that helmet use will increase to approximately 50% after each of the interventions. The National Injury Rate for bicycle use is 50/100,000. Finally, the efficacy rate of helmet use, based on current literature, is assumed to be 85%. Taking these assumptions into account, the following formula should be applied: Number of head injuries averted ⫽ .30 ⫻ target population ⫻ 50/100,000 ⫻ 0.85 Teaching Note: Depending on the background experience of the preceptor and the amount of time available, the preceptor may choose to present the answers to Questions 2 and 3 to the students. As an example, these answers are shown in Table 3. This table is for preceptor purposes. Question 2: Using the information provided, how many head injuries were averted with your option? Question 3: What is the cost per head injury averted? Teaching Note: After each group has completed the work, the whole class reconvenes. Each group presents the answers for their option. The class then addresses the following questions. Question 4: Which is the most cost-effective option? Question 5: Do you have any significant concerns about presenting this option as the “best” option when you provide your recommendation to the Health Advisory Board? Consider the perspective of each option when answering this question. Does a health department have a different point of view about the costs they must invest in an intervention than do legislature or society as a whole? Am J Prev Med 2003;24(4S) 147 148 American Journal of Preventive Medicine, Volume 24, Number 4S Table 3. Bicycle helmet cost-effectiveness exercise sample answer sheet* Target Population Change in number of helmet users Program cost Legislative option Community option School option 125,000 37,500 Publicity $50,000 Enforcement $10,000 450,000 135,000 Media $100,000 Health Education (FTE—$40,000) Distribution (FTE—$30,000) ⫽ $170,000 At cost ($10) for indigent children and adolescents Assumption: BUY ($10 ⫻ 25,000) SELL ($10 ⫻ 20,000) —Program ($170,000) ⫹ —Helmets purchased by agency but not sold ($10 ⫻ 5,000) ⫽ $220,000 —Program ($170,000) ⫹ —Helmets purchased by agency but not sold ($10 ⫻ 5,000) ⫹ —Helmets purchased by parents ($25 ⫻ 110,000) ⫽ $3,170,000 57 $220,000/57 ⫽ $3,860 $3,170,000/57 ⫽ $55,614 84,000 25,200 Publicity $25,000 Distribution (0.5 FTE—$15,000) ⫽ $60,000 Provide helmets None Total cost (narrow perspective) Program only ⫽ $60,000 Total cost (societal perspective) —Program ($60,000) ⫹ —Helmets purchased by parents ($25 ⫻ 37,500) Head injuries averted Cost/head injury averted (narrow) Cost/head injury averted (societal) ⫽ $997,500 16 $60,000/16 ⫽ $3,750 $997,500/16 ⫽ $62,344 ⫽ $40,000 At cost ($10) for indigent school children Assumption: BUY ($10 ⫻ 16,800) SELL ($10 ⫻ 13,500) —Program ($40,000) ⫹ —Helmet purchased by agency but not sold ($10 ⫻ 3,300) ⫽ $73,000 —Program ($40,000) ⫹ —Helmets purchased by agency but not sold ($10 ⫻ 3,300) ⫹ —Helmets purchased by parents ($25 ⫻ 8,400) ⫽ $418,000 11 $73,000/11 ⫽ $6,636 $418,000/11 ⫽ $38,000 *This sample answer sheet is provided for preceptor purposes. A complete preceptor version is available from the Preventive Medicine Program, SUNY–Upstate Medical University, Syracuse, New York. Section E Economic Evaluation B. What are the strengths and weaknesses of each analysis? When the cost-effectiveness of a program is interpreted, the perspective from which the analysis was performed must be taken into account. In other words, was the analysis done from a broad perspective where all costs and benefits to the population are considered or was it done from a narrow perspective where only costs or benefits to a certain subgroup were addressed? In general, a societal perspective is the broadest perspective. In contrast, an analysis done from the point of view of a hospital or an insurance company provides a much more narrow perspective. C. What questions are best answered by each method? Question 1: From what perspective did you conduct your analysis in Section D? Consider the perspective of each option when answering this question. (For example, does a health department have a different point of view than does the legislature or society as a whole?) How would your results change if you were to conduct your analysis from a societal perspective? Thus far in the case, cost-effectiveness has been used to determine the cost per head injury averted. There are different techniques available to conduct an economic analysis, one of which is cost-benefit analysis. Refer to the recommended reading to address the following questions. Question 2: A. What is the difference between cost-effectiveness analysis (CEA) and cost– benefit analysis (CBA)? Finally, because an economic analysis is based on certain sets of assumptions about variables, it should include a sensitivity analysis in which the assumptions are challenged to see how much they affect the outcome of the analysis. Examples of variables for which sensitivity analysis is helpful include success rate of the intervention, valuation of costs of the intervention, or valuation of the benefits. An example of sensitivity analysis is available in the recommended reading by Gaspoz. Question 3: A. In your analysis of the cost-effectiveness of bicycle helmets, what were the most important variables? B. How would changes in these variables affect the outcome of the analysis? Question 4. Taking perspective, type of economic analysis, and sensitivity analysis into account, which preventive approach do you now think is the most cost effective means to decrease death and injury due to bicycle-related accidents in your county? The Case-Based Series in Population-Oriented Prevention (C-POP) is funded by grants from the Josiah Macy, Jr. Foundation and the Health Resources and Services Administration, U.S. Department of Health and Human Services. Am J Prev Med 2003;24(4S) 149