International Journal for Quality in Health Care 2005; Volume 17, Number 4: pp. 287–292 10.

1093/intqhc/mzi040
Advance Access Publication: 26 April 2005

Measuring the quality of hospital

tuberculosis services: a prospective
study in four Zimbabwe hospitals
CHARLES HONGORO1,2, BARBARA McPAKE2 AND PETER VICKERMAN2
1
Aurum Health Research, Economics and Health Systems, Johannesburg, Gauteng, South Africa, 2London School
of Hygiene and Tropical Medicine, Public Health and Policy, London, UK.

Downloaded from https://academic.oup.com/intqhc/article-abstract/17/4/287/2886503 by guest on 25 April 2020

Abstract
Objective. To show how the use of a prospective approach to measuring the quality of services for a specific diagnosis can
generate useful information for improving the quality of services in environments with limited information technology and data.
Design. Tracer approach focusing on intensive treatment of tuberculosis in hospital. The study was conducted in Zimbabwe in
1999. Local tuberculosis management guidelines were first translated into explicit quality assessment criteria and a panel of
public health experts assisted in weighting different factors (structural and process) of the criteria. Factor weightings were
based on both local knowledge and experience, and potential contribution of a factor to the likelihood of a positive outcome.
A total of 138 patients was recruited into the study cohort at admission and followed up to discharge. An assessment of what
was done to and for the patient was made for the entire hospitalization episode using explicit criteria. Comparisons were made
between actual and maximum performance scores.
Setting. The study was conducted at four regional referral hospitals. The hospitals serve at least six secondary hospitals, and
several public and private primary care facilities. The hospitals have a dual role as they also provide secondary care to their
immediate catchment population.
Results. Notable quality gaps are observed between actual and maximum quality levels in all four hospitals although the size of
the gap differed significantly. Variation in the quality of services between the hospitals is explained by distinguishable differ-
ences in structural and process aspects of tuberculosis management.
Conclusions. It is feasible to conduct prospective quality assessment in developing countries with minimal disruption of routine
activities. The study also showed that prospective exploration of health care quality for a specific diagnosis can provide insights
into hospital-level quality issues. Such information is useful for monitoring and improving the quality of hospital services in
general.
Keywords: quality of services, tuberculosis, Zimbabwe

Health systems around the world are being reformed. Pressure Many have grappled with the conceptualization and opera-
for change derives from the need to improve access, effi- tionalization of the quality of care concept [2–6]. Attempts to
ciency, effectiveness and quality of health services. How- be comprehensive have often led to quality definitions that
ever, it remains unclear what the impact of these reforms are difficult to measure in practice. The study adopted the
has been on quality of services [1]. To understand this Institute of Medicine [7] definition that: ‘Quality of care is the
requires facing the challenge of assessing the quality of degree to which health services for individuals and popula-
health services in developing countries where information tions increase the likelihood of desired outcomes and are con-
technology and systems for data collection are still in their sistent with current professional knowledge’ because this
embryonic stage. definition is focused and could be translated in the study context.
The difficulties of measuring quality of health services corre- Donabedian’s [3] structure–process–outcome paradigm is
spond to those of defining it. The measurement method used commonly used in quality assessment studies. In developing
depends on how quality is defined and what is feasible given countries, quality assessment studies have tended to focus on
the prevailing circumstances. This study sought to define and structural and process aspects of quality [8,9] because of the
measure quality in a manner that was both sensitive to the con- problems of data availability and reliability. Data tend to be
text, and capable of exposing areas responsive to improvement. collected for mostly administrative purposes. Furthermore,

Address reprint requests to Charles Hongoro, E-mail: charles.hongoro@lshtm.ac.uk,hongoro@hotmail.com

International Journal for Quality in Health Care vol. 17 no. 4

© The Author 2005. Published by Oxford University Press on behalf of International Society for Quality in Health Care; all rights reserved 287
C. Hongoro et al.

these studies have relied on retrospective data the quality and high priority: 2 months of intensive treatment followed by a
completeness of which is often questionable, and patient continuation phase of 4 months. The drug regimens based on
satisfaction surveys. Little attention has been given to assess- WHO guidelines are given in the Essential Drug List and
ing the quality of in-patient services, and even less to using Standard Treatment Guidelines for Zimbabwe [17]. For
prospective approaches. In this study an attempt was made to practical purposes, only the principal diagnosis was consid-
correct for data deficiencies in estimating quality of in-patient ered in recruiting patients. Ninety per cent of all admitted
services by using prospective patient-specific methods. tuberculosis cases were estimated to be HIV positive [15],
Use of specific tracer diseases to explore quality issues in which meant that cases were likely to be homogeneous in that
health care is not new [10]. Use of tracer diseases in combina- respect. Co-morbidities tend to increase hospital stay, but the
tion with explicit management criteria might provide an presence of home-based care programmes for chronic ill-
understanding for improving hospital services quality. nesses at these hospitals might have reduced the effect on
Approaches available for setting explicit quality criteria length of stay.
include review of literature, panel of experts [11], and use of

Downloaded from https://academic.oup.com/intqhc/article-abstract/17/4/287/2886503 by guest on 25 April 2020

consensus building methods like the Delphi technique
Setting quality criteria and case recruitment
[12,13]. When using a panel of experts, it is important to
ensure that the panel is constituted from practitioners working Explicit normative quality criteria for tuberculosis manage-
in the system. ment were developed from the national management guidelines
There are two broad ways of measuring quality perform- with the help of a selected panel of regional experts: three
ance: (i) categorization of care into qualitative divisions such public health physicians and two senior nursing officers.
as ‘excellent’, ‘good’, ‘fair’, or ‘poor’; and (ii) use of quantitative National guidelines formed the core to which other clinically
weighted scores for specified aspects of care. The weight relevant but non-clinical process and structural factors were
should reflect the relative importance of each factor to the added. The criteria included diagnostic, therapeutic manage-
total quality score. A panel of experts or literature review is ment, patient and environmental hygiene, documentation,
used to assign the scores and weights. Providers are then and nursing variables. Each quality component was assigned a
ranked according to the total scores, and also against an score weighted according to its relative importance in increas-
expected yardstick quality score [14]. A provider performance ing the likelihood of the desired positive outcome. Process
index (PPI) is an example of such a total score [12,13]. aspects of quality were entirely patient specific. Organiza-
However, the use of numerical scores faces two major tional and policy-related process factors were excluded
challenges: (i) medical care sometimes has an all or nothing because of physical constraints.
aspect which the component numerical scores might not For each study site, two research assistants (health account-
reflect; and (ii) the manner in which different treatment com- ants) with nursing experience were recruited for patient
ponents are weighted to arrive at a total score remains subjective. recruitment and follow-up. They were trained in the application
In this study, a local panel of experts assisted in designing the of the quality criteria, followed by trial runs on at least two in-
weighting system. patient cases each. Training was essential for standardizing
data abstraction and ratings. Patients with symptoms of
tuberculosis were conveniently recruited from hospital outpa-
tient and casualty departments. Once recruited, they were
Methods visited every day, and as and when necessitated by critical
moments in their treatment process. The assistant recorded
Choice of tracers and case definitions
what was done to and for the patient. Data were collected from
The study was carried out in four tertiary hospitals in Zimbabwe patient charts and interviews (where possible), and ward staff
in 1999. The hospitals are regional referral centres with (on non-sensitive aspects or verification of patient records).
150–235 beds. Tuberculosis was selected for the study A total of 138 cases was recruited from four hospitals.
because it is a major cause of hospital admissions and deaths. Suspected tuberculosis patients that were admitted for pre-
In 1997, 3965 tuberculosis cases were hospitalized (7% of sumptive treatment and/or investigative purposes, and were
total admissions), and 438 patients died (20% of all hospital later on either referred to a specialized infectious disease hos-
deaths) in one province [15]. The study was supported by the pital (if positive) or discharged (if negative), were excluded
Ministry of Health and Child Welfare who provided national from the analysis. Informed consent was obtained from the
tuberculosis management guidelines. patients or their guardians before recruitment in the study.
The case definition for tuberculosis was described accord- Patient codes were used to ensure confidentiality and protec-
ing to site and sputum status or history. Using the former, tion of people’s medical data. The study obtained ethical
sputum-positive tuberculosis refers to a patient with two con- approval from the Medical Research Council of Zimbabwe as
secutive positive sputum smear examinations or with one part of a larger study on hospital performance.
positive smear examination and a chest X-ray suggestive of Data were coded by hospital and patient. Data were com-
tuberculosis. Historically, a new case was defined as a patient puterized and analysed using SPSS. A total of 39 quality var-
who had never received a full course of tuberculosis treat- iables (including background characteristics) was created,
ment or who had received treatment for less than 1 month and each variable entry represented a weighted score (Tables 1
[16]. The treatment protocol for such patients was Category 1 and 2). A total weighted score was computed for each

288
 Quality of hospital TB services

n ing possible similarities in case-severity. However, high levels

case/patient: ∑ Xij , where Xij is the weighted score for each of self-referral (30–51%) reflect defective referral systems.
i
quality variable i for each patient j and n is the number of The hospitals had significantly different average lengths of
applicable variables per patient. The number of applicable stay. Case fatality rates were relatively high probably because
variables depended on whether the overall score was being of high levels of HIV infection.
calculated or whether the score by quality cluster, process or Table 2 shows a translation of the explicit quality criteria
structural, was being calculated. These weighted scores were for tuberculosis into a weighted schema as advised by the
used to calculate the average weighted score per patient/case panel of experts. The weights reflect the importance of a fac-
in each hospital: tor to both local quality expectations and prognosis, and were
used to establish maximum overall and cluster scores. The
 N n 
 PPI measures the extent to which hospital care providers per-
 ∑ ∑ Xij /N  , formed in accordance with explicit criteria.
 j=1i=1  The overall quality scores for all the hospitals are shown in

Downloaded from https://academic.oup.com/intqhc/article-abstract/17/4/287/2886503 by guest on 25 April 2020

Table 3. None of the hospitals fully met the desired standard
where N is the total number of cases/patients in a particular of service. Hospital 2 had the highest score whilst Hospital 3
hospital. The actual scores were compared against the max- had the lowest. Actual scores exceeded 64% of the maximum
imum scores, calculated using maximum scores derived from overall score. However, a shortfall of up to 36% for a fairly
the explicit criteria for tuberculosis management. By calculat- manageable disease such as tuberculosis raises serious ques-
ing subcategory or cluster scores it was possible to examine tions about the quality of service. The overall scores differed
which factors influenced the overall score or quality perform- significantly between the hospitals (P < 0.001) signifying
ance. A PPI was also calculated by dividing the observed dissimilarities in tuberculosis protocol compliance levels
scores (AS) by the maximum scores (MS), and measures the between hospitals. The overall difference between maximum
extent to which hospital care providers performed according and observed overall scores might be explained by the rela-
to explicit criteria, and takes values from 0 to 1. A value of 1 tively large discrepancies that exist between optimum and
means total conformity with the quality criteria and zero actual structural quality scores (rather than between actual
reflects total non-conformity. Statistical comparisons of quality and optimum process scores). To enhance interpretation of
scores across hospitals were made using chi-squared and the overall scores, it is necessary to look at the component
analysis of variance (ANOVA). parts separately.
Attempts to interpret structural quality scores should
recognize the dearth of knowledge on the relationship
Results between structure and process. All but Hospital 4 fell
within the upper quarter of the maximum score even
The general characteristics of the study cohort are shown in though statistically significant differences were observed
Table 1. The mean age of patients ranged from 34 to 37 years, between them (P < 0.001). This means that the basic infra-
and no significant differences were found in sex mix. No structure to potentially provide services of comparable quality
marked differences in the proportion of referred and unre- was available. The low score for Hospital 4 is explained by
ferred cases were observed across the study hospitals suggest- poor ratings in the following factors: availability of drugs,

Table 1 Sample description

Hospital/variable 1 2 3 4 P-value
(n = 31) (n = 41) (n = 37) (n = 29)
....................................................... .....................................................................................................................................................................
1
Mean patient age, years (range) 34 (18–61) 33 (5–36) 26 (1–56) 39 (21–77) 0.003
Patient sex, n (%)
Female 15 (48) 14 (35) 16 (44) 11 (38) 0.5272
Male 16 (52) 26 (65) 20 (56) 18 (62)
Referral status, n (%)
Referred 16 (52) 20 (49) 26 (70) 19 (68) 0.1522
Self-referred 15 (48) 21 (51) 11 (30) 9 (32)
Length of stay(days) 6.1 6.5 9.0 10.3 0.0091
3
Case fatality rate, n (%) 4 (13) 4 (10) 6 (16) 2 (7)

Changes in sample sizes across factors is attributable to missing values.

1
ANOVA test.
2
Pearson χ2.
3
No statistical test because of small numbers.

289
C. Hongoro et al.

Table 2 Criteria for calculation of maximum weighted mean presence of floor beds (overcrowding), inadequate micro-
scores for pulmonary tuberculosis scopes, and poor building condition. For Hospital 2, the
low score is explained by inadequate hand-washing facilities
Criteria Weighting Overall score in the wards and functional equipment, and the presence of
............................................................................................................ floor beds. The high score for Hospital 1 (>70% of the
A Process factors maximum score, 25) was due to better drug availability,
Time of contact with 1 1 working equipment, and the absence of floor beds. Hospi-
staff since arrival at facility tal 3 had adequate space, and water and sanitation facilities.
Laboratory 1 Significant differences in process quality scores were
Sputum collection 1 observed (P < 0.001). Hospital 4 had the highest process
Collection in open air/ 1 score (25/31) whereas Hospital 2 had superior ratings for the
well-ventilated room majority of process aspects except for supervision of sputum
Specimen collected under 1 collection, direct observation of treatment, and patient pri-

Downloaded from https://academic.oup.com/intqhc/article-abstract/17/4/287/2886503 by guest on 25 April 2020

supervision of competent vacy compared with the remaining two hospitals. This means
person that Hospitals 2 and 4 provided relatively better diagnostic,
Laboratory turnaround 1 clinical, and nursing services for in-patients. What is striking
time is that Hospital 4 had the lowest structural quality score but
X-ray taken 1 the highest process score. Hospital 1 had a relatively low pro-
X-ray turnaround time 1 cess quality rating even though its PPI surpassed 69%. Low
Weighing of patient1 1 process quality ratings could be linked to hospital clinical
Total laboratory score 1 7 practices (relatively low protocol compliance—frequency and
Intensive phase treatment2 observation of treatment), patient and environmental
Choice of drug 3 hygiene. Despite the observed statistical differences in the
combination: quality scores, at the aggregate level, the PPIs indicate
HRZE or HRZS relatively high levels of treatment compliance across all four
Drug dosage by weight 3 hospitals
Treatment directly 1
observed
Patient privacy 1 Conclusions
Total treatment score 8
Nursing services Measuring the quality of hospital services is difficult, and
Personal hygiene 2 more so in contexts where information technology is poor.
Elimination 2 Even where data exist, they are of poor quality because the
Nutrition 6 rationale for collection is usually administrative and not
Environmental hygiene 3 linked, for instance, to improving patient management or
Patient records 2 quality assurance activities. The prospective approach used in
Total nursing score 15 this study might have directly or indirectly introduced a Haw-
Total possible process score 31 thorne effect. We sought to minimize this by avoiding direct
B Structural factors observation of treatment, for instance, the administration of
Availability of tuberculosis 8 an injection, and instead checked (through records and
drugs patient interviews) at timely intervals whether an injection
Availability of toilets 1 was given or not. The use of two research assistants at each
Cleanliness of toilets 1 study site for data collection allowed for cross-checking of
Availability of safe water 1 data quality, and it enhanced patient recruitment. No compar-
Availability of lighting 1 isons were made to assess consistency. Such comparisons
Availability of hand-washing 1 could have improved the quality of the results. Overall, the
facilities study showed that prospective assessment of the quality of
Availability of floor beds 1 tuberculosis services in hospitals was feasible and allows for
Availability of microscopes 1 an examination of both patient-specific and non-patient-spe-
Condition of microscopes 2 cific aspects of hospital services.
Condition of buildings 2 At the patient level, it was clear that the management of
Availability of staff 6 tuberculosis services in all hospitals did not meet the local and
(nurse and doctors) international standards because of structural and process defi-
Total possible structural score 25 ciencies. Hospital 4 had two wards (male and female) with
Total weighted score 56 low but similar structural scores. Although Hospital 4 had a
poor structure for delivering tuberculosis services it had rela-
1 tively better process scores attributable to better nursing serv-
Not necessarily a laboratory activity but a necessary precursor to
effective treatment. ices, environmental hygiene and diagnostic response. The

290
 Quality of hospital TB services

Table 3 Tuberculosis mean weighted quality scores across hospitals

Hospitals/quality 1 2 3 4 Maximum P-value1

scores (n = 31), n (SD) (n = 41), n (SD) (n = 37), n (SD) (n = 29), n (SD) score
............................................ ................................................................................................................................................................ ..................
2
Structural score 17.6 (1.05) 16.1 (0.69) 16.4 (0.79) 14.0 (0) 25 <0.001
Process score 21.3 (2.18) 24.0 (3.05) 19.7 (1.71) 25.4 (1.70) 31 <0.001
Overall score 38.9 (2.12 40.1 (3.35) 36.1 (1.88) 39.4 (1.70) 56 <0.001
PPI 69.5 (0.04) 71.6 (0.06) 64.4 (0.03) 70.3 (0.03) 1 <0.001
1
ANOVA test.
2
The score was similar for both the female and male wards where the cases were assessed.

Downloaded from https://academic.oup.com/intqhc/article-abstract/17/4/287/2886503 by guest on 25 April 2020

hospital had the lowest case fatality rate, of 7%. Critical fac- References
tors like patient counselling, direct observation of treatment,
record-keeping, and patient hygiene were often ignored by 1. Heiby JR, A crisis in quality. Int J Qual Health Care 2002; 14
hospital staff in hospitals which had better process quality (suppl.): 5.
scores. 2. Donabedian A. Evaluating the quality of medical care. Milbank
Explanations of quality differences between hospitals Mem Fund Q 1966; 44 (suppl.): 166–206.
that account for the differences in scores allow for con-
sideration of the question of whether differences judged 3. Donabedian A. The Definition of Quality and Approaches to its Assess-
ment. Ann Arbor, MI: Health Administration Press, 1980.
statistically significant have clinical significance. The rela-
tively small absolute size of the differences in scores might 4. Shortbridge MH. Quality of medical care in an outpatient set-
suggest not, but the presence of drug availability and facilit- ting. Med Care 1974; 12: 283–300.
ies for basic hygiene among those factors causing differ- 5. Black N. Quality assurance of medical care. J Public Health Med
ences suggests that differences may in fact be important for 1990; 12: 97–104.
outcomes.
6. Bitran R. Efficiency and quality in the public and private sectors
The absence of adequate facilities and human resources to
in Senegal. Health Policy Plann 1995; 10: 271–283
deliver quality hospital services, particularly for a high-burden
disease such as tuberculosis in such a context, might not 7. Lohr KN, Institute of Medicine. Medicare: A Strategy for Quality Assur-
reflect successes in non-facility-based directly observed treat- ance. Vol. I. Washington, DC: National Academies Press, 1990.
ment (DOTs) programmes but poor funding and coordina- 8. De Geyndt W. Managing the Quality of Health Care in Developing
tion of resources. This situation has been worsened by the Countries. World Bank Technical Paper No. 258. Washington,
high HIV/AIDS prevalence amongst adults (24%). Intensive DC: World Bank, 1995.
phase treatment is a key component of the national tubercu- 9. Gilson L, Magomi M, Mkangaa E. The structural quality of
losis programme and forms the basis for the DOTs pro- Tanzanian primary health facilities. Bull World Health Organ 1995;
gramme. Therefore, the quality of initial treatment for 73: 105–114.
tuberculosis has implications for the success of the national
programme. 10. Kessner DM, Snow C, Singer J. Assessing health quality—the
case of tracers. New Engl J Med 1973; 288: 189–194.
At the hospital level, prospective analysis of the quality of
tuberculosis services allowed for an in-depth exploration of 11. Mainz J. Defining and classifying clinical indicators for quality
key hospital services essential for other pathologies such as improvement. Int J Qual Health Care 2003; 15: 523–530.
radiography and laboratory (diagnostic) services, treatment, 12. Payne BC, Lyons TF, Neuhaus E et al. Method of evaluating and
nursing services, and environmental hygiene. Such informa- improving ambulatory medical care. Health Serv Res 1984; 19: 219–245.
tion is crucial in identifying areas of weakness and con-
13. Mates S, Sidel VW. Quality assessment by process and outcome
straints in the hospital system—a prerequisite for any quality
methods: evaluation of emergency room care of asthmatic
improvement programme. Tracers are used in this study as adults. Am J Public Health 1981; 71: 687–693.
diagnostic tools for quality of hospital services, and it is clear
from the results that interventions may generate notable 14. Montoya-Aguilar C. Measuring the performance of hospitals
quality improvements by focusing on structure and process and health centres. World Health Organization, District Health
Systems, Division of Strengthening of Health Services, WHO/
aspects. The underlying assumption is that appropriate struc-
SHS/DHS/94.2, 1994.
tures and processes increase the likelihood of positive out-
comes. The foregoing is true for both developed and 15. Government of the Republic of Zimbabwe, Ministry of Health
developing countries [18,19]. Nevertheless, the need is and Child Welfare. Benchmark report, 1998. MOHCW, 1998.
greater in developing countries where resources and informa- 16. Government of Republic of Zimbabwe, Ministry of Health and
tion infrastructure cannot generate outcome measures of Child Welfare. Zimbabwe Tuberculosis Control Programme Manual.
quality. MOHCW, 1994.

291
C. Hongoro et al.

17. Government of Republic of Zimbabwe, Ministry of Health and 20. Government of Republic of Zimbabwe, Ministry of Health and
Child Welfare. Essential Drug List of Zimbabwe and Standard Treatment Child Welfare. Revised Standard Treatment Guidelines on Tuberculosis.
Guidelines for Zimbabwe. MOHCW, 1994. MOHCW, 1998.
18. Hostein A, Plaschke H, Schleiker et al. Structural and process 21. Heidermann EG. The contemporary use of standards in health
quality in the management of diabetic emergencies in Germany. care. Geneva: World Health Organization, 1993.
Int J Qual Health Care 2002; 14: 33–38.
19. Urassa DP, Carlstedt A, Nystrom LG. et al. Quality assessment
of the antenatal program for anaemia in rural Tanzania. Int J
Qual Health Care 2002; 14: 441–448. Accepted for publication 6 March 2005

Downloaded from https://academic.oup.com/intqhc/article-abstract/17/4/287/2886503 by guest on 25 April 2020

292