Validation of The WHO Self-Reporting Questionnaire
Validation of The WHO Self-Reporting Questionnaire
Validation of The WHO Self-Reporting Questionnaire
Abstract
Background: In Eritrea, highly centralized mental health care services and lack of trained psychiatric personnel at
primary health care units remain a challenge to the mental health care system. These problems can be minimized by
introducing screening programs with a simple screening tool for mental disorders in the primary health care settings.
Thus, this study aimed to assess the validity of the WHO self-reporting questionnaire 20 (SRQ-20) in Tigrigna version
for use in Eritrean primary health care setting.
Methods: The SRQ-20 was translated into a local language (Tigrinya) in a process of forward and backward transla-
tion. SRQ-20 data were collected in a primary health care setting on 266 respondents. Internal reliability was tested
using Cronbach’s alpha. Factorial validity was done using principal component analysis with varimax rotation to inves-
tigate whether SRQ-20 items properly measure the underlying dimensions of mental illness. Criterion validity was ana-
lyzed by looking at the relationship between the SRQ-20 and Brief Psychiatric Rating Scale using Pearson’s correlation
coefficient. Sensitivity, specificity and the predictive values of the screening instrument were used to assess how well
the results of SRQ-20 correspond with the criterion instrument.
Results: The SRQ-20 had good internal reliability (α = 0.78). Factor analysis yielded two factors, explaining 31.2% of
the total variance. The instrument performed well in detecting common mental disorders, with an area under the
curve (AUC) of 0.879 (SE = 0.23, 95% CI 0.83–0.92) to the overall sample and with optimal cut-off score at 5/6 with sen-
sitivity 78.6% and specificity 81.5%. Cut-off scores were different for women (5/6) and men (4/5). For male participants,
the AUC statistic was 0.877 (SE = 0.04, 95% CI 0.79–0.96) and 0.871 (SE = 0.02 95% CI 0.81–0.92) for female participants.
Conclusion: The Tigrinya version of the SRQ-20 can be used for screening probable common mental disorders in
Eritrean primary health care setting, but cut-off scores need to be adjusted for men and women separately.
Keywords: Common mental disorder, Self reporting questionnaire (SRQ-20), Primary health care, Eritrea
© The Author(s) 2018. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License
(http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium,
provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license,
and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/
publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Netsereab et al. Int J Ment Health Syst (2018) 12:61 Page 2 of 9
physical illness because of diminished immune function, capital city, Asmara [11]. It is estimated that the treat-
poor health behavior, non-compliance with prescribed ment gap (defined as the number of people not receiving
medical regimens and barriers to obtaining treatment for treatment despite their need), for Eritrea is nearly 97%.
physical disorders. Moreover, there is an acute shortage of skilled mental
The fundamental mental health facts report has under- health professionals and mental health services in large
scored the seriousness and increasing prevalence of men- parts of the country [12]. There is also inadequate budget
tal health problems, highlighting that, every week, one in for primary mental health services as most of the budget
six adults experience symptoms of a mental health prob- is used by the only tertiary level psychiatric hospital (St.
lem [3]. Moreover, a recent index of 301 diseases found Mary Psychiatric Hospital).The recent national mental
mental health problems to account for 21.2% of years health survey posits that the St. Mary Psychiatric Hospi-
lived with disability worldwide, indicating mental health tal does not have the capacity to care for all the mentally
diseases are becoming important contributors to the ill in Eritrea and recommends establishment of mental
global burden of disease [4]. The consequences of mental health care wings, prioritizing the zonal referral hospitals
disorders in terms of lost health are significant. Depres- for initial action. The study also underscored the care of
sion is ranked by WHO as the single largest contributor the mental ill should be integrated as part of comprehen-
to global disability (7.5% of all years lived with disability sive primary health care service and be considered with
in 2015) while anxiety disorders are ranked 6th (3.4%) [5]. the routine health service delivery activities [7].
Mental illness is considered a silent epidemic through- The problems of low financial expenditure and inequity
out most parts of Africa. Because of structural and in distribution of mental health facility can be minimized
systemic barriers such as inadequate health care infra- by introducing psychosocial interventions especially
structure, insufficient number of mental health special- screening program for mental disorder in the primary
ists, and lack of access to all levels of care, mental illness health care settings which are accessible and affordable to
has been characterized as a neglected and increasingly the general population. Considering the understaffed and
burdensome problem affecting all segments of the popu- undertrained personnel in primary health care settings
lation throughout Africa. Prioritizing mental health has of the country, the SRQ would play an important role in
also been difficult due to lack of resources, limited fund- improving community’s mental health. The WHO SRQ-
ing coupled with absent or ineffective mental health poli- 20 is not functional and has never been used in Eritrea.
cies [6]. The culturally sensitive nature of the questionnaire and
A recent national mental health survey conducted in the language difference of settings necessitate adaption
Eritrea reports a common mental disorder prevalence of the instrument locally. The SRQ is expected to help
rate of 14.5% in the general population [7]. The 2015 because it is a simple instrument, poverty friendly, timely
global burden of disease study reports 219, 549 cases of and does not need a professional to administer; which is
depressive disorders (4.3% of the total population) and feasible in our country.
156, 599 cases of anxiety disorders (3.1%) in the country There are many psychiatric screening instruments
[8]. The consequences of these disorders in terms of lost developed by the World health Organization (WHO).
health are also considerable. The 2015 global health esti- Nevertheless, these instruments are time-consuming
mate for Eritrea attributes 40, 426 total years lived with and require trained personnel for administration either
disability for depressive disorders (8.2% of years lived through interview or through self-administered ques-
with disability) and 14, 474 total years lived with disabil- tionnaire, being infeasible for use in the primary health
ity for anxiety disorders (2.9%) [9]. care, which, often, are the target settings for screening
In Eritrea, inequitable distribution of psychotropic and early detection of CMDs. This gives rise to the need
drugs, highly centralized mental health care services to of the self-reporting questionnaire (SRQ-20), a 20-item
urban centers and lack of trained psychiatric personnel mental disorder screening instrument developed by
at primary health care units are some of the challenges WHO to be applied in patients attending primary
the mental health program is facing [10]. According to health care settings [13]. Trudy Harpham et al. reviewed
the study of assessment of mental health system in Eri- the performance and use of this questionnaire and con-
trea, similar drawbacks as in the majority of developing cluded that the SRQ-20 as a cost-effective instrument
countries are prevalent. These include absence of revision to measure mental health at community level with high
of the mental health policy, very low financial expendi- face and criterion validity, ease of use and suitability to
ture on mental health and lack of access to mental health administer it by a lay health worker [14]. Several com-
facilities for in the country there is only one mental parative and validation Studies also found the SRQ-20
health hospital with its only outpatient unit and the only to be preferred from many other screening question-
community residential facility; all concentrated in the naires [15–18].
Netsereab et al. Int J Ment Health Syst (2018) 12:61 Page 3 of 9
The SRQ-20 is an instrument with twenty simple to selected using the Daniel and Macfarlane single propor-
understand items which question respondents about tion formula for finite population, n = Z2 × P × Q/E2.
symptoms and problems likely to be present in those with The study respondents were recruited using a simple ran-
neurotic disorder. It includes binary (yes/no) questions dom sampling. Eligible participants included both male
only, with codes “1” which represents the presence of a and female who were in the age range of 18–65 years and
symptom, and “0” if the symptom is absent. The SRQ- who were attending the selected health facilities at the
20 item questions reflect depression, anxiety and psy- time of data collection. Individuals were excluded from
chosomatic complaints, which are all together, grouped the study if they were medically unstable, if they required
under the common mental disorder (CMD) and have referral to a higher level of care, or were not willing to
been found to detect probable cases of it with satisfactory participate in the study.
accuracy [19]. However, the validity, reliability and cut-off
of the SRQ-20 vary in different settings across variety of Data collection instruments
populations (with different culture, language, setting and In addition to the demographic characteristics of the
gender) [15–19]. For this reason, adapting the SRQ-20 in respondents such as age, gender, ethnicity, educational
one’s community setting or population primarily requires level and marital status, the measures utilized in our
validating the screening instrument in one’s own setting. study include the WHO self-reporting questionnaire and
Many developing countries have validated and have been Brief Psychiatric Rating Scale.
using the SRQ-20 in many community-based screening
programs. In Eritrea, the SRQ is not functional to use as The self‑reporting questionnaire
screening instrument in PHC, for its validity is not yet The self-reporting questionnaire is based on the original
assessed. 20 yes/no questions translated to Tigrinya which consists
Therefore, this study aimed to assess the validity of the of 20 items indicative of non-psychotic mental disor-
WHO self-reporting questionnaire 20 (SRQ-20) item ders. The questions are related to certain pains and prob-
in Tigrigna version in Eritrean setting. Specifically, this lems that may have bothered the participant in the last
study evaluated the psychometric properties, optimal 30 days. If the participant thinks the question applies to
gender specific cut-off points for detecting CMDs using him/her, then will answer as yes and the other ways it will
SRQ as a screening tool and assessed the SRQ-20 items’ be no. The translation of these questions included lan-
capacity to screen for mental disorder relative to the gold guage experts. It was translated in the standard back and
standard (BPRS). forth translation format.
and determination of optimal cut-off scores should be whether SRQ-20 items measure properly the underly-
performed [14]. For this study, first all the twenty items of ing dimensions of mental illness. The Kaiser–Guttman
the WHO SRQ-20 were translated to Tigrigna, a widely criterion was employed to extract factors with Eigen val-
spoken local language in Eritrea by a psychiatrist and lan- ues above one [22, 23]. The scree plots were examined
guage experts. For the purpose of conceptual meaning to identify the optimal number of factors for extraction.
adjustment, the instrument was blindly back translated The Kaiser–Meyers–Olkin (KMO) test was calculated to
by other language experts and psychiatrists who were not determine the sampling adequacy for the procedure. Bar-
familiar with the purpose of the study. Any inconsistency tlett’s test of sphericity was calculated to confirm if it was
in language meaning and translation differences were significant to perform a PCA.
resolved through a focus-group discussion.
Criterion validity
Study procedure The relationship between the testing instrument, the SRQ
Prior to the interview, each participant was informed and the criterion (the BPRS) was assessed using Pearson’s
about the confidentiality of the data collection process. correlation coefficient. Moreover, sensitivity, specificity
They were assured that participation was voluntary, and and the predicative values of the screening instrument
could withdraw from the interview at any time. After were used to assess how well the results of SRQ-20 cor-
fully describing the study objectives in a private room, respond with the criterion instrument (BPRS).
consented respondents were separately interviewed from The optimum cut-off scores were chosen with maxi-
other patients for health related questions. The interview mum sensitivity and reasonable specificity at various cut
was conducted during day time to the patients in the off points. The Receiver Operating Characteristics (ROC)
waiting room for outpatient treatment. During the inter- analysis was performed to evaluate the screening prop-
view, the data collector explained that they will be asked erties of the SRQ-20, using BPRS as the gold standard.
health related questions but not exactly about mental The Area under Curve (AUC) statistic was calculated as a
health in order to reduce misleading answers. If the par- measure of accuracy of the SRQ-20.
ticipant was required by the medical personnel in the
OPD then the interviewer would temporarily suspend Results
the interview and continue after the patient completed In this study, out of the 266 participants, more than half
OPD visit. were women (54.5%, n = 145), and about 70% were in
the age group of 18–35 years. The mean age and stand-
Data analysis ard deviation of the participants was 32 (± 11.09). Most
All data were analysed using SPSS (Statistical Package for of the study participants reached high school (35.3%,
Social Sciences) version 20. n = 94) and (62%, n = 167) were married. Tigrigna eth-
nic group was dominant over other ethnic groups (89%);
Reliability this is because the setting was in Asmara city in which
Reliability (internal consistency) of the SRQ-20 question- majority of its population is Tigrigna. Detailed informa-
naire was calculated using a reliability coefficient (Cron- tion about demographic characteristics of the study par-
bach’s alpha). ticipants can be referred in Table 1.
Table
4 Factor loadings of the principal component
analysis in the total sample
Items Factor 1 Factor 2
Discussion
This study aimed to validate the SRQ-20 item in Eri-
trean primary health care settings. In the overall sam-
ple, male to female ratio was almost proportional.
Fig. 3 Receiver operating curve for female participants. AUC = 0.871 Most of the respondents were in the age range of
(SE = 0.02 95% CI 0.81–0.92) 18–25 years, married, Tigrigna ethnicity and middle-
schooled. As can be inferred from the findings, the
SRQ-20 item was found to have a substantial capacity
ability of CMD non-cases detection and PPV was found for screening of common mental disorders in Eritrean
to be 55.3%. primary health care settings. The cut-off scores for the
These are the measures of determining the validity of SRQ-20 differed by gender.
the scale for screening of the PPD. In the present study, the prevalence of CMDs in the
overall sample was found to be 23%. Many epidemio-
Sensitivity = TP/TP + FN = 47/(47 + 13)
logical studies conducted in primary health settings
= 0.7833 = 78.3%
identify mental disorder using screening instruments But now since the results confirm the performance of the
or clinical diagnosis [24]. Since the SRQ20 cannot be instrument in detecting CMDs, the SRQ-20 can be a suit-
a substitute for, or equivalent to, a clinical diagnosis able tool for measuring CMD in PHC settings of Eritrea.
[14], we used the BPRS results to measure the preva- In this study, the balanced sensitivity and specific-
lence rate of CMDs in our country. As per cross-sec- ity was achieved at the higher cut-off score of 5/6 for
tional studies conducted by WHO at 14 sites found out, women, as compared to the 4/5 cut-off score for men. For
about 24% of all patients in PHC settings had a mental overall sample, an optimal local cut-off, 5/6 was found
disorder [24]. The prevalence of CMDs in this study as to balance the trade-off between sensitivity and specific-
measured by the gold standard, was higher (23%) than ity at 78.6% and 81.5% respectively. This is almost similar
the previous prevalence of mental health disorders in with a study conducted in the UAE were a cut-off score of
the country, 14.5% in 2014 [25]. The higher prevalence 5/6 produced a balanced combination of sensitivity and
rate attained from the current study with respect to for- specificity of 78.3% and 75.2%, respectively [33]. Stud-
mer national prevalence rate, may be associated with ies conducted in other countries like Rwanda and South
a steady increment in the prevalence rate of mental Africa also reported gender differences in SRQ scores
disorders globally especially for the low-income coun- with higher cut-off scores for women than men [29, 34].
tries as justified by sustained population growth and However, there are few studies conducted in other coun-
poverty as strong risk factors [26]. As this study was tries like Zambia and the United Kingdom that have
conducted in PHC settings, where respondents came found no differences in the optimal cut-off scores for
chiefly to address their underlying health problems, both sexes [35, 36].
over-reported symptoms of CMDs may had pathologic Although the study has come up with important find-
origin related to it and hence might have contributed ings, due to some limitations, the findings should be
to high rate of prevalence. The result from this study interpreted with caution. As the translated version of
appeared to be lower when compared to prevalence of SRQ-20 is in Tigrigna, other language speakers from
two towns in Ethiopia, 33.6% in Jimma and 32.4% in different ethnic groups may not be beneficiaries of the
Kombolcha [27, 28]. According to the WHO reports instrument and limited diversity in the ethnicity of the
from 14 countries worldwide, the prevalence of CMD participants may not represent and generalize the result
in PHC settings is 24% [24], which is almost similar of the current study to all Eritrean communities.
to the finding in this study. In 2013, a study done in The findings of the study have good implication for
Malawi also reported that CMD prevalence in PHC was addressing the issue of screening for common mental dis-
20.1% [29]. orders in primary health care settings. Here, the SRQ-20
Majority of the CMD cases in our study were females item has been found to be suitable in primary a health
when compared to their male counterparts. This is simi- care setting which is simple, easy to administer and
lar to findings from other studies done in Ethiopia and with a yes/no questionnaire which is convenient for the
Malawi [27–29]. The reason for this could be as a result interviewer and the respondents. Due to resource-poor
of gender disadvantages like social burden and maternity conditions with ineffective means of detecting CMDs in
issues. There is also tendency of over-reporting mental Eritrea, the Tigrigna-version SRQ-20 item is expected
problems by females. to contribute in reducing the prevalence and burden of
In our study, we found good internal consistency of CMDs through its simplicity, acceptability and feasibility.
α = 0.78, almost similar with findings in China (α = 0.79)
[30], but slightly lower than that of South Africa Conclusion
(α = 0.84) [17]. The Tigrigna-version SRQ-20 is a valid and reliable
Similar with other studies, the findings from the pre- instrument which has the capacity to detect cases of
sent study indicates that the SRQ-20 item is a useful tool CMDs in cost-effective manner. The use of the SRQ-20 is
in Eritrean primary health care settings. In this study, the recommended for use in the PHC settings for screening
area under the ROC curve statistic was 0.87. This result probable common mental disorders in Eritrea.
is consistent with findings from other countries’ primary
Authors’ contributions
health care settings where the area under the ROC curve TBN and MMK wrote the article. TBN, MMK, RBT, SGH, YKW, OZT participated in
ranged from 0.76 in Rwanda to 0.92 in Uganda [31, 32]. In study design, data collection and analysis. All authors read and approved the
this study, the AUC for women and men were 0.871 and final manuscript.
0.879 respectively. This indicates that the SRQ-20 was Author details
just as effective in distinguishing between cases and non- 1
School of Public Health, Asmara College of Health Sciences, Asmara, Eritrea.
2
cases in both groups. This screening tool has never been Ministry of Health, Asmara, Eritrea.
validated in our country’s primary health care settings.
Netsereab et al. Int J Ment Health Syst (2018) 12:61 Page 9 of 9