Journal of

Clinical Medicine

Article
An Integrated Account of Expert Perspectives on Functioning
in Schizophrenia
Laura Nuño 1,2, * , Georgina Guilera 2,3 , Emilio Rojo 4,5 , Juana Gómez-Benito 2,3 and Maite Barrios 2,3

1 Clinical Institute of Neuroscience (ICN), Hospital Clinic, 08036 Barcelona, Spain

2 Department of Social Psychology and Quantitative Psychology, University of Barcelona,
08007 Barcelona, Spain; gguilera@ub.edu (G.G.); juanagomez@ub.edu (J.G.-B.); mbarrios@ub.edu (M.B.)
3 Group on Measurement Invariance and Analysis of Change (GEIMAC), Institute of Neurosciences,
University of Barcelona, 08007 Barcelona, Spain
4 Hospital Benito Menni CASM, Sisters Hospitallers, 08830 Sant Boi de Llobregat, Spain;
erojo.hbmenni@hopitalarias.es
5 Department of Psychiatry, International University of Catalonia, 08007 Barcelona, Spain
* Correspondence: nuno@clinic.cat

Abstract: An integrated and interdisciplinary care system for individuals with schizophrenia is
essential, which implies the need for a tool that assesses the difficulties and contextual factors of
relevance to their functioning, and facilitates coordinated working across the different professions
involved in their care. The International Classification of Functioning, Disability and Health Core Sets
(ICF-CS) cover these requirements. This study aimed to evaluate the content validity of the ICF-CSs
for schizophrenia from the perspective of experts. Six three-round Delphi studies were conducted
with expert panels from different professional backgrounds which have played a significant role in the



treatment of individuals with schizophrenia (psychiatry, psychology, nursing, occupational therapy,


social work and physiotherapy). In total, 790 experts from 85 different countries participated in the
Citation: Nuño, L.; Guilera, G.; Rojo,
first round. In total, 90 ICF categories and 28 Personal factors reached expert consensus (reached
E.; Gómez-Benito, J.; Barrios, M. An
consensus from four or more professional perspectives). All the categories in the brief version of the
Integrated Account of Expert
ICF-CS for schizophrenia reached consensus from all the professional perspectives considered. As
Perspectives on Functioning in
for the comprehensive version, 89.7% of its categories reached expert consensus. The results support
Schizophrenia. J. Clin. Med. 2021, 10,
4223. https://doi.org/10.3390/
the worldwide content validity of the ICF-CSs for schizophrenia from an expert perspective and
jcm10184223 underline the importance of assessing functioning by considering all the components implied.

Academic Editor: Agata Szulc Keywords: schizophrenia; rehabilitation; mental disorders; public mental health; Delphi studies

Received: 22 August 2021

Accepted: 9 September 2021
Published: 17 September 2021 1. Introduction
Schizophrenia has long been considered a chronic mental illness that is predestined to
Publisher’s Note: MDPI stays neutral irreversible progressive deterioration [1]. However, the scientific evidence has shown that
with regard to jurisdictional claims in
the course and evolution of the illness are very heterogeneous [2,3], from cases that require
published maps and institutional affil-
several hospitalizations and have an important detriment of their functioning to cases that
iations.
only show one episode, followed by symptom remission and functional recovery. A recent
meta-analysis [4] showed that individuals with schizophrenia achieve symptom remission
in 56% of cases, while recovery, which includes satisfactory psychosocial functioning, is
achieved in around 30% of cases. Therefore, personal, social and occupational functioning
Copyright: © 2021 by the authors. is usually impaired in this population [5,6], but recovery is possible and should be a priority
Licensee MDPI, Basel, Switzerland. in the treatment of persons diagnosed with schizophrenia.
This article is an open access article
In view of this, an integrated and interdisciplinary care system is needed [7] where
distributed under the terms and
professionals from different fields who treat people diagnosed with this health condition
conditions of the Creative Commons
work together not only to address their symptoms, but also the difficulties they present in
Attribution (CC BY) license (https://
their daily functioning, their personal characteristics and the environmental factors that
creativecommons.org/licenses/by/
affect them.
4.0/).

J. Clin. Med. 2021, 10, 4223. https://doi.org/10.3390/jcm10184223 https://www.mdpi.com/journal/jcm

J. Clin. Med. 2021, 10, x 2 of 18

in their daily functioning, their personal characteristics and the environmental factors that
J. Clin. Med. 2021, 10, 4223 affect them. 2 of 17
In the case of individuals diagnosed with schizophrenia, the literature has shown
that the inclusion of psychiatrists [2,4], psychologists [8,9], nurses [10,11], occupational
therapists [12,13],
In the case ofsocial workers
individuals [14,15] and
diagnosed withphysiotherapists
schizophrenia, the [16,17] in interdisciplinary
literature has shown
mental health teams providing integrative care to this population has
that the inclusion of psychiatrists [2,4], psychologists [8,9], nurses [10,11], occupational substantial effects
improving
therapists clinical, socialworkers
[12,13], social and assistance outcomes
[14,15] and [18–20]. [16,17] in interdisciplinary
physiotherapists
mental
Thishealth
shift inteams providing integrative
the therapeutic approachcare to this population
highlights the need for hasa substantial effects
tool that can assess
improving clinical, social and assistance outcomes [18–20].
the full spectrum of difficulties in functioning that a person may have, and all the contex-
This shiftinvolved
tual variables in the therapeutic
and which approach highlights
facilitates the need for and
the coordination a tooljoint
that work
can assess
among the all
full spectrum of difficulties in functioning that a person may have,
the professions involved in the recovery process. Moreover, achieving integrated care and all the contextual
variables involved and which facilitates the coordination and joint work among all the
goals requires a common language and an understanding of the patient’s functioning
professions involved in the recovery process. Moreover, achieving integrated care goals
problems among interdisciplinary team members. The International Classification of
requires a common language and an understanding of the patient’s functioning problems
Functioning, Disability and Health (ICF [21]) covers all these requirements. The ICF, and
among interdisciplinary team members. The International Classification of Functioning,
the integrated
Disability andbiopsychosocial
Health (ICF [21])modelcoverson allwhich it is based, represent
these requirements. The ICF,aand comprehensive
the integratedand
universally accepted framework for describing functioning, disability
biopsychosocial model on which it is based, represent a comprehensive and universally and health in per-
sons with all types of health conditions. The ICF considers that problems
accepted framework for describing functioning, disability and health in persons with all associated with
a types
disease of can be conditions.
health related to Body functions,
The ICF Bodythat
considers structures,
problems and Activitieswith
associated andaParticipation
disease can in
community
be related to life,
Bodywhich in turn
functions, arestructures,
Body influenced byActivities
and Environmental factors andinPersonal
and Participation communityfactors.
Each
life, of these
which incomponents is structured
turn are influenced hierarchically
by Environmental factorsinand
chapters
Personaland categories
factors. Each of(see
theseFig-
components
ure 1). is structured hierarchically in chapters and categories (see Figure 1).

Figure 1. The ICF structure.

Figure 1. The ICF structure.
Given that it has more than 1400 categories, ICF Core Sets (ICF-CSs) linked to certain
Given
health that it has
conditions havemore
beenthan 1400 categories,
developed. ICF consist
The ICF-CSs Core Sets
of a(ICF-CSs)
list of the linked to certain
most relevant
health conditions
ICF categories forhave been developed.
the description The ICF-CSs
of the functioning andconsist of aoflist
disability of theliving
persons mostwith
relevant
a
ICF categories
given for the description
health condition. In the caseofofthe functioning two
schizophrenia, and versions
disabilityofofICF-CSs
personshaveliving with
been
a developed
given healthfollowing the methodology
condition. In the case ofendorsed by the World
schizophrenia, Health Organization
two versions of ICF-CSs [22]:
havethe
been
brief and the comprehensive [23]. The Comprehensive ICF-CS for schizophrenia
developed following the methodology endorsed by the World Health Organization [22]: includes
97 brief
the categories covering
and the the typical spectrum
comprehensive [23]. TheofComprehensive
problems in the ICF-CS
functioningfor of patients with in-
schizophrenia
schizophrenia. The Brief ICF-CS for schizophrenia is a selection of 25 of
cludes 97 categories covering the typical spectrum of problems in the functioning these 97 categories,
of pa-
those considered the most important for the assessment and treatment of
tients with schizophrenia. The Brief ICF-CS for schizophrenia is a selection of 25 of thesepeople with
J. Clin. Med. 2021, 10, 4223 3 of 17

schizophrenia. Both ICF-CSs can be consulted at https://www.icf-research-branch.org/icf-

core-sets-projects2/mental-health/icf-core-set-for-schizophrenia, accessed on 1 July 2021.
In order to apply the ICF-CSs in clinical practice, they must be validated through
various sources of evidence. Hence, the goal of the present study is to evaluate the
content validity of the ICF-CSs for schizophrenia from the perspective of experts in the
treatment of this population, and to identify, from an expert perspective, the potential
repercussions of this health condition in the functioning of persons diagnosed with this
disorder. It is hypothesized that the problems, resources, and environmental factors that
are represented in the ICF-CS for schizophrenia will coincide with those considered to be
the most relevant for understanding and assessing the functioning of this population from
the health professional perspective.

2. Materials and Methods

Six three-round Delphi studies were conducted with expert panels from different
professional backgrounds, which have played a significant role in the treatment of persons
with schizophrenia (i.e., psychiatry, psychology, nursing, occupational therapy, social work
and physiotherapy). The purpose of the Delphi technique is to reach the consensus of
a group of people possessing knowledge of a subject of interest (hereinafter referred to
as “experts”). This is a multi-level procedure in which a series of rounds are conducted
to gather information on a particular topic, so that each stage is built on the results of
the previous stage, providing anonymous feedback to each participant on the opinion
of the rest of the panel [24,25]. This methodology makes it easy to get the opinion of
numerous experts from different backgrounds, moving from individual opinions into
group consensus [26].
The Institutional Review Board Committee of the University of Barcelona approved
the Study IBR00003099.

2.1. Participants
We aimed to obtain a sample of experts that reflected the worldwide variability
of different variables considered to be interest: gender, age, years of experience and
demographic region of origin. To this end, experts from around the world were recruited
from a variety of sources, including through international associations of the analyzed
professions, universities with internship programs for health professionals, and hospitals.
Potential participants were also searched through a variety of bibliographical searches,
LinkedIn contacts, and personal recommendations of the contacted experts. All of them
were sent an initial invitation letter stating the criteria for participating in the study (i.e.,
being professionals in the specific profession of each Delphi study with at least one years’
experience in treating people diagnosed with schizophrenia). Specific knowledge of ICF
was not required, as the responses had to be based on clinical experience. They were
informed through a detailed description of the goals and the Delphi process, and they were
asked for their socio-demographic and professional information.
In total, 1555 healthcare professionals agreed to participate, and from this set, those
who fulfilled the inclusion criteria were invited to participate in the study (specifically,
443 psychiatrists, 223 psychologists, 160 nurses, 127 occupational therapists, 135 social
workers and 22 physiotherapists).
In the first round of Delphi studies, 790 experts participated (71.2% of whom were
invited to participate in the first round) from 85 different countries covering the six WHO
regions. Table 1 shows the details of their socio-demographic and professional char-
acteristics. A total of 638 participants (303 psychiatrists, 137 psychologists, 79 nurses,
73 occupational therapists, 36 social workers and 10 physiotherapists) completed the third
round (80.8% compared to the first round).
J. Clin. Med. 2021, 10, 4223 4 of 17

Table 1. Socio-demographic and professional characteristics of the total sample of participants in the Delphi studies.

Years of WHO Region Treated Population g

Age
Professional Round 1 Women Experience Eastern South-East Western Countries Round 3
Average African a Americas b European d Acute Chronic Rural Urban
Ambit n(%) n (%) Average Mediterranean c Asia e Pacific f n n (%) h
(Rank) n (%) n (%) n (%) n (%) n (%) n (%) n (%)
(Rank) n (%) n (%) n (%)
352 99 47.6 19.5 26 72 17 82 77 78 325 315 207 303 303
Psychiatry 63
(44.5) (28.1) (29–81) (4–55) (7.4) (20.4) (4.8) (23.3) (21.9) (22.2) (92.3) (89.5) (58.8) (86.1) (86.1)
175 110 41.8 11.7 11 47 21 63 20 13 92 149 60 130 137
Psychology 46
(22.2) (62.9) (24–67) (1–42) (6.3) (26.9) (12.0) (36.0) (11.4) (7.4) (52.6) (85.1) (34.3) (74.3) (78.3)
101 64 45.8 20.7 5 25 9 31 13 18 82 89 45 69 79
Nursing 30
(12.8) (63.3) (24–74) (2–54) (4.9) (24.7) (8.9) (30.6) (12.7) (17.8) (81.2) (88.2) (44.6) (68.3) (78.2)
Occupational 92 76 37.7 9.9 13 16 7 42 5 9 49 79 31 60 73
29
therapy (11.6) (82.6) (23–67) (1–44) (14.1) (17.4) (7.6) (45.7) (5.4) (9.8) (53.3) (85.9) (33.7) (65.2) (79.3)
57 39 45.1 10.3 2 17 1 13 11 13 28 53 24 43 36
Social work 20
(7.2) (68.4) (26–72) (1–27) (3.5) (29.8) (1.8) (22.8) (19.3) (22.8) (49.1) (93.0) (42.1) (75.4) (63.2)
13 7 43.2 10.5 2 7 1 3 7 12 3 7 10
Physiotherapy 0 0 8
(1.6) (53.8) (32–62) (1–30) (15.4) (53.8) (7.7) (23.1) (53.8) (92.3) (23.1) (53.8) (76.9)
307 45.5 15.8 57 179 55 238 127 134 583 697 370 612 638
Total 790 85
(48.1) (23–81) (1–55) (7.2) (22.6) (7.0) (30.1) (16.1) (17.0) (73.8) (88.2) (46.8) (77.5) (80.8)
a Participating countries in the African region: Algeria, Botswana, Ethiopia, Ghana, Kenya, Mozambique, Nigeria, South Africa, Uganda, Zimbabwe. b Participating countries in the Americas region: Argentina,
Brazil, Canada, Chile, Colombia, Costa Rica, Cuba, Ecuador, Mexico, United States of America, Uruguay, Venezuela. c Participating countries in the Eastern Mediterranean region: Egypt, Iran, Iraq, Jordan,
Kuwait, Lebanon, Libya, Marocco, Pakistan, Saudi Arabia, United Arab Emirates. d Participating countries in the European region: Armenia, Belgium, Bosnia and Herzegovina, Bulgaria, Croatia, Cyprus,
Czechia, Denmark, Finland, France, Georgia, Germany, Greece, Hungary, Iceland, Ireland, Israel, Italy, Latvia, Lithuania, Macedonia, Netherlands, Norway, Poland, Portugal, Romania, Russian Federation,
Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey, Ukraine, United Kingdom. e Participating countries in the South-East Asia region: Bangladesh, India, Indonesia, Nepal, Sri Lanka, Thailand.
f Participating countries in the Western Pacific region: Australia, Cambodia, China, Japan, Malaysia, New Zealand, Philippines, Singapore, Republic of Korea, Taiwan. g It was possible to choose more than one

option. h Regarding round 1.

J. Clin. Med. 2021, 10, 4223 5 of 17

2.2. Procedure
The six Delphi studies were conducted between 2016 and 2018. All studies followed
the same design to ensure a high level of comparability between the results of each, and it
has been detailed in the previous studies which describe the results obtained from each
perspective: psychiatry [27], psychology [28], nursing [29], occupational therapy [30],
social work [31] and physiotherapy [32]. Anonymity was guaranteed as the process was
coordinated by a research team using an online platform or email, thus avoiding any
interaction between the participants. The identity of the experts was not revealed to anyone
but the research team and the feedback given was anonymous (the percentage of yes/no
answers considering the expert panel as a whole), in order to ensure the independence of
the participants’ opinions.
Each of the studies lasted two months, from the beginning of the first round to the
end of the third round. The participants had two weeks to respond to each round. All the
material and questionnaires were presented in five different languages (namely Chinese,
English, French, Russian, and Spanish) in order to minimize potential language barriers
and encourage maximum participation in different world regions. The answers for each
Delphi round were collected via an online survey system (www.qualtrics.com, accessed on
1 July 2021).
For the first round, each participant was sent an email with a link to the survey
webpage, asking them to list all aspects they considered relevant when assessing and/or
treating people diagnosed with schizophrenia. To facilitate this, they were asked six open-
ended questions covering all the ICF-CS components. Responses were not limited in terms
of word length, although respondents were instructed to be brief and concise, and to avoid
using abbreviations and vague technical terms. The general procedure followed in each
Delphi study and the verbatim questions asked in the first round can be consulted in
Figure 2.
Responses to the first round were linked to ICF categories by two health professionals
following standardized rules [33,34]. Any disagreement between the two independent
coders was reviewed and discussed by two other health professionals in order to reach
a consensus. As Personal factors are not yet categorized in the current ICF system, the
proposed categorization of Nuño et al. [27] was followed. Those ICF categories and
Personal factors reported by at least 5% of the participants were selected for inclusion in
the second and third Delphi rounds. If any category of ICF-CSs for schizophrenia was not
included in this list, it was added.
In the second round, all the panelists who had responded in the first round were
sent a list of the selected ICF categories, as well as a list of the categories proposed for
Personal factors, together with their respective definitions. For example, for the category
b140 Attention functions, the ICF definition (i.e., specific mental functions of focusing on an
external stimulus or internal experience for the required period of time) and inclusions of
the category (i.e., functions of sustaining attention, shifting attention, dividing attention,
sharing attention; concentration; distractibility) were detailed. Participants were asked
to judge, for each category, whether or not they considered the category to be relevant
from their professional perspective for the evaluation and/or treatment of persons with
schizophrenia. They were reminded that the aim was to obtain a final list short enough to be
applicable in clinical practice and sufficiently comprehensive to cover the most important
needs of people with schizophrenia.
Finally, in the third round, participants were asked to re-evaluate the same list of
categories, this time taking into account the feedback they had received on the responses
from the expert group as a whole and their own in the previous round.
J.J. Clin.
Clin. Med.
Med. 2021,
2021, 10,
10, 4223
x 66 of 18
of 17

Figure 2. General procedure followed in each Delphi study.

Figure 2. General procedure followed in each Delphi study.
J. Clin. Med. 2021, 10, 4223 7 of 17

2.3. Data Analysis

A descriptive analysis of the socio-demographic characteristics of the participants
was conducted. Kappa coefficients and 95% bootstrapped confidence intervals (95% CI)
were calculated to evaluate inter-coder reliability in the linking process of participants’
responses. For each Delphi study, the percentage of participants who selected each category
as relevant in the second and third rounds was calculated. In the absence of a universally
accepted definition of consensus [35], and given the experience of previous studies [36],
the consensus was defined as an agreement among at least 75% of participants in the
third round.
When performing the joint analysis of results, all categories that had reached consensus
from the perspective of at least one profession were considered. Expert consensus was
defined as an agreement among more than half of the professional perspectives considered
(i.e., consensus by four or more expert perspectives for to that category).
All variable data were coded in Excel (2016) and analyzed using the Statistical Package
for Social Sciences (SPSS, version 24) [37].

3. Results
3.1. Selected Categories from the Experts’ Perspective
In the first round, 20,551 concepts were extracted from the responses of the experts,
which were linked to ICF categories and Personal factors. As a result of this process, different
sets of categories were presented to each professional group according to the categories
they had identified in the first round. Specifically, in the second and third rounds, between
110 and 135 ICF categories and between 24 and 35 Personal factors were presented to each
expert group. Of these, 90 ICF categories and 28 Personal factors achieved expert consensus
(reached consensus from the perspective of four or more professions).

3.2. Correspondence between Categories Which Reached Expert Consensus and the ICF-CSs for
Schizophrenia
All the categories in the brief version of the ICF-CS for schizophrenia achieved con-
sensus from all professional perspectives considered. Therefore, we will mainly focus on
the comprehensive version of the ICF-CS for schizophrenia. All categories of the ICF-CS
achieved consensus from the perspective of at least one professional group, and 89.7%
(87 categories) achieved expert consensus.
More detailed information on the categories that achieved expert consensus and their
correspondence with the categories present in the ICF-CS for schizophrenia can be found in
Table 2. Moreover, the discrepancies between the results and the ICF-CS for schizophrenia
are shown in Table 3, which details the specific categories that did not match between the
set of categories that achieved expert consensus and the whole categories present in the
ICF-CS for schizophrenia.

Table 2. Number of categories that reached expert consensus and comparison with the categories included in the complete
version of the ICF-CS for schizophrenia.

Body Body Activities and Environmental

Number of Categories Total
Functions Structures Participation Factors
Categories that achieved consensus from at
21 2 50 40 113
least one professional perspective
Categories in the ICF-CS for schizophrenia 17 0 48 32 97
Categories that achieved expert consensus a 17 1 39 33 90
ICF-CS categories for which expert consensus
16 0 39 32 87
was achieved
a Expert consensus: consensus by four or more expert perspectives regarding that category.
J. Clin. Med. 2021, 10, 4223 8 of 17

Table 3. Categories that did not match between the set of categories that achieved expert consensus and the ICF-CS
for schizophrenia.

Number of
Perspectives from
Perspectives from
ICF Component ICF Category Achieving
Which Achieved
Consensus
Consensus
b126 Temperament and
Categories that Body functions All 6
personality functions
achieved expert
Body structures s110 Structure of brain All 6
consensus a and are not
present in the ICF-CS e135 Products and technology
Environmental factors PC, PS, NS, OT 4
for employment
b530 Weight maintenance
Body functions PC, NS, PH 3
functions
d330 Speaking SW, PH 2
ICF-CS categories for d475 Driving OT, PH 2
schizophrenia that did d510 Washing oneself OT, SW 2
not achieve expert d540 Dressing NS, OT 2
Activities and
consensus d166 Reading PH 1
Participation
d210 Undertaking a single task OT 1
d470 Using transportation OT 1
d860 Basic economic
OT 1
transactions
d930 Religion and spirituality OT 1
aExpert consensus: consensus by four or more expert perspectives regarding that category. PC: psychiatry; PS: psychology; NS: nurses;
OT: occupational therapy; SW: social work; PH: physiotherapy.

3.3. Personal Factors

Of all the Personal factors identified in the Delphi studies, 28 reached expert consensus,
and 12 of them achieved consensus from all perspectives. The categories proposed as
Personal factors that reached expert consensus, with information on from which professional
perspectives they were considered and whether they reached consensus on each of them,
can be consulted in S2.

4. Discussion
Through these studies, we identified the problems, resources, and environmental
factors that health professionals most frequently encounter when treating people with
schizophrenia. All the categories that form part of the ICF-CSs for schizophrenia achieved
agreement from the perspective of at least one profession, and 89.7% of the categories of the
comprehensive version achieved expert consensus. Moreover, 100% of the categories that
make up the brief version of the ICF-CS achieved consensus from all of the perspectives
considered. All this supports the high relevance of the categories that form part of the brief
version. The following discussion will therefore focus on the joint analysis of the expert
perspective in comparison with the comprehensive version of the ICF-CS for schizophrenia.

4.1. Body Functions

With regard to the Body functions component, 17 categories achieved expert consensus,
coinciding with 16 of the 17 categories from this component represented in the ICF-CS for
schizophrenia. Twelve of them achieved consensus from the perspective of all the expert
groups considered, and all these categories belong to chapter b1 Mental functions, highlight-
ing the relevance of this chapter for defining functioning in people with schizophrenia from
the perspective of all health professions. All professionals agreed—with agreement from
each perspective being higher than 95%—that categories referring to classical symptoms in
schizophrenia, such as delusions and hallucinations (e.g., b156 Perceptual functions and b160
Thought functions), negative symptoms (b130 Energy and drive functions and b152 Emotional
J. Clin. Med. 2021, 10, 4223 9 of 17

functions), and other typical alterations such as cognitive deficits (b140 Attention functions
and b164 Higher-level cognitive functions) and psychosocial functions (b122 Global psychosocial
functions), are crucial to consider when evaluating and treating this population.
These results also highlight the need for an interdisciplinary approach to all these
functions. Interventions from the different professional profiles have proved to be effective
for improving these areas of functioning. For example, antipsychotic treatment is the
first-choice treatment for reducing positive symptoms and achieving remission of 80%
of symptoms after the first year of treatment [38], but cognitive behavioral therapy [39],
interventions of occupational therapy [12] and nursing [40] or exercises promoted by
physiotherapists [41] have also been shown to be effective in reducing such symptoms.
In this regard, social workers may also contribute to alleviating psychotic symptoms
by conducting appropriate assessments and referrals [42]. Cognitive function is mainly
improved through cognitive remediation therapy [43,44], but it has also been proven
that it can improve through medical interventions [45], occupational therapy [46] and
physiotherapeutic interventions [16]. These are some examples of how interventions
carried out from each professional perspective can produce effects in these affected areas.
However, these positive effects are even higher when applied by means of an integrated
care approach, in which professionals from the different areas work together in a systematic
way with the same objective [19,20].
It is also worth noting that the category b126 Temperament and personality functions
achieved consensus from all of the perspectives considered, yet it is not represented in the
ICF-CS for schizophrenia. This high rate of agreement is consistent with many studies in
the literature that support the fact that this area may be affected in this population [47–49],
and, therefore, its addition to the ICF-CS for schizophrenia should be considered.
Only one category from the Body functions component represented in the ICF-CS did
not achieve consensus from the majority of the perspectives considered. This category was
b530 Weight maintenance functions, which only reached consensus from the perspective of
psychiatrists, nurses and physiotherapists. This suggests that this category is relevant to the
assessment of, and interventions for, persons with schizophrenia, but may not be the most
common target of the interventions of certain professionals, such as psychologists, which
focus primarily on mental rather than other body functions, or OTs, which focus mainly
on helping people recover and participate in significant life roles. However, weight gain
and obesity are very prevalent in this population, increase the risk of weight-related health
problems, such as adult-onset diabetes mellitus and cardiovascular disorders, and are
related to reduced adherence with pharmacological interventions and quality of life [50].
Professionals such as psychologists, occupational therapists and social workers could
play a significant role in reducing its incidence by promoting healthy habits and lifestyles
in individuals with schizophrenia. In fact, first choice interventions to decrease and
manage weight gain in this population are psychoeducation, diet, and physical activity
interventions [51]. Therefore, these results show the need to raise awareness in certain
professions about the relevance of weight-related health problems and the importance of
intervening in them.

4.2. Body Structures

The ICF-CS for schizophrenia does not include any category from the Body structures
component. However, all the expert panels agreed that brain structure was altered in
individuals with schizophrenia, with agreement higher than 90% from all the perspectives
in the category s110 Brain structure. This is supported by the literature, which suggests that
the brain is the principle altered structure in this illness [52] and that other dysfunctions,
such as neuropsychological impairment, are related to its malfunctioning [53–55]. It is well
known that schizophrenia is associated with abnormal structural and functional connec-
tivity [56], although this can be partially restored by antipsychotic medication [45,57]. In
this regard, nurses also play an important role in promoting medication adherence [58],
and they may be among the first to detect non-adherence and non-attendance at follow-up
J. Clin. Med. 2021, 10, 4223 10 of 17

visits [59]. Neurocognitive and social cognitive interventions also aim to improve cerebral
functioning [60–62]. Psychological interventions produce changes in brain structure and its
functioning [63,64], this being the goal of interventions such as cognitive remediation. The
interventions have also been related to improvements in neurocognitive functioning [46],
and could therefore influence these brain structures. Exercise also has important neuro-
biological effects [65]. The coordination of all these services and professionals by social
workers will be crucial for improving cognitive outcomes [66]. Enhancement of this kind,
in turn, leads to improved neurocognition and social cognition and a reduction in the
negative symptoms [67–69], which is essential for functional recovery in individuals with
schizophrenia. Therefore, from the expert perspective, the inclusion of this category (s110
Structure of brain) in the ICF-CS for schizophrenia should be considered. Nevertheless, we
must bear in mind that many other categories that form part of the ICF-CS for schizophre-
nia do indirectly take this structure into account (for example, cognitive functions), given
the correspondence between these functions and the underlying structures.

4.3. Activities and Participation

The Activities and Participation component is the one with the largest number of
categories achieving expert consensus. In total, 39 categories achieved expert consensus,
and of these, 31 achieved consensus among all professional panels considered, with seven
of them achieving an average agreement higher than 95%.
The categories selected covered all the chapters of this component and focused es-
pecially on chapters d6 Domestic life, d7 Interpersonal interactions and relationships, and d8
Major life areas, such as education and employment. Experts also emphasized the possible
problems in areas covered by the chapters d1 Learning and applying knowledge, d2 General
tasks and demands, and d5 Self-care. All the categories of this component for which consen-
sus was reached are included in the ICF-CS for schizophrenia. This reflects the fact that
schizophrenia may have major implications for everyday functioning in all these areas [70]
and illustrates why the main long-term therapeutic goals of all health interventions for
individuals with schizophrenia should go beyond remission of specific symptoms and
focus on improving social functioning [71].
There are nine categories from the ICF-CS for schizophrenia that did not achieve
expert consensus. These categories mainly related to simple activities, such as d860 Basic
economic transactions, whereas consensus was achieved for the equivalent more complex
categories (e.g., d865 Complex economic transactions). These results offer a more positive
view of the abilities of people with schizophrenia, since they suggest that their difficulties
mainly depend on the complexity of the task at hand.
Managing to restore the psychosocial functioning of individuals with schizophrenia
through their performance in key areas of everyday activities, social achievement and
social competence is the cornerstone of functional recovery [72,73]. All the professions play
a key role in achieving this goal. Psychiatric medication has shown a positive effect on the
functioning and quality of life of individuals with schizophrenia [74]. Different approaches
of psychological treatment, such as cognitive-behavioral therapy, social skills training,
cognitive remediation and social cognition training, have also been shown to be effective
in improving psychosocial functioning and participation in community activities [75].
Nursing interventions can also enhance social functioning, for example, through shared
decision making, which has been suggested to improve social perception [76]. Social work
also plays a central role in the identification of problems that affect one in carrying out daily
activities and in the implementation of individualized and collaborative intervention plans
guided by an individual’s lifestyle and preferences [77]. Several studies have shown that the
intervention of OTs enhances social functioning (improving interpersonal communication,
in particular), the realization of daily activities, and working life [78]. Physiotherapists can
also improve one’s ability to perform several daily activities, and their intervention has
also been related to better functional outcomes [17].
J. Clin. Med. 2021, 10, 4223 11 of 17

Multiple interventions are therefore needed to achieve functional recovery, requiring

integrated care and interdisciplinary intervention [19,79]. Ultimately, being able to perform
daily activities and to participate in community life is the main goal of all people, and it
is therefore not surprising that this component represents a priority area of work and a
treatment objective of all health interventions.

4.4. Environmental Factors

The component with the second-highest number of categories showing consensus
was Environmental factors: 33 achieved expert consensus, and 21 achieved consensus from
all professional panels considered.
The agreed categories covered five of the six chapters of this component, with the only
chapter with no category achieving consensus being e2 Natural environment and human-made
changes to the environment. The selected categories concerned in particular the chapters
e3 Support and relationships, e4 Attitudes, and the accessibility of health services (chapter
e5 Services, systems and policies). These results confirm that health professionals attribute
particular importance to the impact of environmental factors on the functioning of a person
with schizophrenia [80,81]. In fact, authors such as Fleischhacker et al. [7] point out the
necessity of combining integrated care with active engagement on the part of people with
schizophrenia, their families, and their communities, highlighting the relevance of paying
attention to environmental circumstances. All these elements working together should
result in better lives for all those affected.
Family interventions, such as psychoeducation for relatives or multifamily group
therapy, have proven to be effective in improving psychosocial functioning and promoting
the well-being of people with schizophrenia, as well as that of their relatives [82,83]. Social
workers also try to intervene in families and in other contextual factors and in how these
factors affect the individual with schizophrenia [84], and nursing interventions have also
been shown to facilitate the achievement of adequate social and therapeutic support [11].
All the categories from this component that form part of the ICF-CS for schizophrenia
achieved expert consensus. Only one category from this component that achieved expert
consensus is not represented in the ICF-CS for schizophrenia (e135 Products and technology
for employment).
All the areas selected may provide health professionals with information about how
the individual functioning of persons with schizophrenia could be improved by promoting
enabling environmental factors and by reducing barriers. In spite of the importance of these
factors, they are usually ignored in the management of schizophrenia [85]. The ICF-CSs
can be useful in this regard, since they enable a thorough assessment of the environmental
barriers and facilitators that affect the daily functioning of a person with a specific health
condition, identifying the environmental aspects which merit intervention.

4.5. Personal Factors

In total, 28 Personal factors reached expert consensus. Of these, 14 achieved an aver-
age consensus higher than 95% across the perspectives achieving consensus. This high-
lights the relevance of considering personal characteristics in the treatment of individuals
with schizophrenia, as well as the importance attributed to this component by all the
professional perspectives.
Many studies support the relevance of the Personal factors that achieved a higher
consensus in our research, such as age at onset [86], premorbid drug use and lifestyle [87],
premorbid social skills [88], premorbid cognitive skills [89] and personal history and
biography [90,91]. Other categories that achieved high consensus, which are also supported
by the literature, are resilience [92], genetic factors [93], premorbid personality [94] and
premorbid intelligence [95].
Health professionals can, in fact, influence some of these personal factors. For example,
psychosocial skills can be improved by psychologists (through social skills training or
social cognition training [75], and nurses can also help to promote them through specific
J. Clin. Med. 2021, 10, 4223 12 of 17

workshops [70]. Moreover, the knowledge of which personal factors may increase one’s
vulnerability to having a psychotic episode is very important given that it will facilitate
the identification of individuals who are at clinically high risk for schizophrenia as well
as then the application of preventive approaches [96]. In such cases, social work holds a
unique position when it comes to identifying these individuals and making appropriate
referrals [97].
Consequently, it would be useful for the ICF system to incorporate the classification of
this component, in order to enable the systematic identification of all personal factors that
influence the functioning of people with different health conditions, so that professionals
can describe them in a detailed and exhaustive manner. Most of the categories regarded
as important in our study coincide among the different perspectives of the various health
professions, which suggests that the proposed list of Personal factors captures the aspects
that merit particular consideration in this population.

4.6. Strengths, Limitations and Future Research

The present study has several strengths. First, the Delphi technique was applied in
such a way as to maximize its possibilities, by including a large number of participants
and facilitating worldwide participation. To our knowledge, this is the first time that an
ICF-CS validation study has been conducted on such a large and diverse sample. Moreover,
the sample was made up of experts highly qualified in the treatment of individuals with
schizophrenia, both acute and chronic, from both rural and urban settings. Another
strength of the study is that participation was possible in five languages, and this is likely
to have been a key factor in achieving such multicultural and multinational representation.
It should also be noted that the response rate across rounds one to three was between 63.2%
and 86.1%, which is considerably high for this kind of study. Finally, the proposed list of
Personal factors constitutes a valuable and innovative aspect of our study.
Nonetheless, some consideration should be given to certain limitations of the study’s
findings. The primary limitation concerns the representativeness of the panel of ex-
perts. Although individuals from all around the world took part, the African and Eastern
Mediterranean WHO regions were under-represented. Possible reasons for this under-
representation include greater difficulties in accessing these professionals and lower num-
bers of these kinds of specialized health professionals in these regions. Moreover, despite
our efforts to recruit a large and representative panel of experienced professionals across
all the areas, the number of physiotherapists and social workers willing to participate in
the Delphi study was relatively small. The sample of occupational therapists and nurses
was also small compared with the number of psychiatrists and psychologists. It is worth
noting that mental health nurses make up the bulk of the mental health workforce, yet only
160 out of 1555 mental health professionals that initially agreed to participate were nurses.
Although experts were recruited through many sources, including the scientific literature,
international associations, and LinkedIn, in many of these sources, all these professionals
remain underrepresented. This article attempts to be supportive by giving a voice to all
these professional groups, in order to promote a multidisciplinary and integrative health
care approach.
In summary, these Delphi studies have documented the areas and aspects that health
professionals consider important in relation to the assessment and treatment of individ-
uals with schizophrenia, and the results largely validate the ICF-CSs for schizophrenia.
Further validation studies from the perspective of families, caregivers and persons with
schizophrenia, as well as studies that examine other sources of evidence of validity, are
now needed in order to complement the present findings and to move towards a definitive
version of the ICF-CS for schizophrenia.

5. Conclusions
Overall, the present study provides important support for the worldwide content
validity of the ICF-CSs for schizophrenia from an expert perspective. The results high-
J. Clin. Med. 2021, 10, 4223 13 of 17

light the relevance, in the evaluation and treatment of individuals with schizophrenia,
of assessing functioning by considering the body functions, participation in activities,
environmental aspects and personal factors that experts have identified. All this suggests
that the ICF and these ICF-CSs provide an effective framework within which to evaluate
and describe functioning in people with schizophrenia and therefore may be a useful tool
in the comprehensive treatment of this population.

Author Contributions: Conceptualization, L.N.; Data curation, L.N.; Formal analysis, L.N.; Funding
acquisition, J.G.-B.; Investigation, L.N., G.G., E.R. and M.B.; Methodology, L.N., G.G., E.R. and M.B.;
Project administration, J.G.-B.; Supervision, E.R. and M.B.; Validation, L.N.; Visualization, J.G.-B.;
Writing—original draft, L.N.; Writing—review & editing, G.G., E.R., J.G.-B. and M.B. All authors
have read and agreed to the published version of the manuscript.
Funding: This research was funded by Spain’s Ministry of Economy and Competitiveness, grant
number PSI2015-67984; Spain’s Ministry of Science, grant number PID2019-109887GB-100 and
Agency for the Management of University and Research Grants of the Government of Catalonia,
grant number 2017SGR1681.
Institutional Review Board Statement: The study was conducted according to the guidelines of the
Declaration of Helsinki, and approved by the Institutional Review Board Committee of University of
Barcelona (IRB00003099).
Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.
Data Availability Statement: The data presented in this study are available on request from the
corresponding author. The data are not publicly available due to containing information that could
compromise the privacy of research participants.
Conflicts of Interest: The authors declare no conflict of interest. The funders had no role in the design
of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or
in the decision to publish the results.

References
1. Frese, F.J.; Knight, E.L.; Saks, E. Recovery from schizophrenia: With views of psychiatrists, psychologists, and others diagnosed
with this disorder. Schizophr. Bull. 2009, 35, 370–380. [CrossRef]
2. Van Eck, R.M.; Burger, T.J.; Vellinga, A.; Schirmbeck, F.; De Haan, L. The Relationship Between Clinical and Personal Recovery in
Patients with Schizophrenia Spectrum Disorders: A Systematic Review and Meta-analysis. Schizophr. Bull. 2018, 44, 631–642.
[CrossRef] [PubMed]
3. Vita, A.; Barlati, S. Recovery from schizophrenia. Curr. Opin. Psychiatry 2018, 31, 246–255. [CrossRef]
4. Lally, J.; Ajnakina, O.; Stubbs, B.; Cullinane, M.; Murphy, K.C.; Gaughran, F.; Murray, R.M. Remission and recovery from
first-episode psychosis in adults: Systematic review and meta-analysis of long-term outcome studies. Br. J. Psychiatry 2017, 211,
350–358. [CrossRef] [PubMed]
5. Kalin, M.; Kaplan, S.; Gould, F.; Pinkham, A.E.; Penn, D.L.; Harvey, P.D. Social cognition, social competence, negative symptoms
and social outcomes: Inter-relationships in people with schizophrenia. J. Psychiatr. Res. 2015, 68, 254–260. [CrossRef]
6. Penadés, R.; Pérez-García, H.; González-Rodríguez, A.; Catalán, R.; Bernardo, M. Treating Cognition in Schizophrenia: Searching
for the Best Evidence-Based Practices. 2016. Available online: https://pdfs.semanticscholar.org/3a5f/dd2e9c2819a3cc1252a0d3
b30be2317544fc.pdf (accessed on 1 July 2021).
7. Fleischhacker, W.W.; Arango, C.; Arteel, P.; Barnes, T.R.E.; Carpenter, W.; Duckworth, K.; Galderisi, S.; Halpern, L.; Knapp, M.;
Marder, S.R.; et al. Schizophrenia-Time to commit to policy change. Schizophr. Bull. 2014, 40, S165–S194. [CrossRef]
8. Jablensky, A.; Castle, D.J.; Dark, F.; Humberstone, V.; Killackey, E.; Kulkarni, J.; McGorry, P.; Morgan, V.A.; Nielssen, O.; Tran,
N.; et al. The 2016 RANZCP Guidelines for the Management of Schizophrenia and Related Disorders—What’s next? Australas.
Psychiatry 2017, 25, 600–602. [CrossRef] [PubMed]
9. Kendall, T.; Whittington, C.J.; Kuipers, E.; Johnson, S.; Birchwood, M.J.; Marshall, M.; Morrison, A.P. NICE vs. SIGN on psychosis
and schizophrenia: Same roots, similar guidelines, different interpretations. Br. J. Psychiatry 2016, 208, 316–319. [CrossRef]
[PubMed]
10. Mahone, I.H.; Maphis, C.F.; Snow, D.E. Effective Strategies for Nurses Empowering Clients with Schizophrenia: Medication Use
as a Tool in Recovery. Issues Ment. Health Nurs. 2016, 37, 372–379. [CrossRef]
11. Virgolesi, M.; Pucciarelli, G.; Colantoni, A.M.; D’Andrea, F.; Di Donato, B.; Giorgi, F.; Landi, L.; Salustri, E.; Turci, C.; Proietti, M.G.
The effectiveness of a nursing discharge programme to improve medication adherence and patient satisfaction in the psychiatric
intensive care unit. J. Clin. Nurs. 2017, 26, 4456–4466. [CrossRef]
J. Clin. Med. 2021, 10, 4223 14 of 17

12. Foruzandeh, N.; Parvin, N. Occupational therapy for inpatients with chronic schizophrenia: A pilot randomized controlled trial.
Japan J. Nurs. Sci. 2013, 10, 136–141. [CrossRef]
13. Shimada, T.; Ohori, M.; Inagaki, Y.; Shimooka, Y.; Ishihara, I.; Sugimura, N.; Tanaka, S.; Kobayashi, M. Effect of adding
individualized occupational therapy to standard care on rehospitalization of patients with schizophrenia: A 2-year prospective
cohort study. Psychiatry Clin. Neurosci. 2019, 73, 476–485. [CrossRef]
14. de Saxe Zerden, L.; Lombardi, B.M.; Guan, T. Integrated Behavioral Health and Social Work: A Global Perspective. Glob. Soc.
Welf. 2019, 6, 49–56. [CrossRef]
15. Fraser, M.W.; Lombardi, B.M.; Wu, S.; de Saxe Zerden, L.; Richman, E.L.; Fraher, E.P. Integrated primary care and social work: A
systematic review. J. Soc. Soc. Work Res. 2018, 9, 175–215. [CrossRef]
16. Firth, J.; Stubbs, B.; Rosenbaum, S.; Vancampfort, D.; Malchow, B.; Schuch, F.; Elliott, R.; Nuechterlein, K.H.; Yung, A.R. Aerobic
exercise improves cognitive functioning in people with schizophrenia: A systematic review and meta-analysis. Schizophr. Bull.
2017, 43, 546–556. [CrossRef] [PubMed]
17. Vancampfort, D.; Nyboe, L.; Stubbs, B. Physiotherapy within the Multidisciplinary Treatment of Schizophrenia. In Physiotherapy
in Mental Health and Psychiatry: A Scientific and Clinical Based Approach; Probst, M., Skjaerven, L.H., Eds.; Elsevier: Edinburg,
Scotland, 2017; pp. 259–263.
18. Amore, K.; Howden-Chapman, P.L. Mental Health and Homelessness. In International Encyclopedia of Housing and Home; University
of Otago: Wellington, New Zealand, 2012; pp. 268–273. ISBN 9780080471716. [CrossRef]
19. Schöttle, D.; Schimmelmann, B.G.; Ruppelt, F.; Bussopulos, A.; Frieling, M.; Nika, E.; Nawara, L.A.; Golks, D.; Kerstan, A.; Lange,
M.; et al. Effectiveness of integrated care including therapeutic assertive community treatment in severe schizophrenia-spectrum
and bipolar I disorders: Four-year follow-up of the ACCESS II study. PLoS ONE 2018, 13, 1–14. [CrossRef] [PubMed]
20. She, S.; Deng, Y.; Chen, Y.; Wu, C.; Yi, W.; Lu, X.; Chen, X.; Li, J.; Li, R.; Zhang, J.; et al. Two-stage integrated care versus
antipsychotic medication alone on outcomes of schizophrenia: One-year randomized controlled trial and follow-up. Psychiatry
Res. 2017, 254, 164–172. [CrossRef]
21. World Health Organization. International Classification of Functioning, Disability and Health; World Health Organization: Geneva,
Switzerland, 2001; ISBN 9241545429.
22. Selb, M.; Escorpizo, R.; Kostanjsek, N.; Stucki, G.; Üstün, T.B.; Cieza, A. A guide on how to develop an International Classification
of Functioning, Disability and Health Core Set. Eur. J. Phys. Rehabil. Med. 2015, 51, 105–117. [PubMed]
23. Gómez-Benito, J.; Guilera, G.; Barrios, M.; Rojo, E.; Pino, O.; Gorostiaga, A.; Balluerka, N.; Hidalgo, M.D.; Padilla, J.L.J.L.; Benítez,
I.; et al. Beyond diagnosis: The Core Sets for persons with schizophrenia based on the World Health Organization’s International
Classification of Functioning, Disability, and Health. Disabil. Rehabil. 2018, 40, 2756–2766. [CrossRef]
24. Hsu, C.-C.; Sandford, B.A. The Delphi Technique: Making Sense of Consensus. Pract. Assess. Res. Eval. 2007, 12, 1–8.
25. Trevelyan, E.G.; Robinson, N. Delphi methodology in health research: How to do it? Eur. J. Integr. Med. 2015, 7, 423–428.
[CrossRef]
26. Yousuf, M.I. Using experts’ opinions through Delphi technique. Pract. Assess. Res. Eval. 2007, 12, 4. [CrossRef]
27. Nuño, L.; Barrios, M.; Rojo, E.; Gómez-Benito, J.; Guilera, G. Validation of the ICF Core Sets for schizophrenia from the perspective
of psychiatrists: An international Delphi study. J. Psychiatr. Res. 2018, 103, 134–141. [CrossRef] [PubMed]
28. Nuño, L.; Guilera, G.; Coenen, M.; Rojo, E.; Gómez-Benito, J.; Barrios, M. Functioning in schizophrenia from the perspective of
psychologists: A worldwide study. PLoS ONE 2019, 14, e0217936. [CrossRef]
29. Nuño, L.; Barrios, M.; Moller, M.D.; Calderón, C.; Rojo, E.; Gómez-Benito, J.; Guilera, G. An international survey of Psychiatric-
Mental-Health Nurses on the content validity of the International Classification of Functioning, Disability and Health Core Sets
for Schizophrenia. Int. J. Ment. Health Nurs. 2019, 28, 867–878. [CrossRef]
30. Nuño, L.; Guilera, G.; Bell, M.; Rojo, E.; Gómez-Benito, J.; Calderón, C.; Barrios, M. An Occupational therapist perspective of the
ICF Core Sets for Schizophrenia. Am. J. Occup. Ther. 2021, 75. [CrossRef]
31. Nuño, L.; Guilera, G.; Solomon, P.; Gómez-Benito, J.; Barrios, M. The Perspective of Social Workers on Functioning for Individuals
with Schizophrenia: A Delphi Study. J. Soc. Social Work Res. 2021, 12. [CrossRef]
32. Nuño, L.; Barrios, M.; Vancampfort, D.; Rojo, E.; Gómez-Benito, J.; Guilera, G. Functioning in schizophrenia: A Delphi study
covering the perspective of physiotherapists. Disabil. Rehabil. 2020. [CrossRef]
33. Cieza, A.; Fayed, N.; Bickenbach, J.; Prodinger, B. Refinements of the ICF Linking Rules to strengthen their potential for
establishing comparability of health information. Disabil. Rehabil. 2019, 41, 574–583. [CrossRef]
34. Cieza, A.; Geyh, S.; Chatterji, S.; Kostanjsek, N.; Üstün, B.; Stucki, G. ICF linking rules: An update based on lessons learned. J.
Rehabil. Med. 2005, 37, 212–218. [CrossRef]
35. Diamond, I.R.; Grant, R.C.; Feldman, B.M.; Pencharz, P.B.; Ling, S.C.; Moore, A.M.; Wales, P.W. Defining consensus: A systematic
review recommends methodologic criteria for reporting of Delphi studies. J. Clin. Epidemiol. 2014, 67, 401–409. [CrossRef]
36. Kaech Moll, V.M.; Escorpizo, R.; Portmann Bergamaschi, R.; Finger, M.E. Validation of the Comprehensive ICF Core Set for
Vocational Rehabilitation from the Perspective of Physical Therapists: International Delphi Survey. Phys. Ther. 2016, 96, 1262–1275.
[CrossRef] [PubMed]
37. IBM Corp. IBM SPSS Statistics for Windows, Version 24.0.2016; IBM Corp: Armonk, NY, USA, 2016.
38. Zipursky, R.B.; Agid, O. Recovery, not progressive deterioration, should be the expectation in schizophrenia. World Psychiatry
2015, 14, 94–96. [CrossRef]
J. Clin. Med. 2021, 10, 4223 15 of 17

39. Nowak, I.; Sabariego, C.; Świtaj, P.; Anczewska, M. Disability and recovery in schizophrenia: A systematic review of cognitive
behavioral therapy interventions. BMC Psychiatry 2016, 16. [CrossRef] [PubMed]
40. Tham, X.C.; Xie, H.; Chng, C.M.L.; Seah, X.Y.; Lopez, V.; Klainin-Yobas, P. Exploring predictors of medication adherence among
inpatients with schizophrenia in Singapore’s mental health settings: A non-experimental study. Arch. Psychiatr. Nurs. 2018, 32,
536–548. [CrossRef]
41. Gorczynski, P.; Faulkner, G. Exercise therapy for schizophrenia. Schizophr. Bull. 2010, 36, 665–666. [CrossRef]
42. Ashcroft, R.; Mcmillan, C.; Ambrose-Miller, W.; Mckee, R.; Brown, J.B. The Emerging Role of Social Work in Primary Health Care:
A Survey of Social Workers in Ontario Family Health Teams. Health Soc. Work. 2018, 109–118. [CrossRef]
43. Best, M.W.; Bowie, C.R. A review of cognitive remediation approaches for schizophrenia: From top-down to bottom-up, brain
training to psychotherapy. Expert Rev. Neurother. 2017, 17, 713–723. [CrossRef] [PubMed]
44. Vita, A.; Barlati, S.; Ceraso, A.; Nibbio, G.; Ariu, C.; Deste, G.; Wykes, T. Effectiveness, Core Elements, and Moderators of
Response of Cognitive Remediation for Schizophrenia: A Systematic Review and Meta-analysis of Randomized Clinical Trials.
JAMA Psychiatry 2021, 78, 848–858. [CrossRef]
45. Kani, A.S.; Shinn, A.K.; Lewandowski, K.E.; Öngür, D. Converging effects of diverse treatment modalities on frontal cortex in
schizophrenia: A review of longitudinal functional magnetic resonance imaging studies. J. Psychiatr. Res. 2017, 84, 256–276.
[CrossRef]
46. Shimada, T.; Nishi, A.; Yoshida, T.; Tanaka, S.; Kobayashi, M. Development of an Individualized Occupational Therapy
Programme and its Effects on the Neurocognition, Symptoms and Social Functioning of Patients with Schizophrenia. Occup. Ther.
Int. 2016, 23, 425–435. [CrossRef]
47. Díaz-Caneja, C.M.; Morón-Nozaleda, M.G.; Vicente-Moreno, R.P.; Rodríguez-Toscano, E.; Pina-Camacho, L.; de la Serna, E.;
Sugranyes, G.; Baeza, I.; Romero, S.; Sánchez-Gistau, V.; et al. Temperament in child and adolescent offspring of patients with
schizophrenia and bipolar disorder. Eur. Child Adolesc. Psychiatry 2018, 27, 1459–1471. [CrossRef]
48. Rettew, D.C.; McKee, L. Temperament and Its Role in Developmental Psychopathology. Harv. Rev. Psychiatry 2005, 13, 14–27.
[CrossRef] [PubMed]
49. Sevilla-Llewellyn-Jones, J.; Cano-Domínguez, P.; De-Luis-Matilla, A.; Espina-Eizaguirre, A.; Moreno-Küstner, B.; Ochoa, S.
Personality traits in recent-onset-of-psychosis patients compared to a control sample by gender. Schizophr. Res. 2017, 195, 86–92.
[CrossRef]
50. Manu, P.; Dima, L.; Shulman, M.; Vancampfort, D.; De Hert, M.; Correll, C.U. Weight gain and obesity in schizophrenia:
Epidemiology, pathobiology, and management. Acta Psychiatr. Scand. 2015, 132, 97–108. [CrossRef] [PubMed]
51. Gurusamy, J.; Gandhi, S.; Damodharan, D.; Ganesan, V.; Palaniappan, M. Exercise, diet and educational interventions for
metabolic syndrome in persons with schizophrenia: A systematic review. Asian J. Psychiatr. 2018, 36, 73–85. [CrossRef] [PubMed]
52. Koychev, I.; El-Deredy, W.; Mukherjee, T.; Haenschel, C.; Deakin, J.F.W. Core dysfunction in schizophrenia: Electrophysiology
trait biomarkers. Acta Psychiatr. Scand. 2012, 126, 59–71. [CrossRef]
53. Chan, S.K.W.; Hui, C.L.M.; Chang, W.C.; Lee, E.H.M.; Chen, E.Y.H. Ten-year follow up of patients with first-episode schizophrenia
spectrum disorder from an early intervention service: Predictors of clinical remission and functional recovery. Schizophr. Res.
2019, 204, 65–71. [CrossRef]
54. Fitzsimmons, J.; Kubicki, M.; Shenton, M.E. Review of functional and anatomical brain connectivity findings in schizophrenia.
Curr. Opin. Psychiatry 2013, 26, 172–187. [CrossRef]
55. Ira, E.; Zanoni, M.; Ruggeri, M.; Dazzan, P.; Tosato, S.; Ira, E. COMT, neuropsychological function and brain structure in
schizophrenia: A systematic review and neurobiological interpretation. J. Psychiatry Neurosci. 2013, 38, 366–380. [CrossRef]
56. van den Heuvel, M.P.; Fornito, A. Brain Networks in Schizophrenia. Neuropsychol. Rev. 2014, 24, 32–48. [CrossRef]
57. Hutcheson, N.L.; Sreenivasan, K.R.; Deshpande, G.; Reid, M.A.; Hadley, J.; White, D.M.; Ver Hoef, L.; Lahti, A.C. Effective
connectivity during episodic memory retrieval in schizophrenia participants before and after antipsychotic medication. Hum.
Brain Mapp. 2015, 36, 1442–1457. [CrossRef]
58. Chang, Y.-T.; Tao, S.-G.; Lu, C.-L. Qualitative inquiry into motivators for maintaining medication adherence among Taiwanese
with schizophrenia. Int. J. Ment. Health Nurs. 2013, 22, 272–278. [CrossRef] [PubMed]
59. Morton, N.K.; Zubek, D. Adherence challenges and long-acting injectable antipsychotic treatment in patients with schizophrenia.
J. Psychosoc. Nurs. Ment. Health Serv. 2013, 51, 13–18. [CrossRef] [PubMed]
60. Cella, M.; Preti, A.; Edwards, C.; Dow, T.; Wykes, T. Cognitive remediation for negative symptoms of schizophrenia: A network
meta-analysis. Clin. Psychol. Rev. 2017, 52, 43–51. [CrossRef]
61. Kurtz, M.M.; Richardson, C.L. Social cognitive training for schizophrenia: A meta-analytic investigation of controlled research.
Schizophr. Bull. 2012, 38, 1092–1104. [CrossRef]
62. Roder, V.; Mueller, D.R.; Schmidt, S.J. Effectiveness of integrated psychological therapy (IPT) for schizophrenia patients: A
research update. Schizophr. Bull. 2011, 37, S71–S79. [CrossRef]
63. Mason, L.; Peters, E.R.; Dima, D.; Williams, S.C.; Kumari, V. Cognitive behavioral therapy normalizes functional connectivity for
social threat in psychosis. Schizophr. Bull. 2016, 42, 684–692. [CrossRef] [PubMed]
64. Penadés, R.; Pujol, N.; Catalán, R.; Massana, G.; Rametti, G.; García-Rizo, C.; Bargalló, N.; Gastó, C.; Bernardo, M.; Junqué, C.
Brain effects of cognitive remediation therapy in schizophrenia: A structural and functional neuroimaging study. Biol. Psychiatry
2013, 73, 1015–1023. [CrossRef] [PubMed]
J. Clin. Med. 2021, 10, 4223 16 of 17

65. Vancampfort, D.; Probst, M.; De Hert, M.; Soundy, A.; Stubbs, B.; Stroobants, M.; De Herdt, A. Neurobiological effects of physical
exercise in schizophrenia: A systematic review. Disabil. Rehabil. 2014, 36, 1749–1754. [CrossRef]
66. Cordeiro, K.; Foroughe, M.; Mastorakos, T. Primary Mental Health Care in the Family Health Team Setting: Tracking Patient Care
from Referral to Outcome. Can. J. Community Ment. Heal. 2015, 34, 51–65. [CrossRef]
67. Lindenmayer, J.-P.; McGurk, S.R.; Khan, A.; Kaushik, S.; Thanju, A.; Hoffman, L.; Valdez, G.; Wance, D.; Herrmann, E.
Improving social cognition in schizophrenia: A pilot intervention combining computerized social cognition training with
cognitive remediation. Schizophr. Bull. 2013, 39, 507–517. [CrossRef]
68. Mueller, D.R.; Schmidt, S.J.; Roder, V. One-year randomized controlled trial and follow-up of integrated neurocognitive therapy
for schizophrenia outpatients. Schizophr. Bull. 2015, 41, 604–616. [CrossRef]
69. Pinkham, A.E.; Harvey, P.D. Future directions for social cognitive interventions in schizophrenia. Schizophr. Bull. 2013, 39,
499–500. [CrossRef] [PubMed]
70. Quee, P.J.; Stiekema, A.P.M.; Wigman, J.T.W.; Schneider, H.; van der Meer, L.; Maples, N.J.; van den Heuvel, E.R.; Velligan,
D.I.; Bruggeman, R. Improving functional outcomes for schizophrenia patients in the Netherlands using Cognitive Adaptation
Training as a nursing intervention—A pilot study. Schizophr. Res. 2014, 158, 120–125. [CrossRef] [PubMed]
71. Bowie, C.R.; McGurk, S.R.; Mausbach, B.; Patterson, T.L.; Harvey, P.D. Combined cognitive remediation and functional skills
training for schizophrenia: Effects on cognition, functional competence, and real-world behavior. Am. J. Psychiatry 2012, 169,
710–718. [CrossRef]
72. Harvey, P.D.; Bellack, A.S. Toward a terminology for functional recovery in schizophrenia: Is functional remission a viable
concept? Schizophr. Bull. 2009, 35, 300–306. [CrossRef]
73. Slade, M.; Longden, E. Empirical evidence about recovery and mental health. BMC Psychiatry 2015, 15, 1–14. [CrossRef] [PubMed]
74. Bailey, K.P. Choosing between atypical antipsychotics: Weighing the risks and benefits. Arch. Psychiatr. Nurs. 2002, 16, S2–S11.
[CrossRef] [PubMed]
75. Kern, R.S.; Glynn, S.M.; Horan, W.P.; Marder, S.R. Psychosocial treatments to promote functional recovery in Schizophrenia.
Schizophr. Bull. 2009, 35, 347–361. [CrossRef]
76. Adams, J.R.; Drake, R.E. Shared decision-making and evidence-based practice. Community Ment. Health J. 2006, 42, 87–105.
[CrossRef]
77. Ashcroft, R.; Kourgiantakis, T.; Fearing, G.; Robertson, T.; Brown, J.B. Social Work’s Scope of Practice in Primary Mental Health
Care: A Scoping Review. Br. J. Soc. Work 2019, 49, 318–334. [CrossRef]
78. Morris, K.; Reid, G.; Spencer, S. Occupational therapy delivered by specialists versus non-specialists for people with schizophrenia.
Cochrane Database Syst. Rev. 2018. [CrossRef]
79. Juntapim, S.; Nuntaboot, K. Care of patients with schizophrenia in the community. Arch. Psychiatr. Nurs. 2018, 32, 855–860.
[CrossRef] [PubMed]
80. Brown, C.A.; Weisman de Mamani, A. The mediating effect of family cohesion in reducing patient symptoms and family distress
in a culturally informed family therapy for schizophrenia: A parallel-process latent-growth model. J. Consult. Clin. Psychol. 2018,
86, 1–14. [CrossRef] [PubMed]
81. Niendam, T.A.; Tully, L.M.; Iosif, A.-M.; Kumar, D.; Nye, K.E.; Denton, J.C.; Zakskorn, L.N.; Fedechko, T.L.; Pierce, K.M.
Enhancing early psychosis treatment using smartphone technology: A longitudinal feasibility and validity study. J. Psychiatr. Res.
2018, 96, 239–246. [CrossRef] [PubMed]
82. McFarlane, W.R. Family Interventions for Schizophrenia and the Psychoses: A Review. Fam. Process. 2016, 55, 460–482. [CrossRef]
83. Okpokoro, U.; Adams, C.E.; Sampson, S. Family intervention (brief) for schizophrenia. Cochrane Database Syst. Rev. 2014.
[CrossRef]
84. Fox, J.; Ramon, S.; Morant, N. Exploring the Meaning of Recovery for Carers: Implications for Social Work Practice. Br. J. Soc.
Work 2015, 45, i117–i134. [CrossRef]
85. Gurak, K.; De Mamani, A.W. Risk and protective factors, perceptions of family environment, ethnicity, and schizophrenia
symptoms. J. Nerv. Ment. Dis. 2016, 204, 570–577. [CrossRef]
86. O’Donoghue, B.; Lyne, J.; Madigan, K.; Lane, A.; Turner, N.; O’Callaghan, E.; Clarke, M. Environmental factors and the age at
onset in first episode psychosis. Schizophr. Res. 2015, 168, 106–112. [CrossRef]
87. Bhalla, I.P.; Stefanovics, E.A.; Rosenheck, R.A. Mental health multimorbidity and poor quality of life in patients with schizophrenia.
Schizophr. Res. 2018, 201, 39–45. [CrossRef] [PubMed]
88. Mahmood, Z.; Burton, C.Z.; Vella, L.; Twamley, E.W. Neuropsychological predictors of performance-based measures of functional
capacity and social skills in individuals with severe mental illness. J. Psychiatr. Res. 2018, 102, 201–206. [CrossRef] [PubMed]
89. Kendler, K.S.; Ohlsson, H.; Keefe, R.S.E.; Sundquist, K.; Sundquist, J. The joint impact of cognitive performance in adolescence
and familial cognitive aptitude on risk for major psychiatric disorders: A delineation of four potential pathways to illness. Mol.
Psychiatry 2018, 23, 1076–1083. [CrossRef] [PubMed]
90. Lee, E.E.; Martin, A.S.; Tu, X.; Palmer, B.W.; Jeste, D.V. Childhood Adversity and Schizophrenia: The Protective Role of Resilience
in Mental and Physical Health and Metabolic Markers. J. Clin. Psychiatry 2018, 79. [CrossRef] [PubMed]
91. Rajkumar, R.P. The Impact of Childhood Adversity on the Clinical Features of Schizophrenia. Schizophr. Res. Treatment 2015, 2015,
532082. [CrossRef]
J. Clin. Med. 2021, 10, 4223 17 of 17

92. Torgalsbøen, A.-K.; Fu, S.; Czajkowski, N. Resilience trajectories to full recovery in first-episode schizophrenia. Eur. Psychiatry
2018, 52, 54–60. [CrossRef] [PubMed]
93. van Erp, T.G.M.; Walton, E.; Hibar, D.P.; Schmaal, L.; Jiang, W.; Glahn, D.C.; Pearlson, G.D.; Yao, N.; Fukunaga, M.; Hashimoto, R.;
et al. Cortical Brain Abnormalities in 4474 Individuals with Schizophrenia and 5098 Control Subjects via the Enhancing Neuro
Imaging Genetics through Meta Analysis (ENIGMA) Consortium. Biol. Psychiatry 2018, 84, 644–654. [CrossRef]
94. Goghari, V.M. Personality dimensions in schizophrenia: A family study. Psychiatry Res. 2017, 251, 162–167. [CrossRef]
95. Leeson, V.C.; Sharma, P.; Harrison, M.; Ron, M.A.; Barnes, T.R.E.; Joyce, E.M. IQ trajectory, cognitive reserve, and clinical outcome
following a first episode of psychosis: A 3-year longitudinal study. Schizophr. Bull. 2011, 37, 768–777. [CrossRef]
96. Fusar-Poli, P.; Borgwardt, S.; Bechdolf, A.; Addington, J.; Riecher-Rössler, A.; Schultze-Lutter, F.; Keshavan, M.; Wood, S.;
Ruhrmann, S.; Seidman, L.J.; et al. The psychosis high-risk state: A comprehensive state-of-the-art review. JAMA 2013, 70,
107–120. [CrossRef] [PubMed]
97. DeVylder, J.E. Preventing Schizophrenia and Severe Mental Illness: A Grand Challenge for Social Work. Res. Soc. Work Pract.
2016, 26, 449–459. [CrossRef]