Multi Stakeholder platform framework PPPAVCD
Multi Stakeholder platform framework PPPAVCD
Multi Stakeholder platform framework PPPAVCD
Agricultural Systems
journal homepage: www.elsevier.com/locate/agsy
Review
H I G H L I G H T S G R A P H I C A L A B S T R A C T
A R T I C L E I N F O A B S T R A C T
Editor: Mark van Wijk CONTEXT: Multi-stakeholder partnerships (MSPs), which involve the collective action of various institutions
such as governments, private companies, NGOs, and donors, have become a popular approach for developing
Keywords: agricultural value chains to improve the sustainability of food systems. However, the growing body of literature
Systematic review presents a range of terms, concepts, and definitions related to MSPs, potentially causing confusion and obscuring
Multi-stakeholder partnerships
the distinctions between different types of partnerships. While the potential benefits of MSPs have been exten
Impacts
sively explored in the literature, there remains a need to investigate their realized impacts across regions to
Developing countries
enable a more robust synthesis.
OBJECTIVE: The objective of this study was to systematically review the literature on MSPs in agricultural
development, specifically examining the various concepts employed and assessing their realized impacts on
farmers, focusing on developing countries.
METHODS: We systematically reviewed the literature on MSPs and identified 147 studies dealing with a different
type of MSPs and 79 studies reporting the impact.
RESULTS AND CONCLUSIONS: We found that twelve terms have been used to describe MSPs. However, a single
MSP initiative may be referred to by different names in various documents, indicating a lack of consensus on the
defining characteristics of such partnerships. Positive impacts were reported on farmers’ economic, social, and
* Corresponding author at: School of Agriculture and Food Sustainability, The University of Queensland, Gatton Campus, Building 8117A, Gatton, QLD 4343,
Australia.
E-mail addresses: maryono@uq.edu.au, maryonomr@apps.ipb.ac.id (M. Maryono).
https://doi.org/10.1016/j.agsy.2023.103792
Received 1 May 2023; Received in revised form 9 October 2023; Accepted 20 October 2023
Available online 1 November 2023
0308-521X/© 2023 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
M. Maryono et al. Agricultural Systems 213 (2024) 103792
technological aspects, as well as environmental adaptation practices to address climate change. Nevertheless, the
reported impacts tend to focus primarily on economic, technological, and social dimensions while giving less
attention to environmental aspects.
SIGNIFICANCE: This paper presents, firstly, a comprehensive overview of the diverse practices of MSPs in
developing countries. Secondly, it analyses the contributions made by MSPs across various commodities and
regions in the development of agricultural value chains.
1. Introduction Experience from Africa indicates that MSPs’ benefits depend on the
crops grown and farmers’ access to fundamental resources such as land
Poverty, food security, and sustainability are critical issues in agri (Adam et al., 2018). There is a need for a more rigorous synthesis and
cultural development, particularly in developing countries (Timmer, evaluation of the impacts of MSPs implemented across regions to inform
2014; Demont and Rutsaert, 2017; Viteri Salazar et al., 2018; Turner adaptive management of the approach to understanding better ‘what
et al., 2020). The complexity of agricultural development problems re works’ in the process design and facilitation. More evidence on the
quires value chain actors and stakeholders to interact and collaborate to impact of MSPs is also needed for policymakers and donor organizations
achieve greater development outcomes (Sartas et al., 2018). Multi- to guide future development and implementation. This will facilitate the
stakeholder partnerships (MSPs) in agriculture denote a collective ac identification of interventions that can effectively improve the liveli
tion of various institutions such as government, private companies, hood of smallholder producers and promote environmentally friendly
NGOs, and donors intended to support developing and improving agri agricultural practices in response to climate change.
cultural value chains (Bäckstrand, 2006; Dentoni et al., 2018; Tinarwo While there are studies that have reviewed the contributions of
et al., 2018). A growing body of literature suggests that MSPs can be a MSPs, most of them tend to focus on a particular type of MSP or have
practical approach to solving economic, social, and environmental been applied to a specific country or limited to a few regions (Hussain
problems to achieve impact at scale (van Tulder et al., 2016; de Bakker and Perera, 2004; Poulton and Macartney, 2012; Ponnusamy, 2013;
et al., 2019; Totin et al., 2020). The approach has also been considered Schut et al., 2018; de Bakker et al., 2019; Mangeni, 2019; Barzola Iza
fundamental to achieving the sustainable development goals (SDGs) of et al., 2020; van Ewijk and Ros-Tonen, 2021). The most recent system
the United Nations 2030 Agenda (Rein and Stott, 2009; Eweje et al., atic review focused on the contribution of MSPs in sub-Saharan Africa
2020). and on those that enable knowledge co-creation (van Ewijk and Ros-
The emergence of MSPs over the last two decades has attracted the Tonen, 2021). Only some studies, if any, have adopted a systematic
attention of academics and practitioners as they seek to understand approach to analyze and synthesize the diversity of terminology used to
different practices under the broad umbrella of MSPs (Barzola Iza et al., reflect the adoption of MSPs. To fill the gap, we explore the diverse
2020). It is being studied from multiple and different streams of disci practices and evaluate the impact of MSPs by systematically reviewing
plinary angles and stakeholders’ viewpoints (van Tulder et al., 2016; the literature on the MSPs in agricultural development in developing
Barzola Iza et al., 2020), thus leading to a broad range of terms used to countries. We analyze and synthesize the findings of 147 studies for the
refer to the different forms of MSPs (Narrod et al., 2009; Cavatassi et al., definitions of MSPs and their contribution to the economic, social,
2011; Dentoni and Peterson, 2011; Köhne, 2014; Thiele et al., 2016; technological, and environmental aspects of smallholder producers in
Téno and Cadilhon, 2017; Hermans et al., 2019; Totin et al., 2020). developing countries.
Researchers often adopt terms, concepts, and definitions from different
disciplines with little reference to existing ones, leading to confusion and 2. Methods
blurring the boundaries between the different types of MSPs (van Tulder
et al., 2016). Therefore, clarifying and scrutinizing the different prac We conducted this systematic review following the Preferred
tices and terminology used under the MSPs framework is necessary. Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)
Various organizations have implemented MSPs. Since the early (Moher et al., 2009), which requires all steps in the review process to be
2000s, international development organizations have promoted part transparent and well-documented (Fig. 1). Unlike a narrative literature
nerships between public and private actors to provide public goods review, it uses a clear and systematic method to identify, select, and
(Rein and Stott, 2009; van Tulder et al., 2016). As a result, it has seen the analyze relevant documents from a set of formulated research questions
top one hundred largest firms in the world involved in partnerships with and pre-specified inclusion criteria to minimize the potential for selec
‘non-market’ actors to maintain long-term competitive advantage tion bias (Moher et al., 2009; Koutsos et al., 2019; Snyder, 2019). The
(Dentoni and Peterson, 2011; van Tulder et al., 2016; de Bakker et al., systematic review enables us to identify the diverse practices in imple
2019). Research organizations used MSPs to enhance the impact of menting MSPs and their contributions to agricultural development. It
research further than its initial development (Sartas et al., 2018; Schut also identifies research gaps in the literature and possible areas for
et al., 2019). For instance, using MSPs in agricultural innovation systems future research.
is expected to create an enabling environment for institutional and
technological innovation (Sartas et al., 2018). Numerous development
agencies, particularly in Africa, also promoted MSPs as drivers of change 2.1. Scope
to promote access to knowledge and resources, enhance learning, and
negotiate interest (Adekunle and Fatunbi, 2012; Adekunle et al., 2014; A specific set of inclusion and exclusion criteria were developed, as
Van Paassen et al., 2014). presented in Table 1. For inclusion, studies had to present a quantitative,
While the potential theoretical benefits of the MSPs have been qualitative, or conceptual analysis of the collaborative action of multi-
extensively discussed in the literature, the realized impacts still need to stakeholders in agricultural development in developing countries. The
be examined, highlighting the challenges in monitoring and evaluation quantitative studies included were based on statistical analyses,
in practice (van Tulder et al., 2016). It also implies that greater attention manipulative experiments, or studies that combined multiple quantita
is needed to identify and evaluate the impacts of MSPs. Some studies tive methods. The conceptual analysis included reviews and theoretical
reported that MSPs had helped small-scale producers’ livelihood studies.
through improvement in income (Cavatassi et al., 2011; Thiele et al., We only included documents that discuss MSPs as a focus of their
2011) and capacity building (Narrod et al., 2009; Totin et al., 2020). analysis rather than only stating them in passing. We excluded studies
on other types of partnership under private regulation, such as business-
2
M. Maryono et al. Agricultural Systems 213 (2024) 103792
3
M. Maryono et al. Agricultural Systems 213 (2024) 103792
for each database and included alternative search terms where needed. exclusive, as impacts could be categorized into two or more categories.
The document selection process was completed in two steps iteratively. Microsoft Excel was used for data extraction and analysis, including
In the first step, the search string specified words related to agriculture cross-tabulation, frequency distributions, and data mapping.
(Search 1 in Table 2), followed by the subsequent search using search
string specified words related to multi-stakeholder partnership (Search 2 3. Results
in Table 2). The second step was a search of two inclusion terms linked
by a Boolean operator (#1 AND #2). The search was unrestricted and 3.1. Descriptive characteristics of included documents
included studies on different documents, the plot, farm, household,
country, regional, and global scale. The search strings in Table 2 were We analyzed a final selection of 147 documents for the first part of
specific to our research objectives and produced an initial database that the review. The descriptive analysis Fig. 2 presents an overview of the
reflects the available literature. distribution of the included documents by year of publication. It shows
that attention to and documentation of collaborative partnerships in
agriculture development has increased over time. Since 2004, the
2.3. Screening
number of studies has increased significantly. Of 147 documents
included in this study, 93% were published between 2010 and 2022. The
All documents identified during the search stage were screened
remaining documents were published before 2010.
based on the inclusion criteria. Studies that did not meet the inclusion
The majority of the included documents (59 documents) were
criteria were excluded from the review. In the initial search, we iden
qualitative studies (40%), and the quantitative studies accounted for 53
tified a total of 2353 entries. All documents were then screened to meet
documents (36%) (Fig. 3). The qualitative studies focused on the
the inclusion criteria. Covidence, a systematic review screening soft
descriptive analysis of case studies and outlined the implications for
ware, was used in the screening process. A total of 388 documents were
policy based on the functioning and outcomes of MSPs (Thiele et al.,
excluded due to duplication. The screening process then continued with
2011; Kilelu et al., 2013; Klerkx et al., 2013; Sell et al., 2018). Some of
a review of the title and abstract. A total of 1668 documents were
the quantitative studies involved impact evaluations using a counter
excluded as they were irrelevant to the topic. The multistage screening
factual approach (comparing participants and non-participants) (Cav
process produced 297 documents eligible for a full-text review. In
atassi et al., 2011; Kersting and Wollni, 2011; Mango et al., 2015; Palis
addition, we manually searched the references of the remaining docu
et al., 2017; Bisseleua et al., 2018). Others employed statistical analyses
ments to identify any documents not captured during the initial search.
to test the relationships between farmer participation in MSPs with farm
This process identified 19 additional documents. The documents were
productivity, yields and poverty reduction (Hartwich et al., 2005; Ladele
then content-analyzed and selected according to the inclusion/exclusion
and Akinwale, 2016; Lawal et al., 2019).
criteria. This process resulted in the exclusion of 169 documents for the
Regarding geographical representation, most studies (67%) covered
first research question and 237 documents for the second research
the African region (Fig. 4 and Fig. 5). Asia was the second most repre
question. The final screening process yielded 147 documents included in
sented study area (25%), while comparatively fewer studies discussed
the analysis of the first research question to explore the diverse practices
South America (7%). The over-representation of studies conducted in
of MSPs, and 79 documents included to address the second research
Africa is probably due to the widespread research projects implemented
question to evaluate the impact of MSPs (See Appendix A Supplemen
in the region supported by international organizations such as the
tary data).
Consultative Group on International Agricultural Research (CGIAR)
(Hounkonnou et al., 2018; Pamuk and Van Rijn, 2019). Most of the
2.4. Data extraction and analysis included studies (Fig. 6) focused on cereal (19%). The next most
researched commodity was livestock and dairy (17%), followed by fruit
Extracted data, piloted extensively, include bibliographic informa and vegetables (14%), while aquaculture received much less attention
tion, study location, design and methodology. For the first research (3%).
question, we collated data on the characteristics of the MSPs. This
included their type, objective, intervention, location, commodity, the
initiator and partners involved. For the second research question, the
data extraction focused on the reported impacts of the identified MSPs.
The impacts were grouped into four broad categories: economic, social,
technological, and environmental. The categories were not mutually
Table 2
Search strings used in the literature search.
Search Search string
Number
4
M. Maryono et al. Agricultural Systems 213 (2024) 103792
findings resonate with Kilelu et al. (2013) and Sartas et al. (2019) that
multi-actors collaborative arrangements in practice and the literature
use different terminologies.
These 12 terms are similar in their emphasis on the collective action
process involving multi actors from different sectors, such as in any
combination of government, private companies, research institutions,
NGOs, farmer associations, or cooperatives in developing the agricul
tural value chain. Collaboration among stakeholders is also a central
element of all MSPs terminologies used. However, there are differences
where such a term was used, which can be divided into two groups. The
Fig. 4. The geographical scope of the documents. The map shows the number of documents included in the review by country (n = 204). Some documents cover
more than one country.
5
M. Maryono et al. Agricultural Systems 213 (2024) 103792
Fig. 7. Different types of MSPs. a. distribution of different types of MSPs and their share by region. b. Distribution of different types of MSP by year of publication.
first group comprises the multi-stakeholders process, multi-stakeholders predominantly discussed in the context of implementing practical agri
engagement, multi-stakeholders interaction, and multi-stakeholders cultural interventions and conducting action research.
collaboration. This group generally discusses analytical concepts Innovation platforms and multi-stakeholder platforms were used
rather than practical intervention approaches. The second group, on the interchangeably or as multi-stakeholder innovation platforms (Makate
other hand, encompasses several types of partnerships, such as multi- and Mango, 2017; Mango et al., 2017b; Agboton et al., 2018). Both are
stakeholder initiatives, multi-stakeholder sustainability alliances, fundamental elements of the so-called Integrated Agricultural Research
public-private partnerships, multi-stakeholder partnerships, innovation for Development (IARD) which are widely used in the African region
platforms, and multi-stakeholder platforms. These partnerships are (Adekunle and Fatunbi, 2012; Fatunbi et al., 2013; Adekunle et al.,
6
M. Maryono et al. Agricultural Systems 213 (2024) 103792
2014; Swaans et al., 2016; Makate and Mango, 2017; Agboton et al., domination of research institutions in MSPs leadership. These in
2018; Sparrow and Traoré, 2018). The IARD refers to a form of dividuals or institutions are highly motivated and committed to driving
participatory-based action research for development to improve the institutional change and are often better equipped to leverage resources
welfare of smallholder producers by facilitating networks between and create new institutions.
farmers and market players (Sparrow and Traoré, 2018). It can be seen In the context of MSPs, institutional entrepreneurs can act as inno
as a new approach for organizing research to address complex problems vation brokers, facilitating interactions across different levels and
in agricultural development as a comparison to traditional research and addressing social, technical, and institutional issues. By leveraging their
development approaches (Maru et al., 2018). expertise and resources, they can help enhance the system’s overall
The data presented in Fig. 7 shows that there are regional differences functioning and ensure that all stakeholders are represented. However,
in the terminology used to refer to multi-stakeholder platforms (MSPs). it is important to note that the dominance of research institutions in MSP
The terms innovation platform and multi-stakeholder platform were leadership can also have drawbacks, such as a lack of diversity in per
commonly used in Africa. Specifically, 58 documents used the term spectives and a potential bias towards research-driven solutions.
innovation platform, and 11 documents used the term multi-stakeholder Therefore, ensuring that all stakeholders are adequately represented in
platform. These terms were widely used in the region as a significant MSPs and that institutional entrepreneurs are committed to promoting
number of research projects under the umbrella of IARD, such as the inclusivity and collaboration is important.
Volta Basin Development Challenge (VBDC) (Mariami et al., 2015), Nile
Basin Development Challenge (NBDC), Sub-Saharan Africa Challenge 3.3. Analysis of the impact on smallholder
Programme (SSA CP), Convergence of Sciences-Strengthening Agricul
tural innovation System (COS-SIS), and Humidtropics, were imple In the second part of this review, we delve into the impact of MSPs on
mented in the region by global research institutions such as CGIAR smallholder livelihoods, considering four dimensions: economic, social,
(Adekunle et al., 2014; Swaans et al., 2016; Bisseleua et al., 2018; technological, and environmental. Our analysis focuses on the inter
Pamuk and Van Rijn, 2019; Totin et al., 2020). In comparison, the term mediate and ultimate impacts directly linked to the intervention activ
public-private partnership was more commonly used in Asia (19 docu ities. In this section, we present the reported impact of MSPs as
ments). This suggests that different regions may have different per documented in the included studies, combining critical findings from
spectives or approaches towards MSPs, which could affect how these quantitative and qualitative impact evaluations. Qualitative research is
platforms are designed, implemented, and evaluated in different beneficial in understanding the effectiveness of sensitivity to context and
contexts. the heterogeneity of different geographical areas, time frames, and
activities.
3.2.2. Leadership of partnership Of the 147 documents analyzed in the first part of this study, only 79
The leadership of MSPs is assumed by various institutions, as illus documents examined the impact of MSPs. Among these, 42 studies used
trated in Fig. 8. However, the studies reviewed here display a significant quantitative methods to analyze the impact, while 31 employed quali
bias towards the leadership of certain institutions, with research in tative ones. The remaining four studies used quantitative and qualitative
stitutions heading the majority of MSPs (69%). The data reveals that methods to assess the impact. The use of mixed methods in these studies
government-led MSPs represent 14% of the studies, while leadership by highlights the significance of understanding the results and the reasons
NGOs and the private sector accounts for an equal share of 7%. In behind the impacts of MSPs on smallholders. Fig. 9 depicts the number
contrast, farmer organizations only make up 2% of the included studies. of documents that evaluated each indicator and the corresponding
The dominance of research institutions in leading MSPs can be number of impacts identified.
attributed to two primary factors. Firstly, research institutions have In the quantitative analysis of impact, some studies employed
been using MSPs as a tool to implement the IARD approach to achieve rigorous research designs, such as double differences in both ‘partici
their goals, as highlighted in previous studies (Nkonya et al., 2013; pants vs. non-participant’ and ‘baseline-endline’ (Cavatassi et al., 2011;
Mango et al., 2015). Secondly, the formation of MSPs usually follows a Kersting and Wollni, 2011; Nkonya et al., 2013; Mango et al., 2015; Palis
two-step process, with researchers typically from research institutions et al., 2017; Bisseleua et al., 2018; Pamuk and Van Rijn, 2019). Others
leading the first step, which involves identifying key challenges and use only baseline-end line without counterfactuals (Dione et al., 2019;
focusing primarily on research and technology transfer to improve Osorio-Garcia et al., 2020), while some employed cross-sectional data
smallholders’ technical and entrepreneurial practices through local ca taken from participants without counterfactuals (Mariami et al., 2015;
pacity building (Swaans et al., 2016). The second step often involves Ladele and Akinwale, 2016; Makate and Mango, 2017; Mango et al.,
mobilizing high-level actors, such as private companies and the gov 2017b, 2017a; Lawal et al., 2019).
ernment, to access greater markets and achieve critical affirmative As most documents discussed impact in multiple dimensions, the
policies (Van Paassen et al., 2014). total number of documents discussing a particular dimension of impact
The concept of ‘champions’ (Klerkx et al., 2013; Swaans et al., 2016), exceeded the total number of documents included in the second part of
or ‘institutional entrepreneurs’ Van Paassen et al. (2014) can explain the this review. Fig. 9 illustrates the impact indicators reported in the
studies, with the economic impact being the most commonly discussed
dimension, featured in 93 documents. The technological impact was the
second most discussed dimension, featured in 76 documents. The social
and environmental impacts were discussed in 71 and 21 documents.
7
M. Maryono et al. Agricultural Systems 213 (2024) 103792
Fig. 9. Impacts of MSPs reported in the included documents. A single study could have reported multiple impacts that fall within or under different categories; thus,
the sum across all the categories is above the total number of documents.
(4 documents), increased production scale (3 documents), increased using a counterfactual study in Ecuador. A yield improvement of up to
gross margin (3 documents), reduced cost (3 documents), increased ef 300% in potato production was reported in Rwanda (Fatunbi et al.,
ficiency (2 documents), and reduced price risk (1 document). The re 2013). Palis et al. (2017) reported an 11% increase in rice yield, while
ported impacts on the economic dimension highlight the success of Totin et al., (2020) found a 25% increase in potato yield. Similarly, IITA,
multi-stakeholder collaboration in improving the livelihoods of small 2014 reported an 85% increase in soybean yield, while Adam et al.
holders by providing them with better access to finance, market, and (2018) reported a 150% increase in potato yield and a 600% increase in
technology. milk yield.
An increase in yield and income is the most commonly reported Increased yield mostly leads to an increase in farmers’ income from
impacts resulting from collaborative activities (Cavatassi et al., 2011; production activities. For example, MSPs have revolutionized tomato
Fatunbi et al., 2013; Ladele and Akinwale, 2016; Sanyang et al., 2016; production, increasing farmer incomes by an average of 39% in East
Makate and Mango, 2017; Ponnusamy et al., 2017; Téno and Cadilhon, Africa (Keatinge et al., 2011), and soybean and pulses have increased
2017; Adam et al., 2018; Sell et al., 2018; Lawal et al., 2019). An in income by up to 50% in Mozambique (IITA (2014)). Some studies
crease in yield is typically measured as changes in production per unit of measured higher income in participants than non-participants, such as
land area or animal for livestock, as well as the volume of dairy products. Cavatassi et al. (2011) and Palis et al. (2017). Osorio-Garcia et al. (2020)
For instance, Cavatassi et al. (2011) found a significant increase in the documented participating farmers’ income increases by comparing
yield of coffee production for participants compared to non-participants before and after the program using baseline and end-line data. Other
8
M. Maryono et al. Agricultural Systems 213 (2024) 103792
studies measured income improvement using perceptual data. For cassava production in association with other food crops and leguminous
example, Ponnusamy et al. (2017) reported an increase in income of up species, as reported by Bisseleua et al. (2018). In addition to cost savings
to 20% among farmers participating in a program in India. In Ethiopia, and increased efficiency, MSPs have also been found to reduce the risk of
Sell et al. (2018) found that 60% of participants reported higher yields, price fluctuations in soybean production in Indonesia, according to
while most non-participants reported smaller yields due to drought. Susilowati (2018).
The income received by farmers is not solely dependent on the
amount of production but also on the farm gate price they receive. 3.3.2. Impact on the technological dimension
Several studies have highlighted the positive impact of MSPs on A total of 76 documents were analyzed to measure the technological
improving farmers’ selling prices and revenue. For example, Totin et al. impact of a range of interventions. The most frequently discussed impact
(2020) found that the increase in selling price, up to 47%, was the key concerning technology was the increased market access and informa
driver of increasing revenue for dairy farmers. Similarly, Cavatassi et al. tion, which was mentioned in 22 documents. This was closely followed
(2011) reported that participating farmers in their program could sell at by improved production technology, which was discussed in 21 docu
30% higher prices than non-participating farmers, leading to improved ments. Another notable impact was the increased access to inputs such
livelihoods. as seeds and fertilizers, as mentioned in 14 documents. Additionally, 11
Improved product quality and collective marketing have also been documents reported an improvement in post-harvest technology. The
reported to contribute to higher prices for farmers. For instance, Adam remaining documents discussed the increased adoption of technology
et al. (2018) reported a significant price increase of up to 900% due to (5) and the improved quality of seed access (3). Overall, the techno
improved product quality over an eight-year program period. Mariami logical impact of interventions was found to be diverse, with various
et al. (2015) found that improving links between value chain actors and benefits related to market access, production technology, and input
strengthening marketing relationships through MSPs resulted in better access.
selling prices and reduced production costs. The most commonly reported impact within the technological
Several studies conducted across different countries, including Bur dimension is increased market access, as discussed in 17 documents.
kina Faso, Malawi, Nigeria, East and Central Africa, and India, have Participants in MSPs have been found to have higher levels of access to
reported that MSPs have been effective in helping farmers access finance market information compared to the counterfactual group. This impact
(Téno and Cadilhon, 2017; Makate and Mango, 2017; Lawal et al., 2019; has been observed in various countries such as Ghana, Malawi, Nigeria,
Nkonya et al., 2013; Ponnusamy et al., 2017). The involvement of Burkina Faso, East and Central Africa, West Africa, and Latin America
financial institutions such as national banks in the MSPs has also been (Mariami et al., 2015; Makate and Mango, 2017; Ladele and Akinwale,
shown to increase farmers’ access to finance (Acosta, 2009; Nkonya 2016; Téno and Cadilhon, 2017; Nkonya et al., 2013; Sparrow and
et al., 2013; Thorpe and Maestre, 2015; Téno and Cadilhon, 2017; Traoré, 2018; Cavatassi et al., 2011; Namazzi et al., 2015). For example,
Murekezi et al., 2018). This access to finance is crucial for farmers to increased interaction and access to the market in MSPs in Ghana’s in
finance their farming operations, including purchasing necessary inputs tegrated crop-livestock value chain have reduced transaction costs
like seeds and fertilizers (Makate and Mango, 2017). However, Makate (Mariami et al., 2015). The multi-stakeholder partnership has also suc
and Mango (2017) highlighted that smallholder farmers’ access to credit ceeded in linking farmers to the high-value fruit and vegetable markets
from formal financial institutions in Malawi is severely limited as it in Kenya and India. This success was attributed to the implementation of
requires collateral and imposes conditions beyond most smallholder good agricultural practices and market-oriented management inter
farmers’ reach. vention in partnership programs (Narrod et al., 2009).
The study by Totin et al. (2020) found that the multi-stakeholder The second most frequently reported impact of MSPs on the tech
platform significantly increased farmers’ potato and milk production nological dimension is the improvement in production-related technol
revenue, suggesting that the partnership has positively impacted ogy, including crop and livestock management, water management,
farmers’ economic well-being. In the case of rice production in the fertilization and feeding practices, and sustainable farming technologies
Philippines, Palis et al. (2017) reported an increase in revenue, but and methods. Counterfactual studies involving a control group found
higher fertilizer prices during the evaluation period meant that this in that participants had better production technologies than the control
crease in revenue did not necessarily lead to an increase in income for group (Cavatassi et al., 2011; Nkonya et al., 2013; Palis et al., 2017;
farmers. Therefore, while revenue is an essential indicator of the success Pamuk and Van Rijn, 2019). Baseline and endline studies without a
of MSPs, it is important to consider other factors such as input costs, control group also found an improvement in livestock production
market prices, and other external factors that affect farmers’ income. technology practices to control endemic diseases in Mali (Dione et al.,
According to various sources, the multi-stakeholder partnership 2019). Descriptive analyses through cross-sectional surveys (Fleury
program has been recognized for its significant contribution to et al., 2008; Mariami et al., 2015; Ladele and Akinwale, 2016; Mango
improving employment opportunities, including in ACDI/VOCA (2008), et al., 2017b; Ponnusamy et al., 2017; Téno and Cadilhon, 2017; Lawal
Dror et al. (2016), and Ponnusamy et al. (2017). This has been achieved et al., 2019; Totin et al., 2020) have also shown a positive impact on
by introducing new activities, such as mushroom and vermicompost production-related technology.
production in India, resulting in a 10% increase in employment for The beneficiaries of MSPs successfully adopted new production
women farmers, as reported by Ponnusamy et al. (2017). Similarly, in technology, such as soil fertility technologies and water-saving tech
the Himalayas region, Dror et al. (2016) noted that the program had nologies (Cavatassi et al., 2011; Palis et al., 2017). However, the eval
enabled women to participate in the collection, transportation, and uation of agricultural technology adoption shows mixed and
retail of milk, leading to increased employment opportunities for them. inconclusive results in East and Central Africa (Nkonya et al., 2013).
The MSPs have demonstrated the potential for sustainable development Pamuk and Van Rijn (2019) documented that participation in MSP ac
by creating employment opportunities, particularly for women in rural tivities was particularly strong in capacity building for farmers, leading
areas. to increased adoption of crop management and soil fertility
Keatinge et al. (2011) and Mariami et al. (2015) have reported that technologies.
MSPs can lead to reduced production costs. For instance, the MSP pro An increase in access to agricultural inputs as a significant impact
gram implemented in the Philippines has resulted in significant cost was reported in 14 documents. Studies have shown that participants had
savings for rice cultivation, including a decrease in fuel consumption higher access to agricultural inputs compared to the control group in
and costs of up to 44%, irrigation labour by up to 11%, and irrigation East and Central Africa (Nkonya et al., 2013), as well as in other regions
costs by up to 39%, as noted by Palis et al. (2017). Moreover, farmers such as Ghana and Nigeria (Mariami et al., 2015; Ladele and Akinwale,
participating in MSPs have demonstrated greater economic efficiency in 2016). In Rwanda, Totin et al. (2020) found that MSPs focused on
9
M. Maryono et al. Agricultural Systems 213 (2024) 103792
combining higher-quality seeds with access to market-preferred vari actors to solve common problems and improve joint planning and co
eties led to an increase in the use of the most preferred potato varieties ordination of activities between value chain partners (Téno and Cadil
among participants to meet market demand. The formation of seed hon, 2017). Farmer associations help shape farmer decisions to focus
farmers’ groups was identified as the driver of the increase in seed ac more on the market than production (Mango et al., 2017b). Participa
cess. Téno and Cadilhon (2017) also reported an increase in access to tion in multi-stakeholder activities strongly influences collective mar
agricultural inputs, which was attributed to the closer relationships keting, which in turn has a positive impact on the income of smallholder
(social capital) between farmers and various public and private agri households (Mango et al., 2017a).
cultural service providers for livestock, agriculture, and animal health Eleven studies have reported an increase in information sharing, as
services. evidenced by Mariami et al. (2015), Dror et al. (2016), Ladele and
The impact on improving post-harvest technology has been discussed Akinwale (2016), and Lawal et al. (2019). Specifically, Mariami et al.
in eleven documents, including studies by Fatunbi et al. (2013), Mariami (2015) found that MSPs improved communication and information
et al. (2015), Mango et al. (2017b), Pamuk and Van Rijn (2019), and sharing between farmers and value chain actors. This improvement in
Totin et al. (2020). Totin et al. (2020) found that the platform’s co- relationships and behaviour change among various social and economic
funding of adequate storage areas reduced post-harvest loss, resulting actors in Africa has been noted by Sanyang et al. (2016). Lawal et al.
in improved post-harvest technology. Meanwhile, Fatunbi et al. (2013) (2019) also reported an improvement in information sharing.
reported that the MSP increased the production and value addition of Four studies, including Nkonya et al. (2013), Ladele and Akinwale
sorghum drinks, banana juice, and wine through improved processing (2016), Hanyani-Mlambo et al. (2017), and Bisseleua et al. (2018), have
and marketing. shown an increase in access to extension services. Bisseleua et al. (2018)
reported that participants had greater access to training, extension
3.3.3. Impact on the social dimension agents, and social network members for knowledge exchange than non-
A total of 71 documents have reported positive impacts of MSP on participants. Nkonya et al. (2013) discovered an increase in the number
the social dimension. Among these, the most frequently discussed of visits made by agricultural extension agents to the sites of MSPs in
impact is the increase in knowledge, education, and training, which was East and Central Africa. Téno and Cadilhon (2017) noted that MSP led to
reported in 20 documents. The second most commonly reported impact better access to information and exchange, capacity development,
is the strengthening of collective action, which was reported in 19 strong linkage and trust between members, and joint planning. Pon
documents. Other reported impacts include improved information nusamy et al. (2017) observed psychological improvements resulting
sharing (11), increased food and nutrition security (7), increased from public-private partnerships, including an enhancement in team
extension (6), improved women’s empowerment (3), reduced conflict spirit, risk-bearing ability, and the ability to handle group activities.
(3), and improved networking (2). Four studies have reported an improvement in food and nutrition
MSPs have been shown to positively impact knowledge, education, security as an impact of MSPs (Mango et al., 2015; Hanyani-Mlambo
and skill training, according to 20 documents reviewed. The training et al., 2017; Makate and Mango, 2017; Adam et al., 2018). Mango et al.
activities conducted within these programs have improved the capac (2015) found that the MSP activities have reduced food insecurity and
ities of MSPs members, which has resulted in improved farmer capacity improved dietary diversity for participants in southern Africa. Makate
in using better production and marketing techniques, as well as better and Mango (2017) reported that households in Malawi stated that their
farm, post-harvest, and marketing enterprise management (ACDI/ food security had improved through partnership activities and pointed
VOCA, 2008; Fleury et al., 2008; Dabire et al., 2017; Makate and Mango, out that the food produced was enough to feed the family. Adam et al.
2017; Palis et al., 2017; Ponnusamy et al., 2017; Téno and Cadilhon, (2018) also found that participants of MSP in Rwanda have more extra
2017; Bisseleua et al., 2018; Osorio-Garcia et al., 2020). The training income, which led to more access to food. In addition, the interventions
and dissemination of innovation activities and study tours have also promoting local processing have also improved the food quality for
increased the capacity and experience of farmers in applying particular households. Hanyani-Mlambo et al. (2017) found that participants had
technologies, such as the application of AWD (Palis et al., 2017). better food security than non-participants. They found a correlation
MSP has been found to improve knowledge and skills in vegetable between market participation and improved food security among
production using micro-dosing innovations, face-to-face interaction households, as participating in markets provided them with access to a
with scientists and other stakeholders, and timely access to information wider range of foods, resulting in greater dietary diversity.
in Nigeria, based on a perceptional Likert scale survey (Lawal et al., Three studies explicitly reported the positive impact of MSPs on
2019). Training participation has increased the likelihood of partici women’s empowerment (Ponnusamy et al., 2017; Adam et al., 2018; Sell
pating in the market and has led to improved decision-making. As a et al., 2018). Ponnusamy et al. (2017) reported that partnerships helped
result, farmers can conceptualize marketing issues and formulate solu women farmers improve their confidence, team spirit, and access to
tions to improve their income (Mango et al., 2017b). However, the training, markets, and financial services. Sell et al. (2018) found that
changes in knowledge and skills have shown variation in different re women’s skills and capacity in communication improved, and they were
gions, depending on education and social environment (Ponnusamy more likely to express their opinion and start small-scale processing or
et al., 2017). business activities. Adam et al. (2018) reported that women who
MSP is considered a collective action among a range of stakeholders. participated in such partnerships had extra income and were more food
However, 19 documents mainly refer to the collective action among secure. Another social dimension impact reported was societal harmony
farmers who form cooperatives or farmer associations. Participants in and reduced conflict between community members (Adam et al., 2018),
MSPs with cassava farmers in Nigeria showed significantly higher levels achieved through collaboration and reduced competition among diverse
of collective action than non-participants (Bisseleua et al., 2018). The actors.
formation of farmers’ associations also led to other activities besides
production, such as facilitating collective marketing instead of individ 3.3.4. Impact on the environmental dimension
ual marketing (Totin et al., 2020). Collective action is crucial in ensuring Our examination of the data indicated that the collaboration among
that smallholder farmers are not left out of the value chain, especially stakeholders had a beneficial effect on farmers’ interactions with the
when dealing with strict food safety standards (Narrod et al., 2009). environment. However, in the studies we reviewed, the environmental
Through collective action, farmers can easily acquire third-party certi impacts of these partnerships were the least frequently reported. There
fication, hire technical experts, develop infrastructure, such as pack were only 21 studies that reported improvements in environmental
aging houses, or access information (Narrod et al., 2009). impacts, such as enhanced soil and water conservation (10 documents),
Collective action has enabled farmers to work with value chain better soil fertility (3 documents), reduced chemical use (3 documents),
10
M. Maryono et al. Agricultural Systems 213 (2024) 103792
decreased water use (3 documents), increased adaptation and resilience 3.4. Discussion
to climate change (1 document), and reduced waste (1 document).
The studies predominantly reported improvements in environmental 3.4.1. Different types of MSPs
impacts related to natural resource conservation, focusing on soil and Our systematic review indicates that the literature on MSPs includes
water conservation. These improvements were documented in studies 12 terms that describe collaborative efforts in multiple developing
such as ACDI/VOCA, 2008, Dror et al., 2016, Dabire et al., 2017, Makate countries. The various names of MSPs are sometimes used inter
and Mango, 2017, Mango et al., 2017b, Bisseleua et al., 2018, and changeably in documents. Additionally, a single MSP initiative may be
Pamuk and Van Rijn, 2019. In one study, a majority of participants referred to by different names in various documents, suggesting a lack of
(82%) reported adopting new environmental conservation practices as a consensus on the defining characteristics of this type of partnership.
result of the program, while 90% adopted conservation agriculture, However, the included documents reveal three main terminologies used
leading to higher market participation and output compared to non- to refer to MSPs at the operational/practical level: public-private part
adopters (Mango et al., 2017b). nership, multi-stakeholder initiative/sustainability alliance, and multi-
Various studies have shown that MSP has positively impacted stakeholder platform/innovation platforms. The main different prac
farmers’ knowledge, resilience, and adaptation to climate change tices of these three MSPs are depicted in Fig. 10.
(Osorio-Garcia et al., 2020; Zhang et al., 2018). According to ACDI/ Figure 10 displays how MSPs can be classified based on their ob
VOCA (2008), Palis et al. (2017), and Lawal et al. (2019), the MSP has jectives, models, initiators, partner involvement, time frame, and part
resulted in decreased utilization of water, chemicals, and waste. In ner expectations. Public-private partnerships (PPPs) typically have well-
addition, the uptake of sustainable farming technologies and methods defined and specific objectives that can be accomplished in a shorter
has improved soil fertility (Makate and Mango, 2017; Adam et al., 2018; period. These partnerships are generally more formal and involve a
Pamuk and Van Rijn, 2019). These factors are expected to contribute to smaller number of pre-selected partners, often initiated by the govern
longer-term sustainable production and improve yield and income ment. On the other end of the spectrum, multi-stakeholder platforms or
beyond the study period. Farmers’ adoption of conservation agriculture innovation platforms address more complex and systemic challenges
practices has been linked to longer-term impacts on farm productivity over a longer time frame. These MSPs employ a collective impact
and food security rather than directly to climate change (Osorio-Garcia approach rather than a limited joint program and engage with diverse
et al., 2020). Furthermore, the platform has had no negative effects on actors from different sectors. This type of partnership can be an informal
the environment or farmers’ health (Cavatassi et al., 2011). or formal arrangement. Between these two extremes is the multi-
stakeholder initiative/sustainability alliance, which resembles a stra
3.3.5. Other impacts reported tegic alliance between multinational companies and non-market actors
According to seven documents, there were two extra impacts like NGOs to address social and environmental issues. This type of
observed. These two additional impacts, were the positive influence on partnership is primarily led by a company that is committed to
local infrastructure (3 documents) and the positive alteration in policy improving its sustainability practices and reducing its environmental
(4 documents). These impacts emphasize the wider and enduring effects impact.
of partnerships on the communities they serve.
By improving infrastructure, such as building irrigation, new facil 3.4.2. Diverse focus of intended impacts and variability in intervention
ities and providing training venues, partnerships can help to create a effectiveness
more conducive environment for agricultural development, which can The systematic review we conducted reveals that MSPs positively
benefit farmers and other actors in the value chain (Adam et al., 2018; impact farmers’ economic, social, and technological aspects, as well as
Mdemu et al., 2020; Zhu et al., 2021). Similarly, partnerships can also their environmental adaptation practices, to address climate change.
influence policy change at the local and national level, which can Additionally, there are benefits in developing local infrastructure and
significantly impact the sustainability and growth of the agricultural policy change to support the sustainable value chain. Nevertheless, even
sector (Cadilhon et al., 2016; Palis et al., 2017; Retnoningtyas et al., though MSPs are popular in agricultural development literature, we
2021; Lamboll et al., 2021). By advocating for policy changes that found only a limited number of studies that assess their impacts. Out of
support the development of the agricultural value chain, partnerships the 147 studies included in the review, only 79 assessed the impacts,
can help to create an enabling environment that promotes investment, which were derived from a population of 2353 documents screened.
innovation, and growth (Cadilhon et al., 2016; Palis et al., 2017; Among the studies that evaluated the impacts, the results mainly focused
Retnoningtyas et al., 2021; Lamboll et al. 2021). on the economic, technological, and social dimensions, with less
It’s important to note that these additional dimensions of impact are emphasis on the environmental aspects.
closely interconnected with the four dimensions of impact previously There are two primary reasons why the reported impacts tend to
reported. For example, improved infrastructure can lead to increased focus on economic and technological aspects. First, the primary goal of
productivity and efficiency, resulting in higher incomes and better MSPs is to enhance the livelihoods of smallholder farmers by imple
livelihoods for potato farmers in Rwanda (Adam et al., 2018). Similarly, menting interventions that modify technical practices and increase
positive policy change in terms of reducing value-added tax and production (Fatunbi et al., 2013; Kilelu et al., 2013; Dror et al., 2016;
removing the limitation on urban dairy farming in Tanzania can create Teno and Cadilhon, 2016; Makate and Mango, 2017; Téno and Cadilhon,
an enabling environment that can create new opportunities for eco 2017; Bisseleua et al., 2018; Totin et al., 2020). Second, the collabora
nomic development (Cadilhon et al., 2016). tive activities, especially the Innovation Platforms and Multi-
The additional dimensions of impact reported in these studies stakeholders Platforms, are utilized as instruments to magnify the
demonstrate the critical role of partnerships in addressing complex impact of agricultural research by creating an environment that fosters
development challenges and creating sustainable solutions for agricul the scaling of innovations like new technologies, practices, and business
tural development. By addressing not only the immediate needs of models that support the transformation of agriculture (Sanyang et al.,
farmers and other actors in the value chain but also the broader context 2016; Totin et al., 2020). However, it is worth noting that while the
in which they operate, MSPs can help to create a more resilient and included studies emphasized and reported the economic and techno
sustainable agricultural sector that contributes to the overall develop logical impacts of MSPs, the recognition to address social and techno
ment of a country or region. logical impact is also growing.
Although participants have experienced positive impacts, further
investigation is necessary to examine the distribution of benefits across
different socio-economic backgrounds. This is due to the unequal
11
M. Maryono et al. Agricultural Systems 213 (2024) 103792
Fig. 10. Comparison of primary practice used in MSPs (adapted from Peterson et al. (2014) and Tinarwo et al. (2018)).
distribution of benefits among participants of varying socio-economic et al., 2014; Hounkonnou et al., 2018; Dominic and Gupta, 2019; Totin
statuses, as evidenced by research conducted by Makate and Mango et al., 2020). Furthermore, the institutional frameworks set up at a
(2017) and Sparrow and Traoré (2018). Sparrow and Traoré (2018) higher tier through government actions are crucial. Aligning MSP ac
found that MSPs are more effective for participants with larger land tivities with political agendas can significantly bolster and refine scaling
holdings, diversified production systems, higher productivity, and a efforts (Totin et al., 2020). As emphasized by Seifu et al. (2022) and
higher proportion of products sold. However, these partnerships were Kilelu et al. (2017), not addressing institutional change effectively may
less effective for poorer farmers who had lower crop production, smaller lead to minimal impact.
investments, and limited product sales. Similarly, studies by Mariami Our review provides valuable insights into under-researched aspects
et al. (2015) suggested that the impact of MSPs was greater for partic of MSPs. Firstly, studies outside the African region have received less
ipants with higher annual incomes. attention and need to be explored further to enhance the generalisability
of MSP impacts across different contexts. Secondly, the impact evalua
3.4.3. Key factors of successful intervention tion of MSPs on environmental aspects and behaviour change of farmers
For an MSP to be successful in the innovation process, it requires in addressing environmental issues is an important area for research.
three critical elements: adept facilitation, a comprehensive approach to Thirdly, the distributional benefit of MSPs’ impacts across diverse socio-
the value chain, and a commitment to addressing institutional change. economic backgrounds of farmers needs to be studied to ensure equi
Effective facilitation is essential in balancing power dynamics and table outcomes. Lastly, there is a need to examine the governance of the
overcoming barriers to collaboration between diverse stakeholders value chain and how value chain actors respond to changes brought
(Namazzi et al., 2015; Dror et al., 2016; Adam et al., 2018; Sell et al., about by MSPs.
2018; Hounkonnou et al., 2018). Research by Pamuk and Van Rijn In-depth studies on these four aspects are necessary for a compre
(2019) suggests that partnerships with more active members are more hensive assessment of the overall impact of MSPs, taking into account
successful in achieving the outcomes, while partnerships with too many patterns of change compared to non-participants. By understanding the
stakeholders may be less successful. impact of MSPs on these under-researched aspects, we can identify areas
The success of the MSPs also emphasizes the importance of identi where further improvements can be made to increase the effectiveness
fying and organizing beneficiaries that effectively overcome barriers to of MSPs in agricultural development in developing countries.
entry. The success of MSPs can be attributed to their intervention across While our review provides valuable insights into the impacts of
the value chain by combining facilitating market access and production MSPs, we acknowledge that some potential biases may affect our find
support (Narrod et al., 2009; Cavatassi et al., 2011; Van Paassen et al., ings. Language bias is a notable limitation in our study, particularly
2014; and Dror et al., 2016; Totin et al., 2020). On the output side, MSPs given our emphasis on English-language documents, which could lead to
reduced transaction costs by bypassing intermediaries, enabling farmers the underrepresentation of studies in other languages. Furthermore, we
to obtain a greater share of returns from their production activities by observed a disproportionate focus on the African region in the docu
accessing a more lucrative market. On the input side, MSPs introduced ments we analyzed. This geographic imbalance might stem from the
and supplied market-preferred varieties, high-quality seeds, and effi predominant interest of research organizations in this particular region.
cient farming techniques. These interventions significantly impacted the We recognize the inherent challenges in identifying and selecting
agricultural value chain, increasing productivity and promoting sus studies, compounded by the widely acknowledged issue of publication
tainable practices. bias. This bias emerges when documents predominantly report out
The included documents also highlight the critical role of MSPs in comes that are viewed more favorably (Kepes et al., 2023; Ayorinde
addressing institutional change in order to achieve the intended out et al. 2020). Studies that report significantly positive impacts of MSPs
comes. This encompasses alterations in rules, norms, values, regulatory may have a higher likelihood of being published than those reporting
frameworks, governance structures, and the dynamics of actor re negative or insignificant outcomes (Bizikova et al., 2020).
lationships (Hounkonnou et al., 2018). MSPs drive innovation among These biases may result from a constrained focus on particular dis
smallholders by implementing institutional changes, broadening op courses, specific research methods, or select geographical regions. We
portunities beyond just the farm level (Kilelu et al., 2013; Van Paassen mitigated these biases through a systematic and transparent approach in
12
M. Maryono et al. Agricultural Systems 213 (2024) 103792
our framework and review process, incorporating grey literature as reported in this paper.
recommended by Kepes et al. (2023) and Ayorinde et al. (2020). Overall,
our findings align with van Ewijk and Ros-Tonen’s (2021) review of Data availability
MSPs in sub-Saharan Africa, which also noted a predominance of reports
on positive outcomes. We did identify two documents highlighting the The data used is available in the Supplementary
underperformance of MSPs in achieving their intended outcomes (Seifu
et al., 2022; Kilelu et al., 2017). Discerning whether the limited number Acknowledgements
of studies highlighting negative results is a product of underreporting or
genuine successes in the field remains a challenge. We acknowledge the scholarship support to Maryono from the
Another potential bias is the spillover effect on the control group, Indonesian Endowment Fund for Education (Lembaga Pengelola Dana
which may have dampened the impact of MSPs. For example, some Pendidikan - LPDP), Ministry of Finance, Indonesia. Findings and con
studies in East and Central Africa have found that farmers in control clusions are those of the authors and do not necessarily reflect the view
regions adopt practices observed in MSP participant regions after of the funding agency or any other Indonesian government offices or
forming partnership activities, which could blur the distinction between political organizations.
the control and intervention groups (Nkonya et al., 2013). Overall, it is
important to recognize and account for these potential biases in future
Appendix A. Supplementary data
research on MSPs to ensure that findings accurately reflect the true
impacts of these partnerships.
Supplementary data to this article can be found online at https://doi.
org/10.1016/j.agsy.2023.103792.
3.5. Conclusions
References
This systematic review highlights the varied use of the MSPs
approach in agricultural development across developing countries. To ACDI/VOCA, 2008. Final Report : Colombia Specialty Coffee and Support for USAID/
develop agricultural value chains, MSPs generally involve a collabora GOC Public-Private Partnership Program. USAID, Washington DC.
tive process that brings together multiple actors from various sectors, Acosta, RR, 2009. A public-private partnership approach to building the capacity of
smallholder farmers: The case of fresh tropical vegetables in the Philippines. Acta
such as government, private companies, research institutions, NGOs,
Horticulturae 809, 127-34.
and farmer groups. Nevertheless, MSPs are sometimes referred to by Adam, R.I., Misiko, M., Dusengemungu, L., Rushemuka, P., Mukakalisa, Z., 2018. Gender
various names in different contexts. Nonetheless, these approaches were and equitable benefit-sharing mechanisms through agricultural innovation platforms
utilized both as analytical concepts and practical intervention strategies in Rwanda. Community Dev. 49 (4), 380–397.
Adekunle, A.A., Fatunbi, A.O., 2012. Approaches for setting-up multi-stakeholder
to tackle complex issues in agricultural development. platforms for agricultural research and development. World Appl. Sci. J. 16 (7),
The review also indicates that research institutions have played a 981–988.
dominant role in leading the majority of MSPs. Thus, there is a need to Adekunle, A.A., Ayanwale, A.B., Fatunbi, A.O., Olarinde, L.O., Mango, N.,
Nyikahadzoi, K., Siziba, S., Oladunni, O., Nokoe, S., Musinguzi, E., Baidu-Forson, J.,
elaborate on different MSPs that are led by other actors, i.e. farmers, 2014. Exploring the Potentials of Integrated Agricultural Research for Development
which is understudied. This finding underscores the need for greater in Southern Africa. Forum for Agricultural Research in Africa (FARA), Accra, Ghana.
diversity and inclusion of other actors, including farmers, in the design Agboton, S.M., Sèwadé, P.L., Ayenan, M.A.T., 2018. Successes and challenges in tackling
constraints in soybean processing through multi-stakeholder innovation platforms in
and implementation of MSPs to ensure that all stakeholders are Agricultural Research for Development: an evidence from the consortium Soja du
adequately represented, and their interests are appropriately addressed. Bénin. Afr. J. Rural Develop. (AFJRD) 3 (1), 579–595.
The review highlights the positive impacts of MSPs on farmers’ Ayorinde, AA, Williams, I, Mannion, R, Song, F, Skrybant, M, Lilford, RJ, Chen, 2020. Y-F
2020, ’Assessment of publication bias and outcome reporting bias in systematic
economic, social, and technological aspects, as well as environmental reviews of health services and delivery research: A meta-epidemiological study.
adaptation practices to address climate change. Moreover, the study PLOS ONE 15 (1), 1–17.
finds some benefits in developing local infrastructure and policy change Bäckstrand, K., 2006. Multi-stakeholder partnerships for sustainable development:
rethinking legitimacy, accountability and effectiveness. Eur. Environ. 16 (5),
to support sustainable value chains. However, the impact analysis tends
290–306.
to focus heavily on economic, social and technological dimensions while de Bakker, F.G.A., Rasche, A., Ponte, S., 2019. Multi-stakeholder initiatives on
neglecting the environmental aspects. Moreover, MSPs were found to be sustainability: a cross-disciplinary review and research agenda for business ethics.
more successful in the intervention process when they emphasized three Bus. Ethics Q. 29 (3), 343–383.
Barzola Iza, C.L., Dentoni, D., Omta, O.S.W.F., 2020. The influence of multi-stakeholder
primary components: adept facilitation, an all-encompassing approach platforms on farmers’ innovation and rural development in emerging economies: a
to the value chain, and addressing institutional change. systematic literature review. J. Agribus. Develop. Emerg. Econ. 10 (1), 13–39.
While the review demonstrates the potential of MSPs to deliver Bisseleua, D.H.B., Idrissou, L., Olurotimi, P., Ogunniyi, A., Mignouna, D., Bamire, S.A.,
2018. Multi-stakeholder process strengthens agricultural innovations and
positive outcomes for sustainable agricultural development in devel sustainable livelihoods of farmers in southern Nigeria. J. Agric. Educ. Ext. 24 (1),
oping countries, it highlights the need to consider the broader impact on 29–49.
farmers’ sustainable livelihoods. Specifically, future research should Bizikova, L., Nkonya, E., Minah, M., Hanisch, M., Turaga, R.M.R., Speranza, C.I.,
Karthikeyan, M., Tang, L., Ghezzi-Kopel, K., Kelly, J., Celestin, A.C., Timmers, B.,
explore the distribution of benefits across different socio-economic 2020. A scoping review of the contributions of farmers’ organizations to smallholder
backgrounds to assess the equitable impact of MSPs on sustainable agriculture. Nature Food 1 (10), 620–630.
development in agricultural communities. This information can provide Cadilhon, J.-J., Pham, N.D., Maass, B., 2016. The Tanga dairy platform: fostering
innovations for more efficient dairy chain coordination in Tanzania. Intern. J. Food
critical insights for policymakers and practitioners seeking to enhance Syst. Dyn. 07 (2), 81–91.
the effectiveness of MSPs in promoting sustainable agriculture in Cavatassi, R., González-flores, M., Winters, P., Andrade-Piedra, J., Espinosa, P.,
developing countries. Thiele, G., 2011. Linking smallholders to the new agricultural economy: The case of
the Plataformas de Concertación in Ecuador. J. Dev. Stud. 47 (10), 1545–1573.
Statement: During the preparation of this work the author(s) used
Dabire, D., Andrieu, N., Djamen, P., Coulibaly, K., Posthumus, H., Diallo, A.M.,
ChatGPT in order to improve the readability of the article. After using Karambiri, M., Douzet, J.M., Triomphe, B., 2017. Operationalizing an innovation
this tool/service, the author(s) reviewed and edited the content as platform approach for community-based participatory research on conservation
needed and take(s) full responsibility for the content of the publication. agriculture in Burkina Faso. Exp. Agric. 53 (3), 460–479.
Demont, M., Rutsaert, P., 2017. Restructuring the Vietnamese Rice sector: towards
increasing sustainability. sustainability 9 (2).
Declaration of Competing Interest Dentoni, D., Peterson, H.C., 2011. Multi-stakeholder sustainability alliances in Agri-food
chains: a framework for multi-disciplinary research. Intern. Food Agribus. Manage.
Rev. 14 (5), 83–108.
The authors declare that they have no known competing financial Dentoni, D., Bitzer, V., Schouten, G., 2018. Harnessing wicked problems in multi-
interests or personal relationships that could have influenced the work stakeholder partnerships. J. Bus. Ethics 150 (2), 333–356.
13
M. Maryono et al. Agricultural Systems 213 (2024) 103792
Dione, M.M., Traore, I., Kassambara, H., Sow, A.N., Toure, C.O., Sidibe, C.A., Sery, A., Makate, C., Mango, N., 2017. Diversity amongst farm households and achievements from
Yena, A.S., Wieland, B., Dakouo, M., Diall, O., Niang, M., Fomba, C.O., Traore, M., multi-stakeholder innovation platform approach: lessons from Balaka Malawi.
Fall, A., 2019. Integrated approach to facilitate stakeholder participation in the Agricult. Food Secur. 6 (1).
control of endemic diseases of livestock: the case of peste des Petits Ruminants in Mangeni, B., 2019. The role of public-private partnerships (PPPs) in ensuring technology
Mali’. Front. Veterin. Sci. 6. access for farmers in sub-Saharan Africa. Afr. J. Food Agric. Nutr. Dev. 19 (1),
Dominic, D.M., Gupta, J., 2019. Institutionalization of dairy innovation platforms: a 14137–14155.
study on Samagra project in Kerala. Indian J. Dairy Sci. 72 (5), 542–546. Mango, N., Nyikahadzoi, K., Makate, C., Dunjana, N., Siziba, S., 2015. The impact of
Dror, I., Cadilhon, J.J., Schut, M., Misiko, M., Maheshwari, S., 2016. Innovation integrated agricultural research for development on food security among
Platforms for Agricultural Development: Evaluating the Mature Innovation Platforms smallholder farmers of southern Africa. Agrekon 54 (3), 107–125.
Landscape, Innovation Platforms for Agricultural Development: Evaluating the Mango, N., Makate, C., Lundy, M., Siziba, S., Kefasi, N., Fatunbi, O., 2017a. Collective
Mature Innovation Platforms Landscape. Routledge, New York. market participation for improved income among smallholder farming households: a
Eweje, G., Sajjad, A., Nath Shobod, D., Kobayashi, K., 2020. Multi-stakeholder case of Balaka innovation platform in Malawi. Afr. Crop. Sci. J. 25, 97.
partnerships: a catalyst to achieve sustainable development goals. Mark. Intell. Plan. Mango, N., Makate, C., Lundy, M., Siziba, S., Kefasi, N., Fatunbi, O., 2017b. Influence of
Fortcomming. multi-stakeholder innovation platform approach on smallholder farmers marketing
van Ewijk, E., Ros-Tonen, M.A.F., 2021. The fruits of knowledge co-creation in decisions. Afr. Crop. Sci. J. 25, 109.
agriculture and food-related multi-stakeholder platforms in sub-Saharan Africa – a Mariami, Z.A., Cadilhon, J.-J., Werthmann, C., 2015. Impact of innovation platforms on
systematic literature review. Agric. Syst. 186, 102949. marketing relationships: The case of Volta Basin integrated crop-livestock value
Fatunbi, O., Adewale, A., Buruchara, R., Nyamwaro, S., 2013. Integrated Agricultural chains in Ghana. Afr. J. Agricult. Resource Econ. 10 (4), 328–342.
Research for Development ...From Concept to Practice. Forum for Agricultural Maru, Y., Sparrow, A., Stirzaker, R., Davies, J., 2018. Integrated agricultural research for
Research in Africa (FARA), Accra, Ghana. development (IAR4D) from a theory of change perspective. Agric. Syst. 165,
Fleury, P., Petit, S., Dobremez, L., Schermer, M., Kirchengast, C., De Ros, G., Magnani, N., 310–320.
Struffi, L., Mieville-Ott, V., Roque, O., 2008. Implementing sustainable agriculture Mdemu, M., Kissoly, L., Bjornlund, H., Kimaro, E., Christen, E.W., van Rooyen, A.,
and rural development in the European Alps assets and limitations of local projects Stirzaker, R., Ramshaw, P., 2020. The role of soil water monitoring tools and
based on multi-stakeholder participation. Mt. Res. Dev. 28 (3–4), 226–232. agricultural innovation platforms in improving food security and income of farmers
Hanyani-Mlambo, B., Mudhara, M., Nyikahadzoi, K., Mafongoya, P., 2017. Socio- in smallholder irrigation schemes in Tanzania. Intern. J. Water Resourc. Develop. 36
economic differences between innovation platform participants and non- (sup1). S148-S70.
participants: the case of smallholder dairying in Zimbabwe. Livest. Res. Rural. Dev. Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., The, P.G., 2009. Preferred reporting
29 (8). items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 6
Hartwich, F., Gonzalez, C., Vieira, L.-F., 2005. Public-Private Partnerships for (7) p. e1000097.
Innovation-Led Growth in Agrichains: A Useful Tool for Development in Latin Murekezi, P, Menezes, A, Ridler, N, 2018. Contract farming and public private
America? International Food Policy Research Institute, Washington DC. partnerships in aquaculture Lessons learned from East African countries. Food and
Hermans, F., Geerling-Eiff, F., Potters, J., Klerkx, L., 2019. Public-private partnerships as Agriculture Organization of the United Nations (FAO), Rome, Italy.
systemic agricultural innovation policy instruments – assessing their contribution to Namazzi, S, Muchunguzi, P, Lamers, D, Sole-Amat, A, van Asten, P, Dubois, T, Afari-
innovation system function dynamics. NJAS - Wageningen J. Life Sci. 88, 76–95. Sefa, V, Tenywa, MM, Mugisa, I, McCampbell, M, Sartas, 2015. Crop-livestock-tree
Hopewell, S., Clarke, M., Lefebvre, C., Scherer, R., 2007. Handsearching versus electronic integration in Uganda: The case of Mukono-Wakiso innovation platform’, in
searching to identify reports of randomized trials. Cochrane Database Syst. Rev. Innovation Platforms for Agricultural Development: Evaluating the Mature Innovation
2007 (2), Mr000001. Platforms Landscape. M 79–97.
Hounkonnou, D., Brouwers, J., van Huis, A., Jiggins, J., Kossou, D., Röling, N., Sakyi- Narrod, C., Roy, D., Okello, J., Avendaño, B., Rich, K., Thorat, A., 2009. Public-private
Dawson, O., Traoré, M., 2018. Triggering regime change: a comparative analysis of partnerships and collective action in high value fruit and vegetable supply chains.
the performance of innovation platforms that attempted to change the institutional Food Policy 34 (1), 8–15.
context for nine agricultural domains in West Africa. Agric. Syst. 165, 296–309. Nkonya, E., Edward, K., Oduol, J.B., Pali, P., Farrow, A., 2013. Initial impact of
Hussain, I., Perera, L.R., 2004. ’Improving agricultural productivity for poverty integrated agricultural research for development in East and Central Africa. Afr. J.
alleviation through integrated service provision with public-private sector Agricult. Resource Econ. 8, 172–184.
partnerships: examples and issues. Intern. Water Mange. Instit. v, 26p. Working Osorio-Garcia, A.M., Paz, L., Howland, F., Ortega, L.A., Acosta-Alba, I., Arenas, L.,
paper 66. Chirinda, N., Martinez-Baron, D., Findji, O.B., Loboguerrero, A.M., Chia, E.,
IITA, IIoTA, 2014. Public-Private Partnership for Innovation in Soybean and Cowpea Andrieu, N., 2020. Can an innovation platform support a local process of climate-
Value Chains in Mozambique. International Institute of Tropical Agriculture, smart agriculture implementation? A case study in Cauca, Colombia. Agroecol.
Mozambique. Sustain. Food Syst. 44 (3), 378–411.
Keatinge, J.D.H., Easdown, W.J., Hughes, J.D.A., Tenkouano, A., 2011. Small and Palis, F.G., Lampayan, R.M., Flor, R.J., Sibayan, E., 2017. A multi-stakeholder
medium-scale public-private partnerships in horticulture for development: an partnership for the dissemination of alternate wetting and drying water-saving
example from AVRDC - The world vegetable center. Acta Hortic. 921, 25–34. technology for rice farmers in the Philippines. AIMS Agricult. Food 2 (3), 290–309.
Kepes, S., Wang, W., Cortina, J.M., 2023. Assessing publication Bias: a 7-step User’s Pamuk, H., Van Rijn, F., 2019. The impact of innovation platform diversity in
guide with best-practice recommendations. J. Bus. Psychol. 38 (5), 957–982. agricultural network formation and technology adoption: evidence from sub-
Kersting, S., Wollni, M., 2011. Public-private partnerships and GLOBALGAP standard Saharan Africa. J. Dev. Stud. 55 (6), 1240–1252.
adoption: evidence from small-scale fruit and vegetable farmers in Thailand. In: Peterson, K., Mahmud, A., Bhavaraju, N., Mihaly, A., 2014. The Promise of Partnerships:
Paper presented to EAAE 2011 Congress Change and Uncertainty Challenges for A Dialogue between INGOs and Donors. FSG, Washington DC.
Agriculture, Food and Natural Resources, Zurich, Switzerland, 2011-09-02. Ponnusamy, K., 2013. Impact of public private partnership in agriculture: a review.
Kilelu, C., Klerkx, L., Omore, A., Baltenweck, I., Leeuwis, C., Githinji, J., 2017. Value Indian J. Agric. Sci. 83 (8), 803–808.
chain upgrading and the inclusion of smallholders in markets: reflections on Ponnusamy, K., Bonny, B.P., Das, M.D., 2017. Impact of public private partnership model
contributions of multi-stakeholder processes in dairy development in Tanzania. Eur. on women empowerment in agriculture. Indian J. Agric. Sci. 87 (5), 613–617.
J. Dev. Res. 29 (5), 1102–1121. Poulton, C., Macartney, J., 2012. Can Public–Private Partnerships Leverage Private
Kilelu, C.W., Klerkx, L., Leeuwis, C., 2013. Unravelling the role of innovation platforms Investment in Agricultural Value Chains in Africa? A Preliminary Review. World
in supporting co-evolution of innovation: contributions and tensions in a smallholder Dev. 40 (1), 96–109.
dairy development programme. Agric. Syst. 118, 65–77. Rein, M., Stott, L., 2009. Working together: critical perspectives on six cross-sector
Klerkx, L., Adjei-Nsiah, S., Adu-Acheampong, R., Saïdou, A., Zannou, E., Soumano, L., partnerships in southern Africa. J. Bus. Ethics 90 (1), 79–89.
Sakyi-Dawson, O., van Paassen, A., Nederlof, S., 2013. Looking at agricultural Retnoningtyas, H., Yulianto, I., Soemodinoto, A., Herdiana, Y., Kartawijaya, T.,
innovation platforms through an innovation champion lens: an analysis of three Natsir, M., Haryanto, J.T., 2021. Stakeholder participation in management planning
cases in West Africa. Outlook Agricult. 42 (3), 185–192. for grouper and snapper fisheries in West Nusa Tenggara Province, Indonesia. Mar.
Köhne, M., 2014. Multi-stakeholder initiative governance as assemblage: roundtable on Policy 128, 104452.
sustainable palm oil as a political resource in land conflicts related to oil palm Sanyang, S., Taonda, S.J.B., Kuiseu, J., Coulibaly, N., Konaté, L., 2016. A paradigm shift
plantations. Agric. Hum. Values 31 (3), 469–480. in African agricultural research for development: the role of innovation platforms.
Koutsos, T.M., Menexes, G.C., Dordas, C.A., 2019. An efficient framework for conducting Int. J. Agric. Sustain. 14 (2), 187–213.
systematic literature reviews in agricultural sciences. Sci. Total Environ. 682, Sartas, M., Schut, M., Hermans, F., Asten, Pv, Leeuwis, C., 2018. Effects of multi-
106–117. stakeholder platforms on multi-stakeholder innovation networks: implications for
Ladele, A.A., Akinwale, J.A., 2016. Challenges of multi-stakeholder partnerships for research for development interventions targeting innovations at scale. PLoS One 13
agricultural support services provision in rice production in Benue state, Nigeria. (6), e0197993.
Nigerian J. Rural Sociol. 16 (4), 22–28. Sartas, M., van Asten, P., Schut, M., McCampbell, M., Awori, M., Muchunguzi, P.,
Lamboll, R, Nelson, V, Gebreyes, M, Kambewa, D, Chinsinga, B, Karbo, N, Rukonge, A, Tenywa, M., Namazzi, S., Sole Amat, A., Thiele, G., Proietti, C., Devaux, A.,
Sekeleti, M, Litaba Wakun’uma, W, Gutema, TH, Henjewele, M, Kampanje-Phiri, J, Leeuwis, C., 2019. Factors influencing participation dynamics in research for
Masikati-Hlanguyo, P, Quaye, W, Duah, S, Kivuyo, M, Nyanga, P, Akuffobea development interventions with multi-stakeholder platforms: a metric approach to
Essilfie, M, Asafu-Adjaye, NY, Clottey, V, Martin, A, 2021. Strengthening decision- studying stakeholder participation. PLoS One 14 (11), e0223044.
making on sustainable agricultural intensification through multi-stakeholder social Schut, M., Cadilhon, J.J., Misiko, M., Dror, I., 2018. Do mature innovation platforms
learning in sub-Saharan Africa. Int. J. Agricult. Sustain. 19 (5–6), 609-35. make a difference in agricultural research for development? A meta-analysis of case
Lawal, B.O., Akintayo, O.I., Ayoola, O.T., Oyedokun, M.O., Taiwo, L.B., Oyedele, D.J., studies. Exp. Agric. 54 (1), 96–119.
Adebooye, O.C., 2019. Farmers’ participation in vegetable innovation platforms in Schut, M., Kamanda, J., Gramzow, A., Dubois, T., Stoian, D., Andersson, J.A., Dror, I.,
Southwest Nigeria. Acta Hortic. 1238, 157–164. Sartas, M., Mur, R., Kassam, S., Brouwer, H., Devaux, A., Velasco, C., Flor, R.J.O.Y.,
14
M. Maryono et al. Agricultural Systems 213 (2024) 103792
Gummert, M., Buizer, D., McDougall, C., Davis, K., Tui, S.H.K., Lundy, M., 2019. Horton, D., 2016. Multi-stakeholder platforms for linking small farmers to value
Innovation platforms in agricultural research for development. Exp. Agric. 55 (4), chains: Evidence from the Andes. In: Innovation for Inclusive Value-chain
575–596. Development: Successes and Challenges. International Food Policy Research
Seifu, M., van Paassen, A., Klerkx, L., Leeuwis, C., 2022. A state-initiated multi- Institute (IFPRI), pp. 249–268.
stakeholder platform as an instrument to build agricultural innovation system Thorpe, J, Maestre, M, 2015. Brokering Development: Enabling Factors for Public-Private-
capacity: a case study from Ethiopia. Innov. Develop. 1–22. Producer Partnerships in Agricultural Value Chains. International Fund for Agricultural
Sell, M., Vihinen, H., Gabiso, G., Lindström, K., 2018. Innovation platforms: a tool to Development (IFAD, Rome, Italy.
enhance small-scale farmer potential through co-creation. Dev. Pract. 28 (8), Timmer, C.P., 2014. Food security in Asia and the Pacific: The rapidly changing role of
999–1011. Rice. Asia Pacific Policy Stud. 1 (1), 73–90.
Snyder, H., 2019. Literature review as a research methodology: an overview and Tinarwo, J., Babu, S.C., Iyappan, K., 2018. Improving Food System Resilience through
guidelines. J. Bus. Res. 104, 333–339. Better Governance Lessons from Multi-Stakeholder Partnerships in Zimbabwe. The
Sparrow, A.D., Traoré, A., 2018. Limits to the applicability of the innovation platform International Food Policy Research Institute (IFPRI), Washington DC.
approach for agricultural development in West Africa: socio-economic factors Totin, E., van Mierlo, B., Klerkx, L., 2020. Scaling practices within agricultural
constrain stakeholder engagement and confidence. Agric. Syst. 165, 335–343. innovation platforms: between pushing and pulling. Agric. Syst. 179, 102764.
Susilowati, K.D.S., 2018. Creating economic value through multi-stakeholder partnership van Tulder, R., Seitanidi, M.M., Crane, A., Brammer, S., 2016. Enhancing the impact of
(case study on the black soybean farmers development programme of Unilever cross-sector partnerships. J. Bus. Ethics 135 (1), 1–17.
Indonesia). Intern. J. Monetary Econ. Finance 11 (3), 289–297. Turner, J.A., Horita, A., Fielke, S., Klerkx, L., Blackett, P., Bewsell, D., Small, B.,
Swaans, K., Cullen, B., van Rooyen, A., Adekunle, A., Ngwenya, H., Lema, Z., Boyce, W.M., 2020. Revealing power dynamics and staging conflicts in agricultural
Nederlof, S., 2016. Dealing with critical challenges in African innovation platforms: system transitions: case studies of innovation platforms in New Zealand. J. Rural.
Lessons for facilitation. In: innovation for Inclusive Value-chain Development: Stud. 76, 152–162.
Successes and Challenges. International Food Policy Research Institute (IFPRI), Van Paassen, A., Klerkx, L., Adu-Acheampong, R., Adjei-Nsiah, S., Zannoue, E., 2014.
pp. 303–328. Agricultural innovation platforms in West Africa: how does strategic institutional
Teno, G., Cadilhon, J.J., 2016. Innovation platforms as a tool for improving agricultural entrepreneurship unfold in different value chain contexts? Outlook Agricult. 43 (3),
production: The case of Yatenga province, northern Burkina Faso. Field Actions Sci. 193–200.
Report 9. Viteri Salazar, O., Ramos-Martín, J., Lomas, P.L., 2018. Livelihood sustainability
Téno, G., Cadilhon, J.J., 2017. Capturing the impacts of agricultural innovation assessment of coffee and cocoa producers in the Amazon region of Ecuador using
platforms: an empirical evaluation of village crop-livestock development platforms household types. J. Rural. Stud. 62, 1–9.
in Burkina Faso. Livest. Res. Rural. Dev. 29 (9). Zhang, L., Hu, J., Li, Y., Pradhan, N.S., 2018. Public-private partnership in enhancing
Thiele, G., Devaux, A., Reinoso, I., Pico, H., Montesdeoca, F., Pumisacho, M., Andrade- farmers’ adaptation to drought: insights from the Lujiang flatland in the Nu River
Piedra, J., Velasco, C., Flores, P., Esprella, R., Thomann, A., Manrique, K., (upper Salween) valley, China. Land Use Policy 71, 138–145.
Horton, D., 2011. Multi-stakeholder platforms for linking small farmers to value Zhu, G., Li, X., Zhang, Y., 2021. Multi-stakeholder involvement mechanism in tourism
chains: evidence from the Andes. Int. J. Agric. Sustain. 9 (3), 423–433. Management for Maintaining Terraced Landscape in important agricultural heritage
Thiele, G., Devaux, A., Reinoso, I., Pico, H., Montesdeoca, F., Pumisacho, M., Andrade- systems (IAHS) sites: a case study of Dazhai Village in Longji terraces, China. Land
Piedra, J., Velasco, C., Flores, P., Esprella, R., Thomann, A., Manrique, K., 10 (11). https://doi.org/10.3390/land10111146.
15