Water Sanitation Urban Poor

FINAL REPORT
for
Evaluation of Water &

Sanitation for the Urban
Poor’s DFID-Funded
Programme
2012-2015
Jerry Adams
Richard Franceys
Richard Luff
Submitted by: February 2016

Irene Pietersen
ipietersen@intrac.org
+44(0)1865 201851
Oxbridge Court,
Osney Mead,
Oxbridge Court
Oxford,Mead
Osney
OX2 0ES, UK
Oxford
www.intrac.org
OX2 0ES, UK
www.intrac.org
WSUP Evaluation l INTRAC Draft Final Report l February 2016
List of abbreviations
AdeM Aguas da Regiao de Maputo
AFD Agence Française de Développement
APR Annual Progress Review
BC Behaviour Change
CBO Community based organisation
CMM Conselho Municipal de Maputo
CPM Country programme manager
CUA Conselho de Regulação de Águas
DAC Development Assistance Committee
DFAT Department of Foreign Affairs and Trade (formerly AusAid)
DFID Department for International Development
DMA District Metered Areas
DP Discussion Paper
EIB European Investment Bank
ERL Evaluation, Research and Learning
FGD Focus group discussion
FSM Faecal Sludge Management
GAMA Greater Accra Metropolitan Area Sanitation and Water Project
IFI International Financing Institutions
INGO International Non-governmental Organisation
IPP Institutional Pro-Poorness (scale)
IRC International Rescue Committee
KMA Kumasi Metropolitan Assembly
LIC Low Income Consumers
LWSC Lusaka Water & Sanitation Company
MCC Millennium Challenge Corporation
MHM Menstrual Hygiene Management
MLGRD Ministry of Local Government and Rural Development
MoU Memorandum of Understanding
NGO Non-governmental organisation
NRW Non-Revenue Water
NWC Nairobi Water Company
NWSC National Water and Sewerage Corporation
O&M Operation and maintenance
PAWS Partners for Water and Sanitation
PG Programming Guide
PN Practice Note
PP Perspective Piece
PPA Partner Performance Assessment
PTA Parent Teacher Association
RCT Randomised controlled trial
2
SDO Sub-district Officers

SHEP School Health Education Programme
SME Small/medium enterprise
SMEC Small/medium enterprise capacity (scale)
TB Topic Brief
ToC Theory of Change
ToR Terms of Reference
UN United Nations
UNC University of North Carolina
UNICEF United Nations Children’s Fund
USP Unique selling point
VEI Vitens Evides International
VfM Value for money
WASH Water, Sanitation and Hygiene
WB World Bank
WEDC Water, Engineering and Development Centre
WES WASH Environmental Sustainability (scale)
WHO World Health Organisation
WSP Water and Sanitation Program
WSSCC Water Supply and Sanitation Collaborative Council
WSUP Water & Sanitation for the Urban Poor
WT Water Trust
WUC Water Utility Capacity (scale)
3
Contents
List of abbreviations ......................................................................................................................... 2
1. Executive Summary ................................................................................................................... 5
2. Introduction .............................................................................................................................. 12
2.1 Context ........................................................................................................................ 12
2.2 Purpose, Scope and Objectives of the evaluation...................................................... 12
2.3 Evaluation methodology.............................................................................................. 13
2.4 Value for money (VfM) ................................................................................................ 14
3. Assessment and analysis by Outcome ................................................................................... 18
3.1 Outcome One .............................................................................................................. 18
3.2 Outcome Two .............................................................................................................. 39
3.3 Outcome Three ........................................................................................................... 52
3.4 Outcome Four ............................................................................................................. 59
4. Overall Assessment and Conclusions ..................................................................................... 75
4.1 Relevance ................................................................................................................... 75
4.2 Effectiveness ............................................................................................................... 76
4.3 Impact ......................................................................................................................... 81
4.4 Sustainability ............................................................................................................... 82
5. Recommendations ................................................................................................................... 84
5.1 Relevance ................................................................................................................... 84
5.2 Effectiveness ............................................................................................................... 84
5.3 Impact ......................................................................................................................... 86
5.4 Sustainability ............................................................................................................... 87
Annex One – List of consulted documents .................................................................................... 88
Annex Two – Revised work plan ................................................................................................... 89
Annex Three – Evaluation Matrix................................................................................................... 90
Annex Four – Country programme visit criteria for selection and visit outline .............................. 91
Annex Five – Country visit programmes ........................................................................................ 93
Annex Six – List of stakeholders interviewed ................................................................................ 95
Annex Seven – Supporting questions frameworks ........................................................................ 97
Annex Eight – Household and Infrastructure surveys – note ...................................................... 107
Annex Nine – Household and Infrastructure surveys analysis .................................................... 108
Annex Ten – Further data on achievements against Outcome 4 ................................................ 109
4
1. Executive Summary
1. WSUP has developed itself, with DFID support, into a key global resource for facilitating pro-
poor institutional change in lower-income country utilities and municipalities, as well as in
supporting private sector and household involvement in ensuring safe and sustainable public
health improvements.
This independent evaluation has found that WSUP has used DFID funds strategically and
transparently to deliver results over the term of this grant agreement. It has exceeded its
targets to deliver access to WASH for poor urban communities by over 30% and has
influenced utilities and other service providers to provide WASH to a significant number of
indirect beneficiaries. It is also making strong progress in supporting and catalysing
institutional change of service providers. Although funding that WSUP has leveraged was
below target amounts it still represents a very significant level of progress in the period under
consideration. In support of these activities WSUP, has undertaken a series of successful
and valuable research programmes, produced an impressive raft of publications which
document their leading role in pro-poor urban WASH, achieved a notable presence and
stature at international conferences and supported training through its Masterclasses and
Masters modules.
Background
2. WSUP has a strategic portfolio of six countries in Africa and South Asia where it works in
close partnership with local service providers and national government. Through the delivery
of financially viable and effective service provision at a representative scale in target cities,
and by strengthening the capacity of local partners to provide pro-poor services under viable
business models, WSUP aims to trigger investments for the scale-up of service delivery at
the city, and ultimately the national level. WSUP is committed to sharing the learning from its
work with the wider sector to inspire replication and ensure maximum global impact.
3. The DFID-funded programme, titled Water & Sanitation for the Urban Poor and implemented
by WSUP, runs from December 2012 – March 2016. The programme targets the adoption
and replication of effective urban water, sanitation and hygiene (WASH) models by WASH
service providers, national governments and international financing institutions across the six
countries where WSUP has an established presence. It is further characterised by a
comprehensive programme of research, publications and communications. The grant amount
is: £14.9m. The proposed total number of people benefiting from the programme directly is
3.9 million people and a further 12.5 million people benefitting indirectly.
Methodology
4. The evaluation team employed a variety of qualitative and quantitative methods to ensure
that the evaluation was both rigorous in its approach and rich in its content. These include
documentary review1, (semi-structured) interviews; focus group discussions; and analysis of
Household survey in six countries. In addition, the evaluation team undertook approximately
50 individual interviews and carried out two field visits to Kenya and Mozambique.
1 See Annex One for the summary list of documents reviewed by the evaluation team.
5
Verification of data sources

5. To verify the accuracy and plausibility of WSUP reporting, the review team conducted a
preliminary review of the Annual Reports to DFID (June 2015). The field visits, further
secondary data analysis and Skype conversations with key informants and stakeholders
enabled the team to verify the accuracy of the reporting2.
Results
Outcome One
6. Programme reporting provides a positive assessment of WSUP’s value for money per
beneficiary at an aggregated level for all six countries. WSUP reports indicate that this
equates to £3.77/direct beneficiary and £2.97 for both direct and indirect beneficiaries3. This
compares favourably to the estimate given in the proposal to DFID of £8.10/direct beneficiary
and is in the lower quartile of the reported range in the DFID WASH portfolio review of
between £0.91 and £18.80.
7. The numbers of direct programme beneficiaries reported by the programme is significantly

above the projected total (approximately 33%). Whilst the numbers of people with access to
improved water and sanitation are important, their significance is more in their increasing
robustness. Findings from the evaluation support the data from the WSUP Household
survey, to show that people – those at the lower end of the market – having been enabled to
access to water and sanitation, are being translated into use. There is evidence of people
having a firm grip on, and moving up the water and sanitation ladder.
8. An important strength of this programme relates to its twin track ‘accompanying, capacity
building and modelling/piloting’ approach of engaging with communities and service
providers. From the perspective of the communities that WSUP engages with, this is the
point at which the accompaniment they receive meets and engages with utilities and service
providers. WSUP is creating an effective demand mechanism that can work with, and link
effectively to, utilities and service providers who remain the responsible service entities. The
programme also has a very clear emphasis on gender, with women clearly consulted and
represented. This is strongest in the involvement of women in the local groups that discuss
the facilities to be developed.
9. The numbers of beneficiaries who have been indirectly benefitted through the influence of the
WSUP programme are also significant, despite being much lower than projected. It is our
view that the numbers given represent conservative estimates and, more importantly, do not
adequately reflect the critical contribution that the programme is making to scaling up. For
WSUP the numbers of people influenced by the programme provide a strong understanding
of how the programme is being replicated and built on through its direct influence.
2Comment is made on the methodology to count the numbers of people who indirectly benefitted from the
programme.
3 Information received from WSUP 3/02/2016.
6
Outcome Two
10. WSUP had a remarkably ambitious target for institutional change, planning to achieve for 12
cities to get to a 15/25 level (WSUP self-designed capacity measurement scale4). The most
recent WSUP assessment (January 2015) suggests that they have only seen two out of the
ten cities functioning at this WSUP assessed level. However, eight out of the ten cities
reached 10 or above, from a baseline average score of 7.2 and six cities have achieved the
adjusted target of 12.5. Considering the improvement in scores over the period (it being a
very short period in institutional change terms), the figures suggest a more impressive 78%
improvement on average across the nine cities (only 9/10 having a baseline score). Individual
city level improvements have ranged from 31% (for the highest baseline scored city) to 170%
(for one of the lowest baseline scored cities).
11. Additional information reviewed in this evaluation suggests that WSUP’s capacity
measurement scales may not have fully captured the extent of the institutional change
WSUP has delivered. WSUP Country Programme Managers, supported by very able staff in-
country and by the team in London, have done a remarkable job in enabling ‘fragile’
institutions in challenging contexts to move forward in learning to fulfil their ‘service to all’
mandate. The extent of development in the utility ‘pro-poor’ units, also amongst communities
and municipalities, and with the delegated management contractors, is truly impressive.
However, the fragile institutions remain fragile, staff changes and government changes
continually challenge all such improvements and the key value of WSUP being an on-going
presence, as some form of ‘call-down ‘free’ management consultant’ is an invaluable and
necessary resource, at least for the foreseeable future, to ensure these institutional gains are
both maintained and built upon.
Outcome Three
12. WSUP had an equally ambitious target for leveraging US$220m of additional finance
mobilised and made more effective. The figures given to the evaluators suggest that a total of
US$151m has been mobilised by the September 2015 reporting date, adjusted to US$174.4
end February 2016. Of the $151m figure, over 80% is represented by three advisory inputs
with respect to making concessionary finance more effective – one in Ghana and two in
Zambia. That target has been achieved at the 75% level. The target for private finance
leveraged has been met, in effect, but the target for public and households finance is
significantly under-achieved.
13. Although the targets were originally developed from an understanding of likely opportunities
in each country, it is clear that they were over-ambitious. The evaluators have also noted
how it would be possible to adjust the methodology to claim higher numbers for household
investment in water and sanitation for example, and for concessional finance to claim further
additional indirect leveraged finance, at least in the two countries visited for the evaluation.
4 The WSUP Capacity Measurement Scale comprises five sub scales: Water Utility Capacity (WUC) scale; City
Sanitation Management (CSM) scale; Small and Medium Enterprise Capacity (SME) scale; Institutional Pro-
Poorness (IPP) scale; WASH Environmental Sustainability (WES) scale; Institutional Inclusivity (INC) scale. The
total score for a particular scale is generated by adding together the ratings for each sub-scale.
7
14. We believe that these results are a true reflection of progress, and represent a very
significant level of progress in the period under consideration, recognising that both public
and concessional finance, as examples, have such lengthy and unpredictable gestation
periods. We suggest this financing outcome is a useful indicator of progress but likely the
least valuable, being the most uncontrollable, of all the outcome indicators.
Outcome Four
15. WSUP has undertaken a series of successful and valuable research projects with potential,
but as yet unrealised longer term global impact. It is not clear if these will contribute
particularly towards country level objectives, but will likely have more resonance in the
countries in which they were conducted. WSUP intends to move towards a more nationally
relevant, owned and run research agenda that relates more to its, and other national sector
actors going forward. It is considered this shift will bring greater value and more immediate
impact for WSUP’s country level work.
16. WSUP has produced an impressive raft of publications that fill many gaps in the knowledge
market, and document their leading role in pro-poor urban WASH. These are synonymous
with and support the WSUP knowledge leadership brand. They have also achieved a notable
presence and stature at international conferences, drawing heavily from their publications.
Going forward they can be more selective and potentially extend their reach, as they can now
draw upon the recently completed research reports.
17. The Masterclasses are highly effective and greatly valued, delivering immediate and medium
term learning and impact. While they are currently ad hoc, as they do not have an obvious
pathway to a more sustainable learning environment, this can be developed, especially if
WSUP sees these alongside the evolution of the Masters modules. The university accredited
Masters modules are an ambitious initiative which are consistent with WSUP leadership and
demonstrating ways of working. These are really about mid to long-term learning and so
require considerable money and time investment. They may lose out on opportunities for
more immediate impact, and they will not be very effective, unless used by the intended
audience of sector professionals, which was often not the case.
Conclusions
18. Overall our conclusion is that the WSUP Programme and approach is highly relevant to
enabling poor people living in urban areas to be able to access and clean water and
appropriate sanitation through the appropriate national entities. The four outcome areas work
together in a synergistic way. Specifically we would highlight the approach of working with
utilities and poor communities to develop approaches to water and sanitation provision that
engage the two groups in a shared search for appropriate solutions.
19. WSUP’s Unique Selling Point (USP) is to enable low-income country stakeholders to engage
with and deliver services to the poor and fulfil their (not always initially understood) pro-poor
mandate. Understanding the combination of characteristics5 that make up their USP is critical
to understand how WSUP can do what it does. In so doing it helps understand what WSUP
5Long term accompaniment; catalyst and innovator; deep craft i.e. understanding; projects get a
place ‘at the table’; appetite to take action; senor staff with sector expertise; tenacity; independent and
impartial; financial feasibility of poor to pay; building professional awareness.
8
must continue to keep doing and to what extent other organisations can or cannot replicate
and take WSUP’s work to greater scale.
20. WSUP works to support service providers to fulfil their critical pro-poor function in as viable a
manner as possible. In this respect WSUP does not confront duty bearers with their
obligations, rather it works with them through demonstration, capacity building and long-term
support. None the less, advocacy on rights and holding service providers to account are
important tools that WSUP should take account of.
21. In order to lever bigger system change and confront under-investment in sanitation as a
whole, WSUP needs to build some strategic relationships with key stakeholders such as the
WB, UNICEF and WaterAid, in order to lever more support for the sanitation agenda.
22. Given WSUP’s clear focus on the importance of enabling access for people with disabilities,
as well as other people with increased needs, it is important that it provides a consistent and
clear approach to inclusive access, whilst not compromising the legitimate needs of other
users.
23. Learning/knowledge management is a key part of how WSUP can lever change, through its
credibility as an expert, knowing how things really work, being able to innovate and catalyse
change. Given this is so essential, WSUP should be much more strategic in its learning. This
could be supported with better planning, resources and with the addition of some
complementary initiatives.
24. The influencing work that the Country Programme Managers (CPMs) and others do in
relation to Outcomes 2 and 3 is critical and forms part of their ongoing work. Given the lack of
time, measures must be put in place for CPMs to address this and allow CPMs to re-prioritise
work areas.
25. One of the hypotheses that the evaluation explored and tested is that given that WSUP, is
trying to influence utilities, it needs its own core utility experience/partner. Both for the
knowledge of ‘mechanical’ management approaches (with organic elements as and when)
and for the reputational/promotional aspect that might influence fragile target utilities to be
more receptive. It is our view that the quality and capability of the CPMs to open doors may
be as important, if not more important, than having a recognisably competent utility as a
significant partner. We also recognise WSUP’s ability to influence the establishing of a pro-
poor unit in a utility to be second to none.
26. Provision of, and investment in sanitation is a much greater challenge than water supply.
WSUP has rightly focussed a lot of energy and achieved success with its work on FSM and
some very useful work on small scale sanitation finances. Valuable as this work is, there is
now a need to grapple with the fundamental sanitation obstacles. WSUP needs to be able to
balance the aspirations to move towards sewerage when topography allows, to support on-
site as transitional solutions, and to recognize when these are the longer term solution.
27. WSUP has undertaken a successful research programme. However, this route is not without
costs and the careful framing of research questions can become limiting itself. It is suggested
that WSUP could go one step further and step away from the restrictions and costs of high
9
level research and instead see research as just one possible means to answer big questions
and to help unlock innovation and progress moving forward.
28. WSUP has found it challenging to be able to account for the numbers of people influenced by
its work. We suggest that these numbers are very important, both for WSUP as well as for
other sector players. The critical issue for WSUP is to see where it has contributed towards
moving engagement in water and sanitation from pushing and influencing beyond the tipping
point where the changes have become dynamic and have a life of their own.
29. It is critical to consider the extent to which location specific knowledge/understanding can be
replicated elsewhere and/or how much context determines what solutions can be used
elsewhere. It is recognised by WSUP that it must “copy - adapt and not copy – paste”. We
want to stress the importance of maintaining (and further developing) this approach. This
work could be supported by a decision making manual that explains these key context
factors with examples from WSUP’s programme.
30. A key finding from the evaluation is that sustainability is built through the engagement and
buy in of key stakeholders. For hygiene behaviour change it is clear that the key stakeholders
are the communities Principals and Teachers and the local departments of Education. An
issue to consider is how other potential stakeholders such as business could engage to
ensure a harmony of objectives that would see issues such as market penetration and
coverage longer term desirables rather than as specific elements of such a relationship.
31. The programme has a well-developed M&E System. The M&E system can be refined to
support better learning and accountability. We would suggest that WSUP make more use of
its well-developed linkages with communities and its field staff to develop a programme of
longitudinal studies to be able to monitor longer term sustainability.
Recommendations
WSUP has developed itself, with DFID support, into a key global resource for facilitating pro-
poor institutional change in lower-income country utilities and municipalities, as well as in
supporting private sector and household involvement in ensuring safe and sustainable public
health improvements. It can now build on its status and position as a market leader/influencer
to take its role as an influencer to a higher level and use its cadre of ‘model utilities’ and,
most critically, key people in those utilities along with its Programme staff, to influence that
decision making and direction of WASH provision and support by IFI’s and International
donors.
WSUP should focus on extending its programme to two new countries with at least one of
them in Asia. It should develop a more strategic partnership with the World Bank at the global
and national level. It should also build upon selective engagement with UNICEF and UN
Habitat at the national level where they are engaging in urban sanitation work, and, in time,
at the global level. Looking forward, we see a need for more strategic sanitation work to be
undertaken by WSUP that makes the case for a system wide change in attitude to recognise
the need for ongoing sanitation subsidies and appropriate on-site technology. In this it is
important to develop more effective ways to interact with the UK’s water utilities, both to
harness their skills in the more mechanical aspects of utility service and to develop a UK
constituency of support for WSUP.
10
WSUP should take a more holistic view of learning by building a suite of learning packages
and approaches and expand their proposed research programme into an enquiry
programme that can use a variety of means. Recognising the importance of WSUP’s
influencing role, we recommend that it develops its monitoring framework to incorporate
monitoring of sustainability and use.
11
2. Introduction
2.1 Context
Whilst the rapid urbanisation of the last two decades has contributed to bringing millions out of
poverty and helped to bridge the gap between the developed and developing world, it has also
led to significant problems. Many of the people migrating to the cities end up living in urban slum
areas where governments struggle to provide effective water and sanitation systems.
WSUP aims to make a significant contribution to addressing this problem through a tri-sectoral
partnership between the UK private sector, academia and civil society. WSUP believes that
access to safe and sustainable water, improved sanitation and improved hygiene practices
underpins poverty reduction through impacts on health, education and livelihoods. This is
supported by the belief that sustainable and viable improvements can only be made by
strengthening capacity and pro-poor focus in those institutions given the direct responsibility to
deliver services and those entities mandated to support them: public, private or civil society.
2.2 Purpose, Scope and Objectives of the evaluation

The assignment evaluated the Department for International Development (DFID) funded Water &
Sanitation Programme for the Urban Poor Programme (2012–2015)6. The programme is focused
on six countries in South Asia and Africa and targets the adoption and replication of effective
urban water, sanitation and hygiene (WASH) models by WASH service providers, national
governments and international funding institutions. The programme is further characterised by a
comprehensive programme of research, publications and communications aimed at informing
and influencing the sector.
Whilst this evaluation focused on the DFID funded programme, it also considered the input and
contribution of other work carried out by WSUP in the six programme countries – Bangladesh,
Ghana, Kenya, Madagascar, Mozambique and Zambia – through other funding channels, as
these constitute a complementary package.
The programme has four core intended Outcomes that closely map to WSUP’s four strategic
objectives as presented in its organisational Theory of Change (ToC)7. These Outcomes are:
1. Demonstrated models of urban WASH service delivery for the urban poor.
2. Service provider capacity and institutional framework to sustain improvement process.
3. Scale: City wide investment triggered.
4. Influence: Sector influence worldwide.
During the period evaluated, the Programme aimed to improve access to water, sanitation and
hygiene services for 16.5 million people: 3.9 million people directly through WSUP interventions
and an estimated 12.5 million people indirectly through improved capacity and an enhanced
enabling environment for WASH service delivery8.
For this evaluation the Consultants were required to provide an assessment of the programme in
terms of the Development Assistance Committee (DAC) criteria9, namely; A) relevance, B)
6 See Annex Two for the evaluation work plan

7 WSUP Theory of Change 2012-2015.
8 DFID, WSUP Business Case, October 2012
9 Development Assistance Committee (1991) Principles for evaluation of development Assistance. OECD, Paris.
12
effectiveness, C) efficiency, D) impact and E) sustainability, within each of the Programme’s four
outcome areas.
2.3 Evaluation methodology
Evaluation framework
An evaluation framework was developed through the construction of a matrix of the four key
Outcomes described in section 2.2, using the DAC criteria of relevance, effectiveness, efficiency,
impact and sustainability. This matrix is included in Annex Three.
WSUP’s approach, outlined in its Theory of Change, focusses on four key strategic objectives
(corresponding to strategic objectives above).
1. The delivery of service improvements at a representative scale will enable both the
practical application of capacity development of local service providers and removes a key
barrier to a better functioning pro-poor market function through using grant funding.
2. At the same time through the delivery of financially viable effective service provision at a
representative scale.
3. Strengthening the capacity of local partners to provide pro-poor services under viable
business models WSUP aims to trigger investments for the scale up of service delivery at
a city and ultimately at a national level.
4. WSUP’s fourth area of work focuses on sharing learning from its work for the benefit of its
ongoing programmes as well as to the wider sector to inspire replication and to ensure
maximum global impact.
The evaluation approach was based on building up a chain of evidence and analysis that
examined progress to achieving the Outcomes 1 through to 4 of the programme. Our hypothesis
in taking this approach is that it would enable us to build up a credible analytical picture of the
achievement and contribution of the DFID funded programme. Ultimately, the evaluation aimed
to address the major questions of the value added, sustainability and potential for replicating
WSUP’s approach to addressing water and sanitation challenges in urban environments in
diverse environments and contexts.
Key considerations in the proposed methodology

The list of ‘Results Chain Assumptions and Supporting Evidence’ (Annex 1 of the Terms of
Reference (ToR) has been used in developing the question frameworks that were used in the
evaluation process.
In developing the methodology for the evaluation the following key considerations were taken
into account, as they had a bearing on the overall work plan and methodology:
1. An important part of WSUP’s development model is the dynamic linkage between activities
underpinning the four Outcomes. In particular, the ToC indicates that significant work on
Outcomes 1 and 2 needs to be undertaken before significant in-country progress can be
made on Outcome 3 – at least until such time as WSUP’s global influence, both by in-
country networking and as per Outcome 4, permeates through to complementary donors
and finance institutions.
2. Each member of the evaluation team has focused on gathering and analysing the evidence
related to different Outcomes. We refined our approach in that, from that work, the team
worked together to understand the interlinkages between the different Outcomes. Questions
13
were asked in terms of what the evidence is, that work on each of the Outcomes supports
and leads to (and is sufficient for) the other Outcomes.
3. The extent to which each country can be considered representative of WSUP programmes
with sufficient evidence to form a view on the achievements against Outcomes 1-3, and to a
lesser extent Outcome 4, is an important question. WSUP uses a development model which
it believes is highly replicable, though understands the limitations of models being used as
templates.
To address this, the achievements on each of the Outcomes were assessed across each of the
country programmes.
The evaluation did not require a close examination of public health outcomes. For this evaluation
it was only possible to get an indicative understanding of these outcomes through interrogating
WSUP’s reporting, through limited site visits and through interviews with key informants.
Additional insights came from the WSUP managed Household surveys that provided indications
of the demand (through take-up) for the WSUP interventions, and the economics, at a household
level, of accessing better quality water and sanitation services.
Attribution
A key element of the evaluation was to examine the validity of the Theory of Change given in the
WSUP proposal document. The country programme visits were used as an opportunity to test
the Theory of Change against possible counterfactual and alternative approaches.
An important element of the evaluation was to carry out an exploration of the counterfactual –
that is: what are the alternative means by which these outcomes might have been achieved?10
We explored, through the literature and limited interviews, the means by which other
stakeholders have sought to deliver improved water and sanitation to the urban poor and
reflected the effectiveness and efficiency of such approaches against the WSUP model.
The comparators chosen are:
a) World Bank’s (WB) specialist inputs through their Water and Sanitation Program.
b) Water Operator Partnerships (noting some level of overlap with the WSUP approach
through the involvement of Vitens Evides International (VEI) in some of WSUP’s
activities).
c) Suez Environment’s Access to water and sanitation services for all Programme.
d) Water Services Trust Fund (Kenya specific).
2.4 Value for money (VfM)

WSUP has defined VfM in terms of:
 Programme management costs. This is achieved through having small staffing numbers
in both the UK and the six country programmes and through a rigorous management of
programme costs.
 The cost/direct beneficiary (total programme cost divided by total number of direct
beneficiaries).
10 Option 4 WSUP Business Case para 67, page 26.
14
 Finance mobilised by the programme.

We made an assessment of the value for money of the programme through:
 A detailed assessment of costs/direct beneficiary.
 An assessment of the finance mobilised by the programme (estimated target of US$220
million in the Business Case11) across the different areas.
Note: We did not propose to make an assessment of WSUP programme management costs in
terms of value for money. However, the approach to having small staffing numbers will form an
assessment of programme effectiveness and influence.
Gender
Gender is a key cross-cutting issue for the programme12. A quantitative assessment of the
gender impact of the programme could be made primarily through Outcome 1 in terms of the
number of women and girls with access to improved water and sanitation and with improved
hygiene knowledge. This was, however, unlikely to give many insights on the effectiveness of the
programme at meeting the needs of women and girls for access to safe water and sanitation
facilities.
To be able to make an assessment of the gender effectiveness of the programme, the evaluation
team reviewed the documents and reports (for example WSUP Practice Note (PN) 14). These,
together with information gained through the ‘Household survey’ (both through the specific
questions aimed at provision for women and girls), provided insights that were probed further in
the country visits.
Data from reports and case studies was triangulated in the country visits through semi-structured
interviews and (where possible) Focus Group Discussions (FGDs) with women’s groups during
site visits.
Verification of M&E data

An important element of the evaluation was to verify the accuracy and plausibility of WSUP’s
reporting, which was a key data source. The review team conducted a preliminary review of the
Annual Reports to DFID (June 2015). The reports indicated that WSUP is on track to achieve or
exceed the targets for Outcome One (84.4% achievement in both water and sanitation, 400% of
hygiene and 148% in Environmental Sanitation). The field visits, further secondary data analysis
and Skype conversations with key informants and stakeholders enabled the team to verify the
accuracy of the reporting13.
Country programme Visits

An important element of the validation process has been through carrying out two country
programme visits. A consideration for assessing which country programmes to visit was the
11The total volume of finance that is projected to be mobilised by the replication of models developed in this
programme is US$220m (para 87, WSUP Business Case).
12For this evaluation whilst the team gave a special focus to gender we also included as a matter of good
practice concern for and attention to access and disability.
13Comment is made on the methodology to count the numbers of people who indirectly benefitted from the
programme.
15
ability of a country programme to demonstrate a full span of programme interventions across

Outcomes 1-4, and ideally over a range of WASH activity types – water, sanitation and hygiene.
The team developed a set of criteria14 to make the decision which two country programmes
would be most suitable. Scoring of the six country programmes showed clear differences
between the new and more established countries. Looking at the three high scoring country
programmes, the team visited Kenya and Mozambique, rather than Madagascar, as the
programme in Mozambique was understood to have a stronger research evidence base, as well
as the Masters course delivery, as compared with Madagascar. Kenya was understood to have
developed its activities through expansion to additional cities in a manner which allowed for an
assessment of activity development over time. See Annex Five for the country visit programmes.
Data gathering tools

The team employed a variety of qualitative and quantitative methods to ensure that the
evaluation was both rigorous in its approach and rich in its content. These include: documentary
review; (semi-structured) interviews15, focus group discussions; and analysis of WSUP’s survey
data16. The main tools used were:
Semi-structured Interviews with programme stakeholders
The evaluation team undertook approximately 50 individual interviews with WSUP staff, local
government staff, utilities, International Non-Governmental Organisations (INGOs) and sector
stakeholders in the course of the review.
A list of the stakeholders proposed to be interviewed was provided by WSUP. This list was found
to be appropriate and useful: some additional stakeholders were added. Annex Six contains the
list of stakeholders interviewed. A matrix of questions based on the programme Outcomes and
evaluation criteria was developed for use in the semi-structured interviews (over Skype as well
as in-country). The high level matrix is included in Annex Three, Evaluation Matrix, and the
detailed matrix in Annex Seven presents the Supporting questions frameworks by Outcome.
Focus group discussions (FGDs) with communities and user groups
FGDs with community and user groups primarily addressed the research questions under
Objective 1. WSUP commissioned three FGDs with community and user groups in each of the
six countries, as part of the survey process described below. These took place within defined
survey areas linked to intervention types (see sampling for the surveys below).
In addition to these, the team ensured that there was an opportunity to have first-hand
discussions with specific groups of beneficiaries/user groups and service providers. A small
focus group discussion was suggested as the most appropriate format. Lines of enquiry were
based on the question areas in Annexes Two and Seven. FGDs with communities and users
were limited to the two countries selected for visits. However, they provided a triangulation point
to assess the measures of satisfaction with WASH services in the Household survey.
14 See Annex Four and the Inception Report for more details of the selection criteria applied.
15 To assess the effectiveness of the Masterclasses the team followed up on programme participants.
16We would like to note here that, as the designs, sampling and administration of the surveys was done by
WSUP, without input from the evaluation team, the team worked within any limitations that brought and received
prompt clarification about the surveys from WSUP when needed.
16
Household and Infrastructure surveys

WSUP contracted six separate consultants (one in each country) to conduct:
 A Household survey of users (300 households across three defined survey areas17).
 An Infrastructure survey (in two or three defined survey areas).
 At least one FGD (in three survey areas), discussed above.
Analysis of the data from the Household surveys has contributed towards the overall assessment
of the Programme; a note on the surveys is included in Annex Eight and the full analysis is
provided in Annex Nine (a separate document). The Infrastructure survey was less relevant for
this evaluation: the questions were either more operational in nature, or provided similar
information as the Household survey, but without the benefit of a counterfactual.
Following the country visits and the final semi-structured interviews, the team came together to
highlight key findings and issues across the programme. To ensure quality, each team member
was responsible for drawing together the findings and analysis for the Outcome(s) they were
responsible for. Evaluation report inputs were peer reviewed by team members as well as by
INTRAC’s Quality Assessment process18.
17Survey areas were defined contiguous or non-contiguous geographic areas where WSUP has established
under the DFID-funded programme either water, sanitation or hygiene interventions (or a combination of these).
Survey areas were identified and defined by local consultants conducting the surveys.
18 Section 7, INTRAC Proposal.
17
3. Assessment and analysis by Outcome

3.1 Outcome One
3.1.1 Objectives from Logframe
Output One:
 Demonstrated models of urban WASH service delivery for the urban poor
 Programmes delivering service improvements to a representative population of urban
poor in each town/city, demonstrating strong local ownership and informing city wide
water and sanitation programmes
Activity One
 Design and deliver joint programmes of improving water supply in low income/peri-urban
areas of target cities in partnership with water utility, VEI and local civil society;
 Monitor and evaluate success of service delivery
Activity Two
 Design and deliver joint programmes of improving sanitation services in low income/peri-
urban areas of target cities in partnership with the mandated service provider
Activity Three
 Develop partnerships with range of partner organisations to collaborate with: Unilever etc.
 Design and deliver large scale mass media hygiene promotion campaigns focused on
core messages in partnership with the private sector, service provider partners etc. with
support from relevant ministries - WSUP intervention areas
 Monitor and evaluate success of campaigns including uptake of hygiene messages
Activity Four
 Design and deliver joint programmes of improving total sanitation in low income/peri-
urban areas of target cities in partnership with the mandated service provider
18
3.1.2 Outline of Achievements

Table 1 – Numbers of people with improved WASH services19
Outcome Cumulative Baseline Milestone 1 Milestone 2 Milestone 3 Target
Indicator 1 Total (Dec 2012) (31st Dec (31st Dec (31st Dec (31st Dec
2013) 2014) 2015) 2015)
Adjusted to
March 2016
Numbers of Planned 2,076,600: 1,015,000: 1,335,000 1,654,000: 3,995,000)

men, women cumulative f 1,059,576
and children total
m
with improved
1,018,02420
WASH
services: Direct Achieved 2,729,580: 3,674,837: 2,345,081 8,630,205
beneficiaries cumulative
[gender- total
disaggregated:
f - female
m - male]
343,560 541,573 676,115 1,521,041

Direct
beneficiaries
(water)
166980 254,042 578,435 921,874

Direct
beneficiaries
(sanitation)
1,764,450 2,311,010 792,220 4,868,132

Direct
beneficiaries
(hygiene
Direct 454,590 568,212 298,311 1,319,158

beneficiaries
(environmental
sanitation)
19 Source: Household surveys, FGD records, quarterly annual reporting, evaluations

20 Data has not been disaggregated for the final figures.
19
The assessment of the value for money of the programme was made through a detailed
assessment of costs/direct beneficiary.
Table 2 – Value for money
Value for money – programme cost per

beneficiary in GB£
Total Programme Costs £32,552,000
Direct beneficiaries 8,630,205

Cost per beneficiary (direct) 3.77
Indirect beneficiaries 2,315,000
Cost per beneficiary (direct +
£2.97
indirect)
Programme reporting provides a positive assessment of WSUP’s value for money at an

aggregated level for all six countries. WSUP reports indicate that this equates to £3.77/direct
beneficiary and £2.97 for both direct and indirect beneficiaries.21 This compares favourably to the
estimate given in the proposal to DFID of £8.10 and is in the lower quartile of the reported range
in the DFID WASH portfolio review of between £0.91 and £18.80.22
3.1.3 Impact on households using WSUP supported WASH services

The Household survey provides key indications on the quality of WASH services provided by
WSUP, in terms of both tangible benefits for users (access, price, distance) and satisfaction with
the services. The Household survey allowed comparison against non-WSUP users23 (who may
have different profiles) but also users’ previous water and sanitation arrangements. The box
below summarises key findings as a result of these comparisons; further details are in Annex
Nine (a separate document).
Key findings – Household Survey
Water (all six countries)
 The results for Bangladesh appear exceptional, in that the results for WSUP users are
extremely positive both in comparison to other users and when compared with WSUP
users’ previous arrangements; it is also the only country that has focussed on shared
network connections (which are also the most common source for non-WSUP users)
 In other countries the results appear more mixed, and these are also where WSUP users
are using water points/kiosks/standpipes (reflecting the dominant source in their respective
markets).
 There appear to have been major shifts in the types of water source in the market in two
countries (Ghana and Kenya); the survey responses indicate that former seems to be
connect to shifts associated with WSUP programmes, while the latter is more likely to be
exogenous.
21 Information received from WSUP 3/02/2016

22 DFID Water, Sanitation and Hygiene Portfolio Review 2012.
23 WSUP users include those using services that may be supported as well as directly provided by WSUP.
20
 There are indications that WSUP water interventions are benefiting the lower end of the
market within the surveyed areas (which themselves are poorer urban areas). These
indications arise from positive comparisons with previous water source for WUSP users,
but negative comparisons with non-WSUP users. WSUP users also appear to be trading
off price for convenience (see below), which would indicate poorer households.
 A lower relative cost of water for WSUP users appears to be the most consistent result
across all countries; however this is not reflected in satisfaction with price in contexts
where water prices for all users seem to have increased. Improved satisfaction with water
quality is also present in nearly all countries.
 With the exception of Bangladesh, WSUP water sources are showing few benefits in terms
of convenience (availability and length of trip to fetch water) for the average user, when
compared with the alternatives. When compared with previous sources, the length of trip
for WSUP users is reduced in some contexts, suggesting WSUP is benefiting harder to
reach users. However, in some countries there is evidence of availability benefits for those
with the worst provision previously (i.e. at the bottom of the distribution).
 In some countries there have been availability benefits for those with the worst provision
previously.
 User satisfaction with water sources tends to mirror tangible results, except where there
have been major shifts in the market (for example where the price for water across the
market has increased).
Sanitation (four countries)
 For sanitation, both tangible and satisfaction measures are generally positive across the
four countries for which data was available (Bangladesh, Ghana, Kenya and Mozambique),
when comparing WSUP with non-WSUP users and with previous provision.
 The profile of WSUP and non-WSUP users appears relatively similar, suggesting WSUP is
intervening across the market for sanitation within the surveyed areas (which may of
course already represent poorer urban areas).
 Communal/shared toilets are the most common for both WSUP and non-WSUP users, and
household toilets, the second most common overall. However, in three of the four
countries, WSUP users are using those sanitation types in different proportions to Non-
WSUP users.
 WSUP users are predominantly using facilities with septic tanks or ‘sewered’ facilities,
where available. Previous arrangements for WSUP users, and the current arrangements
for non-WSUP users tend to be more basic technologies such as open cesspits, latrines
with cleanable slabs and unimproved latrines.
 There have been significant shifts in sanitation provision for both households using WSUP
supported facilities and those not doing so. The distance to the nearest sanitation facility
has dropped across all countries, while the amount spent per month has increased.
 WSUP users are on average paying more for sanitation than non-WSUP. However, they
are more likely to be satisfied with the amount paid.
 WSUP sanitation facilities are rated very positively on most aspects of quality, when
compared to non-WSUP. The only exception is the comfort during menstruation and
provision of a container for disposal of sanitary products in Kenya and Mozambique.
 Taken together, the findings indicate that WSUP has focussed on improved hygiene (via
the toilet technology) and the quality of sanitation facilities, and that users are willing to pay
for those despite fewer tangible improvements in terms of distance and access.
21
3.1.4 High level comment on progress

People with access
The numbers of direct programme beneficiaries reported by the programme is significantly above
the projected cumulative total (an increase of approximately 33%). The most significant increase
is in the numbers of people accessing hygiene. From the data provided through WSUP’s
reporting it is clear that the programme is demonstrating significant achievement in delivering
water, sanitation and hygiene across its six country programmes.
Whilst the numbers of people with access to improved water and sanitation are important their
significance is more in their increasing robustness:
Figure 1: Illustration of the move of people with access, through use to sustainable impact
• Evidence of
• People with
people using • Maintenance
access
facilities and
Access • Numbers Use Sustainability
Improvement,
• Household
reported
surveys • Evaluations
• Reports
Findings from the evaluation (document review, country visits and interviews) support the data
from the Household survey, showing that numbers of people (especially poorer people), having
been enabled to access water and sanitation, are being translated into use. There is evidence of
people having a firm grip on, and moving up the water and sanitation ladder.
Examples:
 Kenya, Nairobi: Adams/Franceys – upgrading of toilets from communal to sewered
communal to sewered household shared.
 Kenya, Nakuru: improvement by landlords of sanitation facilities following demand for
renting of rooms/that have access to good sanitation.
An important strength of this programme relates to its twin track ‘accompanying, capacity building
and modelling/piloting’ approach working with communities and service providers. From the
perspective of the communities that WSUP engages with and especially the poor, this is the point
at which the accompaniment they receive meets and engages with support, capacity building,
modelling and piloting of utilities and service providers. This point has been made in previous
evaluation reports24 that recognise the relevance and effectiveness of WSUP’s strong
accompaniment approach at community level (Outcome One) that effectively links to its capacity
building of utilities and service providers (Outcome Two). A quote from an evaluation of the
Stone Family Foundation funded Programme in Zambia illustrates this:
WSUP’s ‘advocacy from the inside’ work (combining demonstration work on the
ground and targeted capacity building) is directly influencing the mind-set of
242013. AusAid: Phase I Mid-term Review of Sustaining and Scaling Pro-poor Urban Water and Sanitation
Services in Maputo Initiative.
22
Lusaka Water & Sanitation Company, Water Trusts and other players in the
sector25.
People influenced by the programme – Indirect beneficiaries

The numbers of beneficiaries who have been indirectly benefitted through the influence the
WSUP programme has had on other providers are also significant (see Table 2 above) despite
being much lower than projected (12,560,000). The total number of beneficiaries influenced by
the programme are, however, below the targeted number in the Logframe. It is our view that the
numbers given represent conservative estimates and, more importantly, do not adequately reflect
the critical contribution that the programme is making to scaling up. This is primarily due to the
guidance given26 that has been found to be quite complex to calculate such numbers that country
programme officers have erred on the side of caution when providing information.
The obvious challenge when trying to identify and then count ‘influenced beneficiaries’ is first to
be able to assess when and what changes have occurred that could possibly relate to the
influence of the programme. A subsequent challenge is to assess whether the influence was
direct (as in the example below) or more indirect. In both direct and indirect influencing it is
important to assess the issue of significance. Mayne’s Paper on ‘Contribution Analysis’27 is
helpful in structuring and assessing the evidence needed for this.
As mentioned above, for WSUP the numbers of people influenced by the programme are
important. When reported on for each urban location, they provide a strong understanding of how
the programme is being replicated and built on through its direct influence.
For example, WSUP influencing of Nairobi Water Company (NWC) and the World Bank is
contributing to the development of proposals that, when implemented, will add another 15,000
water and sewerage connections (estimated to be an additional 75,000 people)28.
In this respect the numbers of people influenced by the programme provide an important
indication of how influence is being translated into tangible and measurable change. When
aggregated the numbers of people influenced by the programme represent a helpful
understanding of how the whole programme is having a wider influence.
However, it would also be helpful to be able to differentiate between direct influencing and wider
or more indirect influencing. This point is expanded further in Section Four of this report.
252015. Stone Family Foundation Evaluation: Consultancy to carry out the final evaluation of SFF support to
improved water and sanitation in Chazanga and Kanyama.
26The calculation of indirect beneficiaries is based on the City Capacity Scale scorings: for every city where the
median CCS scoring (of the five sub-scales) at end of programme exceeds 15, the LIC population of that city are
counted as indirect beneficiaries.
27 2008. ILAC Brief 16. Contribution analysis: An approach to exploring cause and effect. John Mayne.
28 Adams Skype interview Mugo Kariuki. WSUP Kenya Country Programme Manager, 09/02/2016
23
Pro-poor29
There is very firm anchoring of the programme to support and engage poor people in the
provision of WASH. This is brought out in the Household survey where analysis of data indicates
that the programme is targeting the ‘lower end of the market’ for water services in the surveyed
areas30.
Whilst there is good evidence to show how poor people are getting a firm grip on the water and
sanitation ladder and are in some instances getting better access and/improving their own
access, there is also evidence to show that some ‘improvements’ can result in a lower quality of
service. For example in Kenya, developments in block/compound housing and latrines in Nakuru
have resulted in a lower ratio of toilets/household as landlords counteract building better latrines
by building more blocks31.
Gender
The programme has a very clear emphasis on gender, with women clearly consulted and
represented. This is most strongly seen in the involvement of women in the local groups that
discuss the types of water and sanitation facilities to be developed.
WSUP is supporting increased participation and leadership by women in WASH interventions,
particularly in WASH committees across its programmes. WSUP is also facilitating discussions to
encourage women to take part in sanitation management and assume lead roles in community
WASH activities32.
Embedding of gender awareness and especially Menstrual Hygiene Management (MHM) for girls
is key. The challenge is for hygiene and MHM work to be embedded and linked to communities
and with local education/health authorities in the same way that WASH is embedded and linked
in communities and utilities. There is some evidence (see WSUP Paper on Gender and WASH
referenced above) of progress in this area. However, the paper also brings out the challenges of
making progress in this area, in particular issues of privacy, security and social taboos. The
following comment from the 2014 Zambia Annual Progress Review (APR) helpfully illustrates the
challenges faced:
On implementation of ‘School of 5’ Hygiene promotion activities: WSUP
conducted the intervention using Stone funding in 50 schools, reaching 100,000
children aged 6-11. However, the WSUP Zambia team shared their concerns that
the intervention has limited sustainability, due to a number of factors beyond
WSUP’s control: these included lack of continuous water supply in some schools,
29 Note: We did not propose to make an assessment of WSUP programme management costs in terms of value
for money. However, the approach to having small staffing numbers will form an assessment of programme
effectiveness and influence.
30 The Household survey only shows pro-poor bias for water. However, it is not the best evidence as the
surveyed areas are all likely to be slum areas. Better evidence of pro poor would be how they chose the wards
that WSUP has focussed on, and the types of water/sanitation that predominate in those areas (i.e. most basic
kinds).
31Adams visit to WSUP Kenya Programme, Nakuru. Interview a ’Block landlord’ in Lawanjiru. 59 households had
access to 23 toilets/bathrooms. Currently the landlord was building more rooms to rent. He was not going to
increase the number of toilets/bathrooms. Increasing houses by 26. Ratio of toilets per family will be 1:4. WHO
guidance is for 1:2 for shared latrines.
32 WSUP Paper: ‘Gender and WASH: Experience and examples from WSUP’s Programmes’.
24
inadequate handwashing facilities, and lack of mechanisms for onward

monitoring33.
3.1.5 Key comment and analysis

3.1.5.1 Relevance
Water, Sanitation and Hygiene
The overall relevance of the WSUP programme in engaging with the challenges of rapid and
increasing urbanisation in Africa and Asia is as relevant now as at the outset of the programme.
The need to expand water and sanitation systems is critical. In order to do this it is essential to
address the constraints that face the sector.
WSUP’s approach based on its ToC, is to engage with communities at the same time as working
with utilities and service providers. WSUP works with communities to assist them to identify the
most appropriate water and sanitation service delivery options (water kiosk, shared connections,
pre-paid dispensers, household connections, community, shared or household toilets) and to
assess the most appropriate financial approach to ensure sustainable service provision.
The engagement of local committees takes many forms from agreeing on the types of toilets to
be built, mobilising the local community to agree the process of building toilets and making their
contribution.
Below an example of participation: a comment from Bangladesh:
We praise the WSUP team’s efforts to ensure community participation. Thanks to
consultation meetings with communities prior to construction, during which the
WSUP engineers explained design options and consulted with the community on
the same, most users feel they participate in decision making. Users also liked
that community-based Infrastructure Implementation Committees (IIC) oversee all
procurement and construction activities34.
The outcomes of this context and market-driven approach are reflected in the results of the
Household survey. WSUP programmes in different countries are deploying quite different mixes
of water and sanitation infrastructure, depending on the prevailing local conditions. In addition,
there are indications that local markets for water and sanitation in some countries have shifted
considerably. In Kenya, for example, both WSUP and non-WSUP users report previously relying
on mobile vendors, yet they were almost completely absent from the market at the time of the
survey. In addition, the price differential for WSUP water and sanitation users (slightly lower for
water, slightly higher for satiation) is relatively small (in spite of significant reported shifts in
prices) suggesting that interventions are appropriate for the markets.
A critical element of this approach is the linking of a clear demand from communities with the
development of an informed and market driven pro-poor approach from utilities. Whilst WSUP is
not explicitly taking a rights-based approach, instead focusing on service delivery to the poor (a
‘universal service obligation’ approach), it is helpful to see how their approach effectively links
‘rights holders’ and duty bearers’ (seeing the utilities as contractors who are acting on behalf of
the government as the ultimate duty bearer) in the supply of water.
33 WSUP Zambia APR 2014.

34This was one of the recommendations of the 2013 APR which WSUP Bangladesh have implemented
successfully.
25
Gender inclusive approach

The purposeful inclusion of women in the community groups and implementation committees is
an essential element in WSUP’s approach and is clearly evident. The findings from this
evaluation agree with previous studies that the programmes clear and meaningful involvement of
women on community based organisations (CBO’s) and User Groups is beneficial. The following
comment from a Focus Group discussion with a group of women committee members illustrates
this perspective:
Women are seen to play an important role in the process. “Women are
responsible for water, sanitation and hygiene for the family. It is therefore
important that women are involved in the process. It is also important to recognize
that men need to be engaged as well”. Comment from women’s FGD Amandla,
Maputo35.
Disability inclusive approach

The messages on evidence of disability inclusive design are more mixed, with a number of
programmes supporting the inclusion of inclusive design for all community and shared latrines,
whilst in other communities the onus was on families to provide support: Examples of inclusive
and non-inclusive approaches are:
 Maputo (Chamancula C Bairro). Inclusive design for people with disabilities in
community/shared toilets, even where there were no people with disabilities in the group
to be served. There was also a recognition that having toilets with more space supported
women and girls /facilitates Menstrual Hygiene Management (MHM)36.
 Kibera in Nairobi community toilets would usually have a step as they would have been
built on a previously constructed pit latrine. During the evaluation site visit (13/1/2016) no
facilities to assist access were observed and when questioned families/attendants stated
that anyone with a disability would be assisted by a member of their family or by the
facility attendant37.
 In Kibera a family were building a household toilet and connecting it to the sewer line.
Their main reason was that the grandmother was elderly and had arthritis and was
struggling to access the communal toilets38.
Inclusion challenges
The evaluation visits to both Kenya and Mozambique noted that the size of toilet cubicles was
often restricted. This was due to either space limitations on the plots, landlord preference or cost.
Whilst this is being accommodated by households, it was noted as a challenge in the Focus
Group Discussion with six female local committee members39.
35Focus group discussion with six women representatives from three local committees in the Bairros
(Communities) of Amandla, Aeroporto B and Chamanculo C, Maputo. January 21 st 2016.
36It is also important to note that this the requirement for ‘disabled access’ was an essential criteria of a DFAT
grant agreement. Adams field visit. Mozambique, January 2016.
37 Adams interview with Zadia, attendant at the Al Safia Washrooms Mabara, Kibera Nairobi.
38 Adams interview with family in Kibera, Nairobi. 13 January 2016.
39 FGD Amandla, Maputo 21/1/16, Adams.
26
Recognising that it is important that the package for toilets has limited options available, in terms
of design, the critical message is that community groups are able to engage in the discussions to
agree on the toilet package they want. In effect, WSUP is creating an effective and responsive
demand mechanism that can work with and link effectively to utilities and service providers who
remain the responsible service entities.
With two contrasting approaches to addressing access for people with disabilities, it is essential
that their needs are taken into account when planning WASH facilities, especially when
considering community facilities that will need to be designed for both current and future needs.
While this point is addressed in the WSUP Urban Programming Guide it is clear that the
application of the principles outlined40 is not consistently being applied and that consideration
should be given as to how it can be integrated into discussions on the development of
appropriate sanitation solutions.
3.1.5.2 Effectiveness
Water
The WSUP programme has a clear and effective poverty focus. This is confirmed through the
results of the Household survey that show a focus and benefit to poorer people. This has been
achieved through WSUP’s lead and championing of a pro-poor focus that supported the utilities
to develop Low Income Consumers (LIC’s) and Pro-poor Units in utilities that have developed
mechanisms to provide affordable and appropriate ways for poorer communities to access clean
water. For this Outcome WSUP has focused on working with communities and the development
of community groups so that poorer people are supported to access and pay for water.
Meetings with Pro-Poor Unit and LIC staff in Nakuru, Nairobi and Mombasa (Kenya) and Maputo
(Mozambique) confirmed the commitment of the utilities in providing services to poorer people,
strongly influenced by the inputs from WSUP. It was evident that in these situations the
recognition of poor people as customers and the support/linkage with local groups was
supporting the development of more robust services and support to consumers, especially poorer
ones41. Whilst the examples given are illustrative of similar progress across all country
programmes, it is important to note that in some situations the LIC Units are not yet sustainable
and will need further support (DWASA Bangladesh42). In Ghana there is a challenge with Ghana
Water Company Ltd that sometimes gives the impression of seeing their poverty focus more as a
means of attracting financial support/funding43.
It is also important to note that in many (if not the majority) of situations, water provision is often
limited by supply constraints. Water resources development has often not expanded at the rate
of population growth and of pro-poor distribution networks. Ensuring fair access for all, through
Non-Revenue Water programmes helps to even out demand and ensure that there is just about
enough to go round. There remains the challenge of how to deal with illegal connections –
40 WSUP Urban Programming Guide; 2014. Crosscutting: Empowering Vulnerable Groups; pp 46 - 47

41 Adams Meeting Pro-poor Unit Nakuru Water; Adams/Luff meeting AdeM, Maputo.
42 WSUP Bangladesh APR 2014.
43 WSUP Ghana APR 2014.
27
something that WSUP has helped to address, for example by running TV adverts in Bangladesh
(in collaboration with WaterAid)44.
Sanitation
WSUP is making strong progress in demonstrating appropriate and relevant approaches to the
provision of sanitation in poor communities. In Ghana there is a strong focus on compound
sanitation that has the potential to achieve citywide scale and to influence sanitation provision at
the national level45. The Ghana and Madagascar programmes have also undertaken initiatives to
develop/enhance laws to ensure access and provision of sanitation facilities, for example:
 Kumasi, Ghana: enhancing existing bylaws that require landlords to provide compound
sanitation46.
 Madagascar: initiative by WSUP and Care on the introduction of a legal text on hygiene,
now being rolled out across the Conselho de Regulação de Águas (CUA). Each
household must have one latrine. In addition, the text places a heavy emphasis on
environmental cleanliness and household guidance on solid waste disposal.
The numbers of people with access to sanitation are important because they are strong/robust
numbers. Of more importance, since it links effectiveness to sustainability and impact, is the
evidence of care and maintenance and of upgrading and improvement of facilities undertaken by
communities, sometimes without further WSUP support. This was observed during the field visit
to Kibera, Nairobi in four main ways:
 Upgrading by moving from communal latrines to sewered communal latrines to
sewered shared compound household latrines and, occasionally, individual
household sewered latrines.
 Improved services: hot showers in Kibera47.
 Moving from communal water points to household water points with a resulting
improved willingness to pay, particularly when accompanied by the lower tariff
enabled by accessing the lifeline block in the tariff structure.
 Improving facilities to enable improved access to people with disabilities48.
An important question to ask about the numbers of people with access to water and sanitation is
whether the numbers are robust, with evidence of access being transformed into use and
improvement? A clear strength of the WSUP approach is the strong accompaniment and the
linkage of rights holders with duty bearers (the formal government service entities), which builds
commitment and leads to embedding and sustaining behaviour change (BC). This, it is

46
WSUP Ghana APR 2014.
47Adams’ field visit to Mabara Community, Kibera, Nairobi 13/1/2016. Example of community latrines with
showers which had been upgraded to provide hot showers (cost KES 10/for a shower). Very popular.
48Adams’ Kibera visit report. Examples of Upgrading from Pit latrines to Improved latrines. Communal toilet block
that had been upgraded from a pit latrine. Rehabilitated toilet blocks used by 15 families. Three toilets and a
shower upgraded from pit latrines. New toilets built on top with drainage to the sewer line. Available to about 40
households. Two examples of families upgrading to household toilet facilities.
28
understood through other research, leads to long term improvements in health. This
demonstrates a strong potential for longer term sustainability and shows that the numbers of
people with access have a good degree of robustness.
Faecal Sludge Management (FSM)

WSUP country programmes have taken a number of different approaches to address challenges
in managing FSM. Supporting the development of local FSM services (for example in Maputo
and Lusaka) and major initiatives in Ghana with the Clean Team and SWEEP in Dhaka,
Bangladesh. The Ghana and Bangladesh initiatives highlight WSUP’s ability to engage local
businesses in addressing the challenge of FSM. The Gulshan Clean and Care business, known
as SWEEP in Bangladesh, has developed strong links with the Dhaka Water Supply and
Sewerage Authority to help them to the business49.
Despite the progress made by the SWEEP and Clean Teams in developing FSM Services, there
remains a challenge for stand-alone private service providers that the fees they can charge are
often not high enough to cover the full costs. This is especially true for poorer families/
consumers. Reading of other Country Reports shows this as a common factor50. For consumers,
specifically poorer consumers who are more likely to have poorer access (being further from an
access point) the cost of pit emptying is often prohibitive/too high. The following example from
Bangladesh illustrates this point:
Communities have taken up the pit emptying service (SWEEP). They are
happy with the quality and speed of service provided – even though they are
more expensive than previous service providers. Currently, WSUP anticipate
emptying costs ranging from 5,000 BDT to 10,000 BDT. According to findings
from the Household survey, such high costs will only be affordable if several
households contribute; it is too expensive for a single household51.
It can also be difficult for small operators to get the right equipment. This was reported as a
challenge at a Focus Group discussion of FSM operators in Maputo52. However, with the
recognition that the amount the operators could charge was not enough, this challenge may be
more about the business model in terms of consumers’ willingness to pay, or for the extent to
which the service needs to be subsidised to achieve societal public health benefits for the
poorest, than about availability of equipment. The WSUP’s report on behaviour research noted
that:
The most important steps towards improving environmental sanitation in Dhaka
is to address the absence of any faecal sludge management system. To
improve the quality and cleanliness of shared facilities, behaviour change
49 These links are illustrated in an article in News Today on 5 February 2016:

http://www.newstoday.com.bd/index.php?option=details&news_id=2405416&date=2015-03-12
50Mozambique: Prices too low for FSM operators. Poorer consumers unwilling/unable to offer to pay more.
Difficult to access some houses. Zambia.
52 Adams FGD Interview with nine FSM operators in Maputo 21/01/2016.
29
targeting the central role that landlords and community managers play can be
particularly effective53.
The critical issue to address is therefore the sanitation value chain: 54

Figure 2 – Sanitation value chain graphic
It is clear that the economics of FSM are particularly challenging. WSUP are involved in trying to
address FSM challenges across a number of their programme areas. In addition to the Clean
Team and SWEEP initiatives, the WSUP Zambia team are working with Lusaka Water trialling an
approach to FSM that aims to introduce a full FSM service to the peri-urban areas of Kanyama
and Chazanga in Lusaka. A key premise of the programme is that it argues that FSM services
are given the best possible chance of success when embedded within a mandated local service
provider that is willing to innovate and continuously iterate55. The FSM service being offered in
these two areas is still at a formative stage and there is clear recognition by the different
stakeholders in the project that it still has a long way to go to be able to claim success.
Unsurprisingly with such a challenging issue, we have not seen evidence that WSUP’s
programming to date has been able to fully address these challenges. It is important to recognise
the work that WSUP is involved in looking to ways to address the FSM challenge. The approach
of WSUP to addressing the challenge of FSM in Kanyama, Lusaka illustrates the role WSUP has
to play in trying to find new modalities for FSM. It is also clear that there is unlikely to be one
solution that can be applied everywhere. Rather, it is more likely that through experimentation
across different situations that models will be developed that illustrate key factors to be taken into
account when developing an appropriate FSM response. For the peri-urban areas of Lusaka a
key factor is seen as the need for a mandated local service provider who is willing to be able to
continually innovate.
53‘Can Behaviour Change Approaches Improve The Cleanliness and Functionality of Shared Toilets?’ WSUP
2016.
54 Image from WSUP Urban Programming Guide 2014.
55 Topic Brief July 2015: Introducing Safe FSM service in low income urban areas: lessons from Lusaka.
30
Maintenance budgets for toilets

Maintenance of toilets has always been a challenge with often inadequate or limited attention
given to having adequate operation and maintenance (O&M) budgets. The Kenya, Bangladesh
and Ghana Annual Progress Reviews in 2014 all noted challenges, primarily due to insufficient
investment and attention to budgeting for O&M56. Specific examples include:
 Chamanculo C, Maputo. Of four communal sanitation facilities visited, the majority did not
have full maintenance budgets (possibly in part because they were new facilities).
 Bangladesh: from a survey, user groups and the Latrine Management Committee are not
yet fully capable of maintaining and operating facilities in a sustainable way. To
elaborate, they do not know when the pit will require empting and how much it will cost.
Moreover, none of the groups/committees has yet established a robust system for
collecting funds to pay for pit emptying and other major maintenance or repair costs.
Survey respondents say they will jointly make a decision on who will empty the pit and
how to come up with the funds once the pit needs emptying57.
There has been some good progress on O&M. As an example the 2015 Bangladesh APR noted:
The sustainability survey results for communal toilets in Bauniabadth and
Roopnagur were highly positive. The toilets in Bauniabadth in particular were
among the most impressive observed by the reviewers in WSUP country
programmes, satisfying key criteria including cleanliness, lighting, ventilation,
security, and O&M financing arrangements.
In considering the challenge of having effective O&M budgets for standalone facilities (networked
facilities being responsibility of the utility) we wonder whether operation and maintenance
aspects follow design and build rather than being seen as integral to it. Taking this approach,
O&M budgets would evolve to take account of operation. For example including better ways to
assess pit faecal sludge levels and requirement for emptying.
Gender
WSUP has been effective at involving women in activities to support the development of
improved access (and use) of water and sanitation facilities. Specifically through the gender
mainstreaming involvement of women on committees:
 Data from Mozambique gives the numbers of women and men on local committees as
257 and 178 respectively.
 Bangladesh: Given that gender inclusiveness, similar to MHM, is a relatively new focus
for WSUP Bangladesh, it is impressive to see how awareness has been created among
target populations – even if they are still small in numbers58.
56WSUP Kenya APR 2014. Section 6. Sustainability Assessment; WSUP Bangladesh APR 2014. Key Findings
(p5); WSUP Ghana APR. School Sanitation.
58Possible exception from Household survey is Kenya and Mozambique where comfort during menstruation and
disposal container were poorly rated for WSUP sanitation facilities in HH survey.
31
There is a recognition by women that improving the understanding of women’s and girls’ needs
must also involve men59. However, it is also recognised that involving men is not always easy.
The 2014 Annual Progress Report from Bangladesh provides a helpful example of the
challenges faced:
….of the 17 community volunteers who had been trained to disseminate
sanitation and hygiene messages only one was male. The challenge identified
was it was easier for women to attend sessions during the day (as many of
them don’t have an official job) – and that it is only culturally appropriate for
women to enter other household’s during door-to-door awareness raising
campaigns60.
Hygiene
WSUP has worked on a number of different approaches to address the hygiene objectives of the
programme, both independently and also with other international organisations, for example
UNICEF, WaterAid and Unilever. It has also forged linkages with the local Ministries and
Departments of Education, for example the Ghana School Health Education Programme (SHEP).
A major partnership has been the ‘School of Five’ initiative, sponsored by Unilever, which
teaches children to wash their hands on five occasions during the day. Whilst strong results have
been reported (see box) there have also been negative reports.
In the Bonsaaso Millennium Village cluster in Ghana, there was a 22% increase in handwashing
among children in the School of Five programme compared to children not in the program. The
average time of handwashing with soap was much longer and the frequency increased. As a
result, the children spent 40% more time washing their hands every day. Lifebuoy also
implemented a School of Five programme in Zimbabwe where there was an increase in
handwashing from eight % before the programme’s implementation in schools to 74% after the
programme61.
However, the long term effect of large scale handwashing and hygiene promotion campaigns on
handwashing behaviour, diarrhoea and respiratory infections is still not known. While intensive
small scale interventions have achieved improvements in these areas, it is not clear whether
realistic and scalable interventions delivered across large areas achieve effects large enough to
be of public health interest. The WSUP Bangladesh Programme for example, reported the
problem of drop off once the programme had finished and soap was no longer being provided.
The London School of Hygiene, together with Unilever and Cliff Investment Trust are embarking
on a study in Bihar, India where a Unilever School of Five programme is being implemented
between 2015 – 2017, to evaluate the effect of this campaign on handwashing behaviour in
school aged children and their mothers, and on diarrhoea and respiratory infections in the same
children and their younger siblings62.
In other initiatives WSUP has worked with schools to support the improvement of sanitation and
hygiene, with a focus on Menstrual Hygiene Management (MHM) across a number of countries:
59 Point made by women at an FGD discussion in Amandla Bairro, Maputo, Mozambique, 01/01/16.
61 2013 Cheeseman G.M. Unilever’s Lifebuoy Soap Aims to Expand Handwashing Program. Triple Pundit.
622015 ClinicalTrials.gov Evaluation of the Unilever Lifebuoy School-Based Handwashing Campaign (School of
5) In Rural Bihar, India.
32
Madagascar. The Wash Friendly Institutions is a DFID funded collaboration of six partners
including WaterAid that is working across schools, churches and hospitals. From its inception in
2002 it has now gained considerable momentum. In 2014 around 1,800 primary schools are
participating (approximately 15% of the total number of primary schools for the country).
Bangladesh. WSUP (through the non-governmental organisations (NGO) Forum) have
significantly increased knowledge on hygiene and sanitation among target communities. This is
great progress towards behaviour change and is complementing construction/rehabilitation of
toilet facilities. Trained community members are very motivated to disseminate key messages
and they demonstrate profound understanding of the value of good hygiene and sanitation. To
illustrate, one female FGD participant emphasised that improving hygiene standards throughout
her community requires everyone to practise good personal hygiene.
Mozambique. WSUP is working with 11 primary schools in one District of Maputo to support the
improvement of essential infrastructure – new toilets and where appropriate improved water
supplies. A critical part of this initiative, which mirrors the initiatives in Madagascar and other
country programmes is the strong linking and development of relationships between parents,
schools and local authorities/Departments of Education.
The benefit of such linkages is seen in the support and development of local initiatives to support
and fund improved MHM management and O&M. The linkage with the Department of Education
provides opportunities to share and potentially replicate achievements to other districts and
areas63.
The Bangladesh Annual Report 2015 highlights the impact of effective advocacy:
We highlight the directive issued by the Department of Primary Education as a
major advocacy achievement for WSUP which has helped to secure buy-in from
District and Sub-district Officers; the sustained engagement of Sub-district
Officers (SDOs) will be crucial to the success of the programme. WSUP has
established a strong rapport with local government administration which needs to
be capitalised on, and through which WSUP now has a “tremendous advocacy
opportunity.”64
It is also encouraging to see that WSUP are exploring innovative ways to make the public aware
of the issues. For example in partnership with WaterAid, WSUP have tried using a TV campaign
to explore innovative ways to make the public aware of the need to hygienically use communal
latrines, saving water, storing water safely, etc.
3.1.5.3 Efficiency
The business case for the programme estimated the expected costs/beneficiary both direct and
indirect of £8.10 per direct beneficiary and £2.60 per indirect beneficiary65. These are estimated
to be well within the range of costs reported in the DFID WASH portfolio review (between £0.91
and £18.8066. Data from WSUP reports gives the costs per direct beneficiary at £3.77 and total of
63 Meeting with Director of Education Municipality of Maputo (CMM) who affirmed the importance of WSUP’s role
in modeling and supporting the improvement of sanitation facilities as a way of engaging parents in supporting
the Municipalities limited budgets and help to embed good practice.
65 DFID (2012) Water and Sanitation for the Urban Poor – Business Case. EDRM no. 3568211.
66 DFID 2012. Water, Sanitation and Hygiene Portfolio Review.
33
direct and indirect beneficiaries at £2.97 (see table 2). The costs per direct beneficiary are
significantly lower than the projected costs in the business case and we suggest this is due to
lower costs. The similarity of the projected cost per direct and indirect beneficiary is most likely to
have been influenced by the significant reduction in the estimates of indirect beneficiaries.
Whilst the cost per beneficiary is helpful to make a crude assessment and comparison of the
efficiency of the programme, it is of more use to dig underneath the numbers to draw out lessons
on potential sustainability and continued relevance.
An important indicator of efficiency is the extent to which infrastructure and ongoing management
costs are recovered via user fees, while still retaining users (including among the poor). The data
from the Household survey shows that user fees for WSUP supported infrastructures are broadly
in line with market rates, taking account of the fact that WSUP may be servicing a poorer
segment of the market. The amount spent by WSUP users is on average slightly lower than the
market rate for water, and slightly higher for sanitation. The indications are that WSUP is not
overly subsidising water and sanitation services:
Table 3 – Amount spent on water for current source (US$)
Amount spent on water for current source, WSUP, non-WSUP users
All in US$ Bangladesh Ghana Kenya Madagascar Zambia Mozambique Average
Non-WSUP 0.281 1.426 1.109 0.404 1.693 1.422 1.056

users
WSUP 0.146 1.419 0.837 0.353 1.156 1.337 0.875

users
Amount spent on water for previous source, WSUP, non-WSUP users
All in US$ Bangladesh Ghana Kenya Madagascar Zambia Mozambique Average
Non-WSUP 0.289 1.131 1.467 1.134 - 1.005

users
WSUP 0.210 0.953 1.360 1.424 (0.934) 0.987

users
Note: local currencies converted to US$.
Non-Revenue Water (NRW)

An important area of efficiency gains for the WSUP programme is in the work done on Non-
Revenue Water (NRW). WSUP has played a critical role with Water Utilities in helping them to
address the problems of Non-Revenue Water through improving leak detection, addressing
issues of illegal connections and vandalism and helping utilities to identify and get payments from
non-paying customers. Whilst a part of this achievement directly relates to the work done with the
water utilities, its success illustrates again the long term benefits of creating an effective linkage
between consumers and customers with utilities and service providers.
Efficiency of service is also gained through developing alternative approaches to paying for water
through pre-payment schemes, such as those in Naivasha and Nakuru in Kenya (though
recognised to be costly to the service provider), discounts for legal connection fees
(Bangladesh), lower water tariffs (Kenya, Nakuru) and staged payments (various programmes).
Benefits of alternative approaches to advanced payments:
34
 Good buy-in from community. Women could go and purchase/top up their fobs/tags with
water credit.
 Stopped fights, evened out supply of water (as a limited commodity), reduced
vandalism67.
Potential challenges to maintaining efficiency gains are:
 Weak embedding of O&M design/management/costs into sanitation.
 Challenges to charging (subsidizing in Bangladesh).
 High cost to utilities of electronic prepaid meters.
 Weakness of pro-poor approach (Ghana Water).
 FSM costs too high for poorer consumers to bear without some level of subsidy.
3.1.5.4 Impact
Numbers of people indirectly supported through influence of WSUP
The influencing numbers that WSUP gives as a result of its work are very important as they
demonstrate how sustainability is moving beyond the local and in this respect they demonstrate
the ‘catalytic’ nature of WSUP’s approach.
The data from WSUP’s reporting shows that the total number of indirect beneficiaries reached by
the programme is 2,315,000. This is significantly different to the projected number given in the
business case of 12,560,000. The methodology that WSUP adopted to assess the number of
indirect beneficiaries of the programme is based on applying their City Capacity Scale (CCS)
scorings:
For every city where the median CCS scoring (of the five sub-scales) at the end of
programme exceeds 15, we count the Low Income Consumer population of that
city as indirect beneficiaries.
WSUP recognises that this approach has not been helpful in estimating the indirect beneficiaries
of the programme. In part this is due to the decision to go for a median rather than an average
score.
Despite the figures being reported, there is strong evidence from the field visits, reports and
conversations with Programme Managers that WSUP is having a significant influence and that
the programme is benefitting substantial numbers of low income communities through the
application of its model of support and demonstration. The following diagram illustrates this:
67 Adams. Notes from Naivasha and Nakuru visit.
35
Figure 3 – Diagrammatic representation of linkage between direct and indirect beneficiaries
Direct beneficiaires.
People benefitting directly
from Water and Sanitation
Key elements of embedding
new models of working and
models of ineraction and
dialogue between
communities and service
providers
Utlilities and Service

providers change their
attitudes and practice and
embed pro- poor services
as part of their business
strategy.
Direct & Indirect

Indirect beneficiaries beneficiaries
People have access to People utiise water and
water and sanitaiton as a sanitation, demand better
result of developments by services that is possible
utilities and service through a better
providers relationship with
utilities/service providers
The following example from the Mozambique country programme illustrates how the programme
is influencing utilities to provide services to communities as a result of their work with WSUP:
WSUP is working with the local utility Aguas da Regiao de Maputo (generally
known by its previous abbreviation as AdeM) in one District in Maputo in a number
of different ways: tertiary network extension, Non-Revenue Water (NRW), Low
Income Consumers (LIC’s) and District Metered Areas (DMA’s). AdeM is
implementing the WSUP model in other areas.
A rough estimate of indirect beneficiaries related to their work with AdeM could be estimated as
follows:
 Network extension: AdeM have constructed 400kms of water supply network. Assuming
that 30% of these were in low-income areas where the model was influenced by the
WSUP model, we estimate that this is equivalent to around 30,000 beneficiaries;
 Non-Revenue Water: since the creation of this department, and WSUP’s support over the
last three years, NRW has reduced from 56% to around 46%. With daily water production
of 180,000 m3/day with 80% delivered to supply, the 10% NRW reduction on 144,000
m3/day means 14,400 m3/day of water recovered; assuming an average consumption
per capita of 90 litre, this represents capacity to serve around 160,000 additional
beneficiaries;
36
 As the District Metering Area model proved to be very positive, AdeM decided to
establish at least one DMA per operational area, so seven DMA are in progress to be
established. We think it is reasonable to assume that in each DMA there will be 12,000
direct beneficiaries and 3,500 indirect beneficiaries, giving a total of 108,500
beneficiaries.
 LIC models have proven to give very positive results regarding invoices delivered meter
reading ratios, collection ratios and billing ratios and better visibility of AdeM close to the
low-income consumers. Because of these benefits AdeM is planning to replicate the
model, gradually, in all low-income areas. Clearly with significant numbers of
beneficiaries.
The evidence presented above primarily indicates that the programme is having a major impact
in terms of equity, i.e. pro-poor services. In addition there is also some evidence, from the
Household survey as well as field visits, of improved benefits to users, e.g.:
 Water of reduced costs, increased quality and in some countries increases in
convenience.
 Sanitation – the impacts are probably more from a public health perspective in reducing
the numbers relying on open cesspits and unimproved latrines (even though this
evaluation has not been able to assess public health impacts,68 existing evidence would
link such improvements to public health benefits).
From the evidence given above it is clear that the WSUP Programme is having a significant
impact beyond its direct programme areas and that this can be estimated. It is also clear that the
numbers of people indirectly benefitting from the programme can provide an important and
helpful indication of the influence of the WSUP programme in contributing to wider catalytic
changes in practice towards the provision of services to low income consumers that needs to be
captured.
3.1.5.5 Sustainability
Water and Sanitation

As mentioned under the sub section on Relevance above, the WSUP approach of forging strong
linkages between utilities/service providers and communities is in effect embedding sustainability
into the process of the programme, rather than it being seen as a separate objective. This results
in a joint commitment to sustainability based on a merging of supply and demand perspectives.
From a service delivery and community perspective it is:
 The combination of strong engagement with local communities – recognising and
encouraging ‘voice’.
 The development and demonstration of appropriate models that are scalable at
community level (they can be improved/adapted to local needs and conditions as well to
enable families and communities to move up the water and sanitation ladder).
 Clear pro-poor focus seen through the development of Low Income Consumer/Pro-poor
Units in utilities with structures to support delivery and of and payment for services.
68We were given substantial ‘informal evidence’ of improvements – fewer cases of diarrhoea, less money spent
on medicine, days off sick etc. – from our field visits.
37
 Effective Non-Revenue Water (NRW) approaches that work with communities and poor
(previously disenfranchised) groups.
 Utilities and service providers want to replicate and expand services.
o Example AdeM in Maputo. The Informal Settlements Department in Nairobi
becoming a self-standing business region in the utility69.
The development of this synergy supports and embeds sustainability and expanding impact in a
dynamic and catalytic way that is evidenced through:
 Expansion of services (direct to indirect beneficiaries).
 Reduced costs due to lower instances of vandalism and illegal connections.
 A better use of an often limited resource – more for all and buying of time as new
resources come on stream.
 Better services for women and girls.
Hygiene
For WSUP’s work on hygiene the challenge with all of its initiatives is how to ensure
effectiveness, impact and sustainability. A clear finding from WSUP’s experience is that it is
essential to understand both the demand and supply side issues and that these then need to be
addressed with local communities, local authorities and with input from organisations such as
Unilever. The challenge for WSUP is how to foster, develop and sustain these relationships. With
a number of potentially good practice examples from different country programmes, the
strengthening of support from WSUP UK is critical in providing support and to encourage cross-
learning. Underpinning this is the need to document and share learning, especially to document
the work in schools and to unpack how to stimulate and sustain social motivation in an urban
context.
Gender
Evidence from the field visits and interviews with women representatives of local committees
suggests that the embeddedness of a strong gender awareness and ‘voice’ is present across
WSUP’s programme areas. At the same time there is also an understanding that it will take time
for progress to move from being embedded to totally recognised as integral, essential and ‘the
way we do things around here’. It will be essential for WSUP to continue its focus and support.
69 Interview with Tim Hayward, WSUP Advisory
38
3.2 Outcome Two
Output Two:
 Service provider capacity and institutional framework to sustain improvement process
 Demonstrable efficiency and effectiveness improvements embedded in service
provider operations and institutional capacity strengthened ensuring the sustainability
of improvements delivered to low income consumers
Indicator 2.1
 Number of target cities achieving a score of 15/25 or more on the WSUP SME
Capacity (SMEC) scale
Indicator 2.2
 Number of target cities achieving a score of 15/25 or more on the WSUP Water Utility
Capacity (WUC) scale
Indicator 2.3
 Number of target cities achieving a score of 15/25 or more on the WSUP City
Sanitation Management (CSM) scale
Indicator 2.4
 Number of target cities achieving a score of 15/25 or more on the WSUP WASH
Environmental Sustainability (WES) scale
Indicator 2.5
 Number of target countries achieving a score of 15/25 or more on the WSUP
Institutional Pro-Poorness (IPP) scale
3.2.2 Outline of Achievements
Table 4 – Achievements against output indicators for Outcome 2

Baseline Target
Output
(Dec (31st Dec 15 & above >12.5 >10
Indicator 2.1
2012) 2015)
Number of Planned 2 12
target cities
achieving a
score of 15/25 Achieved 2 4 2
or more on the
WSUP
Small/medium Achieved % (&
17% (20%) 50% 67%
Enterprise % of 10 cities)
Capacity
(SMEC) scale Source
Quarterly and annual reporting, annual assessments, independent verification
documents
Baseline Target
Output
(Dec (31st Dec 15 & above >12.5 >10
Indicator 2.2
2012) 2015)
39
Planned 0 9
Number of
target cities
achieving a Achieved 3 4 1
score of 15/25
or more on the Achieved % (&
33% (30%) 78% 89%
WSUP Water % of 10 cities)
Utility Capacity Source
(WUC) scale
documents
Baseline Target
Output
(Dec (31st Dec 15 & above >12.5 >10
Indicator 2.3
2012) 2015)
Number of
target cities Planned 1 5
achieving a
score of 15/25 Achieved 3 1 2
or more on the
WSUP City Achieved % (&
Sanitation 60% (30%) 80% 120%
% of 10 cities)
Management
(CSM) scale Source
documents
Baseline Target
Output
(Dec (31st Dec 15 & above >12.5 >10
Indicator 2.4
2012) 2015)
target cities
score of 15/25
or more on the
WSUP WASH Achieved % (&
Environmental 17% (10%) 100% 133%
% of 10 cities)
Sustainability
(WES) scale
Source
documents
Baseline Target
Output
(Dec (31st Dec 15 & above >12.5 >10
Indicator 2.5
2012) 2015)
target cities
achieving a
Achieved 3 4 2
score of 15/25
WSUP 60% (30%) 140% 180%
% of 10 cities)
Institutional
Pro-Poorness Source
(IPP) scale Quarterly and annual reporting, annual assessments, independent verification
documents
Baseline Target
Output
(Dec (31st Dec 15 & above >12.5 >10
Indicator 2.6
2012) 2015)
40
Planned unspecified
Number of
target cities
score of 15/25
WSUP (40%)
% of 10 cities)
Institutional
Inclusivity (INC) Source
scale Quarterly and annual reporting, annual assessments, independent verification
documents
For Outcome Two, reporting, according to the WSUP reported data above, analysed from the six
individual country reports by the evaluators, indicates that WSUP is failing to make strong
progress according to the scoring level target initially proposed. However, taking into account the
longer-term nature of the challenge of institutional development, the adjusted scoring, checking
on performance at the >12.5 and >10 scales, indicates that WSUP is indeed making responsible
progress through building long term, trusted, non-competitive partnerships with water utilities,
municipalities, regulators and private operators across the six programme countries. Capacity
development initiatives supported by WSUP are being embedded into organisation structures.
With regard to the number of people indirectly benefitting from the programme WSUP expects a
total of 5.1 million people to benefit indirectly from the programme overall by the end of Year 3,
41% of the original target. Reporting on internal and partner capacity is informed by an annual
capacity assessment rating tool; the evaluation team reviewed the results of the 2014 round,
reported January 2015. We note that in this subsequent year of operations there are likely to
have been continuing improvements.
3.2.2.1 Overview
WSUP’s Theory of Change is that “demonstrable efficiency and effectiveness improvements
embedded in service provider operations and institutional capacity strengthened, [will] ensure the
improvements continue.” This is expanded to explain in the ToR for this evaluation that “through
the delivery of financially viable and effective service provision at a representative scale in each
city, and by strengthening the capacity of local partners to provide pro-poor services under viable
business models, WSUP aims to trigger investments for the scale-up of service delivery at the
city and ultimately the national level.” This approach is demonstrably relevant to serving the
needs of the urban poor.
WSUP have developed useful and appropriate Capacity Scales in order to track their
performance in these areas. They have chosen to focus upon 1) Water Utility Capacity 2) City
Sanitation Management 3) Small and Medium Enterprise 4) Institutional Pro-Poorness 5) WASH
Environmental Sustainability and 6) Institutional Inclusivity (INC).
We have reviewed the reporting to DFID, which is based on scoring of progress against the six
scales, mentioned above. Whilst there is some case study information given in the reports that
assists in understanding of progress and challenges, we have chosen to focus on the different
types of organisations and assess progress and issues in that way. By doing so we aim to
provide a more nuanced and helpful analysis and comment.
For this evaluation, across the six countries, we have chosen to focus more specifically on the
different types of organisations rather than the institutional responsibilities in the WSUP Capacity
Scale, though the intention is very similar. We recognise there are nine main organisational types
or areas of institutional change to be evaluated:
41
1. Water and sanitation (sewerage) utilities (e.g. JIRIMA, DWASA, NCWS, MWSA,
Naivawass, Nakuru WSC, AdeM, LWSC, GWCL).
2. Municipalities (e.g. CUA, DCC, KMA).
3. Private sector delegated management contractors (e.g. EMA, Maputo).
4. Community/NGO delegated management providers (e.g. Kenyama, Umande).
5. Private sector service providers (e.g. Clean Team, SmartLife, Gulshan Clean &
Care).
6. Government Ministries/policy-makers/environmental agencies/national agents (e.g.
Ministries of Water and Environment; Ministries of Local Government; WSTF Kenya).
7. Government regulators (e.g. WASREB, PURC, NWASCO, CRA).
8. Civil society (e.g. WUA, Madagascar).
9. Other donor programmes (e.g. WSP, WoP, SUWASA, IFIs.)
3.2.3 Key comments and analysis

3.2.3.1 Relevance
This Outcome is the core deliverable for the WSUP programme, utterly relevant, and the
most challenging. WSUP is designed to enable society’s existing organisations - public,
private and civil society - to be more effective in serving the low-income households within
their service areas, delegated or chosen. There are many varied reasons, sometimes good,
sometimes disappointing, often understandable, why a low-income country’s entities are not
delivering effectively to the poor. One key issue to be remarked upon is that the donors have
spent the last two decades requiring service providers, particularly with respect to water, to
be more commercially minded in order to be viable … which has led many of them to a)
remove the ‘free’ stand posts delivering water to the poor, which used to be common-place
(also now therefore, by default, burdening the poor with the cost of paying for the vendor in
addition to the water) and b) to focus on service in areas where the provider can be more
confident of selling viable quantities of water per connection. This in order to service the
fixed costs that make up such a high proportion (80-90%) of a networked service providers
costs.
We agree with WSUP that the service providers and institutions that capacity development
and institutional change is targeted at are fundamentally weak and start from a low base in
terms of human and financial capacity. It is within this context that national and local
government processes, national elections, political unrest and multiple, often competing
demands, from multiple stakeholders puts additional strain on these poorly capacitated
organisations.
The WSUP challenge therefore, as an initially ‘unknown’ external agency, is to deliver
institutional change in a public sector body (usually), which has become ‘frozen’ in a
particular delivery mode with both individuals and organisations benefiting from the status
quo. The Lewin model of change70 explains the need to ‘un-freeze’ an organisation before it
can ‘change’ so as to then ‘re-freeze’ after that change to consolidate and ‘solidify’ the new
70 See for example http://www.change-management-consultant.com/kurt-lewin.html
42
behaviours. This is a hard enough challenge for change agents to deliver when
commissioned by an organisation’s leaders to deliver change. WSUP is tasked to deliver
some level of un-freezing and change as an external agent who, at least to begin with, may
not necessarily be welcomed in this ‘self-volunteered’ role, with the concomitant suspicions
as to motives and benefits.
The necessity to gain credibility, in each country, as a knowledgeable and useful change
agent, along with the need to give early visible results to donors, justifies the initially stand-
alone requirements of Outcome 1. It also explains the more difficult to measure benefits of
Outcome 4, particularly those relating to public sector staff exposure visits, Masterclasses
and long-term Masters module classes. This in the context of the acknowledged long-term
challenge of institutional change: ‘generational development’ or ’20 years’ according to some
practitioners. WSUP has been functioning effectively in its initial countries since 2006. It
could be argued that this present evaluation is taking place approximately half-way through
the necessary change period. And that is assuming that it is possible for organisations to
out-perform the socio-economic trend-line to some extent – that trend-line relating to
effective demand as well as to supply.
The evaluators would also explain that the models of organisational change have been
developed in the context of private sector behaviours in high-income countries. The
organisations that WSUP is required indirectly to change, in the context primarily of the
public sector in low-income countries, described above in WSUP’s description as
fundamentally weak could also be described as ‘fragile institutions’, this description
deliberately appropriating the terminology of ‘fragile states’. The challenges of limited
capacity, politics and changing politicians, and corruption are self-evidently manifest in a
state’s institutions as much as in the state.
The fragile states index not only focuses upon conflict, insurgencies, politics and corruption,
but also recognises the challenges of public services and demography. The demographic
bulge in the countries where WSUP is working is leading not only to the rapid growth of the
urban informal settlements, but also to the challenge of water demand being significantly
higher than achieved water production, let alone water supply – sometimes the demand
supply balance being out by a factor of one third, sometimes by a factor of two or three. To
expect or require an institution to change its approach and actively target serving the urban
poor when it doesn’t have sufficient ‘product’ (water) to deliver to its existing customers in a
viable manner is a challenge. Particularly when that service to conventional customers has
been the target of the previous un-freezing and change programmes, supported by most
institution’s relatively newly designed staff incentives based on commercially oriented key
performance indicators. It is necessary to recognise, in such circumstances, that pro-poor
change takes very particular expertise, as well as time.
Finally in this consideration of relevance, in the context of evaluating WSUP’s progress, it is
helpful to recognise that within the Burns and Stalker71 categorisation of organisations –
‘contingency theory’ – public utilities and municipal entities are focused upon serving, and
geared up in capacity terms, to serve, ‘mass markets’ with ‘simple technologies’. The
resulting organisation is characterised as ‘mechanical’ in that it is designed to be ‘formalised
71Organisational Change as a Development Strategy, Chapter 2, Hage, J. and Finsterbusch, K. (1987) Lynne
Rienner after Burns, T and Stalker, G. The management of innovation, Tavistock, 1961
43
and large scale’. It should be efficient (in theory if not always in practice), ‘producing in
quantity and providing a standardised service or product which can capitalize on economies
of scale.’ Whereas serving the unserved informal settlements, delivering a ‘decentralised,
relatively non-hierarchical service, based on teamwork and networks, specialised around
professional expertise, non-formalised and small scale, which is innovative or adaptive,
produces quality and non-standard goods and services in small numbers’ requires an
‘organic’ organisation. Usually the antithesis of a mechanical organisation and definitely not
an approach that is welcomed in bureaucratic public entities.
So the relevant challenge for WSUP is to enable a ‘mechanical’, ‘fragile’ (also noting the
frequent staff changes) organisation, which has not itself commissioned a change agent, to
incorporate an ‘organic’ element of service provision aimed at a group of potential customers
that it has previously been led to believe should not be served and to which it in reality has
insufficient product to deliver to meet any new demand. This is the target of Outcome 2.
Using the same organisational understanding, the private sector, another key target for
WSUP interventions, whilst undoubtedly more responsive than the public sector, is likely to
have been trapped in a ‘craft’ level organisation approach – meeting ‘local demand for
individual or batch markets run by family businesses or partnerships functioning in local
markets with low capital and skill requirements’. Therefore inherently un-scalable.
The public sector organisations, benefiting from economies of scale in addition to default tax-
based subsidies, therefore have to be ‘nudged’ by various external interested parties into
moving towards a ‘mechanical-organic’ organisation and the private sector has to escape the
‘craft’ service delivery model in an inherently capital intensive sector. Here the private sector
faces the disadvantages of delivering a private cost of capital from the contributions of the
poorest, in competition with the subsidised cost of capital always accessed by the public
provider, where the low-income consumer has to meet significant costs not borne by higher-
income beneficiaries.
The ‘nudging’ referred to defines the need for WSUP itself to be inherently ‘organic’ in its
approach, that is innovative, flexible, responsive, opportunistic with never any assurance that
investing ‘x’ over ‘y’ period will deliver the desired institutional change. A major
understanding of contingency theory, referred to above, is that organisations cannot be both
efficient (low cost of output) and innovative (number of new outputs or procedures) at the
same time. ‘The organic model is designed to be innovative rather than efficient’ (ibid).
We find the assumptions in WSUP’s Theory of Change, and in its scoring approach to
understand that change, to be both comprehensive and relevant. The evaluators have seen clear
evidence of this results chain in action and delivering the desired outcomes, discussed below as
related to effectiveness.
The evaluation of this Outcome 2 challenge, increased capacity and institutional change, is
based on the evaluators visit to the Kenyan Country programme as part of this evaluation
(with development underway in four cities, Kenya representing the most complex approach),
also influenced and informed by earlier visits to WSUP country programmes in Mozambique,
Zambia and Ghana. These visits have been complemented by a detailed review of WSUP’s
(very informative) internal evaluations, monitoring information and the various country
business plans.
44
The overall capacity areas that WSUP uses for scoring are well supported by ‘sub-scales’ to
facilitate as objective as possible scoring. WSUP explains that “The idea is that the scales
provide a holistic assessment of capacity of these partners – they also touch on things like
gender inclusion and environmental friendliness. It is a numeric rating plus narrative comments
and explanations. WSUP teams in each country were asked to complete one rating themselves
(facilitated by the M&E Officer) and request two key partners to self-rate their capacity. All three
ratings will be combined (and an average calculated from numeric rating and narrative
comments.” This is a systematic and sophisticated approach, the addition of partners to self-rate
is a powerful tool to aid the WSUP assessment and to guide partners into recognising the need
for their own assessment tools. However, we note the variability and the ‘optimism bias’ in the
utility self-scoring and have therefore reported figures here based on Country Programme
Manager scoring.
The average improvements in scores per city are shown in Table 5 along with the percentage
development. We recognise that all such scoring is subjective but nonetheless is an important
indicator of progress.
Table 5 – Achievements by city against baseline indicators for Outcome 2
Achievements by city against baseline indicators for Outcome 2
Country Programme Manager

Output Indicator 2 Scoring
Mar 12 - Jan 13 Percentage

Jan-15
Various Improvement
Bangladesh - Dhaka 5.8 10.5 81%
Ghana - Kumasi 5.8 12.5 116%
Kenya - Nairobi 8.8 12.5 42%
Kenya - Naivasha 7.3 13.3 82%
Kenya - Nakuru 8.8 13.2 50%
Kenya - Mombasa 6.2 10.8 75%
Madagascar - Antananarivo 5.8 13.3 130%
Mozambique - Maputo 10.4 13.7 31%
Zambia - Lusaka 5.8 15.7 170%
Average overall nine cities 7.2 12.8 78%
Source : Evaluator's analysis of WSUP Capacity Index
In Kenya, the country programme where most focus was addressed to institutional development
issues, the evaluators discussed the claimed outcomes with utilities at Managing Director,
Technical Director and Pro-Poor Unit leader levels, Private Service Providers, Regulator and
International Financing Institutions (IFIs).
Utility service providers – It is apparent that some countries have found it easier to address the
needs for change in utility service providers, whereas others have had faster initial success
through the supporting or private organisations. There is therefore no straightforward and
constant ‘sample size’ for each of these nine categories. Each country programme has its own
unique mix of involvement and influencing and has rightly and necessarily operated in something
45
of a reactive mode, feeling its way forward and responding to opportunities as and when
available.
Overall it is fair to say that WSUP has had a remarkable success in enabling the establishment of
‘pro-poor units’ (variously labelled by individual utilities) in the main public service providers in
each country where they are operating. To enable the delivery of these ‘organic’ sub-entities with
a differing mix of professional capabilities (sociologists/’marketers’ as well as engineers) is a very
significant step forward and reflects powerful institutional development. This has been delivered
in public organisations that are traditionally reluctant to accept new departments and is a striking
achievement which sets the scene for long-term, sustainable, inclusivity of low-income
consumers. Although the nature of poverty with respect to WASH service delivery may change
over time, it is this evaluator’s experience of the UK water sector that ‘organic’ flexibility in
serving lower-income customers is a requirement that continues.
To enhance availability of water for sale through service extensions, WSUP has, very
appropriately, targeted support to utilities also through programmes of Non-Revenue Water
minimisation. Such programmes necessarily incorporate both technical aspects, to find and fix
leaks in pipes and joints, as well as commercial aspects, to improve billing and find illegal
connections. This has been an important contribution by WSUP to utilities to support the more
conventional ‘mechanical’ aspects of a utility’s operations. By this means WSUP not only
supports the delivery of an important output but also avoids becoming trapped in what others in
the utility business might easily see as a ‘pro-poor ghetto’ with no relevance to the rest of the
organisation. Similarly the technical innovations that WSUP has supported through its pilot
demonstration programmes have had relevance to the rest of the organisation.
Municipalities – The evaluators had limited access to municipalities in their field visits. Earlier
conversations in Ghana, with Kumasi Metropolitan Assembly (KMA) for example, testify to the
value placed by Municipal officials upon having responsive support from WSUP in solving some
of their most difficult service challenges. Conselho Municipal de Maputo (CMM), the Maputo
municipality, was very positive about WSUP’s role and engagement with them and were
supportive of WSUP’s approach. However, their own commitments to supporting sanitation work
was not realised at this stage for a variety of reasons. The sanitation tariff is a key factor to make
the sanitation business environment work for larger scale investment, but CMM appear to be
unable to commit to this to date as the idea is politically controversial. Thus, investments in
sanitation by the municipality remain limited at this stage.
Private sector delegated management – The private sector delegated management
contractors, Maputo as above and Naivasha as another example, are delivering an important
service. WSUP has shown skill and perseverance in supporting them to deliver effectively, along
with enabling them to have a more recognised status as formal sub-contractors. The customer
focus of the delegated contractors is an important signal to the utility, and its pro-poor unit, as to
how a ‘craft’ organisation has to be totally focused upon the interests of its customers in order to
survive. However, the evaluators recognise the importance of such contractors as ‘transitional’
providers that is they have a role which should only continue until the designated utility is able to
fulfil its mandate to serve all consumers in its prescribed area. The transition might take a
generation but ultimately the delegated management contractors will require access to additional
bulk water supply, access to local groundwater supplies being ultimately unsustainable and/or
unaffordable with regard to treatment capacity. And as utilities become more mechanical and
efficient they find that it is more cost effective to absorb delegated outliers with respect to service
provision (though their need to use sub-contractors continues, but with a different focus).
46
This normal development can be seen in the reported coming connection of Mirera, Naivasha to
the utility’s bulk water supply. It is presumed that the community on-selling of that water will
continue in its present form for a while longer (though the fluoride treatment at the kiosks can
presumably be discontinued). This does not negate all that WSUP has delivered in Mirera-
Karagita (and that donors have paid for) over the past years but is a positive sign of an
appropriate evolution of the service provider’s capacity towards long-term sustainability.
Delegated community providers – Similarly to the delegated private providers, the delegated
community providers are being enabled to serve an area where the utility has so far failed. The
work that WSUP has undertaken to formalise the contractual arrangements, enhance the
monitoring and support the technical and commercial expertise of the community providers such
as the Kenyama Trust in Lusaka has been excellent. However, as explained earlier, it means that
poor consumers in such areas pay more than those receiving a supply from the main utility. This
is a better solution than the alternative but again should be seen as a transitional approach.
WSUP should support not only the delivery of better delegated community providers in the
present but also, at the right time, the adsorption of that provider into the formal utility to the
benefit of the consumers. Service reservoirs and distribution networks should be designed with
the transition in mind.
Private sector service providers – The private sector service providers have not been a key
focus of this evaluation though we have been able to talk with the SmartLife manager in Nairobi.
We understand that WSUP Enterprises, under which SmartLife in Kenya, and Clean Team in
Ghana were established, had been acting as a separate entity in order to tap into global interests
for private sector product innovation and development. Our understanding is that however
innovative and creative such enterprises have been, and however good the products they have
developed, they are inherently disadvantaged by being required to deliver a more expensive
product (being unable to take advantage of utilities’ significant economies of scale) whilst having
to price that product to achieve a return sufficient to pay for capital maintenance as well as a cost
of capital if they are to be ‘sustainable’72. This in a situation of serving the poorest consumers
which may be worsened by government taxes on private enterprise and all without being able to
access government subsidies. This rather disappointing overview, recognised by WSUP, does
not negate the work done in this area, as such activities can again form a most useful transitional
service. The key learning is that such approaches have to be able to access subsidies (bulk
water access, enhanced sludge disposal access) if they are to a) achieve reasonable
sustainability and b) not to disadvantage their poor customers.
3.2.3.3 Efficiency
The discussion of the relevance of institutional change also indicates that this is not an area
where it would be reasonable to expect an enhanced level of efficiency. Supporting the
development of the organic capacity of an institution as an outsider, necessarily in a similarly
organic manner, is not conducive to efficiency. However, we recognise that the country
programmes have been run in an extremely lean manner, based primarily on the capability of
very competent Country Programme Managers, supported by visits and consultants as and when
necessary.
To investigate further the level of efficiency of WSUP we have considered other modalities of
supporting utilities and municipalities, the counterfactuals. These are compared in the outline
72 This comment is based on a review of current documentation as well as a number of visits to review the work
of both the SmartLife and Clean Teams by Richard Franceys.
47
below. These alternative approaches show a noticeable bias towards networked water supply,
and are given a) to demonstrate the wide recognition of the pro-poor service challenge in the
international development community, and therefore the relevance of WSUP’s work, and b) to
bring out the particular characteristics of WSUP’s approach.
Table 6 – Successes and limitations of other modalities

Other modalities Successes Limitations
Kenya has the Water Has been strong on acting Has not been able to follow through on either
Services Trust Fund73 as a conduit for pro-poor innovations or institutional pro-poor strengthening.
financing to smaller utilities. WSTF does not have the adaptive, organic,
management consultancy approach that enables
institutional change in fragile situations.
Zambia has the Delivering services to the It is not clear that the DTF, as a public body, has
Devolution Trust Fund poor been empowered to have quite the necessary
flexibility at this time.
USAID Sustainable Successful projects More project-focused approach of SUWASA as

Water and Sanitation delivered in a wide opposed to the more country-focused approach of
in Africa (SUWASA) variety of countries WSUP, the developments initiated by SUWASA are
program unlikely to have ‘re-frozen’ sufficiently to fix the
benefits in place.
GIZ Delivers a significant Does not have the long-term national country
number of pro-poor, utility programme manager relationship
and government
institutional development
oriented activities.
Partners for Water Strong individual PAWS never attained the in-country networking
and Sanitation partnerships and presence or the scale and breadth of
(PAWS) mentoring demonstration pilots leading to lasting institutional
development that have been such a critical benefit
of the WSUP approach.
Water and Sanitation WSP has had to take a Does not have WSUP’s enhanced flexibility and
Program (WSP) much more ‘project’ programmatic approach over a longer time has
approach with delivery of delivered additional benefits. The evaluators were
specified inputs in the hope told that WSP had been trying for some time to get
of achieving desired a low-income consumer unit in Mombasa Water but
outputs within a relatively the WSUP approach subsequently succeeded
short time-frame. within one year.
Ongoing ‘field-level innovative management
consultant’ capabilities of a WSUP will be absent.
Vitens Evides Does impressive work in Much less of the ‘organic’ flexibility of WSUP in
International (VEI). supporting utilities in the creating innovative solutions to serve the poor, not
more conventional being set up to deliver and facilitate pro-poor units
‘mechanical’ aspects of with pilot innovations or to support private sector or
utility water supply. civil society alternative providers in sanitation, for
example.
73 http://www.waterfund.go.ke/
48
Does not have the expertise and deliverables in the

specifically low-income, informal settlements
aspects of service delivery.
Suez Environment Global expertise in water However, the costs of ‘privatisation’, political as
Services For All - and sanitation for the urban well as financial, led to the loss of the majority of
Tailor Made Services poor, based on their global such contracts and the reduction in capability of the
For Unserved Areas’ concession contracts. Services for All department until recently with a
programme resurgence through large-scale pro-poor work in
Algiers and Mumbai.
Respondents have commented that WSP, as one example, has had to take a much more
‘project’ approach with delivery of specified inputs in the hope of achieving desired outputs
within a relatively short time-frame. WSUP’s enhanced flexibility and programmatic approach
over a longer time has delivered additional benefits. The evaluators were told that WSP had
been trying for some time to get a Low Income Consumer unit in Mombasa Water but the
WSUP approach subsequently succeeded within one year. It is understood that the skills
and capacity of WSP, as a separate Trust Fund entity within the World Bank, are being
absorbed into the ongoing World Bank lending operations. This should be to the benefit of
pro-poor investments within larger World Bank financing, but the ongoing ‘field-level
innovative management consultant’ capabilities of a WSUP will be absent. This suggests a
growing opportunity for WSUP, with DFID support, to act as a most useful interface through
utility partners to enable the large scale financial investments that all utilities need to have an
adequate pro-poor focus. An issue we come back to in the evaluation of Outcome 3, next
section.
3.2.3.4 Impact
It can be understood that a public utility exists to serve the poorest (the rich can always manage
to look after themselves) but that a utility can only serve the poorest by accessing the economies
of scale, ability to cross-subsidise, as well as ability to access national subsidies, by serving all
city residents in a standard ‘mechanical’ manner. However, there could well be other
‘mechanical’ areas that WSUP might support for the long-term benefit of the poorest. For
example enhanced bulk water supply and treatment, pumping costs and distribution network
optimisation, telemetry, information technology and billing systems, customer care programmes,
water quality testing and monitoring, etc. could all be supported through consultancies and utility
partnerships. Such areas often being the target of IFI investments we consider that these more
conventional areas of utility operations could benefit from WSUP partnering of utilities to both
enhance and expedite those complementary investments. Referring earlier to the ‘fragile’ nature
of many formal utilities, staff with limited experience and resources are often required to respond
to a wide range of institutional challenges. The constant flow of donors and experts can be
overwhelming, particularly in the context of rapidly changing local political priorities which senior
staff also have to manage. Ongoing peer to peer utility relationships and partnering through
WSUP type support could form an important bridge between utility decision-makers and the
international finance institutions. Those institutions bring very necessary capital for investment
but also have increasingly demanding criteria for feasibility assessment, an area for potential
WSUP support.
We have seen clear evidence of pro-poor service extensions, both for water through communal
provision and through household/yard tap provision, as well as for sanitation through communal
toilets and now household/compound housing sewered toilets. These demonstration pilots,
49
generally being well-based in the work programme of the pro-poor utility units, can only have a
long-term multiplier effect. Both Maputo and Nairobi utilities explicitly volunteered that the work
WSUP had started had now been incorporated into their ongoing programmes.
It is clear from conversations with the regulator in Kenya (and previous conversations in Maputo
and Accra) that regulatory staff value having access to WSUP for ongoing discussions with
respect to areas of concern and interest which then result in consultancy support to deliver
important outputs. The formulation of a sanitation tax in Mozambique and pro-poor indicators in
Kenya are of real benefit to low-income consumers to which the in-country presence and
immediacy of response that WSUP can deliver are a significant contribution to the desired
outcomes.
Conversations and visits with WSUP staff in-country have demonstrated the commitment to low-
income consumers and the broader aspects of civil society and it is apparent that WSUP has
linked to specific civil society organisations whenever and wherever possible and in an
appropriate manner.
We find WSUP’s work in increasing capacity and institutional change to be utterly relevant to the
needs of serving the urban poor. They have achieved this relevance, very appropriately, across a
range of different service providers, including small and medium enterprises, and support
entities. We find clear evidence that service providers have realigned their institutional focus to
be pro-poor. Similarly, with regard to effectiveness we find that service providers are now able to
more effectively deliver services to urban poor, including women and girls. Not always and not
everywhere yet but with very marked improvements across service areas and consumer groups.
In doing so service providers in each category are beginning to improve their efficiency in
comparison with their previous work – as evidenced by regulatory reports in addition to WSUP
capacity scoring. We note that WSUP necessarily supports a range of service delivery models,
some now being more efficient as transition suppliers, even if less efficient than the ultimate
model.
We have noted the impact of these institutional developments, described much more fully in the
reporting of Outcome 1, and see evidence of enhanced institutional sustainability in both the
ongoing roll-out of service access to the urban poor and the subsequent service delivery. We
recognise the limited sustainability of some transitional service providers and, noting the
understandable time-lag in such transitions, recommend innovation in sharing of subsidies (taxes
and transfers) to the smaller scale, private and community, providers. This to overcome their
inherent losses and tendency to limited environmental and social sustainability.
The WSUP results chain assumptions are that these inputs-activities-outputs will enable public
service providers through: “having the capacity to adopt and scale up effective models
contributes to service providers and local/national government committing to adopt and scale up
effective pro-poor models at a city wide level; and that with capacity developed and changed
attitudes to serving low-income areas [will] form part of the core business strategies of service
providers; whilst political developments and social stability do not significantly undermine service
providers’ commitment to scaling up effective pro-poor models at citywide level.”
To the extent possible, the evaluators have verified these reported outputs, in the context of the
results chain, during the country visits and through calls to the non-visited countries.
Overall there is a remarkably strong endorsement of the WSUP approach to facilitating and
supporting pro-poor WASH change. The capability in networking and then using that network to
innovate, pilot service innovations, encourage scaling-up of developments, flexibility support
50
institutions in accepting change (benchmarking visits, in addition to office furniture when

necessary (literally), being highly valued) and the sense of a longer-term commitment to
supporting change, were highly regarded.
We believe that in this most challenging area of institutional development, having been given the
time to respond to the pilot innovations and capacity-building experiences, WSUP’s work has
been remarkably good. Or as one utility respondent reflected “I” We cannot praise too highly the
results that the WSUP Country Programme Managers (CPMs), their national staff and their
international facilitating staff, have achieved in institutional development.
It is of course also clear that at the margins of such varied and necessarily adapting programmes
there are weaknesses where the enabled partner has not been able to follow through at the
required level of quality. For example Mombasa Water was seen to be installing new yard taps
for low-income households that were not robust enough and of sufficient quality to deliver long-
term benefits. Nairobi City Water was not taking advantage of the remarkable new sewers in
Kibera to have trapped disposal points for grey water. However, the nature of WSUP as an
ongoing ‘critical friend’/’management consultant’ will enable such issues to be addressed sooner
rather than later. This would not have been the case in a one-off project intervention.
It is also clear that fragile institutions in semi-fragile states continue to be fragile, both in capacity
and resources and economic growth and management, etc. notwithstanding the benefits of
institutional development. The never-ending change-over of public utility staff in key positions,
when the WSUP investment in training and sensitising appears to be bearing fruit, is both
frustrating to WSUP staff and programmes and a likely regularly irregular set-back. But that is the
nature of development with fragile institutions. The benefit of a longer-presence facilitator such
as WSUP is that they are available to support the next holder of the post, and in the not so
uncommon situations where that new person is not particularly receptive, WSUP has the
capacity to wait for that person to be moved and, with the level of networking demonstrated, to
influence at Board and Managing Director level for the person to be moved. But this is only with
the long-term presence that WSUP has been allowed.
51
3.3 Outcome Three
Output Three:
 Scale - City wide investment triggered
 Pro-poor policies and institutional processes adopted by local service providers
and authorities and supported by financially viable service improvements,
triggering investment from a range of funding sources
Indicator 3.1
 Total volume (US$) of household investment in water and sanitation [in WSUP
intervention districts of all target cities] US$10.67m (US$14.67m)
Indicator 3.2
 Total volume (US$) of public investment in pro-poor water and sanitation [in all
target cities] US$42.23m (US$47.23m)
Indicator 3.3
 Total volume (US$) of private investment in pro-poor water and sanitation [in all
target cities] US$3.54m (US$6.54m)
Indicator 3.4
 Total Volume (US$) of non-national concessionary finance for pro-poor water and
sanitation [in all target cities] US$165.4m (US$190.6m)
Target: US$220m finance mobilised and more effective: households, local private sector, public
sector and IFIs. This in addition to the baseline mobilised finance (US$37.2m) at the
commencement of this programme, the figures in brackets above recognising the cumulative
totals.
3.3.4.1 Relevance
The anticipated inputs for Outcome 3 refer to evidence of WSUP financial models related to
business planning and strategies for different types of service providers, supported by
appropriate contractual frameworks. Through these inputs, and the supporting inputs from the
demonstrated service models and the programmes of increasing capacity and institutional
change, it has been anticipated that WSUP would be able to mobilise significant additional
funding for pro-poor services. This would be achieved by ”engaging with IFI offices, sharing
evidence, participating in project design processes, mobilising grant funding, data collection at
HH level, facilitating private sector investment by entrepreneurs and the local banking sector,
mobilizing service provider budget lines through programmes.”
To monitor the amount of financing leveraged, the WSUP London team developed a guidance
note for use by the organisation. It is important to note that the interpretation of data collected
was on the conservative side, which has led to underestimates due to some investments not
being included.
52
Household finance mobilised includes:

Money that households (water/sanitation users in WSUP programme locations) that would not
otherwise have been spent on pro-poor WASH (e.g. fees for water from water kiosk or communal
stand pipe; fees for use of laundry block or shared sanitation facility). This can include capital
investment in infrastructure or service, recurrent expenditure.
Public finance mobilised includes:

Money that the public sector (e.g. municipalities, local or national governments and line ministries
with the mandate to water and/or sanitation) has spent on pro-poor WASH, that would not
otherwise have been spent on this. This can include public sector investment in water or
sanitation services or hygiene.
Private finance mobilised includes:

Money that the private sector has spent on pro-poor WASH that would not otherwise have been
spent on this. This can include local business investment in sanitation or water services or
hygiene, international private investment in water/sanitation services or hygiene.
Concessional finance mobilised includes:

Investment by International Financing Institutions (IFIs) that WSUP has influenced to be spent on
pro-poor WASH so WSUP influence has caused an IFI investment to be spent in a more pro-
poor manner (or a more efficiently pro-poor manner)74.
With regard to relating activities to outputs WSUP has assumed that “Financial incentives,
businesses cases and plans which demonstrate financial viability affect decision making
processes when making investments in urban WASH, notwithstanding politics and vested
interests; all of the identified sources of scale up finance are in a position to invest in service
delivery improvements.”
Again, we would agree with the caveat that utilities are indeed influenced by political
expectations of the level of services to be delivered as much as by any viability of their business
planning, which is then tempered by the IFI requirements for multiple special factors to be
incorporated in any design, pro-poor service being only one of several. The time taken to come
to an agreement on these issues before any implementation can start explains part of the slow-
take up of leveraged finance claimable by WSUP.
WSUP also assumes that “householders can afford to invest in service WASH improvements for
themselves and their communities”. We find this to be correct but only where any such
investments have been facilitated through staged payments and only where the service is directly
seen as relevant and, quite often, aspirational.
WSUP does not account for finance leveraged according to any distinction between water,
sanitation or hygiene. Based on the scoring for access to improved services, which is
approximately two thirds water to one third sanitation, if the same ratio is being achieved for
finance leveraged we would judge that to be realistic at this stage of development.
Overall, we find these categories and assumptions to be entirely relevant to the finance
leveraged outcome.
74 The full guidance document includes examples of potential finance sources for each category.
53
For Outcome Three early progress has not continued as strongly with the total achievement
reported to end September 2015 of US$151.6m of finance mobilised or better targeted as a
result of the WSUP programme, or 68% the overall programme target of US$228M. Because of
this, WSUP re-forecast its target for Outcome Three to a total of US$182.35m of finance by the
end of implementation – circa 80% of the logical framework target. By end February 2016 WSUP
reported $170.8m finance mobilised (no breakdown given), which is 77% of the original logframe
target, 94% of the revised target.
Table 7 – WSUP mobilised finances
WSUP December 2015 Assessment
Type Target Amount (in US$)
TOTAL – Household $10.7 $4,518,574
TOTAL – Private $3.5 $3,237,219
TOTAL – Public $42.2 $22,766,217
TOTAL – Concessional $165.4 $142,167,781
TOTAL – FINANCE TOTAL $221.8 $172,689,791
WSUP explains that its Outcome 3 target of leveraging US$220m finance was from a ‘bottom-up
plan, country by country’ best judgment at the time of the original business plan – and clearly a
stretch target. Although referred to by some now as a ‘finger in the wind’ type guesstimate, it is
apparent that it was a useful approximation for a target for direct leveraging when there were so
many unknowns as to in-country responses to WSUP’s involvement. As is normal, donor
approaches change over time and, for example, the withdrawal of Australian Aid funding from
urban programmes in Africa, following a change in government, led to a significant reduction in
one aspect of funding. Similarly, the expectation that the then developing link through Vitens
Evides International would leverage finance from the Dutch Foreign Ministry has not come to
fruition, for a variety of reasons we understand.
We note also that there has been no claim with regard to financing leveraged through WSUP
Advisory, unless it has been achieved in one of the six focus countries. This would only ever be
indirect financing, but being based on the experience and knowledge gained through working in
the six focus countries, is clearly an outcome of the DFID support. WSUP defines finance
mobilised as ‘triggered’ (= new money that would not otherwise be spent on WASH) or
‘influenced’ (= money that was already allocated to WASH, but WSUP has influenced the way in
which the money is spent, i.e. for low-income areas)’.
The most recent figures available to the evaluators from WSUP, for September 2015, report
cumulative finance mobilised of US$151.6m as against the target of US$220m. Of this, US$3.7m
is reported as leveraged through households, US$3.2m through private enterprise, US$21.2m
through the public sector and US$123.5m through concessional finance, as in Table 7 above.
54
At country level the total amounts leveraged (to December 2015) are:
Table 8 – WSUP leveraged amounts by country
WSUP December 2015 Assessment (in US$m)
Bangladesh $15.4m
Ghana $33.9m
Kenya $6.6m
Madagascar $5.5m
Mozambique $3.2m
Zambia $108m
We presume that the final figure for finance up to March 2016 will probably be closer to the
US$220 million target. Overall our observation is that there have been other, possibly significant,
amounts of finance leveraged that have not been accounted for. For examples of this see the
comments below regarding WSUP Kenya influencing investment from World Bank through
Nairobi City Water and Sewerage Company (NCWSC) and the Mozambique example of WSUP
evidence of influence of the utility AdeM to invest substantially in pro-poor work in other areas of
Maputo. WSUP Bangladesh supporting the World Bank to work with Dhaka Water and Supply
and Sanitation Authority75 is another example.
Overall we find there is clear evidence of leveraging of finance: we find a range of approaches to
quantify that leveraging. We would advise that indirect leveraging should be explicitly
acknowledged and we would recommend a more realistic suggestion for the next period, target
perhaps being an inappropriate word.
If the larger numbers, related to World Bank or other IFI contracts, can only be claimed in the
context of consultancy or advice those would represent valid outputs of WSUP involvement but
this might better be counted as another category again, perhaps using the word ‘advisory’. This
would lead to three categories of finance leveraging: direct (relating to ongoing WSUP
programme or WSUP acting as an active partner in another agency’s programme and including
discounted cash flows of future consumer payments), indirect (where there is evidence that a
WSUP programme has led to an ongoing use of that approach by others) and advisory where
WSUP special expertise has been contracted for occasional inputs to another agency’s
programme.
Following on from this analysis of WSUP’s reported finance leveraging the evaluation has found
that it has not been straightforward to design finance models which are both scalable and
replicable, being appropriately adapted to specific contexts and stakeholders. Rather, we find
that WSUP’s particular strength has not been to mimic the IFIs in finding scalable models (which
generally have not scaled) but to be remarkably flexible and organic in adapting its solutions and
approaches to the varying contexts and stakeholders. The benefit to stakeholders of having an
in-country WASH management consultant has been to experiment and pilot a remarkable range
of approaches, rather than deciding that there is one model which can be rolled out regardless.
75Skype interview with Abdus Shaheen, WSUP Bangladesh Country Programme Manager (also with Habeeb,
Sanitation Programme Manager and Asis, Water Programme Manager), 10/02/2016, Adams.
55
With regard to the wider changes that have occurred in investment programmes and policy at the
city level we find that all the IFIs and donors have a clear pro-poor agenda and that increasingly
national institutions are recognising the need to follow the same approach. WSUP is facilitating
this process by being exceptionally innovative and creative in each location such that the ‘it won’t
work here’ excuse cannot be used.
The evaluation team has necessarily acted as an auditor to the reported finance mobilised, that
is spot-checking in the countries visited and following through with the team member responsible
as to how they determined the key performance indicator of finance mobilised by households, the
local private sector, the public sector and IFIs.
We find that the figures reported have generally been counted ‘conservatively’, that is staff have
been concerned not to exaggerate or over-count the likely leveraged finance. For example, future
cash flows, for example from households, have not been discounted to present values, as is
normal in economic analysis, therefore not taking advantage of the approximately ten times
multiplier effect of a 30 year cash flow of tariffs that could have been assumed from enabling
household connections.
As an example, WSUP reports that they have facilitated over 5,500 household connections in
Maputo – this has mobilised approximately US$600,000 in household investment through
connection fees. It would have been reasonable to claim the present value of the discounted
cash flows relative to the ongoing future water consumption tariffs leveraged by those new
connections. Similarly ‘a small/medium enterprise (SME) in Naivasha invested US$35,000 in
infrastructure for water supply delivery’ which has also led to households paying long-term for
water consumed from that enterprise.
We have also noted that innovations developed by WSUP that have been taken up by other
donors/financial providers may not always have been counted. We believe that the present
counting system is robust enough for direct financial leveraging but that there is a case to be
made for also recognising ‘indirect financial leveraging’, over and above the direct.
A good example of this indirect leveraging can be seen in the effects of the ‘condominial’
sewerage project in Kibera, Kenya. The WSUP team has counted the household investments
made to connect to the sewerage line developed initially to serve communal toilets and latrine
emptying points. However, it has not counted all of the government funded additional communal
toilets which have subsequently been enabled by the presence of that sewerage development
and that project’s extensions of the sewerage to give an improved service. It has also not
counted the finance leveraged through donors to other projects who are now constructing
condominial sewers in other informal settlements in Nairobi. However, Nairobi City Water and
Sewerage Company confirm that all such projects flowed from the visit supported by WSUP for
NCWSC technical staff to visit Brasilia, the ‘home’ of condominial (community/reduced cost)
sewerage. There they became convinced by the approaches and materials used and accepted
this solution for use in Nairobi, both for the immediate WSUP supported project and for all
subsequent sewerage projects in the informal settlements.
WSUP does not account for finance leveraged according to any distinction between water,
sanitation or hygiene. Based on the scoring for access to improved services, which is
approximately two thirds water to one third sanitation, if the same ratio is being achieved for
finance leveraged we would judge that to be realistic at this stage of development.
Overall we believe that WSUP has been mobilising finance and making it more effective at a rate
that is consistent with ensuring that it really is effective in improving services to people in very
challenging situations. It appears that the US$220m objective has not been achieved but there
has been a good leveraging across the main stakeholder components. Changes in investment
56
patterns that have been observed are directly relevant, have been as efficient as the critically
necessary ‘organic’ flexibility and innovativeness allows, have had a significant impact and are
generally sustainable when delivered with the framework of a utility or delegated water
management contract.
3.3.4.3 Efficiency
With regard to timing of investments and timely release of funds for investments, we recognise
that the large investments with IFI support continue to take a long time to progress through the
project cycle and that implementation capability when funds are eventually available for release
is as limited as always. Again, it is to the credit of WSUP that they have been able to make so
much progress in such a relatively short time in a sector which is inherently slow to absorb new
developments.
Any levels of efficiency that can be surmised from the information available are heavily
influenced by the amounts leveraged through international concessionary finance. We have
therefore looked more closely at the US$123.5m claimed under that heading. This includes three
very significant amounts, one from Ghana and two from Zambia, thus implying very different
levels of efficiency in leveraging finance between those two countries and the other four, as in
Table 8, above.
These three amounts include a US$30m claim for concessional financing leveraged in Accra,
Ghana (not Kumasi as initially reported) and the US$86m for concessional finance for two
projects in Zambia. It has not been possible to audit these numbers through country visits.
Representing 94% of the concessional finance reportedly leveraged, which in itself represents
over 80% of all the leveraged finance, these two country claims have been questioned through
direct communication with WSUP London.
WSUP has a Memorandum of Understanding (MoU, not a contract as previously reported) to
develop technical recommendations for the low income consumers component of the Greater
Accra Metropolitan Area Sanitation and Water Project (GAMA). US$30m of the total US$150m
budget has been allocated to improving services to low income consumers. The MoU with the
Ministry of Local Government and Rural Development (MLGRD) clarifies WSUP’s role in working
alongside WB projects under different lines of funding to better focus the Bank’s funding on
effective delivery of sustainable water and/ or sanitation services for LICs. We accept the claim of
this leveraging of finance.
In Zambia WSUP Advisory was subcontracted to design the €35m LIC component of a €160m
project with Mulonga Water Company in Zambia to be financed by the European Investment
Bank (EIB) and Agence Francaise de Developpement (AFD). WSUP Zambia team, it has been
explained to us, have had a strong involvement in this sub-contract. Also in Zambia, WSUP has
a signed MoU with the Millennium Challenge Corporation (MCC) in Zambia to assist in better
targeting its US$350m infrastructure investment in Lusaka, to ensure impact for low income
areas. The MoU was signed in 2014 and is valid for three years, so this work is ongoing. WSUP
claimed US$35.5m in 2014 with respect to this involvement. This again would appear to be a
good example of leveraging of finance, depending upon the specific role in ‘working alongside.’
WSUP advises that they have had direct influence on the MCC programme ... “but it is so difficult
to measure”.
57
3.3.4.3 Impact
In considering to what extent effective pro-poor targeting of low-income components within IFI
investment projects provides examples for further replication and scale-up in other countries and
contexts we find that the ‘mechanical’ aspects of water and sanitation service delivery are very
scalable but that the nature of the areas where very poor people develop their informal (/illegal)
housing are so variable, that the capability of the ‘organic’ approach of WSUP staff is critical and
non-scalable. For example, the geological conditions in Kenyama, Lusaka require such different
solutions (blasting for trenches and raised latrines due to regular flooding) than the social
conditions in Mukuru, Nairobi (very high density compound ‘dormitory’ housing) to the
institutional conditions in Mombasa (water vendor conflicts), etc.
It appears to us that the experience to date, and the types of financial leverage being achieved,
indicate that specific financial models (over and above the basic model of a capital intensive
networked utility being able to achieve economies of scale in delivery of low cost services to all)
are not particularly scalable.
Investments tend to be challenged in their sustainability, particularly for sanitation, where the
sector has assumed that there really ought to be self-financing profitable stand-alone circular
economy sanitation solutions in low-income economies, with the poorest paying at full cost-
reflective rates, when all our own now high-income country historical experience suggests
otherwise.
In the context of concluding this section on leveraging financing it is perhaps helpful to consider
the extent to which the present expectation for sustainable pro-poor services is at some level a
distraction. This is because society has ‘always’ subsidised the rich to have networked water and
sanitation, through taxes and now transfers in many countries, so it might be seen as illogical to
require the poor suddenly to be cost-reflective consumers. It is a bit like requiring ‘sustainable
welfare payments’. The task is to enable public sector entities, society’s chosen agents, to
discharge their responsibilities to achieve public health protection for all. This has always
required a mix of tariffs and taxes in high-income countries, only delivered as societies have
allowed their wealth to be invested in incremental public health improvements rather than other
desired societal benefits. Tariffs alone have rarely been sufficient in delivering potable water and
environmentally acceptable wastewater disposal. Now that global society desires such
improvements to be made ahead of any particular country’s economic carrying capacity, it would
be illogical to demand that pro-poor services should be funded purely by tariffs. This is even truer
where the appropriate solution is through local private enterprise. Private entities requires full
cost recovery, including a full cost of capital component (‘profit’) without any possibility of cross-
subsidy within the city. A burden the higher-income households connected to the public system
have never had to bear.
Nevertheless, it is clear that in the six countries under consideration additional finance is being
mobilised and made more effective through the inputs of WSUP. Households, communities, the
local private sector, the public sector and IFIs are being enabled to contribute additional finance
to support better water and sanitation services for all.
58
3.4 Outcome Four
Outcome Four:
 Influence: Sector influence worldwide
 Learning and analysis documented and disseminated using a range of media,

challenging the sector on urban water and sanitation service delivery and informing the
design and delivery of large scale investment programme
ACTIVITY 4.1 – Research and Analysis
i) Identify, design and implement major research projects in four core areas: market-driven
sanitation models, faecal sludge treatment technologies, progress linked finance approaches
and impact evaluation and monitoring
ii) Produce high-quality analysis and lead sector debate on innovative and financially viable
process and intervention models for pro-poor urban WASH using results from SO1 to validate
models
ACTIVITY 4.2 – Documentation and Dissemination
i) Influence institutional decision-making and private-sector initiatives in the focus countries,

through coordinated country-level strategies centred around a) targeted meetings and sector
workshops with key stakeholders, and b) in-country documentation
ii) Document and disseminate programme experience through publications (Practice Notes,
Topic Briefs (TPs) and Discussion Papers (DPs)) and wider communications strategies aimed
at targeted audiences in-country and internationally
ACTIVITY 4.3 – Training
i) Develop MSc module with a sector leading academic organisation
ii) Partner with in-country learning institutions and roll out MSc module
iii) Develop short professional training course for service provider staff to be delivered at
national/ regional/international level as appropriate
3.4.2 Outline of achievements (see Annex Ten for full details and analysis of outputs)
Output Indicator 4.1 Target

(31st Dec 2015)
Number of research Planned 6
projects implemented, (cumulative total ) (8)
documented and
disseminated Achieved 4
(cumulative total) (6)
Comments others being finalised (complete by Mar 2016)
59

(31st Dec 2015)
Number of WSUP Planned 15 PNs & 6 TBs (23 PNs & 8 TBs)
publications produced and (cumulative total )
disseminated (Practice
Achieved 11 PNs, 9 TBs, 3 DPs, 3 Perspective Pieces
Notes - PNs; Topic Briefs -
(cumulative total) (PPs), 2 Programming Guides (PGs)
TBs)
Comments expects to publish seven additional
publications before 31st March 2016
(31st Dec 2015)
Number of people from Planned 150: 75 f/75 m
low-income countries (cumulative total )
receiving
Achieved 201. (Estimated to be 17 females but this is a
professional/postgraduate
(cumulative total) guess from names where listed).
training linked to the
WSUP programme Comments Masters module rolled out in 5 WSUP
[gender-disaggregated: f - countries with the 6th - Zambia cancelled so
female/m - male] numbers down 15-20 or so.
2 Masterclasses ran in Uganda and Zambia
Outcome Indicator 4 Planned Target 15
Number of WSUP models (cumulative total)
documented, disseminated
Achieved WSUP response awaited
and informing the sector
(cumulative total)
Comments
Other key activities not included in logframe or without targets set

Conferences (international) – Target; none specified.
 26 conferences in total, with 14 attended in 2015, a few in 2013 (exact numbers
unknown).
Masterclasses – Target; none specified.
 Masterclasses have been run at Africa regional level in 2013 - 60 people (Uganda) and
2015 – 54 people (Zambia). (No gender breakdown).
3.4.3 High level summary of Achievement

The work undertaken under Output 4 seeks to document and disseminate learning and analysis
in order to challenge the approach to working with urban LICs, both nationally and internationally,
in order to make efforts more pro-poor. Under this work stream a series of knowledge products –
research papers, publications, conference outputs, non-published papers from Masterclasses
and a Masters module – have been produced (see Annex Ten for full details of what was
undertaken). Much of the leadership and organisation for these activities and associated
knowledge products came from the global level, specifically from the Evaluation, Research and
Learning (ERL) team.
60
WSUP and WSP are really the only two organisations that have developed in depth knowledge
products in this field, noting some contributions from SUWASA. Most of WSUP’s specialised
urban WASH knowledge for LICs has been produced in just over three years. WSUP has added
as much to this body of knowledge to that produced by WSP over the course of its 25 years of
existence. Therefore, WSUP has established itself convincingly and credibly as a knowledge
leader within a brief time frame, which makes this effort all the more impressive.
Our understanding of WSUP’s unique selling point (USP, see section 4) includes three
characteristics that WSUP brings that relate to WSUP’s work under Outcome 4;
 Really knowing how things work, i.e. deep craft as illustrated in its documentation.
 Showing financial feasibility of/for poorer customers, with much documentation in this
area.
 Identifiable as a catalyst and innovator both in thinking and action, which is demonstrable
in its knowledge products.
In addition to these knowledge products, key staff at the national and global level bring significant
breadth and depth of expertise and so are the key actors who play a critical knowledge
leadership role. It is by being a credible knowledge leader, underpinned by these knowledge
products, that WSUP is able to lever influence. Beyond the specific DFID programme, WSUP
Advisory supported by WSUP’s Sanitation Advisor Georges Mikhael is working on a WSUP
Advisory led programme in India, which provides an example of knowledge leadership. However,
the influencing/knowledge leadership work of key individuals is not projectised and so is clearly a
less tangible and measured part of WSUP’s influencing work. It is highlighted as a critical, yet
often invisible, aspect that must be supported by a strong institutional learning culture, a point
explored more in section 4.
WSUP included three key questions in the ToR in relation to Outcome 4, which are addressed in
the summary below and in more detail in the following section:
1. What has been the impact of dissemination activities conducted under this programme on
external perceptions of WSUP’s work? Are WSUP’s current dissemination channels
conducive to achieving WSUP’s influencing objectives?
2. What evidence exists for the uptake of service delivery models implemented under this
programme by other service providers and WASH-implementing organisations?
3. To what extent have the research projects commissioned under this programme
contributed to country-level objectives?
Overall the collective knowledge products that WSUP have developed fill clear gaps in the
knowledge market, adding considerably to what little documentation there has been by others.
This has created a very positive perception of WSUP’s work and it is considered that this will be
highly influential. The broad range of dissemination channels adopted definitely support WSUP’s
influencing objectives. These could be enhanced going forward. There is definite evidence of
uptake of service models, as described elsewhere in this report, though of course knowledge
products are just one, though critical, part that supports this uptake.
Research – WSUP has undertaken a series of successful and valuable research projects with
potential, but as yet unrealised longer term global impact. It is not clear if these will contribute
particularly towards country level objectives, so will have more resonance in the countries in
which they were conducted. WSUP intend to move towards a more nationally relevant, owned
and run research agenda that relates more to it and other national sector actors going forward. It
61
is considered this shift will bring greater value and more immediate impact for WSUP’s country
level work.
Publications – WSUP has produced an impressive raft of publications which fill many gaps in
the knowledge market and document their leading role in pro-poor urban WASH. These are
synonymous with and support the WSUP knowledge leadership brand.
Conferences – WSUP has achieved a notable presence and stature at international
conferences, drawing heavily from their publications. Going forward they can be more selective
and potentially extend their reach, now being able to draw upon the recently completed research
reports.
Masterclasses – The Masterclasses are highly effective and greatly valued delivering immediate
and medium term learning and impact. While they are currently ad hoc, as they do not have an
obvious pathway to a more sustainable learning environment, this can be developed, especially if
WSUP sees these alongside the evolution of the Masters modules.
Masters modules – University accredited Masters modules are an ambitious initiative that are
consistent with WSUP leadership and demonstrating ways of working. These are really about
mid- to long-term learning and so require considerable money and time investment. They may
lose out on opportunities for more immediate impact, and they will not be very effective unless
used by the intended audience, (sector professionals) which was often not the case.

3.4.4.1 Relevance
Activity 4.1 – Research
In broad terms, WSUP considers high quality research as key to bringing an evidence base to
underpin their own work and in turn influence others. DFID has for some years placed the
highest premium on research work to provide a robust evidence base and this is in line with
global trends with ever increasing emphasis placed on strong evidence. The investments in
research are clearly highly relevant in general terms, but specific terms it needs to be considered
how relevant the chosen topics were. WSUP set up a consultative process76, with inputs sought
at a research planning workshop77, and the final decisions taken internally by WSUP based upon
their internal analysis. Then a newly formed Research Steering Group confirmed the overall
relevance of these, based upon their knowledge. No literature review was carried out. The
process of narrowing the research choice down needed to be quite rapid because of the lead
time to undertake research.
As noted by Guy Norman78, WSUP’s research had approached some very specific questions, in
order to generate clear findings. Asking questions like “what are the most cost-effective urban
sanitation investments?” may have programmatic breadth and seem more relevant, but trying to
answer very big questions risks leading to diluted research efforts of little strong value.
In considering the relevance of the areas of research undertaken there are a few key
observations:
76 Interview Guy Norman.

77 Urban WASH Research: Building the Evidence Base, 21 January 2013.
78 Email Guy Norman, 28th January 2016.
62
a) WSUP balanced its own practitioner perspective with that of knowledgeable academic
bodies in making a choice of topics that clearly have a very direct link with WSUP’s core
work.
b) Research projects chosen related to the following WASH sub sectors; 1- hygiene/
behaviour change (BC), 2- sanitation, and 3- water. Given the greater complexities
associated with workable urban sanitation and BC approaches, it is considered very
relevant to put such emphasis on sanitation and BC.
c) A global research agenda may not meet the specific needs of national service providers,
as noted by Maputo water utility79.
d) In the field of Urban WASH for LICs there is a very long research ‘shopping list’ to
undertake. In future, there could be value in undertaking a more comprehensive review,
perhaps with WB/WSP, rather than just with research institutions.
Activity 4.2 – Dissemination

Publications
WSUP has brought together an impressive and significant body of new publications and case
studies. These were produced in a range of different publication types (short and long formats)
with different purposes in mind. These added considerably to the body of knowledge for urban
WASH for LICs. WSUP did not set out to establish what critical ‘missing’ sector knowledge
needed to be documented and then finding out if their experience fitted this, rather they
documented what they had learnt, which meant the process was more organic than planned80.
None the less a stock-take of the publications (see Annex Ten) reveals the high number of
publications that focus on finances, as well as a good number on sanitation, so WSUP has
added real depth in these areas. These are clearly both highly relevant given the challenge of
sanitation and to demonstrate WSUP’s key belief that the poor are potentially viable consumers.
Only WSP Global experience documented81 has comparable detailed knowledge across the
board.
The WSUP publications survey82 quantifies organisation/users who downloaded docs. The urban
guides were the most popular amongst downloaders, (perhaps because many in this group
appeared, because of their organisation type, too often to be making general research enquires).
However, the publications survey significantly underestimates the use of publications as it does
not capture soft copy distribution amongst organisations after initial downloads and hard copy
users (it is not possible to easily quantify this). WSUP Advisory, as a user of WSUP programme
knowledge products felt that the publications of greatest potential value were papers related to
financial aspects, as these in particular are shared with and cited by other organisations83. What
would have been useful to produce in addition were internal ‘how to’ manuals84 which would
have been beneficial for WSUP project management as well as sector wide influence.
79 Interview with AdeM in Maputo.

80 Interview Guy Norman 16th December 2015.
81 WSP Global experience on expanding services for the urban poor.
82 Publications survey full, January 2016.
83 Interview Tim Hayward.
84 Antonio Junior - WSUP water programme coordinator.
63
Conferences
Global conferences, a key accompaniment to WSUP’s publication and research work, are clearly
a way for WSUP to lever further pro-poor change, particularly at the analytical and to some
extent policy level. At the start of the project period WSUP sought to attend and present at
international conferences where suitable opportunities presented themselves. During the early
period major conferences, for example Water, Engineering and Development Centre (WEDC),
and some less significant conferences were attended with WSUP both presenting and chairing
sessions. Once WSUP’s knowledge momentum was built up, it was possible to attend more of
the strategically significant events and regional sanitation conferences, as well as events
focussed on key topics such as FSM. National conferences also play an important role, as noted
in the Zambia report85.
During these conferences it was notable that no less than 17 conference outputs were on
sanitation, with one on water, five on WASH generally, three gender related, and one on
behaviour change86. It is reassuring to see so much attention given to sanitation, though there is
much more to be said on what is different about urban behaviour change. A closer review of
conference outputs (other than the gender presentations given at SUWASA, MHM, Kings WASH
week conferences) suggests that there could have been more opportunities to mainstream
gender within some other presentations, e.g. by describing the different roles and experiences of
girls and women, alongside boys and men.
Activity – 4.3 learning

Masterclasses
The overall value of peer learning and exchange visits is consistently identified in WSUP reports
as key to support learning, expose participants to other ways of operating and encourage
adoption of new approaches. Two regional Masterclasses were run in Africa in 2013 and 2015,
both of which were very well attended with a very good selection of quite senior WASH sector
representatives. The Masterclasses drilled down into a series of critical topics and really sought
to unpack these and provided useful documentation to support learning87. These added real
depth in key topic areas such as LIC – pro-poor units as highlighted by Tim Hayward88. The
identification of Masterclasses and the choice of topics within these were clearly highly relevant
to the needs of the participants and undoubtedly show real added value.
Masters module
WSUP’s analysis was that there is a real lack of expertise and understanding about how to
undertake WASH provision for LICs in urban areas so there was a real need to build professional
and sector capacity over the medium and long term. This was a point strongly emphasised in
Mozambique by WSUP’s course delivery partner university there89. At the time of the
development of these materials, no other taught material for urban WASH for LICs as a complete
module existed. The development of the Masters module is entirely consistent with WSUP taking
leadership and bringing innovation to develop demonstration models, in this case for learning,
and the initiative potentially added significant value by filling an obvious gap in structured
85 WSUP Zambia APR 2014.

86 See Annex Ten for listing of all conferences attended.
87 WSUP-Masterclass-2013-fuller-report-GN-Dec2013, and a series of documents from the 2015 Masterclass.
88 Key informant interview Tim Hayward, WSUP Advisory.
89 Key informant interview Nelson Matsinhe - Universidade Eduardo Mondlane, Maputo.
64
learning. It also had the potential added benefit of bringing together all WSUP’s knowledge
across the spectrum of its work, placing it into a coherent whole, in a way which the urban
programming guide really only started to do. The modality of delivery was envisaged to be
development of an accredited module, which would then be run through partner university
establishments in all of WSUP’s country programmes. This would bring the topic to the heart of
developing countries, which is considered very appropriate.
Activity 4.1- research
In considering whether the research projects were effective, this evaluation looked initially to
WSUP’s own appraisal of the value of this work90 which considered the likely value-of-findings to
be excellent for three research projects, good for one and adequate for one. A review of the
outputs shows that broadly speaking three of these (behaviour change, scaling up sanitation,
environment) have produced a thoroughly researched evidence base which provides detailed
and convincing knowledge in these areas.
The Bangladesh BC project is a valuable enquiry that examines whether key messaging can
effect hygiene behaviour. It also demonstrates the very well established principle that facilities
also need to be in place (the so called enabling environment) to support messaging. This is
useful to ‘prove’, even though it might be considered self-apparent to those with sector familiarity.
However, though the research methodology takes account of gender it is surprising that the
results of the randomised controlled trial (RCT) are gender neutral and do not distinguish
between the experience of women/girls and men/boys. The Leeds environment study more
precisely quantifies the impact of providing a higher level of water service to LICs on city wide
water resources, and demonstrates that the overall resource impact is not significant. It appears
to have effectively achieved what it set out to do and might well prove to be a very convincing
piece of research. The scale study Triggering Increased City-Level Public Finance for Pro-Poor
Sanitation Improvements conducted by the Urban institute seems particularly interesting as it
really drills down to unpick how change in sanitation investment might occur. In this sense it
particularly useful as its application goes beyond the specifics of the locations (Ghana/Kenya/
Mozambique) it was conducted in, and offers an analysis of more generic patterns, something
that could be built upon for a context analysis tool (see section 4). The Cranfield study was very
focussed seeking to unlock a particular problem and assist the Ghana Clean Team in its work.
Though not a big investment, it appears rather inconclusive, though it still needs to be finalised.
The initiative was particularly complex and faced a number of difficulties,91 including import of
equipment to Ghana, and these complications have really impeded the effectiveness of this
particular piece of work. As noted elsewhere in the report, the FSM management is a particularly
tricky piece of the puzzle that is very important to ‘crack’. However, a research modality may not
be the most effective way to undertake technology development. This is a point made to DFID in
the research report92; “This research also highlights the daunting logistical challenges of
technology testing in distant locations, and WSUP will not get lightly involved in such testing in
future”.
90 Research-for-DFID-AnnualReportJan16-GN-26Jan2016.docx
91 As explained by Richard Franceys.
65

Publications
WSUP has chosen a variety of publication formats, with two originally envisaged and now
expanded into five format types. These short form (Topic Briefs, Practice Notes) technical WASH
briefs are widely used by WHO, WEDC, and Oxfam, so are considered to be a useful way to
provide this sort of very practical information. The publication survey93 showed that the longer
form urban programmes guide was the most downloaded publications, notably the Practice Note
on faecal sludge management, a known gap area was the most popular Practice Note. There
was also a lot of interest in the Ghana Clean Team.
WSUP has produced a significant number of publications that include a lot of data and analysis
of financial aspects of WASH for LICs, which is much needed and very powerful. Some earlier
APRs94 raise the issue of needing convincing data to show utilities that LICs can pay in
Bangladesh. The need for better understanding about life cycle costs was highlighted for
Mozambique. This shows that the 14 publications which focussed on financial aspects95 are very
much needed, though no doubt there is more to be done to satisfy the needs of a variety of
stakeholders. While there are a few publications that bring out gender or focus on girl’s/women’s
issues, gender as a cross cutting theme is not so evident throughout other publications. It is
recognised that some are too short and by their nature gender is not relevant, but others are
rather quiet on gender when more could be said. Overall, as noted by WSP/WB, these
publications definitely help put WSUP on the map and have influence96.
Using downloads as proxy for effectiveness we can see that bigger publications are most
downloaded, but as noted above, this does not capture hard and soft copy distribution. The use
of the shorter formats does allow a live, dynamic, process of contribution, encouraging individual
country ownership, allowing these countries to circulate within their constituents. It is evident that
the teams in Mozambique, Kenya and Madagascar have been much more prominent in getting
their experience set down97, something that is not unnoticed by WSUP in its annual reviews.
However, all countries have contributed, so it has been a highly effective way of drawing out
WSUP’s experience across the board.
French and to a lesser extent Portuguese translations are limited. An overwhelming amount of
publication downloads were in English: 4,791, with 153 French and 50 Portuguese downloads.
This may not tell us much about the effectiveness of language translations, as it seems likely that
hard copy language translations of WSUP experience (not included in the survey) are the format
that would be preferred to most national mid- and senior level sector staff. The WSUP Annual
Progress Review Mozambique 2014 raised the question of the need for further translations.
Conferences
The review of conference material/outputs98 shows that WSUP has established a very effective
presence at these events. Overall WSUP have taken on a strong leadership role in the
conference circuit, playing facilitation roles, as well as presenting/co-presenting, sometimes with
93 WSUP Publications Survey May 2014 – October 2015.

94 WSUP Bangladesh APR 2013, Mozambique.
95 See Annex Ten for breakdown in topics.
96 Key informant interview Peter Hawkins WB Mozambique.
97 See analysis in Annex Ten.
98 Files on conferences attended showing presentations and associated papers.
66
the production of associated papers. Leadership of presentations has been by WSUP’s ERL
team, meaning they have become the global knowledge holders/owners. This is considered as
highly effective as it ties in well with the overall learning and research agenda led by this team. It
also enables an overview, drawing on all country experience to be weaved together. In
conclusion WSUP have occupied a high profile learning space at global WASH conferences and
this has contributed to their international profile and brand.

Masterclasses
Masterclasses and learning exchange visits are considered to be highly effective as these are
very good means by which peer learning and exchange occurs. As noted by WSUP’s African
Cities for the Future report99 peer learning is a very powerful way of spreading learning and
successes, as well as helping develop pro-poor champions. Masterclasses and Bilateral
Learning exchange visits, were described by Tim Hayward of WSUP Advisory as “brilliant”100.
AdeM (Maputo city network water supplier) staff cited the visit to Manila as being very influential,
as did Ghana staff101. Kenya and Bangladesh staff requested further exchange programmes102.
Much of the driver behind the regional sanitation conferences (SACoSan, AfricaSan, etc.) is
bringing South to South actors together to learn and to some extent compete to be better. The
Masterclasses very much embrace this spirit so are considered highly effective.
It is suggested that Masterclasses are particularly useful for mid-level sector professionals who
attended these events to effect short and medium term changes. However, it needs to be
appreciated that part of the success of these is that WSUP also has long term relationships with
key utility staff, so the ability to ‘go deep’ is much enhanced by the trust and relationships built
up. WSUP plays more than a convening role, it is a trusted partner that is allowed to set the tone
and determine content. Thus Masterclass preparation has been led by WSUP in-house
experience, and proved to be a wonderful pro-poor learning platform with a captive audience and
has allowed for cross fertilisation of ideas. For example, one theme discussed was organisational
structures within utilities for extending services to low-income communities.
Masterclasses were not evaluated, though feedback from the Zambia Masterclass was reported
as overwhelmingly positive. The 2015 Zambia class listed key commitments and learning, which
is considered as good proxy for effectiveness. Participants often cited that “opportunities to learn
from the experiences of other institutions in the sector are extremely rare”, such opportunities
provide welcome inspiration and a chance to reflect on what might be possible in their own city.
As noted in their 2015 report; “Following on from the success of the first Masterclass in Kampala
in December 2013, WSUP believes the event provides a valuable forum for knowledge exchange
among WSUP partner institutions”103.
99 African Cities for the Future Water & Sanitation for the Urban Poor October 2014 Final Report | October 2009
to September 2014.
100 Key informant interview Tim Hayward, WSUP Advisory.
102 WSUP Kenya APR 2013, WSUP Bangladesh APR 2013.
103 WSUP six monthly report, July 2015.
67
Masters module
The key driver for the Masters module was to build sector capacity for work on WASH for LICs in
urban areas, which is very much a medium- to long-term goal if targeted at recent graduates.
Brian Reed (WEDC) commented that WSUP also took on an employment centred approach to
the course, in as much as they wanted more knowledgeable WASH LIC personnel in the
employment pool.
WSUP planned to have material developed by a global expert establishment, with delivery by
national university partners and accreditation for the modules, a worthy ambition in order to bring
added, lasting value and credibility. Thus the development of the module was contracted out to
WEDC and the brief included a Mozambique country scoping visit. WEDC recognised and
highlighted to WSUP some of the complexities of embarking upon development of Masters
modules to WSUP104. It highlighted the challenge of ’docking‘ with in-country Masters courses,
for example that this would not easily fit pre-existing courses, potentially limited capacity of
university staff to deliver and weak student ability/grounding in basic principles from their first
degree, much of which proved to hold true. Though WEDC highlighted this in general terms,
there is no evidence that they conducted a clear feasibility assessment, so some of these
problems were not that well stated and had to be addressed subsequently.
As a consequence of the national university limitations, the original materials developed by
WEDC for Mozambique were very generic in nature, and pitched at a lower level to meet the
capability of partner university lecturers and students. However, it is also noted that WEDCs
does not have any real urban LIC WASH experience105, so was not in a position to produce a
higher-level module that would have added value. Even more disappointingly WEDC failed to
include much of WSUP’s very good experience in the first draft of material. WSUP addressed
this by bringing in Richard Franceys who presented his own materials in order to run the
prototype in Mozambique, which enabled WEDC to take these and finalise a much improved
package. While final outputs were of good quality, this point is highlighted as it illustrates the
extent to which urban LIC knowledge still remains a niche that most organisations are unfamiliar
with, so effectiveness is dependent upon real expertise.
There was some variation in the length of the module (from between 5-8 full time equivalent
days) though delivery for all courses included core materials and one field visit. The only delivery
which has been formally accredited within an existing Masters programme is the Mozambique
delivery. However, as noted by Guy Norman “working with the slow moving bureaucracies of
universities precluded the ambition of accreditation elsewhere if roll out targets were to be met,
so this is something that will have to be left for future programmes”106. Partnering with southern
universities is complex as WSUP want to allow flexibility and encourage local ownership, but this
will not contribute to WSUP’s wider goals if local lecturers deviate significantly from the core
materials.
104 Key informant interview with Brian Reed who undertook the visit.
105Views of both Richard Franceys when he re wrote materials for Mozambique delivery and Richard Luff when
looking at the suitability for adaption for another purpose.
106 Guy Norman.
68
In terms of course effectiveness the course evaluations107 and interviews with Reed and
Matsinhe108 highlighted that the course was indeed considered effective. However, the biggest
factor that compromised the overall effectiveness was the fact that participants were often not, in
fact, recent graduates, for which a taught module is considered more appropriate. Instead many
participants appeared to be mid-level sector personnel, as confirmed by Mozambique University
partner Nelson Matsinhe. This also appeared to be the case for participants in Bangladesh,
Kenya and Madagascar. WSUP reported that the approach was modified in light of this
experience to move towards a non-accredited roll-out of the courses targeting sector
professionals109. Another significant limitation in the Masters module was that there were not
enough country specific materials, delivery in Kenya being the case cited by Bill Peacock110.
Finally, the course itself would benefit from sitting within a framework that also considers in
greater depth context factors such as planning and urban social mobilisation, a point highlighted
by Nelson Matsinhe.
While the medium and long term vision of WSUP for courses embedded in Southern universities
is laudable, it is considered too ambitious. Feedback from WEDC, from the partner in
Mozambique and the fact that the course was cancelled in Zambia, demonstrate the complexities
of trying to situate this learning material within national university systems. It should still be noted
that there is definitely appetite in some countries to retain the idea of the university run and
accredited Masters module. For example Chittagong University would like to run the module as
part of a Masters course111 and the WSUP CPM Carla Costa was extremely positive about the
initiative.
In conclusion, it would have been more effective for WSUP to have aimed for short/medium term
sector enhancement by running stand-alone short courses which would have been more
adaptive to participants learning needs, included more WSUP and national sector experience
and allowed more exchange. This was very much the approach recommended by Nelson
Matsinhe. WSUP could then sought to have these subsequently situated within university
courses at a later date, if opportunities arose.
3.4.4.3 Efficiency
Broadly speaking, WSUP thinks research generally offers value for money. It has provisional
plans112 to have a £3 million research portfolio over four years, so approximately a five-fold
increase in expenditure. The DFID conducted WSUP Annual Review 2014 concluded that overall
the “research programme continues to deliver good value for money”.
In considering efficiency, it is important to reflect upon what sort of influencing WSUP seeks to
achieve: is it at the global and/or national level, to influence on issues that WSUP identifies, or
what urban policy makers and service providers want. High level, methodological robust, in depth
research with randomised controlled trails (RCTs) is costly. However, in order to have global
107 Master modules course evaluations from Mozambique, Kenya, Madagascar and Ghana.
108 Nelson Matsinhe – Universidade Eduardo Mondlane, Maputo.
109 Regine Skarubowiz.
110 Interview Bill Peacock 16th December 2016.
111 Key informant interview Abdus Shaheen, WSUP CPM Bangladesh.
112 Minutes of WSUP Research Strategy Meeting, 19th August 2015.
69
recognition and fit a global influencing agenda, it needs this level of investment. However, if a
national level influencing agenda is pursued, the cost would be less and would likely resonate
more with national organisations. It would also provide a sector rallying point for a variety of
sector actors and give rise to more national ownership. A national research body partnering with
globally recognised institutions could be more efficient way of delivering robust, credible
evidence, whilst better meeting national needs and ownership, something which WSUP is
considering going forward.
At the service provider level, a more action orientated research approach is likely to meet the
direct needs of such stakeholders. Some of the research topics could be pursued using in depth
survey techniques and mobile phone surveys and might offer viable alternatives that would
provide enough data to be influential. At the WSUP research event on 4 February 2016, it was
observed that maintaining the gold standard of RCT may need to yield in the face of alternatives,
for example surveys using mobile phone apps.

Publications
The budget was around £60,000, which enabled publication of 20 short form publications and
eight long form publications. An indicative cost per publication type was as follows.
Table 9 – Indicative costs per publication type
Short form Long Form Programme
Type of publication
(PN, PP) (TB, DP) Guide
Indicative average cost per publication
£52 £2,867 £9,750
(not including in-house labour costs)
Conferences
These are considered a key accompanying activity to the research agenda and publications.
Costs for conferences were not available. Outputs for conferences included a range of posters,
PowerPoints, papers or sometimes chairing sessions.
Activity – 4.3 Learning

Masterclasses
WSUP estimates that the total cost was in the order of £60,000113 for the 2015 Masterclass (no
costs were provided for 2013 though these are likely to be similar). Costs arise from paying for
flights and hotels of the participants (these are regional events) and the venue hire. Expenditure
amounts to a cost of about £280/per day, which is roughly speaking equivalent to costs for an
advanced training course, and is considered reasonable value for money. As noted above these
classes are hugely effective in invoking a change in attitude and open eyes to new ways of
running utilities to be more pro-poor, so are – in terms of exposure and learning – considered to
be excellent value for money.
Masters module
A £50,000 contract was agreed with WEDC to develop the Masters training materials (which was
paid for by both the Australian Department of Foreign Affairs and Trade (DFAT, formerly AusAid)
and DFID). Figures provided by WSUP indicate that the running cost for five courses has been
113 Email Sam Drabble 5th January 2016.
70
£38,500 to deliver 39 full time equivalent days (some days were part time) of training in five
countries. This means the cost (excluding WEDC costs) per person/day varies between £14 and
£90, depending upon which country the course was delivered in114. The figures for Mozambique
in particular were very high, as additional contracts were made to bring in additional external
specialist lecturing support, which was considered necessary in light of the weaknesses of the
original materials prepared by WEDC. In total close to £90,000 was spent to deliver material for
87 students, about £1,000 each, or about £180/person/day.
As noted above the absence of pre-existing taught materials meant there was no real choice
other than to commission their development. However, WEDC did not have the experience to
deliver materials of the right quality. Given the very limited pool of expertise, an in-house contract
would have delivered much better value for money as it could have directly drawn in WSUP’s
own experience. Contracting Southern universities to deliver was potentially cost effective and
helped with ownership and sustainability. However, in reality all deliveries to date, except
Bangladesh have involved a WEDC lecturer in module planning and actual delivery (which was
not possible in Bangladesh because of a security threat at that time)115. It shows that
assumptions about national capacity to deliver being adequate did not hold (as highlighted by
WEDC), meaning that additional costs were incurred to run the modules.
Alternatives to delivery of university accredited short modules could have been considered. A
short course format, unaccredited and run on an ad hoc basis, would certainly have been easier
and more cost effective to run in the short/medium term. This could have been conceived of as a
demonstration model that others could later have built upon, much like WSUP’s work under
Outcome 1. However, ad hoc training initiatives offered by a series of organisations over many
years, will prove to be very expensive and not at all sustainable. The case of National Water and
Sewerage Corporation (NWSC) in Uganda, which invested heavily in 90s and 2000s and at great
expense, was cited as an example of the limitations of a piece meal approach116.
A distance learning approach might have provided an interesting alternative. WEDC estimates
that distance learning costs three times as much to develop compared to a conventional face to
face training, but this of course neglects to account for the savings on accommodation,
subsistence and flight/travel costs. The above figures seem to suggest distance learning could
break even after running for three years.
3.4.4.4 Impact
Five of the research reports have just been finalised. WSUP have outlined a dissemination plan
that is contained in its research dissemination timetable. It is intended that results will be
published in peer-reviewed journals117. As noted, WSUP is believed to be seen as impartial and
independent, so undoubtedly independent rigorous research will reinforce this perception of
legitimacy and in so doing WSUP’s impact. However, it is clearly too early to predict the impact.
Member of the panel at the WSUP research event on 4 February gave a strong endorsement of
the research areas and the research findings. There also was, though, acknowledgment that it
114 Details in Annex Ten.

115 Key informant interview Guy Norman.
116 Key informant interview Brian Reed.
117 Email from Guy Norman/Sam Drabble 15th January 2016.
71
was not really known how policy makers and service providers are influenced at the national
level. As these research projects sought to address issues identified by WSUP and undertaken
by global research bodies, it is considered they will not have immediate impact at the national
level or on the national policy environment. Therefore ‘global’ research should be considered to
be a long term game changer, rather than delivering more immediate change. The value and
impact of such research projects is likely to be more significant for the likes of DFID, the WB, etc.
However, there is no clear evidence one way or another to say what impact these particular
projects will have at the global level, so it would be a useful for WSUP to hear from DFID in order
to understand their views and how they will use this research work to shape their own policy.
Some further evidence on perceived potential impact can be gleaned from sources that point to
next steps for any future WSUP research programme as these show where WSUP feels changes
are necessary. WSUP organised an event with the Bill and Melinda Gates Foundation at the
University of North Caroline (UNC) 2015 about urban WASH research going forward118 (this
document was not available for review). WSUP’s research steering group119 recommended that
“WSUP needs to carefully analyse from the outset how to leverage WSUP’s position in the sector
to engage relevant practitioners and policymakers with the research”. The steering group further
noted some over-arching questions of broad interest to the sector should be identified and any
future research programme should retain a focus on creating new knowledge and new tools, and
utilising these tools in the field to make full use of WSUP’s position ‘on the cutting edge’. Guy
Norman120 suggested that, going forward, research should be nationally led and focussing on
national issues. What emerges is that in future WSUP wishes to work in way that has more
impact at the national level, is more directly relevant to national policy makers and service
providers and is more closely aligned with WSUP’s own work.

Publications
These are considered to be high impact from a visibility and branding perspective as they are an
easy way to promote WSUP’s knowledge and brand. It is considered that the publications, a very
identifiable WSUP knowledge product, are a critical part of a package that contributes to shifting
service provision to be more pro-poor. The variety and nature of formats, which often document
cutting edge knowledge, are considered to likely be influential in a market with relatively little
Urban WASH LIC documentation. However, in interviews and email exchange with WSP/WB,
UNICEF and WaterAid, no one was able to comment in a very definitive or insightful way about
the impact of such publications.
It is considered, none the less, that the organisations that access publications will likely be where
WSUP will have impact. The WSUP publication survey (which as noted above only captures
some of the users) shows organisations with the highest number of individual downloaders
(excluding WSUP London and its country offices) were WaterAid, UNICEF (mostly UNICEF UK),
and World Vision. Other notable users were UN Habitat (5 publications), WSP (8), WB (14). 18%
of downloads were in the UK and 26% from Africa, and there was a high number of downloads in
India 8%, perhaps because WSUP Advisory is undertaking work there and WSUP programme
118 http://www.wsup.com/2015/10/28/research-for-urban-sanitation-where-next/
119 WSUP-Research Steering Group – Meeting 4-23 January 2015 minutes.
120 Interview Guy Norman 16th December 2015.
72
used to work in Bangalore. Unsurprisingly usage and potential impact is likely to be strongest
where WSUP has a presence.
Conferences
The question of whether conferences, specifically the regional sanitation conferences, are
effective and what sort of impact these might have has been considered by a Water Supply and
Sanitation Collaborative Council (WSSCC) review of the regional sanitation conferences, led by
Piers Cross (this does not appear to be available for public review, but this might be useful to
obtain). However, there is no doubt that at the international level WSUP is making a significant
contribution to sector knowledge on urban WASH solutions at sector conferences, “using events
such as Africa Water Week, the African Water Association Conference and Stockholm Water
Week as a means of dissemination of their research and their learnings from their
programmes”121.. DFID reinforced this point in an interview that WSUP has a strong presence
and influence in conferences122. Going forward, now that WSUP is established at the global
conference level and has a good body of documentation, research and evidence behind it,
WSUP could probably be more selective in which conferences they attend to maximise their
potential impact.
Activity – 4.3 Learning

Masterclasses
Of all the knowledge products that were mentioned verbally, the Masterclasses, internal learning
forums and exchange visits were most often cited as being of greatest value. For example,
during the Mozambique visit, staff of the water utility AdeM cited the visit to Manila water in the
Philippines as being very powerful, as it allowed them to see what could be achieved and to bring
practice back to Mozambique and adapt it to the country context. It is considered that these
seem likely to deliver most immediate impact, specifically within those institutions that sent
participants to attend. The 2015 Masterclass lists commitments that participants would take back
into their workplace. The extent they can do so depends upon a number of factors, but the sorts
of commitments made might suggest quite an attitude and culture shift for some participants.
However, there is no follow up survey to verify the potential impact of this and there is no direct
way to measure impact.
Masters module
The investment in development and running of the Masters module courses is really pitched at
recent graduates, so its impact is likely to be mid- to long-term, i.e. 10-20 years. Brian Reed
(WEDC) highlighted that this is a big job, beyond the remit of any one organisation and requires
a more collaborative approach. As noted, above these will not deliver real impact if they are
attended by mid-level professionals who actually need a more tailored experience exchange.
Thus, given the materials are designed for an audience that typically did, in fact, not attend,
these will, to some extent have missed the target. In summary, it is suggested that developing
and running a Masters module is a worthy but rather protracted route to achieve impact. A stand-
alone course would have short and/or medium term impact, as well as potential for longer term
impact too, if available to be run as part of a university course at a later date.
121 DFID annual review for WSUP 2014.

122 Key stakeholder interview Jane Crowder.
73
WSUP appears to be moving towards a national level applied research agenda. The prospects
for national ownership and contribution would appear to be much better. Though the ability for
Southern university partners to fund comprehensive research appears very limited, but local
partnerships and less rigorous and expensive research may hold the key to a more sustainable
research agenda being developed.

Publication
The bulk of the work with associated costs has been undertaken, and a significant range of
knowledge products now exist. While updating and new publications will be required, the existing
publications are available and can be easily adapted and used in other formats in future, without
too much additional investment.
Conferences
Grant support is currently required to allow WSUP to attend further conferences. However, it may
be that now WSUP is established on the conference agenda, it can be more selective and
strategic about which conferences it attends.

Masterclasses
While there is a significant price tag associated with these events, these will continue to need
high/full subsidy until such time when utilities and policy makers have access to budgets to
attend these events and/or the pro-poor business is of a scale to make it justifiable. If WSUP can
devote further attention to the Masterclasses as a very important learning/influencing opportunity,
then new opportunities to build on the concept could be developed. An example might be to
move to a more conference-like format, with WSUP holding limited bursaries, with the majority of
participants expected to be self-funding.
Masters module
WSUP have embarked on a learning initiative, initially choosing to develop and run a module of
taught material that could sit within a Masters course. The engagement with universities is very
much a medium and long term approach, and has an eye on sustainability in the long term. The
business model for running courses did not appear to be well developed, a point made by the
Mozambique counterpart123, who felt that significant subsidies will be required. Upfront costs for
initial material development have been undertaken, though the materials will need further work
in-country to tailor and then there are additional costs for facilities and venue hire and payment
for specialist lecturers’ time. Therefore this initiative is better placed to deliver sustainable
learning programmes but is contingent upon sufficient inputs being made available, which
requires a very long term approach.
123 Nelson Matsinhe - Universidade Eduardo Mondlane, Maputo.
74
4. Overall Assessment and Conclusions

4.1 Relevance
Overall our conclusion is that the WSUP Programme and approach is highly relevant to enabling
poor people living in urban areas to be able to access clean water and appropriate sanitation
through the appropriate national entities. The four Outcome areas work together in a synergistic
way. Specifically we would highlight the approach of working with utilities and poor communities
to develop approaches to water and sanitation provision that engage the two groups in a shared
search for appropriate solutions. This work is then supported by a mix of research, training and
capacity building that supports the development of capacity as well as sharing of ideas and
knowledge. The financing work contributes to and is supported by work on the other Outcomes.
4.1.1 WSUP’s Unique Selling Point (USP)

Carrying out an analysis of WSUP reports and documentation, and undertaking interviews with
key stakeholders, along with the two country visits, provided an opportunity to really get ‘inside’
WSUP and hear about the combination of key characteristics that add up to what we perceive is
a winning combination of WSUP’s Unique Selling Point (USP). In the 2009 evaluation by Alison
Barrett124, she considers whether WSUP is unique and concludes this is not material125.
However, we suggest that understanding the combination of characteristics, whether unique or
not, is critical to understand how WSUP can do what it does. In so doing it helps understand
what WSUP must continue to keep doing, and to what extent other organisations can – or cannot
– replicate and take WSUP’s work to greater scale. For example, WSUP Advisory was
developed in part to provide greater leverage and replication. Though of course they do not
combine all the characteristics of the WSUP programme, the question arises to what extent they
can lever change with just some of these characteristics to hand.
Senior staff
with sector
expertise
Appetite to
take action Tenacity
Independent
Projects get a and impartial
place at the table
WSUP’s
USP
Financial
Deep craft i.e feasibility of
understanding poor to pay
Long term
accom-
paniment Building
Catalyst and professional
innovator awareness
Figure 4 – WSUP’s key characteristics
124
Review of the DFID-funded Water and Sanitation for the Urban Poor WSUP. Final Report. December 2009.
Report Prepared by: Alison Barrett for The TI-UP Resource Centre.
125
Alison Barrett. 2009. Review of the DFID-funded Water and Sanitation for the Urban Poor WSUP Final
Report.
75
The diagram depicts ten key characteristics we see working together to bring about the change
that WSUP has been so successful at. As WSUP fully understands, it is the combination of
these, undertaken evidently in different ways, primarily under Outcomes 1, 2 and 4, that leads to
changes under Outcome 3. These then feedback either directly into WSUP work under its
different Outcome areas, or spiral out to new areas and locations through the work of other
agencies and organisations.
One aspect that should be highlighted is that WSUP is not a rights-based organisation, in the
way in which many INGOs work as ‘Rights-based organisations’. WSUP does not put its main
focus on lobbying and advocacy. Instead, it works to support service providers to fulfil their
critical pro-poor function in as viable a manner as possible. In this respect WSUP does not
confront ‘duty bearers’ with their obligations, rather it works with them through demonstration,
capacity building and long-term support. We observed this in the context of WSUP necessarily
working with ‘fragile’ utilities to enable a far more conducive way of working and mutual respect.
None the less, advocacy on rights and holding service providers to account are important tools
which WSUP should take account of. We suggest WSUP should do this more consciously, but
not by doing this directly, but rather working indirectly with others who do, thereby keeping this
work at some distance and so as not to undermine trust and working relationships.
WSUP have developed a new Theory of Change for the next planning period going forward. The
diagram of key characteristics is really another lens through which to examine how WSUP is able
to achieve its wider influence. In this respect it would be valuable to test these ideas to
understand to what extent they are substantially right. The Ghana programme prioritises
influencing over and above demonstration in a way other country programmes do not
consider126. The work on the sanitation tariff in Mozambique127 is another prime example of good
influence (though the tariff is yet to be realised). Given these have been cited as particularly
strong examples, it would be useful to examine these cases in greater detail. There is a wider
question to consider by asking how WSUP’s work is influencing DFID and others. Finally, while
we believe WSUP has become a market leader, combining characteristics in a powerful and
perhaps unique way, it might be considered to what extent others are catching up, or indeed
falling away, and if this this matters.
In conclusion WSUP’s Unique Selling Point (USP) can best be described as follows:
WSUPS’s USP is to enable low-income country stakeholders128to engage with
and deliver services to the poor and fulfil their (not always initially understood) pro-
poor mandate
4.2 Effectiveness
4.2.1 Imbalances/system bias

WSUP works in an environment with a number of systemic biases, as depicted in the diagram
below. The system default is towards networked systems, poor people who have some money to
pay, water and men. It is evident that WSUP is able to work on the areas of lesser priority: this is
considered very positive and must be maintained. In broad terms, WSUP manages well to give
126 WSUP Ghana APR 2014 and email clarification Sam Drabble 19/02/2016.
127 WSUP Mozambique APR 2014.
128 This wording has been used rather than utilities to include other key actors such as municipalities.
76
particular attention to women and to sanitation129 and balances networked/non-networked

systems – it should continue to do so going forward130.
In order to lever bigger system change and confront under-investment in sanitation as a whole,
WSUP needs to build some key strategic relationships with key stakeholders such as the WB,
UNICEF131 and WaterAid in order to lever more support for the sanitation agenda.
Less
priority Greater priority
On site non
networked systems Networked systems
People with little
ability to pay People with some ability
to pay
Sanitation
Water
Women
Men
Figure 5 – System bias (imbalances)
4.2.2 Addressing inclusion effectively

The WSUP programme works hard to address issues of access and inclusion for people with
disabilities. This is clear from the guidance in the Urban Programming Guide that responded to
challenges in approaches in the field. Given WSUP’s clear focus on the importance of enabling
access for people with disabilities as well as other people with increased needs (such as the
elderly or frail) it is important for WSUP to provide a consistent and clear approach to inclusive
access. That enables WSUP programme staff and communities to demand and develop inclusive
approaches that benefit those with special access needs whilst not compromising the legitimate
needs of other users. While recognising the value of Guidance documents, such as the Urban
Programming Guide, there is the inevitable challenge of whether people will always consult and
refer to such documents (especially when they are large documents/guides). There is also the
inevitable challenge of such guidance getting out of date. In order for WSUP to maintain and
build on its position of working at the ‘cutting edge’, we would want to propose that it should look
to developing a system of key personnel across the organisation who can act as the ‘go to’
people for advice and support on each of the main concept areas of WSUP’s work.
4.2.3 Strategic learning

Learning/knowledge management is a key part of how WSUP can lever change, through its
credibility as an expert, knowing how things really work, being able to innovate and catalyse
129During interview with Jane Crowder, DFID, it was stated that sanitation is the DFID priority going forward to
support the urban agenda.
130During interview with Jane Crowder, DFID, it was stated that sanitation is the DFID priority going forward to
support the urban agenda.
131 Interview Mark Henderson, former UNICEF Mozambique WASH chief. WSUP Bangladesh APR 2014.
77
change, etc. Given this is so essential, we believe, while we recognise that WSUP undertakes
strategic learning at all levels, WSUP should be much more strategic in its learning. Learning
could be supported with better planning, resources and with the addition of some complementary
new initiatives.
 There are examples of learning between countries, e.g. the Mozambique Community-led
Total Sanitation programme coming from Madagascar132. However, this could be
enhanced, e.g. by information sharing between countries, which sets experiences down
side by side to invite comparison, for example through regular learning webinars.
 We recognise and applaud the work that WSUP has done to develop guidance notes and
publications, particularly the development of the Urban Programming Guide and the
Urban Water Supply Guide. However, there is still an absence of sufficient guidelines and
‘how to’ manuals/guidelines in country offices. This is a point that Manuel Alvarinho of the
regulator CRA in Mozambique made133. “We emphasise that there is still a need for
simple manuals, simple guidelines, to allow us to select the most appropriate models for
a given town or city district”. WSUP is a small organisation with some of its knowledge
disbursed, but there are compelling reasons to consolidate this in the form of manuals to:
a) ensure consistency of approach within team and over time, b) build overall sector
capacity, c) share with other country offices, d) allow WSUP management to have
effective oversight, e) address a clear demand from external stakeholders. However, it
was noted that there is a lack of writing skills and influencing skills in the teams,134 so
WSUP needs to think through how this could best be undertaken.
 At present Practice Notes tend to be rather static – once written they stay in the ‘file’.
WSUP needs to consider how they can be updated and developed.
4.2.4 Advocacy
WSUP business is about working with and influencing WASH sector professionals and
organisations. There are two schools of thought135 around who is best placed to undertake
influence and advocacy work;
a) Sector professionals with expertise, or
b) Staff recruited for and dedicated that can lobby and advocate.
We are of the view that given that WSUP’s work is primarily about capacitating service providers
from the inside rather than holding duty bearers to account, it is considered that professional
advocates are very unlikely to have credibility with senior sector personnel, cannot bring in depth
sector knowledge and will not secure a place at the WASH negotiating table. We also note that
WaterAid’s advocacy work is targeted at a higher or non-sector specific level, so working with
generalists. In this sense it may be that WSUP also needs dedicated WASH advocates at the
global level, but not based at the country level.
The influencing work that the CPMs and others do in relation to Outcomes 2 and 3 is critical and
forms part of their ongoing work but of course such work is hard to projectise, budget for and
132 Vasco Parenta – WSUP Sanitation Coordinator

134 Key informant interview Guy Norman.
135 Interview Guy Norman and Neil Jeffery 16th December 2015.
78
measure. However, this is an issue/challenge, because CPMs do not have enough time to
undertake this work. It could be more visible and have associated outputs time that could be
allocated more explicitly to this – a point well made in the Annual Progress Review Mozambique
2014 which draws from experience under DFAT project; “Explicitly allocate resource to
influencing activities as part of future proposals and log frames”. Given the lack of time, for CPMs
in particular, who carry a management and admin load, is a clear impediment to achieving this.
Measures must be put in place to address this and allow CPMs to re-prioritise work areas.
4.2.5 Involvement of water utility expertise in WSUP’s board

It has been one of the hypotheses of this evaluation that WSUP, striving to influence utilities,
society’s chosen core service provider entity, needed its own core utility experience/partner. Both
for the knowledge of ‘mechanical’ management approaches (with ‘organic’ elements as and
when) and for the reputational/promotional aspect that might influence fragile target utilities to be
more receptive. The initial WSUP partnership vision of Thames Water playing that role faded
some years back and the more recent hope that Vitens Evides International (VEI) might play that
role has similarly failed. WSUP has attempted to incorporate this capacity through south-south
partnerships (Manila Water, NWSC Uganda) and through consultants who have worked for
utilities.
It is our view now that the quality and capability of the Country Programme Managers to open
doors may be as important, if not more important, as having a recognisably competent utility as a
significant partner. We also recognise that WSUP’s ability to influence the establishing of a pro-
poor unit in a utility to be second to none. It is clear from respondents that the ability to take staff
from utilities where WSUP is starting to work on ‘benchmarking visits’ to other cities where they
have been working for longer, has been a powerful multiplier effect.
However, external utility/municipality involvement would seem to be an area that WSUP might
consider developing, either in its own staff (it is not clear that WSUP has any staff with actual
working utility experience), or perhaps on secondment from a utility or through a stronger
partnership with one or more experienced utilities.
It appears that, for now, the UK water utilities have given up on their profit-making vision of
working internationally, hence their reluctance to be involved. However, it is equally clear136 that
staff in those utilities would prefer to support water development in lower-income countries
through using their professional skills as much as (if not much more than) for example running up
a mountain to raise funds for WaterAid. There are areas of utility change management, over and
above Non-Revenue Water management, where international utilities could deliver critical
support to lower-income country, often fragile, utilities, if a long-term partnership commitment
could be developed. This would also deliver benefits through the development of a long-term
constituency of support for WSUP within the UK.
4.2.6 Faecal Sludge Management

Provision of and investment in sanitation is a much greater challenge than water supply. This
was evident throughout this evaluation: the obstacles facing financial viability of FSM, continuing
underinvestment in sanitation, bias towards sewerage systems as an end state and higher costs
of sanitation (compared to water). WSUP has rightly focussed a lot of energy and achieved
success with its work on FSM and some very useful work on small scale sanitation finances.
136 Institute of Water Regional Meeting, Cranfield University, November 2015.
79
Valuable as this work is, there is now a need to grapple with the fundamental sanitation
obstacles:
a) It does not pay for itself, so will always need subsidies/transfers to service the poor
(unlike water). Specifically, the rich receive hidden subsidies while the discourse is that
the poor can and should have self-financing systems.
b) Low tech/on-site solutions (with FSM) that may be more suitable for the poor are seldom
favoured end states for many municipal service providers, who invariably see sewerage
as the desired goal, sometime stated explicitly, often not. However, there is a need to
accept that on-site solutions may be required as an end state.
WSUP needs to be able to balance the societal aspirations to move towards sewerage when
topography allows, to support on-site and on-plot as transitional solutions, and to recognize when
on-site has to be the longer term solution. This needs a major shift in attitude to understand
where taxes are required to be sustainable. Without this, much of WSUP’s work will remain as
useful FSM fine tuning, but this will not address the bigger underlying challenges.
4.2.7 Urban Behaviour Change (BC) models – understanding urban motivation

To date, relatively little has been done to explain and illustrate key differences between
BC/Hygiene promotion in urban areas, when compared to rural areas. WSUP therefore do not
yet have an urban poor/LIC BC model ready and ‘off the shelf’. This is not to say that WSUP
programme staff do not intuitively know some of this and put it into their practice, but rather it is
not set down, and experience has not been shared between countries. The Bangladesh research
project137 touches upon this by looking at mass media, but does not go deeper. Work
undertaken by one of the evaluators138 notes that “Hygiene promotion is significantly more
complex to undertake in urban areas…. In the much busier and noisier urban environment
Hygiene Promotion work needs to understand if/how to use cash/material incentives to lever
behaviour change…It must be driven by a strong urban social marketing approach” and goes
onto conclude that it is “not about promoting BC, but more marketing/selling it”.
4.2.8 Research
WSUP has undertaken a successful research programme conducted through research
establishments with global recognition. High level research is an area that many at the global
level value, considering it critical to develop robust evidence and which offers VfM. However, this
route is not without costs and the careful framing of research questions can become limiting
itself.
The environment project (Leeds) is really about gathering evidence for advocacy, which this sort
of research is well placed to do. However, the scale (Urban Institute) project is about
understanding how things work, while the treatment research (Cranfield) is really about
innovation and both building understanding and innovation are not necessarily best served by
“research” projects.
WSUP understands and plans to move towards more considered formulation of research
questions informed by programme needs going forward, which is to be welcomed. However, it is
suggested WSUP’s key characteristics of innovation and ‘deep craft’ may not always be best
137 Using mass media behaviour change messaging to improve communal toilets in Dhaka, Bangladesh.
138
Review of humanitarian WASH preparedness and response in Urban and Peri-Urban areas. Richard Luff 29th
September 2014.
80
supported by high level research. Therefore, WSUP could go one step further and step away
from the restrictions and costs of high level research and instead see research as just one
possible means to answer big questions and to help unlock innovation and progress moving
forward. We recognise it might appear challenging to define and manage outputs but is more
consistent with WSUP’s core ‘organic’ way of working than a somewhat mechanistic and rigid
high level research programme.
4.3 Impact
4.3.1 WSUP’s role in influencing wider change. The importance and meaning of numbers
An important aspect of WSUP’s work is for it to influence wider changes. These can be
influenced directly through its work, for example where a utility that it is working with expands its
service coverage to other areas applying principles and practices gained from the WSUP
partnership139. Nairobi City Water and Sewerage stated this to be the case.
This first area of ‘direct’ influence is very clear. It can be measured (numbers of people with
improved services, etc.) and also the evidence for WSUP’s influence can be cross-checked.
The second area of influence is where WSUP has influenced wider changes indirectly. An
example of such influence would be for example, where UNICEF140 has adopted a WSUP model
and is replicating in a completely separate context and situation (or country).
As reported in section 3, WSUP has found it challenging to be able to account for the numbers of
people influenced by its work. We suggest that these numbers are very important, both for
WSUP as well as for other sector players. Because of WSUP’s model (see Relevance and USP),
the numbers of people impacted and the contexts where WSUP has influenced work give a very
valuable perspective on the broader effectiveness and impact of its work. In essence, the critical
issue for WSUP is to see where it has contributed towards moving engagement in water and
sanitation from pushing and influencing beyond the tipping point to where the changes have
become dynamic and have a life of their own. With the sheer scale and enormity of the challenge
of urban water and sanitation, this is the impact that WSUP should seek to contribute towards
and understand how it is influencing those higher level dynamic changes.
4.3.2 Understanding the context

It is critical to consider the extent to which location specific knowledge/understanding can be
replicated elsewhere and or how much context determines what solutions can be used
elsewhere. This is conceptually well understood by WSUP, as evidenced from a number of
sources: the Mozambique Annual Review141 acknowledges that while context differs, learning still
has value outside its place to origin. This was well articulated by the explanation that “WSUP
must copy - adapt, not copy – paste”142. Bill Peacock noted that it is critical to understand the
enabling environment in a country and be able to get stakeholders to grasp this. Guy Norman
suggested that knowledge has to be context specific and so the idea that packages/models that
can be seen as exportable, is an approach WSUP is moving away from in the new business
139 AdeM Maputo. Meeting with AdeM Maputo (20/01/16). Correspondence Carla Costa (10/02/16).
140 This is given as an example recognizing that UNICEF are beginning to move into working in Urban Sanitation.
142 Antonio Junior - WSUP water programme coordinator.
81
plan. This idea is also evident in the thinking behind how any future research programme would
be run: this should be more about context specific learning rather than more generic learning and
with a move towards national and away from international143 engagement.
We want to stress the importance of maintaining (and further developing) the approach of the
‘copy and adapt’ approach. The evidence from this evaluation is clear that the Masterclasses and
exchange visits to other countries are powerful means to enable to the process of ‘copy and
adapt’. There are overarching issues that occur in nearly all locations with patterns that are
clearly evident, though the relative significance of these of course varies. Thus, some knowledge,
for example as per the PNs, TBs, etc. is transferable, but ‘the whole package’ is not.
We suggest there is a need to separate location specific knowledge from the business
environment and other contextual factors. It is evident that in WSUP’s considerable body of
knowledge they recognise this and have started to identify patterns and help guide others
through a top line decision-making process144. Key factors and the enabling environment are set
out in the WSUP Advisory ‘everyone forever’ diagram. This work could be supported by a
decision making manual that explains these key context factors more with examples from
WSUP’s programme145. This sort of tool could be very usefully accompanied by a critical look at
WSUP’s array of country experience set out in its PNs and TBs, with a complementary
explanation which outlined in what other environments this experience could be transferred and
how it might be adapted. For example, how a municipality would know whether WSUP’s
compound sanitation strategy in Ghana146, or communal toilets in Maputo147 would be applicable
in Chittagong.
4.4 Sustainability
4.4.1 Embedding sustainability through developing a shared agenda/approach.

As mentioned under effectiveness a critical aspect of WSUP’s USP is the way in which the
organisation successfully links utilities and communities (especially poorer communities) to
effectively collaborate on the provision, delivery and use of improved water and sanitation. This is
achieved through a strategic mix of support, capacity building, demonstration, modelling and
experimentation. A question to raise is how WSUP can build on this model in other areas –
specifically in hygiene behaviour change.
A key finding from the evaluation is that sustainability is built not just through the processes
outlined above, but also through the engagement and buy-in of key stakeholders – in utilities and
communities (through the development of CBOs).
For hygiene behaviour change it is clear that the key stakeholders are the communities (CBOs,
Parent Teacher Associations (PTAs), as well as Principals and Teachers and the local
departments of Education. An issue to consider is how other potential stakeholders, such as
business, could engage in such a way as to ensure a harmony of objectives that would see
143 Interview Guy Norman.

144 Bangladesh 2013 Annual Review. TB 004 & TB 006.
145 WSUP’s publication Main Global guidance Note for the Urban Poor.
147 PN 002: Financing communal toilets: the Tchemulane project in Maputo (February 2011).
82
issues such as market penetration and coverage as longer term desirables rather than as
specific elements of such a relationship.
4.4.2 Improving M&E to be able to assess sustainability

This evaluation does not comment on the M&E system as a whole and whether this is fit for
purpose. It focusses on where data generated from the system directly supports the Outcome
areas 1, 2, 3 and 4. WSUP has built up an extensive and impressive monitoring framework. This
is testimony to the effective work of the Evaluation Research and Learning (ERL) team. The
team appears positioned to be genuine knowledge holders, given they conduct annual
programme reviews (APRs), lead presentation at conferences, lead on consolidating and
compiling learning. In this sense their work goes beyond collection of data for management
purposes and enables them to perform a genuine learning function. It is also notable that the
process of the APR includes feedback on quality of support from UK teams and contributes to a
virtuous circle of learning148. We would however take this opportunity to propose areas where the
M&E system can be refined to support better learning and accountability.
A critical point, recognised by WSUP, is that a lot of the data collected relates to improved
access rather than detailed evidence of sustained use and change. We would suggest that
WSUP should make more use of its well-developed linkages with communities and its field staff
to develop a programme of longitudinal studies. We would propose that it develops and builds in
a straightforward sustainability monitoring system. Options include adopting a sampling approach
of visiting a selection of sites every year or of using a more qualitative methodology such as the
‘Reality Check’ approach149. This can be applied across WSUP’s main programme areas and
also (together with its utility partners) in some areas where it is to have an influence through
leveraging.
In working through this evaluation process we have been aware of the sizeable amounts of
information, reports and documents produced. A significant challenge for a programme such as
this is how to ensure that data, particularly M&E data, can be used to helpfully and quickly get
across new concepts, as well as information on progress and challenges. With the importance of
the linkages that WSUP has with utilities and service providers, it is important that WSUP look to
ways of providing and sharing information that enables them to engage with and use such
information. Many organisations are now looking at using infographics as a means of sharing and
communicating information in ways that minimise language barriers and support ownership and
learning.
149 http://www.reality-check-approach.com/what-is-rca.html
83
5. Recommendations
These recommendations are provided in the knowledge that WSUP is well into planning for the
next phase of its work, so some of these ideas may already have been raised and agreed upon.
Recommendations are provided for each of the main overarching findings and the corresponding
section number is provided here for reference.
5.1 Relevance
5.1.1 Recommendation 1 (relates to 4.1.1 WSUP’s USP)

WSUP has developed itself, with DFID support, into a key global resource for facilitating pro-poor
institutional change in lower-income country utilities and municipalities, as well as in supporting
private sector and household involvement in ensuring safe and sustainable public health
improvements.
WSUP can now build on its status and position as a market leader/influencer to take its role as
an influencer to a higher level and use its cadre of ‘model utilities’ and, most critically, key people
in those utilities along with its Programme staff (particularly CPM’s, supported by its UK team), to
influence that decision making and direction of WASH provision and support by IFI’s and
International donors.
We note the consideration in the next business plan with respect to working in additional
countries and we strongly recommend that WSUP extend its programme to two new countries,
with at least one of them being in Asia. A proposal for consideration would be to develop new
country programmes in Myanmar and Nigeria, noting the very different challenges in both.
5.2 Effectiveness
5.2.1 Recommendation 2 (relates to 4.2.1 Imbalances/system bias)

Continue and expand work on sanitation by:
 Developing a more strategic partnership with the World Bank at the global and national
level, primarily to further the sanitation agenda, through joint research and learning,
conferences and in-country strategy development. This also in the context of the change in
the role of WSP.
 Build upon selective engagement with UNICEF and UN Habitat at the national level where
they are engaging in urban sanitation work, and in time at the global level. There is a clear
opportunity to support UNICEF as it develops its urban programme.
 Include attendance at regional sanitation conferences in regions where WSUP does not
have country programmes, along with selective attendance at World Bank events.
5.2.2 Recommendation 3 (relates to 4.2.2 Addressing inclusion effectively)

In order for WSUP to maintain and build on its position of working at the ‘cutting edge’ we would
want to propose that it should look to developing a system of key personnel across the
organisation who can act as the ‘go to’ people for advice and support on each of the main
concept areas of WSUP’s work.
84
5.2.3 Recommendation 4 (relates to 4.2.3 Strategic learning)

Enhancing learning
Take a more holistic view of learning by building a suite of learning packages and approaches,
which can be deployed flexibly for both WSUP/partners and in support of other organisations,
drawing from the excellent work already undertaken. This to include:
 Adapting the Masters module to be standalone (i.e. not university accredited at this time)
available for face-to-face and on-line learning. (In the course of time these can
subsequently be re-introduced into Masters courses, as and when opportunities arise).
 Develop an enhanced programme of peer-to-peer learning based upon occasional
exchange visits and improved Masterclasses and complemented with regular in-country
and remote learning through skype webinars, etc. (For example, Masterclasses could
require potential participants to submit papers/topics for discussion and these could be
used to vet participants, getting more participant involvement and allowing follow up on
key learning.)
 Understand how more internal learning opportunities, particularly about other WSUP
country programmes, can be provided to WSUP staff.
5.2.4 Recommendation 5 (relates to 4.2.4 Advocacy)

Strengthening its advocacy capacity at country programme level
Explicitly allocate resources to influencing activities at the national level as part of future
proposals and log frames, considering simple ways to projectise this work, so it can be better
managed and measured. Seek ways to ensure senior staff can expand their influencing role by
for example simplifying and reducing the management burden.
5.2.5 Recommendation 6 (relates to 4.2.5 Involvement of water utility expertise in WSUP’s

board and activities)
Work with WSUP Advisory/key European utilities to develop a concept of whole system analysis
that will support and enable WSUP to influence high level capacity and effectiveness with
decision makers, funders, national and local governments.
Develop more effective ways to interact with the UK’s water utilities, both a) to harness their skills
in the more mechanical aspects of utility service delivery, potentially through long-term partnering
between utilities with a developmentally appropriate ‘one in, one out’ staff exchange approach,
and b) to develop a UK constituency of support for WSUP.
Use of appropriately sensitised UK utility Directors to facilitate high-level interaction and

advocacy in WSUP target countries.
5.2.6 Recommendation 7 (relates to 4.2.6 FSM)

Looking forward we see a need for more strategic sanitation work to be undertaken by WSUP
that makes the case for a system wide change in attitude to recognise the need for ongoing
sanitation subsidies and appropriate on-site technology. This could be achieved by undertaking:
 A macro level analysis that sets down evidence from different parts of the world of how
subsidies/tariffs/transfers are required, carefully explaining the extent to which the poor
bear a disproportionate cost burden or not.
85
 Justification that shows where on-site sanitation systems (with FSM) are a valid end state
rather than a means to a ‘sewered’ end state, building upon work by Mara150 at al. This is
critical to move municipalities and utilities beyond ambivalence towards onsite sanitation
investment. This analysis should be built up from the sanitation value chain (as
highlighted in the Stone Family Foundation funded project final evaluation151), while
taking into account population density, lack of tenure/illegality of settlements, benefits of
community management of systems.
 Situate WSUP’s work on FSM within this bigger picture to know where efforts should be
concentrated.
 Advocacy and communication work that sets down the case for this change in approach.
It is suggested that a lay person’s explanation that sets down key financial, social and
technical differences with water supply would be a helpful part of this.
5.2.7 Recommendation 8 (relates to 4.2.7 Urban BC models – understanding urban

motivation)
Consolidate WSUP programme experience and define research work to develop a much clearer
understanding about the specificities of urban BC. This must place emphasis on drawing out the
different experiences of men and women and take account of context factors/the enabling
environment.
5.2.8 Recommendation 9 (relates to 4.2.8 research)

WSUP expand their proposed research programme into an enquiry programme that can use a
variety of means; high level research conducted with international academic establishments,
extensive surveys, enquiry through national academic establishments, commissioning innovation
etc.
5.3 Impact
5.3.1 Recommendation 10 (relates to 4.3.1 WSUP’s role in influencing wider change. The
importance and meaning of numbers)
Further develop country protocols and guidance by carefully examining cases this evaluation has
highlighted and considering cases in other WSUP countries. Consider how WSUP Advisory and
WB/WSP might measure their influence. Use this to build an improved protocol with examples to
illustrate.
5.3.2 Recommendation 11 (relates to 4.3.2 Understanding the context)

Developing a decision-making guidance note/monograph that sets down key factors that must be
considered as part of context analysis. Draw WSUP’s collective experience of templates/models
together in direct comparison and so providing a menu of options explaining which of WSUP’s
templates/interventions models are suited to which situations, taking account of context factors/
analysis.
150 Low cost sewerage, Duncan Mara. 1996

151 Final evaluation of SFF-funded WSUP project in Chazanga, Mtendere and Kanyama – Final Report.
86
5.4 Sustainability
5.4.1 Recommendation 12 (relates to 4.4.2 improving M&E to be able to assess

sustainability)
Recognising the importance of WSUP’s influencing role we recommend that it develop its
monitoring framework to incorporate monitoring of sustainability and use. Such an approach,
whether through taking a random sampling approach with communities on an annual or bi-annual
basis, or using more qualitative approaches such as the ‘Reality Check’ approach, will support its
work in influencing service providers to copy/adapt reflect and learn from its work.
We also recommend that with the density and complexity of information and concepts that
WSUP look to more visual means of communicating and enabling stakeholders and other sector
players to learn from and use information that it provides.
87
Annex One – List of consulted documents

WSUP Programme documents
- Business case
- Theory of Change 2012-2015
- Results Chain
- Budget and budget assumptions
- Logical Framework
- Organisational chart
WSUP Publications
- Practice Notes 1-20
- Discussion Papers 1-7
- Topic Briefs 0-14
- Perspective Pieces 1-3
- Urban Programming Guide
- Urban Water Supply Guide
- Climate Proofing Reports Naivasha & Antananarivo
- ‘How to Climate Proof’ Report
WSUP reports to/from DFID 2012-2015

- Annual and Six Month Reports
- Briefing on WSUP Madagascar Programme & DFID Contribution
- DFID Annual Reviews
WSUP Internal Quarterly Reports

- Narrative Quarterly Reports of Q2 2015-2016
- WSUP Programmes Outcomes Dashboards Q2-4 2014-2015 and Q1-2 2015-2016
Annual Progress Reports from years 2013, 2014 and 2015

- Bangladesh
- Ghana (not 2015 report)
- Kenya
- Mozambique
- Madagascar (not 2015 report)
- Zambia (not 2015 report)
- African Cities of the Future Annual Reports 2010, 2011, 2012, 2013, 2014
Research
- WSUP Impact study
- Behaviour Research Quarterly Reports
- City Briefs on Scale from Nakuru, Maputo and Ghana West
- WSUP Desk Reviews on Scale
- WSUP Quarterly Reports on Scale
- Treatment Outputs and Findings reports from Accra and Nairobi
- Masters Theses on Treatment, by Louisa Fearn and Collette Genevaux
External Evaluations
- AUSAID Mozambique
- EC LINDA Zambia
- GPAF Kenya
- ODI 2011 of BMGF Maputo and Antananarivo
- Stone Family Foundation Zambia
- TIUP 2009 of WSUP
88
Annex Two – Revised work plan

WSUP Evaluation Months
Nov-15 Dec-15 Jan-16 Feb-16

Activities and deliverables
23-29 30-6 7-13 14-20 21-27 28-3 4-10 11-17 18-24 25-31 1-7 8-14 15-21 22-28
Phase 1 - Inception
1 Briefing and discussion with WSUP, collection of documents
Initial interviews with selected WSUP staff and key/critical
2
stakeholders
3 Initial document review and country selection (criteria)
Develop evaluation workplan, methodologies and questions -
4 14-Dec
drafting of and submit inception report
2. Data collection
A. Work Prior to country visits
5 Detailed documentation review
Phone, Skype, possibly e-mail surey, with WSUP staff, SP, WSP,
6
WB and other key stakeholders in 6 countries
First review of WSUP end-line household and infrastructure survey
7
and pubications survey, (raw data available 18 Dec)
Review of WSUP annual assessment of institutional capacity in
8
each city
B. Country visits
Two country visits, including: prep, interviews/WSs/FGD with
WSUP staff and other stakeholders, Theory of Change workshop,
9
visits to project locations, including meetings with community and
SME
C. Work post country visits
10 Revisit of WSUP surveys (final data available 31 December)
11 Follow up face to face interviews with key WSUP staff in London
Skype calls with WB and other key stake holders that WSUP
12
expect to influence
Phase 3. Data analysis and preliminary findings
13 Produce outline of preliminary findings
14 Present preliminary findings to WSUP/key stakeholders
Phase 4. Analysis and Reporting
15 Draft report 12 Feb
16 Feedback on report
17 Finalisation and submission of report 28 Feb
89
Annex Three – Evaluation Matrix

The following table proposes a number of questions for consideration against each Outcome and the DAC criteria.
A) relevance B) effectiveness WSUP C) efficiency D) impact E) sustainability
output
indicator
Outcome 1. Are the models suitable for Have models delivered 3.45m Have models delivered What has been the Have models delivered a
Demonstrated the urban poor, offering improved services; water more efficiently in economic social and health track record of more reliable
models of advantages over other sanitation and hygiene, to the comparison with impact of new models on & regular service provision
urban WASH models? In particular do most economically services available in poor and in particular for over longer periods of time?
service they bring better services marginalised, in particular for other localities, women and girls and how Are poor people, particularly
delivery for for women and girls? women and girls? particularly poorer many people have directly women, able to maintain
the urban unserved or under- benefitted? payments for services on a
poor served areas? regular and ongoing basis?
Outcome 2. Are models for service Are service providers able to 12.5m To what extent have What impact have changes How long have new business
Service delivery relevant for a more effectively deliver services service providers had on service providers & models been established
provider range of service providers to urban poor, including women improved their the way they have with demonstrated service
capacity and from SMEs to utilities? and girls? efficiency in delivered economic, social delivery on this basis? Are
institutional Have service providers comparison with their and health impacts? How business models viable, do
framework to realigned their institutional previous work and with many people have they take account of losses,
sustain focus to be pro-poor? other service delivery benefitted indirectly & are and environmental and
improvement models? there differences in impact social sustainability?
process between females & males?
Outcome 3. What are the wider To what extent have new US$220m To what extent have Are pro-poor and gender Is there the right balance
Scale – City changes that have investments been made in city wide policies attentive models and ways between capital grant/loan
wide occurred in investment unserved or underserved areas prioritised efficiency & of working being accepted investment & revenue versus
investment programmes and policy at where the poor live? have new investments for city wide stakeholders expenditure in areas where
triggered the city level and how been made on the and their business models? urban poor live & more
many people have basis of delivering widely across the city? Is
benefitted? service efficiencies there a city wide resource &
particularly for the waste management plan?
poor?
Outcome 4. Do new models of service To what extent have DFID, IFIs, To what extent have To what extent have urban How have investment
Influence: provision, capacity building UN & service providers adapted global polices and poor populations benefitted policies and practices of
Sector and investment decision their ways of working and investment prioritised from improved urban other organisations
influence making influence or align focused investments to be more efficient service WASH service provision? addressed social, economic
worldwide with emerging global pro-poor as a result of WSUP provision for the poor? and environmental
thinking? dissemination, research & sustainability?
training?
90
Annex Four – Country programme visit criteria for selection and

visit outline
Criteria for country selection (as in the Inception Report)
An important element of the validation process will be through carrying out two country
programme visits. A consideration for assessing which country programmes to visit will be the
ability of a country programme to demonstrate a full span of programme interventions across
Outcomes 1-4, and ideally over a range of WASH activity type – water, sanitation and hygiene.
The team have used the following criteria to make a decision on which two country programmes
would be most suitable:
1. The programme has been established for a sufficient duration of time in order to demonstrate
evidence across Outcomes 1, 2 and 3.
2. There is a significant difference in; city context, scale, political autonomy and stability,
deregulation, in order to more closely test the achievability of Outcomes 2 and 3 under a
variety of conditions.
3. The level of programme documentation available and the ability to hold phone calls with key
stakeholders will highlight if there are gaps in information which warrant a country visit.
4. To include countries where WSUP (and the evaluation team) believes its Theory of Change
is working well and not well.
Following the inception meeting with WSUP staff a further three criteria were added:
5. Maturity of the programme (progress not length of time).

6. Usefulness of the programme to WSUP (areas of focus or innovation, for example Ghana:
Innovative Compound Strategy).
7. Safety considerations/risks.
Each country programme is scored on a 1 (low/poor) to 3 (high/good) scale.
91
Scoring of the six country programmes showed clear differences between the new and more
established countries. Looking at the three high scoring country programmes the team propose
to visit Kenya and Mozambique, rather than Madagascar, as the programme has a stronger
research evidence base as well as the Masters course delivery.
The teams proposed for the country visits are as follows:
Dates Country People
11-17th January Kenya Richard Franceys

WSUP sites: Nakuru/Naivasha, Jerry Adams
Nairobi and Mombasa
18th – 24th Mozambique Richard Luff
January WSUP sites TBC Jerry Adams
The country programme visits will last one week (5 working days) each.
92
Annex Five – Country visit programmes

WSUP Evaluation – Mozambique programme 18th – 22nd January 2016
Date Time Activity
9.00 (JA/RL) Overview of programme in Maputo – WSUP team
11.00 (JA) bi-lateral meetings Dinesh and Gafur – WSUP CDOs
AM
11.00 (RL) meeting Nelson Matsinhe – Universidade Eduardo Mondlane,
Day 1: Maputo
18/1/16 Lunch
14.00 (JA/RL) Meeting with Head of Neighbourhood, Chamanculo C
PM 15.00 (JA/RL) Visit to three Communal Sanitation Blocks (2009 – 2015
construction)
9.00 (JA/RL) visit to Waste Water Treatment Plant – CMM
AM
11.00 (JA/RL) meeting with Manuel Alvarinho – CRA
Lunch
Day 2: 14.30 (JA/RL) meeting with Vasco Parenta – WSUP sanitation coordinator
19/1/16 15.00 (JA/RL) meeting with Victor Fonseca – CMM
16.30 (JA/RL) Meeting with Alfonso Alvestegal – UNICEF WASH Urban
PM
Lead
18.30 (JA/RL) Skype call Mark Henderson Former head of – UNICEF
WASH, Mozambique
9.00 (JA/RL) Meeting Lidia Salatiel, Jose Henrique Barata, Isabelle – AdeM
AM
10.45 (JA/RL) Meeting – Frederique Martins (and two others) – AIAS
Lunch
Day 3: 2.00 (RL) meeting with Adriana Aifaz – WSUP Community Development
20/1/16 Officer
PM 2.00 (JA) Focus group discussion – FSM operators
16.00 (JA/RL) meeting Antonio Junior – WSUP Water Programme
Coordinator
08.00 (JA) Visit to two WSUP intervention schools EP1 Unidade 23 primary
school, Maxaquenab primary school – meeting with principals (school
holidays)
AM 08.00 (RL) Meeting with Arone Tivane – AdeM NRW team and Job
Mutombene – CBO in Mafala
Day 4: 10.30 (RL) Meeting with Susie Kinghan – WSUP Water Specialist
21/1/16 12.30 (JA/RL) Meeting Carla Costa – WSUP Country Programme Manager
Lunch
14.30 (JA/RL) Focus group with women involved in WSUP’s work –
PM community
16.00 (JA/RL) Meeting Vasco Parenta – WSUP Sanitation Coordinator
9.00 (JA/RL) meeting with Pedro Paulino – FIPAG (cancelled)
AM
10.30 (JA/RL) Meeting Carla Costa – WSUP country programme manager
Day 5:
Lunch
22/1/16
13.00 (JA/RL) Feedback to team
PM
14.30 (JA) meeting with Councillor for Education – CMM
93
WSUP Evaluation – Kenya programme 11th – 15th January 2016
Tim Activity
Date
e
AM RF – Meeting with Kariuki Mugo, WSUP Country Programme Manager
Day 1: Lunch
18/1/16 RF Meeting with Jack O'Regan – WSUP Enterprises
PM
RF Meeting with Chris Heymans – WSP World Bank
RF Travel to Mombasa. Meet with: Philip Oyamo
AM Interview with Acting (and this day departing) MD – Kombo Rajab
Introduction with VEI Project Manager
Day 2: Lunch meeting with Philip Oyamo and Emily Kirigha (WSUP Mombasa
sociologist)
19/1/16
Meeting with Nancy Wanyinyi the very recently replaced Head of 'Low
income consumer services Department'
PM
Visit with Emily to newly connected (that morning) households in a low-
income settlement
JA arrive. JA & RF briefing meeting WSUP team
AM
Day 3: 11.00 JA & RF Kibera visits to communities and individuals
20/1/16 Lunch
PM Meeting with WSUP Team
08.00 (JA) Visit to Naivasha. Meet James Kariuki, WSUP Programme
Manager Naivasha. . Field visit to Karabila (Water Kiosks) and Menerva
AM
(Water Kiosks and shared latrines). Meet representatives of Water Users
Associations (WUA’s)
Day 4: Lunch
21/1/16 RF Meeting with Peter Njaggah, Director of Technical Services, WASREB
JA Naivasha: Visit to Kasaranye to see an FSM operation. Met with PHO
PM (Samuel) & Clinical Officer (Fred). Discussion on recent cholera outbreak.
Travel to Nakuru
RF Meeting with Nahashon Muguna, Technical Director, Nairobi Water
JA visit to Nakuru Water and Pro-Poor Unit. Met Grace Head of
Communications.
Visit to Rhonda and Lawanjiru to see Water meters.
AM
RF Final de-brief meeting with Country Programme Manager (CPM) Kariuki
Mugo
Day 5: RF Meeting with Dr Andrew Trevett, UNICEF Kenya WASH Director
22/1/16
Lunch
JA Nakuru: Visit to Nakuru Water Utility Western Zone Payments Office.
Field visit to Mwaritie. Visit to Southern Zone Nakuru: Manjari area. Return to
PM
Nairobi.
Evening: RF/JA Final debrief
RF Subsequent interviews with WSP team members: Lilian Achieng Otiego;
Clifford James through Kevin Bedner/Glen Pearce Oroz
94
Annex Six – List of stakeholders interviewed
Name Role Organisation Country Date of

interviews
Abdus Shaheen Country Programme WSUP Bangladesh 04/01/2016,
Manager 10/02/2016
Adriana Aifaz Community WSUP Mozambique 20/01/2016
Development Officer
Alfonso Alvestegal WASH Urban Lead UNICEF Mozambique 19/01/2016
Dr Andrew Trevett WASH Director UNICEF Kenya 22/01/2016
Anisur Sarkar Programme Engineer WSUP Bangladesh 10/02/2016
Antonio Junior Water Programme WSUP Mozambique 20/01/2016
Coordinator
Anita Layden Programme Manager WSUP Bangladesh 11/01/2016
Anne Walsh Head of Innovations & WSUP UK 27/01/2016
Consumer Needs
Arone Tivane NRW Team AdeM Maputo Mozambique 21/01/2016
Baghi Baghirathan Programme Director, WSUP UK, 17/02/2016
WSUP Madagascar Madagascar
(and formerly for
WSUP Mozambique)
Bill Peacock Director of WSUP UK, Kenya, 16/12/2015
Programmes; Mozambique
Programme Director
for WSUP Kenya,
Mozambique
Brian Reed Lecturer in Public WEDC, UK 11/01/2016
Health Engineering Loughborough
University
Carla Costa Country Programme WSUP Mozambique 18/01/2016
Manager
Head of Maputo Mozambique 18/01/2016
Neighbourhood
Chamanculo C Bairro
Chris Heymans Senior Water & World Bank Kenya 18/01/2016
Sanitation Specialist,
WSP
Clifford James WSP World Bank Kenya 15/01/2016
Dinis Namburete CDO WSUP Mozambique 18/01/2016
Emily Kirigha WSUP Mombasa WSUP Kenya 19/01/2016
Sociologist
Frederique Martins AIAS Maputo Mozambique 20/01/2016
Abdul Gafur CDO WSUP Mozambique 18/01/2016
Georges Mikhael Head of Sanitation WSUP UK 17/02/2016
Grace Kabubu Head of NAWASSCO Kenya 22/01/2016
Communications
Guy Norman Head of Evaluation, WSUP UK 16/12/2015
Research and
Learning
Habibur Rahman Deputy Programme WSUP Bangladesh 10/02/2016
Engineer
Issaka Balimah Musah Country Programme WSUP Ghana 08/01/2016
Manager
Jack O'Regan SmartLife Manager WSUP Kenya 18/01/2016
Enterprises
95
James Kariuki Project Manager WSUP Kenya 14/01/2016
Jane Crowder Water and Sanitation DFID UK 21/12/2015

Adviser
Jose Henrique Barata AdeM Maputo Mozambique 20/01/2016
Job Mutombene CBO, Mafala AdeM Maputo Mozambique 21/01/2016
Kariuki Mugo Country Programme WSUP Kenya 11-
Manager 16/01/2016,
09/02/2016
Kombo Rajab MD Mombasa Water Kenya 19/01/2016
(now replaced)
Lidia Salatiel AdeM Maputo Mozambique 19/01/2016
Lilian Achieng Otiego WSP World Bank Kenya 11/01/2016
Lovy Rasolofomanana Country Programme WaterAid Madagascar Email
Manager Madagascar (Skype not
possible)
Manual Alvarinho President CRA Maputo Mozambique 19/01/2016
Mark Henderson (Former) Head of UNICEF Mozambique 19/01/2016
WASH
Nahashon Maingo Technical Director Nairobi Water Kenya 21/01/2016
Muguna NCWSC
Nancy Wanyinyi LIC Head of Mombasa Water Kenya 19/01/2016
Department
Neil Jeffery Chief Executive WSUP UK 16/12/2015
Officer
Nelson Matsinhe Lecturer and Universidade Mozambique 18/01/2016
Researcher Eduardo
Mondlane
Paul Gunstensen Director of WASH Stone Family UK 05/01/2016
Foundation
Peter Hawkins Country Team World Bank Mozambique 15/01/2016
Leader, WSP
Philip Oyamo Project Manager, WSUP Kenya 12/01/2016
Mombasa
Zaituni Kanenje Pro-poor Programme NAWASSCO Kenya 15/01/2016
Co-ordinator
Peter Njaggah Director of Technical WASREB Kenya 21/01/2016
Services
Regine Skarubowiz Monitoring and WSUP UK 16/12/2015
Evaluation Manager
Reuben Sipuma Country Programme WSUP Zambia 08/01/2016
Manager
Sam Drabble Research and WSUP UK 16/12/2016
Evaluation Manager
Samuel Kingori Wairia PHP Nakuru County Kenya 15/01/2016
Susie Kinghan Water Specialist WSUP Mozambique 21/01/2016
Sylvie Ramanantsoa Country Programme WSUP Madagascar 26/02/2016
Manager
Tim Hayward WSUP Advisory WSUP Advisory 29/01/2016
Timothy Wanjohi Manager Southern NAWASSCO Kenya 15/01/2016
Zone
Vasco Parenta Sanitation WSUP Mozambique 21/01/2016
Coordinator
Victor Fonseca Civil Councillor CMM Maputo Mozambique 19/01/2016
96
Annex Seven – Supporting questions frameworks

A) Relevance – The extent to which the aid activity is suited to the priorities and policies of the target group, recipient and donor.
B) Effectiveness – A measure of the extent to which an aid activity attains its objectives
C) Efficiency – Efficiency measures the outputs -- qualitative and quantitative -- in relation to the inputs
D) Impact – The positive and negative changes produced by a development intervention, directly or indirectly, intended or unintended
E) Sustainability – Sustainability is concerned with measuring whether the benefits of an activity are likely to continue after donor funding has been
withdrawn
Key questions and means to gather data in relation to Outcome 1
Outcome 1. Demonstrated models of Urban WASH service delivery for the urban poor. Programmes deliver service improvements to a representative
population of urban poor in each town/city, demonstrating strong local ownership and informing city wide water and sanitation strategy.
Questions/data checks Possible Data sources Notes/comparisons/counterfactuals

Output level. Demonstration. ACTIVITY 1.1 Improved Water and Sanitation Services
Relevance Is the programme able to deliver models of urban Country Programme and Annual Reports. Is it possible to drill down into the data
WASH that are accessible to the urban poor Publications, updates and reports on access of to compare the relevance of different
disadvantaged groups. models? Are there preferences/more
Interviews with user groups in-country desired models?
Household surveys Perceptions of other stakeholders
Effectiveness Has the programme achieved the numbers of people Country Programme and Annual Reports, Compare actual numbers
with access to water and sanitation – scale? User and Household surveys (disaggregated) with targets and
How appropriate/effective have marketing Interviews with CP staff, user groups and other milestones). This needs to be looked at
approaches been? stakeholders (WaterAid, WSP etc.) together with Outcomes 2 and 3 to
(To what extent have low income consumers living in understand whether the numbers are
the targeted areas benefitted directly from the water perceived to be ‘scalable’
and sanitation services implemented under this Speak to other stakeholders
programme) Link to other WSUP programmes – Clean
Team in Ghana.
Efficiency Have the different country programmes been on track Cost per beneficiary (disaggregated Important to look for differences
to deliver the numbers of people with access (global/country/programme). Comparison with between new and established
other organisations CARE, WaterAid) programmes.
97
Has the programme delivered cost efficient models to Reports and cases of access. Assumption that delivery of numbers
different urban poor groups Household surveys supports more investment – leading to
Interviews country programme team; Focus more people with access
Group discussions with user groups and site visits Are there significant differences
between higher cost and lower cost
programmes?
Impact Is there evidence of models being developed/ Reports, Would be good to see whether there is
replicated locally or by others? Interviews with country programme staff and key any evidence of preference for any non-
Is there evidence of models informing strategy (locally stakeholders. Country programme visits: WSUP models (if available or known).
or further) interviews with sector stakeholders Is there any evidence of negative impact
of models
Output level. Demonstration. ACTIVITY 1.2. Improved Hygiene Knowledge
Relevance Is the programme able to deliver models of hygiene Annual Reports and updates. Important to compare stories of
education that are relevant to the urban poor? Household surveys Interviews with user groups successes from publications, reports and
Are there models of hygiene education relevant to in-country programmes Country programme updates with Household survey and
specific groups – women, girls, schoolchildren visits: FGD’s with beneficiaries and user groups country programme site visits
Effectiveness Has the awareness created through the hygiene Annual reports Interviews with key stakeholders Annual reports give numbers of people
education programmes resulted in changes in Site visits: interviews with user groups who have been covered
behaviour
Efficiency What are most cost-efficient models of hygiene Annual reports. Check with other actors/stakeholders –
education for different groups? Studies WaterAid, UNICEF
Publications Need to explore replicability of models
Interviews country programme staff (such as ‘School of 5). What are key
factors in sustaining hygiene behaviour?
Impact What has been the impact of hygiene education Household survey data Evaluation Reports Have there been any negative impacts
work? Studies Interviews with community/user groups or challenges as a result of improved
Evidence of sustained change or sharing/replication hygiene
beyond the project areas?
98
Output level. Demonstration. ACTIVITY 1.3 – Environmental Sanitation

Relevance Is the programme able to deliver models of Country Reports
environmental sanitation that are relevant to the Clean Team Ghana
urban poor? Country programme visits
Effectiveness Have environmental sanitation programmes resulted Site visits Clean Team Ghana. What about non
in significant changes to urban environments? Focus group meetings with user groups WSUP models and approaches:
Studies? WaterAid, others?
Efficiency What are most cost-efficient models of Review of reports. Stakeholder interviews: Investigate Clean Team Ghana against
environmental sanitation? Representatives of Clean Team Ghana country other WSUP non WSUP models
visits
Impact Is there evidence of environmental sanitation work Studies From WSSCC field studies or rural
supporting/impacting on sustaining demand for Evaluations sanitation – improved environmental
sanitation and water and behaviour change country programme site interviews sanitation resulted in additional benefits
and supported a desire to maintain a
healthy environment. Is this the case for
urban environments? What are the
drivers?
Output Level. Gender. Cross-cutting
Relevance Is the programme delivering models of Water and Studies Programme has been fairly quiet on this
Sanitation provision that are relevant to the needs of Interviews with country programme staff, User apart from Menstrual hygiene and
women and girls? groups (especially women headed user groups) schools? What is perceived relevance of
what they are doing?
Effectiveness Has the programme delivered to women and girls Reports The logframe gives a baseline of 51%
Disaggregate data across country programmes to coverage of women and
see where there are significant differences milestones/targets of 52%. Is it possible
Country programme visits: to analyse whether the programme has
provided better quality of access and
Site visits
use that is changing attitudes and
FGD’s with user groups practices and health benefits?
Efficiency Are there additional costs to ensuring access to Interviews with country programme staff Site Check with other stakeholders -
women and girls (latrines) are these acceptable to visits WaterAid
communities? FGD’s with communities and women’s groups
99
Impact Is there evidence of the programme positively Evaluations? Is there any evidence of attitude studies
impacting on attitudes towards appropriate and FGD’s with women’s groups on men and women
effective provision of relevant WASH models for the The Household surveys may provide some
Urban Poor that are gender inclusive insights to follow up on.
WSP country programme staff (Kenya and
Mozambique
WSUP Practice Notes

Outcome 2. Service provider capacity and institutional framework to sustain improvement process. Demonstrable efficiency and effectiveness
improvements are embedded in service provider operations and institutional capacity is strengthened, ensuring the improvements continue. – leading to
12.5m people with improved access to sustainable, affordable and viable WASH services
Considering the semi-structured interview questions below, related to the Theory of Change statements but all in the context of WSUP’s capability,
knowledge, skills, attitudes and practices, as opposed to the alternative capacity development offerings. What is WSUP’s USP? Is it selling ‘pro-poor
units’, NRW and FSM? Asking in-country to score WSUP relative to the competitors? E.g. In what ways better/different from Vitens/WSP_WB/SUWASA?
What sort of ‘animal’ do you think characterises WSUP? Does it have/have access to the right skills and knowledge? How strong/effective the link with
water utility capacity?

Output level. 2.1 Having the capacity to adopt and scale up effective models contributes to service providers and local/national government committing to adopt
and scale up effective pro-poor models at a city wide level
Relevance Are models for service delivery appropriate Water utility Alternative models from WSP/WOP/
for a range of service providers from SMEs SME Service Provider SUWASA/KFW/WSTF (Kenya)
Effectiveness
to utilities? How long have new business
Efficiency Municipality
models been established with
demonstrated service delivery on this basis? Regulator
Impact
Are business models viable, do they take WSUP Programme Manager
Sustainability account of losses, and environmental and
social sustainability?
100
Output level. 2.2 Capacity developed and changed attitudes to serving low income areas form part of the core business strategies of service providers
Relevance Have service providers realigned their Water utility Staffing knowledge, skills, attitudes and
Effectiveness institutional focus to be pro-poor? SME Service Provider practices
Are service providers able to Municipality
Efficiency more effectively deliver services to urban
Regulator
Impact poor, including women and girls? To what
extent have service providers improved WSUP Programme Manager
Sustainability
their efficiency in comparison with their
previous work and with other service
delivery models?
Output level. 2.3 Political developments and social stability do not significantly undermine service providers’ commitment to scaling up effective pro-poor models at citywide
level
Relevance To what extent is the socio-economic- Water utility Key informant interviews
Effectiveness political context conducive to early pro- Municipality Newspaper articles search
poor service improvements? Regulator
Efficiency
WSUP Programme Manager
Impact
Sustainability
Outcome Increased capacity and institutional change – leading to 12.5m people with improved access to sustainable, affordable and viable WASH services
Relevance What results have changes had on service Water utility Alternative approaches from WSP/WOP/
Effectiveness providers & the way they have delivered SME Service Provider SUWASA/KFW/WSTF (Kenya
economic, social and health impacts? Municipality
Efficiency How many people have benefitted
Regulator
Impact indirectly & are there differences in
impact between females & males? WSUP Programme Manager
Sustainability Community/households
101

Outcome 3. Scale – City wide investment triggered. Pro-poor policies and institutional processes are adopted by local service providers and
authorities and supported by financially viable service improvements, triggering investment from a range of funding sources for the expansion of service
delivery.

Output level 3.1 Finance models designed to mobilise scale up finance are replicable and can be appropriately adapted to specific contexts and stakeholders
Relevance What are the wider changes that have WSUP Financing records What is WSUP’s USP relative to other
Effectiveness occurred in investment programmes and Water utility donors/agencies?
policy at the city level and how many SME Service Provider
Efficiency people have benefitted? To what extent
Municipality
Impact have new investments been made in
unserved or underserved areas where the Regulator
Sustainability WSUP Programme Manager
poor live?
Output level 3.2 Effective pro-poor targeting of low income components within IFI investment projects provides examples for further replication and scale up in
other countries and contexts
Relevance To what extent have city wide policies WSUP Financing records To what extent are IFIs benefiting from
Effectiveness prioritised pro-poor & have new Regulator WSUP approaches, or leading, or simply
investments been made on the basis of WSUP Programme Manager on parallel track?
Efficiency delivering service efficiencies particularly
IFI Country staff
Impact for the poor?
Sustainability
Output level 3.3 Pledged budget/investments for low income areas are released/delivered upon within reasonable timeframes
Relevance Timely release and spending of funds? WSUP Financing records
Effectiveness Water utility/municipality
Efficiency Regulator
Impact
Sustainability
Output level 3.4 A meaningful proportion of this money goes to on-site sanitation and associated services
Relevance WSUP Financing records
102
Effectiveness What proportion of funds are spent on Water utility/municipality Why the on-site sanitation focus when
Efficiency sanitation and hygiene promotion? Are pro- Regulator communities would often prefer (reduced
poor and gender attentive models and ways WSUP Programme Manager cost) sewerage?
Impact of working being accepted for city wide
Sustainability stakeholders and their business models?
Outcome Mobilising finance and making it more effective- US$220m finance mobilised and more effective: households, local private sector, public sector and IFIs
Relevance Is there the right balance between capital WSUP Financing records What is WSUP’s USP relative to other
Effectiveness grant/loan investment & revenue versus Water utility/municipality donors/agencies?
expenditure in areas where urban poor live Regulator
Efficiency & more widely across the city? Is there a
Impact city wide resource & waste management
plan?
Sustainability
Outcome 4. Influence: Sector influence worldwide Scalable, viable, effective and replicable approaches for pro-poor urban water and sanitation service
delivery are documented, backed by evidence, disseminated widely and influencing decision makers in service providers and the wider sector

Output level. Evidence. ACTIVITY 4.1 - Research and Analysis
Relevance Did the research areas break new Literature search of research in these areas Important to understand if these were
ground/was it the most relevant to Guy Norman for choice of areas recognised as the or some of the most
undertake? WEDC or International Rescue Committee (IRC) – important questions to answer/areas to
Netherlands (not involved in research programme) influence
Effectiveness To what extent have the research Traction and Scale studies in Maputo How effective is research in achieving
projects commissioned under this Scale: study in Bangladesh this? Evidence based programmes is
programme contributed to country-level evaluation of Zambia FSM increasingly stipulated
objectives?
103
Efficiency Were reports developed within a Documentation on drop box Cost analysis will not be undertaken.
reasonable timeframe and at least one Costs will be mapped and may be
peer reviewed? commented on.
Impact What has been the impact of the Guy Norman
research work? WSUP Country Programme Managers
Users or research work (to be identified)
Output level. Learning. ACTIVITY 4.2 - Documentation and Dissemination
Relevance Was there a plan for what need to be Guy Norman for choice of areas Focus on publications
documented and how were topics WEDC and WSP for other views on documentation
identified? What was useful, not useful needs
and what was not covered that could
have been?
Effectiveness Did documentation and dissemination 2015 WSUP publications survey.
provide new learning to organisations, Publication user satisfaction survey early in Year 2
was it widely used and changed the way Reports from network meetings
organisations worked? Has translation
been important? How many slots did
WSUP present in network meetings
compared to meetings attended and
what was their influence?
Efficiency Was delivery within expected Budget costs for publications Cost analysis will not be undertaken.
costs/time? What other channels of Guy Norman for choice of topics for publication Costs will be mapped and may be
dissemination beyond publications and Other user organisations e.g. CARE, WaterAid commented on.
workshops could have been used?
Impact What has been the impact of WSUP publications survey.
dissemination activities conducted under Guy Norman
this programme on external perceptions
of WSUP’s work?
Are WSUP’s current dissemination
channels conducive to achieving WSUP’s
influencing objectives?
What was the impact of the
Masterclasses
104
Output level. Training. ACTIVITY 4.3 – Training

Relevance How were the main audiences and the Guy Norman
learning processes chosen? WEDC??
Why were there parallel process Global Mozambique’s Eduardo Mondlane University
and Mozambique Course evaluation reports
What has WSUP learnt from its
programme going forward for next
business period?
Effectiveness Did training provide new learning? Guy Norman
How were participants able to apply this WEDC??
in their work? Mozambique’s Eduardo Mondlane University
Course evaluation reports
Efficiency What were the costs/time involved? Guy Norman Cost analysis will not be undertaken.
Was delivery within expected WEDC?? Costs will be mapped and may be
costs/time? Mozambique’s Eduardo Mondlane University commented on.
Could learning have been delivered in Course evaluation reports While Masterclass and MSC module have
other ways, e.g. unaccredited short different purposes and audiences it will
courses, apprenticeships, internet be useful to compare inputs and outputs.
learning programmes, artisan rather than
professional level.
Impact What has been the impact of learning Course evaluation reports While Masterclass and MSC module have
courses? different purposes and audiences it will
How does relative impact of Masters be useful to compare impact.
module compare with the
Masterclasses?
Outcome level. Influencing (as per the INTRAC proposal)
Relevance Do new models of service provision, Guy Norman
capacity building and investment WSUP Country programme staff
decision making influence or align with Government and utility staff in Kenya and
emerging global thinking? Mozambique
DFID
WSP?
105
Effectiveness To what extent have DFID, IFIs, UN and Guy Norman

service providers adapted their ways of WSUP Country programme staff
working and focused investments to be Government and utility staff in Kenya and
more pro-poor as a result of WSUP Mozambique
dissemination, research & training?
DFID
WSP?
Efficiency To what extent have global polices and Guy Norman
investment prioritised efficient service WSUP Country programme staff
provision for the poor? Government and utility staff in Kenya and
Mozambique
Impact To what extent have urban poor Guy Norman
populations benefitted from improved WSUP Country programme staff
urban WASH service provision? Government and utility staff in Kenya and
What evidence exists for the uptake of Mozambique
service delivery models implemented A Partner Performance Assessment (PPA) for DFAT
under this programme by other service (formerly AusAID) of the WSUP programme in
providers and WASH-implementing Maputo which DFAT has funded for four years
organisations?
Sustainability How have investment policies and Guy Norman

practices of other organisations WSUP Country programme staff
addressed social, economic and Government and utility staff in Kenya and
environmental sustainability? Mozambique
Evidence of influence in other countries DFID
where WSUP Advisory works.
WSP?
To what extent does the membership
model lever influence?
Evidence of influence with others WB,
ABD, UN agencies
106
Annex Eight – Household and Infrastructure surveys – note

WSUP have contracted six separate consultants (one in each country) to conduct:
 A Household survey of users (300 households across three defined survey areas*)
 An Infrastructure survey (in two or three defined survey areas)
 At least one FGD (in three survey areas)
Analysis of the data from the two surveys will contribute towards the overall assessment of the
programme. It will be used to further inform and refine the methodology, primarily in highlighting
areas to focus on in interviews with user groups and beneficiaries in the two country visits.
Both Household and Infrastructure surveys are being undertaken using the mobile data collection
platform Mwater, using the same questionnaires in each country. Review of the questionnaires
suggests they will provide data relevant to objective 1 against the criteria of relevance and
effectiveness of the WSUP interventions, and to a lesser extent, efficiency, impact and
sustainability. The inclusion of non-WSUP users and questions of around previous water and
sanitation arrangements provide a limited counterfactual.
The survey areas are geographically limited areas benefiting from a WSUP intervention – they
may be one single area or a group of non-contiguous areas. In each country, the consultants
must define three survey areas covering different types of intervention. They are selected on the
basis that two intervention types have been rated by WSUP as strong, and one is rated moderate
or weak. Only interventions with impacts at household are included (i.e. school-based
programmes are excluded).
Sampling for the Household survey is complex and multi-stage, and may vary from country to
country. For the Infrastructure survey, a random sample of infrastructures is stipulated within the
survey areas. The sample size (minimum 150 WSUP intervention users in each country
programme) may limit the power of the survey in observing differences between countries and
between intervention types etc. The evaluation team will review the final achieved samples and
establish whether and what statistical testing may be applicable to the sample designs. At this
stage, we do not propose linking the analysis of infrastructure and Household surveys at a micro-
geographic level (survey area or intervention catchment level), due to the sampling complexity
and limited sample sizes.
Following the country visits and the final semi structured interviews the team will come together
to highlight key findings and issues across the programme. To ensure quality each team member
will be responsible for drawing together the findings and analysis for the Outcome/s they are
responsible for. Inputs will be peer reviewed by team members as well as by INTRAC’s Quality
Assessment process (Clare Moberly and Dan James)152.
152 Section 7, INTRAC Proposal.
107
Annex Nine – Household and Infrastructure surveys analysis

This is presented as a separate document.
108
Annex Ten – Further data on achievements against Outcome 4

Outline of achievements at the output level
4.1 Research
 Target six research reports, achieved four, others being finalised (complete by March
2016)
 Projects started in 2013/2014. Four reports submitted in December 2015
 Project submissions vetted at concept submission stage
 Projects will be peer reviewed when these are submitted to academic journals
 Total expenditure £537,000
Summary overview
WSUP
Country Relates Self-
Research Approx.
Title research to review154
bodies cost153
conducted in Outcome likely value-
of-findings
BEHAVOIR - Using mass media ICDDR,B

behaviour change messaging to John Outcome
Bangladesh £220,000- excellent
improve communal toilets in Dhaka, Hopkins, 1
Bangladesh Stanford
TREATMENT - The Last Piece of the

Puzzle – Testing modular systems for Cranfield Outcome adequate
Ghana £38,000-
treatment of waste from container University 1
toilets
ENVIRONMENT – Impacts of slum

water supply Ghana and Leeds Outcome excellent
£84,000-
improvements on city-level water and Kenya University 2
energy consumption
SCALE - Triggering Increased City-

Ghana, Kenya
Level Public Finance for Pro-Poor The urban Outcome good
and £195,000-
Sanitation Improvements: institute 3
Mozambique
A Research Agenda
IMPACT project – Population

Density, Neighbourhood-Level
Outcome
Sanitation Access, and Health in Mozambique ?? ?? excellent
2&3
Urban Maputo 29 APRIL 2013
(externally funded MapSan)
153 PERL DFID04 budget October 2015 spreadsheet

109
Transforming Waste consortium Outcome

Zambia ?? ??
looking at FSM in Lusaka 1
4.2 Publication development & dissemination, in-country meetings and events,
international events
Publications
Practice note Topic brief Discussion Perspective programme

(PN) (TB) paper piece (PP) guides
DFID project
15 6 0 0 0
target
Produced 11 9 3 3 2
55% French
Translation 50% French
45% Portugs
Total produced
(including those
20 15 7 3 2
before the
project period)
003 pro poor
TB6 Tenure finance (W 3, G
tenancy (W 1, G 0), 006 Creating P02
PN 1, 2, 8, 12, Urban
5), TB11 Business Value Entrepreneurial
Gender/women 15 selected programme
Getting to Scale and ecosystems
mentioned survey (W 0, G guide (W 30, G
in Urban Development success stories
0) 14)
Sanitation (W 1, Impact in the (W 0, G 0)
G 0) WASH Sector -
(W 2, G 0)
 WSUP expects to publish seven additional publications before programme funding

officially ends on 31st March 2016 – all of which will be related to our research projects –
five short-form and two long-form – two relating to water and five to sanitation155.
 Practice Notes (short documents discussing specific solutions for pro-poor urban water
and sanitation, usually based on a particular WSUP experience), Topic Briefs (longer
documents that take a more analytical look at successful and scalable models being
demonstrated by WSUP and its partners in a range of countries), Discussion Papers (aim
to stimulate critical debate about the major challenges facing WASH service provision for
the urban poor, and to propose radical new approaches for overcoming these challenges)
and Perspectives (take a view of the sector as a whole, inviting discussion of broader
issues among all agencies involved in WASH interventions).
 Publications produced drawing upon material from following WSUP programmes
Publication by country (excluding programme guides)

Bangladesh Ghana Kenya Madagascar Mozambique Zambia
4 5 15 13 12 6
Publication by main topic (excluding programme guides)
Water Sanitation Hygiene Gender Finance Other
4 7 1 2 14 9
155 Email Sam Drabble 28th January 2016
110
 Total approx. Cost of publications around £60,000156
Type of publication Short form (PN, Long Form (TB, Programme

PP) DP) Guide
Number produced in programme period 14 12 2
Approximate amount spent on

£7,400 £34,400 £19,500
writing/editing/layout consultancy
Indicative average cost per publication (not

£529 £2,867 £9,750
including in-house labour costs)
Conferences (international)
 Target; none specified.
 26 conferences, 14 attended in 2015, a few in 2013 (exact numbers unknown).
 The major WASH conferences attended; World Water week (2014, 2015), WEDC
conferences (2013/14/15), Africasan (2015), Sacosan (2016), UNC conference (2013,
2014, 2015)
 17 conference outputs on sanitation, one on water, five on WASH, three gender related,
one on Behaviour Change.
WSUP
Confe- lead/joint Gender/ Outputs/
What Where When What
rence lead Women role
session
Assessing demand for pit
latrine emptying services in
Kanyama + Sanitation in
Nakuru, Just
WEDC schools: successes and 2013 NA san
Kenya posters
challenges for
sustainability. Experience
from Mozambique
Programmatic approach to
WASH develop sustainable Brisbane ppt +
Mar-14 NA (G3, W6) san
con sanitation services in Australia paper
Maputo, Mozambique
MHM in the school Kampala Note ppt + Hygiene
MHM Aug-14 NA
environment Uganda topic paper gender
ppt +
Gender mainstreaming: Pretoria Note
SWASSA NA facilitated gender
experience from Maputo South Africa topic
workshop
Improving pro-poor water
Pretoria
SWASSA services: experience from NA (G0, W0) ppt wat
South Africa
Maputo
Pretoria
SWASSA Designing sanitation tariffs NA ppt san
South Africa
Capacity, Drive, Incentives: University of ppt +
Engaging the private North discussio
UNC Oct-14 Yes san
sector with urban Carolina, n
sanitation USA panellist
156 Email Sam Drabble 5th January 2016
111
Tracking urban sanitation

services Improving faecal
sludge management Hanoi ppt +
WEDC Sep-14 NA (G1, W0) san
services through Vietnam paper
monitoring in Lusaka,
Zambia
Bringing toilets back to
Kumasi's compound
Hanoi ppt +
WEDC houses: landlord and Sep-14 NA san
Vietnam paper
tenant behaviours and
motivators
Financing sanitation: Dakar Facilitate
Africasan May-15 Yes san
achieving 0.5% of GDP Senegal d session
Financing sanitation: a
Dakar Facilitate
Africasan local government May-15 Yes (G0, W0) san
Senegal d session
perspective
Supporting Scale-Up of
Hanoi ppt +
FSM3 FSM Enterprises in Dhaka, Jan-15 NA (G0, W0) san
Vietnam paper
Bangladesh
ppt +
Kings Embedding gender
discussio
WASH inclusiveness in urban London Mar-15 NA gender
n
week WASH programme design
panellist
ppt +
Kings Working with institutions:
discussio
WASH financing and maintaining London Mar-15 NA WASH
n
week urban WASH
panellist
ppt +
Scaling
Institutional behaviour discussio
for Bangkok Mar-15 Yes (G0, W0) WASH
change n
impact
panellist
Facilitate
d session
Stockhol Sustainable City Sanitation
+
m world – from planning to Stockholm Aug-15 Yes san
discussio
water implementation
n
panellists
Stockhol Exploring urban sanitation

m world at the nexus of government Stockholm Aug-15 Yes (G0, W2) ppt san
water and enterprise
Stockhol
Public Finance for WASH: Facilitate
m world Stockholm Aug-15 Yes WASH
making it happen d session
water
ppt +
SUWAS Market-based sanitation Kampala
May-15 Yes facilitated San
A solutions Uganda
workshop
University of
UNC Research for urban North Facilitate
Oct-15 Yes San
2015 sanitation: where next Carolina, d session
USA
University of
Primary research around
UNC North Facilitate
public finance for WASH: Oct-15 Yes WASH
2015 Carolina, d session
ways forward
USA
Developing the market for ppt +
Loughboroug
WEDC local WASH businesses: discussio
h Jul-15 Yes san
2015 lessons from pit emptying n
UK
in Africa panellist
112
Sustainable Development
Loughboroug ppt +
WEDC Goals (SDGS) and their
h Jul-15 Yes facilitated WASH
2015 impact on the WASH
UK session
sector
Public Private Partnerships ppt +
SACOSA DhakaBangl
in FSM in the SAARC Jan-16 Yes facilitatin san
N 2016 adesh
Region g session
ppt +
SACOSA FSM - supporting the full Dhaka
Jan-16 Yes facilitatin san
N 2016 sanitation chain Bangladesh
g session
WASH
Sanitation sustainability Facilitatin
sustainab Amsterdam Jun-14 Yes san
tracking g session
ility forum
Conferences national
 In addition, National conferences were attended though not part of budget/log frame (this
is partial listing including responses from Zambia and Mozambique).
- Zambia - April 2013: Presentation on what WSUP is what we do + FSM business
model in the WATSAN quarterly meeting; (Commercial Utilities, Learning institutions
and industry
- Zambia 2015 WSUP attended and made presentation during the Urban Sanitation –
Professionalisation of emptying service Sustainable Sanitation and Hygiene for All –
Urban Program Manila, Philippines, 30 November -3 December, 2015. This was a
learning event.
- Mozambique in 2013: WSUP had three presentations related with: FSM, Standpipes
and Equity and inclusion, challenges in WASH
- Mozambique 2013: participation in the National Conference on Communication in
HIV where WSUP presented a poster about Inclusivity in WASH; attended an AIAS
meeting for discussion of the “Namialo and Monapo Sanitation Master Plan” where
WSUP sanitation models where presented;
- Mozambique WSUP presented to a workshop about “Reflections on the Education of
the Girls” organised by Ministry of Education and Save the Children; the WSUP
participant also moderated the workshop which had about 50 participants from
MINED, national and international civil society organisations, universities and school
councils.
- Mozambique WSUP was invited by MINED to share their experiences working in the
education sector, in a National Meeting with MINED partners; 55 people attended
(30W&25M).
- Mozambique WSUP presented WSUP sanitation work in a workshop under the
theme “the evolution of environmental impact assessment in Mozambique, sanitation
International Learning visits

(This is a partial listing, including responses from Zambia and Mozambique).
 Mozambique in 2013: A visit was organised to NCWSC Nairobi Informal Settlements
Department, to learn and share experiences related to LIC services and establishing a
LIC unit. There were six participants from AdeM (1M+1W), FIPAG (1M+1W), CRA (1M)
and CMM (1M).
113
 Mozambique In 2014: Related with UCLTS a learning visit was realised to Tana with four
people (2M & 2W), representing AIAS, WSUP, SCDS and CMM.
 Zambia – facilitated the learning visit for Zambian LWSC and Ministry of Local
Government and Housing during the WSUP PIW that was held in Maputo in 2013.
4.3 Masters module
 Target; 150: 75 f / 75 m.
 Achieved 87. Estimated to be 17 females but this is a guess from names where listed.
(Numbers will increase with addition of Zambia class – February 2016.)
 The WASH in urban low-income communities’ module was developed by WEDC under
contract.
 Five short Courses run in five WSUP country programmes, the course in Zambia was
programmed in February 2016 but will not be run.
 Course materials available on WSUP web site157
 All been run in conjunction with national Universities and with WEDC (except Bangladesh)
delivering the courses.
 Courses were spread over a 5 – 15 days’ period, equivalent to 5-8 days’ full time.
Evaluation scoring for Masters module delivered:

Bangladesh Ghana Madagascar Kenya Mozambique
No evaluations 33% very good overall ranking; overall ranking very No evaluation
included 47% good 47% very good good 33%, good scoring given
47% good 56%
seemed to be There was Mostly mid-level All of the students
pitched at more suggestion that it professionals that responded
mid-level could be aimed at attended agreed or strongly
professionals more senior agreed that the
people. module has helped
them develop a
greater
understanding of
the subject
 Costs are summarised: £50,000 contract with WEDC to develop training (paid by both
DFAT and DFID), cost for five courses has been £38,500 to deliver 39 days (some day’s
part time) of training in five countries split between DFID and DFAT.
Zambia Kenya Mozambique Ghana Madagascar Bangladesh

November November
planned for November 9 Sept 1 to 15, May 25 to
dates 16 to 25, 15 to 19,
Feb. 2016 to 13, 2015 2014 29, 2015
2015 2015
157 http://www.wsup.com/programme/resources/
114
WARREC
Universidade
University of (Jomo
location/ Eduardo
Zambia, Kenyatta KNUST IST-T ITN-BUET
Delivery by Mondlane
Lusaka University
Maputo
Nairobi)
classroom
4.5 15 4.5 7 4.5
days
days for field
0.5 1 0.5 1 0.5
trip
Numbers
20 14 15 13 25
attending
No of Govt 1
No of women 8?? 3?? 3?? 3??
cost of
implementati N/A 4,973.30 20,100.00 1,054.71 2,051.87 10,417
on (GBP)
Cost per
49.73 89.73 14.06 19.73 83
head/day
Masterclasses
 Target; none specified.

 The Masterclasses were introduced into this work stream after the proposal first reported
to the Board in May 2013 and first reported to DFID in our July 2013 Six-Monthly Report
under Output 4.3 (Professional training)
 Masterclasses have been run at Africa regional level in 2013 (Uganda) and 2015
(Zambia). Attendance; 60 people (2013) to 54 (2015) people. (no gender breakdown)

Cost about £60,000 for 2015 Zambia class158
 Internal planning meetings, functioning in a similar way to Masterclasses, with sessions
on a range of topics, sometimes with external speakers, for WSUP staff as part of annual
planning meeting to improve our internal learning. These were held annually, so four
within the lifespan of the DFID programme: Horsley, UK, in 2013; Antananarivo in 2014;
Accra and Kumasi in 2015; and Cambridge (forthcoming) in 2016.
 Exchange visits, not stated as an activity in budget and log frames.
Masterclass Focus Evaluation feedback Women/gender
Water and
Sanitation for Low Report with learnings
Income Consumers, Focus on themes and success but no evaluation of
Kampala, Uganda stories; LICs and FSM. value of the
2-5th December, Masterclass
2013
158 Email from Sam Drabble 11th January 2016
115
1) Structuring responsibility for low-

income consumers within water (w 0, G 0) session
Urban WASH
utilities, 2) Building effective city- Organisational
Masterclass,
wide Faecal Sludge Management Follow up action structures within utilities
Lusaka, Zambia.
(FSM) systems, 3) Developing provided for extending services
9th -12th February
sanitation surcharge models to to low-income
2015
support sanitation improvement in communities
urban low-income communities
116

Water Sanitation Urban Poor

Uploaded by

Copyright:

Available Formats

Water Sanitation Urban Poor

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Water Sanitation Urban Poor

Uploaded by

Copyright:

Available Formats

FINAL REPORT

Evaluation of Water &

Submitted by: February 2016

SDO Sub-district Officers

Verification of data sources

7. The numbers of direct programme beneficiaries reported by the programme is significantly

2.2 Purpose, Scope and Objectives of the evaluation

6 See Annex Two for the evaluation work plan

2.3 Evaluation methodology

Key considerations in the proposed methodology

2.4 Value for money (VfM)

10 Option 4 WSUP Business Case para 67, page 26.

 Finance mobilised by the programme.

Verification of M&E data

Country programme Visits

ability of a country programme to demonstrate a full span of programme interventions across

Data gathering tools

Household and Infrastructure surveys

3. Assessment and analysis by Outcome

3.1.1 Objectives from Logframe

3.1.2 Outline of Achievements

Numbers of Planned 2,076,600: 1,015,000: 1,335,000 1,654,000: 3,995,000)

343,560 541,573 676,115 1,521,041

166980 254,042 578,435 921,874

1,764,450 2,311,010 792,220 4,868,132

Direct 454,590 568,212 298,311 1,319,158

19 Source: Household surveys, FGD records, quarterly annual reporting, evaluations

Value for money – programme cost per

Total Programme Costs £32,552,000

Direct beneficiaries 8,630,205

Programme reporting provides a positive assessment of WSUP’s value for money at an

3.1.3 Impact on households using WSUP supported WASH services

Key findings – Household Survey

Water (all six countries)

21 Information received from WSUP 3/02/2016

3.1.4 High level comment on progress

People influenced by the programme – Indirect beneficiaries

inadequate handwashing facilities, and lack of mechanisms for onward

3.1.5 Key comment and analysis

33 WSUP Zambia APR 2014.

Gender inclusive approach

Disability inclusive approach

40 WSUP Urban Programming Guide; 2014. Crosscutting: Empowering Vulnerable Groups; pp 46 - 47

44 WSUP Bangladesh APR 2014.

Faecal Sludge Management (FSM)

49 These links are illustrated in an article in News Today on 5 February 2016:

The critical issue to address is therefore the sanitation value chain: 54

Maintenance budgets for toilets

Amount spent on water for current source, WSUP, non-WSUP users

All in US$ Bangladesh Ghana Kenya Madagascar Zambia Mozambique Average

Non-WSUP 0.281 1.426 1.109 0.404 1.693 1.422 1.056

WSUP 0.146 1.419 0.837 0.353 1.156 1.337 0.875

Amount spent on water for previous source, WSUP, non-WSUP users

All in US$ Bangladesh Ghana Kenya Madagascar Zambia Mozambique Average

Non-WSUP 0.289 1.131 1.467 1.134 - 1.005

WSUP 0.210 0.953 1.360 1.424 (0.934) 0.987

Non-Revenue Water (NRW)

67 Adams. Notes from Naivasha and Nakuru visit.

Figure 3 – Diagrammatic representation of linkage between direct and indirect beneficiaries