EVPA Measuring and Managing Impact

EUROPEAN VENTURE PHILANTHROPY ASSOCIATION
A PRACTICAL
GUIDE TO
MEASURING
AND MANAGING
IMPACT
EUROPEAN VENTURE PHILANTHROPY ASSOCIATION JUNE 2015

2 A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
Published by the European Venture Philanthropy Association

This edition June 2015
Copyright © 2015 EVPA
Email: info@evpa.eu.com
Website: www.evpa.eu.com
Creative Commons Attribution-Noncommercial-No Derivative Works 3.0.

You are free to share – to copy, distribute, display, and perform the work
– under the following conditions:
• Attribution: You must attribute the work as

A PRACTICAL GUIDE TO MEASURING AND MANAGING IMPACT
Copyright © 2015 EVPA.
• Non commercial: You may not use this work for commercial purposes.
• No Derivative Works: You may not alter, transform or build upon this work.
• For any reuse or distribution, you must make clear to others the licence
terms of this work.
Authors: Dr Lisa Hehenberger, Anna-Marie Harling, Peter Scholten

Design and typesetting: Pitch Black Graphic Design The Hague/Berlin
ISBN 9789082316087
# !
! ! "
!

"""
$ " !
This publication is supported under the EU Programme for Employment and Social
Solidarity – PROGRESS (2007–2013).
For more information see: http://ec.europa.eu/progress
or opinion of the European Commission.

JUNE 2015 3
A PRACTICAL
EVPA is grateful to:
Fondazione CRT for the support
of its Knowledge Centre
GUIDE TO
MEASURING EVPA is grateful to:
Acanthus Advisers, Adessium
AND MANAGING
Foundation, BMW Foundation
and Omidyar Network for their
structural support
IMPACT
DR. LISA HEHENBERGER, ANNA-MARIE HARLING AND PETER SCHOLTEN | JUNE 2015

JUNE 2015 5
TABLE OF CONTENTS
Preface 6 5.0 Step 4: Verifying & Valuing Impact 76

Executive Summary 14 5.1 What? 76
5.2 How to? 78
Part 1: 5.3 Practical tips 84
Introduction and Overview 28 5.4 Recommendations for managing
1.0 Introduction and Overview 29 impact 85
1.1 Background 30 5.5 Worked Example 85
1.2 How is social impact currently
measured by social investors and 6.0 Step 5: Monitoring & Reporting 86
venture philanthropists? 30 6.1 What? 86
1.3 Five-step framework 31 6.2 How to? 87
1.4 Methodology 33 6.3 Practical tips 93
34 6.4 Recommendations for managing
impact 93
Part 2: 6.5 Worked Example 94
The Impact Measurement Process 36
2.0 Step 1: Setting Objectives 37 7.0 Managing Impact 95
2.1 What? 37
2.2 How to? 37 8.0 Conclusions 101
2.3 Practical tips 46
2.4 Recommendations for managing Part 3:
impact 46 Case studies 102
2.5 Worked Example 46 9.1 Case Study:
Ferd Social Entrepreneurs investing
3.0 Step 2: Analysing Stakeholders 48 in The Scientist Factory 103
3.1 What? 48 9.2 Case Study:
3.2 How to? 49 Impetus Trust investing in Blue Sky 109
3.3 Practical tips 54 9.3 Case Study:
3.4 Recommendations for managing Oltre Venture investing in PerMicro 116
impact 54 9.4 Case Study:
3.5 Worked Example 55 Esmée Fairbairn Foundation investing
in Social Impact Partnership
4.0 Step 3: Measuring Results: (developed and run by Social Finance) 121
Outcome, Impact and Indicators 57 9.5 Case Study:
4.1 What? 57 Auridis investing in Papilio 126
4.2 How to? 60
4.3 Practical tips 70 Part 4:
4.4 Recommendations for managing Appendices 132
impact 70 10.0 Glossary of Terms 133
4.5 Worked Example 71 11.0 Sources 137
Preface

JUNE 2015 7
PREFACE
Introduction to the second edition from Lisa Hehenberger,

Research & Policy Director of EVPA
This is the second edition of A Practical Guide to Measuring and Managing Impact
the uptake of the Guide, the remaining challenges that practitioners face, the contribution
on impact measurement and management.
When we started developing the Guide in 2011, we responded to the need of EVPA members
for more clarity in terms of impact measurement. We had noticed that the problem was not
the lack of information, but rather the absence of guidance in how to make sense of the
information on impact measurement. Therefore, we engaged in a meta-analysis of almost
1,000 different approaches as included in resources such as the TRASI database1 curated
by the US-based “Foundation Center”. From these approaches, EVPA, informed by the
convening of an Expert Group of twenty-seven venture philanthropy and social invest-
ment practitioners, consultants, academics and representatives of other organisations
involved in impact measurement, selected the most commonly used approaches and then
commonalities between various approaches to come up with clear recommendations on

how to measure impact.
We discovered during the process that the most important aspect of impact measurement
is not the actual value or numbers you obtain from the exercise, but the integration of
an impact approach in the organisation so that impact becomes an integral part of the
entire management or investment process. By undertaking and learning from the process
of measuring impact, an organisation can work more effectively towards achieving societal
measuring to also managing impact.
Uptake of the Guide and recent developments

The EVPA guide has been well received, with over 2,000 copies downloaded since its
launch in April 2013 and more than 500 hard copies distributed. It has been translated
th
Industry Survey2,
outlined in the Guide.
1. http://trasi.foundationcenter.org/
2. Hehenberger, L., Boiardi, P.,
Gianoncelli, A., (2014), “European
Venture Philanthropy and social
investment 2013/2014 – The EVPA
Survey”, EVPA.
PREFACE
Step 1 90 % of VPOs that perform

86 each step of societal
Step 2 73 impact measurement
79
FYs 2011 and 2013
Step 3 90
88
Step 4 72
72
2013 n=92
Step 5 84 2011 n=57
72
0 20 40 60 80 100 numbers in %
outcomes and trying to assess the impact of its activities, pointing to the importance the
practice has for VP/SI practitioners, and their increased sophistication in the use of the
practice.
Outputs Outcomes Impact Objectives of social impact

measurement by %
84 87 of respondents
70
n=91
multiple choice
numbers in %
to claim. Many challenges remain for both funders and investees who still consider impact
measurement a complex and technical practice. However, we do believe that the Guide and
the dissemination and policy work around it have contributed to raising awareness for the
topic of impact measurement and management in our sector.
EVPA and its members are being recognised as important actors in the practice of impact
on impact measurement, and we have participated in and contrib-

3
uted to the report produced by the Working Group on Impact Measurement of the taskforce
4
3. http://europa.eu/rapid/
on social impact investment established by the G8. press-release_IP-14-696_
en.htm?locale=EN
4. http://www.
socialimpactinvestment.org/
Commission set up it Sub-group on Impact Measurement to advise the Commission on reports/Measuring%20Impact%20
WG%20paper%20FINAL.pdf

JUNE 2015 9
PREFACE
of the Guide during one of the meetings. The experts in the Sub-group agreed that the
European Standard of Impact Measurement should be set at the process level (adopting
social sector level and they have to be chosen in accordance with relevance to the social
organisation measuring impact). The European Standard was formally adopted in June
20145 and the report can be downloaded for free6.
Another important development for our sector has been the study conducted by the
Working Group on Impact Measurement (WGIM) of the Taskforce on Social Impact Invest-
ment established by the G8. The WGIM was composed of experts from the G8 countries
(later to be G7) as well as representatives of the European Commission and OECD. WGIM
data collection and data analysis.
Considering the uptake of the Guide and its contribution to European and Global standards
currently stands today. Core principles that have come out of these work streams, and that
will guide our work on impact measurement going forward, are as follows:
• Impact measurement has to be relevant to the organisation measuring so that it becomes

part of their management system and helps them improve their work to achieve greater
impact.
• Impact measurement also needs to be proportionate to the organisation at hand, keeping

in mind that it is a means towards an end, not an end in itself.
Attempts are being made at standardising social impact measurement indicators, across
Several databases (e.g. IRIS, Global Value Exchange) exist that have collected key perfor-
mance indicators commonly used. Reporting standards are already being developed by
social investors in cooperation with investees in many parts of Europe (e.g. Social Reporting
Standard in Germany). New and more sophisticated tools (e.g. Sinzer, PULSE) have been
created and are being developed to support practitioners in measuring and managing
impact.
Next steps for EVPA
measurement in 2007, EVPA members are still in need of additional training and guidance
on impact measurement. The lack of benchmarking measures, the absence of standards in 5. http://europa.eu/rapid/
terms of evidence needed, and the lack of data of the impact of funders on their investees press-release_IP-14-696_
en.htm?locale=EN
are a few frequently mentioned issues. We particularly see the need to make the research
6. http://ec.europa.eu/social/main.
even more hands-on, practical and relevant with concrete case studies that run through
PREFACE
an evolving document that incorporates new developments and provides up-to-date and
practical guidance. Therefore, we have decided to invest further in research on the topic.
Concretely, the next steps include conducting and writing a number of in-depth case studies
to be published in a separate report on how to measure and manage impact in practice, to
be further developed as training material.7 This research will provide VP/SI practitioners
with practical real-life and in-depth cases of how impact measurement can be performed.
measuring impact. We also aim to include, where relevant, any new, upcoming method-
ologies in impact measurement and emergent issues (e.g. how to evaluate outcomes, issue
of proportionality, levels of evidence needed and use of control groups, measurement
standards that allow comparability, etc.).
Other plans include developing a micro-site on impact measurement, to develop a

community of practice, making research on impact measurement a process of continuous
learning that builds on existing knowledge and on the experiences of VP/SI organisa-
tions. The micro-site will help EVPA collect knowledge and best practices around the topic
and make it available for practitioners in a user-friendly way. Through the community of
practice we will also collect experiences and practical cases to help EPVA and its members
upgrade and revise the learnings.
It is our aim that the research on impact measurement and management will allow us to
provide even more practical guidance that will facilitate the work of VP/SI practitioners
actor in terms of setting standards in the VP and Social Investment industry, and thus
reduce problems of multiple standards in impact measurement that increase the work
of both investor and investee. And we should never lose track of the overall purpose of
impact measurement; to help both funders and investees work towards greater impact –
while being relevant and proportionate.
Lisa Hehenberger
Research and Policy Director, EVPA
7. A shortened and simplified

version of the case studies will
then be used for training purposes
at a later stage.

JUNE 2015 11
PREFACE
Introduction from Daniela Barone Soares,

Chief Executive of Impetus Trust
you it helps them prove that what they do makes an impact, gives funders reassurance that
their money is well-spent, and provides the stories and case studies they need for further
programmes, and inform their decision-making.
Venture Philanthropy Organisations (VPOs), like Impetus, work for a social sector where
the work of impact measurement is driven by the need to extract maximum value from our
the imprint of past successes and failures.
the data on their own activities to make informed resource allocation decisions, or build an
This Practical Guide to Measuring and Managing Impact is a timely resource with a wealth
of much-needed information for Venture Philanthropy Organisations (VPOs), and the SPOs
Centre, because sharing best practice is an essential part of the development of our sector.
VPOs are in a strong position to take impact measurement and management practice to the
next level. Collecting relevant data, and crucially putting it to good use, is a key challenge
for SPOs. Our unrestricted funding backs the unglamorous, but essential, work of building
At Impetus, we are committed to building this capacity in the organisations we support.

We build relationships of trust that allow us to push our organisations to achieve more than
they might have thought possible. Our deep understanding of the sector is complemented
by the private sector expertise we bring in, and our long-term engagement means that
collecting the data, and putting in place the virtuous circle that connects performance data
to performance improvement.
PREFACE
And of course we are an SPO too! We are acutely aware that we have a duty to expend the
resources entrusted to us for maximum impact, and that we only identify impact through
innovation. We believe SPOs should be intelligent risk takers, with venture philanthropy
providing the ultimate risk capital. Data allows you to know when you are taking a risk,
as well as whether it pays off. And when the “pay off” can mean changing the lives of
thousands, or even millions, we all need to know about it.
Daniela Barone Soares

Chief Executive, Impetus Trust

JUNE 2015 13
PREFACE
Expert Group composition
EVPA is grateful for the contribution of the following Expert

Group to the development of this manual.
edition of the report was published in April 2013.
Name Organisation
Brad Presner Acumen Fund
Ken Ito Asian Venture Philanthropy Network
Claudia Leissner Auridis
Bettina Windau Bertelsmann Stiftung
Richard Kennedy CAN Breakthrough
Camilla Backström Charity Rating / NAYA AB
Nalini Tarakeshwar CIFF
Uli Grabenwarter EIF
Iana Petkova / Esmée Fairbairn Foundation
Gina Crane
Emeline Stievenart ESSEC Business School
European Foundation Centre
Øyvind Sandvold FERD Social Entrepreneurs
Fabrizio Ferraro IESE Business School
Anne Holm Rannaleet IKARE / EVPA Board
Meredith Niles Impetus Trust
Filipe Santos INSEAD
Sarah Gelfand IRIS / GIIN
Thomas Kagerer LGT Venture Philanthropy
Eva Varga NESsT
Lorenzo Allevi Oltre Venture
Emma Lane Spollen One Foundation
Pieter Oostlander Shaerpa
Alex Nicholls Skoll Centre for Social
Entrepreneurship
Social Evaluator
Claire Coulier Social Impact Analyst Association
Jeremy Nicholls SROI Network
Sophie Robin Stone Soup / ESADE Business School
Executive summary

JUNE 2015 15
EXECUTIVE SUMMARY
(“VPO/SI”), and more generally at impact investors, foundations and any other funders
interested in generating a positive impact on society. Throughout the document, we use
is to create a roadmap or guidebook to help VPO/SIs navigate through the current maze of
existing methodologies, databases, tools and metrics on social impact measurement. There-
fore, we do not take a stand to recommend a particular tool, but rather have attempted to
distil best practice from the various ways of measuring and managing social impact. The
and managing impact. We would like the manual to become a working document that
evolves with new versions over time as our industry knowledge develops.
The manual should be useful both for beginners in impact measurement, who are consid-
ering how to get started, and for more advanced investors who are struggling with how to
better integrate an impact focus into everyday investment management decisions. Within
the VPO/SI, the person (or team) assigned to measure impact will be the natural reader/
user of the manual, but we also recommend executive directors, boards of directors and
investment managers to use the manual as a reference for key decisions on topics such as
resource allocation, deal selection and investment management. The manual uses plenty
by the impact measurement initiative (IMI) Expert Group members. The manual does not
Our starting point has been to devise a process of Impact measurement for a VPO/SI wanting
to measure the impact of their investment8 in a Social Purpose Organisation (“SPO”). The
(level of SPO) and how the VPO/SI itself contributes to that impact (level of VPO/SI).
This process is the “how to” of impact measurement and is often what is most needed by
venture philanthropy organisations and social investors in general to get started. Analysis
of existing methodologies for impact measurement and the experience of working together
with VPO/SIs showed that most methods and tools to measure impact share a general
framework.
8. We use investment through-

out as including the range of
grants, loans and equity.

EXECUTIVE SUMMARY
ring 1. The 5 steps of social

ito ing Ob Sett
on eport jec impact measurement
R t
an M
in e s
d
g
5.
iv
Managing
pact
&
Impact
Stak
4. uing Im
2. Ana
ifying
eh old g
Ver
lys
l
Va
er
in
s
Results Source: EVPA

3. Measuring
The steps are presented in sequential order and we recommend that VPO/SIs go through
the steps in this order. However within the process it is possible to go back to steps and
revise them as you gain more information and more familiarity with the process. Some
implementing them in practice.
The goal of impact measurement is to manage and control the process of creating social
impact in order to maximise or optimise it (relative to costs). Managing impact occurs
continuously and is facilitated by integrating impact measurement in the investment
management process. It is important to identify what may need to change within the
investment management process so that you are able to maximise social impact. That is
why Managing Impact is the core of the impact measurement process. For each step in
the process, one should consider how this relates to the everyday work of funding and
building stronger social purpose organisations.
clearly sets out the differences between inputs, outputs, outcome and impacts.

JUNE 2015 17
EXECUTIVE SUMMARY
The Impact Value Chain
SPO’s Planned Work SPO’s Intended Results
1. Inputs 2. Activities 3. Outputs 4. Outcomes 5. Impact
Resources (capital, Concrete actions of Tangible products Changes resulting Outcomes adjusted
human) invested in the SPO from the activity from the activity for what would have
the activity happened anyway,
actions of others &
for unintended
consequences
€, number of people Development & Number of people Effects on target Attribution to changes
etc. implementation of reached, items sold, population e.g. in outcome. Take
programs, building etc. increased access to account of alternative
new infrastructure education programs e.g. open air
etc. classes
€50k invested, 5 people Land bought, school New school built with Students with Students with access
working on project designed & built 32 places increased access to to education not
education: 8 including those with
alternatives: 2
Inputs: all resources, whether capital or human, invested in the activities of the
organisation.
Activities: the concrete actions, tasks and work carried out by the organisation to
Outputs:
activities.
Outcomes:
Social
Impact: outcomes.
(i) what would have happened anyway (“deadweight”); (ii) the action of others (“attribution”);
(iii) how far the outcome of the initial intervention is likely to be reduced over time (“drop
off”); (iv) the extent to which the original situation was displaced elsewhere or outcomes
displaced other potential positive outcomes (“displacement”); and for unintended conse-
quences (which could be negative or positive).
EXECUTIVE SUMMARY
impact numbers.
In what follows, we provide a quick glance at the recommended impact measurement

process as detailed in the manual.
Step 1: Setting Objectives
to be considered at both the level of the VPO/SI and the investee SPO. Often VPO/SIs do
relevant and what is not when faced with scarce resources.
(i) Level of the VPO/SI.
questions upfront:
a. What is your motivation for measuring social impact?
There are many different purposes for using impact measurement and these could each
imply different target audiences and outlook.
b. What resources can you dedicate to impact measurement?
c. What type of SPOs are you working with?

The maturity i.e. the stage of development of the SPO will potentially limit the type of
information that the SPO can provide to you.
d. What level of rigour do you require in your impact analysis?
Depending on how accountable you expect your investees to be, you can increase the
rigour of your analysis and thereby reduce the risk of any impact claims made.
e. What is your time frame for measuring impact?
The time period over which you measure impact should be determined by the most
important outcomes and estimated length of time required to achieve them. But in
practice there may be internal or external pressures to invest for a certain period of time.
general conclusions about the impact of the SPO.

JUNE 2015 19
EXECUTIVE SUMMARY
a. What is the overarching social problem or issue that the VPO/SI is trying to solve?
but a clearly articulated response is necessary to be able to choose investments that can
contribute to solving the social issue that the VPO/SI is addressing.
c. What are the expected outcomes?

The VPO/SI should evaluate the expected outcome of its investment in the SPO, i.e. the
expected outcome of the SPO and how the VPO/SI expects to contribute to achieving
that outcome. It is important to consider potential unintended consequences of the
(ii) Level of the SPO.

At a minimum you should answer these questions about the SPO:
a. What is the social problem or issue that the SPO is trying to solve?
The response should include information about the nature and magnitude of the
problem or opportunity; which populations are affected; whether the matter is changing
or evolving as well as in what way it is changing or evolving.
b. What activities are the SPO undertaking to solve the social problem or issue?
This should include a description of exactly what the SPO is doing to try to effect a change.
c. What resources or inputs (as per the impact value chain) does the SPO have and need
to undertake its activities?
This should include the time, talent, technology, equipment, information and other assets
SPO to solve the issue.

d. What are the expected outcomes?
This should include what the SPO must achieve to be considered successful and will form
the basis of the milestones against which the SPO will be measured. It is also important
Recommendations for managing impact:

• A VPO/SI must formulate its overarching social problem or issue so as to choose
investments in SPOs that can contribute to solving that social issue.
• Understanding the current and expected social impact of an SPO early in the decision-
making process is extremely valuable: it creates a common understanding of the
impact of an organisation; allows the VPO/SI and SPO to “speak the same language”;
and facilitates assessment of achievement of impact at later stages. A VPO/SI should
convince the SPO of the value of impact measurement, provide assistance where
• Decisions have to be made about the amount of time and resources that a SPO should
dedicate to impact measurement.
EXECUTIVE SUMMARY
Step 2: Analysing Stakeholders
as, “Any party effecting and/or affected by the activities of the organisation.”
This is an important step because the VPO/SI needs:
• To understand the expectations of the stakeholders, their contribution to and the potential
• The co-operation of the main stakeholders in the impact measurement process.
Applying to both the VPO/SI and SPO level, there are two aspects to stakeholder analysis:
(i) ; which includes stakeholder mapping (direct and indirect
-
ality, accountability and relevancy) and analysis of stakeholder expectations.
(ii) Stakeholder engagement; which includes communicating with the selected stakeholders
and is vital to be able to understand their expectations and, later in the process, verify
if their expectations have been met. This is described in more detail in Step 4.

• A VPO/SI must get the buy-in of key stakeholders (donors/investors, staff/human
managed and their contributions are aligned.

•
• When a VPO/SI makes an investment in a SPO, stakeholder analysis should be

included during the due diligence phase. As the investment period proceeds, it is
advisable to regularly get back to these stakeholders to verify that their expectations
are being met (more details on how to do this in Step 4).
• Consider upfront when would be the appropriate time to revisit stakeholder analysis
streams, new business lines, policy changes).
Step 3: Measuring Results: Outcomes, Impact and Indicators

This step again occurs at two levels:
•
work on the SPO.

•
impact and indicators.
the work of the VPO/SI on the SPO. Outputs are directly related to the activities of the
organisation i.e. what is done to try and make a change in the base case, hence these are

JUNE 2015 21
EXECUTIVE SUMMARY
generally easier to measure. Outcomes and impacts are related to the expected and unex-
pected effects of the activities of the organisation, hence they are outside the scope of the
Internal vs external focus:

Inputs Internal Focus the use of outputs,
outcomes or impacts
Organisation
Outputs
Base Case Changed Case
Outcomes
External Focus
Impacts
Source: EVPA
Indicators are used to show progress towards or away from outputs or outcomes. If output
indicators are required these should be sourced as much as possible from public databases
such as IRIS, Global Value Exchange or other databases. If these output indicators point in
the same direction as the outcome you are targeting or if there exists independent research
as outcome indicators. If not we recommend the following process to select outcome indi-
cators:
(i) as change statements, target statements or benchmark statements.

(ii) Select outcomes: you may have a list of outcome statements but you must select those
outcomes that are most important, material, useful and feasible (in achievement not in
measurement).
(iii) Select indicators i.e. identify two to three factors that provide measurable evidence
for a sub-optimal situation. There are four aspects to a good indicator:
a. Indicators should generally be aligned with the purpose of the organisation, although
b. Indicators should be “SMART”.
comparable with those used by others.

d. More than one indicator should be used, with preference for two or three.
Impact itself is a technical and often academic discussion including concepts such as drop
off, displacement, deadweight and attribution. The rationale for this guide is to remove the
complexity around the issue and provide practical guidance.
EXECUTIVE SUMMARY
The ability of an organisation to measure impact will depend on the sector and geography
in which it is operating. The propensity of European governments to move towards pay for
performance contracts means the measurement of impact is becoming more important for
those organisations active in these areas. However, for many organisations, access to inde-
pendent statistics and the creation of control groups in order to assess displacement, dead-
weight, drop off and attribution is not possible due to the expense and specialist skill-set
to carry them out. In these cases we encourage organisations to measure impact by calcu-
lating outcomes and acknowledging those factors that may mean that the outcomes are not
equal to the impact i.e. can increase or decrease impact. In some cases it may be possible to
think about some evidence as to what a control group may look like and could be used for
comparison purposes, for example based on research of comparable situations elsewhere.

•
important to assess the impact of the work of the VPO/SI on the SPO.
•
• The VPO/SI should ask the SPO to focus on those indicators that are directly related
additional indicators required for the VPO/SI to satisfy its impact measurement needs
should be collected by the VPO/SI.
• The expected outputs, outcome and impact, and the corresponding indicators should
SPO.
• Clarify at the beginning of the relationship (i.e. during due diligence and within deal
structuring) who is responsible for measuring what. This can evolve over time and
should be reviewed on an annual basis.
Step 4: Verifying & Valuing Impact

In this step, we need to verify whether the claim we make on having positive social impact
is true, and if so, to what extent (i.e. to what value). The responses to these questions will
allow us to re
feedback loop in the impact measurement process. This step also helps identify the
impacts with the highest social value, which can help an organisation focus their resources
towards those initiatives that create most impact on society.
Again, this step needs to occur at two levels: both at the level of the VPO/SI as well as at
the level of the SPO.

JUNE 2015 23
EXECUTIVE SUMMARY
valued. It may also be necessary for VPO/SIs to verify at regular intervals that the expecta-
tions of other stakeholders (donors/investors and human resources) are met so that correc-
tive actions can be undertaken if necessary.
At the level of the SPO, it is important to verify whether the outcomes make sense for the
stakeholder i.e. if the outcomes were realised during the timeframe and in the quantities
expected.
Verifying impact can be done via:

• Desk research:
be triangulated with other data (external research reports, databases, government statis-
tics etc.);
• Competitive analysis: comparing the results of the SPO with other comparable organisa-
tions in terms of similar issues, geographies and populations targeted;
• Interviews / focus groups: asking neutral questions to a representative sample of your
key stakeholders. This format can be particularly useful when the VPO/SI is assessing
neutral party conduct these interviews so as to ensure SPOs are comfortable providing
the most truthful responses).
The next step is to understand if the outcome was important i.e. of value to the stakeholder.
Numerous techniques and methodologies exist for measuring value created. We have
chosen not to list all the possible techniques preferring instead to cite certain useful refer-
• Qualitative: storytelling, client satisfaction surveys, participatory impact assessment

groups, progress out of poverty index.
• Quantitative (monetisation): techniques for valuing e.g. perceived value / revealed pref-
Whether you select a quantitative or qualitative technique or a combination of both for
example, often governments tend to prefer quantitative approaches whereas the general
public may prefer qualitative methods.
EXECUTIVE SUMMARY

• Perform this step at the beginning of an investment (as part of the due diligence), at
least once during the investment period (to check that the impact is achieved and
valued) and again at the time of exit (as a way to check that the desired impact has
been achieved and makes sense).
• Make clear assignments between the SPO and VPO/SI about who is responsible for
which parts of the verifying and valuing process.
• Unless you verify whether you have created value through your support of the SPO,
you cannot credibly make that statement.
• VPO/SIs should use independent studies to assess the value they provide to the
SPOs as directly questioning investees may be a delicate matter, resulting in them not
always providing truthful answers.
• VPO/SIs should verify at regular intervals that the expectations of other stake-
holders (donors/investors and human resources) are met so that corrective action
can be undertaken if necessary.
Step 5: Monitoring & Reporting
through the indicators set in the third step; and reporting – transforming data into present-
able formats that are relevant for key stakeholders. Monitoring and reporting are iterative
processes that go hand in hand because what is monitoring to one stakeholder is reporting
to another e.g. when a VPO/SI is monitoring the progress of an investee SPO, that SPO is
reporting relevant data to the VPO/SI. As in the other steps we must consider the process
at two levels: the VPO/SI and SPO.
(i) Monitoring
make sense to the key stakeholders, they need to start collecting data in a systematic way.
In practice, the type of system may be considered upfront but we urge organisations to
go through the impact measurement process at least theoretically prior to setting up the
system to understand the type of information that needs collection.
The VPO/SI should be collecting and analysing data on:

•
•
•
There is also a need to evaluate if the SPO is effectively monitoring its activities and
outcomes e.g. are the selected indicators appropriate (providing a balanced picture of the
situation and picking up potentially positive and negative aspects) and if the VPO/SI has a
role to play in improving the impact measurement practices of the investee

JUNE 2015 25
EXECUTIVE SUMMARY
The SPO needs to evaluate the outcomes or impacts that are being achieved through the
activities of its organisation and the practical lessons that can be learned from the results.
With this information it is then possible to decide what actions are needed to increase
impact.
(ii) Reporting
Once the data has been collected and analysed, an organisation needs to consider how
to present this information. The purpose of reporting affects the information that should
be included. Depending whether the focus is on an internal or an external audience, the
various stakeholders may require different types of reports. The stakeholder analysis
conducted in Step 2 should guide the development of reporting, considering the stake-
One of the challenges of the social sector is that each SPO needs to report in different ways
to each funder. Some initiatives (e.g. Social Reporting Standard) are trying to overcome this

• To remove a reliance on and/or culture of “gut feeling”, VPO/SIs should work with
the SPO to develop an impact monitoring system that can be integrated into the
management processes of the organisation.
•
requirements – if not, a VPO/SI should be prepared to contribute to improving it
to the SPO as a management tool.

•
negotiation with the investor in order to decide how costs should and/or could be split.
• When working with very early stage SPOs and helping them develop business plans,
integrate requirements on impact measurement at this stage.
• Agree on reporting requirements upfront with SPO and co-investors to eliminate the
burden of multiple reporting on the SPO.
• Manage expectations about frequency and level of detail for reporting, and the way
what frequency).
EXECUTIVE SUMMARY
Managing Impact
better manage the impact generated by its investments. To manage impact, the VPO/SI
-
tive actions if the overall results deviate from expectations. It will also have become clear
that impact measurement is very closely aligned to your investment management process.
Given most VPO/SIs are aiming to maximise impact, the corrective actions taken may
apply as much to the investment management process as to impact measurement itself.
Managing impact in the investment process

Investment process
Investment
strategy Deal Due diligence Deal Investment
Exit
screening (detailed screening) structuring management
Decide on the Assess whether invest- Dig deeper into Map outputs, out- Regularly assess Perform thorough
overarching social ment opportunity fits questions asked in comes and impacts impact results against analysis of impact
impact objectives with VPO/SI strategy setting objectives. and decide on key key indicators. results against
of the VPO/SI – by asking questions indicators against objectives –
these will guide the detailed in setting Perform stakeholder which progress will Verify and value verifying
investment process. objectives. analysis. be measured. reported results at and valuing
regular intervals. reported
Verify and value Decide on monito- results.
expected results. ring and reporting Revise indicators
content and frequency if significant changes
and assign responsi- are made in the
bilities. business and impact
model.

JUNE 2015 27
PART 1:
Introduction
and Overview

JUNE 2015 29
INTRODUCTION AND OVERVIEW
1.0 Introduction and Overview
(“VPO/SI”), and more generally at impact investors, foundations and any other funders
interested in generating a positive impact on society. Throughout the document, we use the
term “VPO/SI” to refer to such social sector funders.
impact, providing practical tips and recommendations for how it works in real-life situations.
For that purpose, the manual is a roadmap or guidebook to help VPO/SIs navigate through
the current maze of existing methodologies, databases, tools and metrics on social impact
measurement. The manual does not recommend a particular tool, but rather attempts to distil
best practice from the various ways of measuring and managing social impact. The manual
should be useful both for beginners in impact measurement, who are considering how to get
started, and for more advanced investors who are struggling with how to better integrate an
impact focus into everyday investment management decisions. The manual does not consider
trigger a movement towards best practice when it comes to measuring and managing impact.
We would like the manual to become a working document that evolves with new versions
over time as our industry knowledge develops.
how the VPO/SI itself contributes to that impact. It focuses on devising a process of impact
measurement for a VPO/SI evaluating the impact of their investment in a SPO. This process
is the “how to” of impact measurement and is often what is most needed by VPO/SIs to get
started. The ultimate goal is for impact to become an integral part of the investment manage-
ment process. Within the VPO/SI, the person (or team) assigned to measure impact will be
the natural reader/user of the manual, but we also recommend executive directors, boards of
directors and investment managers use the manual as a reference for key decisions on topics
such as resource allocation, deal selection and investment management.
In order to ensure the inclusion of the opinions and experiences of various stakeholders, EVPA
convened an Expert Group consisting of twenty-seven venture philanthropy practitioners,
consultants, academics and representatives of other organisations involved in impact measure-
-
siastically contributed their time and knowledge to the development of this document. The
members of the Expert Group are listed in the preface and we are extremely grateful to them.
that were developed by the impact measurement initiative (IMI) Expert Group members. In
this version of the manual, we also include the feedback received during the workshop we
organised on the topic with 80 participants and individual feedback collected during a consul-
impact that attempts to integrate the elements of impact measurement into the investment
recommendations and practical examples. Five concrete and detailed case studies are
provided to further show how real VPO/SIs are dealing with impact measurement. These
addressing the challenges they face in measuring impact. Finally, the document provides a
glossary and additional resources.
1.1 Background
Venture philanthropy (VP) works to build stronger investee organisations with a social
to increase their social impact. Although we use the word social we include impacts that
maybe social, environmental or cultural. The venture philanthropy approach includes both
the use of social investment (equity and debt instruments) and grants. The key character-
istics of venture philanthropy include high engagement, organisational capacity-building,
and performance measurement.
An integral part of the VP approach is the measurement of performance; placing emphasis
accountability and transparency. The focus of this manual is social impact measurement.
1.2 How is social impact currently measured by social investors and venture
philanthropists?
The rationale for undertaking this impact measurement initiative was inspired by the
out-come of a workshop on impact measurement organised by EVPA in June 2011, and
the results of the 2011 EVPA Survey of European VPO/SIs, collecting data on 50 VPO/SIs
based in Europe with investments in Europe and abroad. The general opinion that came
out of the workshop was that there was a strong need for further direction on how to
approach impact measurement.
The second annual EVPA survey of Venture Philanthropy and Social Investment in Europe9,
released on 1st March 2013, collecting data on 61 VPO/SIs also reinforced the importance
of social impact measurement.
The key highlights of the survey with respect to impact measurement were as follows:
• There is increased attention to measuring social impact: The focus on social impact measure-
ment increased, with 90% of respondents measuring social impact on at least an annual
9. Hehenberger, L.; Harling,
basis during the investment period. Although impact measurement still occurs less
A., (2013), “European Venture
Philanthropy and Social Investment
2011/2012”, EVPA.

JUNE 2015 31
• VPO/SIs still focus on outputs more than outcomes or impact:
Nevertheless we saw an increase in the percentage of VPO/SIs attempting to measure

changes in outcome or impact.
• Increase in budget assigned to impact measurement:
a median spend of €15k.

• Lack of standardisation indicates a high degree of fragmentation in the use of impact measurement
tools and systems:
they were not using a standardised tool to measure social impact. Among those that did
use such a tool, the most frequently mentioned were Social Evaluator and SROI, although
a quarter of people did say they were using IRIS indicators or theory of change. Interest-
ingly, when asked whether they used one of the steps of the 5-step process developed
herein, between 70-90% of respondents used each of the steps.
• Impact measurement is not fully integrated into the decision-making process: 53% never or only
sometimes take the social performance into account before releasing new funds.
• Impact measurement does not inform employee compensation: Only 12% of the VPO/SIs
include social performance in the compensation schemes for their own staff.
The outcome of the workshop and the results of the EVPA Industry Survey 2011/2012 rein-
measurement.
1.3 Five-step framework

Analysis of existing resources on impact measurement and the experience of working
with VPO/SIs showed that most methods and tools to measure impact share a general
framework. This general framework was the starting point for the discussions on impact
measurement.
1. Setting Objectives: setting the scope of the impact analysis (why and for whom), the level
(portfolio of social investments/individual social enterprise) and what the desired social
• Level of investee (social issue to be solved, inputs/activities, expected outcomes)

2. Analysing Stakeholders: ranking the multitude of potential stakeholders in order of

priority, weighing their contribution to the completeness of the analysis against the
resources required, and analysing their inputs (if any), activities and potential outputs.
• Level of VPO/SI (employees, board of directors, investors / donors)
•
3. Measuring Results – Outcome, Impact and Indicators: measuring the output, outcome and
impact10 that derive from your activity for the key stakeholders, and understanding how
•
portfolio level indicators)
•
the SPO)
4. Verifying & Valuing Impact:
it indeed was valued by the key stakeholders – considering quantitative and/or
qualitative methods (by calculating the social value of an investment or otherwise) and
comparing the results of the work against relevant benchmarks.
•
investee etc.)
• Level of investee (verifying and valuing impact for key stakeholders)
5. Monitoring & Reporting: collecting data and devising a system to store and manage the
data as well as integrating this information into overall operations and reporting the
data to relevant stakeholders.
• Level of VPO/SI (what systems are required to collect, store and manage data, reporting
formats)
• Level of investee (collection, management and reporting requirements for the SPO)
The manual presents the steps in a sequential order and we recommend that VPO/SIs go
through the steps in this order. However within the process it is possible to go back to steps
and revise them as you gain more information and more familiarity with the process. Some
certain stakeholders on a frequent basis, therefore in practice you may need to gain the
information required for Steps 2 and 4 at the same time.
-
tions for the investment management process. Given VPO/SIs are interested in maximising
impact it is important to identify what may need to change within the investment manage-
ment process so you are indeed able to maximise impact. Within this manual we call this
managing impact. For each step in the process, the VPO/SI should consider how it relates to
the everyday work of funding and building stronger social purpose organisations.
10. The definition of these terms
are explored in section 1.5.

JUNE 2015 33
The 5 steps of social

ring 1. impact measurement
n ito rting Ob Sett
o epo jec
R t
an M
in e s
d
g
5.
iv
Managing
Ver g Impact
&
Impact
Sta
2. Ana
ifying
ke
4 uin
h old g
lys
l
Va
er
in
.
Results
3. Measuring Source: EVPA
1.4 Methodology
research on impact measurement and the practical experience of working with VPO/SIs
-
lated to the Expert Group in the spring of 2012. Between April and July of 2012, six webinars
were held, each webinar related to a particular step in the process (plus an introductory
asked to prepare a presentation, including a case study on a particular step. The experiences
the frame-works put forward in this manual to ensure it is well grounded in the practice
of EVPA members and other social investors. The data gathered from the Expert Group
members was complemented with more in-depth interviews with selected VPO/SIs.
workshop provided initial feedback. Between November 2012 and March 2013 additional
feedback was garnered from VPO/SI practitioners in order to improve the guide. The
timeline of the Impact Measurement Initiative is shown below.
Timeline of the Impact

“Version 1.0” Measurement Initiative
Manual
Consultation
Period April 2013
1st Draft
Manual & November 2012
Case Study Workshop – March 2013
Development
Webinar
November 2012
Series April – August
2012
April – July
2012
1.5 Definition of social impact

There is currently a large amount of discussion and debate around social impact meas-
certain frequently used words in the impact measurement dialogue.
clearly sets out the differences between inputs, outputs, outcome and social impacts. The Impact Value Chain
SPO’s Planned Work SPO’s Intended Results
1. Inputs 2. Activities 3. Outputs 4. Outcomes 5. Impact
Resources (capital, Concrete actions of Tangible products Changes resulting Outcomes adjusted
human) invested in the SPO from the activity from the activity for what would have
the activity happened anyway,
actions of others &
for unintended
consequences
€, number of people Development & Number of people Effects on target Attribution to changes
etc. implementation of reached, items sold, population e.g. in outcome. Take
programs, building etc. increased access to account of alternative
new infrastructure education programs e.g. open air
etc. classes
€50k invested, 5 people Land bought, school New school built with Students with Students with access
working on project designed & built 32 places increased access to to education not
education: 8 including those with
alternatives: 2
Inputs: all resources, whether capital or human, invested in the activities of the
organisation.
Activities: the concrete actions, tasks and work carried out by the organisation to
Outputs:
activities.
Outcomes:
Social
Impact: outcomes.

JUNE 2015 35
(i) what would have happened anyway (“deadweight”); (ii) the action of others (“attribution”);
(iii) how far the outcome of the initial intervention is likely to be reduced over time (“drop
off”); (iv) the extent to which the original situation was displaced elsewhere or outcomes
displaced other potential positive outcomes (“displacement”); and for unintended conse-
quences (which could be negative or positive).
impact numbers. This is a general recommendation however we accept that there are
certain organisations (for example those who interact with government for pay for perfor-
impact numbers.
example11. Let us look at an investment in an organisation that focuses on increasing access

to education for primary school age children in developing countries. We have introduced
the key factors from the case in the impact value chain above to illustrate the difference
between input, out-put, outcome and impact.
The theory of change for this organisation is that lack of access to education is a key factor
in preventing the poor from moving out of poverty. Hence to increase access to education
the organisation builds educational infrastructure in developing countries. Its inputs are
the money invested and the people employed to build the educational infrastructure.
Their principal activity (although it may have other complementary ones) is building new
schools. One particular output would be a new school built with places for 32 primary
school children, although the actual outcome with respect to increased access to education
is only 8 as 24 of the other potential primary school children were kept at home to work
on the harvest and do other essential work for the family. In fact, the impact is even less
its activity: of those 8 primary school children, 6 were already receiving some form of
education through open air classes and visiting teachers.
This example shows the importance of understanding the difference between impact,
out-comes and outputs when considering the social impact of a SPO.
Liechtenstein, (2011), “In search

of gamma: an unconventional
perspective on impact investing”.
PART 2:
The Impact
Measurement
Process

JUNE 2015 37
THE IMPACT MEASUREMENT PROCESS

STEP 1: SETTING OBJECTIVES
In the following sections, we will go through each step in the impact measurement process.
For each step, we will explain what it means, how the step is implemented at two levels
(i) at the level of the Investor, the VPO/SI, and (ii) at the level of the Investee, the SPO itself;
provide concrete recommendations and illustrate by using a real-life example. The reason
why the manual contemplates two levels is because a VPO/SI achieves impact indirectly
by investing in a SPO that is solving a particular social issue. A VPO/SI needs to consider
both levels and how to achieve an appropriate alignment between the two.
The two levels of impact

Investor measurement
(VPO/SI)
Achieve Impact:
Ensure progress towards
impact objectives
Investee
(SPO)
2.0 Step 1: Setting Objectives
2.1 What?
to proceed with the impact measurement process and this can lead to overburdening the
SPO and even the VPO/SI with excessive data collection requests.
-
tives should be set at two levels:
(i) At the level of the VPO/SI; and
(ii) At the level of the SPO
2.2 How to?
Level of VPO/SI
their relationships with the SPOs. Our conversations with VPO/SIs have highlighted that
often VPO/SIs begin with an opportunistic approach to venture philanthropy and social
investment. There may also be other issues e.g. the views of potential donors / investors

from them due to opportunistic approaches or forceful donors / investors.
(i) What is your motivation for measuring social impact?

There are many different purposes of impact measurement and these imply different target
audiences and outlooks.
A VPO/SI may want to use impact measurement for several reasons. The following list
is not exhaustive but provides the main reasons why a VPO/SI should strive to measure
impact. Each motivation in turn has implications for how impact is measured:
1. A tool to assist with investment selection: allowing the VPO/SI to prioritise where to invest
its resources for greatest impact. In this case the target audience of the impact measure-
ment will be internal to the VPO/SI, most likely the VPO/SI portfolio managers, and the
outlook will be forward looking.
2. Evaluation of the progress of a SPO: again the target audience is internal, however this will
also include the management and board of the VPO/SI as well as the individual portfolio
managers and, rather than being prospective, monitoring occurs on a continuous basis.
3. A management tool to ensure that social impact is integrated into strategy and operations: is of
great use to the management of the VPO/SI. This form of impact measurement would
also be done on a continuous basis.
4. Facilitation of aligning of incentives: can be done either with an internal audience in mind:
incentive schemes for portfolio managers based on social impact achieved to steer their
-
achieved. In both these cases there are elements of continuous but also retrospective
measurement of impact.
5. Reporting purposes: so it can communicate the social impact achieved to external stake-
holders in order to facilitate marketing or fundraising efforts. This is almost always done
on a retrospective basis.
In practice a VPO/SI is likely to use impact measurement for a combination of purposes.
Even though Ferd Social Entrepreneurs (please refer to case study) has only one owner
rather than a large external investor group, they still believe it is important to measure
and to owner Johan Andresen that it is possible to create social impact in a country with
a well-developed welfare state and to motivate other investors to follow a VP approach.

JUNE 2015 39

12
cites 3 different reasons for impact measure-
ment:
(i) Track their progress i.e. for monitoring: hold themselves accountable for what they
do and how they do it by measuring inputs, activities and outputs of their work as
well as those of their investments.
(ii) Inform their strategies i.e. as a management tool: test assumptions and track
achievements by measuring outputs, outcomes and impacts as well as under-
standing how and why they have succeeded or failed.
goals by measuring outcomes and impact, sharing results and collaborating with
partners to understand what works and why in the populations they serve.
The table below provides an overview of the principal motivations for measuring impact
and their associated audiences and outlooks.
Motivation Target Audience Outlook

Investment Selection: prioritise where Internal (VPO/SI portfolio Prospective
to invest resources for greatest impact managers)
Monitoring: evaluate progress of the Internal (VPO/SI portfolio Continuous

SPO against milestones, with increased managers, management, board of
transparency VPO/SI)
Management tool: a framework to Internal (VPO/SI portfolio managers Continuous

integrate social impact into strategy and management)
and operations
Aligning incentives: generates Internal (VPO/SI portfolio Continuous/

incentive schemes that steer work managers, management and Retrospective
towards achieving impact, and/or for
setting funding milestones with the management)
SPO
Reporting: communicate impact to External (other stakeholders) Retrospective

external stakeholders for marketing
and fundraising purposes
Source: EVPA
Reality Check
The reality is that no two VPO/SIs are the same, and your understanding of your motiva-
tion for impact measurement needs to be framed in the context of what is reasonable with
your resources, the type of SPOs you invest in, the level of rigour you require in your
analysis and the timeframe you are considering for your analysis.
12. Nelson & Ratcliffe, (2010),

“A Guide to Actionable
Measurement
Gates Foundation.

(ii) What resources can you dedicate to impact measurement?
resources available, the higher your expectations can be as to what you can achieve from
impact measurement and the greater the rigour and complexity that can be applied in the
process. But with limited resources, what you would like to achieve from impact measure-
ment needs to be much more tempered and focused.
to set up the process and the second those required to implement and use the process.
Depending on the complexity of your approach, you could expect to spend three to six
months establishing the methodology and training the team. To implement and use the
process you could expect to have one person dedicated part time to impact measurement.
The aim is for impact measurement to become an integral part of the investment process
so that it is used by all VPO/SI team members on a daily basis, but it is useful to have
someone responsible for the overall process.
A resource called “The Good Investor: A Book of Best Impact Practice”13 focuses on
integrating impact measurement into the investment process. The guide recommends
impact investors to include the following functions to make impact measurement an
integral part of the investment process:
• An investment team that understands the essentials of impact measurement.

• Some in-house expertise regarding impact analysis (either within the investment
team, or active in supporting it).
• A person with a head of impact role (if not a full time position, this responsibility is
• An investment committee with diverse membership, including social and invest-

ment expertise, with members who are able to read impact reports, understand the
key parameters at play, and integrate impact into the making of reasoned investment
decisions.
The issue of constrained resources is often heard as a reason preventing VPO/SIs and SPOs
from getting started on impact measurement. Sometimes this is more a mental barrier, and
13. Hornsby, A. & Blumberg, G.,
(2013), “The Good Investor: A Book
on impact measurement without incurring high costs. Other times impact measurement of Best Impact Practice”. Investing
is seen as a burden, driven by VPO/SIs and/or to be outsourced to external consultants. for Good.
http://cdn.goodinvestor.co.uk/
wp-content/uploads/2013/01/
A survey of 1000 SPOs in the UK by New Philanthropy Capital 14 showed that more than thegoodinvestor.pdf
14. Ní Ógáin, E.; Lumley T.;
Pritchard, David., (October
2013), “Making an Impact: Impact
that SPOs said they found when they did measure their impact was not increased
measurement among charities and
funding but improved services! social enterprises in the UK” New
Philanthropy Capital.

JUNE 2015 41

On the other point, consultants can indeed provide useful guidance and advice; however
allocating internal resources to focus on impact measurement is vital. To perform a good
impact analysis it is important to know the organisation well and assigning internal
resources to these tasks ensures any learning about impact measurement remains within
the organisations so as to inform their strategy, structure, policies and procedures i.e. to
improve services.
LGT Venture Philanthropy (“LGT VP”) estimates that it took them six months to
establish the methodology and another 3 months to train the team. For them termi-
nology was the main issue as it was important to establish a common dialogue within
-
nitions, guidelines and examples. However despite the clarity in the framework it took
a while for the team to get up to speed. LGT VP then mapped the logic model to the
complications and increased the time needed for the team to become comfortable with
the approach.
(iii) What type of SPOs are you working with?
will potentially limit the type of information that the SPO can provide. You should also
consider what assistance the SPO requires in order to provide you with the data needed
to measure impact. In addition, the complexity of the issue that the SPO addresses may
also constrain your impact measurement process and should be considered upfront when
deciding on the scope of your impact measurement.
(iv) What level of rigour do you require in your impact analysis?

A point that we develop further in Step 2 is the concept of how accountable do you expect
your investees to be when assessing their impact? By increasing their accountability you
increase the rigour of your analysis and thereby reduce the risk associated with any impact
claims made. However the ability to do so depends on the type of SPOs you are working
with and the resources that can be dedicated to the process.
(v) What is your time frame for measuring impact?

In theory time frame should not be the driver for impact measurement as the time period
over which you measure impact should be determined by the most important outcomes
and the estimated length of time required to achieve them. However in practice VPO/
SIs may have external or internal pressures to invest for a certain period of time, which
although there is often a temptation to measure only outputs, especially when looking at
shorter investment periods (less than 5 years), all VPO/SIs should aim to go a step further
and concentrate on the outcomes of their investments. We discuss the difference between
outputs, outcomes and impacts as well as how to select appropriate indicators in Step 3.

(i) What is the overarching social problem or issue that the VPO/SI is trying to solve?
Some VPO/SIs decide early on to focus on a particular social issue such as the problem
of youth unemployment. Others have a broader social sector focus, which makes it more
response should include information about the nature and magnitude of the problem or
opportunity; which populations are affected; whether the issue is changing or evolving and
in what way it is changing or evolving. This analysis will allow you to understand the base
case and therefore, at a later stage, allow you to see whether there has been any change from
this base case. A clearly articulated response is necessary to be able to choose investments
in SPOs that can contribute to solving that social issue that the VPO/SI is addressing. For
the impact measurement process, the VPO/SI needs to consider this question clearly before
starting to make investments, and regularly revise and adapt as its investment strategy
develops.
(ii) What objective does the VPO/SI want to achieve?
what changes it wishes to achieve as opposed to the base case social issue previously iden-
questions to assist in answering the question on the VPO/SI – SPO relationship include:
• What is the problem SPOs are facing?
• What solutions are available which are provided by the VPO/SI?
• What is the correlation between these two points?
(iii) What are the expected outcomes?

This should include what the VPO/SI must achieve in order to be considered successful
and will form the basis of the milestones against which the VPO/SI will be measured.
For the VPO/SI it is important to evaluate the expected outcome of its investment in the
SPO, i.e. the expected outcome of the SPO and how the VPO/SI expects to contribute to
achieving that outcome. Given these are likely to evolve over time, it is best to organise
VPO/SI that provides a large grant to one of the players in a particular social sector and
region may distort the market by creating an unfair competitive advantage (even though
instruments.
Tools such as theory of change, logic model or the initial steps of social return on invest-
ment (“SROI”) may be useful at this stage.

JUNE 2015 43

In 2011, Noaber Foundation (“Noaber”) changed its strategy completely to focus solely
on healthcare. The rational for this change was that Noaber saw one of its roles as that
of connecting people/organisations and creating synergies so as to achieve impact at
an aggregated scale. This was more feasible when investees were active in the same
collective impact.
Level of SPO
of support systems, methods and tools are available. Tools recommended to assist VPO/SIs
of methodologies such as SROI or balanced score card (which are themselves based on the
theory of change) are equally useful when working with SPOs on this step of the process.
The manual has extracted the commonalities of the various tools mentioned to come up
At a minimum you should answer these questions about the SPO15
(i) What is the social problem or issue that the SPO is trying to solve?
As per the recommendation for VPO/SIs, the response should include information about
the nature and magnitude of the problem or opportunity; which populations are affected;
whether the issue is changing or evolving and in what way it is changing or evolving. This
analysis will allow you to understand the base case and therefore, at a later stage, allow
you to see whether there has been any change from this base case.
(ii) What activities are the SPO undertaking to solve the social problem or issue?
This should include a description of exactly what the SPO is doing to try to effect a change.
demonstrating how each activity relates to another.
(iii) What resources or inputs, as per the impact value chain, does the SPO have and need to
undertake its activities?
This should include the time, talent, technology, equipment, information and other assets
available to conduct the activities. Ideally it should also consider whether a mismatch
exists between the activities and the resources available to execute those activities. As a VP
investor, you should also consider what would be your contribution to helping the SPO to
solve the issue (access to networks, capacity building etc.) as a key input.
15. Elaborated from Centers for
“Framework for program evaluation

in public health”.

(iv) What are the expected outcomes?

As per the recommendation for VPO/SIs, this should include what the SPO must achieve
in order to be considered successful and will form the basis of the milestones against which
the SPO will be measured. Given these are likely to evolve over time, it is best to organise
Ferd Social Entrepreneurs investing in The Scientist Factory illustrates a complex issue.
Their vision is that by providing interesting and exciting after-school science classes to
primary school children, more children will be inspired to consider natural sciences as
a career path and opt for science classes in high school and at university. Trying to show
the timeline involved as well as the problem of attributing any decision by the children
guidance is required from the VPO/SI. The VPO/SI can work in collaboration with the
are not opposing it is feasible to move ahead with the relationship, but in the case they are
not, then serious questions need to be asked on the appropriateness of the investment as
part of the investment selection process.
“First of all an organisation needs to be clear about its goals: What would you like your
organisation to be recognised for in terms of actual changes? What is the long-term
change you want to see as a result of your work? Once these basic questions have been
answered it is crucial to think about the pre-conditions that need to be in place for the
long term impact: what changes need to happen at what level – within the target group,
the community and the society as a whole – to lead to the desired impact? How do
-
ities and initiatives contribute to change? What can only be achieved through collabora-
They added that, “It is important to think out of the box when developing a theory of
change. While an organisation explores how change happens it is tempting to simply
explain why it does the respective activities. There is often a certain hesitation to leave
the comfort zone of what one already knows. This new thinking requires considering
many external factors which lead to the desired change and which one may not have
freely. There is no reason to fear the outcomes, as this process is valuable and can only

JUNE 2015 45

lead to an increased under-standing of the underlying reasons for existing or future

activities. We consider it as standard good practice for the communities (clients/target
groups) to be actively involved in coming up with the theory of change (constituency
voice), as well as in all relevant planning steps along the way (not as a one-off effort).”
“SMART” Objectives16
in the context of venture philanthropy and social investment.
and time bound:
the outcome or goal and what requirements are necessary.

•
a quality metric, quantity, time and/or cost-effectiveness. Measurable means not
•
resources (time, human, capital, technology) it has at its disposal. It should allow for
some stretch to encourage the organisation to meet its goals.
•
•
evaluation period that has been set by the SPO and/or VPO/SI.
(or outreach) and outcomes, which usually have a quantitative target with a deadline.
16. Doran, G. T., (1981), “There’s

17
In a S.M.A.R.T. way to write
management’s goals and objectives”.
Management Review, Volume
70, Issue 11 (AMA FORUM), pp.
35–36.
17. Examples thanks to Social
Performance Management
Resource Centre:
http://www.themix.org/social-
performance

2.3 Practical Tips

•
considered at both the level of the VPO/SI and the SPO.
• Often VPO/SIs do not spend enough time upfront considering why they want to measure
what not when faced with scarce resources.
2.4 Recommendations for Managing Impact

• A VPO/SI needs to formulate clearly what is its overarching social problem or issue so as
to be able to choose investments in SPOs that can contribute to solving that social issue.
• Understanding the current and expected social impact of an organisation early in the
decision process is extremely valuable: it creates a common understanding of the impact
of an organisation among all stakeholders; allows the VPO/SI and SPO to “speak the
same language” and assess at a later stage whether impact has been achieved.
• A VPO/SI should convince the SPO of the value of impact measurement, provide assis-
• Decisions have to be made about the amount of time and resources that a SPO should
dedicate to impact measurement.
2.5 Worked Example18

Throughout the manual we will be illustrating the various steps in the process through the
use of a worked example. The worked example focuses on a VPO/SI that is investing in
early-stage SPOs in Africa.
In setting the scope of impact measurement they needed to consider that their rationale for
impact measurement was driven by three reasons:
• -
• Ongoing monitoring: facilitating their offering of technical assistance.

• Reporting: to existing shareholders as well as to assist in raising additional funding from
other parties.
In addition, like many VPO/SIs, the investment team is small and resources are tight, which
frames how much time and money they can dedicate to impact measurement. However
they do have a de facto head of impact and are keen to pursue a rigorous process focusing
on outcomes even if they may not be able to accurately determine impact (according to the
18. Thanks go to Beyond Capital Fund
for introducing us to this example,
technical assistance and measuring impact as well as the other inherent characteristics of which is inspired by and elaborated
http://saner.gy
seven years.
The views contained in this
document are those of EVPA and not
of Beyond Capital Fund.

JUNE 2015 47

research and decided that focusing investments in the sectors of water, sanitation and health
on early-stage SPOs, technical assistance, particularly access to networks and mentoring, is
-
tion that is aiming to build and scale viable sanitation infrastructure in Kenyan slums,
beginning with Nairobi.
• Social problem or issue19: 2.6 billion people do not have access to adequate sanitation and
this number is not decreasing despite billions of dollars of aid. The resulting disease and
water pollution cause 1.7 million deaths and a loss of $84 billion in worker productivity
disease and death.

• Activities: the model involves 4 parts: (i) building a network of low-cost sanitation centres
in slums; (ii) distributing them through franchising to local entrepreneurs; (iii) collecting
the waste produced; (iv) processing the waste into electricity and fertiliser.
• Resources or inputs: equipment (sanitation centres, vehicles for collection, digesters to
ground in Kenya to supervise building of sanitation centres and selection of franchisees,

employees to collect waste products and transport to digesters, operators of digesters to
produce electricity and fertiliser); partners (implementation partners for education about
-
nance organisations to support franchisee purchases); funding (grants and investments
from foundations and social investors).
• Expected outcomes: positive expected outcomes at a local level include increased access to
sanitation facilities for slum dwellers, increased employment levels among slum dwellers,
improved health for toilet users and overall slum; increased income for toilet operators;
improved environmental situation (less waste in open waterways). At a national level,
positive outcomes could include a decrease in the number of power shortages, a decrease
in carbon emissions, a decreased reliance on imported fertilisers and a decrease in the
use of chemical fertilisers leading to positive environmental effects. Potential negative
outcomes could be displacement with respect to existing operators of toilets in the slum;
-
tions in sales and hence livelihoods of existing producers of fertiliser.
19. Source: Sanergy website –

http://saner.gy

STEP 2: ANALYSING STAKEHOLDERS
3.0 Step 2: Analysing Stakeholders
3.1 What?
social impact, the stakeholders to be considered will be different.
VPO/SI investments generate value for a variety of stakeholders. We will analyse the stake-
that it is relevant for both levels of analysis:

“Any party effecting and/or affected by the activities of the organisation.”
There are different categories of stakeholders (which are not necessarily mutually
enhance or decrease impact.
Stakeholder analysis is an important part of impact measurement because:

• We need to understand the expectations of the stakeholders, their contribution to and the
potential impact our work will have on them. If these expectations are in opposition
A SPO that focused on getting long-term unemployed people back into employment
based on a variation of “welfare to work” programmes is a good example. For two years
these people received a salary from the SPO (subsidised by the government) rather than
from the employing company. Other than the participants themselves, two important
stakeholders were the government (subsidising the salaries for two years) and the
employing company (accepting to take on the long-term unemployed for two years). For
the government, the expectation was that after the 2-year period, the people receiving
company. However, the company saw this as an opportunity to have free labour for
2-years and did not intend to hire the participants at the end of the 2-year period. Unsur-

JUNE 2015 49

• The co-operation of the main stakeholders in the impact measurement process is critical.
ted from the products or services of an organisation. For them these real case studies
advantaged people.
3.2 How to?

Applying to both the VPO/SI and SPO level, there are two important aspects of stakeholder
tasks: (a) stakeholder mapping, (b) stakeholder selection and (c) understanding stakeholder
expectations.
(a) Stakeholder mapping
set in Step 1 at the level of the VPO/SI and the SPO.
Level of VPO/SI
At the level of the VPO/SI we need to remind ourselves what is the overall scope of the
This will ensure that when the VPO/SI reaches Step 5 it is in a better position to customise
its data analysis and prepare the various reports.
affected by the intervention.
Level of SPO
At the level of the SPO, we have already answered questions around the issue being
addressed, the activities of the SPO, the available resources and the expected outcomes.
These answers should guide us as we list the direct and indirect contributors as well as the
As an example we can consider a SPO that supports ex-offenders in seeking employment

with the aim of reducing re-offending rates. In this case we can highlight certain stakeholders
within this framework. The direct contributors are the staff at the SPO, the indirect contributor

Direct Indirect
Contributor Direct contributor e.g. Staff at SPO Indirect contributor e.g. family of
ex-offender
Beneficiary
offender who is the focus of the SPO e.g. those people who do not
offender being employed
Source: EVPA
(b) Stakeholder selection
Level of VPO/SI
For a VPO/SI the stakeholder selection process as it relates to the scope of impact measure-
ment and eventual reporting in Step 5 should be relatively straightforward. For example, if
your focus of impact measurement is investment selection then your key stakeholders will
be the staff of the organisation (portfolio managers especially) and the board of directors
-
urement is external reporting and communication then you will select those stake-holders
mostly affected by this activity i.e. investors / donors.
the intervention. The key stakeholders contributing would be donors/investors in terms
consequences one would need to consider other organisations or communities that might
be affected by the intervention. That should be part of the due diligence process.
Level of SPO
To mitigate potential selection bias when asking the SPO to provide a list of stakeholders
for you to contact you can:
• Explicitly ask the organisation to include some parties where the outcomes were not
ideal.
•
the SPO but who are familiar with its work.
• Always ask the stakeholders to discuss the successes and failures they have experienced.
• A the end of the interaction with the stakeholders ask them to identify other parties with
-
tives of the VPO/SI and those of the SPO you should be able to rank those stakeholders

JUNE 2015 51

in order of importance. Our suggestion is not to try to measure everything so you should
point concerns regarding resources (time, manpower, capital) come to the fore, as you must
decide what level of accountability (further described below) you want to accept in order
to perform a valid analysis.
Two important questions you can ask to help reduce the number of stakeholders are:
(ii) How relevant is the stakeholder group to my primary mission?
One common question asked is how to decide which stakeholders experience material
out-comes in advance. Clearly this entails risks that should be acknowledged and can
be framed as per the question of how accountable the SPO should be. Additionally by
focusing on stakeholder groups relevant to the primary mission and not including certain
stakeholders in the analysis we may miss large positive or negative outcomes that would
affect our overall impact analysis. However it is important to keep in mind that this is a
learning process and as you go through the impact measurement process you can reassess
(c) Stakeholder expectations
Level of VPO/SI
Once you have selected the stakeholders you should understand their expectations. It is
expectations are managed and their contributions are aligned. For example, this means
money, staff and consultants should know what the goals are that they are trying to achieve
with their work, and SPOs should know what the VPO is expecting them to change. If for
example certain investors/donors have very different expectations to yours, then you may
need to consider how appropriate an investor/donor they are for your VPO/SI, so as to
avoid potential issues at a later stage.
In your engagement with SPOs it is also important to understand what they expect from
these topics.
Level of SPO
With the list of 5 to 10 stakeholders you should then understand their expectations.
-
alises for each of them in a tangible way may differ considerably. For example, in the UK
a social impact bond linked to an organisation that aims to reduce the re-offending rate of
ex-prisoners has the UK government and the entrepreneur of the SPO among its stakeholders.

-
larly budgetary and prison over-crowding) caused by re-offenders, whereas the entrepre-
neur sees impact more in increasing the quality of life of the ex-prisoner so they have no
desire to re-offend.
Importantly there is a distinction between differing expectations, something that is natural

and inherent in venture philanthropy and social investment, and opposing expectations,
which as we demonstrated earlier, can be disastrous for the success of a VP investment. If
it is found that stakeholders do have opposing expectations, then you should take action
in the form of assessing how this may impact on the success of the investment and decide
whether or not to continue or not with the investment. Generally the recommended way to
more detail in the section on stakeholder engagement below.
Mapping stakeholders with respect to accountability: a more in depth way of thinking

about stakeholder relevance
A more in-depth way to consider mapping and then selecting the most relevant stake-
holders is to determine the level of accountability of the SPO in question. For example
spectrum of levels of accountability between these two concepts and illustrate them
you would focus on the employment outcomes.
generalised for the whole group. For example we would consider the trainees who
gained employment but we would not consider the extent to which family support
was critical.
group but analysing this for sub-groups. For example you would consider those
trainees who gained employment and who had family support.
-
ciary group and sub-group. For example you would also consider those trainees
who gained employment who had family support and those that did not.
(v) Accountability for material positive and negative outcomes on a selection of the
For example you would consider the trainees with family support, those without,
the families of the trainees and the employees of the organisation but you would
not consider all stakeholders.
(vi) Accountability for material positive and negative outcomes on all stakeholders e.g.
SROI. For example you would consider the trainees with family support (positive

JUNE 2015 53

outcome, may gain employment), trainees without family support (negative
more depressed meaning they are less likely to gain employment in the future),
families of the trainees, employees, suppliers, funders etc.
It is evident that focusing solely on level 1 will bring a quicker estimation of social
impact. However there is higher risk that the impact is misstated and that the SPO
could even be having an overall negative social impact. Level 6 is certainly a slower
and more resource intensive way of considering the social impact of the SPO, however
there is less risk that impact is misstated as social impact on all potential stakeholders
is considered. This trade-off is a decision for the VPO/SI and should be based on its
motivation for impact measurement, its resources (human, capital, time) as well as the
relationship with the SPO and its resources and motivations.
1. Accountable for SPO intended outcomes on Stakeholder mapping

6 and selection based
2. Accountable for material but only positive on the concept of
5 accountability
whole groups
3. Accountable for material but only positive
4
analysing these for sub-groups.
4. Accountable for material outcomes on main
3
2 5. Accountable for material outcomes on some

1
6. Accountable for material outcomes on all
SPO
Source: EVPA20
(ii) Stakeholder engagement

Engaging in communication with the selected stakeholders is recommended to be able to
understand their expectations and, later in the process, verify if their expectations have
been met, which is discussed in more detail in Step 4.
For VPO/SIs, this means engaging regularly with donors/investors, staff and other human
resources, as well as with the investee SPOs so that you are aware of their expectations and
can correct any misalignment before further harm is done.
20. Inspired by conversations with

Jeremy Nicholls, SROI Network
due diligence process of a VPO/SI. (now Social Value UK).

Within each particular stakeholder group you should aim to construct a sample that is of
an appropriate size and diversity, for example a mix of male and female, older and younger
people. The size of the sample will depend on the reach of the SPO. However what is
important is ensuring a good sample selection that is non-biased and random. Discussions
20–120 respondents for the sample to be credible, depending on the size of the population.
The communication channel selected should be appropriate for the stakeholder, and may
require different methods for different stakeholders. For example, an elderly population
will need to be approached via face-to-face interviews, while a group of youths can be
polled via internet surveys. One point to keep in mind in any interaction however is the
importance of “neutral” questioning, so that the stakeholders can give their answer without
overt direction or pressure from the VPO/SI.
(for example the families of the ex-offenders). Our recommendation is that if a stakeholder
is to be included in the analysis then a method of communication should be found, even
if this is via an intermediary. Without engaging with the stakeholders it is impossible to
understand their expectations and then verify whether those expectations have been met.
3.3 Practical Tips

• Begin stakeholder analysis focusing on a small number of relevant stakeholders and
expand from there, rather than trying to measure everything in one go. This guide is not
set up to advise you to measure everything!
•
(with the understanding that this is a learning process so over time the risk of missing
large positive or negative outcomes decreases).
• As you become more experienced in impact measurement you can consider those stake-
• Engaging with stakeholders on multiple occasions may not be feasible. Assess when is
the optimal time to engage and then ensure all possible preparation has been completed
prior to such time in order to get the most out of the interaction.

•
-
•
being aligned, and otherwise implement corrective measures.

JUNE 2015 55

• When a VPO/SI makes an investment in a SPO, stakeholder analysis should be part of

the due diligence phase.
• To avoid wasting resources, increase the intensity, i.e. more stakeholders, more involve-
ment from the same stakeholders and higher numbers involved from each group, up to
the number required for a non-biased and random sample of the analysis as you increase
• As the investment period proceeds, regularly get back to the stakeholders to verify that
their expectations are being met (more details on how to do this in Step 4).
• Consider upfront when would be the appropriate time to revisit stakeholder analysis
lines being entered, changes to policy environment etc.
3.5 Worked Example

In our example we focus on the stakeholders of the SPO. The stakeholders are the toilet
users, the toilet operators, the waste collectors, the broader slum dwellers, the employees
Direct Indirect
Contributor Toilet operators

organisations
VPO/SI
Beneficiary Toilet users Slum dwellers

Toilet operators
Waste collectors Government
Farmers
Existing fertiliser companies Existing
power companies
Source: EVPA
The VPO/SI would rank the importance of these stakeholders as follows: toilet users, toilet
operators, broader slum dwellers, waste collectors, SPO employees, farmers, existing ferti-
liser producers, existing power companies, government. Given the resources and time that
the VPO/SI has available, the early-stage nature of the SPO and the view that these stake-
holders are most relevant for the VPO/SI to decide if it is achieving its mission; the VPO/SI
broader slum dwellers.


You can consider the expectations of these stakeholders as follows:

• Toilet users: pay an amount of money to use a clean toilet, they therefore expect the toilet
to be clean and may expect to have fewer health problems.
• Toilet operators: earn income from the toilets and pay the franchise fee. They expect to
have a steady stream of customers for their toilets and the necessary franchisor support
from the SPO in case of any problems with the toilet.
• Broader slum dwellers: if the installation of toilets results in less human waste in the
slums then all slum dwellers may have fewer health problems. However it is unlikely
that slum dwellers will necessarily have this expectation.
Although these expectations do differ, none of them are opposing, therefore we can assume
-
niques and the advice on sampling detailed above.

JUNE 2015 57

STEP 3: MEASURING RESULTS: OUTCOME, IMPACT, INDICATORS
4.0 Step 3: Measuring Results: Outcome, Impact, Indicators
4.1 What?
outputs, outcomes, impact and indicators.
Outputs:
activities.
Outcomes:
Social
Impact: outcomes.
(i) what would have happened anyway (“deadweight”); (ii) the action of others (“attri-
bution”); (iii) how far the outcome of the initial intervention is likely to be reduced over
time (“drop off”); (iv) the extent to which the original situation was displaced elsewhere
or outcomes displaced other potential positive outcomes (“displacement”); and for unin-
tended consequences (which could be negative or positive).
Many VPO/SIs and SPOs may be tempted to focus their measurement on outputs, but
often, simple output measures say very little about the actual outcomes. Imagine a nature
conservancy organisation whose mission is to conserve natural species, which measures
member-ship numbers (an output measure) as a measure of its effectiveness. From 1980 to
effective and achieving their mission. However, the membership numbers might have
increased due to the escalating problem of depleted biodiversity. Indeed, if they were to
look at the number of species present in the geographic area where they are active during
on an output measure, which was not aligned with their mission of conserving species, they
were unable to measure the true impact of their work. On the other hand, output measures
For example if their mission had been to increase awareness of the nature conservancy
issue then membership numbers (despite being an output measure) could have been one
of the relevant indicators.
The difference between outcomes and impact can very quickly become theoretical when
considering concepts such as attribution, deadweight, drop-off and displacement. In
reality there is no tool or methodology to accurately measure these aspects. The types of

studies, which would stand up to scrutiny (e.g. randomised control trials etc.), are very
costly, time consuming and may also open up ethical questions when it comes to excluding
for this manual is to be a practical guide hence we recommend that for VPO/SIs and
SPOs to measure impact that they calculate the outcomes of their investments while
acknowledging
contributed (e.g. the effect of the welfare state in developed countries) or where there may
be negative effects. i.e. those factors that increase or decrease impact. In some situations
comparing to potential control groups (for example based on research of comparable situ-
ations elsewhere) may also be feasible.
One could argue that impact should be very closely related to outcome, as venture philan-
thropists and social investors should already be aware of the other parties working in their
sector of focus. If there is already a large amount of activity we could question whether
targeting different areas where they can really add value. In practice, a rule of thumb
If outcomes and impacts become too detached from the operations of the organisation, the
organisation will lose ownership of the impact analysis.
the activities of the organisation i.e. what is done to effect change in the base case. These
outputs are internal to the organisation and hence easy to measure whereas outcome and
impact are related to the expected and unexpected effects of the activities of the organisa-
tion i.e. what effects the activities of the organisation have on the base case. These are by
Internal vs external focus: the

Inputs Internal Focus use of outputs, outcomes or
impacts
Organisation
Outputs
Base Case Changed Case
Outcomes
External Focus
Impacts
Source: EVPA

JUNE 2015 59

VPO/SIs and SPOs identify and use indicators to manage outputs, outcomes and determine
impacts.
An indicator can be expressed in different ways, for example as numbers, as a ranking of

systems or as changes in the level of user approval and further be used to express qualita-
tive and/or quantitative information. Quantitative indicators are numerical. Qualitative
indicators are those based on individual perceptions, for example responses to interview
questions. The types of indicators that exist can also be described at a more granular level
different types of indicators as no one type of indicator is better than another; its suitability
depends on how it relates to the result it intends to describe.
For example21, if a VPO/SI is investing in a SPO focused on increasing access to clean water
outcome could be a reduction in ill health and mortality and a relevant outcome indicator
could be the increase in the number and proportion of the target population with sustained
availability of clean water for domestic use.
with respect to men. An output indicator could be the number of loans given and repaid
as agreed. Two outcome indicators could be the % of women with increased disposable
income; and the expansion of their options towards diverse social and economic roles.
The United Nations Millennium Development Goals are lofty goals22, however the
(i) Halve, between 1990 and 2015, the proportion of people whose income is less than
one dollar a day.
(ii) Achieve full and productive employment and decent work for all, including women
and young people.
(iii) Halve, between 1990 and 2015, the proportion of people who suffer from hunger.
21. Inspired from Ruby Sandhu-
Rojon, UNDP, “Selecting
Indicators for impact evaluation”.
22. http://www.un.org/
millenniumgoals/

then monitored on a country-by-country basis. The indicators selected are:
1.1 Proportion of population below $1 (PPP) per day

1.2 Poverty gap ratio
1.3 Share of poorest quintile in national consumption
2.1 Growth rate of GDP per person employed
2.2 Employment-to-population ratio
2.3 Proportion of employed people living below $1 (PPP) per day
2.4 Proportion of own-account and contributing family workers in total employment
3.2 Proportion of population below minimum level of dietary energy consumption
4.2 How to?
Level of VPO
Although the explanation above focuses predominantly on the SPO; the VPO/SI, via the
outputs, outcomes and impacts and set indicators. The principles of how to select outcomes
and indicators described for SPOs below are equally valid for VPO/SIs.
Measuring impact at the portfolio level is a hot topic in impact measurement at the moment
and there is no common practice yet. VPO/SIs should be aware that the following practices
exist and are being tested by leading VPO/SIs:
• Aggregation of output data e.g. lives touched. VPO/SIs can view the Impetus Trust Impact
Report 2010–201123 as an example. The TONIIC Institute also recently published an
E-guide24 -
cross-portfolio level.
• goals i.e. different indicators per investment
but with an additional overlay of assessing whether or not the goals have been achieved
unconventional perspective on impact investing.” Based on this general idea, the

European Investment Fund is currently experimenting with an approach that uses an
relative measure that can be aggregated. For example, if you make an investment in the
23. Accessed from
www.impetus.org.uk
24. “TONIIC E-Guide: Impact
the relative multiple is 70/65. This multiple can be used to aggregate across the portfolio. Measurement”, (Fall 2012),
TONIIC Institute.

JUNE 2015 61

• . For example, Noaber

Foundation ensures that it aligns the outcomes targeted through its own theory of change
to the outcomes selected by the SPO. If these outcomes are not aligned it does not go
ahead with the investment. Rather than aggregating all individual SPO indicators, the
• Selection of common outcomes at the portfolio level. For example Big Society Capital25 has
number of outcomes in each sector in which it operates. Their focus will be on ensuring
that reporting from investees focuses on these outcomes. However they do clearly state
that although outcomes can be used as a mapping tool to show a VPO/SI where they
are active, an outcome map can only be used for aggregation purposes if truly like-for-
like numbers and contexts are involved and issues such as double-counting have been
dealt with.
important to assess the impact of the work of the VPO/SI on the SPO. As set out in the
Good Investor26, in practice the impact of the VPO/SI on the SPO is apparent in four areas:
• Scale of the investment: the percentage contribution of the investment to the SPO forms a
baseline for the extent to which a VPO/SI can link impacts achieved by the SPO back to
the investment e.g. 25% of SPO capitalised by the VPO/SI translates to 25% of the impact
• Growth and strength of the SPO: -

ience of the SPO, growth in impact generating activities and delivery of services, growth
in outcomes and impact.
• Access to other and further capital: here we consider the “deadweight” of the VPO/SI i.e.
its highest.
• Expertise and networks:
be tracked and valued.
activities of the SPO when a VPO/SI considers its own impact.
The key point for VPO/SIs to remember is that the SPO should report on those outcomes
-
25. More information on the

invests the resources to achieve this. It is important not to overburden the SPO. outcomes matrix can be found at:
http://www.bigsocietycapital.
com/outcomes-matrix
26. Hornsby, A; Blumberg, G.,
(2013), “The Good Investor, A Book
of Best Impact Practice.” Investing
for Good.

Level of SPO
Output and outcome measures are different and should be used in different circumstances.
Output measures are suitable when the focus is on the operational aspects of the SPO (e.g. as
a management tool or for day to day monitoring). However they may also be useful in deter-
there is research that a particular output does result in a particular outcome. For example
participants at an event organised by the SPO (an output measure) would be an appropriate
issue, then counting the number of participants would not be appropriate as it says nothing
about whether the event had any effect on the opinions of those participants.
When focusing on output measures, there are a few databases that include a large number
of output indicators e.g. IRIS and Global Value Exchange. Where possible we would
indicator exists within one of these databases and only if it does not, develop your own
indicator. Indeed a number of VPO/SIs follow this policy.
Standardisation of output indicators serves two important purposes:

-
(ii) Reducing the burden on the SPO, as if all VPO/SIs can request the same output indica-
tors then this reduces the multiple reporting burden of the SPO.
LGT Venture Philanthropy and Bamboo Finance try as much as possible to use IRIS
collaboration with the SPOs.
Outcomes should be your key focus as soon as your rationale for impact measurement
moves beyond the operational towards investment selection, external reporting etc.
Whether you then consider output or outcome indicators as relevant for showing your
progress towards your outcomes will depend on the nature of the business and the
SPO are very directly generating impact or if there exists independent research showing
place at the European level.

JUNE 2015 63

outcomes for various target social sectors in the UK27. The resulting outcomes matrix
the matrix houses a list of the high level outcomes that can be achieved within that
further into detail outcomes, and the indicators that can be used to measure them. The
full outcomes matrix, with the complete list of indicators is not yet complete but will
soon be available for download, it will be integrated into the Global Value Exchange
platform, and will sync with IRIS indicators.
Given our recommendation to VPO/SIs to focus on outcomes and then select appropriate
indicators, the next paragraphs provide guidelines on how to do this in practice.
organisation may state outcomes in a number of different ways28. The desired outcomes
different stakeholders seek different outcomes.
We identify three main types of outcomes:

• Outcomes focused on change: including the increase, maintenance, or decrease in behaviour,
skill, knowledge or attitude e.g. increase immunisation among young children.
• Outcomes focused on targets:
2 year old children in the community according to recommended public health schedules.
• Outcomes focused on benchmarks: including comparative targets, generally related to other
time periods or organisations e.g. increase the current 70% immunisation rate for children
aged 0–24 months to 90% by the year 2015.
(i) Outcomes focused on change
The change or desired effect In what For whom

Such as: Attitude, perceptions, Population group, programme
increase, decrease, maintain, knowledge, skill, behaviour, participant, client, individual,
improve, reduce, expand condition, agency, family, neighbourhood 27. Hornsby, A. ; Blumberg, G.,
organisation, community (2013), “The Good Investor, A Book
of Best Impact Practice.” Investing
Example: Awareness of environmental Among community members for Good.
Increase protection activities 28. Organisation Research Services.
“Outcomes for Success!” A product
of the Evaluation Forum, Jane
Reisman, Judith Clegg, (2000),
pg 3–22 (inclusive).

(ii) Outcomes focused on targets
The amount of change For whom In what
Such as: Population group, programme Attitude, perceptions,

Percentage, rate, ratio, participant, client, individual, knowledge, skill, behaviour,
amount family, neighbourhood condition, agency,
organisation, community
Example: Of community members Will increase their involvement

55% in environmental protection
activities
(iii) Outcomes focused on benchmarks (converted from a target statement)
The amount of change For whom In what Against what standard
Example: Of community members Will increase their involvement As compared to the 2010 rate
55% in environmental protection or To exceed the national
activities standard of 50%
There is however an issue when using any form of percentage statements in that without a
proper context you cannot know whether the change you are seeing is positive or negative.
For example if the % of community members who are active in environmental protection
increases from 55% to 60% but the community itself reduces in size, then the % increasing
on its own does not tell you much about whether more or less people are involved in envi-
ronmental protection activities.
(ii) Selecting outcomes

Outcomes are often lofty and abstract, so how do you set a concrete target for whether the
desired outcome has been achieved or not? This is where indicators come into play because
if you claim you have an outcome you need to be able to measure it.
Having gone through the process you may have a number of outcome statements, but it
is important to select only the relevant outcomes as informed by your mission, rationale
for impact measurement and the stakeholders you are focusing on. Some methodologies
to select among outcomes. To assist in the selection you can ask yourself the following
questions29:
• Which outcomes are most important to achieve (this will depend on the prioritisation
you assign to the stakeholders)? Which are most closely related to the core business of
the organisation? 29. Organisation Research Services.
“Outcomes for Success!” A product
• Are the outcomes material -
of the Evaluation Forum, Jane
ence for the key stakeholders? Reisman, Judith Clegg, (2000),
pg 3–22 (inclusive).

JUNE 2015 65

• Which outcomes are most useful? Which will provide the best information for manage-
ment decision-making, investment selection, reporting or whatever other purpose you
have for impact measurement?
• Which outcomes are most feasible? Which are most likely achievable with the resources
available? Which are likely achievable within the designated evaluation period? It is
important to reiterate that this question relates to achievability of the outcomes and not
the feasibility of their measurement.
The Global Reporting Initiative30 uses the principles of materiality, stakeholder inclu-
siveness, sustainability context and completeness, when identifying the topics that
are of relevance. These principles can also be applied in social impact measurement:
• Materiality: -
cant (i.e. require active management or engagement by the organisation) economic,
assessments and decisions of stakeholders.

• Stakeholder inclusiveness: the reporting organisation should identify its stakeholders
and explain how it has responded to their reasonable expectations and interests.
Failure to identify and engage with stakeholders is likely to result in reports that are
not suitable and therefore not fully credible to all stakeholders.
• Sustainability context:
wider context of sustainability i.e. discussing the performance of the organisation in
the context of the limits and demands placed on environmental or social resources at
the sectoral, local, regional or global level.
• Completeness:
performance in the reporting period.
(iii) Selecting indicators

Having selected the outcomes you need to select appropriate indicators. The key challenge
with indicators is to ensure their quality and integrity. Indicators should generate data
that are needed as well as useful because if they are not used carefully they can consume
extensive resources and generate data with little or no value.
A guiding principle for selecting indicators is that if you are looking at a sub-optimal
situation e.g. low self-esteem of adolescents then there must be some measurable evidence
It is that type of evidence that needs to form the basis of the indicator. We recommend that
you select the top three issues that demonstrate that a situation is sub-optimal. These issues
30. “Sustainability Reporting
should form the basis of your indicators. Guidelines”; Version 3.1; Global
Reporting Initiative.

(i) Indicators should generally be aligned with the purpose of the organisation. Although if
comparable with those used by others so that performance can be better benchmarked
and understood in a broader context.
(iv) Indicators i.e. more than one should be used, with a preference for two to three. For
that they take better care of their health, then an appropriate indicator could be the
number of times they visit their doctor in a certain period. However whether this
are taking better care of their health. At least one other indicator is required and a
conclusion can only be drawn about whether the outcome is achieved by seeing if they
all point in the same direction.
31
estimates the likelihood
that an individual falls below the national poverty line, the $1/day/PPP and $2/Day/
quickly collect and verify. These indicators are derived from the most recent national
Standards Measurement Survey, depending upon which dataset has the most complete
information, for each country. All indicators on the national household survey are
ranked according to how strongly they predict poverty levels. The full list of 400-1000
indicators is narrowed to the 100 most powerful ones. Using both statistics and expert
a point value on the original national survey responses. The total score (summing from
0 to 100) is then linked to the probabilities of falling above or below the poverty lines.
-
ment team had the following advice:
•
meaningful to the organisation.
• Choose a mixture of both quantitative and qualitative indicators and refer to already
One helpful question is: why do we need this information and do we have the capacity
to collect it ourselves or is somebody else collecting this already? Is there an easier
community etc.? 31. For more information refer to

progressoutofpoverty.org

JUNE 2015 67

(iv) What do you need to consider if you are aiming to measure impact?
To move from outcome measures to understand if the organisation is having an impact,
• Drop off: relates to the fact that over time the importance of impact decreases. Impacts
and if there are commonalities among them, so as to be able to improve services.

• Displacement: relates to the fact that with some interventions the positive effect that is
seen in a certain group can be offset by the negative effect seen in a different group
opening in a community may bring about the closure of another business already active
in the community.
• Deadweight: relates to a consideration as to what would have happened anyway i.e. in the
as well as the negative consequences of no intervention (increasing the impact of the

organisation).
• Attribution: relates to understanding how much of the change that has been observed is
-
ment etc. taking place at the same time.
• Unintended consequences:
activities, but are not part of the desired effect.
Unintended consequences:
more accurate calculation of impact they should consider the unintended consequences
consequences may be foreseen because, although the results of the activities on a

particular community or group were not intended, they are a clear result of the organi-
assigned indicators. However others may only manifest themselves once the activities
unintended consequences an organisation should review its activities periodically as

part of the monitoring and evaluation process (further discussed in Step 5) and then
The ability of an organisation to measure impact will depend very much on the sector
and geography in which it is operating. For example, in the UK, the development of the
government, access to public sector statistics on costs for reconvictions, the ability to create
a control group through the Propensity Score Matching method and the involvement of
organisations such as QinetiQ and the University of Leicester to independently assess

the method and outcomes (more details in the Esmée Fairbairn Foundation case study
in section 9.4). In fact the propensity of European governments to move towards pay for
performance contracts means the measurement of impact is becoming more important for
those organisations active in these areas.
However, for many VPO/SIs and SPOs access to independent statistics and the creation
of control groups in order to assess displacement, deadweight, drop off and attribution
is not possible due to the expense and specialist skills needed to carry them out. In these
cases the resources required to estimate these aspects in a rigorous manner are beyond the
scope of most VPO/SIs and SPOs. Therefore we recommend that social impact should be
measured by calculating outcomes, while acknowledging those factors that could serve to
increase or decrease impact. In some cases it may be possible to think about some evidence
as to what a control group may look like and could be used for comparison purposes, for
example based on research of comparable situations elsewhere. To gain an idea of what is
involved in measuring impacts in an academically rigorous manner we would recommend
reviewing the study undertaken of Grameen Danone Foods Ltd32 in Bangladesh by the
paper33
improved cooking stoves.”
Øyvind Sandvold, Business Development at Ferd Social Entrepreneurs shared that,

“We try to be as cost effective as possible in our measurement of impact but still show
meaningful results. Since we are only working within Norway, a country with a well-
-
cators that show the effect and if those numbers were greater or lesser (depending on
the context) compared to average numbers for a comparable group we would claim it
is appropriate to assume that there is an impact. At the same time we always collect
good stories from the SPOs to provide the context behind the numbers, so we have
“witnesses” to strengthen the results. We know that this is by no means “bullet proof”
evidence, but it provides us comfort in our impact beyond reasonable doubt.”
producer in Haiti called MFK illustrates the challenges in moving from outputs to
outcomes and then to impact. MFK dries, stores, roasts and then grinds peanuts into a 32. Found at
www.danonecommunities.com
paste, before mixing them with proteins, vitamins and minerals. The resulting mixture
33. Rema Hanna, Esther Duflo,
is packed into sachets and sold to institutional clients who distribute them for free to Michael Greenstone, Working
Paper 12–10, (April 16 2010),
Reviser (April 30 2012), “Up in
Smoke: the influence of household
behaviour on the long-run impact
of improved cooking stoves”.
34. Source: LGT Venture Philanthropy

JUNE 2015 69

Logic Model applied to MFK34
Model
Resources Organisation Activities
Equipment: peanuts processing factory, transportation vehicles Production of medicines known as RUFs: MFK produces 75MT
Supplies:
and expects to be producing 800MT p.a. by 2015 in an upgraded
Staff: facility.
on the ground in Haiti, trained labour force to run factory,
MFK Agricultural Development: MFK conducts 3–5 workshops
international support team in the USA
p.a. with Georgia University to teach subsistence peanut growers
Partners: institutional programmes / demand for RUFs, how to increase yield and quality of harvests, MFK manages 5
international support for agricultural development operations demonstration plots and sources 40% of its peanuts locally
Funding: philanthropic support to combat malnutrition
Intended Results
Resources Organisation Activities Impact (systemic)
Products: Improve physical wellbeing: Eradicate malnutrition in Haiti
MTs of RUFs produced p.a.: 75 (2011), 800 In 6-8 weeks, a child treated with RUF has 80% Build food security in Haiti
(2015e) likelihood of recovery. Once severe malnutrition has
# of products: 2 (2011), 5 (2015e) been treated the child can survive on a local diet.
MTs of local peanuts purchased p.a.: 40MT Children cured of severe malnutrition before age 5
(2011), 400MT (2015e) perform better at school and develop to be healthier
Services: and stronger.
# of farmers trained in agricultural skills # patients treated p.a.: 80,000
and provided with a stable market at fair
prices: 100 (2011), 1000 (2015e) # patients treated against severe acute malnutrition:
20,000
# children saved from becoming malnourished:

60,000
Improve social well-being:
to avoiding negative impacts, severe trauma and

emotional shock for the family circle
Improve material well-being:
Parents of malnourished children treated with RUFs

can go on with their lives normally as the medicine
Quality of Life Assessments (“QOL”) does not require medical supervision, cooking of
cooling.
No impact
Low impact are provided with technical support and access to a
Medium impact stable market
Strong impact Improve security: N/A
Very strong impact
Improve freedom: N/A

improvement of quality of life. For example MFK improves the health of children. On
average, the families of healthy children have more money than those of sick children
(less spent on medicine etc). Hence, MFK contributes to the material well-being of the
families of the healed children. But how large is that contribution towards the improve-
ment of their quality of life? Answering this question remains a challenge. They try
-
lenging exercise.
4.3 Practical Tips

• It is not advisable to “pick and mix” indicators from existing databases without the back-
ground work associated with following the impact measurement process. It may save
some time upfront, but you are more likely to conclude that you are wasting resources
collecting data on irrelevant points if you have not gone through the process.
•
databases to see whether some are aligned to yours.
•
if the theory of change of a SPO states that providing language training to economic
migrants will empower these people, making them less dependent on government
provided services and therefore reduce costs for the government; then a potential indicator
would be the number of people from this community using government services. In the
short term this number may increase as enhanced language skills mean the people can
now ask for these services, however in the long-term the number may reduce.
• Always try to include at least one non-self-reported indicator for each outcome.
•
measurement on calculating outcomes and acknowledging those factors that could

increase or decrease impact.
• To pick up unintended consequences review your activities periodically as part of the
monitoring and evaluation process (further discussed in Step 5) and then assess what this

•
the portfolio level is a hot topic in impact measurement at the moment and there is no
common practice yet.
•
important to assess the impact of the work of the VPO/SI on the SPO.

JUNE 2015 71

• The VPO/SI should ask the SPO to focus on those indicators that are directly related to
-
tional indicators required for the VPO/SI to satisfy their impact measurement needs
should be collected by the VPO/SI.
• Clarify at the beginning of the relationship (i.e. during due diligence and within deal
structuring) who is responsible for measuring what. The responsibilities of who measures
what could and probably should evolve over time as the SPO grows and develops and
should be reviewed on an annual basis. The expected outputs, outcome and impact, and
made in the business and impact model of the SPO during the investment process.
4.5 Worked Example
in a position to consider outputs, outcomes and impacts, as well as the appropriate indica-
outcomes and impacts.
Business Model
Inputs Activities
Equipment: sanitation centres, vehicles for collection, digester to process Installing toilets
faeces to fertilisers to generate electricity
Recruitment of
of sanitation centres and selection of franchisees, employees to collect franchisees
waste products and transport to digester, operators of digester to
produce electricity and fertiliser
Partners: implementation partners for education about sanitation, Sale of sanitation

technical partners in design of toilets and digesters / composters, services
(via franchisee)
Funding: grants and investments from foundations and social investors Waste removal,
collection and
processing
Electricity generation
Fertiliser production

Expected Effects
Outputs Outcomes Impact
Number of toilets installed Increased access to sanitation Improved physical well-
facilities for slum dwellers being (reduce disease)
$ revenue from toilet sales
Number of toilet operators Increased employment levels Improved material well-

among slum dwellers being
Improve health for toilet users Improved physical well-

total) and overall slum being
Number of visits to toilets Increased income for toilet Improved material well-
operators being
$ income of toilet operators
Kg waste collected (assuming Improved environmental Improved physical well-

kg processed = kg collected) situation in slums (less waste in being
waterways)
kWh of electricity produced Decreased number of power Improved energy security

shortages / outages Improved environment
$ revenue from electricity sales
Decreased carbon emissions
Kg of fertiliser produced Decreased reliance on costly Improved material well-

imported fertilisers being
Kg of fertiliser sold
Decreased reliance on chemical Improved environment
$ revenue from fertiliser sales
fertilisers
-
tions of the SPO, we need to set certain output indicators. These output indicators need to
be in line with the theory of change shown above and to promote standardisation we will
look where possible to use the IRIS indicators.

JUNE 2015 73

Output IRIS Indicator

Number of toilets installed PI9601: Number of units installed by the SPO during the
reporting period
$ revenue from toilet sales PI1775: Revenue from the sales of the product or service
during the reporting period
Number of toilet operators PI2758: Number of micro-entrepreneurs distributing the
Number of visits to toilets PI8783: Average number of client visits to facilities during
the reporting period
Number of users of toilets (per toilet PI4060: Number of individuals who were clients during the
reporting period
$ income of toilet operators PI4881: Total earnings generated by the micro-
Kg of waste collected Not within IRIS so indicator created as: Number of kgs of
waste collected from the toilets during the reporting period
kWh of electricity produced PI8706: Energy produced during the reporting period
$ revenue from electricity sales PI1775: Revenue from the sales of the product or service
Kg of fertiliser produced PI1290: Amount of product or service produced by the

organisation during the reporting period
Kg of fertiliser sold PI1263: Amount of the product or service sold by the

organisation during the reporting period
$ revenue from fertiliser sales PI1775: Revenue from the sales of the product or service
Despite being important for monitoring the operations of the SPO, these output indicators
do not necessarily tell us whether the SPO is making progress towards its outcomes. To
naturally focus on those outcomes related to physical and material wellbeing, over and
above those related to the environment. We mentioned previously that the stakeholders of
therefore focus on the following outcomes arranged according to the themes of material
and physical well-being.
Improved physical well-being:

1. Increased access to sanitation facilities for slum dwellers
2. Improved health for toilet users and overall slum
3. Improved environmental situation in the slum (less waste in waterways)

Improved material well being:

1. Increased employment levels among slum dwellers
2. Increased income for toilet operators
Given the technical equipment needed to test the level of sewage in the slum waterways
the VPO/SI decided to focus on the remaining four outcomes. With each of these you need
to think of the two to three issues that evidence the situation is sub-optimal at present in
order to select appropriate indicators.
For increased access to sanitation facilities, two appropriate outcome indicators can be
found among the output indicators detailed above:
• Number of toilet units installed by the SPO during the reporting period.
• Number of individuals that were clients in the reporting period.
However an important indicator to add to these two would be to understand how the sani-
tation situation has evolved generally:
•
(including latrines etc.) units installed (by the SPO or by any other organisation) during
the reporting period.
For the improved health of the toilet users and slum dwellers, the users may have to be
surveyed so as to collect data on the following indicators:
• Number of days a toilet user has not been able to be up and about during the reporting
period due to some stomach related illness (deliberately left broad to include the possi-
bility of diarrhoea, intestinal worms etc.).
• Number of outbreaks of typhoid or cholera in the slum area served by the toilets during
the reporting period.
• Average Number of days a slum dweller has not been able to be up and about during the
reporting period due to some stomach related illness.
For the increased employment levels among slum dwellers, it is important to track the
following indicators:
• Proportion of community with some form of regular income through full time and part
time work as at the end of the reporting period.
• Number of employees (toilet operators, waste collectors etc.) of the SPO, including
full-time and part-time (but not temporary), as at the end of the reporting period that
reside in the community where the toilets are situated.

JUNE 2015 75

For the increased income of the toilet operators, another of the output indicators can be
used as well as two indicators that point towards increased wealth:
•
services.
• Proportion of toilet operators with all their children attending school.
•
(e.g. iron, aluminium, tile, concrete, bricks, stone, wood).
Of the eleven outcome indicators selected, three are also used as output indicators and
one other (SPO employees from the slum) should be relatively easy for the SPO report.
However the remaining seven, which are required to show the progress (or not) towards
the target outcomes, require further investment of time and resources (e.g. information
gathering via surveys) on the part of the SPO. Given the SPO is claiming these outcomes
they should be willing to spend the time necessary to collect the data. However in very
early stage entrepreneurs it is important for the VPO/SI and SPO to agree when this level
of reporting should begin, although the VPO/SI should not be too accommodating in this
respect.

STEP 4: VERIFYING & VALUING IMPACT
5.0 Step 4: Verifying & Valuing Impact
5.1 What?
management for a long time. The initial focus of this research and within commercial
organisations was on the quality of the product offered i.e. the process of how the product
was made or the service was provided. Management then realised that quality alone was
not enough to satisfy the customer so the focus moved to client satisfaction. Today the
focus has moved again, to demonstrating the value the customer gets from the product or
service. The importance of putting customer value at the centre of your assessment is not
-
tions and VPO/SIs.
(i) VPO/SI level: as a VPO/SI you believe you are creating value by providing non-
SPO values this assistance, you cannot credibly make that statement. It may also be
necessary for VPO/SIs to verify at regular intervals that the expectations of other stake-
holders (donors/investors and human resources) are met so that corrective actions can
be undertaken if necessary.
out-comes and indicators (Steps 1–3), we need to know whether we are really making
progress towards the desired change and the desired outcomes. We need to know
the expected amounts.
The focus of this step is predominantly on the second level, the SPO, given there are
generally more challenges in this area. However VPO/SIs should not overlook the impor-
Given that, with respect to the SPO level, we may have different stakeholders with different
expected outcomes; we need to verify the results at the level of these stakeholders. This can
be a time consuming activity, so it is preferable to start with your most relevant stakeholder
In addition, when we verify whether the outcome makes sense for the stakeholders and
if the expected outcomes are realised (within the timeframe and quantity expected) we
also need to verify whether this outcome was important i.e. valuable to the stakeholder(s).
The latter is what we call “valuing impact”. In other words: we need to verify whether the
claim we make on having positive social impact is true, and if so, to what extent (i.e. to
what value). The responses to these questions
and associated indicators, creating a positive feedback loop in the impact measurement
process and enabling us to effectively manage impact.

JUNE 2015 77

Clearly at this point we are assuming that the SPO is a going concern and given the work
done in developing the services that the SPO already knows that the outcomes are of some
value. Otherwise one would question why the SPO has implemented the products or
Value is (whoever that may

be). VPO/SIs and SPOs normally incur costs, to create value for other stakeholders (i.e. the
incurred by one stakeholder in order to create value for another. This is one of the reasons
Net VC
Benefits Sacrifices
Attributes Outcomes Monetary Non Monetary
Relationship
Goods Quality Strategic Benefits Price
Costs
Service Quality Personal Benefits Search Costs Psychological

Costs
Core Product
Social Benefits Acquisition Costs Time
Features
Added Service
Practical Benefits Opportunity Costs Effort
Features
Customisation Financial Benefits Distribution Costs
(Marketing VC) (Derived VC) Learning Costs
Costs of Use
Maintenance Costs
Disposal Costs
Source: Woodall, 2003,
(When reduced = Sale VC) “Conceptualizing Value for the
Customer”

This step is also important in assessing whether the SPO has improved its product or
service delivery post VPO/SI intervention (this is why it would be important to make such
an assessment at the beginning and the end of the intervention). The choice of method
depends on the mind-set of the VPO/SI, the characteristics of the investors (whether more
or less focused on numerical or emotional value), and resources available.
Additionally, verifying and valuing impact helps identify the impacts with the highest
social value, which can help the SPO and VPO/SI focus their resources towards initiatives
that create most impact on society.
Verifying and valuing results should not only be done at the last phase of an investment: it
should be repeated as a “reality check” at several points during the investment and value
creation process of a VPO/SI. We recommend that this step be performed at the beginning
of an investment (as part of the due diligence), at least once during the investment period
(to check that the impact is achieved and valued) and again at the time of exit (as a way to
check that the desired impact has been achieved and makes sense).
One question that is often raised is who is responsible? At the level of the VPO/SI, it must
be the VPO/SI that takes responsibility for verifying and valuing the impact of their non-
too busy or do not have the time or the incentives to do it. VPO/SIs often do not want to
“burden the investee”. It is up to the VPO/SI to encourage the SPO to dedicate the time
and the resources to this step, given it adds more credibility to any information provided.
Unfortunately in practice, many investors in the social sector tend to “trust their gut
feeling” rather than invest in the verifying and valuing process. We hope that this step
that the outcomes are valuable to the key stakeholders.
5.2 How to?
Verifying results
What we need to verify is what has been developed through the rest of the impact measure-
ment process.
Level of VPO
the value they provide to their SPOs as directly questioning investees may be a delicate
matter not always providing truthful answers.
Level of SPO
With respect to the SPO these are the outcomes that the SPO plans to or claims to be deliv-
ering i.e. how the key stakeholders are / were affected by the work of the SPO.

JUNE 2015 79

In other words, in both cases, we are triangulating the information we have received by
verifying it against other sources.
There are three principal approaches for this:
(i) Desk research

By looking at external research reports, databases, government statistics etc. it is
indicators. This can be done by the VPO/SI and/or SPO or can be outsourced. For the
VPO/SI this desk research can and should occur at various points in the investment
process. Prior to investment this information provides data on the size and impor-
tance of the issue and establishes a base case. During the investment this data is useful
for triangulation purposes.
(ii) Competitive analysis

We can compare the data of the organisation with the data of other comparable organ-
isations operating in similar geographies on similar issues. We can ask the question
of whether the activity has been tried before and what were the results and learnings.
But the danger with this method is that organisations may only share “good” results
(iii) Interviews / Focus groups

Probably the best way to verify expectations and results is to ask the stakeholders: by
personal interviews or in the form of focus groups. In both cases you ask your stake-
holder about the results of the intervention. This is particularly the case when the
One of the most crucial issues is to ask questions in a neutral way so as to prevent ”leading
It is preferable to have a neutral interviewer (i.e. relative outsider) asking open questions
such as “What do / did you need?” “What has changed?”
is ensuring that the sample is representative.
We highlight a few references that we believe are useful for this method:
• http://www.roguecom.com/interview/overview.html
• http://techinlibraries.com/cowgill.pdf

Valuing results
There are numerous methods and techniques to measure the value created. They can be
divided in two categories:
• Qualitative
• Quantitative (monetised)
value, instead we focus on a number of the most commonly used methods and describe
more about a particular technique.
(i) Qualitative
• Storytelling
Almost all organisations use storytelling in one way or another. These stories can be
-
tured approach which describes the outcomes of an intervention / investment from the
point of view of a stakeholder. Through structured interviews, stakeholders are asked
about their experiences with the organisation. Every interview is executed with the same
framework of questions. Finally a picture (story) will emerge about the change that
the particular stakeholder experienced. A number of frameworks are available on the
internet to help create a structured interview and hence effective story.
Website: http://www.eldrbarry.net/roos/eest.htm
The reason why storytelling is popular is that numbers do not always tell a story, and it
is often easier to communicate the value of an outcome through a story. The downside of
storytelling is that it is generally unclear how many people are having or have had, that
particular experience i.e. the story may not be representative. We recommend the use of
storytelling as one component of valuing; not as the only way of valuing.
• Client satisfaction survey
of research, including online questionnaires, interviews, focus groups, etc. Often this will
be done by an outside organisation.
One Foundation commissions independent feedback from their grantees through a

quantitative survey, carried out by Centre for Effective Philanthropy every second
year. The Grantee Perception Report® (GPR) shows an individual philanthropic
funder its grantee perceptions relative to a set of perceptions of other funders whose
grantees were surveyed by CEP.35
- 35. More information can be found at

cial support provided to their investees. www.effectivephilanthropy.org

JUNE 2015 81

Using client (or customer) satisfaction research can deliver important information on the
value of the product or service to the stakeholders. However do note that “satisfaction”
You should therefore include questions focused on value, for example: how important is
Many organisations ask the question, “How many interviews or how much feedback is
required? 40%? 80%?” In general it really depends on availability of resources. In reality
number of interviews. It is better to have 20% coverage of a good representative group,

than 50% of a non-representative group.
• Participatory impact assessment (focus groups)

This method is popular in developing countries with target groups that cannot read or
write as it makes it possible to rank preferences among stakeholders through the use of
item (the offer from the SPO or VPO/SI for example) is inserted. Participants receive
small stones as “money” and can rank their preferences by paying more or less stones to
the different products.
• Progress out of poverty index (Grameen Foundation)

We have already described this method in Step 3 where the focus was on selecting appro-
priate indicators. In Step 4 we would use the PPI as a measure of the effectiveness of the
group in question.
(ii) Quantitative (monetisation)

Different stakeholders are likely to require different techniques. We identify two
principal techniques:
• Perceived value
• Cost-savings / cost reallocation
-
niques in themselves, rather frameworks using one of the two techniques.
In the case study of Esmée Fairbairn Foundation investing in the social impact bond
managed by Social Finance, it is clear that the bond focused on cost-savings because
this was the governments focus and because the buy-in from the government was
crucial to the whole structure.

For individuals and/or target populations cost-savings are hardly ever relevant as it is
not generally the individual or broader population who is bearing the cost. For them,
perceived value methods should be use. On the other hand, governments, institutions
and organisations generally prefer cost-saving methodologies, given this is their focus.
across the portfolio. But be aware that this can only be done if in each case you are
looking at either values or costs i.e. they are genuinely like-for-like quantities.
(a) Perceived value

• Perceived value / revealed preference
These techniques infer prices from related market-traded goods. The idea is that people
these methods may be referred to as contingent valuation methods. Basically they

address two main questions to infer:
(i) Willingness to pay

(ii) Willingness to accept
Because these methods use “money” in the research, the answers of the respondents may
be biased: either they give strategic answers (lower value when they are afraid their will-
they may give unrealistically high answers.
• Mitchell, R. and R. Carson, (2005). “Using surveys to value public goods; the contingent
valuation method”. Washington USA.
• Champ, P., Koyle, K. and Brown, T., (2003). “A Primer on nonmarket valuation”. Dordrecht
(NL): Kluwer.
• The Value Game
combines participatory impact assessment (as described above) and the willingness to
pay-method, without the “money-component” of willingness to pay. Participants rank
pictures with relevant products, and a picture of the service/activity/impact with
unknown value, in order of their preference. The ranking gives information about the
rating of the service and can be compared to the (money) value of the surrounding products.
More information on Value Game can be found at www.valuegame.org

JUNE 2015 83

(b) Cost-savings
• Cost-saving methods / stated preferences
most commonly used methods are:
- Prevention costs method: for example when a new hospital treatment results in a shorter
stay in hospital for the patient, hospital costs will be prevented.
- Travel cost method: for example the costs people are incurring to get to a service. These
costs indicate the minimum price they are paying to receive the service.
- Hedonic pricing model: is measuring the value of a change, resulting from changes in
the environment. For example: a house has a value of 1 million euros. When an airport
is built right beside the house, the value may drop down (although the house is still
the same).
- Well-being valuation: a recently developed technique for valuing the effect, in monetary
-
lating the compensating variation necessary to maintain the same level of well-being
after suffering from a particular health problem, and is hoped to offer a solution to the
problems of revealed preference and contingent valuation methods. Ref.: www.ncbi.
nlm.nih.gov/pubmed/17380470
- These methods give a good indication of the volume of value created, and are popular in
•
The basic idea of a QALY is straightforward. It takes one year of perfect health-life expec-
tancy to be worth 1, and regards one year of less than perfect life expectancy as less than
1. The QALY is based on the number of years of life that would be added by an interven-
tion. Each year in perfect health is assigned the value of 1.0 down to a value of 0.0 for
death. If the extra years would not be lived in full health, for example if the patient were
to lose a limb, or go blind or have to use a wheelchair, then the extra life-years are given
a value between 0 and 1 to account for this.
Although one treatment might help someone live longer, it might also have serious side
effects. For example, it might make them feel sick, put them at risk of other illnesses or
leave them permanently disabled. Another treatment might not help someone to live
as long, but it may improve their quality of life while they are alive (for example, by
reducing their pain or disability). The QALY method helps to measure these factors so
that we can compare the cost of different treatments for the same and different condi-
tions. A QALY gives an idea of how many extra months or years of life of a reasonable
quality a person might gain as a result of treatment.

QALYs have been criticised because there is an implication that some patients will be
refused or not offered treatment for the sake of other patients and, yet such choices have been
made and are being made all the time. However big the pot, choices still have to be made.
Further information on QALY can be found at:

http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/QALY.pdf
The above list is not and is not meant to be conclusive. We have highlighted some of the
often-used methods that we think are most prevalent and most useful. More information
on these techniques (and many others) can be found in the TRASI database: http://trasi.
foundationcenter.org/
Pieter Oostlander from Shaerpa shared that, “What we do in relevant cases is:
• Search for academic research that supports claims that are made on impact in the
analysis;
•
or organisation are actually acknowledged by those groups;
• Search for statistical information and academic research to underpin the values of
stakeholders to assess the perceived value with them.”
5.3 Practical Tips

• Take this step (more) seriously as it may prevent poor investments, and can create a
learning and entrepreneurial environment.
•
example: learning and growth of SPO).

•
•
• Your choice of quantitative or qualitative techniques or a combination of both to value
by the prioritisation you assign to different stakeholders.

• The amount of time you will need to verify and value impact should be budgeted upfront
in your time plan for the year (VPO/SIs in the Expert Group suggested 5–7% of time) as
it is vital to talk to people to ensure that impact is being achieved.

JUNE 2015 85


• For a VPO/SI, verifying and valuing results should not only be done at the last phase
of an investment: it should be repeated as a “reality check” at several points during the
investment and value creation process of a VPO/SI.
• Make clear determinations between the SPO and VPO/SI regarding who is responsible
for which parts of the verifying and valuing process.
• Unless you verify whether you have created value through your support of the SPO, you
cannot credibly make that statement.
• VPO/SIs should use independent studies to assess the value they provide to their SPOs
as directly questioning investees may be a delicate matter not always providing truthful
answers.
• VPO/SIs should verify at regular intervals that the expectations of other stakeholders
(donors/investors and human resources) are met so that corrective actions can be under-
taken if necessary.
5.5 Worked Example

The VPO/SI in our worked example wants to verify and value the technical assistance
ensures that it tracks all the pro-bono assistance it provides to the SPO in terms of type,
hours, and where possible assigning a $ value to how much that assistance would cost if
it were to be purchased on the market. On an annual basis it surveys all its investees to
developed with the help of an external consultant. However it now does the surveying and
the necessary tweaking itself to reduce costs.
At the same time the VPO/SI collects as much data for the verifying process as possible by
using desk research combined with competitive analysis. For example it tracks government
data on disease outbreaks in and around Nairobi as well as keeps an “ear to the ground” on
the activities and results of any similar companies working in a similar setting (although
not necessarily the same country).
they would consider (perceived value or Progress out of Poverty Index). Given the early
and consolidating its sales. Additionally the required outcome indicators already take up
other requests at this early stage.


STEP 5: MONITORING & REPORTING
6.0 Step 5: Monitoring & Reporting
6.1 What?
through the indicators set in the third step; and reporting – transforming data into present-
able formats that are relevant for key stakeholders. Monitoring and reporting are iterative
processes that go hand in hand because what is monitoring to one stakeholder is reporting
to the other, e.g. when a VPO/SI is monitoring the progress of an investee SPO, that SPO
is reporting relevant data to the VPO/SI. When considering monitoring and reporting we
again consider the step at two levels: the VPO/SI and SPO.
(i) Monitoring
sense to the key stakeholders, they need to start collecting data in a systematic manner to
considered upfront, however we urge organisations to go through the impact measure-

ment process at least theoretically prior to setting up the system so us to understand the
type of information that needs collecting and therefore avoid technological related issues
at a later stage.
A VPO/SI needs to systemise the data it tracks (from the SPO as well as independently)
The SPO needs to collect and track data related to the indicators set in Step 3 and the
A VPO/SI should also gain an understanding of the ways its investees are already gathering
data and assess whether or not the relevant data is collected in a systematic way. VPO/SIs
need to monitor the information received from the SPO (timeliness, completeness, quality
of information provided etc.). In fact some VPO/SIs go a step further and look beyond
to monitor in the future, if the SPO is able to develop according to their strategic plan.
A VPO/SI also needs to monitor data about its own activities as a high engagement
each investee and the total costs thereof. It also involves tracking how the support is used
by the SPO and where the gaps are.
to be processed, performing the necessary analyses to gain a better and more complete

JUNE 2015 87

implemented. The organisation uses the data collected to analyse the results against the
The VPO/SI needs to analyse its role in the change process asking questions such as: Is the
there resources available to implement such corrective actions?
(ii) Reporting
Once the data has been collected and analysed, an organisation needs to consider how to
present this information. The purpose of reporting affects the information that should be
included. Depending whether the focus is on an internal or external audience, the various
stakeholders may require different types of reports. The stakeholder analysis conducted in
One of the challenges of the social sector is that each SPO needs to report in different ways
to each funder. Some initiatives (for example the Social Reporting Standard) are trying to
overcome this problem, but there is still a problem of lack of standardisation that leads to
Social Reporting Standard

An initiative to deal with multiple reporting requirements is the Social Reporting
Standard (SRS)36. Apart from providing guidelines on reporting, it provides the
following framework for impact-oriented reporting of SPOs:
1. Problem to be solved
2. Scale of the problem
3. Contribution of the organisation to a solution and expected impact
4. Actual social impact
• resources used (input)
• work performed (output)
• impact (incl. outcome)
5. Plan and outlook
6. Organisation
7. Finances
6.2 How to?

A single system can be used that includes functionality for both monitoring and reporting,
so that the monitoring system is set up to produce reports, or different tools can be used
for each part.
36. www.social-reporting-standard.de

(i) Monitoring
Level of VPO/SI
Monitoring at the VPO/SI level is not yet well developed in many cases, whereas the moni-
toring of individual investments is a much more widespread practice. Depending on how
the VPO/SI measures impact at portfolio level, the data collection needed and the necessary
analysis will be different. It is important that the approach used is coherent. For example,
of a certain population, data should be collected at regular intervals to assess changes in

that particular population, and an evaluation should be made as to the contribution of the
VPO/SI to that change. Implementing the monitoring into the standard internal processes
of the VPO/SI and assigning a person responsible for this aspect should go some way to
addressing the problem.
Auridis, for example, has developed an investee database using Microsoft Access. It
Auridis does not aggregate output, outcome, or impact data because the indicators
are not comparable across the portfolio. Some very basic aggregate indicators such as
“number of lives touched” can be aggregated across its investments.
PULSE37 is a numeric metric data collection and reporting tool and was created between
Acumen Fund and engineers from Google.com and developed with help from the Skoll
Foundation, the WK Kellogg Foundation, the Lodestar Foundation and Salesforce.com
Foundation. A VPO/SI that works with multiple organisations, and has metrics to track
and report back to your stakeholders, can use PULSE to facilitate the process.
understanding of the data already collected by the SPO and assess whether the data is of
indicators, but not on outcomes.
The key recommendation for any VPO/SI is not to ask the SPO to collect data that will not
be useful to the management of the SPO itself. The danger is to start asking the SPO for long
lists of data that take time and effort to collect, when in the end only some of this data is
truly relevant. This is why it is so important for both VPO/SIs and SPOs to go through the
37. http://pulse.app-x.com

JUNE 2015 89

relevant indicators, before starting to collect data. If an organisation discovers at this stage
3 and reconsider the indicators to align them with the real situation at hand.
There is a need to evaluate if the SPO is effectively monitoring its activities and outcomes
e.g. are the selected indicators appropriate (providing a balanced picture of the situation
and picking up potentially positive and negative aspects) and if the VPO/SI has a role
to play in improving the impact measurement practices of the investee. If the VPO/SI
provide key resources.
In the case of Papilio, the investee of Auridis highlighted in the case study on Step 5,
the investee itself developed an information system to collect relevant data. The Papilio
team previously used a mix of Excel sheets, Word lists, and paper lists spread all over
funder, the Papilio team started to develop their own information management system.
The recommendations when implementing an information management system are as
follows:
• Usability is the key success factor of any system.

• The underlying processes are more important than technology, which is why the
system has to be well planned.
• Unless the SPO has relevant technological skills, it is advisable to hire a separate IT
consultant to implement the system.
• The whole SPO team and some of the other (external) users need to be integrated in
to be useful for them.
The development of an information management system needs an iterative process and

a lot of end consumer testing and reversing.
Level of the SPO

The SPO needs to evaluate the outcomes or impacts that are being achieved through the
activities of the organisation and the practical lessons that can be learned from the results.
With this information it is then possible to decide what actions are needed to increase
impact.
38. http://1068899683.n263075.test.
For example, in the social balanced scorecard developed in the UK for social enterprises38,
prositehosting.co.uk/wp-content/
- uploads/2013/03/SEL-Balanced-
Scorecard-article.pdf

It provides a template to be used as a management tool for the social enterprise. An example
is included in the table below:
Outcome: Promote financial sustainability

Indicators Targets Targets Targets Targets Initiatives Owner
Q1 Q2 Q3 Q4
Increase trading % of income 1% of 2% of 3% of 4% of Launch of Director of

income from new overall overall overall overall online shop marketing
budget budget budget budget
services
For Streetfootballworld, it is important to harmonise all monitoring and evaluation

activities with the theory of change and the strategic priorities of an organisation. There
is no point in collecting data that is not used either for reporting or for learning and
improvements. They have often encountered data overload on the one hand and a lack
of relevant data that helps organisations respond to internal or external questions about
their programmes on the other hand. If an organisation has a clear theory of change and
-
isation is still on track towards achieving them. In addition, it is essential to consider
community and investor requirements from the start to avoid adding additional data
collection, which often results in isolated forms and reports rather than synchronised
work streams. Involving stakeholders (communities/target groups, investors, board
etc.) in the development of the theory of change and the resulting monitoring and eval-
uation system is a very good practice to ensure backing and support for what you want
investors or partners who want to know different things about the programmes, and
what communities/customers consider effective and necessary.
(ii) Reporting
For a VPO/SI, reporting can be external or internal, but generally it is related to reporting
to donors or investors. This reporting has different levels of detail depending on the stage
of the investment process. At a deal screening phase, the report to investors includes a low
level of detail, whereas much more information will be reported on after a due diligence
has been conducted. Once the investment has been made, the agreed-upon impact targets
should be communicated to investors. During the investment period, reporting should
allow investors to determine whether impact targets are being met, and at the end of an
investment, a detailed report should be completed with more long-term impacts included
and how the VPO/SI has helped the SPO achieve those. A VPO/SI should also consider
how to report the progress of its entire portfolio.

JUNE 2015 91

VPO/SIs and SPOs should agree before the investment the level of reporting required.
Considerations include:
• What to report on: which information should be included in the report?

• Format of reporting: which format can easily be used by the SPO based on the management
system they have developed, and is clear and transparent for the VPO/SI?
• Ownership: who is responsible for reporting within the SPO?
• Frequency and time horizon of reporting: how often (monthly, quarterly, annually, etc.)
should the reporting take place and what should be the time frame included for compar-
ison (one year, three years)?
If the VPO/SI co-invests with other funders, they should consider the possibility of devel-
oping common reporting frameworks so that the SPO is spared the burden of multiple
requirements. As long as the funders are able to extract the necessary information from the
report, they should not necessarily push their own format.
Some indicators may be reported more frequently than others. Typically, output indica-
tors can be captured more frequently than outcome indicators that might require more
time and effort to collect relevant data. The SRS recommends that the reporting period
-
duced Pulse in their Salesforce tool to capture the key indicators. Investee organisations
frequency of reporting is every 3 months. Social impact might be reported less often, as
frequently must be balanced.
Ferd Social Entrepreneurs (“Ferd”) do not expect the social entrepreneurs to report
every month - only once or twice a year, and then maybe spending a couple days at
a maximum each time. For them it is crucial to keep the amount of time as low as
possible (for many SPOs the gathering of output data is more or less automated). For
Ferd, they spend time collecting data perhaps 3-4 weeks a year, as well as a lot of time
more time. Standard reporting formats for both SPOs and VPO/SIs may help a little,
but Ferd believes they need to tailor the reporting for each of the SPOs to make it as
relevant as possible for their business.

PhiTrust does not ask for social impact reporting on a monthly basis, but rather some
top-line criteria every six months and a more in-depth look at the impact performance
intensive process for some of their investees (notably those who deal with an array of
stakeholders) and the information is not always easily collected. PhiTrust have seen
that many of their investees have or are moving towards creating internal reporting
metrics and at least annual social impact/performance. The internal format allows them
to both take on the process formally in-house with all of the internal buy-in necessary
as well as produce one document that they can share with all of their investors. For
investees choosing to develop these internal formats, PhiTrust have worked closely
with the entrepreneurs in the development of documents to ensure that both the pres-
entation format and content is as relevant as possible to their own needs.
in their collaboration and collective efforts. For them good reporting should always
progress made towards their desired outcomes, as well as key lessons learnt. While they
strongly believe in standardising reporting requirements, they also see some limitations
in its form and cover a high percentage of the basics, so that only a small amount of
additional information needs to be adapted for other stakeholders. At Streetfootball-
world they use SRS as a basis for reports to Ashoka and the Schwab foundation. They
have also observed an increasing openness by investors to accept existing (standard-
ised) reporting formats rather than asking for burdensome additional information and
strongly encourage existing and new investors to support such proposals.
The Global Reporting Initiative39

quality:
• Balance:
performance to enable a reasoned assessment of overall performance.
• Comparability: Issues and information should be selected, compiled and reported
consistently. Reported information should be presented in a manner that enables
could support analysis relative to other organisations.

• Accuracy:
39. “Sustainability Reporting
Guidelines”; Version 3.1; Global
Reporting Initiative.

JUNE 2015 93

• Timeliness: Reporting occurs on a regular scale and information is available in time for
stakeholders to make informed decisions.
• Clarity: Information should be made available in a manner that is understandable and
accessible to stakeholders using the report.
• Reliability: Information and processes used in the preparation of the report should be
-
tion and that establishes the quality and materiality of the information.
6.3 Practical Tips

• Early stage SPOs may not be ready to implement a complex monitoring and reporting
system – start with simple (e.g. Excel) and increase the level of sophistication as the
organisation matures and can free up resources.
• For the VPO/SI, it can be challenging to aggregate results across the portfolio with many
different systems and types of impacts. Some tools such as Pulse may be helpful.
• Try to share the development of systems with others, and do not invest on your own
formats e.g. Excel.

• -
cies. The Social Reporting Standard is a positive step in that direction.
6.4 Recommendation for Managing Impact

• To remove a reliance on and/or culture of “gut feeling”, the VPO/SI should work with
the SPO to develop an impact monitoring system which can be integrated into manage-
happens at the same time), tools to be used and responsibilities.

• -
ments – if not, be prepared to contribute to improving it through pro-bono partners or
-
ment tool!
•
the investor in order to decide how costs should and/or could be split.
• When working with very early stage SPOs and helping them develop business plans,
integrate requirements on impact measurement at this stage.
• Agree on reporting requirements upfront with SPO and co-investors to eliminate multiple
reporting for SPOs.
• Manage expectations about frequency and level of detail for reporting, and the way
what frequency)?

6.5 Worked Example

In our example, given the SPO is at a very early stage it is sensible to begin collecting data
through the use of spreadsheets. The VPO/SI can take the initiative with other investors
and create a template for reporting that the other investors are also happy to receive. This
technical assistance to the SPO, one such assistance could be the use of pro-bono consult-
ants to help the SPO develop a more robust (vs. Excel) internal monitoring system to facil-
information.
The VPO/SI itself should have some form of internal system for collecting and aggregating
data (where feasible). Given the VPO/SI is also a young organisation, this system can begin
as an access database, but given the VPO/SI has plans in the short term to implement a
“Salesforce” style CRM system, moving the monitoring to a system based on Pulse may be
a good medium term option.

JUNE 2015 95
MANAGING IMPACT
7.0 Managing Impact
The goal of impact measurement is to manage and control the process of creating social
impact in order to maximise or optimise it (relative to costs). The impact measurement
generated through its investments. To manage impact, the VPO/SI should continuously
impact measurement process as lessons are learned, additional data is collected, or the
learning process.
Throughout the document, the impact measurement process has been related to the invest-
ment management process of the VPO/SI. Given that most VPO/SIs are aiming to maximise
impact, the corrective actions taken may apply as much to the investment management
process as to impact measurement itself. In the table that follows, the components of the
impact measurement process have been integrated into the overall investment process of
Managing impact
measurement integral to their investment process. Such an approach may facilitate the end in the investment
goal of maximising impact: process
Investment process
Investment
strategy Deal Due diligence Deal Investment
Exit
screening (detailed screening) structuring management
Decide on the Assess whether invest- Dig deeper into Map outputs, out- Regularly assess Perform thorough
overarching social ment opportunity fits questions asked in comes and impacts impact results against analysis of impact
impact objectives with VPO/SI strategy setting objectives. and decide on key key indicators. results against
of the VPO/SI – by asking questions indicators against objectives –
these will guide the detailed in setting Perform stakeholder which progress will Verify and value verifying
investment process. objectives. analysis. be measured. reported results at and valuing
regular intervals. reported
Verify and value Decide on monito- results.
expected results. ring and reporting Revise indicators
content and frequency if significant changes
and assign responsi- are made in the
bilities. business and impact
model.
Source: EVPA
Several VPO/SIs that have worked many years on impact measurement, such as Noaber
Foundation and LGT Venture Philanthropy, have fully integrated impact analysis into their
investment process. Although the aim is for the impact measurement to become an integral
part of the investment process so that it is used by all VPO/SI team members on a daily
basis, it is useful to have someone responsible for the overall process. Drawing from the
recommendations on managing impact developed for each step in the impact measurement
process, the following elements should be taken into account when developing an invest-
ment strategy and for the investment process as a whole.
MANAGING IMPACT
Investment strategy
issue is that the VPO/SI is trying to solve. A clearly articulated response is necessary to be
able to choose investments in SPOs that can contribute to solving the social issue that the
VPO/SI is addressing. For the impact measurement process, the VPO/SI needs to consider
this question clearly before starting to make investments, and regularly revise and adapt
-
tives, the VPO/SI should consider what changes it wishes to achieve as opposed to the
SI. The role of the VPO/SI will be to provide the SPO with the support needed to help the
For the VPO/SI, it is important to get the buy-in of key stakeholders (donors/investors,
key stakeholders should happen upfront by making sure they understand and support
-
-
tives continue being aligned, and otherwise implement corrective measures.
-
ment of impact at the portfolio level is a hot topic in impact measurement at the moment
and there is no common practice as of yet.
Deal screening
process, narrowing down the type of SPO that will be considered for investment. For each
potential investment, it is important to evaluate the expected outcome of its investment in
the SPO, i.e. the expected outcome of the SPO and how the VPO/SI expects to contribute
to achieving that outcome.
Due diligence (detailed screening)

At the due diligence stage, it is vital to gain a detailed understanding of the current and
expected social impact of the SPO. It not only reduces the risk of making the wrong invest-
ment, but also creates a common understanding of the impact of an organisation among all
stakeholders and allows the VPO/SI and SPO to “speak the same language”.
Stakeholder analysis should be an integral part of the due diligence phase. To avoid wasting
resources, it is advisable for VPO/SIs to increase the intensity (i.e. more stakeholders, more
involvement from the same stakeholders and higher numbers involved from each group

JUNE 2015 97
MANAGING IMPACT
(up to the number required for a non-biased and random sample) of the analysis as it
becomes more likely that the investment will be realised.
If a SPO is claiming a certain outcome then they need to prove it. If the SPO cannot deliver
the data then the VPO/SI must consider whether they will bring in the expertise and
provide the necessary support so the system for data collection can be set up (although not
necessarily assisting the SPO in collecting the data per se) or question whether the SPO is
an appropriate investment at all.
It is useful as part of the due diligence phase to check whether the impact monitoring
Otherwise, the VPO/SI may need to contribute to improving it through pro-bono partners
or other resources – and those costs should be factored in before making an investment
decision.
Deal structuring
The resources of any SPO are limited and decisions have to be made about the amount of
time and resources that a SPO should dedicate to impact measurement. An important role
of the VPO/SI is to convince the SPO of the value of impact measurement, provide assis-
exactly what they want to deliver makes it is much easier at a later stage to assess whether
this has been achieved.
It is important to clarify in the deal structuring phase who is responsible for measuring
what. The responsibilities of who measures what could and probably should evolve over
time as the SPO grows and develops and therefore should be reviewed on an annual basis.
The expected outputs, outcome and impact, and the corresponding indicators should be
theory of change and hence in line with their operational process. Any additional indicators
required for the VPO/SI to satisfy its impact measurement needs should be collected by the
VPO/SI. Also make clear determinations between the SPO and VPO/SI regarding who is
responsible for which parts of the verifying and valuing process – and when would be the
appropriate time to revisit stakeholder analysis during the investment period.
To remove a reliance on and/or culture of “gut feeling”, it is essential that the VPO/SI
works with the SPO to develop an impact monitoring system which can be integrated into
impact happens at the same time), tools to be used and responsibilities. The cost to support
and maintain such a system (including personnel time and costs) should be part of the
how costs should and/or could be split.

MANAGING IMPACT
management tool! When working with very early stage SPOs and helping them develop
business plans, it is useful to integrate requirements on impact measurement.
Reporting requirements should be agreed upfront between the VPO/SI and the SPO, if
possible involving co-investors in the decision-making process to eliminate a multiple
reporting burden for the SPO. Managing expectations about frequency and level of detail
process.
Investment management
investment process. Some indicators may be reported by the SPO more frequently than
others. Typically, output indicators can be captured more frequently than outcome indi-
cators that might require more time and effort to collect relevant data. VPO/SIs usually
months or on an annual basis during the investment period. For a VPO, it is not enough to
work of the VPO/SI on the SPO. It is recommended that VPO/SIs use independent studies
to assess the value they provide to their SPOs, as directly questioning investees may be a
delicate matter not always providing truthful answers.
to policy environment etc. It is advisable to get back to the key stakeholders to verify that
their expectations are being met. Verifying and valuing results should be repeated as a
“reality check” at several points during the investment and value creation process of a
VPO/SI. We recommend that this step be performed at least once during the investment
period to check that the impact is achieved and valued.
changes can be made and corrective actions implemented. The VPO/SI together with the
decide which strategies and interventions worked and which did not. The indicators set at
impact model of the SPO during the investment process.

JUNE 2015 99
MANAGING IMPACT
For NESsT, managing impact takes place at the SPO and VPO/SI level. At the SPO level,
the SPO and as signals for both the SPO and NESst for required intervention if things
VPO/SI with their parti-cular “toolboxes”. At the VPO/SI level, they track portfolio
strategy so as to better reach their goals. The most important question for NESsT is how
to decide when to intervene: how far off track to you need to go? What processes can be
put in place? Can this be supported in investment documentation? The NESsT approach
is to review performance data three times a year and that is when discussions about
intervention take place. They also relate their decision to scale or exit a SPO based on
performance and impact data.
Exit
At the time of exit, a VPO/SI should aim to measure the outcomes of the investment against
-
mended in Step 4. The resulting information will be useful for the VPO/SI itself to assess its
success as a “high-engagement” investor and take away learnings for future investments.
It will also be used to report back to donors and investors on the “social return” on their
investment. The impact of the SPO itself may also be a selling argument when “handing
over the baton” to future social investors.
MANAGING IMPACT
The chart below illustrates how LGT VP integrates social impact into the overall invest-
ment process and who is involved and what is produced. Prior to an investment,
during the due diligence process, the principal users of the social impact information
are internal to LGT VP both at a team and board level. However once the deal is in
impact targets. Post investment, monitoring and reporting assesses how the SPO has
performed relative to the social impact targets. Here there are two principal audiences,
internal and external, and LGT VP produces different reports for each group.
Deal screening Preliminary review Deal execution Due diligence
Who uses the LGT VP Team Board LGT VP Team / Board

social impact / Orgs
information?
How is the Initial understanding Deeper understanding Board approval of
social impact of impact of impact engagement
information
Decide if org will be Board approval of
used?
presented to the board resources for local DD
Deliverables Impact model Impact model Investment memo
light (impact targets)
disburse capital
Post-Investment
Portfolio controlling & reporting Exit
Monitoring
Y1 Y2 Y3 Y4
Who uses the
social impact Team + Board + Funders/Investors + Public
information?
How is the
social impact Assess and report on achieved social impact
information (qualitative and quantitative)
used?
Deliverables Impact reports
(internal + public)

JUNE 2015 101
CONCLUSIONS
8.0 Conclusions
for VPO/SIs and other funders interested in generating a positive impact on society. For
that purpose, we researched the various existing approaches, interviewed several VPO/
Expert Group that helped us develop practical case studies. Importantly, we tried not to
be partial to any existing approach, but rather attempted to provide practical recommen-
best practice on impact measurement. We envisage further work during coming years to
provide more high-level guidelines on impact measurement and reporting following this
hands-on practical guide. To support the implementation of our recommendations in the
manual, EVPA will launch a training centre.
A survey40 of 1000 charities by New Philanthropy Capital in the UK in 2012 cited a number
of barriers preventing SPOs from using impact measurement to its full potential. Among
these barriers was the point that SPOs do not know how to decide outcomes or where to
was the fact that different investors ask SPOs for different types of information – over two
thirds asking their investees for information tailored to them. We believe that within this
manual we have the foundations of a shared measurement system for venture philanthro-
build on the content of the practical guide to create a code of good practice, which can then
be disseminated further across the sector.
At EVPA, we encourage our members to work hard to measure, monitor and report impact,
but also to increasingly integrate an impact approach into each important decision along
the investment process, from deal selection to exit. This is why managing impact is at the
core of the impact measurement process. For each step in the process, one should consider
how this relates to the everyday work of funding and building stronger social purpose
organisations. Our aim is for this practical guide to encourage more and better work on
40. Pritchard, D; Ní Ógáin, E;

Lumley, T., (2012), “Making an
impact: impact measurement among
charities and social enterprises in the
UK” New Philanthropy Capital.
PART 3:
Case Studies

JUNE 2015 103
CASE STUDIES
9.1 Step 1: Setting Objectives

Case study: Ferd Social Entrepreneurs investing in The Scientist
Factory (“TSF”)
-
sation TSF supported by the venture philanthropy organisation Ferd Social Entrepreneurs
is trying to solve the issue. It then discusses the impact measurement undertaken by the
Introduction – social issue

The challenge in question in this case study is that we live in an era where natural sciences
and technology develop at a very rapid rate. The gap between what children learn in
school and what is happening in the real world grows bigger every year. At the same time,
research within natural sciences and technology is central and vital for the development
of society. The issues we face in the years to come include climate change, food production
and distribution, and medicine and health. These issues cannot be solved without the use
of natural sciences and new technologies. TSF was founded in 2002 to meet the educational
the number of students who choose an education in natural sciences, and to develop an
interest in research and technology among children and young people.
we did. Ferd itself is part of the Norwegian industrial group of the same name. Ferd recog-
nizes its corporate social responsibility as an integral part of its business activities. We also
consider it natural to play a role beyond this, principally by supporting social entrepreneurs
people – and particularly children and young people – can realise their opportunities and
ambitions. The due diligence process with TSF was initiated by the CEO of Ferd at the same
time as he hired the CEO of FSE.
The investment decision of FSE was based on a “gut feeling” regarding the impact that TSF
would have and a belief in the work of the entrepreneur Dr. Hanne Finstad. After inter-
value for the participants (learning, motivation, fun) and hopefully in a longer term create
a change in the way natural science is taught in Norwegian schools.
How TSF assesses impact: starting point

Our starting point for measuring the impact of TSF before participating in the EVPA initia-
tive was to check what the organisation itself was already doing. In the autumn of 2011,
after TSF had been running for 10 years a web-based survey was launched to seek answers
CASE STUDIES
to what we considered to be the most important question: does TSF matter? The survey was
answered by 75 out of 229 participants who followed a TSF course in the period 2002–2003
(a response rate of 32%, which is considered high for this type of survey).
The following results were found:

• all respondents have positive memories of the course
• 93% are still interested in natural sciences
• 18 respondents (25%) state that the TSF courses were an important factor when they
decided to choose natural science in their secondary education
• a positive correlation between the level of motivation and the interest in natural science
• a positive correlation between the number of courses pupils attended and the focus on
natural science in their secondary education
Motivation and interest in the natural sciences among children are the main goals of TSF,
and are therefore important factors that they have in mind when creating the courses. At
physics and biology. The participants perform experiments and receive a few facts about
the different assignments before, during, and after each course session. Furthermore, TSF
always asks the pupils if they feel that they have learned something from the course. 99% of
the participants say that they learned a lot, or something, and many of them can talk about
the topics they have learned in their own words. Therefore, Ferd has reason to believe that
the courses provide a good learning platform where children gain knowledge, in addition
to giving them exciting and motivating experiences in natural sciences.
In the context of impact measurement the important question for us is whether the survey
provides information about the impact created by TSF.
To consider whether impact is measured with this survey we would have to ask certain
critical questions:
• Can we establish a base case? i.e. What percentage of children generally choose a natural
sciences-focused education later on (secondary/tertiary)?
• What percentage of participants of TSF classes chose a natural science focused education?
• How many of the participants in TSF were already interested in natural sciences?
• How many of them would have chosen to focus on the natural sciences anyway (a concept
that in some methodologies is called dead weight)?
Although the survey can provide us with some information about the % of participants of
TSF classes that choose a natural sciences focused education, it cannot provide answers to
the latter two questions. However some comparison of the TSF group to the general popu-
lation of Norway can probably be made.
One of the suggestions of the Expert Group, and particularly relevant for this step was the
importance of accurately describing the situation at the start of the period under analysis

JUNE 2015 105
CASE STUDIES
changes that TSF would like to achieve.
One of the biggest problems for us is that the typical starting age of participants in TSF
classes is 10–12 years (5/6 graders). The reason for targeting children of this age was that
their interest and curiosity and not lose it on the way to higher education. However when
education (high school) or were at the beginning of their tertiary education (college/univer-
remember their views back in 2002/2003 and think about whether TSF was responsible for
any change in opinion or increase in motivation towards the natural sciences.
Input from the Expert Group – Setting Objectives

From our discussions in the Expert Group we decided to take another look at TSF and how
we could potentially understand and measure their social impact.
Firstly we realised that it was vital to understand why we (as Ferd Social Entrepreneurs)
one owner, so our focus does not need to be directed to a larger external owner group.
Nevertheless we believe it is important to measure impact for a number of reasons:
impact in a country that has a well-developed welfare state. And in addition, that small
amounts of money (as a proportion of the total welfare spend) can achieve quite some
impact.
2. To encourage the social entrepreneurs themselves to measure social impact so they can
improve their sales message and more effectively compete for government contracts or
sell their products / services.
3. To more effectively manage our portfolio. Our focus is on how we scale social impact
(versus scaling the economics per se) so we need to have a very clear understanding
ensure any scaling strategy focuses on these areas.

4. To select investments. Although we have not yet fully implemented social impact
analysis into our investment process we know this is important. We have scarce resources
(in terms of people and available funds) so we need to be sure we are spending our time
and funds with the social entrepreneurs that are generating the most social impact.
5. To motivate other investors to follow a VP approach. We feel that if we can demon-
strate our own social impact then it will be easier for other organisations to work with
dominant player and it would be good to have some other organisations doing the same.
CASE STUDIES
Quote from Johan Andresen41
but also an enormous challenge, to try to measure impact in order

to document that real social value is created, a value that someone,
private, business, or government should be willing to pay for”.
whether TSF is creating a system change in the way in which natural sciences are taught at
primary schools in Norway. It was basically a retrospective evaluation of TSF, which would
allow us to better consider any future investments in TSF and also work with the entrepre-
neur on their scaling strategy.
Secondly, from the Expert Group discussions, it was decided to use the theory of change to
The rationale for this choice was that it is a simple framework that can guide you as you
time. We generally spend a lot of time (in person) with our entrepreneurs so we gain a good
understanding of what they are doing and why. We found that the theory of change was a
good technique / methodology for helping an entrepreneur “get down on the paper” the
key points of what they are trying to achieve. However, it is important to highlight that in
-
With regards to TSF we asked ourselves the following questions:

• What is the social issue TSF is trying to solve and why is it important?
•
• When can TSF be considered successful?
• What would happen without TSF?
41. http://www.forbes.com/sites/
rahimkanani/2012/09/16/
entrepreneurship/

JUNE 2015 107
CASE STUDIES
Our responses were as follows:

• Social issue: There is a lack of young people who choose an education in natural sciences
caused by the way education is delivered: many teachers in primary school are not able
equipment. This has a negative affect on the self-image among children regarding this topic
therefore not enough children choose/focus on natural sciences in their secondary education.
As we made this statement, we realised that there were even more questions that we
needed answered upfront especially to try and understand why this topic is important.
On the one side we can make the statement that research within natural sciences and
technology is central and vital for the development of society. As stated above, we believe
that the issues society faces (climate, food production, distribution, medicine and health)
encouraged to provide sources and support for this statement, which led to more questions.
the macro level questions can probably be answered from generally available statistics but
the mezzo and micro levels would require us to conduct surveys from a large enough sample
Macro level:
• How many young people choose an education in the natural sciences (at the moment)?
•
• i.e. how large is this gap?
• Mezzo level:
•
•
• And what kind of equipment?
Micro level:
• What is the level of interest of the participating children in natural science?
• How many children have a negative self-image regarding this topic?
• And what causes a negative self-image regarding this topic?
TSF Solution: The (after-school) courses provided by TSF are designed to create a long-
lasting inner motivation for natural sciences for children from 9 till 13 years old. There
are 4 to 6 courses within a school year and each course (2–3 hours) focuses on a different
topic. The courses aim to create positive experiences, through fun and creative teaching
senses). Alongside the courses for children TSF also offer, “teach the teacher” programmes.
CASE STUDIES
Success: We decided that there were two ways of considering whether TSF had been
successful. First the implementation of the teaching principles of TSF within the primary
education system of Norway, and second the increase in the number of students who
choose an education in natural science (secondary and beyond).
What would happen without TSF: Despite acknowledging the importance of this question
for us to be able to truly measure impact, we think that this is a question is almost impos-
sible to answer due to the age of participants. What children want to be when they grow
up when asked at the age of 10 is in most cases not what they end up being. One could try
to set up randomized control groups to provide more information and data but we believe
this would be a waste of resources. We therefore will be focusing on comparing the % of
participants in TSF classes who elect natural sciences at the level of higher education to the
average in Norway.
Conclusions
TSF is the key differentiator or not. The participant children may have parents or siblings
with this interest, they may have an exceptional teacher, choose a role model with that type
of background, watch a video clip on YouTube, etc. and one of these events may be the true
trigger. However if the proportion of children from TSF classes choosing natural sciences
state that the TSF courses were important to them – then it is safe to make the assumption
that TSF works. And that is the best answer we think we can get and for us it is good enough!
Recommendations
From our experience and the Expert Group discussions we have a number of recommenda-
tions for other VPO/SIs:
• -
exactly what the social entrepreneur wants to deliver then it is much easier at a later stage
to assess whether this has been achieved. It may be easier to follow a “gut feeling” but it
•
as this then sets the tone for your future collaboration with the entrepreneur. The theory of
change can help the entrepreneur better express their goals, aided by the VPO/SI.
• The investor should convince the social entrepreneur of the value of considering impact
measurement and using the theory of change methodology, even if it means they have a
few hours less sleep some nights!

JUNE 2015 109
CASE STUDIES
9.2 Step 2: Analysing Stakeholders

Case Study: Impetus Trust Investing in Blue Sky
-
ment in Blue Sky Development and Regeneration (Blue Sky).

There is substantial evidence that employment is the single most effective way to enable
an individual to escape the cycle of re-offending. Blue Sky offers ex-offenders “a proper
level positions in the grounds maintenance and recycling sectors for up to six months.
The ex-offenders work in small teams and are supervised by an ex-offender team leader,
who serves as a mentor, and the Blue Sky team provides additional pastoral support (for
choice. As employees near the end of their six months with Blue Sky, they are offered help
In 2008, Impetus invested in Blue Sky, a social enterprise that helps reduce re-offending by
employing ex-offenders and supporting them into sustained onward. Impetus Trust works
to break the cycle of poverty by investing in ambitious charities and social enterprises
plus specialist business support delivered on a pro bono basis plus careful hands-on invest-
ment management in the UK. Since its launch in 2002, Impetus has invested in 24 charities
and social enterprises, helping them achieve average annual growth in income and people
helped of 23% and 30%, respectively.
Working Definition of Stakeholder Analysis
organisation who is effecting and/or affected by the venture philanthropy investment.”
parties that contribute to the change experienced by the ex-offenders, as well as stake-
-
tive, as an investor, the investee is also a stakeholder, one with which it is important for the
investor to build and maintain a positive relationship. When we think about stakeholder
CASE STUDIES
Impetus’s Approach to Stakeholder Analysis vis-à-vis the Blue Sky Investment

The Impetus team considered stakeholder analysis at three stages during its investment in
Blue Sky: as part of our pre-investment due diligence, as part of our ongoing investment
Pre-investment due diligence: We do not necessarily need, at this stage, a perfect under-
that our assumptions have validity.
Stakeholder analysis is important during this phase because we cannot develop a picture
of the impact that is created by the organisation without identifying and then speaking to
who else, besides Blue Sky, plays a role in creating the change that is experienced by stake-
holders.
A key consideration at all stages but which is of particular importance during the pre-
investment stage is stakeholder selection: in order to form an accurate view of the impact
created by the organisation, we need to ensure that the stakeholders we interview have not
been “cherry picked.” The need for a balanced view from stakeholders is complicated by
the fact that we often have to rely on the potential investee to identify and provide access to
its service users and other parties who are closely familiar with its work, and the potential
investee obviously has an interest in presenting as positive a view of its work as possible.
to maintain contact with individuals who drop out of their programme. To mitigate the
potential for selection bias, we have a number of strategies:
• When developing the list of service users / other partners to be interviewed, we explic-
itly ask the organisation to include some parties where the outcomes were not ideal.
•
the potential investee, but who are familiar with its work.
• In all interviews, we ask stakeholders to discuss both successes and failures that they and
others have experienced, and we ask them to identify any other parties with whom they
Also of critical importance to Impetus at the pre-investment stage is effective resource allo-
-
mately make an investment. We therefore seek to employ a method of stakeholder analysis
the investment. We typically invest c. two to three days of staff time in total in stakeholder
analysis during due diligence, and this was the case with Blue Sky. This is largely managed
in-house, although we do on occasion commission external consultants on a pro bono basis,
always led by an Impetus investment executive, to assist with due diligence.

JUNE 2015 111
CASE STUDIES
We begin with conversations with the senior management team probing the theory of
change and developing a picture of who are the key stakeholders who would need to
be interviewed prior to making an investment decision. Our discussions led us to the
following conclusions:
• Key social change the organisation is trying to achieve: reducing re-offending through
the employment of ex-offenders.
• -
ment and from support in turning their lives around) and government/local communi-
ties (which save money through fewer crimes and related costs).
• Other parties that might contribute to this change: We understood that families, probation
their lives around, and we wanted to investigate further how much of the observed
change could be attributed to these groups.
• -
tively): we were interested in understanding what happens to employees who drop out
step one. We focused our work at this stage on three groups:
• Employees: We interviewed some current and former Blue Sky employees to understand
better what changes for them and how Blue Sky contributes to this. We also wanted
to know if there were any negative changes experienced and the extent to which the
employees felt that other parties were responsible for helping them turn their lives
around. Finally, we wanted to understand more about the employees, in particular, how
similar they are to an “average” person coming out of prison, so that we could develop
an understanding of how much change we might expect to have occurred even without
Blue Sky. Through our due diligence, including interviews with employees, reference
employee database (and comparison with publicly available datasets), we got comfort-
-
vention was very decisive and without it, the change would not have happened. This
other stakeholders at this time.

• Government / local communities: We did undertake some reference checking with local
authorities that were familiar with Blue Sky, but they were primarily able to comment on
the quality of the work performed by the employees rather than the social impact created.
CASE STUDIES
the potential for cost savings when positive outcomes are achieved.
• Potentially displaced employees:
this time.
On-going Investment Management: Having committed to invest in Blue Sky, our primary
investment management model involves

• Monthly meetings with the CEO to review progress against investment milestones and to
get an update on developments within the organisation.
•
• Periodic site visits, including informal interviews with employees, and preparation
of further case studies each year to help keep our understanding of the work “on the
ground” fresh.
• Quarterly reporting by Impetus to our Board on progress against investment milestones.
•
within our portfolio; more regular reporting to particular stakeholders, such as investors
in our various initiatives or co-investors in particular organisations, on a case-by-case
basis.
analysis include:
• A change to outcomes being achieved: we would want to understand what is driving this
new outcome and whether there are any stakeholder groups that need to be considered
•
• New business lines being entered: we would want to look at whether these involve
different market dynamics and therefore increased displacement or other positive or
negative impacts that we would need to consider.
• New recruitment practices: we would want to understand whether this changes the
make-up of the employee group, either increasing or decreasing what we should assume
about deadweight and attribution.
• Changes to the policy environment: we keep an eye on the overall environment around
ex-offenders and employment of hard to reach groups to understand the impact that
other actors might have on the Blue Sky employees and our assumptions around dead-
weight and attribution.

JUNE 2015 113
CASE STUDIES
Given our relatively light-touch and by exception approach to stakeholder analysis during
the on-going investment management phase, stakeholder engagement usually takes up no
more than a day or two of staff time per year and is managed entirely in-house.
One-Off Impact Analysis: Mid-investment, we decided to undertake an in-depth review
to quantify the social value created by the Blue Sky intervention using a methodology
that would be externally recognised as rigorous. We chose to conduct a social return on
investment (SROI) analysis that was capable of achieving assurance by the Social Value UK
(formerly SROI Network), which meant closely following the principles laid out in their
guide to SROI.
The stakeholder analysis and engagement undertaken during this phase was similar to,
and document all the assumptions we had made regarding stakeholders. As an initial step,
and many more). We then screened this list for relevance (how relevant is the stakeholder
of stake-holder experiences by constructing a sample that was of an appropriate size and

diversity. For example, when selecting which employees to interview, we went for a mix of
male and female, current and former, older and younger employees, and we also explicitly
sought out some employees who had left the programme early. Next, we conducted detailed
stake-holder interviews focused on understanding how they experience and contribute to
the social change delivered by Blue Sky. Finally, we “played back” our conclusions to the
there were any additional points that we needed to consider. We engaged a team of about
four people from Blue Sky and Impetus to carry out the stakeholder engagement, and in
total, we invested about eight to ten days of staff time to select the stakeholders, develop
the interview questions, set up and carry out the interviews, and then to follow up with the
stakeholders once the SROI analysis was complete. Although there were some arguments
in favour of engaging a third party to conduct the stakeholder engagement during this
phase, we ultimately opted to conduct this work in-house in order to proceed quickly and
stake-holders. This led to an interesting debate within the team working on the analysis as
to how to treat the families of ex-offenders. We knew that in the relatively small number
CASE STUDIES
could have had a material impact on our calculation of the overall value created and which
therefore should be captured as part of the analysis. We could point to a number of third
party studies that enumerated in detail the positive impact on families of the ex-offender
Blue Sky families directly to substantiate these claims, and the Blue Sky team felt strongly
that this would be inappropriate. Their position was that an important part of their inter-
vention is that they treat their employees as would any other employer, and we did not
that would happen at other companies. This left us in the unsatisfactory position of
knowing that there was material value that we were not able to capture and include in the
analysis. Because we wanted to have our study externally assured, we excluded the value
of changes that we believe are experienced by this group of stakeholders.
Another challenge we encountered was around extracting from the stakeholder engage-
ment the evidence we needed for our SROI analysis in a manner that preserved the integrity
of the process and of our relationships with the stakeholders. For instance, we needed the
employees to share with us as much detail about what had changed, both positively and
negatively, in their lives as a result of working with Blue Sky. However, even though we
made it clear that we wanted employees to be as open an honest with us as possible, we
could not get around the risk that some current employees might be nervous about saying
questions, but some employees were understandably reluctant to open up with us about
a time in their life that has been challenging, meaning that some of their answers to our
a way to ask for additional information gently and without leading the employees towards
-
-
ingful to them. Consistently, employees told us that the most meaningful change they had
experienced was that they had managed to secure and hold onto their freedom, and they
also consistently told us that it was impossible to put a value on this – that it was literally
resources that provide suggestions on how to engage with stakeholders (see Sources in the
Appendix), which we found helpful. Ultimately, though, each situation is unique and will
to preserve the dignity of the stakeholders and the integrity of your relationship with them.

JUNE 2015 115
CASE STUDIES
Lessons Learned / Tips for Other Venture Philanthropy Organisations

In the end, the results that we achieved from the more rigorous, structured examination
of stakeholders that we performed as part of our SROI analysis did not lead us to materi-
ally different conclusions as those achieved during our due diligence: the more rigorous
contributors to the social change achieved by Blue Sky that had not been considered earlier.
may safely rely on intuitive processes that work for them. For newer investors or ones
Appendix). VP investors will also need to think about how much time and resources they
with primary stakeholders to be quite useful and would encourage any VP organisation
to make this a regular part of their investment procedures. However, our experience with
the SROI analysis we conducted also led us to conclude that direct engagement with stake-
holders may not be the only way to understand value creation; we believe there is a place
for considering available third party research with similar stakeholders, particularly if
there are resource constraints that would prevent an organisation from conducting their
own stakeholder engagement or if there are ethical issues involved, as was the case with
Blue Sky.
Our top tips in approaching stakeholder analysis are to:

•
particular stage.
• Put it in proportion: Stakeholder analysis is an area that could, in theory, consume as
much resource as an organisation is willing to invest in it. Be thoughtful about how much
is required for your current stage of investment.
• Keep it current: Stay in touch with your stakeholders regularly and be clear about what
• Sample soundly: Try to construct stakeholder samples that are of an appropriate size and
CASE STUDIES
9.3 Step 3: Measuring Results – Outcome, Impact and Indicators

Case Study: Oltre Venture investing in PerMicro

Oltre Venture started its activity in 2006 and has been investing in social enterprises since
then; bringing capital, managerial skills and knowledge to the social sector. Its purpose is
to assist companies that have a social impact to create value through the creation of sustain-
that seek primarily a social impact. The challenge is to attract private capital in a sector
social return.
the other hand we aim to create a durable positive impact on communities involved in
several private investors, some corporations and one bank foundation (Fondazione CRT).
Currently Oltre focuses its investments on microcredit institutions (20%), social housing
Our approach to venture philanthropy and social investment is tailor made for the market
in which we are investing: Italy. We truly partner with the organisations we invest in
and this is made very clear from the outset. For us understanding impact begins right
embedded in their mission, for example offering services at a price at least 50% lower
than the market rate to customers who would not usually have access (e.g. low cost dental
-
nance). We generally invest in very early stage or start-up organisations so our key focus is
period (7–10 years) then it is unlikely to survive after we exit meaning any potential social
impact is then lost. In addition the investors in our fund expect at least the return of their
PerMicro Case
An example of one of our investees is PerMicro. PerMicro is a microcredit institution
“non-bankable” populations through microcredit, providing loans directly to businesses

and individuals. Operating initially in the multi-ethnic neighbourhoods of Torino, PerMicro

JUNE 2015 117
CASE STUDIES
is based on the concept of network credit: the social network of reference is the interme-
diary between PerMicro and the clients, providing a moral guarantee and supporting them
Its business model has been recognised and rewarded also at European level and won
amount of €11,4m. The average duration of a loan is 36 months, the average size of a loan
is €4,000 for family loans and €7,300 for business loans. Oltre Venture currently owns 12%
group) among its shareholders as its industrial partner.
Current approach to measuring results
condition to achieve its mission. PerMicro wants to understand the outputs, outcomes, and
impact of its activity and has developed an in-house approach to impact measurement that
PerMicro has developed different types of reports and performance screening tools, which
• On-going Performance Tracking & Management: PerMicro produces monthly,

quarterly, and annual reports that summarize its activities, which are shared during
monthly commit-tee meetings. The indicators in the report include measures of outreach,
stated in the business plan (break-even point).
Type of Report Information Covered Purpose

Monthly Reports • Client information: nationality, gender, civil status, •
business activity of clients and purpose of the loan. attrition rate
• Loan information: disbursed and outstanding port- • Provide detailed information on the monthly activity
folio, the number of contracts, the number of opened of PerMicro
Risk Reports • Bad debt • Evaluate cost of risk

• Repayment • Evaluate quality of portfolio
• Other performance measures • Set benchmarks among branches, evaluate other risks
• This is a tool under development. It will be a monthly • Provide a comprehensive view on the social and
report and will provide information across the economic performance of PerMicro
following areas:
• Administration
• Production and development
• Risk and recovery
CASE STUDIES
The form, content and frequency of the reporting were agreed between Oltre and PerMicro
at the beginning of our investment and focus on the operations of PerMicro, for the reasons
• External Reporting: PerMicro produces a series of different reports for different stake-
holders. Equity investors are the stakeholders that are mostly interested in the assess-
(achievement of the break even point and value created through their investment). Apart
from clients and investors of PerMicro, other interested stakeholders are mainly local
municipalities and in general public institution working in the nearby environment,
Target Audience Information

Potential clients of PerMicro • Social reports • Communication instruments
Investors (e.g. Oltre Venture) • Qualitative reports about outreach • Business plan
(monthly) and client satisfaction. • Social reports
• Reports that monitor portfolio risk • Market research
• Balance sheet and income statement
Other stakeholders (networks, • Social reports • Market research

government) • • Reports on clients
Indicators
modelling, with constant monitoring and review of the business plan made available by
monthly budget reports. On the Social side, PerMicro constantly monitors the demographics
of clients it reaches, comparing this with its goals and its mission, as well as monitoring the
Although we are aware of and follow the development of standardised indicators (from the
likes of IRIS and Global Value Exchange) and can understand what these organisations are
us to assign indicators in Italian, and this is not possible with the current IRIS taxonomy.
Secondly we believe it is important to work with the SPO in the development of the indica-
tors given the peculiarities of each of the organisations we work with.

JUNE 2015 119
CASE STUDIES
Financial side Social side

Objectives Financial objectives: break even point Social objectives: lending to non-
bankable people
Indicators Financial data: balance sheets, income Client Demographics: gender,

nationality, education, age
Client Engagement: account types,

pending loans, non performing
loans
Impact Measurement
Following the social and economic contextualization of the microcredit institution activity,
PerMicro goes a step further in the evaluation of impact, focusing on the analysis of changes
made in the quality of life of its clients (or their families and local communities) and deter-
mining whether there have been any positive, negative or neutral effects.
•
•
To identify and measure impact, one must prove in a credible manner that changes observ-
able in clients, with reference to the different analysis levels, are directly related to the
some potential methodologies to evaluate the impact of its activity. These methodologies,
however, presented some hurdles in terms of cost of implementation and of the so-called
attribution problem, which is more marked in the western world, where the existence of a
more structured public welfare system makes it hard to isolate the effect of micro lending
from other types of intervention.
after taking a microloan.
The end of the evaluation period was set to be end of 2014, at which time PerMicro was also
As an investor we are fortunate that PerMicro themselves were willing to commit the
required resources to these more in depth studies about their impact and it does provide us
with further information to communicate to our own stakeholders. However if PerMicro
CASE STUDIES
were not keen to perform these impact studies we would not require them to do so as we
Timeline Phase Actions

June-September Sample of non-bankable
2012
• Absence (or presence for less than 6 months) of a bank account
• Loans with credit institutions
• Loans with banks
2. Selection of a sample composed of non-bankable people within PerMicro

portfolio who received a loan in 2010.
September- Interview 3. Phone interview with the client or the bank to understand whether after the
December 2012 disbursement of a microloan the client become bankable i.e. did they start a
stable relationship with a bank, open a bank account asked and/or obtain
another loan.
January 2013 Elaboration of data 4. Analysis of data. Impact evaluation.
Conclusions
so we predominantly use impact measurement as a management tool, focusing on output

indicators to understand how the business is progressing vis-à-vis its business plan. This is
to isolate the longer-term effects of any organisation we are supporting to provide an

-
ments, we can accurately measure the number of loans disbursed and number of new busi-
nesses created, but to go a step further and consider how that relates to the physical well-
using randomized control groups would probably be required and then we also have the
the purpose of the study were selected not to so as to have an appropriate control group.
We may eventually consider a more comprehensive study of the “impact” of our fund, but
that is most likely to occur once we close the fund and are distributing the proceeds to its
be in place for at least half the time of our total investment period and we think it is more
important to focus our efforts on supporting the entrepreneurs in growing their business.
We recommend other social investors to develop ways of measuring results that are clearly

JUNE 2015 121
CASE STUDIES
9.4 Step 4: Verifying & Valuing Impact

Case Study: Esmée Fairbairn Foundation investing in Social
Impact Partnership (developed and run by Social Finance)
Introduction
Esmée Fairbairn Foundation aims to improve the quality of life for people and communities
in the UK both now and in the future. We make grants in the region of £30–35 million every
year in the arts, education and learning, the environment and social change. In addition,
we operate a £21 million Finance Fund, which invests in organisations that aim to deliver
at the start of their grant of investment. We then track progress towards these outcomes
and investments it is not a good use of resources for us to independently verify or value
the impact achieved, although for strategic interventions on particular sectors or themes or
for large individual grants we may commission a broader evaluation, we do not generally
verify or value the impact that is achieved by our investees.
We are aware that verifying and valuing impact is becoming an important topic in the
social return.
Investment: Social Impact Partnership (developed and run by Social Finance)
Finance with the aim of reducing re-offending in a cohort of prisoners. A social impact bond
is an outcomes-based contract in which private investors pay the costs of an intervention,
are made to the investors by the public sector if the agreed improved social outcomes take
place. If outcomes do not improve, then investors do not recover their investment.
progress. It is hoped that, in a time of reduced public sector spending, social impact bonds
will be a way of attracting new investment in interventions with positive social outcomes.
CASE STUDIES
% of cost savings Ministry

Structure of the Social
Investors from reducing
re-offending
of Justice Impact Partnership:
£5 million
Social Impact Bond
On-going operating funding
Reduction
re-offending
St Giles Ormiston Other
YMCA
Trust Trust Interventions
Support in prison, Support to Providing a Support needed

at the prison prisoners’ families community base by the prisoner,
gates and in while they are in in prison and the
the community prison and post community.
release Funded as the
need is identified
3,000 male prisoners sentenced to less then 12 months
Source: Social Impact Through Effective Finance, Emily Bolton, Social Finance, Ltd., 2010
Approaching Valuing Impact – Considering Objectives, Stakeholders & Impact
Finance, the Government, potential investors and the voluntary sector.
Social Finance canvassed offenders, prison staff, local stakeholders, voluntary organisa-
help stop the revolving door of short sentenced re-offending. They also began talks with
the Ministry of Justice to understand what might make a difference if an alternative source
of funding was found to deliver support to this target group. In addition, Social Finance
engaged with Trusts and Foundations, some of whom were already committed to the
Criminal Justice sector, to test whether they were prepared to support an untested but
potentially transformational proposition.
After 18 months of intense discussions, a contract was signed with the Ministry of Justice
• Provide intensive support to 3,000 short-term prisoners leaving Peterborough prison

over a six year period, leading to a reduction in re-offending of at least 7.5% or more
which would trigger payments to investors; and
• Prove the social impact bond as a model which could attract new investment in future.

JUNE 2015 123
CASE STUDIES
Step 2: Stakeholder Analysis

Most of the stakeholders of the Social Impact Partnership are taking part in or are directly
• Government – the public sector (Ministry of Justice).

• Investors – 17 charitable foundations, primarily from the UK and two from the US;
• Service Providers – voluntary sector charities (St Giles Trust, Ormiston Children and
Families Trust and YMCA) provide the core services, supplemented by additional
•
represent the control group.
• -
ment team works alongside the service providers to provide pre-release services.
Step 3: Measuring Results: Outcomes, Impacts and Indicators
future for their family). For us, the most important outcome was reducing re-offending,
however the cost-saving element for the government became a key driver given the impor-
tance of the government in facilitating the whole transaction
Outcomes:
• A proven reduction in re-offending in a cohort of short-term sentence prisoners.
• A wider impact on the social investment market – evidence on whether this model works,
or how it can be improved, which is taken up by the market.
Indicator:
• Reduction in the frequency of reconviction* events (number of times an offender is recon-
victed at court in the 12 months following release from prison calculated using data held
on the Police National Computer) of the cohort group when compared to a comparison
group of prisoners discharged from other prisons during the same period (to normalize
*It was agreed to use the indicator of reconviction events rather than re-offending, as
cost savings to Government are linked to reconviction events rather than incidences of
re-offending.
Verifying & Valuing Impact

In the case of the Social Impact Partnership, we were primarily interested in the perfor-
mance of the model itself and what lessons this provides for the future – would the stake-
holders be able to work together to deliver the main goal, reducing re-offending?
CASE STUDIES
How was the Social Impact Partnership to assess the value of reducing re-offending? It
costs of reconviction saved by the public sector (the Ministry of Justice) would represent
both the value of the social outcomes achieved and the return to the private investors in
the Partnership.
Why was the cost-saving methodology selected?

• The outcome metric of the Social Impact Partnership is the foundation of its structure,
which is in essence a contract between the public sector and private investors. The
were the most important measure

• In order for the Partnership model to work, the target social outcomes must be tied to a
desired social change and a direct cost to the public sector.
• Being able to measure clearly and provide evidence for the social outcomes of the invest-
ment and link them directly to the costs saved by the Ministry of Justice by achieving the
social outcomes was essential to attract both the public sector and the private investors
into the Partnership.
•
Costs & Cost-Saving

•
• Data that was available and easily collected on public sector costs.
• Cost calculations that were probability-weighted.
• Average public sector costs per individual.
The cost calculations were limited to the direct cost of a reconviction, and did not include:
insurance costs, costs to victims and costs borne by society for crime prevention due to the
Probability / Cost
The public sector court cost of a reconviction within 1 year (in terms of £13,000
police work, court costs, etc.)
Reconviction cost £13,000
The likelihood of a reconviction leading to a further prison sentence 40%
The costs associated with that further prison sentence £37,000

Average prison cost £14,800
The likelihood of a reconviction leading to a community sentence 60%
The costs associated with that community sentence £6,000

Average community sentence cost £3,600
Average cost of a reconviction within 1 year £31,400
(please note that the data in the chart is illustrative only)

JUNE 2015 125
CASE STUDIES
Impact measurement: current status
cohorts of 1,000 unique short-term male prisoners each over 6 years.
In order to measure the outcome, the Social Impact Partnership tracked a baseline control
to match the cohort to a suitable control group. The method normalized the groups for demo-
assessed by QinetiQ and the University of Leicester.
There are minimum thresholds in place that must be reached to ensure that the outcomes
•
in each of the three cohort groups are reduced by at least 10% relative to a control group.
• If a 10% reduction is not achieved in any of the three cohorts, then the three cohorts are
measured together at the end of the pilot. If a 7.5% reduction is achieved in total, then
investors receive payment for any cohorts that have not been paid for to date.
• There is a cap on total outcome payments to investors. Investors will therefore receive an
increasing return effectively capped at a maximum of 13% per year over the eight-year period.
Limitations of this method:

• Data integrity – the measurement of outcomes assumes that data on the prisoners and the
control group is captured and recorded correctly on the Police National Computer.
• Propensity Score Matching method (“PSM”) – the model assumes that the PSM method-
characteristics.
We are not yet able to value the impact of the Social Impact Partnership, but as a model it has
proved that it is possible to get Government, the voluntary sector and private sector investors
working together for a common goal. Whether this is taken up more widely will depend on its
success, and that of other Social Impact Bonds, which have begun to be taken up in other sectors.
Anecdotally, it is believed that the Social Impact Partnership is already having an impact.
Clients have reported a better control over their lives and lower incidences of re-offending.
released, a further 18 months to track reconviction events and a further 3–6 months to measure
the outcome against the control group.
Lessons Learned
PSM method proved successful in developing an appropriate control group, it is a complex

and time-consuming process. This could be a barrier to replicating the model more widely.
CASE STUDIES
There are also limitations for control group comparison if the social impact bond models
are scaled. With higher numbers, the population from which the control group is derived
becomes smaller, which may restrict the quality of the matching and ultimate results.
The outcome chosen in this case is the one that is best linked to cost-savings for the Govern-
considered from the point of view of other stakeholders, including ourselves. For a charitable
(either through a grant or a social investment) is usually done through a mix of qualitative
Will the estimated cost-savings for Government materialise? It is likely that the outcomes
achieved at Peterborough will be too low to be able to shut a prison wing or close a court;
hence the cashable cost-savings may be limited.
9.5 Step 5: Monitoring & Reporting

Case Study: Auridis investing in Papilio
(“Papilio”).

Auridis invests in organisations and programmes that sustainably improve opportunities
for socially disadvantaged families and their small children.
The investment focus is on the dissemination and replication of successful approaches. The
core portfolio consists of 19 organisations that mostly receive grants for 3 to 10 years. Due to
the fact that we are investing in early childhood development, most investments do preven-
tion work with impact that cannot be easily related to the activities of the investees.
Since 2010 Auridis has been supporting Papilio. Papilio has developed and promotes
a kindergarten programme for early childhood prevention of addiction and violence.
-
tion, with extremely high negative effects on the society and national economy. The likeli-
hood of young people developing a substance addiction or violent behaviour is to a relevant
so-called socio-emotional competences (resilience). Children develop these competences in
abilities to support young children in developing positive social and emotional competences.

JUNE 2015 127
CASE STUDIES
Papilio integrates as a part of the pedagogic concept in the kindergarten, with elements
like the “toys-go-on-holiday-day” or “Paula and the trunk pixies”, a puppet play with
programmes offered in German kindergarten, Papilio accompanies the children during

their whole kindergarten time (as opposed to a curricular one-time activity).
Prevention and impact measurement dilemma

The expected social long-term impact:
• The early development of protection factors (social-emotional competences) prevents
the risks that lead to addiction and violence.
• This forms the basis for a self-paced and independent adult life.
Challenge:
It is a great challenge to measure the long-term social impact of prevention work with
quality assurance.
outcomes based on large-scale randomised longitudinal studies – if at all.
Proxy:
Short-term output indicators can give an indication of sustainable impact.
The Papilio programme is disseminated by way of a train-the-trainer model, with head-

quarters in Augsburg, Germany. Since 2002 close to 5000 child care workers in 11 federal
states all over Germany have been trained with the Papilio programme and approximately
business planning phase from 2010 to 2011 and is currently supporting the growth phase
from 2012 to 2017 (estimated).
Auridis’ approach to Monitoring & Reporting
planning phase (“impact planning”). During this phase a shared understanding of the
social issue, the theory of change, the expected impact, the main levers for organisational
success, and the relevant indicators is developed between Auridis and the investee. During
Auridis and the investee.
We track the development in our investee database which collects information such as
progress reports, and the milestones. The investee database has been developed in house
output, outcome, or impact data of the portfolio organisations, as we believe this data
would lead to misinterpretation.
CASE STUDIES
We support the development of information management and controlling systems of our
systems, bringing in pro bono consultants, and sharing experiences throughout the portfolio.
The level of sophistication of the information management and controlling systems of our
investees varies across our portfolio. We estimate that approximately one quarter of our
investees have good systems in place, on a par with Papilio, which we discuss shortly; one
quarter of our investees are about to develop a robust system; one quarter are considering the
development of an information management system; and for the remaining quarter it is not
an issue on their radar screen given they are very early stage and are needing to focus their
efforts and resources elsewhere.
Papilio’s approach to monitoring and reporting
2005 with 700 children and their families. The results showed positive outcomes for children,
better learning abilities at school, positive effects on cooperation within the kindergarten
team, and a better basis for education partnerships with the parents.
As outcomes are not always easy to measure in the short term we decided to use large scale
output indicators to serve as proxies for outcome. For example: the number of actively prac-
and DVDs for their children, etc. The underlying assumption is that these indicators are good
proxies for the expected long-term outcomes.
1. Aggregation of impact data

Papilio introduced an online, web-based database system for the Papilio trainers to report
their activities to headquarters. Information such as names and contact details of trainers, child
care workers, and kindergarten as well as number, date, place, and participants of trainings
In addition, Papilio tracks the quantities of materials ordered (books, DVDs, educational
material, etc.). The Papilio team gets monthly reports of all aggregated data. The prerequisite
for Papilio to introduce such a tool was a German-language, very simple web-log-in system.

JUNE 2015 129
CASE STUDIES
Financial data tracked by the book-keeping and accounting system as well as output data
recorded in the web-based database are integrated in monthly and quarterly reports. This
is done semi-automatically by the controller of Papilio using Microsoft Excel templates
for summarizing the web-based database and the accounting software. Data processed
output indicators as described above.
3. Stakeholder presentation of the data

The data collected is presented to different stakeholders in different formats:
management. More detailed reports are produced for a variety of funders in accordance
with their respective requirements.
Papilio has started to produce annual reports in accordance with the German Social
Reporting Standard (SRS). The SRS has been developed by a consortium of German high-
impact funders such as Auridis, BonVenture, and Ashoka, in cooperation with experts and
researchers. SRS provides a structure to report on the problem to be solved, the contribu-
tion of the SPO to the solution and the achieved social impact together with organisational
different funders. To the extent this is not the case the reports can be complemented by
additional annexes.
Papilio started to use the SRS structure during its business planning phase. Many of the
elements developed during this phase are being reused for reporting purposes, such as the
investees to do so, preferring to offer them assistance in introducing it. In our view, using
SRS will improve the consistency and comparability of the information that we receive.
CASE STUDIES
Auridis’ investment in Papilio
aggregate the data. As Papilio evolved a more holistic system was needed. Supported by
-
– was put into action in 2012 and will need approximately half a year of implementation. So
far Papilio only counts hard facts, but is thinking about how to measure soft outcomes in the
future.
What has Papilio learned from the development process?

• Usability is the key success factor for the usage of the system. Therefore, simplicity is the
most important requirement for the information management system.
• The underlying processes are more important than technology.
• The process should be steered by an experienced IT developer who can, and does, ask the
team for input regarding the reporting contents and formats required and translates them
into a technical solution.
• The whole team and some of the other (external) users need to be integrated in the develop-
• The development of an information management system needs an iterative process and a lot
of end consumer testing and reversing.
Therefore, user training is required to ensure the same understanding of data and time periods,
have only access to defunct technology) and to agree on reporting timelines. The end-user
should optimally also understand the added value of using the new tool.
The costs for the development of the system were 20,000 to 30,000 Euro for staff time and the
IT developer. In many cases this kind of work does have a high potential for pro bono work
from external consultants and IT companies. The hours saved if the information manage-
(although no calculation was made for this).
Recommendations
We believe that investees should be encouraged to allocate substantial money to information
management, as it is a key to sustainable growth and stakeholder reporting. Excel is only
suitable for the early development stage. In most cases, the necessity to introduce more or less
sophisticated monitoring and evaluation systems only becomes apparent once the scaling-
-
ence, the monitoring and evaluation systems used by one organisation can only inspire the

JUNE 2015 131
CASE STUDIES
development of tailor-made solutions for other organisations with a different business model,
but cannot be transferred “as is”.
Importantly, the investees need external help to implement these systems, which can be facili-
tated by the VPO/SI. In a number of cases, the organisations in the Auridis portfolio were
the “how to questions”.
studies need to be used, especially in prevention work. But be aware to not only count what
is countable – soft facts matter more. However, it is important to be transparent about the
Financing an information management system
-
cant costs would accrue. VPO/SIs should provide cash and encourage their investees to invest
in IT infrastructure to streamline processes and strengthen the operational capacities of the
investee.
Nevertheless, given the usual shortage of money in SPOs, this topic offers the opportunity to
consultant the development and implementation process can be realised with minimum cash
spend. VPO/SIs can play an active role in connecting their investees to service providers
and pro bono resources. Investments in a sound information management system should be
written off in many years and maybe shared with other organisations to make the investment
worth wile.
Typical costs if no pro bono support applied:
Phase Cost range (Euro)

indicative, based on the Auridis
5,000 – 15,000
Realisation 20,000 – 50,000 However, it can be used as a

general indication of expected
Testing 1,000 – 5,000
costs.
Pilot phase 2,500 – 5,000
Yearly operation 1,000 – 5,000

PART 4:
Appendices

JUNE 2015 133
APPENDICES
GLOSSARY OF TERMS 42
10.0 Glossary of Terms Contributors

The people, communities, broader society
Accountability and environment that contribute to the SPO
The obligation of an organisation to account performing its activities. Contributors can
for or take responsibility for the effect of its enhance or decrease the effect of the activi-
activities. ties of the SPO.
Activities
The concrete actions, tasks and work carried -
out by the organisation to create its outputs -
-
Attribution
Attribution takes account of how much of -
the change that has been observed is the sis applied to SPO activities.
how much is the result of actions taken Deadweight

simultaneously by others (e.g. other SPOs, Deadweight is the change that would have
government). happened anyway i.e. the outcomes the ben-
Balanced scorecard if the organisation were not active. This is

Developed by Robert Kaplan and David sometimes called the “baseline” or “coun-
terfactual”. Deadweight includes the pro-
what an organisation means by “perfor-
mance” and measures whether the organi-
sation is achieving desired results. The
Balanced Scorecard translates mission and Displacement
vision statements into a comprehensive set Displacement occurs when the positive out-
-
negative outcomes experienced by another

The traditional balanced scorecard of the group elsewhere (also as a result of the
business world has also been adapted by
Social Enterprise London with the aim of
assisting social enterprises to examine their Drop-off
strategies and desired outcomes, which can Drop-off occurs when, over time, the effects
be tracked over time. of the output and the observed out-comes
The people, communities, broader society -

and environment that a SPO seeks to reach comes sets the scope for how long they are
42. Definitions defined so as to
expected to last. Drop-off occurring within
be aligned with definitions in
affected positively or negatively by the this period is accounted for in assessing the the glossary of Hornsby, A;
activities of the SPO. Blumberg, G., (2013), “The Good
Investor: A book of best impact
practice”, Investing for Good.
APPENDICES
GLOSSARY OF TERMS
Expert group Investee

The Expert Group is the 27 strong group of A SPO that receives investment from a
practitioners, consultants, academics and VPO/SI.
representatives from other networks who
contributed to the development of this prac- Investment
tical guide. We use investment throughout this docu-
Global Value Exchange instruments from grants, loans to equity.

Global Value Exchange is a database
of values, indicators and outcomes for IRIS
stakeholders.
Standards initiative of the Global Impact
Impact Investing Network (“GIIN”) and was devel-
See: Social Impact oped to provide a common reporting lan-
guage for impact related terms and metrics.
Impact investor
IRIS indicators
IRIS indicators are a set of standardised
Impact measurement initiative metrics that can be used to describe an
The initiative undertaken by the European
Venture Philanthropy Association with the
support of the Expert Group to create this
practical guide for impact measurement Logic model
with the aim of spreading best practice in Logic models are usually a graphical depic-
the venture philanthropy and social invest- tion of the logical relationships behind how
ment sector. an organisation does its work i.e. the rela-
tionships between the activities, outputs,
Impact value chain outcomes and impacts.
Represents how an organisation achieves
its impact by linking the organisation to its Materiality
activities and the activities to outputs, out- Materiality refers to an assessment made to
comes and impacts. determine the factors that are relevant and
material to include in a true account of the
Inputs
The resources, whether capital or human,
invested in the activities of the organisation. Monetisation
Indicators Monetisation is the process of transforming
the value of outcomes and/or impacts into a
actions or conditions that assess progress unit of currency. SROI is a way to monetise
towards or away from outputs or out-
comes. Indicators may relate to direct quan-
tities (e.g. number of hours of training pro- Organisation
vided) or to qualitative aspects (e.g. levels In this case an entity working to bring about
positive social impact i.e. the term includes
SPOs and VPO/SIs.

JUNE 2015 135
APPENDICES
GLOSSARY OF TERMS
Outcomes years of life that would be added by a par-

ticular medical intervention and the quality
effects (both long and short term) that result of the life lived during those years.
Revealed preference
Outcomes matrix Revealed preference theory was pioneered
- by American economist Paul Samuelson and
ety Capital in combination with Investing is based on the assumption that the prefer-
for Good and other UK based VPOs, for
use by investors and SPOs to map areas in purchasing behaviour. It tries to understand
impacts are being achieved. of goods, given their budget constraints.
Outputs Social balanced scorecard

The tangible products and services that The traditional balanced scorecard adapted
by Social Enterprise London with the aim of
assisting social enterprises to examine their
Participatory impact assessment strategies and desired outcomes, which can
Participatory impact assessment is the pro- be tracked over time.
cess of engaging people and communities in
the actual measurement of impact on their Social impact
livelihoods, for example through the use of -
focus groups or survey. ties to broader and longer-term outcomes.
Perceived value To accurately (in academic terms) calcu-

-
comes for: (i) what would have happened
have little or nothing to do with the prod- anyway (“deadweight”); (ii) the action of
others (“attribution”); (iii) how far the out-
come of the initial intervention is likely
needs or requirements. to be reduced over time (“drop off”); (iv)
the extent to which the original situation
Progress out of poverty index (“PPI”) was displaced elsewhere or outcomes dis-
Developed by the Grameen Foundation, the placed other potential positive outcomes
progress out of poverty index estimates the (“displacement”); and for unintended con-
likelihood that an individual falls below the sequences (which could be negative or
national poverty line, the $1/day/PPP and positive).
$2/Day/PPP international benchmarks.
Social investor (“SI”)
workers can quickly collect and verify. An organisation pursuing a social invest-
ment approach.
Quality adjusted life year (“QALY”)
Social investment
of health in terms of time (life years) and Social investment is the provision and use
-
with diseases). It is based on the number of cial returns.
APPENDICES
GLOSSARY OF TERMS
The social investment approach has many Theory of change

overlaps with the key characteristics of ven-
ture philanthropy, however social invest- blocks required to bring about a given
ment means investment mainly to gener- long-term goal. This set of connected build-
ate social impact, but with the expectation ing blocks is depicted on a map known as
a pathway of change or change framework,
capital). which is a graphic representation of the
change process.
Social purpose organisation (“SPO”)
An organisation that operates with the pri- Unintended consequences
mary aim of achieving measurable social Unintended consequences are those that
and environmental impact. Social purpose
activities, but are not part of the desired
organisations and social enterprises. effect. They may be foreseen (anticipated but
not intended), or unexpected (positive or
Social return on investment (“SROI”) negative). Unintended consequences often
Social return on investment is a framework relate to effects upon stakeholders other
for measuring and accounting for the broad
concept of value. It tells the story of how
change is being created by measuring social, Value Game
environmental and economic outcomes and The Value Game is a survey tool that asks
uses monetary values to represent them. questions to stakeholders in order to reveal
the value of outcomes. It shows how stake-
calculated e.g. a ratio of 3:1 indicates that an holders value the outcomes they experience
investment of €1 delivers €3 of social value. relative to other products they also value.
Stakeholder Venture philanthropy (“VP”)

Any party that is effecting or affected by Venture philanthropy is an approach that
the activities of the organisation. The most includes both the use of social investment
prominent stakeholders are the direct or (equity and debt instruments) and grants.
The key characteristics of venture philan-
thropy include high engagement, organisa-
staff and volunteers, its shareholders and
investees, its suppliers and purchasers and -
works, multi-year support and performance
those close to them, and the communities in measurement.
which they live.
Venture philanthropy organisation
Stated preference (“VPO”)
Stated preference is a method used to assess Organisations following the venture phi-
the value of an outcome or impact by using lanthropy approach.

JUNE 2015 137
APPENDICES
SOURCES
11.0 Sources

General Resources on Goal Setting
• Locke, E. A. & Latham, G. P., (1990). “A theory of goal setting and task performance”.
Upper Saddle River, NJ: Prentice-Hall.
• Locke, Edwin A.; Latham, Gary P. “Building a practically useful theory of goal setting and
task motivation: A 35-year odyssey.” American Psychologist, Vol 57(9), Sep 2002, 705–717.
• Doran, G. T., (1981). “There’s a S.M.A.R.T. way to write management’s goals and objectives”.
Management Review, Volume 70, Issue 11 (AMA FORUM), pp. 35-36.
• Sept, Naylor and Weston. 2011. “Measuring the impact of social programs: A review of best
practices -
tally Responsible Supply Chain Program: http://www.gsb.stanford.edu/sites/default/
Note that this provides a slightly different framework for considering approaches to
social performance measurement:
•
Models): http://www.wkkf.org/knowledge-center/resources/2006/02/WK-Kellogg-
Foundation-Logic-Model-Development-Guide.aspx
• Grantcraft, The Ford Foundation, “Mapping Change: Using a Theory of Change to Guide
Planning and Evaluation” (Compares “Theory of Change” and “Logic Model”):
• www.theoryofchange.org
• Annie E. Casey Foundation (www.aecf.org). “Theory of Change: A Practical Tool for
Action, Results and Learning”.
Step 2: Analysing Stakeholders

• Accountability, “The Stakeholder Engagement Manual”. http://www.accountability.org/
images/content/2/0/207.pdf
http://www.accountability.org/images/content/2/0/208.pdf
• The New Economics Foundation, “Participation Works!” http://www.neweconomics.
org/publications/participation-works
• Robert Wood Johnson Foundation, “A Practical Guide to Engaging Stakeholders in Deter-
mining Evaluation Questions”
• Forthcoming guide by Social Value UK (formerly SROI Network) “Supplementary
Guidance on Stakeholder Involvement” http://socialvalueuk.org/publications/publica-
tions/doc_download/368-supplementary-guidance-on-stakeholder-involvement
• The Value Game – a stakeholder led valuation tool http://www.valuegame.org/
• Geoff Mulgan, “Measuring Social Value”, Stanford Social Innovation Review, (2010).
http://www.ssireview.org/articles/entry/measuring_social_value
APPENDICES
SOURCES
Step 3: Measuring Results: Outcome, Impact, Indicators

• Nelson & Ratcliffe, (2010), “A Guide to Actionable Measurement
Foundation
• IRIS database of indicators: iris.thegiin.org
• Global Value Exchange database of indicators: http://www.globalvaluexchange.org/
• Social Balanced scorecard and other tools: http://www.proveandimprove.org/tools/
socialenterprise.php
• Ruby Sandhu-Rojon, UNDP, “Selecting Indicators for impact evaluation”
• Millennium Development Goals: http://www.un.org/millenniumgoals/
• Progress Out of Poverty Indicator: http://www.progressoutofpoverty.org/
Step 4: Verifying & Valuing Impact

• http://www.roguecom.com/interview/overview.html
• http://techinlibraries.com/cowgill.pdf
• http://www.eldrbarry.net/roos/eest.htm
• Mitchell, R. and R.Carson, (2005). „Using surveys to value public goods; the contingent
valuation method“. Washington USA.
• Champ, P., Koyle, K. and Brown, T. (2003). „A Primer on nonmarket valuation“.
Dordrecht (NL): Kluwer.
• Wellbeing valuation: www.ncbi.nlm.nih.gov/pubmed/17380470
• Further information on QALY can be found at: http://www.medicine.ox.ac.uk/
bandolier/painres/download/whatis/QALY.pdf
• More information on these techniques (and many others) can be found in the TRASI
database: http://trasi.foundationcenter.org/
Step 5: Monitoring & Reporting

• Social Reporting Standard: http://social-reporting-standard.de/ by Auridis, BonVen-
ture, Phineo, Ashoka, PWC a.o.
http://srs.aufbau-server.de/en : English webpage
• www.mande.co.uk : website on monitoring and evaluation; lot of information, docu-
ments, cases, etc.
• http://www.unfpa.org/monitoring/toolkit.htm (United Nations Population Fund)
• http://web.undp.org/evaluation/handbook PDF-Handbook UNDP.
• “Principles of Good Impact Reporting”, by NPC a.o.
• World Bank, “Monitoring & Evaluation: Some Tools, Methods & Approaches”.
• PULSE (http://pulse.app-x.com)
• GIIRS (www.giirs.org) provides both company and fund impact ratings, each with
current and historical analyses of impact performance for comparative use. In order to
scale the impact investing marketplace, investors require an independent third-party
impact ratings product that is comparable, transparent, and easy to use.

JUNE 2015 139
APPENDICES
SOURCES
Webinars
The Expert Group members were divided into working groups to focus on a particular step
-
tion to the other members of the Expert Group and the case studies found in section 9.1. The
working groups for each step were as follows. A “*” denotes the author of the case study.
• Step 1 - Setting Objectives: Van Dijk, M., Social Evaluator; Presner, B., Acumen Fund;
Kagerer, T., LGT Venture Philanthropy; *Sandvold, O., Ferd Social Entrepreneurs;
Ferraro, F., IESE Business School.
• Step 2 – Analysing Stakeholders: Grabenwarter, U., European Investment Fund; *Niles,
M., Impetus Trust; Kennedy, R., CAN Breakthrough; Robin, S., Stone Soup.
• Step 3 – Measuring results: outcome, impact, indicators: Gelfand, S., the GIIN; Lane
Spollen, E., One Foundation; *Allevi, L., Oltre Venture; Stievenart, E., ESSEC Business
School.
• Step 4 – Verifying & Valuing Impact: Nicholls, J., SROI Network (now Social Value UK);
Varga, E., NESsT; *Petkova, I., Esmée Fairbairn Foundation; Nicholls, A., Skoll Centre
for Social Entrepreneurship.
• Step 5 – Monitoring & Reporting: Scholten, P. Backstrom, C.,
Naya AB; Tarakeshwar, N. *Leissner, C.,
Auridis; Santos, F., INSEAD Business School.
Interviews
• Allevi, L., Managing Director, Oltre Venture (September 24, 2012)
• Blokhuis, M., Director, Noaber Foundation (October 19, 2012)
• Crane, G.,
• Kagerer, T., COO, LGT Venture Philanthropy (September 10, 2012, by email)
• Leissner, C.,
• Luebbering, J; Elsemann, K., Partnership Development, Streetfootballworld (September
10, 2012)
• Lumley, T., Head of Development, New Philanthropy Capital (September 7, 2012)
• Mason, C., COO, Big Society Capital (September 27, 2012)
• Niles, M., Investment Director, Impetus Trust (September 24, 2012)
• Sandvold, Ø., Director of Business Development, Ferd Social Entrepreneurs (September
17, 2012)
Rue Royale 94
1000 Brussels, Belgium
Tel: +32 (0) 2 513 21 31
Email: info@evpa.eu.com
The European Venture Philanthropy Association (EVPA) EVPA is grateful to:

Fondazione CRT for the support
Established in 2004, EVPA aims to be the natural home as well as the highest- of its Knowledge Centre
value catalytic network of European Social Investors committed to using venture

philanthropy and social investment tools and targeting societal impact.
EVPA is grateful to:

investment activities and includes venture philanthropy funds, social investors, Acanthus Advisers, Adessium
Foundation, BMW Foundation
and Omidyar Network for their
structural support
EVPA members work together across sectors in order to promote and shape the
future of venture philanthropy and social investment in Europe and beyond.
Currently the association has over 180 members from 25 countries, mainly based
in Europe, but also outside Europe showing the sector is rapidly evolving across
borders.
EVPA is committed to support its members in their work by providing

networking opportunities and facilitating learning. Furthermore, EVPA aims to
strengthen our role as a thought leader in order to build a deeper understanding
of the sector, promote the appropriate use of venture philanthropy and social
investment and inspire guidelines and regulations.
http://www.evpa.eu.com
ISBN 9789082316087
VENTURE PHILANTHROPY
SOCIAL INVESTMENT

EVPA Measuring and Managing Impact

Uploaded by

Copyright:

Available Formats

EVPA Measuring and Managing Impact

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

EVPA Measuring and Managing Impact

Uploaded by

Copyright:

Available Formats

What is the guide about?

What is the guide about?

Who are the main authors of the guide?

Who are the main authors of the guide?

EUROPEAN VENTURE PHILANTHROPY ASSOCIATION

EUROPEAN VENTURE PHILANTHROPY ASSOCIATION JUNE 2015

Published by the European Venture Philanthropy Association

Creative Commons Attribution-Noncommercial-No Derivative Works 3.0.

• Attribution: You must attribute the work as

Authors: Dr Lisa Hehenberger, Anna-Marie Harling, Peter Scholten

or opinion of the European Commission.

EUROPEAN VENTURE PHILANTHROPY ASSOCIATION

EUROPEAN VENTURE PHILANTHROPY ASSOCIATION

Preface 6 5.0 Step 4: Verifying & Valuing Impact 76

EUROPEAN VENTURE PHILANTHROPY ASSOCIATION

Introduction to the second edition from Lisa Hehenberger,

on impact measurement and management.

commonalities between various approaches to come up with clear recommendations on

Uptake of the Guide and recent developments

outlined in the Guide.

Step 1 90 % of VPOs that perform

Outputs Outcomes Impact Objectives of social impact

on impact measurement, and we have participated in and contrib-

EUROPEAN VENTURE PHILANTHROPY ASSOCIATION

data collection and data analysis.

• Impact measurement has to be relevant to the organisation measuring so that it becomes

• Impact measurement also needs to be proportionate to the organisation at hand, keeping

Next steps for EVPA

Other plans include developing a micro-site on impact measurement, to develop a

7. A shortened and simplified

EUROPEAN VENTURE PHILANTHROPY ASSOCIATION

Introduction from Daniela Barone Soares,

programmes, and inform their decision-making.

the imprint of past successes and failures.

At Impetus, we are committed to building this capacity in the organisations we support.

Daniela Barone Soares

EUROPEAN VENTURE PHILANTHROPY ASSOCIATION

Expert Group composition

EVPA is grateful for the contribution of the following Expert

edition of the report was published in April 2013.

EUROPEAN VENTURE PHILANTHROPY ASSOCIATION

8. We use investment through-

grants, loans and equity.

ring 1. The 5 steps of social

Results Source: EVPA

implementing them in practice.

EUROPEAN VENTURE PHILANTHROPY ASSOCIATION

The Impact Value Chain

SPO’s Planned Work SPO’s Intended Results

1. Inputs 2. Activities 3. Outputs 4. Outcomes 5. Impact

In what follows, we provide a quick glance at the recommended impact measurement

Step 1: Setting Objectives

relevant and what is not when faced with scarce resources.

(i) Level of the VPO/SI.

c. What type of SPOs are you working with?

general conclusions about the impact of the SPO.

EUROPEAN VENTURE PHILANTHROPY ASSOCIATION

c. What are the expected outcomes?

(ii) Level of the SPO.

SPO to solve the issue.