Creating Your Theory of Change1 PDF
Creating Your Theory of Change1 PDF
Creating Your Theory of Change1 PDF
I am glad to say that this initial scepticism has been found to be entirely unwarranted. Since NPC first began to
explore using theory of change to work with charities around impact measurement and strategy, it has become a
core component of our work. We have found it to be enormously useful in helping charities to articulate what it is
that they do and why: what they are trying to achieve, and how their strategies aim to get them there.
Today, we recommend that all social purpose organisations—charities and funders, social enterprises and
investors—build their work on a theory of change. It is at the heart of strategy. It is the foundation for the
development of an impact measurement framework. It should be the cornerstone of attempts to work out whether
and how well you are achieving your mission. NPC’s approach to impact measurement, set out in our Four pillar
report, is built around the concept of theory of change.
Of course, a theory of change is only as useful as its practical application and we should not get lost in the quest
for theoretical perfection. But taken at face value, as a theory to explain how your work is supposed to function, it
is immensely powerful. For us in the charity sector, committed to achieving the best possible outcomes for those
we aim to serve, it helps test whether our understanding of the world is backed up by what happens in reality.
Theories of change, therefore, should not be static—they evolve and improve as our understanding and
knowledge is advanced by evidence and observation.
NPC published its first paper on theory of change in 2012, and it has since become our most popular publication,
downloaded more than 13,000 times. Theory of change is also our most popular training course, with 169
participants over the last two years. In our consulting we have worked directly with 47 clients to help them develop
their own theories of change. While most of this work has been with charities, funders are increasingly seeking
our help too. At the same time, we have seen growth in the wider theory of change market: other consultants and
advisors have built up their own offerings to help social purpose organisations make use of the concept.
After a long time in development, NPC has also published its own theory of change. This process has helped us
to understand the opportunities and the challenges developing a theory of change can present. As an evolving
model of our work, it is sometimes hard to know when it is finished. And producing a theory of change is not the
same as embedding it throughout our work, and using it to drive our strategy and impact measurement. We are
on a journey with theory of change, as is the UK charity sector today.
I am delighted to mark this step with the publication of a new paper on theory of change, sharing the insights we
have gleaned through our work with clients and partners. I hope it helps you get an even better grip on what is at
the core of your work—what you are trying to achieve, and how you plan to get there.
2
CONTENTS
3
Creating your theory of change | Glossary of key terms
Having a common language is important when developing a theory of change, so we have included a glossary of
the terms we use at NPC. It is worth noting that some of the definitions are contested, and it is not essential that
you adopt them precisely. The most important thing is that you understand what we mean in this guide and that
when you are developing a theory of change for your organisation or project, everyone shares a common
understanding.
Final goal: The broader social change a project or organisation is trying to achieve.
Intermediate outcomes: The short-term changes, benefits, learning or other effects that result from what a
project or organisation does. These short-term steps will contribute to a final goal and may include changes in
users’ knowledge, skills, attitudes, and behaviour. A useful way to think about intermediate outcomes is the
outcomes achieved after the project—what service users take away from it.
Activities: The things that an organisation or project does or the way it chooses to deliver a project day-to-day.
Activities are within an organisation or project’s control.
Inputs: The resources that go into the project that a team or organisation needs to be able to carry out its
activities.
Outputs: Products, services or facilities that result from an organisation or project’s activities. These are often
expressed quantitatively; for example, number of users, how many sessions they receive and the amount of
contact they had with a project.
Enablers: Conditions or factors that need to be present or absent to allow an organisation or project’s work to
succeed. The presence or absence of enablers can help or hinder a project. There are two kinds of enablers:
• Internal enablers need to exist inside an organisation for a theory of change to work, and are mostly within
an organisation or project’s control. Internal enablers describe the mechanisms by which an organisation
delivers its work (such as the quality of services, relationships and the values and attitudes of staff).
• External enablers need to exist in the external environment for a theory of change to work, and are often
beyond an organisation or project’s immediate control. External enablers describe the context in which an
organisation works (such as social, cultural, economic and political factors, laws, regulations, and working
with other organisations).
Evidence: Information that you already have or plan to collect that is relevant to supporting or testing a theory of
change.
Assumptions: The underlying beliefs about how a project will work, the people involved and the context. These
are sometimes implicit in a logic model or theory of change, but it can be useful to state them explicitly.
• http://www.jargonbusters.org.uk/
• http://www.theoryofchange.org/what-is-theory-of-change/how-does-theory-of-change-work/glossary/#2
4
Creating your theory of change | Introduction to theory of change
A theory of change is a tool to help you describe the need you are trying to address, the changes you want to
make (your outcomes), and what you plan to do (your activities). The approach can be used for organisations of
all shapes and sizes—from service-delivery charities, to campaigning organisations, to funders.
A theory of change is often represented in a diagram or chart, but a full theory of change process involves more
than this. It should help you consider and articulate the assumptions and enablers that surround your work and
explain why you think your activities will lead to the outcomes you want. It should also challenge you to develop
clear aims and strategies and explore whether your plans are supported by evidence. By the end of a theory of
change process, you should have a clear idea of what your organisation or project wants to achieve and a
strategy to do this.
Charities, funders and social purpose organisations can benefit greatly from creating a theory of change. A range
of possible advantages is discussed in the next section of this paper.
Theory of change is actually a very simple concept. Throughout our work and personal lives we have aims,
objectives and ideas about how to achieve our goals, but we rarely take the time to think these through, articulate
and scrutinise them. All a theory of change process does is to make these assumptions explicit and therefore
more testable.
While the idea itself is simple, the terminology around theory of change can be confusing because there are
variations in the approach. In this document we use the term to refer to the broad family of approaches that all
©
help to articulate the reasoning behind a project. This includes the CES Planning Triangle , logic models,
outcomes chains, and narratives. However, others in the field use theory of change to refer only to the outcomes
chain approach. As an organisation thinking about using theory of change, the most important thing is not to get
bogged down in terminology, but to use the method that suits your needs best.
This guide aims to help you through your theory of change process and support you to produce something that
strengthens the design and delivery of your projects, and gives you confidence in your approach to evaluation and
learning.
5
Creating your theory of change | The benefits of theory of change
During our work we have seen first-hand how a theory of change can help an organisation improve its strategy,
measurement, communication and partnership working. Below we discuss the benefits a theory of change can
bring in each area.
Strategy
• Help teams work together to achieve a shared understanding of a project and its aims. The process of
agreeing a theory of change teases out different views and assumptions about what an organisation is aiming
for and how staff should work together. Using a theory of change to co-develop strategy fosters consensus
and can motivate staff, helping them to feel involved and showing them how their work contributes to long-
term goals.
• Make projects more effective. A theory of change is an agreed statement of what your organisation or
project is trying to achieve. It can help you to identify where activities are not contributing to your goals and
take action, and understand what information you will need to monitor performance.
• Help identify and open up ‘black boxes’ in thinking. A range of assumptions may underlie the design of
your project or approach. The theory of change process should reveal these hidden assumptions, some of
which you may then discover are unfounded, out-of-date or inconsistent with the evidence.
Measurement
• Help determine what needs to be measured (and what does not) so you can plan your evaluation
activities. Some evaluations are misconceived and lack strategy. A theory of change provides a framework
for the evidence you should collect, which will give you greater confidence in your approach.
• Encourage teams to engage with the existing evidence base. The best theories of change are justified by
up-to-date knowledge of what works in a particular field. This could be drawn from your own organisational
data and/or research published by others, such as academics and government departments.
• Act as the basis for claims about attribution. If, by collecting good quality evidence to test your theory,
you can show you have achieved targets and desired changes at each stage, then you have a stronger case
for saying that your project has made a difference.
Communication
• Quickly communicate a project’s aims. A theory of change diagram is a neat way to summarise your work
and communicate it to stakeholders, including funders and commissioners. They may also feel more
confident in supporting your organisation if they know you have been through a theory of change process.
• Bring the process of change to the forefront. All change occurs incrementally through intermediate
outcomes (steps which lead towards your final goal), such as improvements in service users’ skills or
changing public attitudes to disability using a campaign. A theory of change encourages you to focus on
these outcomes and articulate them properly.
Partnership working
• Help with partnership working. Developing a theory of change in collaboration with other organisations can
clarify roles and responsibilities, and establish consistency around outcomes. This could be especially useful
for partnership working between statutory and voluntary sector organisations. A theory of change can also
help train new staff or volunteers and replicate services, as it shows what a service aims to achieve and how.
6
Creating your theory of change | How to create a theory of change
The final goal should describe the change you want to see in service users or beneficiaries. It should be realistic
and succinct; you should not set more than a few final goals, and it is often best to have just one. A final goal
should be relatively long-term, obviously beneficial, and something that funders, commissioners or supporters
would be interested in funding. It must be relevant to the needs of the target population, and plausibly linked to
your project’s activities. Your organisation as a whole should articulate its final goal(s), which should be related to
your charitable objects and aligned to the goals of individual projects.
Thinking about final goals requires you to consider what your project or
Figure 1: Theory of
organisation is accountable for and what is beyond its sphere of influence. You
change elements
may decide to choose a grand aim for your final goal, such as ‘reducing
reoffending’ or ‘beating cancer’. Clearly, charities working in these areas will
aim to achieve these goals, but the extent of success will depend on a wide
variety of conditions and factors, many beyond their control. We suggest
drawing an accountability line between outcomes that you achieve directly
and longer-term goals to which these contribute. You will see accountability
lines in the charts in this guidance; however, where you draw the line is a
matter for your own judgement.
Once you have defined your final goal(s) you need to work backwards through
the steps or intermediate outcomes needed to achieve it. These can be
†
understood as the changes experienced by service users or beneficiaries that
will contribute to achieving your final goal. This is perhaps the most important
part of the process: too often organisations jump from their activities to their
final goals without thinking through the changes that need to happen in
between. Intermediate outcomes must be clearly articulated within your theory
of change and should be things that your project can definitely influence.
Outcomes should be feasible, given the scale of your activities; they should be
short-term, but should link logically to your long-term goal(s); and they should
ideally be supported by evidence.
At this stage, it is important to put aside any thoughts about how you will measure or evidence the theory of
change you are working on. A project should not be designed around what can be measured, and neither should
a theory of change. Any questions about future evidence collection should be left until later.
Once you have established your final goal(s) and intermediate outcomes, you need to consider how your
activities will make this change happen. Take each intermediate outcome in turn and think about how it links to
*
We use the term ‘final goal’. Other terms which mean the same thing are ‘final outcome’, ‘long-term goal/outcome’ and ‘vision’.
†
For most projects, intermediate outcomes relate to change in service users or beneficiaries. However, there are some projects,
like professional development, improved partnership working or organisational restructuring, where the intermediate outcomes
might relate to changes in staff or volunteers.
7
Creating your theory of change | How to create a theory of change
your activities. Consider the features that make the activities successful, and whether you have missed out any
intermediate outcomes.
You then need to think about enablers: conditions or factors that need to be in place for the project to work.
Internal enablers need to exist inside your organisation for the theory of change to work. They are mostly within
your control, and describe how you deliver your work—for example, the quality of services, relationships, and the
values and attitudes of staff. External enablers need to exist in the external environment for the theory of change
to work, and are often beyond your immediate control. They describe the context in which you work—including
social, cultural, economic and political factors, laws, regulations, and working with other organisations.
Throughout the process you should consider what evidence already exists that is relevant to your theory of
change. Ideally this will be in the form of references to published research, but you could also include your own
organisation’s experience and data. You may find some evidence that contradicts your theory. It is important to
think this through and, if necessary, modify what you do to reflect what the evidence tells you. If you do not have
evidence, it is useful to identify your assumptions about why one outcome will lead to another.
Only once you have a theory of change that you are broadly happy with should you start thinking about how to
measure and evaluate it. This is covered later in this guidance.
A theory of change often uses a diagram or chart to describe a project in visual form. However, this should
not be seen as the only element. The best theories of change demonstrate the rationale behind a project
and include:
• An introduction to what a theory of change is and a description of the process you have been
through to create it, including who has been involved.
• An analysis of the context and situation, which discusses the background to the project, and
outlines the problem in society you are trying to address, your target group, and their needs and
characteristics. This is sometimes called a ‘situation analysis’.
• References to existing evidence that relate to the theory of change, including evidence that your
organisation has already collected, and any relevant published research.
• Plans for measurement and evaluation that arise from the theory of change, including the details
of what you need to collect to test whether the theory of change is delivered. This is sometimes called
a ‘measurement framework’.
8
Creating your theory of change | How to create a theory of change
• The purpose of your theory of change: Carefully think through why you are doing a theory of change
before starting. If your main aim is to build a shared understanding across your team, then it will be important
to involve as many people as possible; if you are looking to communicate externally, you will need to work
towards a clear summary diagram and put effort into presentation; and if you want to improve your evaluation
approach, then you may need to focus on understanding the causal rationale underpinning your activities.
• Size and complexity of the project: You should be clear about your scope before you begin. It is possible
to produce a theory of change for a whole organisation or a single project or campaign. One approach is to
develop a high-level, overall theory of change for an organisation alongside more detailed ones for individual
projects.
• Stage of development: Some projects are well-established with a robust evidence base, and producing a
theory of change will be quite straightforward. Exploratory or innovative projects may require more thought.
• Direct or indirect impact on service users: Some projects are processes or system changes; for example,
an organisational restructure, a new IT system or a professional development network. These can be more
challenging to theorise as the link to end users is indirect, while the intermediate outcomes relate to changes
for your colleagues or stakeholders.
Involving people
In NPC’s experience, it is nearly always better to engage a range of people when developing a theory of change.
This may include practitioners, volunteers, managers, services users and external stakeholders. It is not
necessary or efficient to involve everyone with an interest in your work; a group of 3-10 people seems to work
well, depending on the size of the project.
You can develop a theory of change either in a workshop or by talking to people individually. Workshops are more
efficient and tend to be the most common approach. They need to be facilitated so that everyone feels able to
contribute, regardless of their position in the organisational hierarchy.
Developing a theory of change can be a demanding process and people can begin to lose focus after a few
hours. We suggest spending between half a day and a day on it at most. You can always reconvene the group
later, which will allow time for writing-up, reviewing and taking stock. A theory of change is an iterative process: as
well as sending the draft to the people who attended the workshop, you may find it useful to circulate it more
widely for further feedback, for example, to senior managers and partner agencies.
1
Box 2: The Ikea effect
This is the observation that people feel a greater commitment to and ownership of things they have helped
to create. We find this is the case with theory of change; the more people you involve at the start, the
stronger the commitment to the finished article.
9
Creating your theory of change | How to create a theory of change
To answer, you need to consider the existing evidence to support your assumption that each of your activities
contributes to the outcomes you want. You should focus on evidence for causal links between intermediate and
final outcomes; this is the hardest thing to measure and therefore where citing external evidence will be most
useful. Using existing evidence should help you to understand how the intermediate outcomes you aim for
contribute to your final goals.
If there initially seems to be no evidence directly relevant to your project, you can look further afield. There may
be evidence from other contexts or work with different types of service users or social problems that could help.
• Keep it simple! Avoid being too ambitious with your first theory of change.
• In workshops, start by brainstorming or writing on post-it notes to populate a general theory of change
and get your group talking. Only move towards specific issues and refinements once everyone has had
a chance to speak.
• Set aside time at the end of a workshop to write up notes; bear in mind that this will require time and
effort.
• Circulate the draft theory of change map to a wide group of stakeholders. This will help build
consensus and support, as well as gather information.
• Your theory of change will never be perfect. Avoid wasting time worrying too much over wording and
specific links. The main aim is to produce something that everyone broadly agrees with that is useful
for your aims.
• A theory of change should be seen as a working document. You can always update it to reflect
learning and new situations.
10
Creating your theory of change | How to represent a theory of change
•
©
The CES Planning Triangle
• Logic model
• Outcomes chain
• Written narrative
This chapter includes a description of each approach illustrated by examples, all of which relate to a hypothetical
supported housing project for ex-offenders. As you read the description of each approach, you may want to refer
back and forth between the diagrams.
Although theory of change approaches differ in appearance and focus, important aspects remain the same.
Theories of change should always be built on the key elements of consultation, drawing on existing evidence (if it
is available), and backwards mapping from outcomes to activities. They also have similar aims of helping think
through strategy, communicate activities and plan evaluation. In this section we outline each approach, so you
can decide what is appropriate for your organisation.
A theory of change is not only a diagram; it is also the process of thinking through and describing in full how
a project should work. The diagram is a representation and a summary.
• show a coherent causal model—a clear explanation of the causal processes that you think will lead to
your outcomes;
• communicate clearly.
Begin by agreeing your final goal and add this to the top of the diagram—the small amount of space encourages
you to keep it short. After this, populate the middle section with intermediate outcomes, asking yourself;
• What changes will your project help to make for your service users or beneficiaries?
11
Creating your theory of change | How to represent a theory of change
Next, add your intermediate outcomes to the middle section of the diagram. These should all be plausibly linked
to your project and occur within its lifetime. When developing intermediate outcomes, it is good to use words
associated with change, like: more, less, better, and improved. You can also think about intermediate outcomes
as the specific aims of your organisation’s different projects or interventions.
Finally, at the bottom of the diagram you should list the activities you will deliver to achieve the intermediate
outcomes. Each activity should have a direct link to one or more of the intermediate outcomes. If not, ask yourself
why the activity is included? It may help to number the outcomes and put the relevant number(s) against each
©
activity. An example of a CES Planning Triangle for a supported housing project is shown below.
Once the model is complete, you need to think about the indicators you will use to measure whether each
element is achieved. We return to the issue of evaluation later in this guidance.
Logic models
©
A logic model is similar to the CES Planning Triangle but allows for more detail, particularly about how a project
is delivered. Like the previous model, start with your final goal and work backwards to the intermediate outcomes.
It is often helpful to break these down into the changes you are trying to make to beneficiaries’ knowledge and
skills, attitudes and thinking, and behaviours, all of which should logically help them towards the final goal. Once
you have agreed your intermediate outcomes you should focus on activities, this time in more detail. You may
choose to describe:
• Inputs: The resources that go into a project, including the budget, the amount of staff time required, and the
relationships with other organisations.
• Activities: Activities are what you actually do. A logic model gives you space to break this down into different
components: you can describe the assessment process, the steps you take to become familiar with people
and build trust, and the schedule of the different activities you deliver. You can also think about how you
deliver these activities—the unique qualities you bring and what makes your project effective.
12
Creating your theory of change | How to represent a theory of change
• Outputs: These are the products, services or facilities that result from an organisation or project’s activities.
These are often expressed in the quantity of what is delivered; for example, the number of users, how many
sessions they receive and the amount of contact you have with them.
• Enablers: These are internal or external factors that are key to a project’s success. Internal enablers are
within your sphere of influence, whilst external enablers are beyond your immediate control.
An example of a logic model for a supported housing project for ex-offenders is shown on the next page.
A logic model should describe the key aspects that make a project work, from inputs through to the final
goal. It should be clear so that someone else can see what you have done and replicate it. It should also
provide a template for evaluation: data collection should be orientated towards testing whether each
element is delivered.
Logic models are straightforward to produce. They include detail about how a project works, and can therefore
inform decisions about what to evaluate. They are useful for communication as they represent projects in a
compelling way, while the connections between different elements are less complex than those in an outcomes
©
chain (see below). However, there are limitations to both logic models and the CES Planning Triangle .
©
Box 6: Limitations of logic models and the CES Planning Triangle
• The process of creating them can become a pro forma exercise in which people simply catalogue
project components without careful consideration of cause and effect.
• They can encourage people to list too many features, not all of which are fundamental to the change
process.
• They are not good at showing the dynamic features of a project. They create the impression that inputs
and activities happen first, followed neatly by outcomes; in most projects, different aspects occur at
different points in time.
• They have limited scope to plot sequences of outcomes and the subtle aspects of causality. For
example, in many projects there is an assumption that users will go through a change in knowledge
first, leading to a change in attitude and then behaviour, which is harder to reflect through these
formats.
These challenges can be overcome by using outcomes chains and narrative theories of change.
13
Creating your theory of change | How to represent a theory of change
14
Creating your theory of change | How to represent a theory of change
Outcomes chains
An outcomes chain is the diagram most closely associated with the term ‘theory of change’. Outcomes chains
differ from the two previous models because they focus more on causality: which activities are linked to which
©
outcomes in what order. In the CES Planning Triangle and logic models causality is implicit; it is assumed that
outcomes flow from the activities listed. An outcomes chain encourages you to think more about how and why
change occurs.
The main aim of developing an outcomes chain is to agree the central rationale for the project and the
conditions needed for success. Doing so should highlight the strategic choices you are making, allowing you to
discuss them. Outcomes chains are particularly useful if you are designing a new service, project, or campaign.
Developing an outcomes chain can be challenging because it forces you to think in more detail about the
sequence of outcomes, and requires more complicated visual expression than putting words in boxes and listing
inputs and activities. However, this effort can help you test and clarify the thinking behind a project.
Causality is the relationship between an event or events (cause or causes) and a second event or events
(effect or effects), where it is understood that the second is a consequence of the first. It is the key
evaluation question that social sector organisations face: how do we know our work makes a difference?
NPC has written guidance on comparison group studies, which is one way to address this question; but this
3
approach takes time and can be difficult for many organisations. The next best option is to articulate
causal relationships and then test them against the evidence you have—both external evidence that is
already available and evidence you collect through your own evaluation work.
Thinking about causality is particularly valuable because it should help your team agree the most important
aspects of your service, which will enable you to conduct the best possible evaluation. For more
4
information about causality, refer to NPC’s four pillar approach.
An outcomes chain focuses on mapping the sequence in which outcomes are intended to occur; by doing so, it
allows you to identify what is critical to success. Inputs and activities are summarised without much detail to
ensure they do not distract from the main aim of highlighting the key causal processes. Creating an outcomes
chain also requires careful consideration of the internal and external enablers needed for success.
To create an outcomes chain begin, as always, by agreeing your final goal and identifying which intermediate
outcomes lead directly into it. Ask what causes these intermediate outcomes, and then what in turn causes these
causes. You should continue this backwards mapping, focusing on cause and effect, through your work, always
ensuring you are capturing the process of change. You may need multiple outcomes chains to reflect different
elements of the project and different outcomes, which may be interrelated. Only when you are happy that your
outcomes chain captures all the causal relationships required for success across your work should you map your
activities onto it to show where they fit.
Causality is complex; mapping it is a challenge. Many things contribute to change for individuals, organisations
and society; it can be hard to capture them all in a single diagram. We suggest two ways to meet this challenge.
First, you could develop an outcomes chain that works at a high level. This is illustrated in Figure 4, where the
main elements of the housing support project are presented in a causal sequence. The diagram does not include
a description of how services are delivered; the aim is to simplify so that the main causal relationships are brought
to the fore. This high-level outcomes chain should highlight elements that are fundamental to a project’s
success—especially useful if the aim of the theory of change is to communicate to stakeholders, agree on a broad
strategy, and show how a project works.
15
Creating your theory of change | How to represent a theory of change
A second, alternative approach is to focus your outcomes chain on specific aspects of a project, or on individual
outcomes. This can be better than trying to capture all processes and outcomes in a single diagram, which can
take a lot of time and produce a complex chart that is hard to read.
Figure 5 illustrates this approach, showing an outcomes chain that outlines how a supported housing project
works to create pro-social, anti-criminal identities. This diagram is still fairly complex, despite only considering one
intended outcome of the project.
16
Creating your theory of change | How to represent a theory of change
Remember that a theory of change is a simplification of reality, and it is impossible to capture everything.
You will need to make judgements about what is important to include.
• Remove some of the detail from your diagram and include it in an appendix or narrative. Outcomes are
the most important part of the model, so think about taking out outputs and activities.
• Use references and footnotes to signpost people to more information and evidence.
• Think about expressing your theory of change at different levels of detail. A useful analogy is the view
of the earth from space. From high up you can only see the broad shape of things, but as you get
closer more detail comes into view. Similarly, you can develop an outcomes chain that shows broad
causal links alongside more detailed outcomes chains or logic models that describe specific elements
of projects or specific outcomes.
• Think about the sequence of change that a user goes through when they engage in your project or
organisation. A good theory of change can often be built around that sequence, with project activities
and outcomes mapped to each stage.
17
Creating your theory of change | How to represent a theory of change
As well as drawing out causal links, an accompanying narrative provides more detail on the aspects of a theory of
change that are central to achieving your intermediate outcomes and final goal(s). Theory of change diagrams
can be light on detail, so you should think about other key elements that are important for your theory of change
to work. These elements may already be included in the diagram, but this is an opportunity to add detail. Key
elements include:
• The context in which your organisation and its beneficiaries operate. This could be a description of who your
beneficiaries are and why they are engaging in your activities. It could cover wider social, political and
economic circumstances. Ray Pawson, professor of social research methodology, has defined four levels of
context that might be useful to consider: individual capacities, interpersonal relationships, institutional setting,
5
and the wider infrastructural system.
• The assumptions that show why you think one outcome will lead to another. Where academic or your own
evaluation evidence is not available, it is useful to identify the assumptions that underpin your theory of
change. This can help you understand the causal processes involved and avoid misrepresenting what you
are doing and why. If you have identified many assumptions, then prioritise them: include the assumptions
that are most critical to success, have the highest risk, are most unstable, you know least about, or have
implications for long-term success.
• The evidence that shows that your activities achieve your desired outcomes. Where you can, include
evidence that supports your activities, as well as evidence that one outcome leads to another. All
organisations working for social change want their projects to be successful; as far as possible, activities
should be based on knowledge of what works in a particular field. If evidence exists to show that your
activities are effective, you should include it; it will add weight to your theory of change.
• The internal and external enablers that need to exist for your theory of change to happen. Without
enablers, a theory of change cannot happen, so explaining your enablers is important. There may be many
enabling factors in your theory of change, but some areas we have found to be particularly important are:
− The quality of a service, activity or campaign. This will have a bearing on how successful you are at
achieving your goals. Consider questions such as:
- What should service users be thinking, feeling, or doing when they receive the service?
− The process of building relationships with service users or beneficiaries. This will likely be pivotal to a
project’s success. Most social programmes work through their ability to influence and change peoples’
choices, so describing how you intend that to happen is key. Similarly, successful campaigns are often
determined by the strength of relationships with key stakeholders (such as journalists, politicians or other
charities), so engagement of these individuals and groups will also be a determinant of success.
An example of a narrative theory of change for the supported housing project is shown in Box 9 on the next page.
18
Creating your theory of change | How to represent a theory of change
Context
• We work with people who have been released from custody with housing problems along with a range
of needs and a high risk of reoffending behaviour.
• There are numerous causes of offending behaviour. Direct causes include substance misuse, financial
problems, and lack of positive relationships. Indirect causes include criminal identities and attitudes,
lack of understanding of social norms, and limited aspirations.
Assumptions
• Providing secure accommodation offers ex-offenders a stable and controlled environment upon which
other activities can be based, thereby enabling us to address multiple needs.
• The relatively small number of units facilitates the development of close social bonds and mutual
support within the house. This helps people to learn and practice positive relationships with people
outside the house, including their families and the wider community. Relationships between staff and
residents are crucial to the change process.
• Residents need to develop self-esteem, confidence and resolve. They need to feel positive about
themselves and be resilient to setbacks. Change occurs incrementally and is supported across a range
of activities and life skills. Continual support is needed to offset setbacks and risks.
• Residents need to develop a non-criminal identity and need to understand the benefits of pro-social
behaviour, aspire to a crime-free life and feel themselves capable of living this way.
• With this level of time and effort, individuals with complex needs can be equipped to lead independent,
crime-free lives.
Evidence
•
6
Offenders with accommodation problems have been found to be more likely to reoffend.
•
7
High levels of impulsivity have been identified as a significant factor that predicts violent reoffending.
•
8
There is evidence that negative peer influences are a relevant factor in adult reoffending.
• An attitude that supports crime, such as anticipating and evaluating crime as worthwhile, has been
9
found to link to reoffending.
Enablers
• Residents need to develop strong relationships of trust, honesty and openness with staff, which will
facilitate the successful delivery of support (internal enabler).
• Success depends on the referral of appropriate people to the house; in particular we are unable to help
people until severe mental health or substance misuse use problems are stabilised (external enabler).
• Long-term success depends on the availability of good-quality housing for residents to move into, as
well as volunteering, education and work opportunities (external enabler).
19
Creating your theory of change | How to represent a theory of change
In NPC’s experience, the approach most suited to this additional level of detail is the outcomes chain, where the
©
focus is on the sequence of outcomes and causal links. Causality in the CES Planning Triangle and logic models
is implicit, rather than drawn out explicitly, which is why some in the field argue that outcomes chains are the ‘true’
approach to theory of change. For these reasons, the examples in this section are outcomes chains, where we
have focused in on sub-elements of the supported housing outcome map.
Assumptions
To make your assumptions explicit, include this detail in additional boxes between outcomes, to show why you
think one thing will lead to another. Assumptions can include common sense and intuition.
Evidence
Evidence and assumptions are shown in the same colour because they both explain the process of change. If you
have to make a choice, include evidence rather than assumptions as it makes a more compelling case for why
change occurs. Sources of publicly-available evidence include academic research and government data.
20
Creating your theory of change | How to represent a theory of change
Enablers
Internal and external enablers help ensure that your theory of change happens, so include these where
appropriate as additional boxes.
• Meaningful: Does it describe the project or organisation accurately in ways that staff, trustees, volunteers
and stakeholders agree with?
• Well-defined: Is a clear audience, client or user group articulated? Is it clear what you do?
• Comprehensible: Does it enable you to give someone the ‘two-minute story’ of the service? Would a
member of the public understand the theory?
• Do-able: Are the services and activities likely to contribute to the desired outcomes and impact?
• Plausible: Is it realistic? Does it take into account your organisation’s capacity? It should be something that
the programme, project or organisation could really do, not just wish it could.
• Credible: Are people outside your organisation likely to believe it? Is the secondary evidence you include
credible with your stakeholders?
• Testable: Can you test the theory through a series of testable hypotheses? All elements should theoretically
be assessed using research and observation (even though you may not have the resources to assess this
yourself).
Another useful way to test your theory of change is to add in expectations about numbers or flow. For example,
you might predict that 100 people will start on the project, 80 will continue to engage, 60 will achieve intermediate
outcomes, and 30 will achieve final outcomes. This will help you understand the scale of change to expect,
determine whether you have invested resources to achieve the change you want, and assess whether the
expected level of outcomes is worth the effort and resources invested.
21
Creating your theory of change | How to use your theory of change to create a measurement framework
A measurement framework built around your theory of change will ensure you collect information that tells you
what difference you are making. Designing this framework will involve deciding what data to collect, the level of
rigour of evidence you need, and how to go about collecting this data.
11
Our method for building measurement frameworks is set out in detail in NPC’s four pillar approach. This outlines
a process that your organisation can work through to build an effective framework to measure the right things in
the right way. It aims to provide results that can be used to understand and improve charities’ services, as well as
report on their progress.
Building an effective measurement framework relies on several key components that need to work together, as
illustrated in Figure 9. A solid foundation is essential: it requires good leadership and buy-in from senior staff and
trustees, a strong commitment to the value of impact measurement, and the investment of time, energy and skills.
Each pillar in the diagram represents a step in the process of creating a measurement framework. Below, we
explain each step in more detail.
Effective measurement
framework developed
Choose Select
Map your Prioritise
your your
theory of what you
level of sources
change measure
evidence and tools
Strategic vision
Leadership
Case for impact measurement
22
Creating your theory of change | How to use your theory of change to create a measurement framework
1. Has anyone already proved the causal link between outcomes in your theory of change?
If so, you can use desk research to gather this evidence rather than collect new data. For example, an
organisation that aims to reduce re-offending may have ‘reduced negative peer influences’ as an outcome in
its theory of change. There is already research to help demonstrate causal links between reduced re-
offending and reduced negative peer influences, so in this case the organisation could choose to focus on
measuring other outcomes.
This questioning process will help you prioritise a set of outcomes to measure. These outcomes form the basis of
your measurement framework.
12
There are four main ways to make a credible case that what you do really makes a difference:
23
Creating your theory of change | How to use your theory of change to create a measurement framework
1. Statistical approaches look for patterns in quantitative data to see if the effect (ie, the expected outcomes)
frequently follows the cause (ie, the service). Such approaches include before and after comparisons,
correlation, regression analysis, and other statistical models.
2. Experimental approaches compare differences in outcomes between people who receive a service (the
intervention group) and those who do not (the control group). The most rigorous way to select a control group
is to randomly choose who receives the service and who does not in a randomised control trial (RCT). Any
differences in the outcomes of the control group can be attributed to the service or to chance using statistics.
This approach helps distinguish a project’s impact from other factors that might affect the outcomes. The
most common criticism around RCTs are the cost involved and ethical concerns about denying people a
service.
3. Case-based approaches compare cases (eg, individuals, groups of people, or places) within an intervention
or programme or across interventions or programmes, to draw conclusions about causes of the effects and
the impact of programmes. This approach recognises that there is rarely a single cause to any social
outcome and that it is hard to unpick all the various influences.
4. Theory-based approaches describe in detail how a service or programme influences different people at
different times and places using observations by staff, evaluators, and other stakeholders, as well as the
views of beneficiaries, rather than by analysing lots of cases or using a control or comparison group.
These different types of evidence are not mutually exclusive; they can be combined to bolster your claims that a
project or service makes a difference.
The outcomes you prioritise and your decision about how rigorous your evidence needs to be will determine the
type of data you should collect and how you should go about it. There are three key questions to answer:
• Who do you need to collect data from? Who you collect data from should reflect the groups your
organisation interacts with—the beneficiaries of a service, staff members, practitioners using your resources,
or policymakers you are trying to influence. It may also be appropriate to consider groups with whom you
have indirect contact, such as beneficiaries of an organisation that you directly support. Tracking a sample of
people rather than everyone you work with cuts costs without sacrificing the quality of the evidence. This is a
must if you work with a large number of people.
• What type of data should you collect? Data is most commonly categorised as quantitative or qualitative.
Quantitative data helps you to determine whether change has taken place, while qualitative data helps you to
understand why and how. Quantitative data is aggregated evidence of the extent of the impact you are
having. Qualitative data, including case studies, is more effective in communicating the detail of what has or
can be achieved and how. The data types are complementary, and NPC promotes the collection of both
where possible, bearing in mind the mantra: ‘No numbers without stories; no stories without numbers.’
• When should you collect this data? A key decision to make is when and how often to collect data.
Resources, funder requirements and the types of activities you carry out, as well as methodological and
practical considerations, will all influence this decision. You will need to think about collecting baseline data
before an intervention takes place so you have a point of comparison for assessing your project or
organisation’s impact. You should also consider when you will collect the data—a common pattern is to
collect data before and after an intervention—and whether and when to follow up.
24
Creating your theory of change | How to use your theory of change to create a measurement framework
The final stage in developing your measurement framework is identifying how to get your data—the approach you
take and tools you use to collect it. First, you should think about what data already exists and how to use that
where possible. Then you can identify new sources of data and find or develop appropriate tools to collect it.
Figure 10 presents a number of issues to consider when choosing the best tool for you.
Area What to consider when choosing NPC’s tips when choosing tools
tools
• What is the mix of ‘soft’ or ‘hard’ • Choose standardised and validated tools
outcomes in your work? where you can.
• Does the tool fit the outcomes you • Try to get hold of existing administrative/
Outcomes want to measure? statutory data for hard outcomes.
• Can you both measure change and • Use quantitative tools to measure change and
understand why change has qualitative tools to discover how change
happened? happens.
• Do you work with people over a long • Consider how you might follow up with people
period of time? in the long term, such as contacting a random
• Do you work with people one-to-one sample or using statutory data.
Activities/ or in an intensive way? • Resource-intensive tools may not be
intervention • Is your service innovative or appropriate for light-touch interventions.
established? • If you are delivering a new activity or trying to
• Are you trying to scale your scale you should invest in a robust approach
approach? to measuring your impact.
25
Creating your theory of change | How to use your theory of change to learn and improve
Once you have developed a theory of change, described how it is supported by existing evidence, and collected
your own evidence, you can determine whether you have achieved the outcomes you intended. You will also be
in a position to learn from your results and improve your services.
Theory of change should be at the heart of your strategy, and should be a living document. Whether your impact
measurement results are good or show room for improvement, organisations in the social sector should be
constantly striving to improve their services and do better for the people they serve. This is why learning and
improving are so important.
There are two key ways that you can use your results to help achieve better outcomes for beneficiaries:
For example, if you find that one-to-one support from key workers is improving the aspirations and self-awareness
of female ex-offenders, but not their male peers, think about why this might be happening and what you can do to
address it.
You should also consider whether you have sufficient data to allow you to answer these questions—are you
measuring the right things? As you develop a better understanding of how your activities and outcomes link
together, could you update your theory of change to more accurately reflect this?
It is important that your organisation has a culture of continuous learning and improvement. If you decide to make
changes to your services, you should continue to assess your results to see if these changes are working and
delivering outcomes. One way to embed learning within an organisation is to give a member of your board or
26
Creating your theory of change | How to use your theory of change to learn and improve
senior management team responsibility for ensuring you regularly review and act on results data. It is also
important to engage frontline staff and show them that the data they gather is put to good use—share results with
them, and show how you are using the information to improve services.
Our regular training sessions allow you to dedicate a day to understanding theory of change, so you can
choose the right approach and make a start on your own, with our experts on hand to answer any
questions—check the events section of our website for the latest dates. We can also work with you in depth
to develop a theory of change as a basis for clarifying strategy, measuring your impact or thinking about
your place in the sector. Do get in touch at info@thinkNPC.org for more information.
27
Creating your theory of change | Further resources
FURTHER RESOURCES
We have selected further online resources to support you in creating a theory of change.
http://www.thinknpc.org/publications/theory-of-change/
• Charities Evaluation Services’ introduction, Making Connections:
http://www.ces-vol.org.uk/tools-and-resources/Evaluation-methods/making-connections-tools
http://betterevaluation.org/plan/define/develop_logic_model
http://www.sibgroup.org.uk/impact/help/plan/
http://www.theoryofchange.org/
• A Prezi looking at the issue of developing theories of change for complex projects:
http://prezi.com/cjutwzfsfspe/theory-of-change-for-complex-interventions_mary-de-silva/
http://www.scotland.gov.uk/Resource/Doc/175356/0116687.pdf
www.thinkNPC.org/events
28
Creating your theory of change | References
REFERENCES
1
Wikipedia (2014) The Ikea effect.
2 ©
Charities Evaluation Services (2014) CES Planning Triangle .
3
NPC and Clinks (2014) Using comparison group approaches to understand impact.
4
Kazimirski, A. and Pritchard, D. (2014) Building your measurement framework: NPC’s four pillar approach. New
Philanthropy Capital.
5
Pawson, R. (2006) Evidence based policy: A realist perspective. London, Sage.
6
May, C. (1999) Explaining reconviction following a community sentence: the role of social factors. Home Office
Research Study 192. London: Home Office.
7
Craig, L., Browne, K., Beech, A. and Stringer, I. (2004) ‘Personality characteristics associated with reconviction
in sexual and violent offenders’. The Journal of Forensic Psychiatry & Psychology, Vol 15 No 3.
8
Cobbina, Huebner and Berg (2012) ‘Men, women, and post-release offending: An examination of the nature of
the link between relational ties and recidivism’. Crime & Delinquency, Vol.58, 3, p331-361.
9
Brezina and Topalli (2012) ‘Criminal self-efficacy: Exploring the correlations and consequences of a “successful
criminal” identify’. Criminal Justice and Behaviour, 2012: 39.
10
Robert Wood Johnson Foundation (2010) New Grantee Seminar—Creating your Theory of Change.
11
Kazimirski, A. and Pritchard, D. (2014) Building your measurement framework: NPC’s four pillar approach. New
Philanthropy Capital.
12
Befani, B., Davies, R., Forss, K., Mayne, J., Stame, N., and Stern, E. (2012) Broadening the range of designs
and methods for impact evaluation. Department for International Development, Working Paper 38.
13
Ní Ógáin, E., Lumley, T. and Pritchard, D. (2012) Making an impact: Impact measurement among charities and
social enterprises in the UK. New Philanthropy Capital.
14
Kail, A., Plimmer, D., Ní Ógáin, E, Harries, E. and Copps, J. (2014) The Journey to EmploymenT (JET)
framework. New Philanthropy Capital.
29
TRANSFORMING THE CHARITY SECTOR
NPC is a charity think tank and consultancy which occupies a unique position
at the nexus between charities and funders, helping them achieve the greatest
impact. We are driven by the values and mission of the charity sector, to which
we bring the rigour, clarity and analysis needed to better achieve the outcomes
we all seek. We also share the motivations and passion of funders, to which we
bring our expertise, experience and track record of success.
Increasing the impact of charities: NPC exists to make charities and social
enterprises more successful in achieving their missions. Through rigorous
analysis, practical advice and innovative thinking, we make charities’ money
and energy go further, and help them to achieve the greatest impact.
www.thinkNPC.org