Guidance on M&E

for
Civil Society Programs

Prepared for AusAID Program Managers

Linda Kelly
Rosalind David
Chris Roche

December 2008
Demand for Better Governance Unit
 Table of Contents

Acronyms_____________________________________________________________________ 2
A note for senior managers ______________________________________________________ 3
Introduction __________________________________________________________________ 4
Monitoring & Evaluation at the design stage________________________________________ 6
Monitoring & Evaluation during implementation __________________________________ 13
Considerations for reporting ____________________________________________________ 21
Evaluation and redesign________________________________________________________ 23
Annex 1: Monitoring and Evaluating AusAID Community Development Programs Initial
Discussion Paper ______________________________________________________________ 25
Annex 2: Framework for the Analysis of the Quality of Gender Integration in Programs__ 45

1
Acronyms
ALNAP Active Learning Network for Accountability and
Performance in Humanitarian Action
APPR Annual Program Performance Review
AusAID Australian Agency for International Development
BPFA Beijing Platform for Action
CDS Community Development Scheme
CEDAW Convention on the Elimination of All forms of
Discrimination against Women
CPP Churches Partnership Program
CSO Civil Society Organisation
DFID UK Department for International Development
ECDPM European Centre for Development Policy
Management
HAP Humanitarian Accountability Partnership
HIV/AIDS Human Immune-deficiency Virus/Acquired Immune
Deficiency Syndrome
ICR Independent Completion Report
ILO International Labour Organization
IUCN International Union for Conservation of Nature
M&E Monitoring and Evaluation
MTR Mid-Term Review
NGO non-Government Organisation
OD Organisation Development
ODI Overseas Development Institute
PNG Papua New Guinea
QAI Quality at Implementation
UNDP United Nations Development Program
UNEP United Nations Environment Program

2
 A note for senior managers
The following guidance has been developed for program managers who have responsibility
for managing programs focused around civil society engagement. It discusses the different
M&E requirements for such programs, with particular reference to gender in civil society. The
guidance has been developed through review of best international practice (see Annex one)
as well as consultation with AusAID staff and program implementers.
There are several key messages in the guidance that have implications for the time and
resources program managers require in order to be effective in the management and
monitoring of civil society programs. These include:
1. Engaging with civil society is a different approach to aid delivery from other forms of
aid that AusAID utilise. The M&E will likewise be different, usually more participative
and more concerned with change and improvement over time.
2. Good quality work with civil society takes time. It is not helped by constant changes or
new ideas which interrupt ongoing processes. Program managers should be
supported to take longer term approaches to civil society programs, allowing
processes to develop and seeking results over the longer term
3. At the same time, civil society programs usually change and evolve, particularly as
they begin to learn what works, and as they innovate. Contracts and other formal
systems need to be able to support such changes, allowing monitoring and evaluation
processes to lead to better program implementation.
4. Effective civil society processes are usually built on the foundations of trust and
mutually accountable relationships. These take time to construct and need the space
for ongoing dialogue to be created. A major emphasis needs to be put on this,
particularly in the first stages of program development. Tangible ‘results’ can only be
expected after this development.
5. Good M&E for civil society and gender programs will deliberately seek information
from different people and different sources. It can be expected to generate a lot of
information that will not be amenable to simple aggregation. Reporting on outcomes
should not be equated with simple indicators or bland generalisations. Civil society
outcomes need to be understood in context and will vary across locations. Effective
M&E will acknowledge this.
6. In order for different perspectives to be heard, and for genuine feedback to be
generated, the difference in the power of different stakeholders needs to be
recognised. Deliberate steps will need to be taken to minimise these differences and
provide opportunities for those with less power to engage in assessment processes.
7. The manager will have to do more than simply receive reports if they are required to
make good quality judgements about programs. They have to be part of the analysis
process, drawing upon the formal data collection and also informal sources of
information. This will take additional time and ideally should involve the program
manager in being outside the office, engaged with program implementers and also
with those the program seeks to support.
8. In order for Program Managers to manage civil society programs in this way, they will
need their managers to:
a. support creative and innovative ways of approaching their work;
b. model an openness to feedback, genuine dialogue and an awareness of
power and gender relations;
c. assist with and support adaption of processes and systems;
d. make available the necessary time and resources for them to undertake the
management of these programs.

3
Introduction
AusAID supports a range of civil society programs across several countries. These include
large scale bi-lateral programs as well as smaller projects focused on particular sectors and
programs which support the work of non-Government Organisations (NGOs).
Civil society work is different from many of the other areas of development work supported
through the aid program. It explicitly seeks to serve people through enabling them to
undertake their own development, relying as far as possible upon their own skills and
resources. As such it focused on enabling and empowering processes. The ways things are
done, in particular the way in which people are able to control their own development, is as
important, perhaps more important, than what is achieved. This holds true for all aspects of
civil society programs including the monitoring and evaluation of such programs.
In light of these differences and the challenges it presents for aid management, this
document provides guidance for AusAID program managers about how to approach
monitoring and evaluation (M&E) for civil society programs. It provides guidance for the
AusAID program manager about what to look for (and avoid) at the key stages of a program
in order to ensure that civil society processes are maintained and enhanced through the
assessment processes. Attention is also given to how gender should be addressed
throughout assessment processes.
The guidance has been developed after review of international best practice (see Annex
one), which highlighted the specific difficulties of civil society work and the challenges these
raise for M&E. Consultation with AusAID program managers and program implementers has
also informed this guidance.
The guidance is divided into four sections:
1) Section 1 covers design. This section considers aspects of analysis and design which are
relevant to M&E.
2) The second section focuses on implementation and how M&E should be expected to
evolve through this period.
3) The next section looks at reporting.
4) The final section addresses evaluation of civil society and community development
initiatives.
A short section for senior managers is also attached. This short summary is designed to help
them understand and assess the resources and skills they need within their program staff for
managing civil society programs.

The role of the program manager

While the role of the AusAID program manager does vary across Posts and programs, it is
generally true that the manager is expected to manage, not ‘do’. That is they are responsible
for checking on the overall quality of the M&E system, ensuring the agreed system is
correctly applied in practice and then verifying through informal and other means that the
M&E system is capturing the important changes and informing program practice.
As is suggested throughout this following guidance, for M&E of civil society programs the
role expands beyond this. The program manager cannot manage these programs effectively
unless they also engage with some aspects of the M&E process. Nevertheless they remain
responsible to AusAID for the management of programs, not the implementation. The
program manger should see their role as a bridge, linking effective M&E and the AusAID
requirements. For civil society programs this will require some active management and
interpretation of one to the other.

4
 A Summary of some key questions about the Monitoring System

Programme analysis and design

• Are you clear about the theory of change

and power relations?
• Are you clear why you are engaging with
civil society?
• Have you developed relationships of
trust with key partners?
• Has there been adequate engagement
with relevant individuals, organisations or
networks representing gender relations
in this context?
• Do you share a common vision with
partners?
• Is the initial design sufficiently flexible to
allow for future responsiveness to the
changing/ emerging context?
• Have you jointly (with your stakeholders)
articulated what `success’ will look like
and created the processes and space to
review this together?
• Have you reviewed the program to
assess how effectively gender has been
integrated?

Reporting and evaluation Implementation and monitoring:

• Are you avoiding the over-aggregation • Does the Monitoring system

of monitoring data, and looking for a establish regular ways of seeking
variety of ways to for it to be dynamic feedback from multiple
presented? sources about the benefits,
problems and impacts of the
• Have you taken into account the intervention?
different information needs that • Does the system encourage staff
stakeholders may require? and stakeholders to create regular
(sense making) spaces for analysing
• Is the purpose of any review or and reflecting on the underlying
evaluation clear? assumptions or theories of change?
• Does the M&E encourage
• Has adequate time and space been adaptation and responsiveness to
reserved for staff to engage effectively the changing environment so that
in review processes? learning influences on-going
activities/plans?
• Have proposed evaluation processes • Does the M&E seek to explore
(including any team assembled) `surprises’ and understand the
adequately addressed assessment of differing effects on women and men
gender relations? and gender relationships.
• Is the system simple, light and
• Are plans in place to ensure that useable?
review or evaluation findings are fed • Is the monitoring and evaluation
back into planning? system CENTRAL to the
developmental work providing useful
and used information or is it just a
technical add on?

5
Monitoring & Evaluation at the design stage
Good monitoring and evaluation starts at design. Critical to this stage is to ensure the correct
building blocks are in place in order that effective monitoring processes can be developed
during implementation. Failure to give attention to these processes during design will mean
considerable difficulties for later monitoring. For civil society programs, failure to address the
following issues prior to implementation may well mean they are never effectively monitored.
The following guidance is not intended to be a complete guide to designing civil society
programs. It refers to the elements of such programs which should be considered in order
that the monitoring plan is able to be developed and successfully implemented.
1. For design of civil society programs, it is important to situate an analysis of civil
society within a broader picture of change and power relations.
Understanding how social change happens in particular contexts is fundamental to how
poverty reduction and the achievement of gender equality is approached. Change is usually
political, and often powerful interests and distorted incentive systems keep pro-poor gender
sensitive change from happening. Change occurs through complex inter-relationships
between institutions, individuals, and their physical and cultural environment. This is highly
context specific and is the product of a variety of historical economic, social and political
processes. By combining an analysis of power, and what keeps change from happening,
and an analysis of the factors that motivate and stimulate change, we can begin to paint
scenarios for how change might occur in different contexts. Some of the key questions that
need to be asked for all programs, including those which try to engage with civil society, are:

• What are the incentive systems, power structures, gender relations and interests that are
holding a given situation in place?
• What and who are the drivers for positive change in general, and within civil society in
particular?
o To what extent is leadership (or action) by key individuals in power critical? How
might leaders be influenced? What are the key policy and practice changes required?
o To what extent are broader shifts in ideas, beliefs, and attitudes key to changes in
behaviour and practice?
o What are the opportunities afforded by new technology, ideas, networks and
knowledge?
o What historical examples of change or ‘success’ in this, or similar, contexts are there
that can be learnt from, or scaled up?
o Who is driving change through promoting changes in rules, incentives, systems and
innovation? Or who has the potential to do so? What support do they need?
• People are usually at the heart of change. It is therefore important to understand what
motivates, what constrains, and what is required to move from change at the level of a
few individuals or communities to larger shifts in norms and behaviour, whether by
individual leaders, institutions, or amongst the public more broadly.

6
 The Unfinished State: Drivers of Change in Vanuatu

One area that this study looked at was how to strengthen traditional structures, so that they
become an integral part of the state structure. They came across a number of ideas for this
which included;
• Formally recognising the role of chiefs in community governance, and reinforcing their
obligation to respect individual rights, and to work in a consultative manner with their local
communities.
• Supporting churches and NGOs to act as facilitators for institutional development in local
communities, helping to transfer knowledge and skills required to access government
services.
• Revitalising Area Councils by strengthening their role in community development, giving
them a stronger institutional base and allowing them to retain local revenues to fund
development projects.
• Supporting the Island Council of Chiefs to play a more active role in setting regional
development priorities and pushing provincial government to be more responsive.

The report notes that the ‘significance of these ideas is that they represent proposals developed
by ni-Vanuatu for strengthening the democratic process in a culturally appropriate way…. that
would help to counterbalance some of the shortcomings of the formal political structures’. The
authors suggested one of the useful interventions that AusAID could make is to create
opportunities for such a debate, including helping different stakeholders to develop and articulate
their positions.

See: Cox et al (2007) The Unfinished State: Drivers of Change in Vanuatu

2. If AusAID is not clear about the reasons for the civil society program then it will
not be possible to develop a coherent approach to assessment. Be clear about
why you are engaging with Civil Society in the first place.
The role, structure and strength of civil society organisations (CSOs) vary enormously even
within countries. The relation between CSOs and the state, and the private sector is also
extremely varied. As a result different CSOs will play different roles within any given society
and their relative power and autonomy will vary. Therefore any engagement by AusAID with
CSOs needs to be based on a clear understanding of the context and existing power
relations. This includes having a good analysis of AusAID’s own power and the risks as well
as benefits for civil society groups in engaging with AusAID.
CSOs reflect the societies and communities of which they are part. Gender, class or ethnic
inequalities may be embedded in these organisations. A careful analysis of the degree to
which CSOs reproduce or challenge these inequities should be central to decisions about
engagement.
Civil Society groups are also wary of being ‘used’ by donors. Care needs to be taken to
respect the mandates and autonomy of CSOs. In some contexts it may even be
inappropriate and counter-productive for AusAID to engage directly – or even indirectly - with
CSOs.
The key message however is that the program must be clear about why in this particular
situation AusAID seeks to work with civil society. Assessment of outcomes is much harder
without clarity about this starting position 1 .

1
AusAID is currently developing further policy about engagement with civil society under various
programs. If this is unavailable at the time of the design the program manager needs to refer to
his/her senior manager to ensure the program rationale is very clear for all stakeholders.

7
There are some key questions which might help guide the development and design of civil
society programs in different contexts:

• To what extent is there an enabling environment for civil society engagement? 2 What
is the relationship between the state, parliamentarians and different civil society
groups?
• What are the strengths and weaknesses of different civil society organisations in:
o Giving voice to stakeholders and constituents, particularly of people living in
poverty, women and marginalised groups?
o Providing professional and local expertise and increasing capacity for
effective and gender sensitive service delivery, especially in environments
with weak public sector capacity or in post-conflict contexts?
o Promoting public sector transparency and accountability?
o Promoting public consensus and local ownership for reforms, national poverty
reduction, gender equality and development strategies?
o Bringing innovative ideas and solutions, as well as participatory gender
sensitive approaches to solve local problems?
o Strengthening development programs by providing local knowledge, targeting
assistance, and generating social capital at different levels?
• How best might civil society organisations contribute – directly or indirectly - to the
achievement of Millennium Development Goals, National Poverty reduction
Strategies and Gender Equality?
• How best might AusAID either indirectly - through helping to create an improved
enabling environment - or directly - through supporting different civil society roles –
strengthen civil society?
• What staff, resources or partnerships does AusAID need to effectively engage with
civil society in this particular context?
The text box below illustrates the different variety of roles that CSOs play within the health
sector, many of these roles, and others, are played by CSOs in various sectors. CSOs can
also play such roles across multiple sectors for example through human rights work.

Roles of Civil Society in the Health Sector

• Delivering Health services directly or by facilitating community interactions with service providers
• Piloting innovations and sharing lessons learnt
• Under-taking health promotion, information exchange and building informed public choice on
health
• Engaging in Policy setting by representing public and community interests in policy or promoting
equity and pro-poor policies;
• Mobilizing resources to finance health services, or building public accountability and
transparency for resource use.
• Monitoring the quality of care and responsiveness of service providers by giving voice to
marginalized groups

See: http://www.who.int/civilsociety/documents/en/alliances_en.pdf

2
See for example the ARVIN framework developed by the World Bank which is a tool to assess: the freedom of citizens to
associate; their ability to mobilize resources to fulfil the objectives of their organizations; their ability to formulate and express
opinion; their access to information (necessary for their ability to exercise voice, engage in negotiation and gain access to
resources); and the existence of spaces and rules of engagement for negotiation, participation and public debate.

8
3. As part of developing relationships all stakeholders need to come to a shared
view about what success will look like. This is unlikely to be possible at the
beginning of a new civil society program and must be part of the ongoing
processes.
Once it is relatively clear why AusAID might engage with parts of civil society then building
effective relationships and trust is critical. This can take some time and normally evolves as
organisations get to know each other. It can therefore be difficult in the early stages of a
relationship for complete honesty and openness to prevail, particularly if power differentials
between the parties are large. Establishing very specific aims at this stage therefore can be
misleading or unhelpful. However agreeing broad outcomes and what `success might look
like’ can also be a key step in building common understanding and trust.
At this stage therefore the critical issue is determining how best the people or organisations
who are supposed to benefit from this initiative are going to be involved in articulating what it
is the program hopes to achieve, and what `success’ would look like. As AusAID requires
clarity about objectives to meet the Quality at Entry requirements it is important to determine
overall broad objectives at this stage 3 . Over time more specific objectives can be expected
to develop. The program design needs to be flexible enough to enable this development.

The mid-term review of the PNG Churches Partnership Program identified a key reason for its
success was the lack of specific predetermined objectives in the original design. Rather three
overall outcome areas were agreed between AusAID, the churches and their Australian NGO
partners. This allowed each Church and NGO to work together to explore what they could
achieve in each outcome area. This flexibility has allowed for considerable experimentation and
learning leading to significant change in the churches and in their ways of working.

Churches Partnership Program, Mid- Term Review, Final report. May 2007

Whether AusAID is directly or indirectly engaging with CSOs – for example through a sub-
contractor or NGO – it is important that the emphasis in the development of proposals at this
stage is on ensuring effective and genuine processes of participation and dialogue, rather
than defining detailed indicators and plans. This needs to include ensuring the involvement
of different groups of people (based on gender, age, ethnicity etc) – and particularly those
whose voices might not normally he heard. In cases where the intervention will be working
through intermediaries who may not [yet] have a direct relationship with communities, i.e.
research or advocacy groups, it needs to be clear how the interests of those communities
will be properly represented and monitored.

3
For design of civil society programs the notion of objectives is best understood as clarity about the
overall ’intent’ of the program, not as specific statements of tangible results. All collaborations start
with some shared understanding of success. At first establishing and monitoring the relationships that
allow that understanding to be developed is the most important thing, so there is no point in having
more specific objectives at the outset than the shared understanding will bear. But more specific
understanding of success (and therefore objectives) will be developed as the activity goes on.

9
 The Stages of the 5 D Approach
The approach to community engagement is referred to as the 5 D approach because the there are five
steps through which the community is led:
1. DISCOVER: Story telling in groups to discover what has worked in the past or what is valuable and
appreciated.
2. DREAM: Drawing a picture of the ideal school based on what the community is capable of
managing. What direction do we want to move towards, how do we imagine it should look?
3. DESIGN: Deciding priorities with separate inputs from women, youth and men. In a democratic way
choosing together what options there are for getting to where we want to go (based on 1& 2
above).
4. DEFINE: Mapping the available capacities or resources, called assets available to the community
through their own networks, lands and skills. What assets exist now - what do we have that we
can contribute
5. DELIVER: Drawing up an Action Plan to describe: Who will do what when for the whole community?
AusAID PNG Basic Education Development Project: Community Participation Annual Evaluation Report 2008

Depending on the context and the type of change 4 envisaged, different approaches to
defining what success looks like may be needed. In very complex environments 5 where
transformational change is envisaged, this may require focusing on imagining what success
might feel like, look like, and smell like in very creative ways. For example imagining the
“future backwards 6 ” is a technique which deliberately recognises sometimes we need to
break with the present to create a new future. A focus on logic and indicators in these
circumstances and at this stage might destroy the passion, experimentation and creativity
needed in such circumstances.
Whereas in ‘simpler’ environments, or in addressing more immediate concerns where cause
and effect are clearer, for example in providing oral rehydration to children suffering
diahorrea, then a clear logical, linear step by step, process based on best practice would be
appropriate, and success can be defined quite specifically using indicators.

Domains of Change, Reflective Questions and Indicators

Domains of Change describe broad areas of change that are desired. In complex environments
and early in partnerships it is often inappropriate to overly define objectives or indicators at this
stage. Domains of change are usually sufficient to be able to test more experimental approaches,
not least because prescribing precisely how change is going to occur is not possible.

Reflective Questions are generic questions that are consistently asked at all stages of a
program’s life and which reflect the fundamental issues that stakeholder’s agree are key in
monitoring progress. Agreeing these questions and asking them regularly – whilst answering them
in different ways – can help provide coherence in diverse circumstances.

Indicators. Indicators are useful in much simpler environments, where projectable change is
possible and when partnerships are relatively mature. In more complex environments and when
trust is not yet established they can create perverse incentives and constrain creativity and
experimentation.

4
See Discussion Paper and paper by Doug Reeler
5
See Kurtz and Snowden: The new dynamics of strategy: Sense-making in a complex and
complicated world
6
For an example see Simon Zadek’s 2004 paper on the UN and Civil Governance in 2020

10
4. Theories of change which underlie the civil society program need to match the
context and also the analysis undertaken by AusAID and partners.
Once there is broad agreement on desired changes there needs to be a process of
determining how best this might be achieved in ways which are consistent with the context
and nature of the political, economic, social, and cultural environment, recognising that in
many cases defining precisely what you want to achieve, and especially how you will do it
and by when, will not be possible at the outset, and will change over time, as relationships
and learning evolve.
However, attempting to agree with partners a common theory, or hypothesis, of change is
important for at least two reasons: it identifies where there are differences between people,
and it also provides a map of how change might happen which can be monitored over time.
People often have a very clear understanding of what needs to change and what behaviour
should be different in order to bring about that change. By allowing the space and time for
this detailed discussion the understanding of how change is likely to come about in a specific
context is able to be strengthened and based on real knowledge and experience.
Some times AusAID and CSOs will disagree about what needs to change and how. Such
differences should be identified and discussed and in some cases can be resolved or
worked through early in the activity. In other situations differences will remain or emerge
during the course of the program – this is normal. The design should acknowledge different
views where they exist. It should highlight the risks this raises for the activity and how these
will be monitored and managed during the implementation stage.
5. The M&E of the civil society program needs to support the approach being taken to
civil society by the program. Therefore it is very important that M&E data needs,
processes and spaces are addressed as part of the design. In addition this is the
point where gender needs to be properly considered and integrated into the
design.
The design process for civil society programs should be the start of an on-going participatory
monitoring process (see next section) which identifies
a) What are the major areas of change desired from the perspective of different groups?
b) What changes do we expect to see over the next period (1 year/18 months etc)?
c) How will we monitor progress over this period and who will be involved?
It is critically important that these questions are addressed during the design stage, even if
the answers change, or are refined, later. In particular it is necessary to create – and protect
- the necessary space, time and resources for ongoing reflection, sense-making, learning
and adaptation. Activist organisations often prefer to do, rather than reflect. Bureaucratic
organisations often put the emphasis on data collection and reporting, and ignore the
importance of the collective sense making processes. Both therefore need the discipline and
incentives established to create the necessary ‘down-time’ for analysis of and reflection on
the monitoring data.
These processes are also liable to be multi-layered i.e. with communities and front line staff;
between intermediary organisations and AusAID staff; in AusAID itself; between AusAID and
Governments. How the links between these levels are constructed, what overlaps are
possible, how findings are collected and shared with others are all important questions to
address at the design stage 7 .

7
The process of selecting Most Significant Change stories and these being discussed at different levels in
organisations, with feedback being provided, is an example of a multi-level sense-making process.

11
And in particular, gender
Many of the elements of good practice for civil society programs are similar for gender i.e.
the need for good contextual analysis, the importance of empowerment and relationships,
and the challenges of sense-making and addressing power relations. In addition for gender it
is also important to consider the ‘institutional’ obstacles to progress both within AusAID and
within stakeholders. Attention to these areas at the design stage allows for ongoing
assessment of change in gender relations over time in programs.
Some key gender questions 8 for consideration by Program Managers at the design stages
include:

• Is the proposed program based on a rights-based social analysis that demonstrates

the links between poverty and gender?
• Has an adequate assessment been made of past/current approaches to addressing
gender inequality in this context, both in terms of Civil society and in terms of the
Institutional context (i.e. the ‘rules of the game’ at policy level or in terms of norms,
ideas and beliefs)?
• Has there been adequate consultation and engagement with relevant individuals,
organisations and networks representing women’s interests and with particular
expertise in gender relations in this context?
• Are the analysis and the assessment appropriately translated into Program Plans
and strategies that will enhance gender equality?
• Is there an appropriate strategy in place for strengthening gender mainstreaming, for
both AusAID staff and other stakeholders?
• Are there mechanisms proposed for monitoring, evaluating and learning from progress in
both institutional 9 (i.e. organisational policies, staff capacities etc) and programmatic
terms (i.e. development outcomes) included in the design of M&E systems?

Some Do’s and Don’ts at Design Stage

Do
• Engage in effective power and gender analysis as key drivers of change,
• Focus on building a common vision of change tailored to the local context
• Ensure that there is an appropriate and representative mix of perspectives involved,
based on the power and gender analysis
• Ensure that adequate time, space and resources are built into the design for ongoing
reflection, sense-making and learning
• Ensure gender is properly considered at analysis and design

Don’t
• Push CSOs or subcontractors working with CSOs to focus on narrow objectives at the
expense of the process and creation of trust
• Assume that the design or analysis is correct, it will need to evolve over time,
• Instrumentalise CSOs - respect their mandates and autonomy,
• React punitively if your policies are challenged by CSOs, try and navigate different
views with curiousity.

8
See Appendix 2: for a Framework for the Analysis of the Quality of Gender Integration in Programs
9
See Tool 2 on pages 13 and 14 of CIDA’s framework for Assessing Gender Equality Results

12
Monitoring & Evaluation during implementation
Good M&E systems for civil society programs are ones which are:

• Dynamic: Systems which encourage `learning by doing’ and are promoting regular
ways of seeking dynamic feedback from multiple sources about the benefits,
problems and impacts of the intervention.
• Participative and Gender Sensitive: Systems which actively seek to overcome
barriers of gender, age, power, culture and other issues which limit the participation
of all stakeholders in the monitoring and assessment process.
• Reflective: Systems which encourage staff, partners and stakeholders to create
regular space and time for analysing information and reflecting back on the
underlying assumptions or `theories of change’ which underpin the interventions.
• Evolving: Systems which are adapting and changing in order to keep them as light
and simple as possible while providing `real time’ information which informs on-
going improvement of the intervention.
These M&E systems do not have to be complicated. They can be based on simple, creative
processes and simple basic questions which utilise participatory stakeholder feedback,
sound analysis and the use of multiple tools to encourage active stakeholder engagement.
However they do require resources and time to develop.
6. Dynamic: the M&E framework must promote regular ways of receiving feedback
from stakeholders
Good M&E frameworks encourage dynamic feedback. `Learning by doing’ is key to most, if
not all, environments. Each Civil Society initiative ought to promote ways of seeking on-
going feedback from multiple sources about the benefits, problems and impact of the
intervention. At the heart of these processes are three key areas of analysis:
1) What has been done by the Program
• What were the initial objectives, theory of change, assumptions and initial
measures of success?
• What happened throughout the year?
2) What has changed as a result
• Who benefited (women/men/class/caste etc) and who didn’t?
• What were the intended and unintended outcomes as perceived by different
stakeholder groups?
• What were the gender implications?
3) How should the learning from this process inform forthcoming work
• What are the lessons/learning from the past year?
• How should the on-going plans be altered as a result of these lessons?
• How could stakeholders have greater input into (or involvement in) future
activities?
Dynamic feedback processes can take a variety of forms. One form is the introduction of
`Annual Reflection Cycles’. An `annual reflection cycle’ is essentially the implementation
agency creating `space and time’ annually to reflect with stakeholders on `how’ and `if’ the
initiative is making progress towards its objectives or success criteria. Annual reflection
cycles provide an opportunity to reflect back on `theories of change’ or assumptions which
guide the intervention, assess progress against `success criteria’ and involve different
stakeholder groups in the analysis.

13
Use of annual reflection cycles

Each annual reflection cycle will be different depending on the nature, and stage, of the
intervention. Multiple data methods, data sources and tools should be used. Some annual
processes may involve implementing staff and other stakeholders (e.g. government, peer
organisation, bilateral staff) in feedback processes. In other situations, it may be possible to
involve primary stakeholders (or their representatives) in participatory analyses of how (or if)
the interventions are making a difference to people’s lives. Many processes will be multi-
layered but in essence they should be kept simple and as participatory as possible (see
potential key steps below).

Potential steps in an Annual Reflection Cycle

Initial preparation of
reflection process
by program team

Planning of reflection process with

different stakeholder groups

Participatory processes with Participatory processes with Participatory processes with

different stakeholder groups different stakeholder groups different stakeholder groups
• What has been done? • What has been done? • What has been done?
• What has changed as a • What has changed as a • What has changed as
result? (& for whom) result? (& for whom) a result? (& for whom)

Feedback from other data Feedback from other data Feedback from other data
sources sources sources

Discussion with
stakeholders on how
analysis/learning should
influence program

`Sense making process’

Discussion by Program Team of
the learning & implications of
reflection cycle for the CD program

Review of
reflection process

14
Other methodological approaches which invite reflective, dynamic stakeholder feedback
include Appreciative Inquiry, Outcome Mapping, and use of the Most Significant Change
approach. In Appreciative Inquiry the intention is to seek out examples where processes are
improving and changing in ways that people consider positive and ask ‘why’ and ‘what can
we learn from this? 10 ’ Outcome mapping focuses on one particular category of results –
changes in behaviour of people, groups and organisations with whom the programme is
working directly 11 . Most Significant Change involves the collection of collection of significant
change stories emanating from the field level, and the systematic selection of the most
significant of these stories by panels of designated stakeholders or staff 12 .
In addition to processes initiated by AusAID or contractors it is important to establish means
by which communities and other stakeholders can provide feedback when it is important to
them, as part of a process of empowerment 13 . In this way the monitoring and evaluation
process becomes part and parcel of civil society development - rather than a ‘technical add-
on’ - because it strengthens the ability of people to hold others to account. For example,
complaints handling mechanisms 14 are increasingly used in the humanitarian arena, as well
as World Bank programs, and have potential in longer term civil society processes as well.
Citizen Report Cards 15 and Community Based Performance Monitoring have also been used
in a variety of contexts to provide means for communities’ to express their satisfaction with
development projects and social services, and assess their performance. Other innovations
include the work of the International Human Rights NGO Witness 16 who has set up a pilot
using the model of You-Tube, called ‘the Hub’, whereby groups and communities can post
evidence of human rights abuses onto the web. This illustrates the potential of using new
technologies to innovate in this area, and build on experiences using storytelling, theatre and
video as alternative assessment methods.

In the Chiefs Project in Vanuatu, the monitoring system is adapted for each location. In some
locations a set of three questions are posted on a communal wall, allowing participants to
provide their views. In other settings, use is made of the existing informal systems of
discussion. The local facilitators also contribute their reflections and observations. A challenge
for the program is that people tend to avoid negative reflections. However by keeping the
system simple and deliberately seeking different views, the project has been able to surface
different perspectives that have been important for ongoing program development.

Whatever the method used to gather information or feedback about how the program is
proceeding and what it means for people, there are several components which need to be
addressed in the M&E framework for all civil society programs. These include:

10
H. Preskill and T Tzavaras Catsambas (2006) Reframing Evaluation through Appreciative Inquiry.
11
See International Development Research Centre, Canada:
12
See here for MSC Guidelines
13
The AusAID commissioned review of Social Accountability Initiatives undertaken in 2006 offers a
very useful review of a range of potential methods
14
Go to the U4 Anti-Corruption Resource centre for examples and guiding principles for developing
and designing a complaints mechanism
15
See this site for World Banks resources on report cards and community monitoring
16
See Witness, http://hub.witness.org/

15
o Seeking feedback from multiple sources using multiple methods (both qualitative and
quantitative) in both formal and informal ways.
o Involving different stakeholder groups in meaningful ways in different forums to analyse
the benefits/problems and learning from the intervention.
o Deliberately `seeking surprise’ – seeking ways of hearing and understanding the
perspectives of different groups & devising processes that consciously seek new
information (positive and negative change) 17 .
o Devising processes which lead to greater transparency of information about the
initiative with stakeholders and result in greater ownership of the initiative by
stakeholder groups.
o Creating a process which is simple, invites honesty and is a learning experience for
those involved.
A challenge for M&E which focuses on participative and qualitative methods is concern
about the rigour or validity of the information. Both quantitative and qualitative data can be
useful in seeking to understand a situation but the M&E framework must explain how the
information will be checked and verified. Such processes are different for different types of
data as outlined below.
Criteria for Judging the Rigour or Trustworthiness of Information

Conventional Scientific Criteria Qualitative or Participatory Criteria

Internal Validity – the proof of causal Length and Depth of Engagement to build rapport and
relationships trust in order to again quality information

External Validity – the degree to Persistent and Parallel Observation using a number of
which findings can be applied to other observers and stakeholders
contexts or groups
Cross Checking combining different sources, methods
Reliability – the degree to which the and researchers as well as participant checking
findings could be repeated if the
enquiry was done in the same or Expression of Difference & Negative case Analysis
similar situation searching out different views and explanations, particularly
on the basis of gender or class, analyzing change for the
Objectivity – the extent to which worse
multiple observers can agree on a
phenomenon, ensuring that results are Research Diary and peer review making transparent
not due to researcher’s biases means by which information is collected and analysed as
well as its sources

Impact on Stakeholders’ capacities to know and Act -

the process itself should be empowering and developmental
and generate new insights for all.

17
See Irene Guijt (2008) Seeking Surprise: Rethinking Monitoring for Collective Learning in
Rural Resource Management

16
7. Participative and Gender sensitive: In a civil society program we want increased
engagement of people. The monitoring process needs to reflect this, enabling a
wider range of people to engage in assessment and understanding of the progress
of the program itself. However there can be many barriers to such engagement.
An awareness of power and power dynamics is essential in any monitoring process. Without
an awareness of power in the process, all tools and approaches can be manipulative,
shallow and exclusionary. The monitoring process has to work with an understanding of the
many sources of power. 18
When facilitating feedback processes with different stakeholders an understanding of power
is essential. Key questions to ask of the M&E framework include:

• How is the process ensuring that the voices of the most vulnerable and/or least powerful
groups are being heard?
• What `safe’ spaces are being created so that women and marginalised groups feel
comfortable to tell their stories?
• How is the process being facilitated so that difficult, but honest feedback can be given?
• How is the power of the donors, the consultants, the state and the richer (or more
powerful) voices being contained so that space may be given to others to tell their stories
and be heard?
Being sensitive to power relations and thinking creatively about how participatory monitoring
processes can create opportunities for excluded and marginalised groups to have a
voice.This is important in order for those processes to reflect the aims of civil society
programs themselves.
And in particular, Gender
An awareness of gender is also critical in monitoring and feedback processes. Attention to
different experiences of men and women within community development and other programs
is an essential requirement of good quality monitoring. By not actively seeking ways to
involve women in stakeholder feedback processes, women’s voices can be easily
marginalised and/or silenced. Ongoing monitoring and feedback processes should ensure
that women’s perspectives are proactively sought and that monitoring processes analyse the
differing effects of the interventions on men and women. It is important to remember that
gender and gender related issues can be considerably sensitive in community development.
Key issues to consider include:

• How monitoring processes be timed and/or facilitated so that women are comfortable
and able to be actively involved.
• How (or if) women can be involved in monitoring processes in `safe ways’ which enable
their differing voices and opinions to be heard without putting women in vulnerable
situations or subject to recrimination.
• How men can be sensitively and positively involved in discussing the interventions of the
program on gender relations?
• How formal monitoring data captures the differing effects of the program on different
groups of people?

18
See John Gaventa’s ‘Finding the Spaces for Change:A Power Analysis’ and Valerie Miller and Lisa
VeneKalsen’s ‘A New Weave of Power’ and www.ids.ac.uk/ids/particip/

17
In most situations, gender-aware M&E involves proactive initiatives to engage both men and
women. Separate forums, careful planning and sensitive facilitation are often required for
feedback processes to work well. However, it is important to remember that even when
women are involved in feedback processes that women differ by age, class, caste, ethnicity
etc. An awareness of the gender–power nexus is therefore important 19 .

Tsunami response in Sri Lanka: gender watch:

In Batticaloa and Ampara districts in Sri Lanka the Women’s Coalition for Disaster Management
(WCDM) was initiated by Suriya, a local women’s organization. From the middle January 2005
this group has been playing an important role in post-tsunami relief and reconstruction work.
The WCDM also formed an action group called Gender Watch, with local and international non-
government organisations. The initiative enables women to report domestic violence, sexual
harassment and discrimination to the group. The group documents violations in the camps and
distributes the information to international agencies and the government.
A review of the “Gender Watch” after 6 months of operation was requested by the participants
and supported by Oxfam Australia. The most important critical success factors were seen as a)
this was the only network in the area that was driven, run and managed by local women but
supported by INGO’s, UN organizations and government representatives, who had decision
making authority, b) it provided a platform for discussion unavailable elsewhere, c) it was not a
sectoral meeting – most ‘official fora’ were organized on this basis e.g. on health or water - but
looked at all issues that affected women which ranged from shelter, rape and employment
opportunities in a holistic manner.

“.. we sat in our groups to illustrate how we would like our lives to be in five years time..”
Radhamani Mundari, Orissa, India.

In Orissa, India, the `DISHA partners for development’ has played a pivotal role in bringing people
together, motivating and making possible the preparation, execution and review of people’s plans
in 20 villages of Orissa. There have been four essential elements of the process:
• Supporting groups to analyse their own situations – including analysing power relations
existing in every sphere of life – political, social, economic and culture
• Recognising a bias in favour of the marginalised by facilitating a process of poverty
analysis
• Using participatory methods to address key issues – using graphs and visuals such as
maps, drawings, drama and video to support people to analyse and then address the
deep-causes of marginalisation
• Providing space for groups to be involved in planning and monitoring and assessing on-
going work
Regular facilitated processes of assessment help up-date the work. The people are centrally
involved in this process. “we divided into groups. The women formed into two groups, and the
men got into one group. Each of us took different issues to highlighted in Lok Yojana [community
plan]. The task was to see how far we had progressed against the objectives set in our plans, the
difficulties faced while carrying out our plans and what still needs to be done and who will take the
responsibility for taking this forward. Our group discussed the situation of women and whether
there had been any changes in the relationships between men and women during this period. We
drew pictures of our previous situation and where we are now. There has been some gradual
change…” Radhamani Mundari, Orissa, India

See “Shaping our own Destiny: Radhamani’s Story”.

http://www.actionaid.org/assets/pdf%5CRadhamani's%20story%20-%20small.pdf

19
See Linda Mayoux’s website at
http://www.lindaswebs.org.uk/Page1_Development/Gender/Gender.htm

18
8. Reflective: there has to be time and space created as part of the M&E approach for
analysis and ‘making sense’ of the data.
In most M&E frameworks and guidance the emphasis is on defining objectives and
indicators, clarifying data collection processes, and sometimes on analysis of that data. Very
rarely is the important process of making sense of not only this data but other formal and
informal information, given much attention. It is assumed. And it is assumed to be straight-
forward. This is rarely the case.
Good M&E systems create, and protect, the necessary space, time and resources for
reflective analysis or `sense making’. Throughout the year it is important for the M&E
framework to require that the program team create explicit space for team reflection and
analysis 20 . At these `sense making’ fora, people ought to be encouraged to draw on data
from a multiple sources (this includes formal program monitoring data, but also informal
feedback/perceptions and analysis of changes in the external environment) to openly
discuss:

• What is going well and not so well?

• Is the program doing the right thing in the right place at the right time to make
achieve the results it anticipates?
• Is the monitoring providing data that is helpful for analysis?
• How the interventions could be improved?
At one point in the year this `sense making process’ could be linked to an annual
stakeholder feedback process. This is an opportunity for all the stakeholders to collectively
analyse the information and develop a shared understanding of progress against objectives,
learning and the implications for the future direction of the program. This process can link to
the Quality at Implementation reporting. See the next section for further discussion.
The value of making time and space for honest, open, reflective analysis should not be
underestimated. One of the pitfalls of many M&E systems is the lack of time given to making
sense of formal and informal monitoring data. Civil society initiatives are often responding
to complex, changing situations. It is therefore important to make time to understand the
present before engaging in processes which seek to engender change.
All sense making processes should end with deciding how the analysis should affect future
interventions. What should change? Each program team needs to have the ability to change
objectives, revise success criteria and revisit ways of working in line with the thinking and
analysis that comes from these processes. The key test of good M&E systems is the degree
to which the learning and insights are used.

The ACCESS program in Indonesia focused on enabling the CSOs to become learning
organisations. The program collected data using a range of methodologies which included
community and organisational ‘snapshots’, MSC stories, and case studies. The critical step was
having CSO and other partners being part of the discussion and analysis of what meaning this
information had in terms of key issues such as gender, capacity, community empowerment and
so on. Organisations were expected to learn how to learn. In turn they started to move closer to
the people and become more responsive to serving communities.

20
The time given to this analysis will be commensurate with the size and scale of the program, but at
least two full days is advised for each sense making process

19
 The Won Smolbag Project in Vanuatu has developed its own capacity to reflect on its program
and use that reflection to make changes and develop the various program areas. The program
has a formal M&E system which focuses around ten performance questions. Each area of the
program collects data which contributes to answering those performance questions on an annual
basis. The process is supported by a research officer who assists the program staff to consider
how they might collect useful information for their area.
AusAID provides the funding for this research officer and supported the development of the
performance question approach.

9. Evolving: M&E systems for civil society should be expected to be changing and
improving over time.
Just as good civil society initiatives should evolve, so should be good M&E systems. Each
year the program team ought to spend time reflecting on whether the M&E system is
providing pertinent, appropriate data for sound analysis; whether the voices and
perspectives of different stakeholder groups (particularly communities and people) are being
heard in the annual processes; and how to revise systems to keep them as simple and light
as possible while providing `real time’ information which informs on-going improvement of
the intervention.
10. AusAID requirements: maintaining the principles across programming levels
CSO programs receive funding under various programs from different mechanisms and
funding arrangements. AusAID is currently developing guidance for the appropriate level of
focus and reporting for each of these variations. In some cases, where AusAID chooses to
monitor at higher levels of sector and country programs the responsibility for monitoring
resides more directly with AusAID.
Overall however the same principles about good quality monitoring for civil society
engagement will apply for sector programs as for individual activities. The M&E needs to be
dynamic, participative, gender sensitive, reflective and evolving. The process does not
become simpler or more amenable to simplistic aggregated processes for larger or more
complex programs.

Some Dos and Don’ts for the Implementation stage

Do:
• Encourage M&E systems which are seeking regular stakeholder feedback on
plans, strategies & performance.
• Develop M&E systems which are central to developmental work and
organisational learning rather than systems which are technical add-ons driven
by performance targets
• Ensure an awareness of power dynamics in all processes and make space and
time for the most excluded to be heard.
• Create adequate space and time for `sense making’ processes.

Don’t:
• Seek standardised information, pre-set indicators or logical frameworks
• Create complicated M&E systems which take too much time, cost too much
money and don’t provide pertinent, up-to-date information to inform on-going
decision making.

20
Considerations for reporting
11. Monitoring data can lose meaning if it is aggregated and presented in simplistic or
rigid ways.
Civil society M&E tends to develop a wide range of qualitative and qualitative data due to the
differences between contexts and the particular variations that occur when people begin to
take charge of their own development. What indicates effective change in one location will
not necessarily be the same in another. But both are valid and important to understand. It is
important that a focus on reporting ‘outcomes’ is not interpreted as meaning that such
diversity should be ignored or glossed over.
Good M&E should allow for synthesis of key information - that is, reducing large amounts of
information down to key issues and lessons - but should not try to aggregate different types
of data simply to provide an easy-to-read presentation.
Likewise forcing data into formal reports and formal styles can undermine the very enabling
process the civil society program is trying to achieve. If a community group have struggled to
record their achievements and challenges in forms which make sense for them, such as
through video, drawing or other creative methods, then it is important to focus on the
information, not critique the styles and presentation. It may be that the format usually
required by AusAID will have to be changed or varied to meet the possibilities provided by
this type of M&E.
12. Not all information is required by all stakeholders in the same format.
While formal reports are usually required for accountability upwards in AusAID and other
donors, not all stakeholders will find such reports to be the most useful way to receive
information. The M&E framework should allow for the time and resources to present
information in other forms that are appropriate to the stakeholders groups. Partner
governments may prefer to have information presented to them in short oral presentations.
Communities may prefer visual presentations or more creative mechanisms such as drama.
The important issue is to ensure that stakeholders are informed about progress in ways that
enable them to use the information and contribute to decision making about the program.
If AusAID want to assess progress in civil society programs then it is important for them to
engage as part of the sense making processes. Formal reports will not be sufficient for
program staff to be able to ‘know’ if a program is achieving its objectives. This will be a
complex and varying judgement to be made on the basis of a wide range of information.
Some of that information will be made available through formal reports, but as discussed in
the previous section, much will also come from the views of multiple stakeholders, in both
formal and informal ways.
By being part of the sense making process the AusAID program manager can then further
check on both the quality of the M&E processes and the interpretation and analysis of the
data. On the basis of this information and engagement he/she can then have the confidence
to complete the documentation required by AusAID such as the quality at implementation
(QAI) forms.
13. AusAID requirements: Quality at Implementation
The Quality at Implementation (QAI) processes require the AusAID program manager to
make judgements about the activity or program based on the available evidence. Much of
this will come from the formal monitoring systems, (although additional information will also
come from other sources, such as informal discussion). The program manager can take the
opportunity of the ‘sense-making’ process in the formal monitoring to ask for relevant
feedback and input into the QAI process.

21
The judgements for the QAI and the completion of the form remain the responsibility of the
program manager. However given the analysis process is intended to be a shared and
participative process where everyone weighs and considers the available evidence about
progress, it is sensible to also use this opportunity to obtain insight for the QAI.
There will also be considerable other information about the program or activity which is
important for program development and improvement but less relevant for the AusAID
accountability requirements. It is important that consideration of the QAI does not lessen
program attention to this other information.

Some Dos and Don’ts for Reporting

Do:
• Engage with the implementers and the other stakeholders so that you
understand the M&E process in full.
• Be part of the sense making processes so that you can confidently make
judgements about the progress of the program and use this information
for QAI completion
• Require the program to report to all stakeholders in whatever forms are
most useful to them
• Be open to new ways of doing things.

Don’t:
• Ask for simple statements of outcomes which gloss over diversity and
important differences.
• Be driven by predetermined reporting formats or expectations.
• Ignore alternative ways of reporting e.g through photo-stories, video and
other narrative techniques,

22
Evaluation and redesign
14. Evaluation processes need to follow the same principles and approaches as
monitoring.
At some points civil society programs will require evaluation and possibly redesign.
This does not mean changing and checking every part of the program every year, because
civil society programs need time to grow and for their strategies to take effect. But
sometimes it is useful to stand back and ask evaluative questions:

• What has been the value of the work so far?

• Who has it served?
• How could we do better?
The purpose of the evaluation needs to be clear to all involved 21 . While there are different
reasons for evaluation, clarity about the specific purpose enables people to continue to have
some control over their participation and engagement. Where evaluation only serves
external purposes it can be expected that local stakeholders will be unlikely to engage in any
significant ways.
The methodology of the evaluation should build on the monitoring approach used throughout
the program, enabling participation, effective gender analysis and the engagement of a wide
range of stakeholders. It needs to have space for the same sense making processes or
analysis processes outlined in the previous sections. While these may include additional
external views and insights, it is important that those external participants continue to
engage and empower the communities and CSOs who are the focus of the program.
Traditional review teams who enter communities for short periods with no previous
relationships or legitimacy are unlikely to offer much useful contribution to evaluation of civil
society and may do considerable harm to existing processes and relationships.
The evaluation should also consider how adequate the M&E system has been for the
program and how well it has served the civil society intentions of the program.

The midterm review of the PACAP program in the Philippines was useful process
because it focused on understanding the emergent outcomes in the program. It
showed what could be possible for the future. This has led to new emphasis within the
program.

15. AusAID requirements: annual consideration of effectiveness

Every year AusAID is required to consider the effectiveness of the whole of its program, this
process, the Annual Program Performance Review (APPR) requires that country and sectors
review overall effectiveness and progress, including an analysis of the reasons for that
progress.
This may be a good opportunity to also ask evaluative questions about the ongoing civil
society programs. Question that could be considered include:

21
Baser, H. & P. Morgan. 2008. Capacity, Change and Performance - Study Report (Discussion
Paper, 59B). Maastricht: European Centre for Development Policy Management

23
 • Is the original rationale for engaging with civil society still relevant? Do we need to
update this?
• What has been the value of the engagement with civil society, beyond the specific
activity outcomes?
• Could AusAID have achieved this value in other more effective and/ or efficient
ways?
• What are the lessons that have been learned which can be feed back to both
program development and wider policy development?
• What should be changed about the engagement with civil society in this situation?
This process not necessarily part of a formal evaluation. It is an opportunity provided through
the ongoing performance review and quality assurance that AusAID now promotes
throughout the whole aid program.

Some Dos and Don’ts for Evaluation

Do:

• Where possible, seek creative ways of evaluating the program which

reinforce empowerment processes and stakeholder involvement.
• Ensure the evaluative process assesses gender and gender relations
• Encourage the evaluation process to look `out of the box’ for the unexpected
& unintended
• Ensure the evaluation feeds into on-going program learning

Don’t:
• Involve external consultants who have little knowledge & legitimacy
• Expect simple answers and pithy results in complex situations
• Gloss over diversity and the important difference.

24
Annex 1: Monitoring and Evaluating AusAID Community
Development Programs Initial Discussion Paper
Ros David, Linda Kelly, Chris Roche,
July 2008
Preamble
This discussion paper attempts to synthesise Australian and international lessons on
monitoring and evaluation of community development and civil society engagement
programs. The paper forms the first part of a consultation process with key stakeholders
within and external to AusAID for the purpose of developing new guidance for AusAID
officers about good quality practice in monitoring and evaluation (M&E) of community
development and civil society programs.
The paper builds on issues raised at the AusAID Civil Society Network Conference
(Canberra, May 2008) and ideas discussed at a Steering Committee meeting for this project
held in Canberra (17th June 2008). At that meeting it was agreed that this initial discussion
paper would synthesise issues arising from the M&E literature and propose a practice-led
consultation process with AusAID staff and other stakeholders. The consultation process
would, in turn, lead to the development of practical guidelines on the monitoring and
evaluation of community development programs. At this meeting the Steering group
confirmed that gender equity should be addressed through ensuring the M&E of gender
equality within community development initiatives.
The paper is designed to reflect some of the key challenges and learning about civil society
and community development M&E from an international perspective. The paper also reflects
some of the current views about AusAID experience in M&E, particularly as it relates to civil
society and gender programs. The consultation process is expected to add considerably to
the latter area, leading to a more informed understanding of the strengths and challenges of
current AusAID M&E systems as they apply to these practice areas.
The paper is divided into five sections.

• Section 1 introduces the discussion from the perspective of current AusAID

experience.

• Section 2 provides an overview of the literature synthesising key lessons for

international development M&E, with particular reference to community development
and civil society programs.

• Section 3 attempts to identify some pertinent issues related to the AusAID

experience

• Section 4 outlines some useful approaches and tools; and

• Section 5 outlines the intended next steps.

25
The AusAID Experience
AusAID supports a range of community development and civil society programs. This
includes large community development programs focused on funding of local organizations
such as the ACCESS program in Indonesia and the PACAP program in the Philippines, as
well as a range of smaller programs such as the Chiefs program in Vanuatu or the Churches
Partnership Program in Papua New Guinea. AusAID also support Australian NGOs to
undertake work that often includes both community development and civil society
strengthening. In addition AusAID manage a range of programs that include elements of
community development or civil society strengthening as one aspect of a wider program.
Finally AusAID provides support for programs which work directly to improve gender
outcomes, such as the Fiji Women’s Crisis Centre, as well as seeking to mainstream gender
and development across the entire aid program. The general picture is one of a range of
innovative designs, using various implementations methods, which are seeking to operate
effectively in highly varying locations and contexts.
AusAID policy strongly supports the inclusion of such programs within the aid portfolio
(AusAID 2007, 2008), however a quick assessment of a sample of such programs suggest
that there have been some difficulties in assessment of their contribution to the wider
AusAID country and regional strategies.
In part this seems to be about the M&E systems which are applied to the programs. Reports
suggest that conventional, M&E, based on simple indicator based assessment of
predetermined outcomes is often inadequate to capture the wider and more complex results
of such programs (a problem identified for example, in recent assessments of the Chiefs
program in Vanuatu and the Solomon Island Civic Education program). Further that
conventional M&E often fails to convey all the information required to understand a program
(for example, the recent Mid Term Review (MTR) of the Churches Partnership Program in
Papua New Guinea found that most external stakeholders supported the program but
wanted considerably more information about what it did and how it operated).
Further AusAID experience suggests that changes brought about by community
development programs can only be understood with reference to their context (see for
example the Independent Completion Report (ICR) for the Indonesian ACCESS program) or
that changes and outcomes might be interpreted and reported differently by different
stakeholders (for example the MTR of the Vanuatu Women’s centre identified several
different stakeholder groups with varying perspectives on the achievements of the Centre).
Finally, experience suggests that in community development programs different projects
under one program can have varying aims and objectives and quite different outcomes (see
for example the MTR of the Africa APAC program which struggled to draw together the
highly different experiences of six NGOs working across several countries in the broad area
of HIV/AIDS prevention and mitigation). Assessment of overall outcomes therefore is difficult
to understand outside of the specific context and difficult to aggregate in any meaningful
way.
A further problem for many AusAID supported community development programs appears to
be that ‘success’ takes a long time and that the information made available by conventional
M&E therefore tends to focus on activities and outputs (see for example the ICRs of both the
Community Development Scheme (CDS) in Papua New Guinea and the PACAP program in
the Philippines. Both reports identified a lack of information about outcomes as a major
failure of the project M&E systems). AusAID and other stakeholders are left without
information about achievements and about the sustainable benefits of such programs.
Beyond the M&E systems there also appears to be some limitations in articulating the
program rationale for some community development and NGO programs. For example, the
ICR for the CDS scheme in Papua New Guinea concluded

26
 “CDS2 design lacked some clarity of purpose as to what real change or end-of-
program impact it was supposed to bring about, and how necessary or sufficient that
change would be in terms of contributing, meaningfully, to national poverty reduction
objectives. It also lacked meaningful measures of that change or impact.” (CDS ICR,
2008)
This aspect of the problem appears in part to be related to the innovative nature of many of
the AusAID supported community development programs. Design processes have widened
beyond simple, problem based program logic (for example the CPP program in Papua New
Guinea and the support provided for the youth focused organisation, ‘One Smol Bag’ in
Vanuatu) and have begun to utilise new theories of how change will come about. However
the M&E systems do not appear to have kept pace with these design changes. There is less
clarity about why programs are supported and insufficient information about how the way in
which their outcomes contribute to the wider AusAID strategies within a country or region.
Other experience of M&E in international development
AusAID is not alone in struggling to improve its monitoring and evaluation 22 . A review of the
Australian and international M&E literature suggests that other bi-lateral and multi-lateral
donors have had similar experiences. Drawing from that literature there appears to be five
recurrent lessons which have relevance for the AusAID experience.

• There is growing recognition of the non-linear and complex nature of development

and therefore a need to develop M&E system which can capture this more
sophisticated understanding of change processes;

• Single actors can only have a partial knowledge of complex problems and therefore
M&E approaches have to be able to accommodate different perspectives in a
meaningful way;

• Aid agencies need to be aware of, and address, power and gender relations in their
work, including how they assess outcomes;

• Strategies and outcomes needs to be updated in the light of changing circumstances;

and

• Organisational enablers are critical to promoting effective monitoring and evaluation.

Each of these areas is examined in detail below.

Growing recognition of the non-linear and complex nature of development

The literature presents a growing recognition amongst some donors, and academics, of the
non-linear and more complex interpretations of developmental change. There is a linked,
and emerging, understanding of how pre-conceptions of change influence how development
agencies value, assess and interpret information. In addition, there is a growing recognition
of the importance of clarifying the implicit theories, models and drivers of change that guide
agencies approaches to development,
The limitations of a simple linear cause-effect approach to promoting change, based on
certainty, rationality and predictability, is being increasingly challenged in the aid arena. At
the heart of this is the recognition that the development process is made up of a complex

22
A World Bank evaluation of Poverty Reduction Strategy processes indicated that the M&E of its PRSP
processes has been one of the hardest issues to get right. Out of 39 questions the one on M&E received the
most negative response. In response to the question ‘An effective structure to monitor and evaluate results has
been established’, 41 per cent of the respondents ‘disagreed’, or ‘disagreed completely’, whereas a further 21
per cent answered ‘don’t know or unsure’ (World Bank 2004)

27
web of inter-relationships which cannot be ‘managed’ like a project, which are sensitive to
context, and when subject to interventions act in unpredictable ways (Uphoff 1992, Eyben
2007, Krznaric 2007,Guijt 2008). This thinking has been paralleled by thinking in official aid
circles Within DFID the `Drivers of Change’ studies have identified these issues, as did the
AusAID study in Vanuatu (Cox et al 2007). According to DFID this tendency emerges from
the recognition
“..That effective programmes must be grounded in an understanding of the
economic, social and political factors that either drive or block change within a
country [and]….. as a way of applying political economy analysis to the development
of donor strategy” 23 .
This development has been accompanied by attempts to look at the implications of non-
linear change for M&E (Reeler 2007, Eoyang et al 1998).

Examples of Projectable, Emergent and

Doug Reeler, for example, suggests
Transformative Change
that linear forms of ‘projectable’ Projectable Change might include, for example, the
change are the exception rather than running of an immunization program. In these
rule. As a result, the application of circumstances what needs to be done is clear and
orthodox M&E (based on the use of generally agreed, quality standards exist, and the
logical frameworks and time-bound effects of the intervention (i.e reduction in disease), are
indicators) is inappropriate and he known in advance.
proposes other approaches based on
‘emergent’ and ‘transformative’ Emergent Change might include attempting to build
change (Reeler 2007). more effective relationships between civil society and
government. In these circumstances the exact end point
Emergent change represents of this journey cannot be accurately predicted. Rather
the relationships required will need to develop out of the
“..the day-to-day unfolding of life,
ongoing and regular process of trust building and the
[the] adaptive and uneven processes development of mutual understanding, which needs
of unconscious and conscious regular adjustment. In the cases “the path is made by
learning from experience, and the walking it”.
change that results from that”.
In other words the kind of ongoing Transformative Change might include trying to broker
a peace-deal during a conflict. In these circumstances
change we experience every day. He
there is no manual or agreement that can guide
notes that in these circumstances it is precisely how to do this, existing ways of looking at the
important to encourage staff and problem may be part of the reason that the problem
partners to develop as ‘reflective- exists in the first place, and that therefore reframing the
practitioners’ (Schon 1983) and problem, and challenging existing assumptions about its
develop more action-learning causes may be critical. In these cases the vision may
approaches to M&E. be clear but the means of achieving it much less
obvious.
Transformative change describes
more abrupt or radical change often
associated with a crisis, or when latent pressure for change is suddenly released. In these
circumstances Reeler proposes a more questioning approach to M&E. One that explores and
surfaces contradictions and paradoxes, which encourages ‘unlearning’ and ‘double-loop
learning’ (see Pasteur 2006) and which challenges fundamental assumptions of the usual
ways of doing things.

23
See http://www.gsdrc.org/go/topic-guides/drivers-of-change

28
Single actors can only have a partial knowledge of complex problems
A second lesson emerging from the recent development literature is that any single actor
can only have partial knowledge of complex or ‘unbounded’ problems. There is a need for a
broad range of perspectives to understand what is really going on in complex environments.
Indeed, the Paris Declaration is an explicit recognition that the development process needs
to be undertaken in partnership, that different actors have different roles in that partnership,
and that effective relationships between these partners is critical.
The importance of recognising `partial knowledge’ is paralleled by the need to value and
recognise different perspectives. Eyben (2006) writes that whilst
“...it may be impossible to have a total grasp of the complexity of our global society,
each of us has at least some understanding. These varieties of understanding can in
dialogue shape responsible policy, one negotiated by respecting difference, where
those involved appreciate that there are many ways of understanding the world and
its problems”..
Acknowledging partial knowledge, respecting difference and negotiating policy is particularly
important when problems are “unbounded” or “divergent”. For example, where:

• There is no clear agreement about the exact nature of the problem because of its
complexity and multi-causal nature

• There is uncertainty and ambiguity as to how the problem might be addressed.

• The problem has no limits in terms of the time and resources it could absorb.

Many of the problems that confront aid agencies in general and governments, in particular
are of this nature. It is unsurprising therefore that ‘messy partnerships’ or complex
partnership arrangements are needed to deal with such problems and that it is unclear what
success will look like and how this might be achieved (Guijt 2008).
As a review of a number of recent donor studies notes
“[p]ower and Drivers of Change analysis is potentially challenging, because it
questions fundamental assumptions about how development happens. It reinforces
the need for harmonisation of donor approaches to be based on rigorous and honest
debate about different perspectives.” (Dahl-Østergaard et al 2005).
Other authors also stress the need to map out the theory, or theories of change that
underpin a democratic dialogue. This approach …
“..is based on the importance of knowing what implicit or explicit understanding of
social change underpins the process that one is assessing and wanting to learn
from”.(Guijt 2008 citing Pruitt and Thomas, 2007).
The implications of unbounded, complex problems and partial knowledge for M&E are
interesting. In these cases, agreeing objectives and indicators may be problematic and
outlining clear program logic at the outset may not be feasible as it may require prolonged
negotiation with other partners and an unfolding of issues to emerge. Rather than a
predetermined orthodox approach to M&E, Guijt suggests that the monitoring and evaluation
approaches should ‘seek surprise’ (2008).

Aid agencies need to be aware of, and address, power and gender relations
The debates on the necessity of explicitly understanding the theory of change and bringing
together different perspectives, are accompanied by a growing literature on the nature and

29
role of power relations in development. Understanding power relations at all levels is seen to
be important, that is, within communities in terms of gender, class and ethnicity (Guijt &
Shah 1998, Gaventa 2006); between donors and recipient governments (Helleiner 2000,
Hyden 2008), and NGOs and ‘partners’ (Wallace et al 2007); and within development
agencies themselves (Eyben 2006). This has two major implications for monitoring and
evaluation. Firstly that any assessment of community development or civil society
engagement needs to critically review the degree to which power relations and relative
participation are addressed and effected throughout the project cycle. Secondly that
agencies need to be acutely aware of their own power both in terms of how this shapes their
relationships with partners and communities as well as how power is distributed within their
own organisation (VeneKlasen and Miller 2002).
Work on gender relations has perhaps gone furthest in looking at the relationship between
these different levels, notably in how organisations can reproduce the inequalities inherent in
the societies in which they are embedded, unless these issues are explicitly tackled (Goetz
1998).
Much of the effort in many agencies has focused on developing gender sensitive indicators,
and gender disaggregated data and measures with the assumption that it will assist
organizations to: a) take gender equality seriously, b) enable better planning and actions, c)
hold institutions accountable to their commitments on gender equality, d) allow for cross-
national comparisons of gender equality and enable complex data to be condensed into
simple statements about achievements and gaps (BRIDGE, 2007).
However it has also been recognized that some issues are difficult to conceptualize and
measure using orthodox M&E approaches. This includes, for example, issues such as
women’s empowerment or the gender dimensions of poverty. Furthermore, there are
sensitive issues, such as gender-based violence, and sexuality, and sensitive contexts, such
as the gender dimensions of armed conflict which produce particular methodological
challenges (BRIDGE, 2007).
A number of studies also indicate the risks of pre-defining indicators. Nalia Kabeer for
example noted in Bangladesh that some micro finance studies used women’s control over
micro-finance as a key indicator of their empowerment. However this was at odds with
women’s own definition of empowerment and well-being which was about the importance of
joint decision making with their husbands. This indicates how some, purportedly gender
aware M&E approaches can exclude women’s or communities’ lived experiences and
assessments of change (Kabeer, 1998),
The current literature therefore suggests that if development agencies are to overcome their
partial knowledge, work in partnership and negotiate different models and theories of change
this means being aware of their own power and recognising how this can privilege some
voices and perspectives and diminish others. The implications are that monitoring and
evaluation processes need to involve a careful analysis of both what is assessed, but equally
importantly how that assessment is undertaken and whose voices are privileged in the
process.
Strategies and outcomes need to be updated in the light of changing circumstances
A fourth lesson arising from the literature is the importance of adjusting and refining theories
of change, strategies and outcomes in the light of changing circumstances and dynamic
feedback. As David Booth has noted in his review of M&E best practice in PRSP processes
‘rapid feedback on this [i.e. intermediary] level of change is what matters most for
accountability and learning” (Booth 2002).

30
A number of authors have suggested that an over-emphasis on indicators as the key means
of defining and testing progress has diminished organizations’ openness to feedback and
‘surprises’ (Roche 1999, Estrella 2000, Davies 2005, Guijt 2008).
“ the notion of approaching all monitoring through one type of data process (i.e.
indicator-based) …was acknowledged as a crude and inappropriate way to view
information needs..” (Guijt 2008).
The reality is that information about how a project or process is going comes to staff in a
variety of formal and informal forms and through a variety of processes including
observation, discussion and formal reports etc. Over-emphasising indictors can lead to an
organisation ignoring important information, particularly that related to unexpected and
unanticipated effects.
These findings are consistent with approaches that rely less on the blueprint planning model
and more on learning from reality. Indeed some argue that
“..a well-functioning national M&E system that devotes considerable attention to the
twin functions of feedback (lesson learning) and accountability is thus a linchpin of
the new aid paradigm..” (Holvoet, and Renard 2007).
The review of World Bank community development and community driven projects supports
this view, noting that, given the importance of local social and cultural contextual issues in
community development processes, learning by doing is much more useful than application
of designs based on predetermined models of change. This learning by doing has to be
based on constant feedback in order to continually adjust the programs to local
circumstances and learning as they proceed (Mansuri et al 2003).
Organizational enablers are critical in promoting effective M&E and gender
mainstreaming
A fifth lesson from the literature is that both M&E and gender mainstreaming need to be
consistently supported through organisational systems. Numerous studies point to the
importance of a number of organisational enablers that are critical in promoting effective
M&E and gender mainstreaming. Many of them are remarkably similar, they include:

• The importance of leadership from senior managers and boards. This is leadership
which sends strong messages about the importance of learning, accountability and
the critical importance of addressing gender equity as a key means of addressing
MDGs (UNDP 2007). But also leadership which models learning, openness and
honesty, invites feedback, and which engages in dialogue with others in ways that
recognise power differentials and partial knowledge (Eyben 2006, David and Mancini
2004).

• The importance of learning being valued within an organisation so that space, time
and resources are dedicated to discussing feedback, sense-making, reflecting &
learning from experiences, preferably with other stakeholders. As well as shifts in
organizational practices to take risks and manage for outcomes rather than outputs;
and improved capacity to assess results and performance (UNDP 2007) This in turn
means getting the balance right between ‘proving’ success and ‘improving’ practice
through an honest admission of errors and mistakes (Estrella 2000, Guijt 2008).

• The importance of the alignment of incentives in ways that reinforce rather than
contradict each other. This means reviewing both explicit and implicit organisational
incentives:

31
 o Explicit incentives include the linkage between annual appraisals, results and
competency assessments, linked to objectives and indicators,

o Implicit incentives include deeper cultural issues (i.e. whether mobilization of

resources is seen to be more important than results). As AusAID own thematic
performance report on gender notes “[C]ultural issues are also central to
consideration of ownership of gender equality. Gender relations are at the very
heart of any culture, and culture shapes personal and institutional behaviour
(AusAID 2008).

Understanding the dominant culture is therefore important. External pressures, the relative
power of different stakeholders and interests contribute to setting the ‘rules of the game’ for
internal processes. Organisations can then also reproduce the inequities they embody. For
example UNDP’s recent review of its M&E systems noted that corporatist approaches to
M&E can have limited or perverse effects on development effectiveness with more junior
levels manipulating their programmes to fit corporate straight jackets and in doing so divert
attention from local needs and make reporting more about process than substance. The
review also found that there was mixed evidence about whether the corporate M&E
approaches significantly affect the shape of country-level programmes and partnerships, but
there was significant evidence that they could impose unnecessary transaction costs (UNDP
2007).
Additional considerations from the AusAID perspective
The review of current AusAID experience suggests that many traditional M&E approaches
do not serve the information and learning needs of the agency, particularly within the
complex and messy interventions such as community development and civil society
programs. AusAID, alongside many other donors, are missing the rich picture of what is
happening in community development programs and are struggling to assess gender
relations within them. This is often compounded by a lack of clarity of the underlying theory,
or theories, of change which provide an overall rationale for individual projects and
programs.
It seems that within AusAID there a number of different suggestions 24 as to why this problem
may exist, which include:

• being unclear what success (or failure) in community development or gender work
looks like, and over what time-scale,

• being unable to define clear and appropriate levels of objectives, and a tendency to
focus reporting and indicators on activities and processes not outcomes and results,

• being unclear why AusAID wishes to engage with civil society, or undertake
community development, and how this fits into broader programs and strategies,

• being unclear how to distil and communicate diverse and complex information,

• being unclear how to engage to address weaknesses in partners’ M&E systems

• the assessment of changes in gender relations tending to get reduced to demands

for gender disaggregated data,

24
Some of these issues were raised at the Civil Society Network Conference in May 2008. Some have appeared
in the drafts of the terms of reference for this work. Some were raised during the meeting with Steering
Committee in June 2008 and in meetings with individuals in Canberra before and after that meeting.

32
 • a view that organisational systems and planning processes are not flexible enough to
capture the fluidity and complexity of community development processes and
changes in gender relations – and indeed may hinder necessary learning and
adaptation,

• overcomplicating things instead of keeping it very simple,

• Not having the space, time or resources for learning and quality management.

As the literature suggests there are a number of understandable reasons why AusAid and
other donors struggle with these issues. Some of these are methodological and relate to the
challenges of addressing attribution, aggregation, selecting appropriate indicators etc.
Others are related to organisational systems, culture and resourcing. But many also relate
to the complex nature of the community development and the realities of working in
partnership with others.
Arguably many of the issues also relate to different definitions of accountability. For some
development is centrally concerned with accountability i.e. development is about
transforming power relations so people in poverty can hold the powerful to account and
achieve social justice. For others accountability is part of a culture of upward reporting, risk
management and ‘mad-audit disease’ which not only stifles creativity and innovation, but
ultimately diminishes trust. This can leave staff in the not-for-profit and public sectors
becoming less accountable to the citizens they serve (Roche 2006). For those that see
accountability as central to the development process the way forward requires a holistic
version of accountability which would include: transparency; participation; learning and
evaluation; and grievance and redress 25 .
But the literature also suggests that part of the solution is to recognise that we need an
intelligent mix of M&E processes and systems which are compatible with the reality of the
challenges aid agencies face. Practical, and possibly innovative, ways to capture experience
and learning from these programs are needed. Indeed, processes which also enable an
organisation such as AusAID to make use of that information for a range of purposes which
include learning, improving practice, accountability, and policy development.

25
See One World Trust http://www.oneworldtrust.org/?display=index_2006 or Bonbright D. (2007) The Changing
Face of Accountability: A talk at the International Seminar on Civil Society and Accountability, Montevideo, 16
April 2007

33
Possible useful approaches and tools
So what might this intelligent mix include? What other areas of M&E might be useful to
explore and learn from? Again the international literature suggests a range of possibilities
which may inform further development of AusAID approaches and systems.

Social Accountability Initiatives

Recent years have seen a The Uganda Debt Network
growing interest in social
accountability initiatives as a The Uganda Debt Network was established 1996, and had been
means for people living in instrumental in campaigning for debt relief as well as developing
poverty and the organizations a civil society response to holding the government to account for
that support them to hold spending that debt relief in an accountable manner. This was
governments and other actors to assisted by ensuring that debt relief was allocated to a ring‐
account 26 . The importance of fenced Poverty Action Fund established in 1997/8. UDN
these mechanisms is that they established Poverty Monitoring Committees in the first instance
form part of development in 12 out of 45 districts in Uganda, and then in 17. This then
processes which strengthens and developed into Community based Monitoring and Evaluation
empowers the voice of poor systems which involved communities carrying out monitoring
communities vis-a-vis other using their own developed tools which they used to collect
actors. In so doing they are also qualitative data, analysis of information.
providing important information These qualitative assessments were complemented by scoring
and feedback on what is working against criteria developed in a performance scorecard which
and what is not. provided feedback from the local to national level and which
complemented traditional reporting processes which can often
There is clearly potential for lead to overly aggregated data which loses a level of detail and
mechanism such as these to play richness, and which can mean that important extremes of
a role in providing feedback to performance are not captured in a timely way
community development Summarized from Roche (2005)
programs, (Macnamara, IDS
bulletin Jan 2008, Goetz & Jenkins). These might be complemented by more macro studies
and indicators based on a number of attempts to develop means to measure good
governance, democracy, corruption and freedom and the strength of civil society (Civicus,
2008).
Advocacy & Policy work
Clearly there is also much to learn from the monitoring and evaluation of advocacy work.
Advocacy and policy work is generally recognised as non-linear, complex and has particular
challenges regarding attribution. It therefore offers interesting lessons to community
development work. Advocacy initiatives often attempt to monitor key dimensions of change
which are also germane to community development and civil society work, notably: changes
in norms, ideas & beliefs; changes in policies and practices; changes in the capacity of
people [living in poverty] and organisations to advocate for change; and the degree to which
democratic or political space is increasing or decreasing (Miller 1997, Chapman and
Wameyo 2001, Brown and Fox 1999, Roche 1999).

26
For example the AusAID commissioned review by Macnamara (2006) provides a very useful overview of
examples of such initiatives particularly as they relate to health and education services. They include processes
of budget formulation, analysis and monitoring – including gender budget analysis, community based monitoring,
citizen report cards, and grievance and redress mechanisms.

34
 One particular example is ODI’s RAPID
RAPID Case Study of SPHERE
research, 27 which has developed a
useful methodology to assess
One of the most significant policy shifts in the
international humanitarian sector in the last decade has
evaluations of research and policy
been the move to strengthen the accountability of interventions. This approach helps to
humanitarian agencies. The decision to launch the answer questions like: What impact do
Sphere project in 1996 was one of the key policy donor policy documents have on
initiatives associated with this shift. practice? How can policy guidance be
made more effective? How to improve
One of RAPID’s case studies explored the process that the policy environment in order to
led up to this policy initiative, and how buy‐in and promote effective practice?
ownership were achieved. By examining the interaction
between the context, the quality of the action‐research The ODI framework looks at the
that was undertaken to produce the SPHERE charter and interaction between the context; the
standards and the links between researchers, policy and quality and methodology of the policy
decision‐makers the study illuminates a complex and work and advocacy; and the institutional
non‐linear process by using a theory based methodology. linkages, networks and alliances offers
Summarized from Buchanan-Smith (2003) useful insights for the assessment of civil
society influencing and for the degree to
which donor/government policy provides
an enabling environment for civil society.
Gender
There are a number and range of innovations piloted by women’s organizations and others
which offer different ways of assessing change in gender relations. These are often related
to women’s leadership, men and masculinity and campaigns on violence against women.
Brambilla (2001) looks at some of the practical experiences of designing gender sensitive
M&E systems many of which are from NGOs community development initiatives. Some of
these include participatory M&E processes as well as adaptations of ethnographic research
(see text box below).

Gender, Key Informant Monitoring and the Nepal Safer Motherhood Project (NSMP)

The NSMP works to improve maternal health. The project has used Key Informant Monitoring (KIM).
KIM as an adaption of the participant‐observation processes of ethnographic research. Data is
collected by community‐based researchers and used for monitoring and planning. KIM takes as its
starting point the idea that the context is important in shaping health‐seeking behavior and maternal
outcomes. The approach recognizes the importance of building trust and rapport between the
researcher and the researched, as well as acknowledging ethnicity, gender, kinship and age and
associated power relations. As a result KIM has sought to:
• Train women to interview others of similar age and social background.
• Make use of conversational prompts to collect data on barriers to services, quality of care
and women’s decision making.
• establish debriefing workshops for researchers with NSMP female local facilitators to cross‐
check results
• Develop interaction with women and their families, to also become catalysts for dialogue and
change, for example, by convincing family members to take women with obstetric
complications to hospital.

One of the methodological challenges have included researchers becoming over‐committed to

program objectives and overzealous in the pursuit of evidence of better access and service quality.
Summarized from Price, N., and Pokharel, D., 2005

27
See http://www.odi.org.uk/RAPID/

35
Caren-Levy’s development of the ‘Web of Institutionalisation’ approach to gender (adopted
by UNEP, ILO, IUCN) provides a tool for looking at different dimensions within organisations
such as the: policy sphere, the citizen sphere, the organisational sphere, and delivery
sphere. This may be applicable to the kinds of gender assessments that organisations
promoting gender equality may need to conduct as well as a framework for assessing the
progress of larger partner organisations in mainstreaming gender (Caren-Levy 1996).
Finally a number of NGOs such as Oxfam have developed a range of tools and processes
such as: program gender audits; criteria for the assessment of the degree to which
regional/country strategies have mainstreamed gender; methods for evaluating changes in
belief systems in relation to gender related campaigns 28 ; and published material on different
frameworks to guide gender work/evaluations (March et al 1996) 29 .

28
Such as `We Can’ in South Asia which is a campaign designed to reduce violence against women.
http://www.wecanendvaw.org/
29
As NGOs may have something to offer in this area – but may well not make this material available easily –
AusAID might consider arranging to meet with NGO staff working on these issues to share tools, methods and
experiences.

36
The Humanitarian Experience
There are also pertinent approaches which have been developed in the humanitarian
sphere. Agencies working on humanitarian issues recognise the particular circumstances of
M&E of humanitarian work, notably how security, trauma and conflict pose particular
challenges; how rapid and unpredictable change need to be factored in to M&E; and how to
balance tracking short term effects (increasingly through real time evaluations 30 ) and long
term consequences. A number of important initiatives have been developed associated with
particular dilemmas that have been faced for example in Biafra, Somalia, the Great Lakes,
Bosnia etc. This has resulted in efforts to define agreed and collective standards and
principles (the Red Cross Code 31 , SPHERE 32 ); debates on moral dilemmas and how to deal
with them (e.g. Hugo Slim 1997); efforts to enhance collective learning and evaluation
(ALNAP 33 ) and developing initiatives on joint accountability (HAP-I 34 ).
The ALNAP-TEC evaluation of the Indian Ocean Tsunami response questions the degree to
which these quality initiatives are having a sufficient impact. In particular the review suggests
that the biggest potential driver for quality should be feedback to the donor public on the
quality of an agency’s operations. Feedback that should be driven more by information from
the people affected by the disaster than by the agencies’ communication departments.

Gender Watch
Notwithstanding the challenges that
the Tsunami produced, it is
Following the Tsunami in Sri Lanka an initiative called Gender
arguably the case that the
Watch was established by a local Women’s organization
humanitarian agencies has much to supported by International NGO and involving a wider group of
offer in terms of M&E. This sector government and multilaterals. The initiative enabled women to
has developed more innovative and report domestic violence, sexual harassment and discrimination
creative ways of attempting to to the group. The group documented violations in the camps
develop standards, improve and distributed the information to international agencies and
accountability to affected the government. Remedial action taken includes: having a
communities and develop `real time’ government officer suspended for violations; providing
learning that offer some ways protection to five orphaned children; ensuring women have
forward on how to deal with some of access to oral contraceptives; facilitating access to the police in
the issues raised earlier in this the case of domestic violence; temporary shelters being given
paper. to single women who were originally excluded because they did
not possess the right papers, and registering women for the
provision of ration cards so that they can have access to relief
goods.
Summarized from Roche, 2005

30
http://www.odihpn.org/report.asp?id=2772
31
http://www.ifrc.org/publicat/conduct/
32
See www.sphereproject.org
33
See www.alnap.org
34
www.hapinternational.org

37
Organisational Change and Development
Many community development and community driven processes have capacity building as a
central element. A key M&E question therefore relates to the degree to which organisational
capacity has, or has not, been enhanced, whether this includes an organisational capacity to
address gender equality concerns; and the degree to which networks and alliances – rather
than individual agencies – have developed.
A recent European Centre for Development Policy Management report (ECPDM 2005)
suggest that ‘soft systems methods’ are particularly useful in dealing with ‘messy complex
situations characterised by unclear goals, contested strategies and uncertain outcomes’.
Indeed the framework 35 developed by ECPDM and studies undertaken in Papua New
Guinea using this framework 36 seem to offer some useful pointers in this area.

ECPDM review of capacity, change and performance issues in PNG’s Health Sector

This study was one of 20 case studies focused on the process of change from the
perspective of those undergoing that change, and examined the factors that
encourage it, how it differs from one context to another, and why efforts to develop
capacity have been more successful in some contexts than in others.

The use of this method in PNG helps illuminate how the health system in PNG is a
complex of competing and complementary policies, institutional arrangements,
relationships, incentives, and political interests, some of which support efforts to
strengthen sector capacity and improve performance, and others which can
undermine it. The study also helps indicate the importance of Papua New Guinean
culture, traditions and diversity as factors influencing organizational behavior,
stakeholder collaboration, and even the perceived legitimacy of the state.
Summarized from Bolger et al (2005)

Next steps
The intention of this discussion paper has been to provide a basis for further consultation
with key stakeholders about the AusAID M&E systems, as they relate to community
development and civil society programs supported by the agency. The final outcome is
intended to be a set of practical guidelines which include
a. Guidelines or ideas about how programs could go about constructing theories or
models of change that match the complex and varying contexts of many community
development and civil society programs and how these might be communicated
effectively to wider audiences.
b. Guidelines or ideas about how to develop an intelligent mix of M&E approaches that
are coherent and consistent with these theories of change.
c. Guidelines and ideas for addressing gender and related power relations, within
community development monitoring and evaluating process.

35
Watson, D. Monitoring and evaluation of capacity and capacity development. (Discussion Paper 58B).
Maastricht: ECDPM. http://www.ecdpm.org/dp58B
36
See Bolger et al (2005) and Hauck et al (2005)

38
 d. Suggestions for what kind of organisational culture, values & leadership are required
for effective M&E processes within this area of community development and civil
society.

It is recognised that the consultation process could cover a wide range of programs,
(including the many AusAID supported programs which include elements of community
development and civil society development alongside other multiple objectives and
intentions and the wide range of programs implemented by Australian NGOs). In order to
maintain a feasible focus for this process, the proposal is to take a ‘bounded learning
approach’. That is to focus on a limited number of programs, specifically those programs
funded directly by AusAID for community based organisations in Solomon Islands, Papua
New Guinea, Indonesia and Philippines, while also drawing on experience from a range of
other programs as appropriate. These are likely to include the extensive range of community
based programs in Vanuatu, other civil society programs in Papua New Guinea, Solomon
Islands and East Timor and possibly a small number of examples from some Australian
NGO work. Where possible attention will also be given to examples of gender related and
gender focused programs as well as the inclusion of gender focused M&E in community
development programs.
The resultant findings and guidance may well have wider application to other AusAID
programs and their M&E systems, particularly in the assessment of gender outcomes. As
appropriate this information will also be captured and disseminated. However the intention is
to retain a bounded focus in order to see what could ‘work’ in practice within these more
limited set of experiences.
The consultation process is intended to reflect the practice reality of the issues discussed in
this paper. The proposal is to work from ‘practice up’ involving a limited number of Posts to
explore existing M&E practice, the challenges they have with providing information and
analysis for AusAID systems, and how the practices might be better improved and
supported.
The proposed consultation approach will be developed as the next step for this process.

39
References:
AusAID (2008) Gender equality: Annual Thematic Performance Report 2006–07,
AusAIDAusAID (2007), “Approaches to Building Demand for Better Governance”,
December, AusAID.
Baser, H. & P. Morgan. 2008. Capacity, Change and Performance - Study Report
(Discussion Paper, 59B). Maastricht: European Centre for Development Policy
Management,
http://www.ecdpm.org/Web_ECDPM/Web/Content/Download.nsf/0/AE807798DF344457C12
57442004750D6/$FILE/05-59B-e-Study%20_Report.pdf
Beck, T. and Stelcner, M. (1997) Guide to Gender Sensitive Indicators , Gatineau: Canadian
International Development Agency (CIDA) (English and French)
http://www.acdi-cida.gc.ca/INET/IMAGES.NSF/vLUImages/Policy/$file/WID-GUID-E.pdf
(English)
Bolger J, Manide-Filer A, Hauck V, (2005) “Papua New Guinea’s health sector: A review of
capacity, change and performance issues”, Discussion Paper No 57F, ECDPM
Bonbright D. (2007) “The Changing Face of Accountability: A talk at the International
Seminar on Civil Society and Accountability”, Montevideo, 16 April 2007
Booth D, and Lucas H, (2002) ‘Good Practice in the Development of PRSP Indicators and
Monitoring Systems’ Working Paper 172, Overseas Development Institute, London
DFID http://www.gsdrc.org/go/topic-guides/drivers-of-change
Brambilla P, (2001) “Gender and Monitoring: A Review of Practical Experiences Paper
prepared for the Swiss Agency for Development and Co-operation” (SDC), BRIDGE Report
63
BRIDGE (2007) ‘Gender and Indicators', Cutting Edge Pack , Brighton : BRIDGE/IDS
http://www.bridge.ids.ac.uk/reports_gend_CEP.html#Indicators
Brown, L David and Jonathan Fox (1999) 'Transnational civil society coalitions and the
World Bank: Lessons from project and policy influence campaigns’, paper presented at the
‘Conference on NGOs in a Global Future’, Birmingham 10-13th January
Buchanan-Smith M (2003) ‘How the Sphere Project Came into Being: A Case Study of
Policy-making in the Humanitarian Aid Sector and the Relative Influence of Research’, ODI
Working Paper 215,
Caren-Levy (1996) `The process of institutionalising gender in policy and planning The web’
of institutionalisation’ Working Paper No. 74. Development Planning Unit. University College
London. 9 Endsleigh Gardens
Chapman Jennifer and Amboka Wameyo (2001) ‘Monitoring and Evaluating Advocacy A
Scoping Study’, unpublished ActionAid Working Paper series, January 2001, London,
ActionAid
Civicus (2008) Civil Society Index.
http://www.civilsocietyindex.org/
Cox M, Alatoa H, Kenni L, Naupa A, Rawlings G, Soni N, Vatu C (2007) The Unfinished
State Drivers Of Change In Vanuatu, AusAID
http://www.ausaid.gov.au/publications/pdf/vanuatu_change.pdf
Corner, L (2005) Gender-sensitive and Pro-poor Indicators of Good Governance,
http://www.undp.org/oslocentre/docs05/cross/2/Gender-sensitive%20and%20pro-
poor%20indicators%20for%20Democratic%20Governance.pdf

40
Cornwall A and Pratt, G (2004), Ideals in practice: enquiring into participation in Sida, IDS
Cox, M, Alatoa H, Kenni, L, Naupa, A, Rawlings, G, Soni, N, Vatu, C. (2007) The Unfinished
State: Drivers of Change in Vanuatu,
http://www.ausaid.gov.au/publications/pdf/vanuatu_change.pdf
Dahl-Østergaard T, Unsworth S, Robinson M, and Jensen I (2005) Lessons learned on the
use of Power and Drivers of Change Analyses in development cooperation, Review
commissioned by the OECD DAC Network on Governance (GOVNET),
http://www.gsdrc.org/docs/open/DOC82.pdf
David, Rosalind and Antonella Mancini (2004) ‘Going against the flow: Making organizational
systems part of the solution rather than part of the problem, Lessons for Change in Policy &
Organisations, No.8 Brighton, Institute of Development Studies
Davies, R (2001) `Evaluating the effectiveness of DfID’s influence with multilateral: a review
of NGO approaches to the evaluation of advocacy work’, unpublished paper for the
Department for International Development, August 2001, London, DfiD
Davies R and Dart J (2005) The ‘Most Significant Change’ (MSC) Technique: A Guide to Its
Use,
http://www.mande.co.uk/docs/MSCGuide.pdf
Eoyang G, and Berkas T, (1998) ‘Evaluation in a Complex Adaptive System’, unpublished
paper, www.winternet.com/~eoyang/EvalinCAS.pdf
Estrella M, Ed (2000) Learning From Change: Issues and Experiences in Participatory
Monitoring and Evaluation, ITDG
Eyben R (2005) ‘Donors’ learning difficulties: Results, relationships and responsibilities’, IDS
Bulletin Vol 36 No 3 September, IDS (UK)
Eyben R, (2006). Relationships for aid. London: Earthscan Publications Ltd
Eyben R, (2007) ‘Labelling for Aid’ in Moncrieffe J and Eyben R, (2007) The Power of
Labelling, London: Earthscan Publications Ltd
Eyben R, Kidder T, Rowlands J, and Bronstein A (2008) 'Thinking about change for
development practice: a case study from Oxfam GB', Development in Practice 18(2): 201 -
212
Klasen, S. (2006) ‘ Measuring Gender Inequality and its Impact on Human Development:
The debate about the GDI and GEM' , HD INSIGHTS , Issue 2, November 2006, New York :
UNDP
http://hdr.undp.org/network/messageview.cfm?catid=12&threadid=135
Gaventa J, (2006) Finding the Spaces for Change: A Power Analysis, IDS Bulletin Volume
37 Number 6 November 2006 © Institute of Development Studies
http://www.forumsyd.org/conferences/w_metod/w_demokrati/PowerAnalysis_John_Gaventa.
pdf
Goetz A-M Ed. (1998) Getting Institutions Right for Women in Development, Uk: Zed Books
Goetz A-M, Jenkins R (2005) ‘Reinventing accountability: making democracy work for
human development’, Palgrave Macmillan, Basingstoke
Guijt I, November (2007), Assessing and Learning for Social Change: A Discussion Paper,
Institute of Development Studies (UK)
Guijt I, November (2007), Assessing and Learning for Social Change: A Discussion Paper,
Institute of Development Studies (UK)

41
Guijt I (2008), Assessing and Learning for Social Change: A Review, Institute of
Development Studies (UK), IDS Development Bibliography 21
http://www.ids.ac.uk/userfiles/File/participation_team/Db21.pdf
Guijt I (2008) Seeking surprise : rethinking monitoring for collective learning in rural resource
management, PhD Thesis, Wageningen, http://www.future-
agricultures.org/farmerfirst/files/T1d_Guijt.pdf
Guijt I, and Shah K, Eds (1998) The Myth of Community: Gender Issues in Participatory
Development, UK: ITDG
Guijt I and Gaventa J (1998) Participatory Monitoring and Impact Assessment of Sustainable
Agriculture Initiatives an Introduction to the Key Elements, SARL Program Discussion Paper
No.1 IIED London
Haley, N (2008) Strengthening Civil Society to Build Demand for Better Governance in the
Pacific: Literature Review and Analysis of Good Practice and Lessons Learned, SSGM,
ACFID and AusAID, January 2008
Hauck V, Mandie Filer A, and Bolger J, (2005). Ringing the church bell - The role of
churches in governance and public performance in Papua New Guinea (Discussion Paper,
57E). Maastricht: European Centre for Development Policy Management.
http://www.ecdpm.org/Web_ECDPM/Web/Content/Download.nsf/0/67B0248E5EE1DB34C1
25709F002E806A/$FILE/Hauck.MandieFiler.Bolger_church%20study%20PNG_2005_DP%
2057E.pdf
Helleiner G, (2000) “Towards Balance in Aid Relationships: Donor Performance Monitoring
in Low-Income Countries.” Cooperation South Journal No. 2: 21-35.
Holvoet N, and Renard B, (2007) Monitoring and Evaluation Reform under Changing Aid
Modalities Seeking the Middle Ground in Aid-Dependent Low-Income Countries, Research
Paper No. 2007/52, UNU-WIDER
Hyden G., (2008) After the Paris Declaration: Taking on the Issue of Power, Development
Policy Review vol 26 no. 3, ODI
IDS (2008) State Reform and Social Accountability: Brazil, India and Mexico, Volume
38, Number 6, January 2008
Kabeer N, (1998) Money Can't Buy Me Love? Re-evaluating Gender, Credit and
Empowerment in Rural Bangladesh, IDS Discussion Paper 363
Kelly L and Chapman R, (2007) ‘Why understanding organizational values and relationships
is important for assessing aid effectiveness – an NGO perspective’, Paper presented at
Australasian Evaluation Society Conference, Doing Evaluation Better, 3-7 September 2007,
Melbourne.
Krznaric R (2006) How Change Happens: Interdisciplinary Perspectives for
Human Development, Oxfam
http://www.oxfam.org.uk/download/?download=http://www.oxfam.org.uk/what_we_do/issues/
misc/downloads/research_change.pdf
Kurtz and Snowden (2003) The new dynamics of strategy: Sense-making in a complex and
complicated world, IBM Systems Journal Vol 42, No.3, 2003,
http://www.research.ibm.com/journal/sj/423/kurtz.pdf
Mansuri G, & Rao, V (2003), Evaluating Community-Based and Community-Driven
Development: A Critical Review of the Evidence, Development Research Group,
Washington DC: World Bank, http://www.cbnrm.net/pdf/mansuri_g_001_cddfinal.pdf, 39

42
March C, Smyth I, and Mukhopadhyay M, (1996) A guide to Gender Analysis Frameworks,
Oxfam
McNamara S (2006) ‘Review of Social Accountability Initiatives’ Unpublished background
paper for the development of the Demand for Better Governance Program, AusAID
Miller, Valerie (1997) Advocacy sourcebook, Boston, MA Institute of Development Research
Morgan P, Land T, and Baser H, (2005) Study on Capacity, Change and Performance,
Interim Report, Discussion Paper 59A, ECPDM
Preskill, H, and Tzavaras Catsambas, T. (2006) Reframing Evaluation through Appreciative
Inquiry
O’Neill, T., Foresti, M. and Hudson, A. (2007) Evaluation of Citizens’ Voice and
Accountability: Review of the Literature and Donor Approaches. London: DFID.
Pasteur, Katherine (2006) `Learning for Development’ in Rosalind Eyben (ed) Relationships
for aid. London: Earthscan Publications Ltd
Price, N., and Pokharel, D., 2005, ‘Using Key Informant Monitoring in Safe Motherhood
Programming in Nepal’, Development in Practice, Volume 15, Number 2
http://www.gsdrc.org/go/display&type=Document&id=2627
Pruitt, B. and Thomas, P. (2007) Democratic Dialogue – A Handbook for Practitioners.
Washington, DC/Stockholm, Sweden/New York NY: General Secretariat of the Organisation
of American States, International Institute for Democracy and Electoral Assistance, and
United Nations Development Programme,
www.democraticdialoguenetwork.org/page.pl?s=2;p=tools_handbook_dd
Reeler D (2007) A Theory of Social Change - and Implications for Practice, Planning,
Monitoring and Evaluation, South Africa: CDRA,
http://www.cdra.org.za/articles/A%20Theory%20of%20Social%20Change%20by%20Doug%
20Reeler.doc
Roche C, (1999) Impact Assessment for Development Agencies, Oxfam
Roche C, Kasynathan N, and Gowthaman P. (2005), “Bottom-up Accountability and the
Tsunami” Paper Prepared for the International Conference on Engaging Communities,
Brisbane, 14-17 August 2005,
http://www.engagingcommunities2005.org/abstracts/S109-roche-c.html
Roche C, (2006) “Promoting Voice, Choice and Responsiveness: Governance and the Role
of Non-Governmental Actors”, Unpublished Paper for the AusAID /NGO seminar on the Role
of Civil Society in Governance, 2nd May 2006,
Schon D (1983) the Reflective Practitioner: How Professionals Think in Action, USA: Basic
Books Inc
Tambo F, Wells A, Sharma B and Mendizabal E (2007) Multi-donor support to civil society
and engaging with ‘non-traditional’ civil society: A light-touch review of DFID’s portfolio,
London: ODI
Slim H, (1997) Doing the Right Thing, Disasters, vol. 21/3, September 1997
U4 Anti-Corruption Resource centre (2007) Anti-Corruption Complaints Mechanisms,
http://www.u4.no/pdf/?file=/helpdesk/helpdesk/queries/query132.pdf
Uphoff N, (1992) Learning from Gal Oya: Possibilities for Participatory Development and
Post-Newtonian Social Science. Ithaca and London: Cornell University Press

43
United Nations Development Fund, (2006), Measuring Democratic Governance: a
framework for selecting pro-poor and gender sensitive indicators
http://www.undp.org/oslocentre/flagship/governance_indicators_project.html
VeneKlasen, Lisa and Valerie Miller, (2002), A New Weave of Power, People and Politics:
The Action Guide for Advocacy and Citizen Participation, Oklahoma, World Neighbours,
http://www.justassociates.org/ActionGuide.htm
Wallace T, Bornstein L, and Chapman J, (2007) The Aid Chain: Coercion and Commitment
in Development NGOs, Practical Action
Winter J, Hall J and Ivatts, S. (2007) ‘More than the sum of the parts: assessing country level
effectiveness in the Australian international development program’, Paper presented at
Australasian Evaluation Society Conference, Doing Evaluation Better, 3-7 September 2007,
Melbourne.
World Bank 2004 The Poverty Reduction Strategy initiative: an Independent Evaluation,
OED World Bank,
http://lnweb18.worldbank.org/oed/oeddoclib.nsf/24cc3bb1f94ae11c85256808006a0046/6b5
669f816a60aaf85256ec1006346ac/$FILE/PRSP_Evaluation.pdf
World Bank (2004) ‘Citizen Report Card Surveys - A Note on the Concept and Methodology’,
Social Development Notes No.91 Feb 2004,
http://siteresources.worldbank.org/INTPCENG/1143380-
1116506267488/20511066/reportcardnote.pdf
World Bank (2004) Operational Manual For Community Based Performance Monitoring,
http://www4.worldbank.org/afr/ssatp/Resources/HTML/Gender-
RG/Source%20%20documents%5CTool%20Kits%20&%20Guides%5CMonitoring%20and%
20Evaluation/TLM&E4%20%20Manual%20for%20Community%20Based%20Performance%
20Monitoring.pdf
Zadek S, (2004) Civil Partnerships, Governance and the UN, Background Paper For The
Secretary-General’s Panel of Eminent Persons on Civil Society and UN relationships
http://www.ecn.nl/fileadmin/ecn/units/bs/JEPP/civil_partnerships.pdf

44
 Annex 2: Framework for the Analysis of the Quality of Gender Integration in Programs

Level A: Blind Level B: Basic Level C: Moderate Level D: Strong

• No gender • Limited gender disaggregated • Has used a gender-sensitive • Has used an effective gender sensitive planning
disaggregated data. planning process. process
data • Gender analysis is restricted to • Addresses women’s • Includes gender disaggregated data.
• No evidence of women’s roles in isolation from empowerment but stops short • Includes a gender analysis which makes clear the links
consultation with men and/or focuses on only of analysing the balance of – in context specific terms - between gender inequality,
relevant one or two dimensions of power between men and poverty and human rights.
individuals and gender inequality. women and men’s role in • Analyses and responds to the interconnections, in
organisations. • Does not examine connections addressing gender inequality. context specific terms, between gender and other forms
• Little or no gender between gender and other • Addresses, where appropriate, of identity/exclusion e.g. caste, disability and in
analysis forms of social exclusion e.g. the multiple dimensions of particular ethnicity and HIV/AIDS.
• Limited if any those due to caste, class, gender inequality at multiple • Includes gender-specific changes in policies/practices
review of past ethnicity, indigeneity, levels. at micro, meso, macro levels that draws on the gender
practice. vulnerability to HIV/AIDS. • Includes gender-specific analysis.
• Repeated use of • Gender analysis tends to be changes in policies and • Analyses and links changes to external targets as set
gender neutral generic rather than context practice that address the issues out CEDAW, BPFA and other human rights
terms specific. identified in the analysis at least instruments, as appropriate.
• No specific • Strategies have limited focus at micro and meso levels. • Identifies approaches to policy and practice change at
strategies to on gender inequality or the • Includes strategies that are micro, meso, macro levels that empower men and
enhance women’s enhancement of women’s gender sensitive and support women to change gender relations.
status or address status i.e. interventions at only equal participation by men and • Identifies types of partners and/or proposes
gender inequality. one level and in particular do women. partnerships with key organisations at micro, meso and
• No OD strategy for not adequately address • Includes an OD strategy for macro levels with a strong commitment to or capacity
strengthening changes at meso and macro strengthening gender with promoting women’s status or enhancing gender
gender levels. mainstreaming in AusAID and equity.
mainstreaming. • Weak OD strategy if any. in partners. • Includes a gender-sensitive monitoring, evaluation and
learning framework.
• Includes a relevant, cost-effective strategy for
enhancing gender mainstreaming in AusAID and in
partners/allies that addresses key issues identified in
the internal context analysis.

45