NUMBER 5

2ND EDITION, 2010

PERFORMANCE MONITORING & EVALUATION

TIPS
USING RAPID APPRAISAL METHODS

ABOUT TIPS

These TIPS provide practical advice and suggestions to USAID managers on issues related to performance
monitoring and evaluation. This publication is a supplemental reference to the Automated Directive
System (ADS) Chapter 203.

WHAT IS RAPID
APPRAISAL?
Rapid Appraisal (RA) is an approach
that draws on multiple evaluation
methods and techniques to quickly,
yet systematically, collect data when
time in the field is limited. RA
practices are also useful when there
are budget constraints or limited
availability of reliable secondary
data. For example, time and budget
limitations may preclude the option
of using representative sample
surveys.
BENEFITS WHEN TO USE
RAPID APPRAISAL
METHODS
Rapid appraisals are quick and can
be done at relatively low cost.
Rapid appraisal methods can help
gather, analyze, and report relevant
information for decision-makers
within days or weeks. This is not
possible with sample surveys. RAs
can be used in the following cases:

for formative evaluations, to make

mid-course corrections in project
design or implementation when
customer or partner feedback
indicates a problem (See ADS
203.3.6.1);
when a key management decision
is required and there is inadequate
information;
for performance monitoring, when
data are collected and the
techniques are repeated over time
for measurement purposes;
to better understand the issues
behind performance monitoring
data; and
for project pre-design assessment.
LIMITATIONS WHEN
RAPID APPRAISALS ARE
NOT APPROPRIATE
Findings from rapid appraisals may
have limited reliability and validity,
and cannot be generalized to the
larger population. Accordingly,

rapid appraisal should not be the

sole basis for summative or impact
evaluations. Data can be biased and
inaccurate unless multiple methods
are used to strengthen the validity
of findings and careful preparation is
undertaken prior to beginning field
work.

WHEN ARE RAPID

APPRAISAL
METHODS
APPROPRIATE?
Choosing between rapid appraisal
methods for an assessment or more
time-consuming methods, such as
sample surveys, should depend on
balancing several factors, listed
below.
Purpose of the study. The
importance and nature of the
decision depending on it.
Confidence in results. The
accuracy, reliability, and validity of

findings needed for management

decisions.
Time frame. When a decision
must be made.
Resource constraints (budget).
Evaluation questions to be
answered. (see TIPS 3: Preparing
an Evaluation Statement of Work)

USE IN TYPES OF
EVALUATION
Rapid appraisal methods are often
used in formative evaluations.
Findings are strengthened when
evaluators
use
triangulation
(employing more than one data
collection method) as a check on
the validity of findings from any one
method.
Rapid appraisal methods are also
used in the context of summative
evaluations. The data from rapid
appraisal methods and techniques
complement the use of quantitative
methods such as surveys based on
representative
sampling.
For
example, a randomized survey of
small holder farmers may tell you
that farmers have a difficult time
selling their goods at market, but
may not have provide you with the
details of why this is occurring. A
researcher
could
then
use
interviews
with
farmers
to
determine the details necessary to
construct a more complete theory
of why it is difficult for small holder
farmers to sell their goods.

KEY PRINCIPLES
FOR ENSURING
USEFUL RAPID
APPRAISAL DATA
COLLECTION
No set of rules dictates which
methods and techniques should be
used in a given field situation;
however, a number of key principles

can be followed to ensure the

collection of useful data in a rapid
appraisal.
Preparation is key. As in any
evaluation, the evaluation design
and selection of methods must
begin
with
a
thorough
understanding of the evaluation
questions and the clients needs
for evaluative information. The
clients intended uses of data must
guide the evaluation design and
the types of methods that are
used.
Triangulation increases the validity
of findings. To lessen bias and
strengthen the validity of findings
from rapid appraisal methods and
techniques, it is imperative to use
multiple methods. In this way,
data collected using one method
can be compared to that collected
using other methods, thus giving a
researcher the ability to generate
valid and reliable findings. If, for
example, data collected using Key
Informant Interviews reveal the
same findings as data collected
from Direct Observation and
Focus Group Interviews, there is
less chance that the findings from
the first method were due to
researcher bias or due to the
findings being outliers. Table 1
summarizes
common
rapid
appraisal methods and suggests
how findings from any one
method can be strengthened by
the use of other methods.

COMMON RAPID
APPRAISAL
METHODS
INTERVIEWS
This method involves one-on-one
interviews with individuals or key
informants selected for their
knowledge or diverse views.
Interviews are qualitative, in-depth
and semi-structured.
Interview
guides are usually used and
2

EVALUATION METHODS
COMMONLY USED IN RAPID
APPRAISAL

Interviews
Community Discussions
Exit Polling
Transect Walks (see p. 3)
Focus Groups
Minisurveys
Community Mapping
Secondary Data Collection
Group Discussions
Customer Service Surveys
Direct Observation
questions may be further framed
during the interview, using subtle
probing techniques.
Individual
interviews may be used to gain
information on a general topic but
cannot provide the in-depth inside
knowledge on evaluation topics that
key informants may provide.
MINISURVEYS
A minisurvey consists of interviews
with between five to fifty individuals,
usually
selected
using
nonprobability sampling (sampling in
which respondents are chosen based
on their understanding of issues
related to a purpose or specific
questions, usually used when sample
sizes are small and time or access to
areas is limited).
Structured
questionnaires are used with a
limited number of close-ended
questions.
Minisurveys generate
quantitative data that can often be
collected and analyzed quickly.
FOCUS GROUPS
The focus group is a gathering of a
homogeneous body of five to twelve
participants to discuss issues and
experiences among themselves.
These are used to test an idea or to
get a reaction on specific topics. A
moderator introduces the topic,
stimulates
and
focuses
the

discussion, and prevents domination

of discussion by a few, while another
documents
the
evaluator
conversation.
THE ROLE OF TECHNOLOGY
IN RAPID APPRAISAL
Certain equipment and technologies
can aid the rapid collection of data
and help to decrease the incidence of
errors. These include, for example,
hand held computers or personal
digital assistants (PDAs) for data
input,
cellular
phones,
digital
recording devices for interviews,
videotaping and photography, and the
use of geographic information systems
(GIS) data and aerial photographs.

GROUP DISCUSSIONS
This method involves the selection
of approximately five participants
who are knowledgeable about a
given topic and are comfortable
enough with one another to freely
discuss the issue as a group. The
moderator introduces the topic and
keeps the discussion going while
another evaluator records the
discussion. Participants talk among
each other rather than respond
directly to the moderator.

COMMUNITY DISCUSSIONS
This method takes place at a public
meeting that is open to all
community members; it can be
successfully moderated with as
many as 100 or more people. The
primary interaction is between the
participants while the moderator
leads the discussion and asks
questions following a carefully
prepared interview guide.
DIRECT OBSERVATION
Teams of observers record what
they hear and see at a program site
using a detailed observation form.
Observation may be of the physical
surrounding or of ongoing activities,
processes, or interactions.
COLLECTING SECONDARY
DATA
This method involves the on-site
collection of existing secondary
data, such as export sales, loan
information, health service statistics,
etc. These data are an important
augmentation
to
information
collected using qualitative methods
such as interviews, focus groups, and
community discussions.
The

evaluator must be able to quickly

determine the validity and reliability
of the data. (see TIPS 12: Indicator
and Data Quality)
TRANSECT WALKS
The transect walk is a participatory
approach in which the evaluator
asks a selected community member
to walk with him or her, for
example, through the center of
town, from one end of a village to
the other, or through a market.
The evaluator asks the individual,
usually a key informant, to point out
and
discuss
important
sites,
neighborhoods, businesses, etc., and
to discuss related issues.
COMMUNITY MAPPING
Community mapping is a technique
that requires the participation of
residents on a program site. It can
be used to help locate natural
resources, routes, service delivery
points, regional markets, trouble
spots, etc., on a map of the area, or
to use residents feedback to drive
the development of a map that
includes such information.

COMMON RAPID APPRAISAL METHODS

Table 1
Method

Useful for
Providing

Example

Advantages

Limitations

Further
References

INDIVIDUAL INTERVIEWS
Interviews

A general overview of
the topic from
someone who has a
broad knowledge and
in-depth experience
and understanding
(key informant) or indepth information on
a very specific topic or
subtopic (individual)
Suggestions and
recommendations to
improve key aspects
of a program

Minisurveys

Quantitative data on
narrowly focused
questions, for a
relatively
homogeneous
population, when
representative
sampling is not
possible or required

Key informant:
Interview with
program
implementation
director

Provides in-depth, Susceptible to

inside information
interviewer and
on specific issues
selection biases
from the
Individual
individuals
interviews lack the
perspective and
broader
experience
understanding and
Flexibility permits
insight that a key
exploring
informant can
unanticipated
provide
topics

Interview with
director of a regional
trade association

Individual:
Interview with an
activity manager within
an overall
Easy to administer
development program
Low cost
Interview with a local
entrepreneur trying to
enter export trade

A customer service
assessment

Quantitative data
from multiple
respondents

Rapid exit interviews

after voting
Low cost

TIPS No. 2,
Conducting Key
Informant Interviews
K. Kumar, Conducting
Key Informant Surveys
in Developing
Countries, 1986
Bamberger, Rugh, and
Mabry, Real World
Evaluation, 2006
UNICEF Website: M&E
Training Modules:
Overview of RAP
Techniques

Findings are less

generalizable than
those from sample
surveys unless the
universe of the
population is
surveyed

TIPS No. 9,
Conducting Customer
Service Assessments
K. Kumar, Conducting
Mini Surveys in
Developing Countries,
1990
Bamberger, Rugh, and
Mabry, RealWorld
Evaluation, 2006 on
purposeful sampling

Quick data on
attitudes, beliefs,
behaviors of
beneficiaries or
partners

GROUP INTERVIEWS
Focus Groups

Customer views on
services, products,
benefits
Information on
implementation
problems
Suggestions and
recommendations for
improving specific
activities

Discussion on
Group discussion
experience related
may reduce
to a specific program
inhibitions,
intervention
allowing free
exchange of ideas
Effects of a new
business regulation
Low cost
or proposed price
changes

Discussion may be
dominated by a
few individuals
unless the process
is facilitated/
managed well

TIPS No. 10,

Conducting Focus
Group Interviews
K. Kumar, Conducting
Group Interviews in
Developing Countries,
1987
T. Greenbaum,
Moderating Focus
Groups: A Practical
Guide for Group
Facilitation, 2000

Group
Discussions

Community
Discussions

Understanding of
issues from different
perspectives and
experiences of
participants from a
specific subpopulation

Discussion with
young women on
access to prenatal
and infant care

Small group size

allows full
participation

Discussion with
entrepreneurs about
export regulations

Understanding of an
A Town Hall
issue or topic from a
meeting
wide range of
participants from key
evaluation sites within
a village, town, city, or
city neighborhood

Findings cannot be Bamberger, Rugh, and

Mabry, RealWorld
generalized to a
Evaluation, 2006
larger population
UNICEF Website: M&E
Training Modules:
Community Meetings

Allows good
understanding of
specific topics
Low cost
Yields a wide
range of opinions
on issues
important to
participants

Findings cannot be
generalized to
larger population
or to
subpopulations of
concern

Bamberger, Rugh, and

Mabry, RealWorld
Evaluation, 2006

Observer bias
unless two to
three evaluators
observe same
place or activity

TIPS No. 4, Using

Direct Observation
Techniques

Must be able to
determine
reliability and
validity of data

TIPS No. 12,

Guidelines for
Indicator and Data
Quality

A great deal of
information can be Larger groups
obtained at one
difficult to
point of time
moderate

UNICEF Website: M&E

Training Modules:
Community Meetings

ADDITIONAL COMMONLY USED TECHNIQUES

Direct
Observation

Visual data on physical Market place to

Confirms data
infrastructure,
observe goods being
from interviews
supplies, conditions
bought and sold,
Low cost
who is involved,
Information about an
sales interactions
agencys or businesss
delivery systems,
services

WFP Website:
Monitoring & Evaluation
Guidelines: What Is
Direct Observation and
When Should It Be Used?

Insights into behaviors

or events

Collecting
Secondary
Data

Validity to findings
gathered from
interviews and group
discussions

Microenterprise
bank loan info.
Value and volume of
exports

Quick, low cost

way of obtaining
important
quantitative data

Number of people
served by a health
clinic, social service
provider

PARTICIPATORY TECHNIQUES
Transect
Walks

Community
Mapping

Important visual and

locational information
and a deeper
understanding of
situations and issues

Walk with key

informant from one
end of a village or
urban neighborhood
to another, through
a market place, etc.

Info. on locations
important for data
collection that could
be difficult to find

Insiders viewpoint Susceptible to

interviewer and
Quick way to find
selection biases
out location of
places of interest
to the evaluator
Low cost

Map of village and

Important
Rough locational
surrounding area
locational data
information
with locations of
when there are no
markets, water and
detailed maps of
fuel sources, conflict
the program site
Quick comprehension
areas, etc.
on spatial location of
services/resources in a
region which can give
insight to access issues

Bamberger, Rugh, and

Mabry, Real World
Evaluation, 2006
UNICEF Website: M&E
Training Modules:
Overview of RAP
Techniques

Bamberger, Rugh, and

Mabry, Real World
Evaluation, 2006
UNICEF Website: M&E
Training Modules:
Overview of RAP
Techniques

References Cited
M. Bamberger, J. Rugh, and L. Mabry, Real World Evaluation. Working Under Budget, Time, Data, and Political
Constraints. Sage Publications, Thousand Oaks, CA, 2006.
T. Greenbaum, Moderating Focus Groups: A Practical Guide for Group Facilitation. Sage Publications, Thousand Oaks,
CA, 2000.
K. Kumar, Conducting Mini Surveys in Developing Countries, USAID Program Design and Evaluation Methodology
Report No. 15, 1990 (revised 2006).
K. Kumar, Conducting Group Interviews in Developing Countries, USAID Program Design and Evaluation
Methodology Report No. 8, 1987.
K. Kumar, Conducting Key Informant Interviews in Developing Countries, USAID Program Design and Evaluation
Methodology Report No. 13, 1989.

For more information:

TIPS publications are available online at [insert website].

Acknowledgements:
Our thanks to those whose experience and insights helped shape this publication including USAIDs Office of
Management Policy, Budget and Performance (MPBP). This publication was authored by Patricia Vondal, PhD., of
Management Systems International.
Comments regarding this publication can be directed to:
Gerald Britan, Ph.D.
Tel: (202) 712-1158
gbritan@usaid.gov
Contracted under RAN-M-00-04-00049-A-FY0S-84
Integrated Managing for Results II