1996, Number 6

Performance Monitoring and Evaluation

TIPS
USAID Center for Development Information and Evaluation

SELECTING PERFORMANCE INDICATORS

What Are Performance Indicators?

To manage for Simply put, performance indicators are measures that describe how well a program is
results, USAID achieving its objectives.
operating units
Whereas a results statement identifies what we hope to accomplish, indicators tell us
need reliable and specifically what to measure to determine whether the objective has been achieved.
timely data on Indicators are usually quantitative measures but may also be qualitative observations.
They define how performance will be measured along a scale or dimension, without
their program specifying a particular level of achievement. (Planned levels of achievement -- targets
results. -- are separate from the indicators themselves).
Performance USAID operating units have developed hundreds of performance indicators in recent
indicators define years. Common examples include the dollar value of non-traditional exports, private
the data to be investment as a percentage of gross domestic product, contraceptive prevalence rates,
child mortality rates, and percentage of eligible voters voting.
collected to
measure Why Are Performance Indicators Important?
progress, and
Performance indicators are at the heart of a performance monitoring system -- they
are thus an define the data to be collected to measure progress and enable actual results achieved
indispensable over time to be compared with planned results. Thus, they are an indispensable
tool for decision- management tool for making performance-based decisions about program strategies
and activities.
making.
Other ways that performance indicators, and the data collected on them, can be used
include the following:
This Tips offers
advice for to orient and motivate operating unit staff toward achieving results
selecting
to communicate USAID achievements to host country counterparts, other
appropriate partners, and customers and
and useful
performance to report results achieved to USAID's stakeholders, including the U.S.
Congress, Office of Management, and Budget, and citizens.
indicators.

PN-ABY-214
2
Use a Participatory Approach

Reengineering requires operating units to use a participatory approach in selecting indicators for their performance
monitoring system. Collaborating closely with development partners, host country counterparts, and customers at
each step of the indicator selection process has many benefits. It makes good sense to draw on the experience of
others and obtain their consensus throughout the process.

Avoid overly broad results statements. Sometimes objec-

For What Results Are Performance tives and results are so broadly stated it is difficult to iden-
Indicators Required? tify the right performance indicators. Instead, specify those
aspects believed to make the greatest difference to im-
proved performance. For example, rather than using a
Reengineering guidance requires operating units to develop broad results statement like "improved capacity" of a host
performance indicators for all strategic objectives, strategic country institution, clarify those aspects that program
support objectives, special objectives, and USAID-sup- activities emphasize. For example, improved personnel
ported intermediate results (referred to below as SOs and recruitment process, or improved management skills.
IRs) identified in the results frameworks.
Be clear about what type of change is implied. What is
Some means should also be developed for gathering infor- expected to change -- a situation, a condition, the level of
mation on the results supported by development partners knowledge, an attitude, a behavior? For example, changing
and on the status of critical assumptions, although less a country's law about voting is very different from chang-
rigorous standards apply. ing citizens' awareness of their right to vote, which again is
different from their voting behavior. Each type of change is
Also, SO teams are required to collect data regularly on measured by different types of indicators.
activity-level inputs, outputs, and processes to ensure they
are proceeding as expected and are contributing to relevant Also, clarify whether the change being sought is an abso-
IRs and SOs. This implies some thought be given to devel- lute change, a relative change, or no change.
oping indicators for monitoring progress at the activity
level.
--Absolute changes involve the creation or introduction of
something new.
Steps in Selecting Performance
Indicators --Relative changes involve increases, decreases, improve-
ments, strengthening or weakening in something that cur-
Selecting appropriate and useful performance indicators is rently exists, but at a higher or lower level than is consid-
a fairly straightforward process, but requires careful ered optimum.
thought, iterative refining, collaboration, and consensus-
building. Here are some suggestions. Although presented as --No change involves the maintenance, protection or pres-
discrete steps, in practice some of these can be effectively ervation of something that is considered fine as is.
undertaken simultaneously.
Be clear about where change should appear. Is change
Step 1. Clarify the results statements. expected to occur among individuals, families, groups,
communities, regions? Clearly, a change in the savings rate
Good performance indicators start with good results state- for an entire nation will be quite different than for a partic-
ments that people can understand and agree on. ular sector of the business community. This is known as
identifying the "unit of analysis" for the performance indi-
cator.
Carefully consider the result desired. Review the precise
wording and intention of the strategic objective, strategic
support objective, special objective, intermediate result, Identify more precisely the specific targets for change. Who
critical assumption, or result supported by partners. What or what are the specific targets for the
exactly does it say?
3
change? For example, if individuals, which individuals? The key to creating a useful initial list of performance
Average citizens or exporters? All exporters or only export- indicators is to be inclusive. That is, view the desired result
ers of non-traditional agricultural products? in all its aspects and from all perspectives. Allow sufficient
opportunity for a free flow of ideas and creativity.
Study the activities and strategies directed achieving
change. Some activities will produce the desired change Step 3. Assess Each Possible Indicator.
directly, while other activities will produce the change less
directly. For example, activities to develop Next, assess each possible indicator on the initial list. Expe-
microenterprises aim to increase employment directly. rience suggests using seven basic criteria for judging an
Activities to reform economic policies may have the same indicator's appropriateness and utility. These seven criteria
effect, but less directly. Before appropriate indicators can are described in the box on page 4.
be developed, clarity is needed about the expected relation-
ship between activities and their intended results, in order
to understand exactly what changes are reasonable to ex- When assessing and comparing possible indicators, it is
pect. helpful to use a matrix with the seven criteria arrayed
across the top and the candidate indicators listed down the
left side. With a simple scoring scale, for example 1-5, rate
Step 2. Develop a List of Possible Indicators. each candidate indicator against each criterion. These rat-
ings will help give an overall sense of the indicator's rela-
There are usually many possible indicators for any desired tive merit, and help in the selection process. However,
outcome, but some are more appropriate and useful than apply this approach flexibly and with judgment, because all
others. In selecting indicators, don't settle too quickly on the seven criteria may not be equally important.
first that come most conveniently or obviously to mind. A
better approach is to start with a list of alternatives, which Step 4. Select the "Best" Performance Indicators.
can then be assessed against a set of selection criteria.
The next step is to narrow the list to the final indicators that
To create the initial list of possible indicators, tap the fol- will be used in the performance monitoring system. They
lowing sources: should be the optimum set that meets the need for
management-useful information at a reasonable cost.
internal brainstorming by the strategic objective
team Be selective. Remember the costs associated with data
collection and analysis. Limit the number of indicators used
consultations with experts in the substantive pro- to track each objective or result to a few (two or three).
gram area Select only those that represent the most basic and impor-
tant dimensions of our aims.
experience of other operating units with similar
indicators.

CDIE's Tips series provides advice and sug-

gestions to USAID managers on how to plan
Tip: When developing indicators, consider and conduct performance monitoring and
tapping information from a) the PME da- evaluation activities effectively. They are
tabase on indicators other operating units supplemental references to the
have used for similar objectives; and b) on- reengineering directives system (ADS),
going work by technical groups in the chapter 203. For further information, con-
Agency goal areas to develop common or tact Annette Binnendijk, CDIE Senior Eval-
generally used indicators. uation Advisor, via phone (703) 875-4235,
fax (703) 875-4866, or e-mail. Copies of Tips
can be ordered from the Development
Information Services Clearinghouse by call-
ing (703) 351-4006 or by faxing (703) 351-
4039. Please refer to the PN number. To
order via the Internet, address requests to
docorder@disc.mhs.compuserve.com
4
SEVEN CRITERIA FOR ASSESSING PERFORMANCE INDICATORS

1. DIRECT. A performance indicator should measure as closely as possible the result it is intended to measure. It should not be
pegged at a higher or lower level than the result being measured. For example,contraceptive prevalence rateis a direct measure of
the result increased use of family planning methods. But number of service providers trainedwould NOT be a direct measure of the
result improved service delivery. Just because people are trained does not necessarily mean they will deliver services better.

If using a direct measure is not possible, one or more proxy indicators might be appropriate. For example, sometimes reliable data on
direct measures are not available at a frequency that is useful to managers, and proxy indicators are needed to provide timely insight
on progress. Proxy measures areindirect measures that are linked to the result by one or more assumptions. For example, in rural
areas of Africa it is often very difficult to measure income levels directly. Measures such as percentage of village households with tin
roofs (or radios or bicycles) may be a useful, if somewhat rough, proxy. The assumption is that when villagers have higher income
they tend to purchase certain goods. If convincing evidence exists that the assumption is sound (for instance, it is based on research
or experience elsewhere), then the proxy may be an adequate indicator, albeit second-best to a direct measure.

2. OBJECTIVE.An objective indicator has no ambiguity about what is being measured. That is, there is general agreement over
interpretation of the results. It is both unidimensional and operationally precise. To be
unidimensional means that it measures only
one phenomenon at a time. Avoid trying to combine too much in one indicator, such as measures of both access and use. Operational
precision means no ambiguity over what kind of data would be collected for an indicator. For example, whilenumber of successful
export firms is ambiguous, something likenumber of export firms experiencing an annual increase in revenues of at least 5 percent is
operationally precise.

3. ADEQUATE.Taken as a group, a performance indicator and its companion indicators should adequately measure the result in
question. A frequently asked question is "how many indicators should be used to measure any given result?" The answer depends on
a) the complexity of the result being measured, b) the level of resources available for monitoring performance, and c) the amount of
information needed to make reasonably confident decisions. For some results that are straightforward and have tried and true
measures, one performance indicator may be enough. For example, if the intended result increased
is traditional exports, the
indicator dollar value of traditional exports per yearis probably sufficient. Where no single indicator is sufficient, or where there are
benefits to be gained by "triangulation" -- then two or more indicators may be needed. However, avoid using too many indicators. Try
to strike a balance between resources available for measuring performance and the amount of information managers need to make
reasonably well informed decisions.

4. QUANTITATIVE, WHERE POSSIBLE.Quantitative indicators are numerical (number or percentage of dollar value, tonnage,
for example). Qualitative indicators are descriptive observations (an expert opinion of institutional strength, or a description of
behavior). While quantitative indicators are not necessarily more objective, their numerical precision lends them to more agreement
on interpretation of results data, and are thus usually preferable. However, even when effective quantitative indicators are being used,
qualitative indicators can supplement the numbers and percentages with a richness of information that brings a program's results to
life.

5. DISAGGREGATED, WHERE APPROPRIATE. Disaggregating people-level program results by gender, age, location, or
some other dimension is often important from a management or reporting point of view. Experience shows that development
activities often require different approaches for different groups and affect those groups in different ways. Disaggregated data help
track whether or not specific groups participate in and benefit from activities intended to include them. Therefore, it makes good
management sense that performance indicators be sensitive to such differences.

6. PRACTICAL.An indicator is practical if data can be obtained in a timely way and at a reasonable cost. Managers require data
that can be collected frequently enough to inform them of progress and influence decisions. USAID operating units should expect to
incur reasonable, but not exorbitant, costs for obtaining useful performance information. A rule of thumb, given in the reengineering
guidance, is to plan on allocating 3 to 10 percent of total program resources for performance monitoring and evaluation.

7. RELIABLE. A final consideration in choosing performance indicators is whether data of sufficiently reliable quality for
confident decision-making can be obtained. But what standards of data quality are needed to useful?
be The data that a program
manager needs to make reasonably confident decisions about a program is not necessarily the same rigorous standard a social
scientist is looking for. For example, a low cost minisurvey may be good enough for a given management need.