J.M.

Carroll / Interacting with Computers 13 (2000) 43±60 43

Interacting with Computers 13 (2000) 43±60

www.elsevier.com/locate/intcom

Five reasons for scenario-based design

J.M. Carroll*
Department of Computer Science and Center for Human±Computer Interaction, Virginia Tech, Blacksburg, VA
24061-0106, USA

Abstract
Scenarios of human±computer interaction help us to understand and to create computer systems
and applications as artifacts of human activityÐas things to learn from, as tools to use in one's work,
as media for interacting with other people. Scenario-based design of information technology
addresses ®ve technical challenges: scenarios evoke re¯ection in the content of design work, helping
developers coordinate design action and re¯ection. Scenarios are at once concrete and ¯exible,
helping developers manage the ¯uidity of design situations. Scenarios afford multiple views of an
interaction, diverse kinds and amounts of detailing, helping developers manage the many conse-
quences entailed by any given design move. Scenarios can also be abstracted and categorized,
helping designers to recognize, capture and reuse generalizations and to address the challenge
that technical knowledge often lags the needs of technical design. Finally, scenarios promote
work-oriented communication among stakeholders, helping to make design activities more acces-
sible to the great variety of expertise that can contribute to design, and addressing the challenge that
external constraints designers and clients face often distract attention from the needs and concerns of
the people who will use the technology. q 2000 Elsevier Science B.V. All rights reserved.

Keywords: Scenario-based design; Human±computer interaction; Work-oriented communication

1. Introduction

Designers of information systems and applications face a disturbing reality. While there
is plenty of opportunity to do things that make a difference, it is never unequivocal just
what should be done, or even just what the real problems are. The problems can only be
de®nitively analyzed by being solved; the appropriate solution methods must typically
be executed in order to be identi®ed; the solutions must be implemented in order to be
speci®ed. All the while, the designer faces convoluted networks of tradeoff and inter-
dependency, the potential of untoward impacts on people and their social institutions, and

* Tel.: 11-540-231-8453.
E-mail address: carroll@cs.vt.edu (J.M. Carroll).

0953-5438/00/$ - see front matter q 2000 Elsevier Science B.V. All rights reserved.
PII: S 0953-543 8(00)00023-0
44 J.M. Carroll / Interacting with Computers 13 (2000) 43±60

the likelihood that changing cultural and technological circumstances will obviate any
solution before it can be deployed.
Most software engineering methods belong to a methodological tradition that seeks to
control the complexity and ¯uidity of design through techniques that ®lter the information
considered and decompose the problems to be solved. A complementary tradition seeks to
exploit the complexity and ¯uidity of design by trying to learn more about the structure
and dynamics of the problem domain, by trying to see the situation in many different ways,
and by interacting intimately with the concrete elements of the situation (Ackoff, 1979a,b;
Checkland, 1981; SchoÈn, 1983).
Scenario-based design techniques belong to this complementary approach. In scenario-
based design, descriptions of how people accomplish tasks are a primary working design
representation. Software design is fundamentally about envisioning and facilitating new
ways of doing things and new things to do. Maintaining a continuous focus on situations of
and consequences for human work and activity promotes learning about the structure and
dynamics of problem domains, seeing usage situations from different perspectives, and
managing tradeoffs to reach usable and effective design outcomes (Carroll, 1994, 1995).

2. What are scenarios?

Computers are more than just functionality. They unavoidably restructure human activ-
ities, creating new possibilities as well as new dif®culties. Conversely, each context in
which humans experience and act provides detailed constraint for the development and
application of computer technologies. In analyzing and designing systems and software we
need better means to talk about how they may transform and/or be constrained by the
contexts of user activity: this is the only way we can hope to attain control over the
ªmaterialsº of design. A direct approach is to explicitly envision and document typical
and signi®cant user activities early and continuingly in the development process. Such
descriptions, often called ªscenarios,º support reasoning about situations of use, even
before those situations are actually created.
Scenarios are stories. They are stories about people and their activities. For example, an
accountant wishes to open a folder on the system desktop in order to access a memo on
budgets. However, the folder is covered up by a budget spreadsheet that the accountant
wishes to refer to while reading the memo. The spreadsheet is so large that it nearly ®lls the
display. The accountant pauses for several seconds, resizes the spreadsheet, moves it
partially out of the display, opens the folder, opens the memo, resizes and repositions
the memo and continues working.
This is about as mundane a work scenario as one could imagine. Yet even this scenario
speci®es window management and application switching functionality vividly and point-
edly: people need to coordinate information sources, to compare, copy and integrate data
from multiple applications; displays inevitably get cluttered; people need to ®nd and
rearrange windows in the display. Scenarios highlight goals suggested by the appearance
and behavior of the system, what people try to do with the system, what procedures are
adopted, not adopted, carried out successfully or erroneously, and what interpretations
people make of what happens to them. If the accountant scenario is typical of what people
 J.M. Carroll / Interacting with Computers 13 (2000) 43±60 45

want to do, it substantively constrains design approaches to window management and

switching.
Scenarios have characteristic elements (Potts, 1995). They include or presuppose a
setting: The accountant scenario explicitly describes a starting state for the described
episode; the relative positions of the folder and spreadsheet, and the presence of the
accountant. The scenario implies further setting elements by identifying the person as
an accountant, and the work objects as budgets and memos.
Scenarios also include agents or actors: the accountant is the only agent in this example,
but it is typical of human activities to include several to many agents. Each agent or actor
typically has goals or objectives. These are changes that the agent wishes to achieve in the
circumstances of the setting. Every scenario involves at least one agent and at least one
goal. When more than one agent or goal is involved, they may be differentially prominent
in the scenario. Often one goal is the de®ning goal of a scenario, the answer to the question
ªwhy did this story happen?º Similarly, one agent might be the principal actor, the answer
to the question ªwho is this story about?º.
In the accountant scenario, the de®ning goal is displaying the memo in such a way that both
the memo and budget can be examined. A subgoal is opening the folder in which the memo
is located, and a further subgoal is moving the budget to allow the folder to be opened.
Scenarios have a plot; they include sequences of actions and events, things that actors
do, things that happen to them, changes in the circumstances of the setting and so forth.
Particular actions and events can facilitate, obstruct, or be irrelevant to given goals.
Resizing the spreadsheet and moving it out of the display are actions that facilitate the
goal of opening the folder. Resizing and repositioning the memo are actions that facilitate
the goal of displaying the memo so that it can be examined with the budget. Pausing is an
action that is irrelevant to any goal, though it suggests that the accountant's goal-oriented
actions were not completely ¯uent. Notably, actions and events can often change the
goalsÐeven the de®ning goalÐof a scenario.
Representing the use of a system or application with a set of user interaction scenarios
makes that use explicit, and in doing so orients design and analysis toward a broader view
of computers. It can help designers and analysts to focus attention on the assumptions
about people and their tasks that are implicit in systems and applications. Scenario repre-
sentations can be elaborated as prototypes, through the use of storyboard, video and rapid
prototyping tools. They are the minimal contexts for developing user-oriented design
rationale: a given design decision can be evaluated and documented in terms of its speci®c
consequences within particular scenarios. Scenarios and the elements of scenario-based
design rationale can be generalized and abstracted using theories of human activity, enabling
the cumulation and development of knowledge attained in the course of design. Thus, scenar-
ios can provide a framework for a design-based science of human±computer interaction.
In the rest of this paper, we review ®ve of the key challenges for design methods and
illustrate for each the corresponding response of scenario-based design.

3. Challenge: design action competes with re¯ection

Technical professionals are intelligent people performing complex and open-ended

46 J.M. Carroll / Interacting with Computers 13 (2000) 43±60

tasks. They want to re¯ect on their activities, and they routinely do re¯ect on their
activities. However, people take pride not only in what they know and learn, but in
what they can do and in what they actually produce. There is a fundamental tension
between thinking and doing: thinking impedes progress in doing, and doing obstructs
thinking. Sometimes this con¯ict is quite sharp, as when one must stop and think before
taking another step. But frequently it is more a matter of trading off priorities.
SchoÈn (1983, 1987) has discussed this con¯ict extensively in his books on re¯ective
practice. For example, he analyzes a coach reacting to an architecture student's design
concept for a school building; the design includes a spiral ramp intended to maintain
openness while breaking up lines of sight (she calls the idea ªa Guggenheimº):
¼ when I visited open schools, the one thing they complained about was the ware-
house qualityÐof being able to see for miles. It [the ramp] would visually and
acoustically break up the volume. (SchoÈn, 1987, p. 129).
The coach feels that she needs to explore and develop her concept more thoroughly,
noting that a ramp has thickness and that this will limit her plans to use the space under-
neath the ramp; he urges her to draw sections. However, he does not bother to justify this
advice to the student, to allow her to see the rationale behind his advice; as SchoÈn puts it,
he does not reveal ªthe meanings underlying his questionsº (SchoÈn, 1987, p. 132). SchoÈn
regards this as a hopeless confrontation in which no progress can be made on the particular
design project, or on the larger project of understanding how to design. Both the student
and the coach are willing to act publicly and to share actions, but do not re¯ect enough on
their own and one another's values and objectives and on their interpersonal dynamics.
Re¯ection is not always comfortable; it forces one to consider one's own competence,
to open oneself to the possibility of being wrong. As SchoÈn (1987) put it,
The interactions I have suggested emphasize surfacing private attributions for public
testing, giving directly observable data for one's judgments, revealing the private
dilemmas with which one is grappling, actively exploring the other's meaning, and
inviting the other's confrontation of one's own. (p. 141).
In general, people do not like to make themselves self-aware of their own roles as actors
in situations; indeed, being ªobjectively self-awareº impairs performance of nonroutine
tasks (Duval and Wicklund, 1972)Ðlike designing software!

4. Scenarios evoke re¯ection in design

Designers do try to create opportunities for their own re¯ection. They organize design
review meetings in which the whole team works through a set of requirements, a progress
report, or a speci®cation. It is also common to build early prototypes to verify and re®ne
design requirements; one can directly observe prospective users interacting with such a
prototype to make a formative evaluation (Scriven, 1967) of the design. These activities
can facilitate the identi®cation and integration of different perspectives; they can raise
concrete and detailed design issues to guide further work. In this way they help designers
to re¯ect on the work they have already done. However, they do not evoke re¯ection in the
 J.M. Carroll / Interacting with Computers 13 (2000) 43±60 47

Fig. 1. A usage scenario for a multimedia education project.

context of doing design. Though design reviews and formative evaluations are re¯ective
activities, they are ancillary activities that must be coordinated with design itself. Proto-
typing is directed at building and testing software that embodies a design, not on thinking
about the design as it is produced.
Constructing scenarios of use inescapably evokes re¯ection in the context of design.
Consider the scenario in Fig. 1: it succinctly and concretely conveys a vision of the system,
in this case a vision of student-directed, multimedia instruction. It is a coherent and
concrete vision, not an abstract goal or a list of requirements. Elements of the envisioned
system appear in the scenario embedded in the user interactions that will make them
meaningful to the userÐperhaps revelatory, perhaps cryptic, but de®nitely more than
just technological capabilities. For example, the role of natural language query is exem-
pli®ed as a means of locating further case studies that illustrate the principles of harmonic
motion.
The scenario emphasizes and explores goals that the user may adopt and pursue, such as
watching the ®lm clips twice, or skipping the exercises. Some of these goals are oppor-
tunistic, such as investigating the Tacoma Narrows collapse because of experience with a
totally unrelated bridge collapse, or deciding to branch from bridge failures to ¯utes. The
scenario implicitly articulates the usage situation from multiple perspectives: the student
stores and annotates a video clip with speech, raising speci®c requirements for user inter-
face tools and presentation as well as for particular data structures and memory. The
scenario impels the designer to integrate the consideration of such system requirements
with consideration of the motivational and cognitive issues in education that underline the
user's actions and experiences.
The scenario concretely embodies a partial view of the design, and thereby exposes the
design to critique. In this sense, the scenario as an object can function much like a ªsoftº
prototype (though, as we have suggested, the process of designing a scenario more
strongly evokes re¯ection). Indeed, scenarios have been found to be extremely useful as
focal objects in both design reviews and formative evaluations (Carroll and Rosson, 1985;
Karat and Bennett, 1991; Chin et al., 1997). The scenario exposes not only the function-
ality of the system, but speci®c claims about how the user will access that functionality and
what the user will experience in doing so.
SchoÈn (1983, pp. 67±68) drew an important contrast between merely creating or
48 J.M. Carroll / Interacting with Computers 13 (2000) 43±60

identifying elements of the problem context, and ªgiving them reasonº. He characterized
design as inquiry, and the designer as ªa researcher in a practice contextº (p. 68). From this
perspective, the designer's domain expertise is both a prerequisite and a potential liability.
Having a category or a label with which to ªunderstandº a situation often establishes an
insurmountable obstacle to achieving an innovative understanding. In the Guggenheim
example, the design is seen by the student only as a geometry affording patterns of light
and sound. In her sketches, the Guggenheim analogy is static, it is a design ªanswerº that
fails to evoke critical questions about the space underneath the ramp.
Creating and using scenarios pushes beyond static answers. The Guggenheim is an
environment for walking and sitting underneath the ramp, among other activities.
Embodying the activities of the Guggenheim space instantly raises questions about the
space. Indeed, emphasizing user activities makes it easier for designers to integrate re¯ec-
tion and action in their own design practice. A state of subjective self-awareness is
encouraged by designing scenarios because the focus of attention is the activities and
experiences of the prospective user, not the designer's professional virtuosity. Design
work can be very personal, but fundamentally it is about the user's activity and not
about the designer's skills.

5. Challenge: design situations are ¯uid

Design analysis is always indeterminate, because design changes the world within
which people act and experience. The rapid evolution of spreadsheet software in the
1980s does not indicate a failure in the original requirements analysis for VisiCalc, but
rather suggests the extent to which the original spreadsheet programs altered the work
situations in which these program were used (Carroll and Rosson, 1991). Requirements
always change (Brooks, 1995). When designs incorporate rapidly evolving technologies,
requirements change even more rapidly. The more successful, the more widely adopted
and the more impactful a design is, the less possible it will be to determine its correct
design requirements. And in any case, re®nements in software technology and new
perceived opportunities and requirements propel a new generation of designs every 2±
3 years.
There is a tendency to think of the indeterminacies of technology development in
positive terms: the world is getting better, albeit sometimes in ways we did not expect.
Recently this positive attitude has been supplemented with growing acknowledgement of
potential negative consequences, such as pollution of groundwater and the deterioration of
the atmosphere. But both these emphases underanalyze the extent to which design, and
especially the design of new technology, undermines the stability of the world. SchoÈn
(1967), for example, noted that in our era technology development constantly erodes the
manager's concept of ªthe business I'm inº (p. 195) and a worker's notion of the special
skill he or she possesses; it erodes the traditional staging of life into education followed by
steady practice. Even writers, such as McLuhan (1994) who revel in the prospect of
ªlearning a living,º agree about the magnitude of this instability.
The instability of design situations originates inside design teams as well as outside. As
a project goes forward, its funding may be threatened or restructured; various stakeholders
 J.M. Carroll / Interacting with Computers 13 (2000) 43±60 49

or team members may change their interests or priorities, or may even leave the team.
Others with unknown interests and priorities may join. This creates a chronic need for
education and consensus building to ensure that there is agreement as to what the require-
ments are at any given point in time.
SchoÈn (1967) stressed that in the face of great instability, people create illusions of
stability to manage their own uncertainties and potentially disturbing perceptions. This is
consistent with a substantial body of social psychology research: people create logically
tidy interpretations of their experience, even if they must ªadjustº their actual perceptions
in order to do this (Nisbett and Wilson, 1997). People protect their interpretations by
selectivity reconstructing their own experience (Erikson, 1980). And the more reconstruc-
tionist effort people must expend to maintain an interpretation, the more ardently it is held,
even in the face of incontestable discon®rmation (Festinger, 1957).
Given the human bias stability, we should not be surprised by the persistence of the
attitude that design problems can be planfully decomposed into routine subproblems, that
scienti®c principles can be mechanically applied to expose and manage the underlying
orderliness of design problems. From the standpoint of the psychology of designers and
managers, it could conceivably become even stronger as it becomes more clearly inap-
propriate (Festinger et al., 1956). These dysfunctional beliefs about design would not be
such a problem if they could only lead to standstill. The potentially dangerous aspect is
that they can lead to ªsuccessº in accomplishing the wrong things. Designers almost
always design something. If their need for stability induces them to prematurely close
their designs, to cease re¯ecting and critiquing, to declare victory, they may produce a
solution for requirements that no longer exist.

6. Scenarios are at once concrete and ¯exible

To manage an ambiguous and dynamic situation, one must be concrete but ¯exible. One
must be concrete just to avoid being swallowed by the indeterminacies; one must be
¯exible to avoid being captured by a false step. Systematic decomposition is a traditional
approach to managing ambiguity, but it does not afford ¯exibility. Instead one ends up
with a set of concrete subsolutions, each of which is fully speci®ed. Unfortunately, by the
time the set of subsolutions is speci®ed, the requirements often have changed.
Scenarios of use help us to reconcile concreteness and ¯exibility. They are concrete in
the sense that they simultaneously ®x an interpretation of the design situation and offer a
speci®c solution: the scenario in Fig. 1 speci®es a particular usage experience that could be
prototyped and tested. At the same time the scenario is ¯exible, deliberately incomplete
and easily revised or elaborated: in a few minutes, a piece of the scenario could be-written
(e.g. perhaps the associated module opens automatically) or elaborated (e.g. the module
may be opened by following a ªrelated materialsº tag attached to the ®lm clip).
A scenario provides a concrete envisionment of a design solution, but can be couched at
many levels of detail. Initial scenarios are typically very rough. They specify a system
design by specifying the tasks users can or must carry out, but without committing to the
lower-level details of precisely how the tasks will be carried out or how the system will
enable the functionality for those tasks. The narrative in Fig. 1 is at an intermediate level,
50 J.M. Carroll / Interacting with Computers 13 (2000) 43±60

with some detail regarding task ¯ow, but not at the level of individual user±system
interactions.
Thus scenarios provide a stable foundation for action-oriented re¯ection. By being
both concrete and rough, they make explicit the design goal of specifying tasks and
functions in greater detail. But they do this by opening up the issue of how to do
this detailed design, instead of closing it out prematurely. Scenarios allow designers
to provisionally construct a space of user tasks despite the instability in requirements
originating from the context of technology development. Scenarios are particularly valu-
able for managing the instability originating inside design teams: They are broadly acces-
sible to various stakeholders and team members, unlike artifact-orientated decompositions
such as functional speci®cations.
The notion that-scenarios of use are an appropriately concrete but rough design object
extends several earlier analyses. For example, SchoÈn (1983, pp. 277±279) emphasized that
effective re¯ection must be tightly coupled to action: the analysis need not be complete
and consistent, it need only guide a restructuring of the current situation that can produce
new design actions or new insights. His cases are drawn from design domains for which
there are rich languages to allow designers to quickly create and explore situations.
Scenarios can be such a language for the design of human±computer interactions.
In an earlier book, SchoÈn (1967, p. 41) argued that decomposition is useful
chie¯y through the side-effects of eliciting and focusing concrete action, and thereby
reducing uncertainty. But he warned that the resultant decomposition can ªstrangleº
innovation. Ackoff (1979a) similarly developed the notion of ªidealized designº: a
process of planning ªthe system with which the designers would replace the existing
system now if they were free to do soº (p. 191). He argued that such a process can
increase participation among stakeholders, mobilize participants with respect to the
project, expand the concept of what is feasible, increase both creativity and scope of
human and organizational consequences that are considered in the design process; an
idealized design is really just a vehicle for focusing the articulation of possibilities and
the discussion of their consequences.

7. Challenge: design moves have many consequences

Every element of a design, every move that a designer makes, has a variety of potential
consequences. In their work on violin design, the Catgut Society found that additional rib
holes were needed in their new treble violin to pitch the instrument high enough and yet
maintain the neck length for ®ngering. This design move had the intended consequences,
but also forced the designers to consider stronger materials for the ribs, and ultimately to
use aluminum instead of wood. However the aluminum ribs caused a nasal quality in the
lower strings, forcing the designers to reconsider a new wooden rib designÐribs so thin
(0.7 mm) that the designers believed on analytical grounds that they would collapse. A
typical complication in this design case is that the various consequences belonged to
different stakeholders: a ®ngerboard that is too short is a problem for the musician. A
very thin wooden rib is a problem for the instrument maker. A nasal tone is a problem for
the musician and listener (Hutchins, 1981).
 J.M. Carroll / Interacting with Computers 13 (2000) 43±60 51

SchoÈn (1983, p. 101) sees design as a ªconversationº with a situation comprised of

many interdependent elements. The designer makes moves and then ªlistensº to the design
situation to understand their consequences:
In the designer's conversation with the materials of his design, he can never make a
move which has only the effect intended for it. His materials are continually talking
back to him, causing him to apprehend unanticipated problems and potentials.
Thus, the Catgut group made the move of employing aluminum ribs and then noticed
the unanticipated problem of a nasal tone in the lower strings. When a move produces
unexpected consequences, and particularly when it produces undesirable consequences,
the designer articulates ªthe theory implicit in the move, criticizes it, restructures it, and
tests the new theory by inventing a move consistent with itº (SchoÈn, 1983, p. 155).
SchoÈn's approach to managing the interdependencies within a design situation by
treating design as inquiry raises several questions. What directs the designer to investigate
various types of consequences, to listen for various types of backtalk? How are different
types and sources of backtalk integrated in the designer's restructured design theory? The
moves for the treble violin had various consequences. The Catgut group effectively inte-
grated these in their subsequent design work, but how did they do that and, more generally,
how can we support outcomes like that? Designers need a language for this conversation
and they need techniques for managing the consequences.

8. Any scenario has many possible views

Scenarios of use are multifarious design objects; they can be used to describe designs at
multiple levels of detail and with respect to multiple perspectives. The scenario in Fig. 1
provides a high-level task view, but it easily could be elaborated with respect to the
detailed moment-to-moment thoughts and experiences of the user in order to provide a
more detailed cognitive view, or the user's moment-to-moment actions to provide a more
detailed functional view. Alternatively, it could be elaborated in terms of hardware and
software components that could implement the envisioned functionality in order to
provide a system view (use cases play this role in object-oriented software engineering
(Jacobson, 1995; Wirfs-Brock, 1995)). Each of these variations in resolution and perspec-
tive is a permutation of a single underlying use scenario. Indeed, the permutations are
integrated through their roles as complementary views of the same design object.
Scenario analyses can leave implicit the underlying causal relationships among the
entities in a situation of use. In Fig. 1, the envisioned speech annotation capability allows
the user to add a personal comment without the overhead of opening an editor and typing
text. However, the annotation is noncoded, and thus cannot be edited symbolically. These
tradeoffs are important to the scenario, but often it is enough to imply them (this is an
aspect of the roughness property discussed above).
There are times, however, when it is useful to make these relationships explicit. For
example, in another situation Harry may wish to collect and revisit the set of ®lm clips he
viewed and annotated as ªbreakthroughs in forensic engineering.º Unfortunately, his
noncoded voice annotations cannot be searched by string. Thus, this new scenario
52 J.M. Carroll / Interacting with Computers 13 (2000) 43±60

Fig. 2. A view of the multimedia education scenario analyzed as consequences associated with the orientational
video clip and the speech annotation capability.

would end in failure. To understand, address and track the variety of desirable and
undesirable consequences of the original annotation design move, the designer might
want to make explicit the relevant causal relationship in the scenario. Doing so provides
yet another view of the envisioned situation as shown in Fig. 2.
Scenarios can help designers articulate consequences of differing types. For example,
the data structures for the user's workbook might differentiate between annotated and non-
annotated items, allowing annotated items to be retrieved and browsed as a subset. This
would not allow Harry to directly retrieve the set of items with a particular annotation, but
it would still simplify the search. Alternatively, the search problem might be addressed
directly by speech recognition or audio matching, or by including the option of text
annotation. Each of these alternatives would entrain different elaborations for both the
annotation scenario in Fig. 1 and the search scenario discussed above. These elaborations
could then be explored for further consequences and interdependencies.
Thus, one would want to pursue the usability consequences of the various elaborations,
for example, the frustrations of a 90% recognition accuracy. One would want to investi-
gate tradeoffs and interdependencies among consequences of different types: The speech
recognition capability might resolve the search scenario, but it might entail prohibitive
costs in memory and processing. Including an option of text annotation might change the
annotation task in a subtle way, the user would be aware when creating the annotations that
they will later be retrieval keys. Providing a ®nely articulated data structure for the user's
workbook enables ¯exible organization and retrieval, but at the cost of complexity in the
underlying database.
Using scenarios in this way makes them an extremely powerful design representation.
They allow the designer the ¯exibility to develop some use scenarios in great detail, for
example the ones that describe the core application functionality or the critical usability
challenges, while merely sketching other less problematic scenarios. At the same time,
they allow the designer to switch among scenario perspectives and to directly integrate, for
example, usability views with system and software views. Such a ¯exible and integrative
design object is precisely what designers need to manage the nexus of consequences
entrained by their design moves.
 J.M. Carroll / Interacting with Computers 13 (2000) 43±60 53

9. Challenge: technical knowledge lags technical design

SchoÈn (1983, pp. 42±44) emphasized the ªdilemmaº of rigor or relevance throughout
the professions. He describes the ªhigh, hard groundº where practitioners can make use of
systematic method and scienti®c theories, but can only address problems of relatively
limited importance to clients or to society. He contrasts this to the ªswampy lowland [of]
situations¼incapable of technical solutionº (p. 42); here the practitioner can confront the
greatest human concerns, but only by compromising on technical rigor. The design and
development of technology aspires to occupy the high, hard ground, to apply the current
state of technical knowledge systematically, to reduce dif®cult problems to mere corol-
lariesÐand of course to bask in the con®dence that deductive science confers both on its
practitioners and the recipients of their efforts: science is certainty. But at the same time,
technology design and development is inevitably driven to pursue novelty and innovation,
and thereby to slip continuously into the swampy unknown regions.
SchoÈn (1983, pp. 16, 42±44) discusses the ®eld of operations research as an example.
During World War II, operations research developed from the successful application of
mathematics to modeling battle situations like submarine search. After the war, the ®eld
expanded its interests to management problems in general, and successfully produced
effective formal models for relatively bounded problems like managing capital equipment
replacement or investment portfolios. However, it generally failed in complex, less well-
de®ned areas like business management, housing policy, or criminal justice. Remarkably,
the response of the ®eld to this was largely to focus attention on the relatively simple
problems that suited the mathematical models.
Russell Ackoff, one of the founders of operations, lamented the inability and disinterest
of the ®eld in addressing the ªturbulent environmentsº of the real world (Ackoff, 1979b,
p. 94):
managers are not confronted with problems that are independent of each other, but
with dynamic situations that consist of complex systems of changing problems that
interact with each other. I call such situations messes. Problems are abstractions
extracted from messes by analysis; they are to messes as atoms are to tables and
charts¼Managers do not solve problems: they manage messes (Ackoff, 1979b,
pp. 99±100).
This quote evokes the story in which a man loses his car keys one night, but decides to
search for them not where they were dropped, but under a nearby streetlight because the
light is so much better. The man will not ®nd his keys, and Ackoff analogously worried
that solving operations research problems will not lead to ªdesigning a desirable future and
inventing ways of bringing it about.º (Ackoff, 1979b, p. 100).
Our own experience in instructional design illustrates this pattern very well. Modern
instructional design blossomed in the 1960s and subsequently in response to rapidly
expanding needs for technical training (Schriver, 1997). The basic model that emerged
and guided this work was based on hierarchical task analysis: the overall instructional
objectives were successively decomposed into enabling objectives, objectives that enabled
the enabling objectives and so on; at the lowest level, these subskills were described,
drilled and tested (Gagne and Briggs, 1979). The vision of this ªsystems approachº model
54 J.M. Carroll / Interacting with Computers 13 (2000) 43±60

is that the instructional designer carefully orchestrates a series of well-controlled instruc-

tional events that build up the learning hierarchy within the student's mind.
At the time this model was widely adopted, three decades of research on learning and
education had already made it clear that no one ever learns this way. People can learn in
spite of such an approach, but only because they are so adaptable. The whole enterprise
seems to have been con®gured to ignore the most dif®cult issues of learning and education
and to reliably produce mediocre results. In the 1970s and early 1980s, this approach was
pervasive in the design of instruction for computer systems and applications, and
was found to be particularly ineffective for non-programmers. Unfortunately, non-
programmers were the most rapidly growing segment of computer users during those
years. It turned out to be possible to design effective instruction for these users, by study-
ing and supporting actual situations of learning, instead of merely asserting the problem
and the solution in a vacuum of systematic decomposition (Carroll, 1990).
The critical elements for an effective learning experience are activity and discovery that
can be recognized as meaningful by the learner and as supporting current personal inter-
ests, needs and objectives of the learner. People want to learn in a realistic context; they
want to be able to use what they already know to critically evaluate new knowledge and
skills they encounter. These themes are not new discoveries, they had been widely devel-
oped by psychologists like Jerome Bruner, John Dewey and Jean Piaget and were well-
known in 1970. But they do not admit of simple assembly-line instructional designs.
Taking them seriously makes instructional design a very open-ended process; it guarantees
little about either the types of design analysis that will be required in a given case or the
types of instructional designs that will emerge as appropriate and effective in a given case.

10. Scenarios can be abstracted and categorized

Though technical design cannot escape SchoÈn's swamp, designers need to extract
lessons from their experience to guide their work and improve their practice. If technical
knowledge lags design, then designers themselves must formulate what they learnÐ
perhaps becoming creators of technical knowledge more than consumers of it. The Catgut
Society is a beacon for this: at the start of their project there was little relevant technical
knowledge; physicists had a good understanding of vibrating strings, but no signi®cant
understanding of vibrating systems composed of strings vibrating across air cavities parti-
tioned by wooden ribs and bounded by irregularly shaped wooden boxes. The Catgut work
in the end has substantially advanced technical knowledge of complex acoustical systems:
the design work created an ªislandº of hard ground in the swamp of real problems.
Scenarios keep the designer of computer systems and applications in the swamp, but by
their very nature also provide scaffolding to get a view of the design situation from a bit
higher up. The roughness of scenarios is, after all, abstractness: to the extent that a scenario
is rough, it is a description of a space of particular user interactions. This is analogous to
the way in which a jazz score is a rough description of the many ways in which that piece
of music might be performed. Documenting and explaining a scenario provides an account
of a class of possible situations; this creates a generalized design object that can be
immediately employed in further design work by being elaborated in various ways. For
 J.M. Carroll / Interacting with Computers 13 (2000) 43±60 55

example, the scenario in Fig. 1 could guide the design of a multitude of information
systems.
But this is only the most primitive technique for developing design knowledge with
scenarios. Scenarios exemplify particular themes and concerns in work and activity in
work and activity situations. Earlier we discussed two scenarios for a multimedia educa-
tion system. In one (Fig. 1), the user is at ®rst opportunistically exploring an information
structure, but eventually adopts a particular interest that guides his exploration. In the
other, the user wishes to search and organize information that has previously been
browsed.
Described at this level of generality, these are not scenarios unique to multimedia
education system, or even to computers. They are general patterns for how people work
with information. Therefore, it is likely that some of the lessons learned in managing the
ªopportunistic explorationº pattern or the ªsearching under a descriptionº pattern in the
design of any given situation might be applicable in the subsequent design of other
situations. Such a taxonomy of scenarios provides a framework for developing technical
design knowledge.
Scenarios can also be classi®ed in terms of the causal relations they comprise. In Fig. 1,
for example, providing speech annotation simpli®es the actions needed to personalize a
piece of information. In this causal relation, the consequence is the simpli®cation of
organizing and categorizingÐa general desideratum in designing interactive systems.
Generalizing the relation in this way allows the feature associated with the consequence
(in this example, speech annotation) to be understood as a potential means for that conse-
quence, and employed to that end in other design contexts. There is of course no guarantee
that the generalization is correct, that can only be settled by trying to use it and succeeding
or failing. The point is that such candidate generalizations can be developed from scenario
descriptions.
The generalization of the causal relations comprising scenarios can also be carried out
across features: Speech annotation of data helps the user create a personalized view. But
this relation holds independent of whether the data is annotated by speech, by text or by
handwriting. Understanding the relation more generally allows designers to consider any
medium for annotation as a potential means of facilitating a personalized data view.
Scenarios can also be taken as exemplars of model scenarios; for example, Fig. 1
illustrates a model of opportunistic control. Harry pursues the link from bridges to picco-
los, because that is the aspect of the information that interests him. The system was
designed to support this style of use; to that extent it embodies a model of opportunistic
control. Other models are possible of course; many instructional systems would require a
student to complete the current module before allowing a branch to related material in
some other module. Seeing the scenario in Fig. 1 as an opportunistic control scenario
allows the designer to bene®t from prior knowledge pertaining to this model and to
contribute further design knowledge of the model based on the current project.
Designers are not just making things; they are making sense. Particularly in technical
design it is not enough for them to master a craft practice, for there is no stable craft
practice in design domains like computer systems and applications. The turbulence of
technology development leave little unchanged, but particularly in the short run, leaves
little resolved. We may expect that conventional science will eventually systematize
56 J.M. Carroll / Interacting with Computers 13 (2000) 43±60

technical knowledge in new domains, but we also know that it typically will do so after the
wave of technological innovation has swept onwards. The challenge for designers is to
learn as they go.

11. Challenge: external factors constrain design

Designers must have constraints; there are just too many things that might be designed.
Requirements, if they can be identi®ed, are clearly the best source of constraints because
they indicate what sort of design work is needed. But there are many other sources of
constraints. The current state of technology development makes some solutions
impossible and others irresistible: On the one hand, designers cannot use technology
that does not yet exist, though their work often drives technology development toward
possibilities that are nearly within reach. On the other hand, designers, like everyone else,
are caught up in a technological zeitgeist that biases them toward making use of the latest
gadgets and gizmos. In addition, designers are often biased toward deploying technologies
they have used before, even when they are aware of limitations in these technologies.
Earlier we referred to our work in instructional design. In this domain, we found that
many designers continued to use the systems approach even though they knew that it was
inappropriate for their users. They did this because the approach ensured a uniformly
structured, comprehensive result that other instructional designers could recognize as
well executed. This aligned the designer with others employing the same model, creating
a professional community. The many dif®culties the approach created for learners were
viewed as merely part of the research agenda, although in fact few of these problems
received adequate attention.
As Churchman (1970) observes, many systems are designed to serve the ªwrongº client,
for example, hospitals are designed to serve doctors, not patients. In time, of course, a
system designed to serve the wrong client will poorly serve all clients. This is similar to the
confusion of ªcustomersº and ªusersº in identifying design requirements. However, by far
the worst such confusion is entrained by solving tidy problems instead of addressing the
real problem. In the systems approach to instructional design, the mistaken client is the
instructional designer, or perhaps the instructional design ®rm; the underlying intent is to
streamline the production of instruction, not to improve its utility.
Many constraints in design originate in the organizational structures within which
design work is embedded. Only certain types of assumptions and arguments are acceptable
in the business cases that underwrite design projects. For example, well into the 1980s it
was widely believed that executives would never use a keyboard. This belief was based on
a marketing stereotype of executive disdain for clerical tasks, yet it constrained many
technical strategies for the development of computer hardware and software in that period.
Ultimately, appropriate hardware and software were designed for executives, as one can
easily verify in the number of notebook computers in use in the ®rst class cabin of an
airplane. Apparently, the issue had nothing to do with typing per se, but with other aspects
of use situations.
Power structures and stereotypes play assorted other roles. Design projects are often
chartered with a priori commitments to follow a strict decompositional approach: which
 J.M. Carroll / Interacting with Computers 13 (2000) 43±60 57

makes them easy to manage, but unlikely to succeed with respect to serving the needs of
users and clients. In such a context, designers must struggle not only with technical
challenges but with an impossible methodology. Schedules and resources are often
assigned in ways that create on-going con¯icts between system designers and usability
engineers: the usability engineers need to evaluate scenarios and prototypes at every stage
of system development, but if schedules and resources do not provide for this, the usability
work can con¯ict with development work.
In all these examples of external constraints, the designers can become distracted by
ancillary or even perverse factors and lose sight of what is essential in the design project,
namely, the needs and concerns of users. The designer can become ªunsituatedº with
respect to the real design situation, which is not the marketing manager's projections, or
the instructional designer's list of steps, or the software engineer's system decomposition.
The real design situation is the situation that will be experienced by the user, and designers
need to stay focused on that.

12. Scenarios promote work-orientation

Scenarios are work-orientated design objects. They describe systems in terms of the
work that users will try to do when they use those systems. A design process in which
scenarios are employed as a central representation will ipso facto remain focused on the
needs and concerns of users (Carroll and Rosson, 1990).
One can increase the effectiveness of scenarios of use as work-oriented design objects
by couching them at an appropriate level and by directly involving users in creating and
using them. A person's experience of working with a system revolves around understand-
ing and developing task goals, responding opportunistically to intriguing system events,
and planning, carrying out and evaluating courses of action. People do not focus on the
very low-level enabling actions and events that comprise these experiences, such as key-
presses, beeps, mouse gestures and clicks, display updates and font size. Quite often
people are only aware that they are seeing, hearing and doing these things when something
goes amiss. A heuristic for writing scenarios focused on the client's needs and concerns is
to initially couch them at the ªbasic taskº levelÐthe level at which people experience
their own activity (Carroll and Rosson, 1992).
Ackoff (1979a,b) argued that the indeterminacy of design situations made it imperative
that all stakeholders participate directly. Assuming that stakeholders can represent their
own interests, this proposal pretty much guarantees an adequate focus on client's needs
and concerns. However, it may not be the case that all stakeholders can represent their
interests: they will be trying to evaluate their needs as transformed by new technologies
that they may not understand. Moreover, it is not always feasible to include all stake-
holders; in the design of a new spreadsheet application for personal computers there might
be several million stakeholders.
Ackoff's ideas have been re®ned through various developments in participatory design
over the past two decades. One technique is to include representatives clients on the design
team, instead of imagining that every client can participate directly. Of course client
representative on participatory design teams are not representative clients; by de®nition
58 J.M. Carroll / Interacting with Computers 13 (2000) 43±60

Fig. 3. Challenges and approaches in scenario-based design.

only clients not on the design team can be truly representative. However, given this
dilemma, ªrepresentativeº sampling may be a reasonable heuristic. Clients can also be
helped to better represent their interests in design collaborations by taking part in facili-
tated demonstrations of possible scenarios of use (Kyng, 1995; Muller et al., 1995).

13. Some ®nal words

Our objective in this paper was to motivate and preview a framework for managing
design that accommodates the nature of design problem solving as it occurs in the context
of technology development. Our approach tries to facilitate ¯exible design actions
informed by re¯ection on multiple levels and from multiple perspective, including direct
collaboration among team members. We argue that making scenarios of use a focal design
object serves this variety of purposes.
Fig. 3 summarizes the ®ve issues and corresponding approaches we discussed. The key
issue is encouraging, supporting and productively directing re¯ection that is closely and
effectively integrated with action in the design process. Designers want to re¯ect, but they
know from experience that it is impossible to fathom all the consequences and inter-
dependencies. At the same time, they know that can act and immediately see progress
toward a solution, or at least feel that they have eliminated some possibilities. Faced with
this choice, it is always more attractive in the short-term to act. In our work on instruc-
tional design, we noticed an analogous con¯ict in the activities of learners that we called
the ªproduction paradoxº (Carroll, 1990): People know that they must learn new concepts
and skills in order to be able to do new sorts of things, however, they also know that by just
 J.M. Carroll / Interacting with Computers 13 (2000) 43±60 59

trying things out they can see and feel progress, learning as they accomplish something
meaningful.
Scenarios of use help designers manage the production paradox. Creating and elaborat-
ing scenarios is concrete design work; the designer sees and feels progress toward a design
result. At the same time, scenarios are concrete hypotheses about what the people using the
design result will do, think and experience. Thus, in SchoÈn's (1983) terminology, scenar-
ios evoke re¯ection-in-action. SchoÈn stressed the importance for designers to experience
the ªfelt-pathº of the people interacting with their designs. A scenario guarantees this
experience: it presents a potential felt-path to the designer; it is medium through which
designers can envision and explore alternative felt-paths.
Scenarios evoke effective re¯ection in a way that addresses some of the most dif®cult
properties of design. The ¯uidity of design situations demands that solutions be provi-
sional, that commitments be tentative; yet if every design decision is suspended, the result
will be a design space, not a design. A scenario is a concrete design proposal that a
designer can evaluate and develop, but is also rough in that it can be easily altered and
allows many details to be deferred.
The interconnectedness of design decisions, and the variety and extent of any given
decision's consequences, requires designers to consider their decisions from many differ-
ent perspectives: software architecture, marketing, ease of learning, production cost,
usability and so forth. It requires them to consider their decisions at many different levels
of detail in the proposed solutions. Scenarios of use serve as a concrete context for
developing and integrating these different perspectives and levels.
Technical innovation is driven to those regions where technical knowledge is thin.
Designers often have little more than craft practices to guide them in these regions. But
if we see design as inherently a process of inquiry, this lag between codi®ed knowledge
and practice is transformed into a signi®cant opportunity. Design can be a paradigm for
creating and cumulating technical knowledge. Scenarios provide a broad rubric for orga-
nizing and generalizing knowledge attained in design contexts. They can be abstracted and
categorized in various ways, and then employed in new design problems.
Like every human activity, design work occurs in many overlapping social contexts:
technical societies, corporations, states of technology development, industries and so
forth. Each context introduces constraints on possible methods and solutions. Sometimes
these are constructive and appropriate, often they are neither. The cacophony of these
social contexts and constraints can leave designers unsituated with respect to their primary
source of constraintÐthe needs and concerns of the people who will use the system being
designed. Scenarios help keep the whole enterprise focused on past and future situations of
use, they help keep designers focused on what matters most.

References

Ackoff, R.L., 1979a. Resurrecting the future of operations research. Journal of the Operations Research Society
30 (3), 189±199.
Ackoff, R.L., 1979b. The future of operations research is past. Journal of the Operations Research Society 30 (2),
93±104.
60 J.M. Carroll / Interacting with Computers 13 (2000) 43±60

Brooks, F., 1995. The Mythical Man-Month: Essays on Software Engineering, . Anniversary EditionAddison-
Wesley, Reading, MA (originally 1975).
Carroll, J.M., Rosson, M.B., 1985. Usability speci®cations as a tool in iterative development. In: Hartson, H.R.
(Ed.). Advances in Human±Computer Interaction, Ablex, Norwood, NJ.
Carroll, J.M., Rosson, M.B., 1990. Human±computer interaction scenarios as a design representation. In:
Proceedings of the 23rd Annual Hawaii International Conference on Systems Sciences (Kailua-Kona, HI,
2±5 January, IEEE Computer Society Press, Los Alamitos, CA, pp. 555±561.
Carroll, J.M., Rosson, M.B., 1991. Deliberated evolution: stalking the view matcher in design space. Human±
Computer Interaction 6, 281±318.
Carroll, J.M., Rosson, M.B., 1992. Getting around the task-artifact cycle: how to make claims and design by
scenario. ACM Transactions on Information Systems 10, 181±212.
Carroll, J.M., 1990. The Nurnberg Funnel: Designing Minimalist Instruction for Practical Computer Skill, MIT
Press, Cambridge, MA.
Carroll, J.M., 1994. Making use a design representation. Communications of the ACM 37 (12), 29±35.
Carroll, J.M. (Ed.), 1995. Scenario-based Design: Envisioning Work and Technology in System Development
Wiley, New York.
Checkland, P.B., 1981. Systems Thinking, Systems Practice, Wiley, New York.
Chin, G., Rosson, M.B., Carroll, J.M., 1997. Participatory analysis: shared development of requirements from
scenarios. In: Pemberton, S. (Ed.). Proceedings of CHI'97: Human Factors in Computing Systems (Atlanta,
22±27 March), ACM Press/Addition-Wesley, New York, pp. 162±169.
Churchman, W., 1970. Operations research as a profession. Management Science 17 (2), 37±53.
Duval, S., Wicklund, R.A., 1972. A Theory of Objective Self-Awareness, Academic Press, New York.
Erikson, E.H., 1980. Identity and the Life Cycle, Norton, New York.
Festinger, L., Riecken, H.W., Schachter, S., 1956. When Prophecy Fails, University of Minnesota Press, Minnea-
polis, MN.
Festinger, L., 1957. A Theory of Cognitive Dissonance, Harper and Row, New York.
Gagne, R.M., Briggs, L.J., 1979. Principles of instructional design, Holt, Rinehart and Winston, New York.
Hutchins, C.M., 1981. The acoustics of violin plates. Scienti®c American 245 (4), 170±186.
Jacobson, I., 1995. The use-case construct in object-oriented software engineering. In: Carroll, J.M. (Ed.). Scenario-
Based Design: Envisioning Work and Technology in System Development, Wiley, New York, pp. 309±336.
Karat, J., Bennett, J.B., 1991. Using scenarios in design meetingsÐa case study example. In: Karat, J. (Ed.).
Taking Design Seriously: Practical Techniques for Human±Computer Interaction Design, Academic Press,
Boston, MA, pp. 63±94.
Kyng, M., 1995. Creating context for design. In: Carroll, J.M. (Ed.). Scenario-Based Design: Envisioning Work
and Technology in System Development, Wiley, New York, pp. 85±107.
McLuhan, M., 1994. Understanding Media: The Extensions of Man, MIT Press, Cambridge, MA (original
edition, 1964).
Muller, M.J., Tudor, L.G., Wildman, D.M., White, E.A., Root, R.A., Dayton, T., Carr, R., Diekmann, B., Dystra-
Erickson, E., 1995. Bifocal tools for scenarios and representations in participatory activities with users. In:
Carroll, J.M. (Ed.). Scenario-Based Design: Envisioning Work and Technology in System Development,
Wiley, New York, pp. 135±163.
Nisbett, R.E., Wilson, T.D., 1997. Telling more than we can know: verbal reports on mental process. Psycho-
logical Review 84, 231±259.
Potts, C., 1995. Using schematic scenarios to understand user needs. In: DIS'95: ACM Symposium on Designing
Interactive Systems, Ann Arbor, MI,. ACM Press, New York, pp. 247±256.
SchoÈn, D.A., 1967. Technology and Change: The New Heraclitus, Pergamon Press, New York.
SchoÈn, D.A., 1983. The Re¯ective Practitioner: How Professionals Think in Action, Basic Books, New York.
SchoÈn, D.A., 1987. Educating the Re¯ective Practitioner, Jossey-Bass, San Francisco, CA.
Schriver, K., 1997. Dynamics in Document Design, Wiley, New York.
Scriven, M., 1967. The methodology of evaluation. In: Tyler, R., Gagne, R., Scriven, M. (Eds.). Perspectives of
Curriculum Evaluation, Rand McNally, Chicago, IL, pp. 39±83.
Wirfs-Brock, R., 1995. Designing objects and their interactions: a brief look at responsibility-driven design. In:
Carroll, J.M. (Ed.). Scenario-Based Design: Envisioning Work and Technology in System Development,
Wiley, New York, pp. 337±360.