SAAM: A Method for Analyzing the Properties of Software Architectures

Rick Kazman Len Bass, Gregory Abowd Mike Webb

Department of Computer Science Software Engineering Institute Texas Instruments Inc.

University of Waterloo Carnegie Mellon University Dallas, TX, U.S.A. 75265
Waterloo, ON Canada N2L 3G1 Pittsburgh, PA, U.S.A. 15213 {smw}@sei.cmu.edu
rnkazman@watcgl.uwaterloo.ca {ljb,gda}@sei.cmu.edu

Abstract in a software architecture. Examples of such claims are:

We have developed … user interface components
While software architecture has become an increasingly
that can be reconfigured with minimal effort. [20]
important research topic in recent years, insufficient atten-
tion has been paid to methods for evaluation of these archi- Serpent … encourages the separation of software
tectures. Evaluating architectures is difficult for two main systems into user interface and “core” application
reasons. First, there is no common language used to de- portions, a separation which will decrease the cost
scribe different architectures. Second, there is no clear way of subsequent modifications to the system. [22]
of understanding an architecture with respect to an organi-
This Nephew UIMS/Application interface is bet-
zation’s life cycle concerns—software quality concerns
ter that [sic] traditional UIMS/Application inter-
such as maintainability, portability, modularity, reusability,
faces from the modularity and code reusability
and so forth. This paper addresses these shortcomings by
point of views. [26]
describing three perspectives by which we can understand
the description of a software architecture and then propos- The difficulty in assessing the validity of these claims
ing a five-step method for analyzing software architectures arises for two reasons. First, the various architectural
called SAAM (Software Architecture Analysis Method). We descriptions do not use a common vocabulary. When peo-
illustrate the method by analyzing three separate user in- ple develop new architectures, they typically develop new
terface architectures with respect to the quality of modifi- terms to describe them, or use old terms in a new way. It
ability. is, therefore, difficult to compare these new architectures
with existing ones—there is no level playing field. Second,
1 Introduction it is often difficult to link architectural abstractions with
system development concerns. Developers tend to concen-
Software architecture is a topic of growing concern trate on the functional features of their architectures, and
within both the academic and industrial communities. seldom address the ways in which their architectures sup-
Despite the popularity of this topic, little attention focuses port quality concerns within the system development life
on methods for analyzing a software architecture to show cycle.
that it satisfies certain properties. Software architectures The main goal of this paper, therefore, is to establish a
are primarily motivated by software engineering consider- method for describing and analyzing software architec-
ations, or software quality factors, such as maintainability, tures, called the Software Architecture Analysis Method
portability, modularity and reusability. In this paper, we (SAAM). Before analyzing an architecture, we must first
address architectural description and a method of analysis have a way to provide a clear description of it which
with respect to software quality. The method is demon- exposes its main features. We define three perspectives for
strated by means of a case study examining architectures understanding and describing architectures—functionality,
for developing the user interface portion of interactive sys- structure and allocation. We also provide a simple lan-
tems. guage for describing the structural perspective. This lan-
Over the past decade, changes in the software architec- g u a g e i s i m p o r t a n t b e c a u s e i t p e r m i t s d i ff e r e n t
ture characterize the advances in support environments for architectures to be described consistently, forcing a com-
the development of user interfaces. However, it is often mon level of understanding for comparing different archi-
difficult to assess a developer's claims of qualities inherent tectures.
 After we have a way to describe an architecture, we can significant architectural case studies can help to define the
apply the SAAM to analyze it. The main activities field of software architecture, and this is why we chose to
involved in the SAAM are enumerated below. ground our general analysis method in a concrete domain.
1. Characterize a canonical functional partition- Although we have extensive experience in the field of
ing for the domain. Human-Computer Interaction and user interface develop-
ment and analysis, our main interest in writing this paper
2. Map the functional partitioning onto the ar- was not so much for the HCI community but for the gen-
chitecture’s structural decomposition. eral software engineering community interested in under-
3. Choose a set of quality attributes with which standing how clear and coherent software architectures
to assess the architecture. can aid in analysis. Having said that, there is a clear contri-
4. Choose a set of concrete tasks which test the bution in our work for the user interface development
desired quality attributes. community. There is an abundance of literature which
takes a taxonomic approach to classifying the many user
5. Evaluate the degree to which each architec- interface development tools ([4], [11]). Rather than simply
ture provides support for each task. classify various user interface development tools, we pro-
It is a secondary goal of this paper is to demonstrate the vide a means of evaluating their suitability for producing
utility of this general architectural analysis method for the modifiable interactive systems.
evaluation of user interface architectures.
1.2 Overview
1.1 Background
In Section 2, we define the three distinct perspectives,
Our approach to the analysis of software at the architec- functionality, structure and allocation, used to describe a
tural level reflects a growing trend in software engineer- software architecture and we introduce the simple lexicon
ing. That trend emphasizes a better understanding of for the structural perspective. The remainder of the paper
general architectural concepts as a foundation for the demonstrates the application of the SAAM to the case
proof that a system meets more than just functional study of user interface software. In Section 3, we define a
requirements [12]. We agree with Perry and Wolf [18] that canonical functional partitioning for user interface soft-
an architectural description provides multiple perspectives ware and examine how that functionality is allocated for
of different information, though their perspectives would three distinct user interface development environments. In
rightly be considered as perspectives on structure in our Section 4, we discuss one quality attribute, modifiability,
terms. They also promote the use of a small lexicon of ele- important for user interface software. We then identify two
ments to portray structure as do Abowd, Allen and Garlan benchmark modification tasks which are used in Section 5
[2] and Garlan and Shaw [8]. Our small lexicon, however, to evaluate the architectures assumed by the different
resulted more because it was sufficient for the task at hand development environments. In Section 6, we summarize
and not out of any conviction that a small vocabulary was the results of this paper and point to further work on this
essential. topic.
We are not original in our desire to predict the quality
of a software product from a higher level design descrip- 2 Perspectives on architectures
tion. For example, Parnas [17] motivated the use of modu-
larization or information hiding as a means of high-level The architectural design of a software system can be
system decomposition to improve flexibility and compre- described from (at least) three perspectives—the func-
hensibility. Stevens, Myers and Constantine [25] intro- tional partitioning of its domain of interest, its structure,
duced the notions of module coupling and cohesion to and the allocation of domain function to that structure.
evaluate alternatives for program decomposition. The soft- These perspectives reflect a consensus within the software
ware metrics community has used these notions to define engineering community, as witnessed by the literature ([8],
predictive measures of software quality (see, for example, [13], [21], [24]). We will discuss each of these perspec-
the work by Briand, Morasca and Basili [6]). Our analysis tives in turn.
method appeals to a more abstract evaluation of how the
architecture fulfills the domain functionality and other 2.1 Functionality
nonfunctional qualities. We do not present any metrics for A system’s functionality is what the system does. It
predictive evaluation, but instead present an example- may be a single function or a bundle of related functions
based method for performing a more qualitative evalua- which together describe the system’s overall behavior. For
tion. large systems, a partitioning divides the behavior into a
Garlan and Shaw demonstrated how the examination of
collection of functions which together comprise the sys- boxes represent computational components which only
tem’s function but which are individually simple to exist within a process or within another computational
describe or otherwise conceptualize. component (e.g. procedures, modules). Square cornered
Typically, a single system’s functionality is decom- shaded boxes represent passive data repositories (typically
posed through techniques such as structured analysis [30] files). Round-cornered shaded boxes represent active data
or object oriented analysis [22], but this is not always the repositories (e.g., an active database). Solid arrows repre-
case.When discussing architectures for a collection of sys- sent data flow (uni- or bi-directional) and grey arrows rep-
tems in some common domain, such as we are doing here, resent control flow (also uni- or bi-directional). This
the functional partitioning is often the result of a domain lexicon has been defined to meet the descriptive needs of
analysis. In a mature domain (e.g., databases, user inter- this paper; it should not be construed as a complete vocab-
faces, flight simulators, and VLSI design), the partitioning ulary for all structural descriptions.
of functionality has been exhaustively studied and is typi- To understand the overall behavior of a system, we
cally well understood, widely agreed upon, and canonized would need to provide more detailed descriptions of the
in implementation-independent terms. Another common computations possible within components and the overall
name for a canonical functional partitioning is a reference coordination of a collection of components with various
architecture, but we choose not to use that name for fear of connection relationships between them. Such computa-
overloading the term architecture. tional and coordination models should be explicit in an
architectural description [1]. However, we do not include
2.2 Structure such information in our lexicon.
A system’s software structure reveals how it is con-
2.3 Allocation
structed from smaller connected pieces. The structure is
described in terms of the following parts: The allocation of function to structure identifies how
1. A collection of components which represent the domain functionality is realized in the software struc-
computational entities (e.g., a process) or per- ture. The purpose of making explicit this allocation is to
sistent data repositories (e.g., a file); understand the way in which the intended functionality is
achieved by the developed system. There are many struc-
2. A representation of the connections between tural alternatives and many possible allocations from func-
the components, that is, the communication tion into that structure. Software designers choose
and control relationships among the compo- structural alternatives on the basis of system requirements
nents. and constraints which are not directly implied by the sys-
In our analysis of architectures, we make use of a small tem’s functional description. It is the allocation of function
and simple lexicon for describing structure, as indicated in to structure which we will be examining most closely in
Figure 1. this paper, because the allocation choices most strongly
differentiate architectures within a given domain.
Components Connections
3 Description of architectures for user
Uni-/Bi-directional
Process ( ) interfaces
Data Flow
We can now describe a number of software architec-
Computational tures for user interfaces in terms of a canonical functional
Component Uni-/Bi-directional partitioning for the domain, the individual structural
( ) decomposition and the allocation from functionality to
Control Flow
that structure. Once we have these architectural descrip-
Passive Data
tions, we will have the basis for a more intellectually satis-
Repository
fying comparison of different user interface software
architectures. We begin with a description of a canonical
Active Data functional partitioning for this domain before describing
Repository three influential non-commercial user interface architec-
tures—Serpent, Chiron and Garnet.
Figure 1: Architectural notations
3.1 The Arch/Slinky metamodel
Square-cornered boxes with solid lines represent pro-
cesses, or independent threads of control. Round-cornered The Arch/Slinky metamodel of user interface architec-
tures [29] will serve as our canonical functional partition-
ing for this domain. It is an extension of the Seeheim Dialogue
model of user interface management systems [19], and is Dialogue Controller
the result of wide discussion and agreement amongst user Manager
interface researchers and developers. Arch/Slinky identi-
fies the following five basic functions of user interface Application
software:
• Functional Core (FC). This component performs the
data manipulation and other domain-oriented func- Active Database
tions. It is these functions that the user interface is
exposing to the user. This is also commonly called the
Domain Specific Component, or simply the Applica-
tion.
• Functional Core Adapter (FCA). This component Presentation
aggregates domain specific data into higher-level struc-
tures, performs semantic checks on data and triggers Figure 2: The Serpent architecture (adapted from [3])
domain-initiated dialogue tasks.
• Dialogue (D). This component mediates between the one. Presentation modules provide techniques for support-
domain specific and presentation specific portions of a ing interaction at both the logical and physical level com-
user interface, performing data mapping as necessary. pletely independent of application semantics. Different
It ensures consistency (possibly among multiple views presentation modules in a given run-time instance are pos-
of data) and controls task sequencing. sible, although once again, not typical. Given that applica-
tion and presentation modules are separate, there must be a
• Logical Interaction (LI) component. This component way to coordinate a given application component with a
provides a set of toolkit independent objects (some-
presentation component. That is the purpose of the dia-
times called virtual objects) to the dialogue compo-
logue controller. The dialogue component mediates the
nent.
user’s interaction with an application, through the control
• Physical Interaction (PI) component. This component of the presentation.
implements the physical interaction between the user All communication between Serpent components is
and the computer. It is this component which deals mediated by constraints on shared data in the database
with input and output devices. This is also commonly shown in Figure 2. This structure is implemented as an
called the Interaction Toolkit Component. active database—when values in the database change, they
In the remainder of this section, we will provide struc- are automatically communicated to any component which
tural descriptions of the Serpent, Chiron and Garnet user is registered as being interested in the data. This global
interface development environments. Each of these devel- database physically resides in the same process as the dia-
opment environments assumes a structure for application logue controller but is logically independent from all of
built within that system. We will first show that graphical the Serpent components. A dialogue manager sits within
structure assumed by the authors of the development envi- the dialogue controller process and mediates the connec-
ronment. We will then recast that structure using the lexi- tion between application and presentation. The dialogue
con defined in Section 2.2. Superimposed on that manager is further decomposed into a collection of view
structural description will be the allocation of domain controllers (not shown in Figure 2) which provide a finer
function given by the Arch/Slinky canonical functional grain of correspondence between application and presenta-
partitioning. tion objects.
Figure 3 indicates how the five basic functional roles of
3.2 Serpent a user interface architecture, as identified in section 3.1,
Serpent identifies a dialogue controller, the presentation are mapped onto Serpent’s software structure. Dotted lines
and the application as three distinct processes in its archi- are used, in this and following figures, as a visual indica-
tecture [3], as shown in Figure 2. tion of the allocation of functionality to portions of the
Application modules contain the computational seman- structure. This is done for analysis purposes only and
tics required for the application. Although there can theo- should not be interpreted as having any other structural
retically be many different applications contained within a implications.
given run-time instance of Serpent, there is typically only Several things have changed between Figure 2 and Fig-
 Serpent a glue language is used within the presentation
FCA Dialogue Manager Dialogue process to separate physical aspects of a window sys-
D Controller tem/toolkit from logical aspects. This glue, therefore,
supports the subdivision of the presentation process
into physical interaction and logical interaction func-
tions.
This last point is important to remember. We are only
concerned with architectural support in this paper. We do
not discuss other forms of support, such as language sup-
Active Database port as exemplified by the use of glue in Serpent, though
we recognize their potential importance.

3.3 Chiron
FC PI
LI The Chiron architecture [24] was built with the
expressed goals of addressing life cycle concerns of main-
Application Presentation tainability and sensitivity to environmental changes. A
Chiron architecture consists of a client and a server. The
client consists of an application, which exports a number
Figure 3: Serpent architecture (annotated)
of abstract data types (ADTs) which Chiron encapsulates
ure 3. The architecture has been redrawn using the nota- within Dispatchers. Dispatchers communicate with Art-
tion given in section 2.2, and the functional roles have ists, which maintain abstract representations of their asso-
been overlaid on the structure of Serpent, in order that the ciated ADTs in terms of an abstract depiction library
allocation of functionality to structure may be illustrated. (ADL).
There are several points of interest to note in this re- A Chiron server consists of the ADL, a virtual window
characterization of Serpent’s structure: system, and an instruction/event interpreter. The virtual
• there is no structural separation between the PI and LI window system translates abstract interface depictions into
functions in the presentation process; concrete ones. The instruction/event interpreter responds
to requests from Artists to change the abstract description
• there is no structural separation between the FCA and and translates those requests into changes to the presenta-
D functions in the dialogue controller process;
tion. It also responds to events from users and translates
• except for the dialogue manager, all of Serpent’s com- those back into Artist requests.
ponents are monolithic—that is, they provide no archi- The architecture of a typical system developed under
tectural support for subdivision of functionality within Chiron is shown in Figure 4.
a component. Within the dialogue manager, a view The Chiron architecture clearly separates the applica-
controller hierarchy provides an architectural decom- tion (functional core) from the rest of the system, as would
position.There are other forms of support that allow for be expected in a system which was built with the
further subdivision of functionality. For example, in expressed goal of minimizing sensitivity to environmental

Chiron Server Chiron Client

I/E Interpreter Artist Dispatcher

Virtual Window

Artist ADT
System

Dispatcher Application
Artist ADT

Abstract Depiction Dispatcher

Artist
ADT

ADL Library

Figure 4: A typical Chiron architecture (adapted from [24])

 Chiron Server Chiron Client
PI LI D FCA
I/E Dispatcher FC
Interpreter Artist
Virtual Window

Artist ADT
System

Dispatcher Application
Artist
ADT

Abstract Depiction Dispatcher

Artist
ADT
ADL Library

Figure 5: Chiron architecture (annotated)

changes. The dispatchers/ADTs together form the func- emphasis from Serpent and Chiron, which were expressly
tional core adapter. It seems clear that the Artists contain interested in maintainability and separation of concerns.
some of the dialogue, for example, maintaining a corre- The architecture of applications built with Garnet is a
spondence between objects from the application domain single process, with Common Lisp and the X11 window
and interface objects from the presentation domain. How- system as its foundations (above an assumed operating
ever, what is less clear, from the architectural description, system). The Garnet toolkit is built on top of this founda-
is where the “state” of the dialogue lives. For example, tion. At the very top are high-level Garnet tools (which we
where does one put the information that the “paste” option call Garnet applications). Garnet is thus commonly pre-
in an edit menu should be grayed out unless something has sented as a layered system, as shown in Figure 7.
previously been cut or copied? Another type of dialogue
issue is maintaining relationships among the interface Garnet Applications
objects. For example, when a user selects the “Save As”
option in a file menu, something in the dialogue must Widget set
cause a file selection box to be created. Once again, the
location of these sorts of dialogue issues is not clear from Interactors Opal Graphics
Chiron’s architectural description. These dependencies
might exist in the Artists or the ADTs. [28] Constraint system
The physical interaction component appears to be KR object system
located in Chiron’s Virtual Window System component,
and the logical interaction component is encapsulated X11 Common Lisp
within the ADL. As a result of this characterization, we
can provisionally annotate the Chiron architecture as Operating System
shown in Figure 5.
This re-characterization indicates the logical division of Figure 6: The Garnet architecture (adapted from [15])
functionality in Chiron, according to the Arch/Slinky
However, Garnet applications are not strictly layered.
meta-model. Note that by re-characterizing Chiron’s archi-
In a strictly layered system, the nth layer hides all details
tecture in this way, we can now begin to understand its
of the layers n-1 and lower, and is available only to layer
relationships to other architectures, such as Serpent’s. This
n+1. In Garnet applications, Common Lisp, the KR object
task is considerably more difficult when trying to compare system and the Constraint system are services used
architectures based upon their own representations and directly by all layers. There are no prototypical patterns of
claims. What we have done is to develop a common lan- usages for the object services. Consequently, we choose
guage for making architectural comparisons. not to reveal it as an architectural features of Garnet in
3.4 Garnet Figure 7, just as we have chosen not to represent the oper-
ating system in our architectural description.
The emphasis for systems developed under Garnet [15] The Interactors and Opal Graphics together strictly hide
is on control of the runtime behavior of interaction objects all of the X11 calls. Thus, Garnet widgets and applications
and the visual aspects of the interface. This is a different do not have any window manager calls in them, only calls
to the interactors, object-oriented graphics, constraints and In the Evolutionary Development Life Cycle project at
object system. [14] Carnegie Mellon University, we are concerned with the
Given this analysis, we can re-draw Garnet’s architec- architecture of systems which have long projected life-
ture as follows: times—20-30 years. This is an increasingly important seg-
ment of the software market. According to a study by
X11 library Green and Selby, the average age of all software is increas-
PI ing [9]. In systems such as these, modifiability is of para-
mount importance. In the domain of user interface
development, modifiability is frequently cited as a moti-
vating factor for a particular architecture, since some
Interactors Opal Graphics changes are very common. For this reason, we will evalu-
ate user interface architectures with respect to the property
Object/Constraint

of modifiability.
Widget set Modifiability by itself is too abstract to be useful for
LI
System

analysis. In order to understand how to design for modifi-

ability, it is necessary to better understand the ramifica-
tions of this attribute. We look at what sorts of
D modifications to a software system in our domain are pos-
FCA sible. When that is accomplished, we then decide what
Garnet Applications sorts of modifications are likely, or representative of the
FC
domain. We evaluate each candidate architecture accord-
Figure 7: The Garnet architecture (annotated) ing to how well it supports the set of likely modifications,
or benchmark tasks. If the application domain is well
The X11 window system fills the physical interaction understood, this set of benchmark modifications can often
function. The Widgets, Interactors and Opal graphics col- be given a sample distribution, for the purposes of ranking
lectively provide the logical interaction function, making the individual evaluations.
use of the object and constraint services. Finally, applica- This process is akin to choosing a set of benchmark
tions in Garnet fulfill the functions of dialogue, functional tasks to evaluate the performance characteristics of a piece
core adaptor and functional core, again utilizing object and of software or hardware. These modifications are intended
constraint services. to approximately model the type and distribution of tasks
This characterization of Garnet illustrates several which are typical of an organization’s software develop-
points. Garnet applications subdivide logical interaction ment life cycle. This is why a set of example modifications
into three distinct parts. Dialogue, functional core adaptor is required.
and functional core are separated from the rest of the sys- Oskarsson gives an informal characterization of classes
tem but are not further subdivided within a Garnet applica- of modifications in [16]. Drawing upon his work, we have
tion. enumerated the following classes of modifications:
How an architecture partitions its functionality indi- • Extension of capabilities: adding new functionality,
cates the emphases of the architecture. The partitioning is enhancing existing functionality;
the architectural manifestation of the stated goals of Gar-
net: control of the runtime behavior of interaction objects • Deletion of unwanted capabilities: e.g. to streamline or
and the visual aspects of the interface (physical interaction simplify the functionality of an existing application;
and logical interaction components) are paramount. • Adaptation to new operating environments: e.g. proces-
sor hardware, I/O devices, logical devices
4 Analyzing architectural qualities • Restructuring: e.g. rationalizing system services, mod-
ularizing, optimizing, creating reusable components.
One of our key arguments is that software architectures
are neither intrinsically good nor intrinsically bad; they In the user interface domain, two classes of modifica-
can only be evaluated with respect to the needs and goals tions prove most common—adaptation to new operating
of the organizations which use them. It is important to environments and extensions of capabilities. The other
understand the life cycle concerns for a particular organi- classes of modifications—restructuring and deletion of
zation, providing a list of quality factors which can be unwanted capabilities—do not constitute a significant per-
used to discriminate between good and bad architectural centage of the modifications for user interfaces. We will
alternatives for that organization. not include them in our set of modifications.
 Adaptation to new operating environments is a common with OpenLook. What would have to change is both the
modification activity which user interface architectures presentation process and the dialogue.
must support. For example, this paper is being composed There are two ways this change can be accommodated
using a desktop publishing system which is available on in Serpent. If the intended use of the new toolkit is simply
Unix-based workstations running the X window system, to reproduce the look and feel of the old toolkit, then the
Macintoshes running the Macintosh toolkit, and IBM- only modification necessary is to change Serpent’s glue
compatible PCs, running MS-Windows. In each of these code in the presentation process. As discussed in Section
cases, the underlying functionality of the system is identi- 3.2, the glue code provides the logical interaction function
cal. What does change are the devices, both logical and for Serpent, but is not isolated in its architectural descrip-
physical, used to interact with the program. tion, so we conclude that Serpent provides no architectural
The first of our benchmark modification tasks, there- support for this kind of modification. If the intended use of
fore, is to change the toolkit. For example, a move from the new toolkit is to provide new interaction capability,
using Motif to Fresco would be typical of this category of then modifications to the dialogue manager would be nec-
modification. essary, in addition to new glue code.
Because user interface development is highly iterative, We can see from this analysis the purpose of the logical
extensions of capabilities (adding new features, reorganiz- interaction function in the Arch/Slinky canonical partition-
ing the appearance of the interface, reorganizing the ing. Logical interaction provides a buffer between the dia-
human-computer dialogue) occur frequently. This is par- logue and the physical interaction in order to isolate the
ticularly so during the initial development of a user inter- effect of changes between them. Architectures that isolate
face, but such modifications are also common later in the the logical interaction function provide support for this
life cycle. benchmark modification. Serpent, therefore, does not pro-
The user interface life cycle is distinguished from typi- vide architectural support for this modification.
cal software engineering life cycles because it is highly
iterative, relying more on prototyping and empirical test- 5.1.2 Adding a menu option
ing and validation. Requirements for human-computer
Serpent has isolated the dialogue controller, so it is easy
interaction are often not well understood in advance, and
to say that the second modification type—adding an
so iteration and prototyping are often the only ways in
option to a menu—will occur somewhere in that process.
which to evolve a system’s design. Thus this class of
The dialogue controller process contains two parts, the
change is the most common (and most costly overall) in dialogue manager, which governs the behavior of the dia-
the user interface life cycle. logue and the active database, which maintains the state of
We choose as our second benchmark task a modifica- the dialogue. The benchmark modification concerns a
tion to the human-computer dialogue. More specifically, change in the behavior of the dialogue, so it is isolated in
the second benchmark modification is to add a single the dialogue manager. View controllers further subdivide
option to a menu, reflecting some piece of application the dialogue manager so that it easier to isolate a specific
functionality which must be made available to a user. piece of dialogue behavior, such as a menu. Therefore, we
conclude that Serpent provides adequate architectural sup-
5 Architectural analysis for UI tools port for this modification.
Now that we have enumerated our set of benchmark 5.2 Chiron
modifications—changing the window system/toolkit and
adding a single option to a menu—we are in a position to 5.2.1 Changing the toolkit
evaluate the degree to which each of our candidate systems
Chiron goes one step beyond Serpent in providing
provides architectural support for these modifications.
architectural support for this modification. Chiron has iso-
5.1 Serpent lated the logical and physical interaction functions in sepa-
rate structures within a Chiron server. Since the Virtual
5.1.1 Changing the toolkit Window System only communicates with the Instruction/
Event Interpreter and its associated Abstract Depiction
Changing the toolkit in Serpent assumes that we have Library, it is isolated from the rest of the architecture. This
an application running under one toolkit and we want to architectural isolation means that the Virtual Window Sys-
change to another toolkit which is not currently supported tem and Instruction/Even Interpreter communicate via a
within the Serpent run-time environment. For example, well-defined interface. The existence of such an interface
this situation would occur if we had Motif running in a means that different toolkits could be inserted into the
Serpent presentation process and we want to replace it
architecture as long as they comply with the interface con- cycle requirements. SAAM permits the comparison of
ventions. architectures within the context of an organization’s par-
This strict separation of concerns aids modifiability, by ticular needs. This sort of comparison has been hitherto
localizing the effects of a change. Thus, we can conclude quite difficult.
that the Chiron architecture provides significant support The SAAM places strong demands on an organization
for this benchmark modification. to articulate those quality attributes of primary impor-
tance. It also require a selection of benchmark tasks with
5.2.2 Adding a menu option which to test those attributes. The purpose of the SAAM is
Chiron fares slightly less well with respect to the sec- not to criticize or commend particular architectures, but to
ond modification. As stated in section 3.3, the division of provide a method for determining which architecture sup-
dialogue responsibilities between Artists and ADTs is not ports an organization’s needs.
precisely specified. Hence, a modification to the dialogue We applied the method to evaluate user interface devel-
may manifest itself as a change to an Artist or an ADT, or opment environments to determine their architectural sup-
both. If the ADTs are well-structured, changes to them port for two benchmark modifiability tasks. We saw that
should be minimal, in which case a change to the dialogue the benchmark tasks could be cast in terms of the Arch/
would be isolated to the Artists. Slinky metamodel. Therefore, systems with a clear alloca-
For this particular modification, it is reasonable to tion of that functional partitioning to their structure could
assume that no change to an ADT would be necessary provide architectural support for the modifications. We
since we are assuming that we want to make available emphasize that architectural support for a quality attribute
some pre-existing application function which would is only one possible type of support. We may also have
already be defined in one ADT. The modification in this language or tool support. We recognize that an analysis of
case is restricted to an Artist which is associated to the these other types of support is an important topic for future
ADT containing the desired function. We conclude that research.
Chiron provides architectural support for this benchmark Quality attributes is a recognized driver for software
modification. architectural design, outside of the user interface domain
(e.g. [1]). Consequently, we believe SAAM will work for
5.3 Garnet other attributes. This method of evaluation shows the
degree to which an architecture was designed to support
5.3.1 Changing the toolkit selected quality attributes. We see a strong link between
quality attributes and canonical partitionings. More than
Garnet has isolated the toolkit to three components—
one canonical partitioning is possible for a given domain,
interactors, Opal graphics and widgets—and has hidden
each accommodating a different set of quality attributes
any X11 dependencies behind them. Garnet has strictly
[5]. For example, the multiagent PAC model [7] is a candi-
separated physical interaction, and so we conclude that
date canonical partitioning which emphasizes scalability
this modification is supported by the architecture.
and construction efficiency.
5.3.2 Adding a menu option Finally, what is really desirable is to have metrics for
more precisely evaluating attributes in terms of architec-
A dialogue in Garnet is monolithic, and can involve any tures, but our understanding of this topic is not yet suffi-
of the language services which Garnet provides. Thus, cient to define any measurements. For now, we simply
changing a menu in Garnet involves locating and modify- have ways to analyze, compare and rank architectures. We
ing the affected Lisp code, which may be a difficult task in believe that the work of Henry and Kafura [10] on infor-
a complex interface. The second modification is not archi- mation flow points the way to an analysis technique for
tecturally supported by Garnet. Garnet does provide tool architectures, but it must be augmented by techniques for
and language support for this class of change, but as we measuring the understandability and consistency of archi-
stated earlier, that is not the concern of this paper. tectures.

6 Conclusions and future work 7 Acknowledgments

In this paper, we have provided a architectural analysis We would very much like to thank Brian Boesch of
method (SAAM) and used it to evaluate user interfaces ARPA and Dick Martin of the School of Computer Sci-
architectures with respect to modifiability. This method is ence at CMU for their support of this research. We also
based upon a common understanding and representation thank members of the IFIP Working Group 2.7 (User
for architectures and an analysis of an organization’s life- Interface Engineering) and colleagues at the SEI, specifi-
cally Reed Little and Larry Howard, for their influence Interfaces”, IEEE Computer, 23(11): 71-85.
during the formative stages of this work. This work was [16] Oskarsson, Ö. “Mechanisms of Modifiability in Large Soft-
sponsored in part by the National Sciences and Engineer- ware Systems”, Linköping Studies in Science and Technology
ing Research Council of Canada and the U.S. Department Dissertations No. 77, 1982.
of Defense. [17] Parnas, D.L. “On the Criteria To Be Used in Decomposing
Systems into Modules”, Communications of the ACM,
15(12):1053-1058, 1972.
8 References [18] Perry, D., Wolf, A. “Foundations for the study of software
architecture”, SIGSOFT Software Engineering Notes, 17(4),
[1] Abowd, G., Bass, L., Howard, L., Northrop, L. “Structural October 1992, 40-52.
Modeling: an Application Framework and Development Process
[19] Pfaff, G. (ed.). User Interface Management Systems. New
for Flight Simulators”. Software Engineering Institute, Carnegie
York: Springer-Verlag, 1985.
Mellon University Technical Report CMU-SEI-TR-93-14. Pitts-
burgh, PA, 1993. [20] Pittman, J., Kitrick, C. “VUIMS: A Visual User Interface
Management System”, Proceedings of the ACM SIGGRAPH
[2] Abowd. G., Allen, R., Garlan, G. “Using Style to Understand
Symposium on User Interface Software and Technology, Snow-
Descriptions of Software Architectures”. Software Engineering
bird, UT, October 1990, 36-46.
Notes, 18(5):9-20, 1993. Proceedings of SIGSOFT ‘93: Sympo-
sium on the Foundations of Software Engineering. [21] Pressman, R. Software Engineering: a Practitioner’s
Approach, 3rd edition. New York: McGraw-Hill, 1992.
[3] Bass, L., Clapper, B., Hardy, E., Kazman, R., Seacord, R.
“Serpent: A User Interface Management System”. Proceedings of [22] Rumbaugh, J., Blaha, M., Premerlani, W., Eddy, F., Loren-
the Winter 1990 USENIX Conference, Berkeley, CA, January son, W. Object-Oriented Modeling and Design, Englewood
1990, 245-258. Cliffs, N.J.: Prentice-Hall, 1991.
[4] Bass, L. and Coutaz, J. Developing Software for the User [23] SEI, “Serpent Overview,” SEI Technical Report CMU/SEI-
Interface, Addison-Wesley, 1991. 89-UG-2, Carnegie Mellon University Software Engineering
Institute, August 1989.
[5] Bass, L. Kazman, R. and Abowd, G. “Issues in the Evaluation
of User Interface Tools”. Proceedings of the ICSE Workshop on [24] Sommerville, I. Software Engineering, 4th edition. Reading,
Software Engineering and Human-Computer Interaction, Sor- MA: Addison-Wesley, 1992.
rento, Italy, May, 1993. Forthcoming [25] Stevens, W.P., Myers, G.J. and Constantine, L.L. “Structured
[6] Briand, L.C., Morasca, S. and Basili, V.R. “Measuring and design” IBM Systems Journal, 13(2):115-139, 1974.
Assessing Maintainability at the End of High Level Design”, Pro- [26] Szekely, P. “Standardizing the Interface Between Applica-
ceedings of the IEEE conference on Software Maintenance, tions and UIMSs”, Proceedings of the ACM SIGGRAPH
Montreal, Canada, 1993. Symposium on User Interface Software and Technology, Will-
[7] Coutaz, J. “PAC, An Implementation Model for Dialog iamsburg, VA, November 1989, 34-42.
Design”, Proceedings of Interact ‘87, Stuttgart, September, 1987, [27] Taylor, R., Johnson, G. “Separations of Concerns in the Chi-
431-436. ron-1 User Interface Development and Management System,”
[8] Garlan, D., Shaw, M. “An Introduction to Software Architec- Proceedings of InterCHI ‘93, Amsterdam, May 1993, 367-374.
ture”. Advances in Software Engineering and Knowledge [28] Taylor, R. personal communication, July 1993.
Engineering, Volume I, World Scientific Publishing, 1993. [29] UIMS Tool Developers Workshop, “A Metamodel for the
Forthcoming. Runtime Architecture of an Interactive System”, SIGCHI Bulle-
[9] Green, J., Selby, B. “Dynamic Planning and Software Mainte- tin, 24(1), 32-37.
nance: A Fiscal Approach”, Naval Postgraduate School, [30] Yourdon, E. Modern Structured Analysis, Englewood Cliffs,
Monterey, CA, NTIS Report AD-A112 801/6, 1981. N.J.: Prentice-Hall, 1989.
[10] Henry, S., Kafura, D. “Software Structure Metrics Based on
Information Flow”, IEEE Transactions on Software Engineering,
SE-7(5), Sept. 1981.
[11] Hix, D. “Generations of User-Interface Management Sys-
tems”, IEEE Software, 7(5):77-87, 1990.
[12] Kazman, R., Abowd, G., Bass, L., Webb, M., “Analyzing the
Properties of User Interface Software Architectures,” Carnegie
Mellon University, School of Computer Science Technical Report
CMU-CS-93-201, 1993.
[13] Military Standard, Defense System Software Development
(DOD-STD-2167A). Washington, D.C.: United States Depart-
ment of Defense, 1988.
[14] Myers, B., Giuse, D., Dannenberg, R., et al., “The Garnet
Reference Manuals”, School of Computer Science, Carnegie
Mellon University Technical Report CMU-CS-90-117-R3, 1992.
[15] Myers, B., Giuse, D., Dannenberg, R., et al. “Garnet: Com-
prehensive Support for Graphical, Highly Interactive User