Innovations Syst Softw Eng (2008) 4:71–85

DOI 10.1007/s11334-007-0042-z

ORIGINAL PAPER

Patterns: from system design to software testing

Neelam Soundarajan · Jason O. Hallstrom ·
Guoqiang Shu · Adem Delibas

Received: 16 December 2007 / Accepted: 19 December 2007 / Published online: 12 January 2008
© Springer-Verlag London Limited 2008

Abstract Design patterns are used extensively in the design of the particular system by specifying a set of specializa-
of software systems. Patterns codify effective solutions for tion rules that are designed to reflect the scenarios in which
recurring design problems and allow software engineers to the defects codified in this set of PTCTs are likely to man-
reuse these solutions, tailoring them appropriately to their ifest themselves in the particular system. We illustrate the
particular applications, rather than reinventing them from approach using the Observer pattern.
scratch. In this paper, we consider the following question:
How can system designers and implementers test whether Keywords Design patterns · Contracts · Testing
their systems, as implemented, are faithful to the require-
ments of the patterns used in their design? A key considera- 1 Introduction
tion underlying our work is that the testing approach should
enable us, in testing whether a particular pattern P has been Design patterns [4,11,24] have had a profound impact on
correctly implemented in different systems designed using how software systems are built. This is not surprising since
P, to reuse the common parts of this effort rather than hav- patterns capture the distilled wisdom of the software com-
ing to do it from scratch for each system. Thus in the approach munity, and provide effective solutions to recurring design
we present, corresponding to each pattern P, there is a set problems. They allow software engineers to reuse these solu-
of pattern test case templates (PTCTs). A PTCT codifies a tions, tailoring them to the needs of their particular systems,
reusable test case structure designed to identify defects asso- rather than reinventing them from scratch. Patterns are used
ciated with applications of P in all systems designed using primarily during the design phase of software systems. In
P. Next we present a process using which, given a system this paper, we focus on later stages of the software life-cycle,
designed using P, the system tester can generate a test suite the testing and maintenance phases. Specifically, the ques-
from the PTCTs for P that can be used to test the particular tion we are interested in is, how can system designers and
system for bugs in the implementation of P in that system. implementers test whether their systems, as implemented, are
This allows the tester to tailor the PTCTs for P to the needs faithful to the requirements of the underlying patterns? This
question is especially important for large systems since such
N. Soundarajan (B) · G. Shu · A. Delibas
Computer Science and Engineering, Ohio State University,
systems tend to have correspondingly large software teams
Columbus, OH 43210, USA and hence a correspondingly greater likelihood that differ-
e-mail: neelam@cse.ohio-state.edu ent members of the team may have subtly different inter-
G. Shu pretations of the patterns underlying the system’s design.
e-mail: shug@cse.ohio-state.edu Hence the team members may implement their respective
A. Delibas parts of the system in ways such that when these parts are put
e-mail: delibas@cse.ohio-state.edu together, the resulting overall system is not compatible with
the patterns’ intent. Design-related bugs of this kind may not
J. O. Hallstrom
School of Computing, Clemson University,
manifest themselves in test cases that are developed using
Clemson, SC 29634, USA standard approaches such as testing against the functional
e-mail: jasonoh@cs.clemson.edu requirements or even code-based criteria such as statement,

123
72 N. Soundarajan et al.

or branch, or path coverage [3]. The testing techniques and only create the subcontract that specifies how the pattern is
supporting tools discussed in this paper will be of value in specialized for use in a particular application.
reducing the likelihood of such bugs going undetected, and A similar consideration is a primary motivation for the
will aid in their localization. approach we present in this paper. That is, a key goal of our
The problem of ensuring that a system is faithful to the work is that the testing approach should enable us, in test-
patterns underlying its design is even more serious during ing whether a particular pattern P has been correctly imple-
the maintenance phase since system maintenance may be mented in different systems designed using P, to reuse the
the responsibility of team members who were not involved common parts of this effort rather than having to implement it
with the original design or implementation of the system— from scratch for each of these systems. Thus in the approach
and consequently have only a superficial understanding of we develop, corresponding to each pattern P, there is a set
its design. As a result, over time, the design integrity of the of what we will call pattern test case templates (PTCTs). A
system is likely to erode. The approach we develop in this PTCT codifies a reusable test case structure designed to iden-
paper can help address this aspect of the problem as well. tify defects associated with applications of P in all systems
This is because the test suite used to test whether the system designed using P. Next we present a process using which,
is faithful to its underlying patterns can be included as a given any system designed using P, the system tester can
key part of the system’s design documentation in the same generate a test suite from the PTCTs for P that can be used
manner as standard unit tests or other functionality tests. As to test the particular system for bugs in the implementation
the system evolves during maintenance, the design test suite of P. The process allows the system tester to tailor the set
will help system maintainers identify and localize changes of PTCTs corresponding to P to the needs of the particular
that may conflict with the original design. If one of these tests system by specifying a set of specialization rules that are
were to fail, that doesn’t necessarily mean that the changes designed to reflect the structure and the scenarios in which
in question are flawed. It may be that the design needs to the defects codified in the set of PTCTs for P are likely to
be modified; but this decision should be made consciously manifest themselves.
with the design documentation being appropriately updated, An example will help illustrate our approach. Consider
rather than simply modifying the code. We will return to this the classic Observer pattern which we will use as our pri-
point in the final section of the paper. mary case study throughout the paper. According to the stan-
In standard software testing, we test software against its dard description [4,11] of the pattern, participating objects
specifications. These specifications must be formal since, play one of two roles: Subject or Observer. The intent of
otherwise, there is no way to conclude whether a test was suc- the pattern is to keep multiple observers consistent with
cessful or not1 . Similarly, to test whether a system is faithful the state of a single subject. When an object wishes to
to the intent of the patterns underlying its design, we need become an observer of a subject, it invokes the subject’s
formal specifications of the patterns. Attach() method. When it is no longer interested, it invokes
In previous work [12,26,31], we presented an approach to the Detach() method. In addition, Subject includes a
providing precise specifications for patterns and their appli- Notify() method which, according to the intended usage of
cations in the form of pattern contracts and subcontracts. It the pattern, is required to be called whenever the state of the
is against these contracts and subcontracts that we wish to subject is modified. Notify() is required to invoke Update()
test individual systems to validate their design correctness. on each attached observer which must, in turn, update the
An important aspect of this approach to pattern speci- state of that observer to make it consistent2 with the current
fication is that the requirements and behavioral guarantees subject state.
specified in a pattern contract apply to all uses of the pattern; The standard UML diagram for the pattern appears in
a subcontract characterizes how the pattern is specialized in Fig. 1. The subject maintains a set of references to the
a particular application. Thus the wisdom of the community currently attached observers in the variable _observers.
captured in a design pattern is reflected in the corresponding According to the standard description, when Attach() is
pattern contract; it can be reused by every team that uses the invoked, it adds a reference to the attaching observer to
pattern in the design of its system. The individual team need _observers. But if this was all Attach() did, the intent of the
pattern would be violated; this is because the newly attached
observer’s state may not be consistent with the current state
1 When is a test successful? One definition in the testing literature [3] of the subject – it will remain inconsistent until the next
says that a test is successful when it shows that the software does not
meet its specification, since the purpose of the test is to find flaws in
the software. Others [20] use the convention that a test succeeds, or that 2 What precisely terms such as “modified” and “consistent” mean is not
the software “passes” the test if, during the test execution, the software clear from such informal descriptions. The formal contract for Observer
behaves according to its specification. We will use this latter convention which we will see in the next section will resolve this and other ambi-
in this paper. guities in the informal description.

123
Patterns: from system design to software testing 73

Subject Observer Paper organization. Section 2 summarizes our approach to

_observers
pattern contracts and their relation to PTCTs. Section 3 devel-
+Attach(in Observer) +Update() ops the essential ideas of PTCTs and how they are defined
1 *
+Notify() based on pattern contracts. A number of PTCTs for Observer
+Detach(in Observer) are presented based on the contract for the pattern. Section
4 considers how test cases, for a particular system designed
using a given pattern, may be generated from the PTCTs for
for all o in _observers the pattern; a simple system, the simulation of a Hospital, is
o.Update() used for illustration. Issues of coverage (and the associated
metrics) in the context of pattern-centric testing are also dis-
ConcreteSubject cussed. Section 5 briefly surveys elements of related work.
ConcreteObserver
−subjectState *
Section 6 concludes with a summary of the approach, and its
1 −observerState
relation to test-driven design (TDD).
+Update()

2 Pattern contracts
Fig. 1 Observer pattern

The contract for a design pattern P consists of a set of role-

notification cycle. To address this problem, Attach() must contracts, one corresponding to each role of P, and a portion
invoke Update() on the newly attached observer. Failure to that corresponds to the pattern as a whole. The role-contract
do this is a common bug that appears in systems designed for a role R lists the state components of R, and specifies, in
using the Observer pattern. It is also a bug that can be missed standard pre-/post-condition format, requirements that must
during standard (functional) testing since the inconsistency be satisfied by the various methods of R. In a system designed
of the newly attached observer may last only for a short using P, a class C playing the role R will typically pro-
time, depending on when exactly the next modification in vide other methods in addition to those “named” in R. If
the subject state takes place, resulting in a notification cycle these “other” methods were to behave arbitrarily, then the
which will remove the inconsistency. intent of the pattern would be violated even if the methods
In our approach, given that this problem is common to sys- corresponding to the named methods behaved according to
tems designed using the Observer pattern, one of the PTCTs their specifications in the role-contract. In order to elimi-
for this pattern, as we will see, will correspond to this poten- nate this possibility, R’s role-contract will also include an
tial bug. When testing a system designed using Observer, others specification that must be satisfied by all methods
the system tester will specialize this PTCT appropriately to of C, except those corresponding to the ones named in the
generate a set of test cases that will help test the particular role-contract. The portion of the contract that corresponds
system for the presence of this bug. These tests will confirm to the pattern as a whole consists of an invariant over the
that when a new observer in this system attaches to the sub- (role) states of the various objects enrolled, at runtime, in an
ject, it is appropriately updated. In other words, these tests instance of the pattern. The invariant will be satisfied when-
will help ensure that the system is faithful to (this particu- ever control is not inside any of the methods of any of the
lar aspect of) the intent of the Observer pattern; and if the participating objects.
system were to be modified during system evolution, the test One potential problem with formalizing a design pattern
cases and the associated documentation will help the system is that its flexibility might be reduced or even eliminated.
maintainers test for this bug and to identify any violations A number of features of our formalism help guard against
of the intent of the underlying pattern that may be inserted this, the most important of these being the notion of an aux-
during this phase. Thus, the approach will help preserve the iliary concept. An auxiliary concept is a relation over one
design integrity of the system as it evolves. or more states of one or more objects interacting with each
Although some approaches to formalizing patterns have other according to the pattern. Auxiliary concepts are used
been proposed [8,10,19,23] (in addition to our work cited in the role-contracts and the pattern invariant, but their defin-
above), we are not aware of any other work focused on ition are not part of the pattern contract. Instead, definitions
the question of testing a system to see whether it correctly tailored to particular systems, are provided in the subcon-
implements the patterns underlying its design. Given the tracts corresponding to systems. The subcontract for a
widely recognized importance of design patterns and their particular system also includes a set of role-maps, one
use in designing and implementing large software systems, corresponding to each class C playing a role R in the pattern,
we believe that this is an important concern and believe that as applied in this system. The C-R-role-map specifies how
the approach we present will prove valuable for this task. the state, i.e., variables, of C map to the state of R (listed in its

123
74 N. Soundarajan et al.

role-contract), which methods of C correspond to each of the

named methods of R, etc.
As noted in Sect. 1, the pattern contract applies to all
systems designed using the pattern; the subcontract speci-
fies how the pattern is specialized for use in a given system.
Correspondingly, the pattern test case templates that we will
develop in the next section and will apply to all systems
designed using the pattern, will be based on the pattern’s con-
tract. The test cases for a given system will be obtained by
specializing the templates using the information in the sub-
contract and accounting for additional application-dependent
factors, as we will see later.
Consider the Observer pattern. The intent of the pattern, as
noted earlier, is to ensure that when the state of the subject is
modified, the observers “attached” to the subject are appro-
priately updated so that their states become consistent with
the current state of the subject. “Modified” does not, how-
ever, mean a change in any arbitrary bit or byte of the sub-
ject state; rather, some modifications are important enough to
require updates of the attached observers and others are not.
At the same time, which modifications are important enough
in this sense and which are not varies from system to system.
Hence, we use an auxiliary concept, Modified(), a relation
over two states of the subject, to distinguish “important”
changes in the subject state from “unimportant” ones. Sim-
ilarly, what it means for an observer state to be “consistent”
with the subject state varies from system to system; and, Fig. 2 Observer pattern contract (partial)
indeed, from one kind of observer that may be attached to
the subject to another that may be attached to the same sub- interactions are then expressed in the form of appropriate
ject. Hence we use a second auxiliary concept, Consistent(), conditions on the trace variables. As we will see in the next
a relation over an observer state and a subject state, to rep- section, PTCTs will be designed to test whether the sequence
resent this notion. The contract for Observer, a portion of of interactions that occur in a system designed using the pat-
which appears in Fig. 2, is written using these concepts and tern satisfy these conditions.
specifies the requirements that apply to all systems designed The contract for Observer starts by listing the auxiliary
using this pattern. concepts. Next (lines 6–8), we have a constraint that must
Many patterns are concerned with the sequences of inter- be satisfied by the definitions of these concepts in any sub-
actions between various objects, i.e., the sequences of method contract of Observer; we will consider this after consider-
calls invoked on the various objects involved in the pattern. ing the rest of the contract. Next (lines 10–11), we have
Thus, as we noted in Sect. 1, the Notify() method of the the pattern invariant, the reason (or the “reward” [26]) for
Subject role is required to call Update() on each of the using the pattern, that is guaranteed provided we meet all the
observers currently attached to the subject. Such require- requirements specified in the pattern contract. players[] is
ments are expressed in the pattern contract using the trace τ , the array of objects enrolled in the pattern instance in ques-
a ghost variable [15] provided by the formalism. In effect, tion and is another ghost variable of the contract formalism.
when a method m() starts execution, a corresponding trace In the case of an instance of Observer, the first player to
τ , initialized to the empty sequence, is automatically cre- enroll3 in any instance of the pattern is the subject; hence
ated. Each call that m() makes to a named method during
its execution is recorded as an element of the sequence, and 3 The complete pattern contract will also include clauses that specify
includes information about the object on which the method how an object enrolls to play a given role in a pattern instance, how
was invoked, the name of the method, and information about a pattern instance is created, etc. We omit the details of these here;
the associated arguments and return values. A number of see [12,26]. When testing a system designed using the pattern, it is,
of course, necessary that the requirements specified in these clauses
mathematical functions, some of which we will introduce
of the pattern contract (specialized for the particular system) are satis-
as needed, simplify trace manipulation, access to individual fied. Hence appropriate PTCTs must be defined, or these tests must be
trace elements, etc. Contract requirements concerning object included as part of other PTCTs.

123
Patterns: from system design to software testing 75

(a reference to) this object will be in players[0]. The rest in a particular application, that s1, s2 are two states of the
of the objects in players[] will be the objects that enroll to subject, and o1 is the state of an observer. Suppose that the
play the Observer role (by invoking the Attach() method, definitions of Modified() and Consistent() for this application
this being the action required for this purpose as specified are such that each of the following is true:
in the (elided from Fig. 2) enrollment clause for this role). Consistent (s1,o1), ¬Modified (s1,s2), and
Thus the invariant (lines 10–11) states that the state of the ¬Consistent (s2,o1)
first object in players[], i.e., the state of the subject in any
Suppose, at some point in the execution of this system, the
given instance of this pattern, is Consistent() with the states
subject is in the state s1 and the observer in the state o1.
of each of the observers (currently) enrolled in this instance.
Suppose, finally, that the subject state changes from s1 to
Next we have (part of) the Subject role contract. First we
s2. Then, according to the others specification in the Sub-
specify the state of the subject as consisting of obs, used to
ject role, the Notify() method will not be invoked and hence
store references to the currently attached observers. Next we
Update() will not be invoked on this observer and its state
have the specification of Attach(), one of the named methods
would remain as o1. At this point, the current state, o1, of the
of this role. The pre-condition requires that the attaching
observer will be inconsistent with the current state, s2, of
observer not already be attached, i.e., not have a reference
the subject, and the invariant that the pattern was intended
to it in obs. The post-condition states4 that a reference to the
to guarantee will be violated although the individual meth-
object is added to obs; that the subject itself is not modified,
ods in the individual roles, as actually implemented in the
in the sense of the auxiliary concept Modified(), and that one
particular application, satisfy their respective specifications.
call to a named method of the pattern has been made during
The problem is that we have conflicting notions of what a
the execution of Attach(), this being to the Update() method
sufficiently serious modification in the subject state is that
invoked on the attaching observer.
requires the observers to be updated on the one hand, versus
The others specification (lines 20–23) states that any
what it means for the subject state to be consistent with the
other method of the class playing the Subject role should
observer state, on the other. The constraint specified (lines
make no change in obs, and should either not modify (again
6–8) in the contract ensures that this problem doesn’t arise.
in the sense of Modified()) the subject state, or must invoke
Thus while the designers of a particular system are free to
the Notify() method (on the current subject). The (elided)
tailor the definitions of the auxiliary concepts to their partic-
specification of Notify() states that this method invokes
ular needs, they must ensure that these definitions satisfy the
Update() on each observer in obs.
constraint specified in the contract.
The Observer role contract states that the state of this
How realistic is this problem? That is, how likely is it
role consists of the variable sub (that holds a reference to
that the software team responsible for an actual application
the subject to which this observer is attached); that the
designed using the Observer pattern would base their sys-
Update() method does not change the value of sub, and
tem on such mutually conflicting notions of Modified() and
makes the state of the observer to be consistent (in the sense
Consistent()? It depends on the size of the team in question;
of the auxiliary concept Consistent()) with that of the sub-
as the team grows larger, the likelihood of such problems
ject. The others specification also states that it not modify
seem to also grow. But even for relatively small systems, the
sub, and leave the observer in a state that is consistent with
problem can creep in during system evolution; we will see
that of the subject5 .
an example of this later in the paper.
Let us now consider the constraint specified in the con-
Unfortunately, however, specific PTCTs (or, more pre-
tract (lines 6–8). Although the definitions of the auxiliary
cisely, test cases obtained by specializing them for a partic-
concepts, tailored to the needs of a particular application, will
ular application) cannot test these constraints because they
be provided by the corresponding subcontract, these defini-
are general conditions typically involving universal quanti-
tions cannot be completely arbitrary. For example, suppose,
fiers over the states of the various objects enrolled in the
pattern instance. Instead, violations of these constraints may
4 We use the “#” notation in the post-condition to refer to the pre- show up as violations of, for example, the pattern invariant—
condition value of the variable, i.e., its value when the method started although the individual methods invoked as part of the test
execution. case may satisfy their own specifications. We will see this in
5 Standard informal descriptions [11,25] of the pattern seem to suggest
the discussion of the example.
that the other methods of this role should not make any changes in
the state of the observer. But this is unnecessarily restrictive. As the
pattern contract states, all we need is that any changes in the observer 3 Pattern test case templates (PTCTs)
be such that they leave the observer state consistent with the sub-
ject state. This is an example [12,26] of how our pattern contracts can
identify dimensions of flexibility that may be missing in the standard Given the requirements captured in a pattern contract, how
informal descriptions. do we develop appropriate PTCTs that can be used to test

123
76 N. Soundarajan et al.

not the contract requirements are satisfied in those common

situations. These templates must then be specialized to obtain
the actual test cases corresponding to a particular system. The
system tester does this specialization, as we will see in the
next section, with help from JDUnit6 , a testing framework/-
tool that we are in the process of building, by combining
the information in the contract, in the subcontract and the
PTCTs, and taking account of any relevant details about the
particular system.
The language for defining PTCTs is intended to mirror
standard Java programming syntax to support rapid practi-
tioner adoption. Before looking at specific PTCTs, we sum-
marize the syntax of the structure of a PTCT. A PTCT is
represented as a segment of stylized Java code with inter-
spersed asserts that, based on the pattern contract that the
PTCT corresponds to, are expected to be satisfied at those
points. The asserts will be based on the pre- and post-
conditions of the named methods as well as the other spec-
ifications in the various role contracts of the pattern contract.
In addition, some of the asserts may also be based on
Fig. 3 Pattern-centric testing the pattern invariant. Since the PTCT is intended to apply to
all applications of the corresponding pattern, the construc-
tor and method calls used in its definition must conform to
whether systems implemented using the pattern satisfy these the signatures specified in the relevant role contracts. When
requirements? It is important to stress that, given a pattern the actual test cases corresponding to a given application are
contract, construction of the corresponding PTCTs is not a generated from the PTCT, information in the corresponding
mechanical activity. The purpose of the PTCTs is to help test subcontract about the mapping of named methods of the
for common mistakes in implementing the particular pattern. roles to particular methods of the corresponding classes and
Clearly, which mistakes are common and which are not is a the mappings of the role-method signatures to the class-level
question that can be answered only on the basis of experience method signatures will be used to translate the statements in
with systems designed using the pattern. In this section, we the PTCT to appropriate code in the test cases.
will present three PTCTs, the first two corresponding to the A PTCT begins with an initialization portion that creates
Observer pattern and the last one corresponding to the Com- instances of one or more role types. named methods may
posite pattern. These are based on our assessment, based on be invoked to bring the new player objects into states appro-
our own experience with implementing systems and based priate for the test case family being developed. The PTCT
on the reports in the literature, of the most common mis- developer may optionally include calls to other methods,
takes in implementing this pattern. PTCTs corresponding to which serve as placeholders for class methods, which may
other patterns will similarly be based on an assessment of be introduced at the point of test case generation. Since the
commonly reported mistakes made in implementing them. precise number of other method calls required to bring an
But these are, of course, not cast in stone. Indeed, with addi- object into the state appropriate for testing a particular system
tional information about such mistakes, these PTCTs will be varies, the PTCT syntax allows developers to specify that a
refined and additional PTCTs introduced. The main contribu- single other method may be instantiated by multiple class
tion of this paper, then, is the approach presented for testing methods, supplied at the point of test case generation. The
systems to ensure the correct implementation of their under- initialization section typically concludes with a PTCT pre-
lying patterns—not the particular PTCTs presented. We will condition. This assertion, expressed as a standard boolean
return to this point in the final section. expression, captures properties assumed by the PTCT body.
Figure 3 schematically illustrates our approach to test- The check ensures that the class-level constructor and method
ing systems against the patterns underlying their designs.
The pattern contract formally captures the requirements of
the design pattern, the subcontract specifies how the pat- 6 The name of the framework, currently under construction, is intended
tern is specialized in the particular system. The PTCTs are to draw a parallel to JUnit, the popular unit testing framework for Java;
designed based on the most common mistakes that are made the additional D emphasizes that the framework is used to test units of
in applying the pattern and are designed to check whether or design.

123
Patterns: from system design to software testing 77

calls introduced at the point of PTCT instantiation satisfy this

pre-condition.
Next, a PTCT defines a body. The body portion gener-
ally follows a standard testing idiom: First, local variables
are used to store the pre-conditional states of the players
participating in the PTCT. Next, invocations are placed to
the methods of interest, i.e., those relevant to the design
pattern requirements being tested. Finally, the PTCT con-
cludes with one or more assertions used to check that the
relevant pattern requirements are satisfied. These assertions,
as noted above, are based on the method and invariant spec-
ifications in the pattern contract and generally involve the
post-conditional states of the participating players, as well Fig. 4 PTCT for subject.attach()
as the pre-conditional states saved at the start of the body.
There is, however, one important deviation from the stan- that JDUnit provides appropriate getter methods for each
dard testing idiom: Since key parts of the pattern contract role field, including private fields.) Next, we add the attach-
involve the trace τ of method calls, the assertions contained ing observer to pre_obs (line 4) to simplify the expression
in a PTCT will correspondingly include conditions on a spe- of the assertion check corresponding to the post-condition of
cial variable tau, used to record the trace of invocations orig- Attach() (lines 9–10). We additionally store the current state
inating from the PTCT. Note that the PTCT is not allowed of s (line 5), since this is required to check that Attach() not
to contain any code that directly updates this variable; the modify s according to the applicable definition of Modified().
management of tau and updating it to reflect the method A couple of points should be noted here. The getter meth-
invocations as they happen is the responsibility of JDUnit. ods are more involved than simply returning the current value
The one exception to this rule is that a PTCT may clear tau of the field in question. In an actual system designed using the
when the earlier invocations recorded in it are no longer of pattern, the class C playing a particular role R may provide
interest. In addition, JDUnit provides a number of helper a different set of fields from those specified in R’s role con-
functions that we may use in a PTCT; these functions allow tract. As we will see in the next section, the subcontract for
us to obtain information about the current contents of tau. the system will specify the mapping from C’s fields to those
We will see several of these in the PTCTs that we discuss of R. The getter methods provided by JDUnit will make use
next. of this mapping to translate the values of C’s fields to the
There are some important differences in detail between corresponding values of R’s fields. Similarly, the clone()
the trace τ used in the specifications in the pattern contract operation applied to s (line 5) will create an object of what-
and the trace tau maintained by JDUnit. The primary reason ever class plays the Subject role in the particular application.
for these differences is to enable us to construct more useful Finally, JDUnit will use the auxiliary concept definitions
PTCTs. In particular, suppose we have a PTCT that includes provided in the subcontract to evaluate assertions involving
a call to a method m1() followed by a call to method m2(). concept references, such as the one involving Modified()
Thus the invocation of m1() completes before the invocation (line 9). We will return to these points in the next section.
of m2() begins; i.e., the execution of neither is nested inside Let us now consider contract requirements involving
the other. Suppose the trace of method calls that takes place traces. As we saw, in the contract formalism, each method
during the execution of m1() is t1 and that during m2() is m() has an associated trace τ , initialized to the empty
t2. The trace tau maintained by JDUnit allows us to write an sequence when the method starts execution. Information
assert following the completion of m2() that, for example, about calls made by m() to named methods are recorded in
imposes conditions on t2 based on the value of t1 or vice- τ , with m()’s post-condition imposing necessary conditions
versa. We will see these details shortly. on τ . In those cases where a PTCT includes calls to methods
The main portion of a PTCT intended to test the behav- with trace requirements, the PTCT will include assert state-
ior of the Attach() operation of the Subject role of the ments that check relevant conditions on the associated trace
Observer pattern appears in Fig. 4. As discussed earlier, a variables. Indeed, a PTCT may include assert statements
common problem in using this pattern is not updating an that require particular relations to hold across multiple trace
observer when it initially attaches to a subject. In the PTCT, variables, each associated with methods preceding the asser-
we begin by creating a subject s (line 1), and an observer o tion check. To enable easy expression of such asserts, we use
(line 2). Next, we use a “getter” method provided by JDUnit, the following approach. When the instantiated PTCT begins
to retrieve the pre-conditional value of s.obs before the call to execution, the testing framework creates a trace object tau,
Attach() (line 7), and to store this value in pre_obs. Note initialized to the empty sequence. Each named method

123
78 N. Soundarajan et al.

invoked during the execution of the PTCT is recorded as

an element of tau, and includes information about the tar-
get object, the identity of the method invoked, etc. In addi-
tion, and this is the key difference from the use of τ in the
contract formalism, each element of tau itself includes the
trace of methods executed during the associated call’s exe-
cution. Thus, while there is only a single tau object, it main-
tains a branching structure corresponding to the computation
tree rooted at the PTCT. The traces contained in the individ- Fig. 5 PTCT for subject.other()
ual elements of τ may be extracted using a simple accessor
(“tr()”) function provide by JDUnit.
Immediately prior to the call to Attach(), we clear tau sions will be based primarily on the common mistakes that
(line 6). This removes the trace entries associated with the practitioners make in using the given pattern.
preceding constructor calls (lines 1–2), as well as any calls to Note also the use of appropriate temporary objects to
named methods introduced during the specialization process capture pre-conditional values referenced in various post-
in going from the PTCT to actual test cases; in the next conditions; this is common in specification-based testing
section we will see how the specialization may introduce given that post-conditions refer to both the state at the end of
such calls. Hence, when control returns following the call to the method and at its start. The pattern invariant (lines 6–7,
Attach(), tau contains a single entry, corresponding to Fig. 2) could also have been checked as part of the assert
this call. The trace of method calls that occurred during the statement. This would amount to checking that Attach() not
execution of this Attach() is extracted and stored in t1 only invokes Update() on the attaching observer, but also
(line 8). The assert that follows imposes appropriate condi- that the latter method appropriately updates the observer’s
tions on this trace, based on the requirements specified in the state to be Consistent() with the subject.
pattern contract. The first two clauses require that s._obs be Next consider the PTCT shown in Fig. 5, intended to test
appropriately updated (line 9), and that s not be Modified() the behavior of Subject’s other methods. We begin by cre-
(line 10). The last three clauses (lines 11–13) address trace ating a subject (s) and two observers (o1, o2) (lines 1–3),
conditions imposed on Attach(). Together, these clauses and attach both observers to the subject (line 4). We then
require that Attach() invoke exactly one named method, save the pre-conditional value of s.obs (line 5) and the value
and that this call be to the Update() method of the attach- of s as a whole (line 6). tau is then cleared, and some other
ing observer7 . method is invoked on s (line 7). Finally, the trace associ-
The assert statement is, of course, based on the specifi- ated with the other call is saved (line 8), and the behavior
cation of Attach() included in the Subject role contract of the method is checked against the requirements specified
shown in Fig. 2. The effort involved in writing the PTCTs in the pattern contract (lines 9–10). The assert statement
corresponding to a pattern is primarily in identifying the requires that s.obs be unchanged, and that both o1 and o2
common mistakes that are made in applying the pattern and be Consistent() with s.
coming up with appropriate test case templates based on this. In generating test cases for a particular system from this
Once that has been done, the appropriate asserts to include PTCT, the s.other() call will have to be replaced by calls to
are determined by the pattern contract. But even here, the appropriate methods of the class playing the Subject role.
PTCT designer may choose not to include all of the clauses Some of these methods may require additional arguments. In
specified in the role contract as the post-condition of the the next section, we will consider the problem of generating
particular method. Even more importantly, at what points suitable values for these arguments. At this point, however,
to include checks of the pattern invariant and assertions that the more important issue is the mismatch between the assert
require particular relations between the traces associated with statement and the requirements specified in the Observer con-
two or more method calls appearing earlier in the PTCT, are tract. According to the contract (Fig. 2), when s.other()
all decisions the PTCT designer must make. And these deci- terminates, either the state of s must not be Modified(), or
the Notify() method must have been invoked. As we have
7 We obtain the identify of the method invoked and the target object
already seen, this method will in turn invoke Update() on
by using the helper functions mt() and ob() provided by the JDUnit each attached observer, bringing the objects into states con-
framework. These functions may be applied to individual elements of a sistent with the new state of the subject. Hence, the assert
trace in which case they return the identify of the method/target object included in the PTCT is a test of the expected net behavior
involved in that element. Or they may be applied to a trace, in which case
of the participating objects, i.e., it is a more “global” test of
they return the sequence of method identities/target objects involved in
the various elements of the trace. We such usage in the pattern contract the system. There is, however, a risk in using such a PTCT,
in the last section. especially if these were the only ones used. If, for instance,

123
Patterns: from system design to software testing 79

an other method were to modify the state of the subject,

neglect a call to Notify(), but by chance leave s in a state
Consistent() with the states of o1 and o2, the design defect
would go undetected.
In [27], we present a scenario in which this problem mani-
fests itself during system evolution; we summarize the exam-
ple here. A class S1 plays the Subject role, and provides two
fields, f1 and f2. A change in either field is considered to be
a modification of the subject according to the definition of
Modified() supplied in the system’s subcontract. A class O1
plays the Observer role. In an initial version of the system,
O1 is interested only in the value of f1; changes in f2 are Fig. 6 Composite role contract (partial)
ignored. The Update() method of O1 uses an appropriate
getter method to retrieve the value of f1, and then updates Patterns such as Composite, although classified as structural,
the observer’s state to become Consistent() with the state of also have behavioral aspects to them. Indeed, the main pur-
the subject. The Consistent() concept is defined suitably in pose of the Composite pattern is to ensure that each operation
the subcontract. The S1 class includes a bug that omits a call of a composite object is implemented by invoking the cor-
to Notify() when a change in f2 occurs. A PTCT similar to responding operation on each of its components and com-
the one above will not detect this defect since each observer bining the results returned by these invocations appropriately
will remain Consistent() with the subject when the other to obtain the result of the original operation invocation.
method terminates — even if f2 has changed without a cor- A small portion of the Composite pattern contract, in par-
responding call to Notify(). ticular a part of the Composite role contract, appears in
During system evolution, O1 is modified so that the value Fig. 6. The specification of any operation() of this role
of f2 becomes significant. Instances of O1 record the cur- states that when this method finishes, the set of children
rent value of f2 associated with their corresponding subject. is unchanged; that during the execution of this method, there
Hence, the definition of Consistent() is suitably modified, as should have been a call to the corresponding method on each
is the implementation of O1’s Update() method. The new child c; and that the result returned by this method should be
implementation retrieves the values of both f1 and f2, and equal to the value obtained by appropriately combining the
updates the state of the observer appropriately. When the test results of these method calls invoked on the children. Since
case derived from the PTCT shown in Fig. 5 is executed, the the details of precisely how these results are combined to
assert statement will generate an error; the clause involv- obtain the result of the Operation() applied to the com-
ing the Consistent() concept will be violated. The natural posite will vary from one application designed using this
assumption is that the fault lies in an area affected by sys- pattern to another, we use an auxiliary concept, Glue() to
tem evolution. In fact the defect lies in the original S1 class represent this. Thus the final clause of the post-condition
— in particular, the failure of its other() method to invoke of Operation() requires that the result returned by
Notify() when the value of f2 changes. the original call is equal to the value of Glue() applied to
The solution is to revise the PTCT to fully test the interac- the sequence of results in the method calls recorded in τ ,
tion requirements specified in the pattern contract, rather than this being obtained by means of the helper function,
testing for net effects. More precisely, the last two clauses of resultSeq().
the assert in Fig. 5 (line 10) should be replaced with the Figure 7 presents a PTCT to test that the requirements
following conditions: specified in Fig. 6 are satisfied. We create two composites, c1
and c2, a leaf l, and create a suitable composite structure. We
8 a s s e r t ( … clauses from Fig. 5 …
9 && (! M o d i f i e d ( pre_s , s ) ||
then invoke operation() on the top level composite struc-
10 (( t1 . l e n g t h ( ) = = 1 ) & & ( t1 . m ( ) = = " N o t i f y ")) ture and, assert, after the call returns that the trace created
during the execution of this method is equal to the number of
Before concluding this section, we will consider another children of this structure, and that the result returned matches
PTCT, this one for the Composite pattern. Composite is a that obtained by applying the auxiliary concept Glue() on
structural pattern [11] intended to compose objects into a the results returned by the calls recorded in that trace. The
tree structure to represent part-whole hierarchies and to allow definition of Glue() will, of course, be given in the sub-
clients to treat individual objects and composite objects in a contract corresponding to a particular system, for which we
uniform manner. The pattern has three roles, Component, will specialize this PTCT to produce actual test cases. It may
Leaf, and Composite. Component is an abstract role with be worth noting that this PTCT does not completely test the
Leaf and Composite being the two kinds of components. specification in Fig. 6; in particular, while it does test that the

123
80 N. Soundarajan et al.

Fig. 7 PTCT for composite.operation())

number of method calls recorded in the trace t1 is what it

should be according to the specification, it does not test which
methods were invoked in these calls, nor on what objects. It is
straightforward to modify the PTCT to include these checks
and we omit the details.

4 Generating test cases

Given PTCTs such as those in the last section, how do we

generate actual test cases corresponding to a given system S
designed using a particular pattern? Consider, for example,
line 7 of Fig. 4. The method Attach() that is being invoked Fig. 8 Hospital simulation code
in this line may have an entirely different name in S. Indeed,
even the classes playing the Subject and Observer roles
will have their own names appropriate for the application. The role map for Patient as Subject specifies how the
To illustrate these and other issues involved in generating _obs of this patient viewed as as subject may be obtained
actual test cases, we will use a simple Hospital Simulation from the state of the patient. Next, the method maps specify
system, parts of which appear in Fig. 8, designed using the that addNurse() and setDoctor play the role of Attach()
Observer pattern. and that notify() plays the role of Notify(). The role map
The Hospital system consists of three main classes, for Nurse as Observer illustrates a technical problem: since
Patient, Nurse and Doctor. Instances of the Patient class a nurse may be observing multiple patients, each of which
play the Subject role; instances of the other two play the is the subject in a distinct instance of Observer, how do
Observer role. Zero or one doctor object and zero or more we identify the particular patient involved with the current
nurse objects are assigned to observe patient. The variables pattern instance? The contract formalism [12,26] provides
_nurses and _doctor in the Patient class are used to keep the notion of lead object to address this. The lead object in
track of the assigned doctor/nurses. A nurse object tracks a pattern instance is the first object that enrolls in a pattern
the temperature (_temp) of the patient it is observing; a instance, i.e., the one that is in players[0]. The method map
doctor object tracks the heart-rate (_hrtRt). The checkVi- for Nurse maps its update() (applied to the lead object) to
tals() method updates _temp and _hrtRt as needed, and Update() of the Observer role. The role map for Doctor
invokes notify() if needed. The Nurse and Doctor classes as Observer is analogous.
include an update() method to help keep track of the data Next we have the definitions of the auxiliary concepts.
about the patients being observed. They also provide a get- According to the definition of Modified() (lines 25–27), the
Status() method that returns information about the patients. state of a patient is considered to be modified if either _temp
Before considering how test cases corresponding to the value or the _hrtRt value has changed. The definition in lines
Hospital system may be generated from the PTCTs of the 28–29 state that a nurse n’s state is consistent with that of the
last section, we have to consider the subcontract that specifies patient p if n has the correct information about p._temp.
how the Observer pattern is specialized in this system; the The definition (lines 30–31) of consistency of a doctor d
main part of this subcontract appears in Fig. 9. and patient p is similar.

123
Patterns: from system design to software testing 81

suitably specialize the PTCT by defining appropriate test case

specializers.
A key purpose of a test case specializer (TCS) is to limit
the set of generated test cases, i.e., to specialize the PTCT
in such a manner that the test cases that are generated are
the ones most appropriate to the system under test. There
are a number of dimensions in which this specialization has
to be performed. First, the objects constructed in setting up
the scenario for a PTCT are role objects. If R is a role of
this pattern and, in the PTCT, we construct an object ob of
type R, and if, according to the subcontract for the system
in question the class C1 and C2 play the role R, then in an
actual test case, ob may be either a C1 object or a C2 object;
the TCS has to specify which it is.
Second, calls in the PTCT to named role methods (such as
Attach()) have to be replaced by calls to the corresponding
class methods, as specified in the appropriate role maps of
the subcontract. This process, by itself, does not require any
additional information from the TCS – unless more than one
class method plays the role method in question or, as is more
frequently the case, the class method in question requires
additional arguments beyond the ones expected by the role
method. These values will, of course, have to be provided by
the TCS. There are two important considerations to account
for here. First, the application class may specify, as part of
Fig. 9 Hospital-observer subcontract
the pre-condition of the corresponding class method, some
conditions that must be satisfied by these argument values. If
some set of argument values generated according to the spe-
Let us now turn to the main task of this section, generat- cialization in the TCS do not satisfy these conditions, that
ing, from the PTCTs for a pattern, test cases for a particu- test case cannot be used. For example, in the Hospital sys-
lar system designed using the pattern. For any such PTCT, tem, we are not allowed to attach two doctor objects at the
a (large) number of test cases may be generated. Each test same time to a given patient object. In other words, an
case would be obtained by replacing each role object in the attempt to call addDoctor() on a patient object that
PTCT, such as s, o1, and o2 in the case of the PTCT in already has a doctor will violate the pre-condition of this
Fig. 5, by corresponding application objects, as determined method and hence the method may behave in whatever way
by the subcontract, such as a patient, a nurse, and a doc- it chooses and this is not a bug in the system, despite the fact
tor, respectively, in the case of the Hospital system; replacing that, because of such behavior, an assert in the test case
the calls to role methods in the PTCT, such as s.Attach(o1) may fail. At the same time, we cannot always eliminate this
and s.Attach(o2), by calls to the appropriate application type of problem when the test case is generated because some
methods, again as determined by the subcontract, such as of these assertions may be checkable only during execution
s.addNurse(o1) and s.setDoctor(o2), respectively; etc. of the test case. In order to address this type of problem,
While that much is straightforward, there are some impor- JDUnit inserts additional asserts in the test cases it gen-
tant additional issues to consider. For example, it may be that erates immediately prior to calls to any methods. Each of
the constructor functions (or other methods, for that matter) these asserts checks that the pre-condition of the method
of some of the application classes require additional argu- being called is satisfied. If, when a test case is executed,
ments. What values do we supply for these? The PTCT, of such an assertion is not satisfied, that is recorded in the log
course, imposes no requirements on these. As far as the PTCT maintained by the system; but the assertion failure does not
and the pattern are concerned, any values may be used for indicate a failure of the test case8 . The second issue that the
these additional arguments. This means that the number of
possible test cases we can generate will be very large. Which 8 This is a distinction from the JUnit framework where every assertion
of these test cases will actually be useful will depend on
violation is reported as a failure. In order to allow a different treat-
the system, in particular its implementation details. Hence ment for pre-condition violations, in our design of JDUnit, we use an
the human tester using our approach will be required to additional method, preAssert, to specify such assertions.

123
82 N. Soundarajan et al.

tester designing the TCS has to be concerned about in decid- 8 A t t a c h ( o ) { @7 } =

ing on the argument values is producing a large enough set of 9 { D o c t o r ( o ): s e t D o c t o r ( o , /* addnl args */ )
test cases with different values for these arguments to ensure 10 || N u r s e ( o ): a d d N u r s e ( o , /* addnl args */ )};

adequate coverage of the system under test.

Strictly speaking, it should not be necessary for the tester to
Next, each other() call that appears in the PTCT must
provide this information since it is available from the sub-
be converted to a call to one of the methods of the appro-
contract (Fig. 9). But in those cases where multiple methods
priate application class, other than the ones mapped to the
of the same class may play the role of a named method, the
various named role methods. In addition, as in the case con-
TCS will have to specify which of those must be used. This
sidered above, if the method in question requires additional
is, of course, very much needed in dealing with other() calls
arguments, the TCS must provide these as well. And, as in
that appear in the PTCT since, in general, there will be many
the case considered above, we must account for any pre-
class methods that qualify as other methods. In the case of
conditions that may be imposed by the application class in
other*() calls, we must provide the sequence of method
question.
calls that should replace it and the arguments for each call
The final dimension of specialization of the PTCT that the
in the sequence. For example, suppose the other() call in
TCS must account for is concerned with a construct that may
line 5 of the PTCT in Fig. 5 was an other*() call. Then we
appear in a PTCT but, as it turns out, was not not used in
may have the following:
any of the PTCTs in the last section. This is the “other*();”
construct, which denotes a sequence of zero or more calls 8 o t h e r *() { @7 } =
to other methods. This construct may appear in a PTCT 9 { c h e c k V i t a l s (); c h e c k V i t a l s ();}
at points where the PTCT designer wants to allow for the
Here the other*() call is replaced by a sequence of two calls
possibility that a number of additional calls to other methods
to checkVitals(). We can also use the “||” notation to allow
may be needed in order to get the various objects into suitable
for alternative sequences of calls.
states for continuing the test. The TCS will have to specify
The detailed syntax for TCS is very much a work in
the sequence of method calls that must be made in place of
progress. For example, in the situation we just considered,
the other*() construct.
if the tester wanted to test a range of different sequences
Let us now see how these considerations play out when
of other calls, listing them all explicitly would be unde-
applied to some of the PTCTs (for the Observer pattern) when
sirable. It would be more convenient if we could use regu-
we specialize them to test the Hospital system. Consider the
lar expression-like notations for such situations. We intend
PTCT in Fig. 4. Line 2 of this PTCT requires an observer
to apply our approach to a number of patterns and systems
object to be created. Suppose the TCS designer decides that
designed using the patterns to determine the exact notational
this object should be a doctor. In the notation for TCS that
facilities that TCS must include. In addition, practitioner-
we are designing concomitantly with the design of JDUnit,
friendliness will be a key consideration since it is the testers
this may be expressed as:
who are part of individual software teams who will be respon-
8 O b s e r v e r () { @2 } = D o c t o r ( /* args */ ); sible for defining the TCSs for their particular systems.
On the other hand, given that both Doctor and Nurse play Before we conclude this section, one final point about the
the Observer role in this application, the TCS designer may Hospital system is worth briefly discussing. Suppose, dur-
want to create two test cases, one corresponding to each of ing system evolution, it is decided that the nurse objects
these possibilities. This is expressed in our notation as: observing a given patient object should keep track also of
8 O b s e r v e r () { @2 } = D o c t o r ( /* args */ ) the medicine-level, i.e., the value of the _medLvl variable
9 || N u r s e ( /* args */ ); of the Patient class. It would be easy enough to modify the
Nurse class to do so, perhaps by introducing a new variable,
Let us consider another statement that appears in the same
_medLevels, similar to the Nurse._vitals; and to revise
PTCT, the call to Attach() in line 7 of the PTCT in Fig. 4.
Nurse.update() so that the value of _medLevels as well
The following specializes that call so that it is replaced by a
as _vitals is appropriately updated.
call to the setDoctor() method:
While this seems simple and reasonable, if this is all we
8 A t t a c h ( o ) { @7 } = do, there will be a bug in this system! The problem is that, as
9 { s e t D o c t o r (o , /* a d d i t i o n a l args */ )}
it is implemented, the Patient class methods do not invoke
If, as we considered above, the observer that was con- notify() if only the _medLvl value changes. So in those sit-
structed earlier (line 2 of the PTCT) could be either a doctor uations where only this variable changes, the attached nurse
or a nurse, our notation for TCS allows the tester to spec- objects will have incorrect information about medLvl (until
ify which method must be used when replacing the call to the next time the patient’s _temp or _hrtRt changes, trig-
Attach() in line 7: gering a call to notify()).

123
Patterns: from system design to software testing 83

Unfortunately, however, the test cases generated using the can the work described here be new? The standard meaning of
PTCTs we have seen will not catch this error. The problem testing pattern refers to a pattern used to address a recurring
is that, according to the definition of the auxiliary concepts problem in the context of software testing. Indeed, a test(ing)
in Fig. 6, if only the value of _medLvl changes, the state pattern is simply a special type of design pattern [17], one
of the patient is not considered to be modified; moreover, intended to guide the design of testing code. Our work, how-
the state of nurse is considered to be consistent with that ever, is focused on validating the correct usage of design
of the patient as long as the former has the correct infor- patterns through software testing. We note that other authors
mation about the value of the _temp of the latter. Thus any have also considered applying design patterns as part of the
test cases generated from the PTCT in Fig. 5 will succeed system design process to improve testability [1,6]. Again,
when executed. If we were to change the definition of one however, our focus has been on ensuring the correct applica-
of these concepts but not the other, the constraint specified tion of patterns.
in the original pattern contract (lines 6–8 of Fig. 2) will be It is important to note that McGregor [17,18] extends the
violated. Since such a problem could easily arise during sys- traditional notion of a test(ing) pattern to patterns intended
tem evolution, it is important to ensure that our PTCTs check for use in validating design pattern implementations. The
that constraints on the auxiliary concepts are satisfied. At the concept is similar to our use of PTCTs in that each test
same time, the fact that these constraints are typically uni- pattern specifies a particular scenario of application objects
versally quantified over the states of various roles makes this that interact in ways intended to check pattern requirements.
a challenge. This is one of the questions we intend to address However, the scenarios are not based on precise pattern spec-
in future work. ifications. Equally important, McGregor does not consider
test suite specialization, nor automated generation. Still, the
similarities are important.
5 Related work Finally, it may be useful to mention the SABER system
described by Reimer et al. [22]. The system is designed to
The PTCT testing approach builds upon our prior work in detect defects in large Java applications using static analy-
pattern formalization [12,26,31]. The key conceptual ele- sis; SABER operates on systems designed using particular
ments of this work were summarized in Section 2. Several frameworks. Correct use of these frameworks requires that
other groups have also considered improving the precision of certain methods be invoked in certain orders, that others not
pattern descriptions. Some have described techniques based be invoked at certain points, etc. SABER attempts to iden-
on diagrammatic notations that extend standard UML(-like) tify violations of such requirements based on call sequence
syntax [7,21,32]. Closer to our specification approach is that rules specified using an XML dialect. Given the importance
of Eden et al. [9,10]; the authors describe an approach to cap- of call sequence requirements in pattern-based systems,
turing the structural properties of patterns using a logic nota- elements of SABER may be useful in detecting pattern
tion. Patterns are expressed as logic formulae that specify violations statically, prior to pattern testing. We intend to
participating classes, methods, inheritance hierarchies, and explore this possibility as part of future work.
the structural relations among them. Mikkonen [19] describes
a behavioral specification approach based on DisCo [14], a
UNITY-like [5] action system notation. Data tuples represent 6 Conclusion
objects, and guarded actions model their interactions. Helm
et al. [13] describe a contract formalism that bears similarity We began with the observation that design patterns play
to ours. In particular, the formalism has constructs analogous a central role in software practice, influencing the design
to our role contracts, auxiliary concepts, and a limited form of of most major software systems and class libraries. Conse-
trace conditions. The formalism additionally provides some quently, techniques for ensuring the correctness of pattern
support for contract specialization and system-level reason- implementations are likely to have a profound impact on the
ing. However, despite the effort invested in documenting pat- reliability of real-world systems. To this end, we focused
terns precisely, there appears to be relatively little prior work on an approach to testing pattern implementations based
in testing the correctness of systems constructed according on precise pattern specifications. Whereas our prior work
to particular pattern descriptions, either formal or informal. provides the specification foundation, the work reported
By contrast, this has been the focus of our work. here provides the foundation for specification-based testing
Some readers may take note of the fact that the phrase of pattern implementations.
“test(ing) patterns” appears throughout the literature. In developing the testing approach, we observed that pat-
Indeed, there is at least one workshop series devoted to testing tern implementations follow a similar structure. This is not
patterns [28–30], a corresponding online repository [16], and surprising. After all, for a designer to claim that a particu-
widely referenced books that detail the subject [3,18]. So how lar group of classes have been designed according to a given

123
84 N. Soundarajan et al.

pattern, she must have satisfied certain requirements— during testing, use PCT to generate appropriate test cases.
namely, those common to all applications of the pattern in Hence, PCT and TDD are fully compatible, but orthogonal
question. It then follows that the test cases used to vali- testing principles.
date the pattern’s implementation share a common structure
across systems designed using the pattern. The central moti-
vation of our testing approach is that this common structure,
References
applicable to all systems designed using the pattern, should
be reusable. In defining the test cases for a particular system, 1. Baudry B, Le Sunyé Y, Jézéquel J (2001) Towards a ‘safe’ use of
testers should only be required to invest the incremental effort design patterns to improve OO software testability. In: The 12th
necessary to specialize the generalized structure as appropri- international symposium on software reliability engineering, IEEE
Computer Society, Washington, DC, pp 324–329
ate to the system under test. Moreover, the ability to define
2. Beck K, Gamma E (1998) Test infected: programmers love writing
reusable test case structures enables the software engineer- tests. Java Rep 3(7): 37–50
ing community to encode test case families that are likely to 3. Binder R (1999) Testing object-oriented systems. Addison-
reveal the most common pattern implementation errors. Wesley, Menlo Park
4. Buschmann F, Meunier R, Rohnert H, Sommerlad P, Stal M
We described a testing approach that satisfies these goals,
(1996) Pattern-oriented software architecture: a system of patterns.
and the design of a software tool used to assist in test case gen- Wiley, New York
eration. In our approach, pattern test case templates (PTCTs) 5. Chandy K, Misra J (1988) Parallel program design. Addison-
are used to define generalized test case structures reusable Wesley, Menlo Park
6. Dasiewicz P (2005) Design patterns and object-oriented software
across applications of particular patterns. Each template is
testing. In: The 2005 Canadian conference on electrical and com-
expressed using a Java-like syntax amenable to practitioner puter engineering, IEEE Canada, Dundas
adoption; the code is written in terms of the role types and role 7. Dong J (2002) UML extenstions for design pattern compositions.
methods specified by the corresponding pattern specification. In: Mingins C (ed) Proceedings of TOOLS, in special issue of
journal of object technology, vol 1, issue 3, pp 149–161
To test the behavior of a system implemented using a partic-
8. Dong J, Alencar P, Cowan D (2001) A behavioral analysis
ular pattern, the corresponding PTCTs must be instantiated approach to pattern-based composition. In: Proceedings of the 7th
to generate executable test cases. The instantiation process international conference on object-oriented information systems,
is guided by a specification of the mappings between pat- Springer, pp 540–549
9. Eden A (2001) Formal specification of object-oriented design. In:
tern roles and system classes, as well as rules that govern
Proceedings of the international conference on multidisciplinary
system-specific specializations. JDUnit assists in the instan- design in engineering
tiation process by guiding developers in selecting the appro- 10. Eden A (2002) LePUS: a visual formalism for object-oriented
priate PTCTs and pattern specifications, and in introducing architectures. In: Proceedings of the 6th world conference on inte-
grated design and process technology, IEEE Computer Society,
application-specific objects, method calls, and arguments.
pp 149–159
The output of the tool is a set of JUnit test cases used to 11. Gamma E, Helm R, Johnson R, Vlissides J (1995) Design patterns:
validate the correctness of the relevant patterns as applied in elements of Reusable OO Software. Addison-Wesley, Menlo Park
the system under test9 . 12. Hallstrom J, Soundarajan N, Tyler B (2006) Amplifying the ben-
efits of design patterns. In: Aagedal J, Baresi L (eds) Proceedings
We conclude with a short comment on the relationship
of the 9th international conference on fundamental approaches to
between our approach to pattern-centric testing (PCT), and software engineering (FASE), Springer, pp 214–229
test-driven-development (TDD) [2]. In TDD, test cases that 13. Helm R, Holland I, Gangopadhyay D (1990) Contracts: specifying
capture the expected behavior of a component are written behavioral compositions in object-oriented systems. In: OOPSLA-
ECOOP, pp 169–180
before the component is implemented. Hence, TDD dictates
14. Järvinen H, Kurki-Suonio R (1991) Disco specification language:
when test cases must be designed. On the other hand, PCT is marriage of actions and objects. In: Proceedings of the 11th inter-
concerned with what must be tested. One could, as dictated by national conference on distributed computing systems, IEEE Com-
TDD, develop our test cases before a system is implemented. puter Society, Los Alamitos, pp 142–151
15. Jones C (1990) Systematic software development using VDM.
(Of course, relevant portions of the design must be complete
Prentice-Hall, Englewood Cliffs, New York
since otherwise we would not know which PTCTs to instan- 16. Marick B (2007 (date of last access)) Test patterns repository/
tiate.) Alternatively, following a more conventional process, software testing patterns. http://www.testing.com/test-patterns/
we may design and implement the system first, and then, patterns/
17. McGregor J (1999) Testpatterns: please stand by. J Object-oriented
Program 12:14–19
9 A key guideline for using the JUnit framework is that the tests should 18. McGregor J, Sykes D (2001) A practical guide to testing object-
be designed to test behavior rather than individual methods [20]. This oriented software. Addison-Wesley, Menlo Park
also applies to our approach, except that the behaviors in question are 19. Mikkonen T (1998) Formalizing design patterns. In: Proceedings
those corresponding to the use of a particular pattern. Hence, the PTCTs of 20th ICSE, IEEE Computer Society Press, pp 115–124
typically involve multiple roles. Consequently, the instantiated test cases 20. Rainsberger J (2005) JUnit recipes. Manning
involve multiple system classes, by contrast to more traditional unit 21. Reenskaug T (1996) Working with objects. Prentice-Hall, Engle-
tests. wood Cliffs

123
Patterns: from system design to software testing 85

22. Reimer D, Schonberg E, Srinivas K, Srinivasan H, Alpern B, 27. Soundarajan N, Hallstrom J (2006) Pattern-based system evolu-
Johnson R, Kershenbaum A, Koved L (2004) “saber”: smart analy- tion: a case-study. In: Zhang K, Spanoudakis G, Visaggio G (eds)
sis based error reduction. In: Proceedings of “ISSTA ’04”, ACM Proceedings of 18th international conference on software engineer-
Press, pp 243–251 ing and knowledge engineering (SEKE 2006), Knowledge Systems
23. Riehle D (1997) Composite design patterns. In: Proceedings of Institute, pp 321–326
OOPSLA, ACM, pp 218–228 28. Testing (2001) Patterns of software testing 1
24. Schmidt D, Stal M, Rohnert H, Buschmann F (1996) Pattern- 29. Testing (2001) Patterns of software testing 2
oriented software architecture: patterns for concurrent and net- 30. Testing (2001) Patterns of software testing 3
worked objects. Wiley, New York 31. Tyler B, Hallstrom J, Soundarajan N (2006) A comparative study of
25. Shalloway A, Trott J (2002) Design patterns explained. Addison- monitoring tools for pattern-centric behavior. In: Hinchey M (ed)
Wesley, Menlo Park Procedings of 30th IEEE/NASA sofware engineering workshop
26. Soundarajan N, Hallstrom J (2004) Responsibilities and rewards: (SEW-30), IEEE-Computer Society
specifying design patterns. In: Finkelstein A, Estublier J, 32. Vlissides J (1998) Notation, notation, notation. C++ Report
Rosenblum D (eds) Proceedings of 26th international conference
on software engineering (ICSE), IEEE Computer Society, pp 666–
675

123