See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/308541385

Development of Complex Software with Agile Method

Conference Paper · August 2015

DOI: 10.1109/Agile.2015.18

CITATIONS READS
2 289

3 authors:

Alan Braz Cecília M. F. Rubira

IBM University of Campinas
10 PUBLICATIONS 31 CITATIONS 120 PUBLICATIONS 1,654 CITATIONS

SEE PROFILE SEE PROFILE

Marco Vieira
University of Coimbra
251 PUBLICATIONS 2,842 CITATIONS

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Agile@Research View project

design methods for finance View project

All content following this page was uploaded by Alan Braz on 10 April 2018.

The user has requested enhancement of the downloaded file.

 2015 Agile Conference

Development of Complex Software

with Agile Method

Alan Braz Cecı́lia M. F. Rubira Marco Vieira

IBM Research - Brazil Institute of Computing Department of Informatics
Avenida Tutóia 1157, 04007-005 State University of Campinas Engineering
São Paulo, SP, Brazil P.O. Box 6176, 13083-852, University of Coimbra
alanbraz@br.ibm.com Campinas, SP, Brazil 3030, Coimbra, Portugal
cmrubira@ic.unicamp.br mvieira@dei.uc.pt

Abstract—Agile Software Development (ASD) has been on This article proposes a solution named Scrum+CE to guide
mainstream through methodologies such as XP and Scrum the development of robust component-based systems that adds
enabling them to be applied in the development of complex some practices of MDCE+ to the Scrum framework. These
and reliable software systems. This paper is the end result of practices affect the Pregame and Game phases in the aspect of
the Master’s dissertation of the main author, and proposes a discovering and modeling exceptional conditions in the format
solution to guide the development of complex systems based on
components by adding exceptional behavior modeling practices
of Exceptional Stories and more detailed Acceptance Tests.
to Scrum, resulting in the Scrum+CE method (Scrum with Scrum+CE also inserts a required artifact of the High Level
Exceptional Behavior). Architecture and the new role of Architecture Owner.
In order to evaluate the proposed method, a synthetic con- The methodology used to validate the proposed solution
trolled experiment was conducted with three groups.We com- was Synthetic Environment Experiments [6], where a smaller
pared the efficiency of the new process in relation to plain Scrum version of Scrum and Scrum+CE were executed. This reduc-
and the results were the production of a better quality software tion had to be made due to the availability of human resources
but with less features implemented during the same amount of and time to execute the experiment.
time.
The validation hypotheses were that using Scrum+CE
Keywords—Agile Software Development, Scrum, Component- would result in the delivery of (i) less Story Points, since the
Based Development, Reliability, Exception handling, Software En- requirements are more detailed; and (ii) better quality of the
gineering
final software in terms of less defects, once it will drive to a
more robust exception handing code.
I. I NTRODUCTION
The Software Engineering has been suffering changes in II. R ELATED WORK
its methods and practices due to the growing need to develop Radinger and Goeschka [7] proposed an approach to in-
systems in shorter periods with lower costs and satisfying qual- tegrate the ASD and the CBD in small and large projects
ity. By contrast, the complexity of these systems is increasing by combining the technical and organizational issues of both
as well as the need for them to be reliable. Dependability approaches.
is no longer a mere nonfunctional requirement just inherent
to critical systems, such as a flight controller or a financial Nord and Tomayko [8] explored the relationships and syn-
system, but of all software systems that require a certain ergies between design and analysis focused on the architecture-
robustness to not expose sensitive information and maintain centric method and XP Agile methodology, noting that the
a service dependable as much as possible. To address these latter emphasizes the rapid and flexible while the first priority
issues, ideas like Component-Based Development (CBD) [1] to the design and infrastructure.
and Architecture-Centric Development [2] have been applied Behrouz [9] have discussed and shown that despite the dif-
with relative success on developing more reliable software. ferent goals, a combination of Software Reliability Engineering
However, there is a trend of using “lighter” methods of (SRE) and the Agile practice of Test Driven Development
development and project management software. This set of (TDD) have improved the reliability of the developed software.
principles and practices called lightweight were named Agile In this case, we did not focus on TDD approach to increase
Methods or Agile Software Development (ASD) methods by dependability, but we achieved it by adding elements of
several professionals who met in 2001 to formalize what had exception handling at early phases of development process.
already been doing for some years, as Extreme Programming
(XP) [3] and Scrum [4], resulting in the Agile Manifesto [5]. III. BACKGROUND
A. Agile Method
Nevertheless, this simplicity is often confused with infor-
mality and lack of rigor, especially regarding the activities Agile Software Development (ASD) is a set of methodolo-
of modeling, documentation of functional and nonfunctional gies guided by four values and twelve principles defined by
requirements and architectural design. the Agile Manifesto [5]. These values are:

978-1-4673-7153-7/15 $31.00 © 2015 IEEE 97

DOI 10.1109/Agile.2015.18
 1) Individuals and interactions over processes and tools !"#$%&'()*'+$)&','#-&'()*-)+*+$,')'&'()*
(,*&.$*$"#$%&'()-/*0$.-1'(2*')*&.$*,(23*
5(3%()$)&4*'3%/$3$)&-&'()*-)+*62-%*
#2$-&'()*,(2*&.$*2$74$+*#(3%()$)&4
2) Working software over comprehensive documentation (,*$"#$%&'()-/*4&(2'$4
3) Customer collaboration over contract negotiation A$4#2'%&'()*(,*
5())$#&(24*'3%/$3$)&-&'()
4) Responding to change over following a plan $"#$%&'()-/*-44$2&'1$
C

%B
The twelve principles expand the values in more details by 2-

(
%

$/
$1
!"#$%&# !'()$%&#
giving more emphasis to the left items of values than the right

A
;*</-))')= ;*?)&$=2-&'()*&$4&4
*%&#
ones. ;*8:4&$3*-2#.'&$#&72$>
.'=.*/$1$/*+$4'=)
;*@')-/*62-%
;*5/(472$
8%2')&4
B. Scrum Methodology 9

6
+E

'$
7

1
4&

D$
Scrum is an agile software management framework, that 8$%-2-&'()*(,*#()#$2)4*
promotes an iterative and incremental development. It was 0$&6$$)*)(23-/*-)+*
$"#$%&'()-/*#(3%()$)&4 9)-/:4'4*(,*&.$*$"#$%&'()-/* B!"#$%&'()!*#*%+)"),
introduced in 1995 by Schwaber [4] proposing a process ,/(6*-)+*#-&#.$24*2$,')$3$)& '()"-#!*#'!$.'"#-

with three phases: Pregame (Conception or Initiation), Game

(Development) and Postgame (Closure or Rollout).
Fig. 1. MDCE+ practices affecting Scrum phases. Extended from [4]
Despite the Scrum has evolved a lot since the Schwaber’s
1995 [4] paper in terms of removing the software engineer- TABLE I. R ELATION BETWEEN MDCE+ AND S CRUM PHASES .
ing practices and focusing on team organization and project
management, for the sake of this work, we choose to use the Scrum Phases Scrum Events MDCE+ Phases
1. Requirements specification and
first academic reference due to the need to handle exceptional Pregame
Planning
analysis
behavior at the architecture level. This decision was made 2. Management aspects definition
because the High Level Architecture, in our proposed solution, System architecture / 3. Architecture design
high level design
must begin before the first development Sprint, there is, at the 1. Requirements specification and
Pregame. Sprint Planning
analysis
3. Architecture design
Game 4. System analysis
C. Methodology for Exceptional Behavior Definition 5. System design
(MDCE+) Sprint
6. Components implementation
7. Components integration
Ferreira [10] presented a methodology for building fault- Sprint Review
1. Requirements specification and
tolerant systems using techniques of exception handling.This analysis
8. User-acceptance release
methodology is named MDCE, acronym in Portuguese for Integration tests 7. Components integration
“Methodology for the Definition of Exceptional Behavior”. Postgame Wrapping
8. Production Release deploy
It extends the Unified Modeling Language (UML) with new Closure

stereotypes.
Brito [11] extended MDCE defining MDCE+ which aims A. Pregame
to systematize the modeling and implementation of exceptional
behavior in the development of component-based systems. 1) Planning: Besides the regular activities of the Planning
The emphasis on architecture has enabled a better analysis phase of Scrum there were added the following activities (i)
of the flow of exceptions that occur between architectural identify the exceptions and define the exceptional behavior in
components, resulting in efficient handlers, and anticipating the form of Exceptional Stories and (ii) describe the excep-
the correction of possible specification faults. tional assertive in the form of Acceptance Tests either at the
normal behavior User Stories or at the introduced Exceptional
IV. T HE P ROPOSED SOLUTION Stories.
The proposed solution is an adapted agile development During the review of the Product Backlog, all stories should
process based on Scrum process that adds some MDCE+ be revisited in order to add extra Acceptance Tests, derived
practices and techniques to increase the robustness of the de- from the Exceptional Stories, resulting in a formalization of
veloped complex product. The name of this proposed process the exceptional assertive and in consequence generating a more
is Scrum+CE (the CE comes from Exceptional Behavior in robust test set.
Portuguese) and it adds practices at the Pregame and Game
phases of Scrum in order to raise, detail and document the 2) Definition of “Done”: Exceptional behavior should also
exceptional behavior in the format of Exceptional Stories be explicit regardless the definition agreed by the entire team.
and Acceptance Tests more judicious inside the regular User The Architecture Owner has the responsibility of adding the
Stories [12], and also refines and documents the architecture following to it: “. . . and all exceptions were properly han-
highlighting the exceptional components. dled. . . ”.
Figure 1 shows the phases of Scrum with the activities 3) System architecture / high level design: Just like Plan-
from MDCE+ in gray. Scrum+CE doesn’t add new phases to ning, this phase still formatted by the Scrum activities plus
Scrum, therefore it still has Pregame, Game and Postgame. a new role, the Architecture Owner, and a new mandatory
Table I shows the relation between the phases of Scrum and artifact, the High Level Architecture document. Both additions
MDCE+. Note that due to the iterative aspect of the Game have brought the concepts of Architecture-Centric Develop-
phase, composed by sprints, some MDCE+ phases will be ment to Scrum+CE. The architecture has to follow the concepts
repeated. of CBD and have at least a component-level architecture

98
diagram, the most important architectural decisions, used tech- The Pregame and Postgame phases were done by the
nologies and frameworks, and an initial data model. It is author of this paper, while the Game phase was done by
recommended to document it in a collaborative tool, like a the participants. The three implemented source-codes were
Wiki. This would allow the document to be highly available, available to the execution of functional tests and the collection
flexible and support the Agile Manifesto principle that states: of metrics used further in the result analysis.
“The best architectures, requirements, and designs emerge
from self-organizing teams”. A. Pregame execution

B. Game In this phase it was defined the product Vision and Product
Backlog with all stories prioritized and estimated in Story
Scrum+CE is also an iterative process in the format of Points. The role of Architecture Owner of Scrum+CE was
time-boxed sprints with length between two and four weeks conducted by the author for the groups G2 and G3, as well
that repeat continuously until the implementation of every as the roles of Product Owner and Scrum Master for all
requirement of the Product Backlog or the Product Owner three groups. This participation was necessary to control the
declares that the current Increment is valuable as a final variables, requirements, artifacts and adherence to processes
product. With that, the Scrum events, structure and activities by groups, thus, maintaining the validity of the experiment.
remain the same. Every artifact was presented to each group independently.
The MDCE+ practices, highlighted at Figure 1, will affect 1) Scrum groups division: The participants were invited to
the Product Backlog with the addition of Exceptional Stories, participate in a “Scrum in Practice” training. they had to fill
and the Sprint Backlog, with the related tasks dedicated to out a registration form composed by knowledge and experience
the implementation of exceptional behavior. Once inside the questions about Agile, Scrum and Java development and 12
backlogs, either exceptional stories or tasks will the treated by participants were selected. It was created a technical score
the Development Team as ordinary ones. for each participant. The scores were distributed among three
groups. The scores from G1 to G3 respectively were: 253
C. Postgame (36%), 225 (32%) and 233 (33%). G1 had the highest relative
The Postgame still with the wrapping and preproduction score so it was intentionally choose to be the control group.
activities to the end-user release and did not have any change 2) System architecture / high level design: The architecture
in relation to Scrum. of the system was presented for the three groups during the
first Sprint Planning Meeting and followed a simple four layer
V. VALIDATION model as illustrated at Figure 2. In this case, the layers User
Session and System Services were unified and further layers
In order to validate the benefits and applicability of
of server side followed the MVC [13] design pattern. This
Scrum+CE, we designed a controlled experiment in which
architecture was used in Scrum+CE groups G2 and G3. Even
a information-based software system, with dependability re-
not required in Scrum, the same diagram was given to the
quirements relating to data consistency, were implemented.
control group G1, but without the two exceptional components.
The experiment consists of an object (P1), that is, the software
system in Java for the web platform along with its predefined !"#$%&#""'()%*%
9,&/:;'-&/<87& "=,4'&,,:>&/?47&, !8-838,&
architecture following the Model-View Controller (MVC) [13] &+",#-%&#$.'/#"

pattern, and three subjects, that is, three professional groups (>$@

(G1, G2 and G3) with experience this technology that will be "/01,&/
5'-4-4&,
part of the Development Team, as described by Table II;
ABB) +&,-.&- $%&'()*
!"#
TABLE II. E XPERIMENT MODELING .
2034.&
Object G1 G2 G3
P1 O1: Scrum O2: Scrum+CE O3: Scrum+CE 567&%-40'8. 567&%-40'8.
)/&,&'-8-40' )&/,4,-&'7&

Using this format, G1 was the control group, and conse-

quently G2 and G3 were the experimental groups. This kind of Fig. 2. Component-based software architecture diagram exposing the
experiment followed a controlled method called Synthetic En- exceptional components used by the experimental groups G2 and G3.
vironment Experiments [6] where a reduced version, in terms
of duration time, of Scrum and Scrum+CE were executed. From the CBD standpoint, each layer was treated as a com-
So each Sprint will last 1 week (4 hours per day during 5 ponent, and the goal of Game phase was the implementation
coonsecutive days), with a 2-hour virtual work-day including of such components.
a 3-minute Daily Scrum meeting.
B. Game execution
All the professionals selected to implement the project of
the experiment have at least 3 years of working experience The experiment itself was implement in a format of a 40-
developing software in Java and basic training in Agile. The hour Scrum in Practice training. The time was divided in 2
fact that all participants were proficient in Java and Scrum, sprints of 20 hours, and each sprint was composed by 7 virtual
helped to strengthen the internal validity of the experiment, days of 2 hours each. With that, it was possible to simulate
that is, the results would be influenced by the development rigorously the Scrum and Scrum+CE processes. The activities
process instead of their previous knowledge. of the three groups were conducted in parallel with a difference

99
of about ten minutes between a group and another due to was created several test cases, based on the Acceptance Tests,
movement of the author between the rooms. for each delivered story.
1) Sprint 1: At the beginning of the experiment, all partici- The metrics collected in this phase were divide in three
pants were gathered in the same room where they were notified categories: (i) number of requirements; (ii) quality of require-
about their respective groups and directed to a exclusive ments; and (iii) quality of code.
meeting room. After that the groups started the Sprint Planning 1) Requirements delivered metrics: A User Story was only
Meeting and then followed the schedule described at Table III. considered completed if it respected the Definition of “Done”.
Since no initial code was provided, all groups selected only Table IV shows the entire Product Backlog with the stories
two User Stories to be implemented in this first Sprint. delivered by each group and its correspondent Story Points.
At the end of the fifth day, playing the role of Product TABLE IV. U SER S TORIES DELIVERED BY GROUP.
Owner, the author made an individual Sprint Review Meeting
with each group when they demonstrated the features imple- Story Points G1 G2 G3
1 20
mented during this sprint. Only G1 delivered both stories. The 2 5
•
•
•
•
•
•
other two groups, delivered only one story each. 3 8 • • •
4 3 • • •
So at the end of this sprints the Product Backlog of 5 3 • • •
the groups was different. At this point, G1 had two stories 6 2 • •
7 1 • • •
completed, and G2 and G3 only one. 8 3
9 5 • • •
TABLE III. S CHEDULE OF S PRINT 1. 10 2 •
11 2
Day Activity Duration
12 5
Reception and organization 2 hours 13 2
1
Planning Meeting 2 hours Total stories 9 7 8
Daily Scrum 3 minutes Total points 49 45 47
Implementation day 1 2 hours
2
Daily Scrum 3 minutes
Implementation day 2 2 hours 2) Requirements quality metrics: Once the architecture was
Daily Scrum 3 minutes designed to interact with the system using HTTP requests, it
Implementation day 3 2 hours
3
Daily Scrum 3 minutes was built a test suite to run the same set of tests against the
Implementation day 4 2 hours different codes automatically. Table V consolidates the results
Daily Scrum 3 minutes of the functional tests showing the number of tests executed
Implementation day 5 2 hours
4
Daily Scrum 3 minutes against each group code, the amount of failed tests (defects)
Implementation day 6 2 hours and the fail rate.
Daily Scrum 3 minutes
5
Implementation day 7 2 hours TABLE V. N UMBER OF TESTS EXECUTED AND DEFECTS FOUND BY
Review Meeting 2 hours GROUP.

Metric G1 G2 G3
2) Sprint 2: It had a slightly different schedule but followed Tests executed 189 149 161
the same structure of Table III besides the first and last days. Failed tests 66 44 21
Day 1 was composed by a Planning Meeting of 2 hours, a Fail rate 35% 30% 13%
Daily Scrumof 3 minutes and the Implementation day 1 with
2 hours. Day 5 had a Review Meeting with 3 hours and a 3) Code quality metrics: The delivered codes were submit-
Retrospective that took 1 hour. ted to a static analysis in order to evaluate the code quality.
It was used an automated and open-source toll called Sonar
In this sprint, the groups started the Sprint Planning [14]. Table VI shows some relevant metrics provided by this
Meeting reviewing their own updated Product Backlog and tool.
selecting the stories that they would commit with the Product
Owner to deliver at the end of Sprint 2. At the last day, we TABLE VI. C ODE QUALITY METRICS .
gather all participants at the same room and made a collective Metric G1 G2 G3
Sprint Review Meeting. Every group presented the results of Lines of Code (LOC) 2232 1984 1950
their implementation and the author, again as Product Owner, Number of Classes 27 47 38
Number of Exceptions 1 21 2
validated them all. The list of the stories delivered by each Native catch blocks 53 18 33
groups is at Table IV. Created catch blocks 9 12 6
Cyclomatic Complexity (CC) [15] 446 305 288
The Sprint Retrospective was a collective session where we CC by class 15.9 4.5 7.2
discussed how was the feeling of using an agile method with CC by method 5.0 2.4 2.5
time-boxed events to develop complex software. Then it was
reveled to the participants that the process used by the groups
VI. R ESULTS ANALYSIS
were different.
Due to the size of the experiment it was not possible
C. Postgame execution to make a statistical analysis of the collected data. Thus, a
quantitative analysis was performed comparing the control
The first activity of this phase was to build a set of group (G1) and each of the experimental groups (G2, G3),
functional tests to run against each source code available. It resulting in two sets of results: G1-G2 and G1-G3.

100
 A. Requirements delivered VIII. C ONCLUSION
Figure 3 shows the comparison between the User Stories Despite the size of the experiment, in terms of number
and Story Points delivered. of participants and duration, we couldn’t apply any statistical
analysis. So we were unable to generalize this results to every
kind and size of complex software project with some relia-
6()*7 6+7 bility requirement. During this work we realize how hard and
68*)*7 697
,86,'
expensive it is to make software engineering experimentation
68()*7
6887
,86,: due to the need of people availability for a long period of time.
6'*)*7
Even with this challenges, we considered a great experience
6''7 during the author’s master degree dissertation work.
6'()*7
!"#012$%03#" 2$%0&14%35$"
The same experiment could be replicated with more groups
Fig. 3. Stories relation between G1 and G2, and between G1 and G3. or even bigger groups, implementing the same project require-
ments, for the same time-frame, in order to collect more data,
Analyzing these numbers, Scrum+CE resulted in fewer and eventually make it possible some sophisticated statistical
Story Points (6.1% on average) delivered once the two exper- analysis at the results. This is a suggestion of future work that
imental groups gave fewer stories (16.7% on average) which would reinforce this study.
was expected and should be compensated with a better code This work also inspired the software engineering teachers
quality. of our institute to remodel the software engineering practice
Another metric that supports this issue is the amount of courses of the Computer Science and Computer Engineering
Lines of Code (LOC) of the delivered systems, presented in undergraduate courses. We supported this changes at the
line 1 of Table VI. It highlights that the code delivered by G1, course agenda and execution for both 2014 semesters obtaining
which implemented a greater number of stories, was higher amazing results and feedbacks from the students and teachers.
than the other groups. The relative size of G2 to G1 was These experiences will be published in the near future.
11% smaller, and to G3, 13% smaller. With that, the size of
the system in LOC is directly proportional to the amount of R EFERENCES
functionality delivered. [1] C. Szyperski, Component Software: Beyond Object-Oriented Program-
ming. Boston, MA, USA: Addison-Wesley Longman Publishing Co.,
Inc.
B. Requirements quality [2] I. Sommerville, Software Engineering, 9th ed. Harlow, England:
Addison-Wesley, 2010.
The amount of tests executed, listed at Table V, were
[3] K. Beck, Extreme Programming Explained: Embrace Change, US ed.
directly proportional to the delivered requirements at Table IV. Addison-Wesley Professional, Oct.
Since only delivered User Stories were tested, G2 and G3 were [4] K. Schwaber, “SCRUM Development Process,” in Proceedings of
exposed to fewer amount of tests. the 10th Annual ACM Conference on Object Oriented Programming
Systems, Languages, and Applications (OOPSLA), pp. 117–134.
The fail rate is the metric that best represents the results of
[5] K. Beck et al. (2001) Manifesto for Agile Software Development.
the experiment and helps to validate the hypothesis in which Accessed: 17 Apr 2015. [Online]. Available: http://agilemanifesto.org
the use of Scrum+CE over Scrum would increase the “code [6] M. V. Zelkowitz and D. Wallace, “Experimental Validation In Software
quality generated by decreasing the number of defects”. The Engineering,” Information and Software Technology, vol. 39, pp. 735–
fail rate of each group was 35%, 30% and 13% respectively 743, 1997.
for G1, G2 and G3, as shown in Table V. To support this [7] W. Radinger and K. M. Goeschka, “Agile software development for
analyses, we can rely on Table VI to check that G2 and G3 component based software engineering,” Companion of the 18th annual
took different approaches to design the exceptions (Number ACM SIGPLAN conference on Object-oriented programming, systems,
languages, and applications, Oct. 2003.
of Exceptions and Created catch blocks). In other hand, they
[8] R. L. Nord and J. E. Tomayko, “Software Architecture-Centric Methods
delivered equivalent size in terms of Lines of Code (LOC) and and Agile Development,” IEEE Software, vol. 23, no. 2, pp. 47–53, Mar.
Cyclomatic Complexity (CC). 2006.
[9] B. Far, “Software Reliability Engineering for Agile Software Develop-
Considering that a failed test generates a defect, these ment,” 20th IEEE Canadian Conference on Electrical and Computer
results indicate a validation of the hypothesis, since the ex- Engineering CCECE, pp. 694–697, 2007.
perimental groups had lower fail rates than the control group. [10] G. R. M. Ferreira, “Tratamento de exceções no desenvolvimento de
Therefore, the usage of Scrum+CE promoted the creation of a sistemas confiáveis baseados em componentes,” Master’s thesis, IC,
better quality code when compared to G1. Unicamp, Dec. 2001.
[11] P. H. S. Brito, “Um Método para Modelagem de Exceções em Desen-
volvimento Baseado em Componentes,” Master’s thesis, IC, Unicamp,
VII. C ONSOLIDATED RESULTS Oct. 2005.
[12] M. Cohn, User Stories Applied: For Agile Software Development.
As a result of the experiment, we demonstrated that the Addison-Wesley, 2004.
expectation over the method Scrum+CE was proved once there [13] S. Stelting and O. Maassen, Applied Java Patterns. Prentice Hall
was delivered 6.1% less Story Points than using Scrum, but Professional, 2002.
with a decrease of 13.5% in the number of defects found. [14] SonarQubeTM . Accessed: 16 Oct 2012. [Online]. Available: http:
Therefore, using the Scrum+CE delivered less Story Points //www.sonarsource.org
but with a better quality code (fewer defects) in relation to the [15] T. J. McCabe, “A Complexity Measure,” IEEE Trans. Software Eng.,
development that used the Scrum. vol. 2, no. 4, pp. 308–320, 1976.

101

View publication stats