Agility at Scale

Economic Governance, Measured Improvement, and Disciplined Delivery

Alan W. Brown Scott Ambler Walker Royce

AbstractAgility without discipline cannot scale, and In our experience, both within IBM and in the broader
discipline without agility cannot compete. Agile methods are now industry, many change-averse groups perceive agile processes
mainstream. Software enterprises are adopting these practices in as the results of shoddy planning, poor management, and
broad, comprehensive delivery contexts. There have been many chaotic delivery schedules. Consequently, these organizations
successes, and there have been disappointments. IBM’s become increasingly polarized. The process progressives are
framework for achieving agility at scale is based on hundreds of advocating dynamic planning, steering leadership, rapid
successful deployments and dozens of disappointing experiences delivery cycles, integration-first milestones, and empowered
in accelerating software delivery cycles within large-scale teams. They demand change. The process traditionalists
organizations. Our collective know-how points to three key defend their reliance on plan-and-track leadership, overly
principles to deliver measured improvements in agility with high precise plans and specifications, integration-later milestones,
confidence:
and hierarchical teams. They resist change.
1. Steer using economic governance
As grassroots success starts to spread, the supporting
2. Measure incremental improvements honestly
constituencies (the resource managers, financial managers,
3. Empower teams with disciplined agile delivery
sellers, and other support teams in a software business) find
This paper elaborates these three principles and presents
practical recommendations for achieving improved agility in
themselves to be outside resistors. They exaggerate the
large-scale software delivery enterprises. challenges that agile approaches inflict on their traditional
mindset and techniques. For example, they may observe less
Index TermsSoftware process improvement; accelerating precise up-front commitments to resources, less long-term
software delivery; agile development; integration first; economic planning, new progress/quality measures that don’t align with
governance; measured improvement; steering leadership; company baselines, and so forth. They see new techniques as
disciplined agile delivery. problems for their localized turf, not as a source of possible
benefits for the larger business and organizational efficiency.
I. INTRODUCTION In the worst cases, the result of these misaligned teams is
The principles of agile software development are now that agile adoption stalls, stifled by middle management —
more than 10 years old [1]. Achieving agility at scale, project managers, business analysts, contract negotiators, and
however, demands process optimization among groups of others. These supporting stakeholders do not feel empowered
stakeholders as well as applying agile principles in context. by agile approaches; they feel threatened and out of control.
Many practical ideas have been described, piloted, practiced, IBM has worked with hundreds of clients who are
and evolved. Agile practices are now neither novel nor introducing modern agile techniques into their enterprise-
extreme. However, applying agile approaches at large scale scale software delivery approaches. We have found that
remains fraught with complexity, just as it has been for past success in achieving agility at enterprise scale requires three
eras of significant software process improvements. foundational principles:
Most agile adoption efforts begin with small teams of 1. Economic governance. An objective economic
software developers introducing agile practices. These foundation for planning, decision-making, and
practices are not part of any formal organizational progress reporting that resolves uncertainties earlier
transformation; they are simply team efforts to deliver good and unifies constituencies on a shared set of expected
software effectively. When the results are good, other small target outcomes.
software development teams quickly adopt similar practices. 2. Measured improvement. A more honest approach to
Despite early enthusiasm and grassroots success for agile measuring progress and quality outcomes by
practices, agile proponents face barriers to broader acceptance quantifying change trends in the code and test base.
when they come up against organizational challenges in 3. Disciplined agile delivery. A process decision
scaling, governing, and institutionalizing. All too often, the framework that defines a set of practices, measures,
organization rushes into an enterprise-wide agile and know-how. Discipline ensures progress is goal-
improvement program and then confronts disappointment, driven, using a hybrid approach to IT solution
skepticism, and a shortfall in quantified results. delivery.
 First, we base the framework on an economic model for large-scale software delivery team circumstances.
software delivery that recognizes the variations in delivery Interesting examples include IBM [13], Yahoo! [14],
methods that result from using a mix of waterfall, iterative, and and TCS [15]. Each of these case studies emphasizes
agile techniques. This allows us to reason about where, how, the organizational, cultural, and financial challenges
and when agile approaches can provide real benefits to an that must be overcome in specific situations.
enterprise organization. Second, we introduce an incremental, 4. Extending agile principles to broader areas of the
measured approach to agile software delivery improvement. system lifecycle, such as requirements definition [16]
This forms the context for scalable deployment of agile
and project planning [17], and to more controversial
practices as part of a governed organizational transformation.
topics such as whether agile practices can be blended
Third, we extend existing agile software development methods
such as Scrum with a set of disciplined practices for software with more traditional methods to create hybrid
delivery in a larger scale, enterprise context. This creates a approaches such as “water-scrum-fall” [18] and
workable method that provides more freedom to practitioners “wagile” [19]. Many of these lessons are well
through less overhead work and more control to management summarized in Sutherland´s 10-year retrospective on
stakeholders through better measurement of progress and agile adoption [20].
quality insight. Accelerating software delivery at an enterprise scale is
II. RELATIONSHIP TO PREVIOUS WORK being pursued by almost every business that depends on their
software capability as a market discriminator. However, we
IBM’s framework for achieving agility at scale is founded still lack adequate empirical evidence and well-documented
on a long history of exploration and experience in delivering case studies. Our hope is that this paper will provide some
quality enterprise software. As far back as the NATO reports target patterns for others to validate or challenge with their
[2], [3] that proposed the term “software engineering” and the own measured improvement know-how.
original “spiral” and “iterative” models of software
development [4], [5], the industry has sought scalable III. IMPROVING SOFTWARE ECONOMICS
approaches to enterprise software delivery. The culmination of Empirical cost estimation models (like COCOMO II,
that work has been in improvement frameworks such as the SEER, QSM, Slim, and others) can estimate resources to
Capability Maturity Model Integration (CMMI) [6], [7], which within 25 to 30 % for most software projects. This level of
focuses on repeatability and consistency of system and unpredictability requires that software governance techniques
software delivery. In practice, “maturity” is often achieved at accommodate high levels of uncertainty. These cost models
the expense of innovation and flexibility in the delivery include dozens of parameters and techniques for estimating a
processes. wide variety of software development projects. However, they
The pressures in many domains to deliver new features can be simplified into a function of four basic parameters [21]:
more quickly led many teams to focus on agile techniques as
both a reaction and a complement to that previous work. In the Agility
Resources = Complexity * Collaboration * Automation
past decade, work on agile approaches has matured to
encompass a very wide variety of software development
techniques. While these methods were initially targeted at 1. Complexity. The complexity of software is typically
software development in small, collocated teams, their broader quantified in units of human-generated code (product
popularity has led to greater emphasis on scaling agile and test and supporting code) and its quality.
adoption in more complex environments. We can divide that Quantities may be assessed in lines of source code,
work into four main threads: function points, use-case points, or other measures.
1. Several frameworks for scaling agile adoption have Qualities like performance, reuse, reliability, and
been proposed [8], [9], [10]. These frameworks define feature richness are also captured in the complexity
a set of dimensions through which the challenges of value. Simpler and more straightforward applications
applying agile techniques can be described, measured, will result in a lower complexity value.
and systematically addressed. These dimensions 2. Agility. This process exponent typically varies in the
include team size and geographic distribution, as well range 1.0 to 1.25 and characterizes the governance
as regulatory compliance, domain complexity, methods, techniques, maturity, appropriateness, and
technical complexity, and organizational distribution. effectiveness in converging on wins for all
Staged improvement programs can then be executed stakeholders. Better agility (better processes with
while addressing those dimensions. optimized practices for the project at hand) will result
2. Through experiences with specific agile methods like in a lower exponent.
Scrum, scaling extensions and improvements have 3. Collaboration. This parameter captures the skills,
been proposed [11], [12]. These typically look at the experience, motivations, and know-how of the team,
implications of using specific methods in larger, along with its ability to collaborate toward well-
distributed team contexts. understood and shared goals. More effective teams
3. Documented case studies of agile approaches have will result in a lower multiplier.
highlighted the practicalities of adapting them to meet
 4. Automation. This parameter captures the extent of Figure 1 illustrates the expected costs and productivity
process automation, process enactment, improvements based on IBM’s experience and research. We
instrumentation, and team synchronization. Better continue to synthesize this experience into value traceability
tools and instrumentation will result in a lower trees, metrics patterns, benchmarks of performance, and
multiplier. instrumentation tools to provide a closed-loop feedback-
control system for improved insight and management.
The mathematical form of this equation, the empirical data Past measurement approaches [21], [22] suggest three
in the models, and their practical application across thousands genres of software governance. We briefly explore these and
of industry projects indicate that these four parameters are in offer some evolutionary context for why transformations to
priority order of potential economic leverage. A 10% reduction better measurement are needed.
in complexity is worth more than a 10% improvement in the 1. Engineering governance. The waterfall model [23] is
process, which is worth more than a 10% more capable team, practiced by the majority of software development
which is worth more than a 10% increase in automation. teams today because it derives from the predominant
Figure 1 shows ranges and probability distributions of legacy culture of traditional engineering governance.
expected improvements based on our experience. Actual Measures typically include (dishonest) earned value
targets and outcomes are highly dependent on the specific
based on activity/milestone completion, code/test
context. The more significant improvements, like systematic
production, requirements-to-code traceability,
reduction in complexity and major process transformations,
also require more significant investments in time and inspection coverage, and variances between static
resources. Smaller, incremental process improvements, skill plans and actual expenditures. An engineering
improvements, and automation improvements targeted at orientation focuses on static targets, planned activities,
projects or individuals are more predictable. and deterministic predictions to drive the development
Although tools are important, we have learned from field process. Waterfall management is simple, but it is
experience that there is much more economic leverage in overly simplistic for software where uncertainties
reducing complexity, improving process agility, and dominate a project’s timeline.
improving collaboration. Investments in object-oriented 2. Hybrid governance. About 20 to 30 % of system and
languages, Unified Modeling Language, service-oriented software development teams practice iterative
architectures, model-driven development, and reuse can help development techniques [24], where architectures are
design teams build million-line programs with only constructed first and evolved through a sequence of
diseconomies of scale, instill good practices, codify patterns of executable releases. Measures typically include more
success, and avoid patterns of failure. An open thousands of honest earned value based on release content, release
lines of human-generated code. Investments in iterative quality, prioritized risk lists, change management
development and agile methods help to reduce the trends, and variances between dynamically changing
collaboration platform for enhanced collaboration improves plans and actual expenditures. Although iterative
team productivity through integrated change management, development is more complex to manage, therefore
agile project management, , and automated instrumentation.
requiring more project management savvy, it is more
frequently successful.
3. Economic governance. An economic orientation
steers project priorities and results based on the
dynamically changing predictions of the probable
outcomes of the development process. As many as
45% of industry teams claim to practice agile or lean
software delivery techniques [25], [26], [27]. Project
teams focus on early reduction of uncertainty,
continuous integration, asset-based development, and
increased stakeholder interaction, and they use smarter,
collaborative environments. Measurements include
demonstrable user stories, uncertainty reduction
(trends in the variance of estimates to complete),
defect trends, change trends, and rework trends.
Management savvy, combined with meaningful
measurement and instrumentation, results in higher
levels of agility and improved predictability of
outcomes.
Projects that have transitioned to a more agile steering
Figure 1. Ranges of productivity improvement
leadership style based on effective measurement can optimize
scope, design, and plans, reducing unnecessary scrap and
rework. Steering implies active management involvement and A. Integration Testing Should Precede Unit Testing.
frequent course corrections to produce better results. Effective In practice, integration and unit testing proceed in parallel.
steering eliminates uncertainties earlier and significantly However, to accelerate the transformation to increased agility,
improves the probability of win-win outcomes for all it is best to demand that intermediate milestones include
stakeholders. Scrap and rework rates are not driven to zero, but executable test cases of integrated functionality.
to a level that corresponds to healthy discovery, Integrating first to demonstrate the architecturally
experimentation, and production commensurate with resolving significant challenges first will resolve the big uncertainties
the uncertainty of the product being developed. We estimate earlier. It is human nature to address the easy things first to
this to represent about 15 to 20% of total effort spent in show early progress, but there is no economic leverage in
reworking the code and test base, as opposed to the 40% of doing so. By postponing the resolution of uncertainty, you
effort more typical with conventional governance [24]. decrease the probability of success.
Most software organizations are struggling to transform Demanding an integration-first priority will help you
their lifecycle model from a development focus to a delivery increase the agility of the larger project team.
focus and to improve the economic outcomes of software Project management. Lay out plans, resources, and
development. This subtle distinction in wording represents a measures that prioritize integration testing of key usage
dramatic change in the principles that are driving the scenarios in multiple checkpoints as the primary steering
management philosophy and the governance models. mechanism.
A software development orientation focuses on the various Analysis. Analyze the business context or system context
activities required in the development process, while a to define first the integrated behaviors, qualities, and usage
software delivery orientation focuses on the results of that scenarios that represent the holistic value and whose
process. Organizations that have successfully made this elaboration will reduce most of the project uncertainty.
transition have recognized that engineering discipline is Architecture. Elaborate the architecture of the solution
trumped by economics discipline in most software-intensive and the evaluation criteria for incremental demonstration of the
endeavors. Table 1 provides a few differentiating indicators of most important behaviors and attributes, such as changeability,
a successful transformation from conventional engineering performance, integrity, security, usability, and reliability.
governance to more economically driven governance. Design/development. Develop units, services, and
To transform successfully from conventional engineering components that are always executable and testable, evolving
governance to more agile economic governance requires a from initial versions that permit execution within their usage
significant cultural transformation that is best achieved context (that is, satisfy their interface) in a trivial way, and
through the pursuit of two major change themes: 1) integration then progress toward more complete components that meet
testing should precede unit testing, and 2) agility and control quality expectations across their entire operational spectrum.
must be complementary. Testing. Identify testing infrastructure, data sets, sequences,
harnesses, drivers, and test cases that permit automated
TABLE 1. TRADITIONAL GOVERNANCE VS. ECONOMIC GOVERNANCE
regression testing and integration testing to proceed without
reliance on completely tested units and components.
Software Development Software Delivery
Traditional Governance Economic Governance B. Agility and Control Must Be Complementary.
Distinct development phases Continuously evolving systems In most conventional software engineering cultures,
management governance competes with practitioner freedom.
Distinct handoff from development Common process, platform, and Here are two recurring observations from such cultures:
team to maintenance team team for development and
maintenance 1. Where there is a perception of good governance,
practitioners feel choked by repetitive manual
Distinct and sequential activities Sequence of usable capabilities
requirements to design to code to with ever-increasing value
reporting.
test 2. Where practitioner agility and morale are positive,
management feels out of control with dynamically
Role-specific processes and tools Collaborative platform of
integrated, web-based tools and changing baselines.
practices
To accelerate software delivery and achieve agility at scale,
False, early precision in plans and Honest, evolving precision as we need to integrate governance with agility so they are
requirements uncertainties are resolved complementary objectives and not competitive forces. We
Governance through measurement Governance through measurement must provide management with improved steering
of artifact production and activity of incremental outcomes and mechanisms such as progress and control measures, automated
completion progress/quality trends instrumentation, and real-time development analytics. We
Engineering discipline: track Economic discipline: reduce
must also enable practitioners with more freedom to innovate
progress against static plans uncertainties, manage variance, through automation of measurement, traceability and change
measure trends, adapt and steer propagation, process enactment, reduced scrap and rework,
and automated progress reporting.
 The platform of know-how and automation must deliver TABLE 2. HOW IMPORTANT IS MEASUREMENT?
this critical quid pro quo:
Measurement practice Strong Weak
More freedom and less overhead for practitioners
On-time projects 75% 45%
  Late projects 20% 40%
Cancelled projects 5% 15%
Better measurement and predictability for stakeholders Defect removal efficiency 95% <85%
Resource estimates Accurate Optimistic
Client satisfaction Higher Lower
Practitioners need to maximize their productive time to
Staff morale Higher Lower
create product or service capabilities that delight their users.
Practitioners must demand that overhead tasks be minimized
or automated. Management stakeholders, on the other hand, approach. First, we define a framework for prioritizing agile
need more insightful measures and a feedback control loop to practice areas. Second, we introduce a performance
steer progress and quality with better predictability of business measurement framework that emphasizes and encourages an
outcomes. incremental, measured approach to agile practice adoption.
Improving measurement discipline is important because it
correlates strongly with better business performance. The data A. A Software Practice Framework
in Table 2, compiled from more than 150,000 projects across The key to adopting agility at scale is to prioritize the
hundreds of software organizations by Jones [28], illustrates important practices and to segment them into a phased
how measurement discipline can promote more trustworthy approach to improvement. We have seen several agile
communications among stakeholders. The value of that trust adoption efforts experience significant roadblocks when they
can only be quantified coarsely, but the impact is eye-opening. are overly focused on specific techniques and tools, rather than
Covey [29] coined the term “the speed of trust” to on improving the practices that are most important to the
illuminate the necessary ingredient in reducing overhead and organization. It is far too easy to place all the attention on the
improving efficiency: trust. Trust is the secret ingredient activity and lose track of the intended outcome and measured
necessary to achieve the measurement-freedom quid pro quo. improvement that the implementation is intended to deliver.
Trust is not just a matter of ethics; it is a matter of competence: We have found that five core practices constitute the key
track record, transparency, and accountability. prerequisites for efficient adoption of agile approaches.
To reconcile these competing points of view by 1. Iterative development
establishing more trust between constituencies, developers 2. Two-level planning
need to embrace meaningful measurements, and management 3. Whole team thinking
needs to enable and empower agility. 4. Continuous integration and delivery
The following section describes how this crucial exchange 5. Test-driven development
between governors and practitioners (the “give-to-get”
agreement of a quid pro quo) can enable economic governance Addressing these core practices early achieves an explicit
in a large-scale enterprise context. and shared understanding among key constituents. If an
organization is not experienced in iterative development
IV. DEPLOYING AGILE KNOW-HOW AT ENTERPRISE SCALE techniques, the direct transition to an agile, continuous
delivery approach is too much change to swallow at one time.
In the past 10 years, we have introduced agile techniques in Two-level planning and whole team thinking reflect the
hundreds of complex enterprise delivery contexts [6], [30]. predominant project management shifts. The shift to outside-in
Such enterprise organizations typically have these assessment (test-driven development) and continuous
characteristics:
integration ensure that uncertainties are resolved earlier,
1. Diverse software supply chains across many business resulting in improved predictability of outcomes.
units, working in many geographical regions and In support of the agile core practices, our framework
governed by multi-layered reporting, that place identifies five additional areas of improvement: governance
extreme demands on collaboration and integration. and compliance management, requirements management,
2. Complex organizational, technical, and economic change and release management, architecture management,
structures that demand complex decision frameworks and quality management. By examining other software
and complicate meaningful measures. engineering improvement frameworks such as IBM’s Rational
3. Blended technical platforms, processes, and tools for Unified Process (RUP) and CMMI, these five areas represent
software development and delivery that require an the critical areas of concern for most organizations, and offer a
incremental approach to change. comprehensive framework for measured improvement. In
To manage these complexities, IBM’s approach includes a practice, several detailed discussions and workshops may be
strong measured improvement point of view, supported by necessary to identify the specific strengths and weaknesses of
adoption guidance for translating agile methods into the organization, prioritize them, and set target improvement
incremental outcomes. There are two key elements to our objectives. For example, some organizations may focus on
areas of specific competitive advantage (such as speed of TABLE 3. A FRAMEWORK FOR MEASURING AGILE DELIVERY ADOPTION
delivering new features), on areas of known shortfalls (such as
improving software quality), or on alignment with other on- Business Measures IT Measures Agile Measures
going transformations (such as a consistent approach to
requirements management in concert with business analysts). Delivery cycles Overhead Practice adoption
We need to define several dimensions of each practice: Productivity and cost Change Role adoption
 Key concepts. What are the key principles of the reduction efficiency
practice, and how does it relate to other practices?
Quality and customer Requirements Work product
 Work products. Which artifacts are critical for this satisfaction churn adoption
practice, and how will they be maintained?
Skills and reputation Integration Task adoption
 Tasks. What are the most important activities and
stability
usage models for that practice?
 Guidance. What advice and heuristics are useful for Employee Test time Process adoption
the practice? satisfaction
 Automation. How can tooling be applied to reduce
overhead work and improve efficiency and quality? 1. Are we meeting the business objectives? The original
 Measurements. What are the key measures for the business objectives for the agile delivery program
practices, what is the current benchmark, and what are raised expectations with executive management that
reasonable improvement targets? the enterprise software delivery organization would
be faster-cheaper-better. Simple quantitative measures
This last dimension is a key discriminator of success. In were specified to correlate the technical measures
prioritizing these practices, it is essential to define and agree with the business outcomes expected.
on the key measured improvement targets for each practice
2. Are we seeing the benefits we expected? Technical
across the organization. Based on this approach, the
measures were identified as key performance
organization can create priorities around the areas for
improvement. Practices are adopted incrementally based on a indicators (KPIs) that quantified different
measured plan for improvement that is well understood across perspectives of status and progress. The agile delivery
the organization. adoption program was aligned with those KPIs to
rationalize forecasted improvement plans and to
B. Meaningful Performance Metrics estimate expected contributions.
One principle of success for any organizational change is 3. Are we truly agile? Individuals and teams in the
to adopt change incrementally based on measured feedback. organization had their own view of what they
Therefore, it is essential that an agile delivery adoption expected from the agile delivery program. Continual
approach is supported by measures and instrumentation that assessment took place, through surveys and
provide insight into progress and quality improvements.
qualitative interview-based approaches, to understand
Measurement is an important and complicated challenge
how the agile delivery program was proceeding and to
for any enterprise software delivery effort. Much has been
written on this topic, with particular attention in recent years normalize perceptions with objective benchmarks.
on cost control and efficiency in enterprise software delivery. The ability to obtain accurate, timely measures is critical to
Measurement know-how can be applied to the context of agile this simple approach to measured improvement. Our
delivery rollout. In practice, we have found that the most experience has been that the individual practitioners place
critical first step is to understand what ideally could be greater emphasis on improvement if there are a small number
measured, from what practically can be measured, from what of metrics derived directly from the model, code, and test base
effectively is measured. Too often we see agile delivery with clear interpretation. Table 4 shows a good starting point
adoption programs that include sophisticated measuring for specific measures.
schemes where the organization has little history, maturity, or When starting agile adoption, it is important that the
context for those measures. The result is that such schemes business owners deliver more software, more quickly, and
become abused, mistrusted, and ignored. with better quality. Developers and engineers will embrace
No single set of measures for agile delivery adoption will automated measurement of the engineering artifacts (models,
work for all organizations. Rather, in adopting agile principles code, and test bases). Automated instrumentation eliminates
in a large-scale enterprise, we recommend clear, explicit goals the overhead drudgery of metrics collection, progress reporting,
that help to steer the organization toward its objectives. Table change propagation, and traceability, and increases
3 shows a simple three-level measurement approach that we transparency and accountability. Likewise, when governance
used with one large banking organization to baseline their authorities are enabled with more honest measures that
goals and priorities for agile delivery adoption. The goal of correlate better (and well) with dynamically changing progress
this agile delivery adoption scheme was to address three basic and quality trends, they will encourage more change. Royce
questions. provides a more comprehensive treatment of measuring agility
and architectural integrity in [22].
 TABLE 4. SIMPLIFIED MEASUREMENTS FOR AGILE SOFTWARE DELIVERY People first. We foster the strategy of cross-functional
teams made up of cross-functional people. There should be no
Measure Business- Team-Related formal hierarchy within the team, and team members are
Related encouraged to be cross-functional in their skill set and perform
Cycle-time Time from Build/release cycle time
work related to disciplines other than their specialty. The
reduction initiation to increased understanding that team members gain beyond their
delivery of Sprint velocity primary discipline results in more effective use of resources
first and reduced reliance on formal documentation and signoffs.
increment Blocking work items This in turn enables scaling through easier communication and
collaboration.
Time from Requirements  tests Explicit scaling support. The DAD process framework is
initiation to an important part of IBM’s strategy, which explicitly promotes
project Change costs over time
that there is more to scaling than team size and there are
closure multiple scaling factors a team may need to address. These
scaling factors are team size, geographic distribution,
Quality Production Defect trends regulatory compliance, domain complexity, technical
defects per
Change trends complexity, organizational distribution, organizational
100 function
complexity, and enterprise discipline. Each team will be in a
points
Integration trends unique situation and will need to tailor its strategy accordingly.
For example, a collocated team of seven in a regulatory
Continuous Process Practice adoption environment will work differently than a team of 40 spread out
optimization maturity across several locations in a nonregulatory environment. Each
level Variance in cost to of the eight scaling factors will motivate teams to tailor the
complete foundation practices.
Goal-driven. The framework is goal-driven, as summarized
Productivity Function Sprint burndown chart
points per
in Table 5. For each goal, we describe several issues that must
man-year Release burndown chart be considered. We show several options for addressing each
issue and the trade-offs for each option. Suggestions are
provided for what we’ve found to work best; there is no
Constant monitoring of the health of the agile delivery prescription. Context matters and the starting points in our
program results in both governance authorities and framework require some elaboration into each specific
practitioners embracing the measurement with shared situation. For example, consider the goal of identifying the
objectives. This can be used as the basis for early decision initial requirements for your project. There are several issues
making, and provides visibility into the program for all of that need thoughtful consideration when tailoring your
those involved. process: What level of detail (detailed specification,
The final section describes a specific framework through lightweight specification, a list of goals, none) are you going to
which the principles of economic governance and measured capture? What types of modeling (domain, process, user
improvement can be applied to enterprise-scale software interface, nonfunctional) will you perform? For each type, how
delivery projects. will you explore that issue?
V. A METHOD FOR DISCIPLINED AGILE DELIVERY TABLE 5. SOME GOALS ADDRESSED THROUGHOUT A DAD PROJECT
People new to modern best practices, and even many with
agile experience, often ask fundamental questions: How does Inception Phase Goals Construction Phase Goals
architecture fit into agile delivery? When does it make sense to
* Form initial team * Produce a consumable solution
write requirements specifications? What level of detail is
necessary? When should you enhance your whole team testing * Develop common project vision * Address changing needs
efforts with an independent test group? When shouldn’t you?
How do you work successfully with outside teams when they * Align with enterprise direction * Move to deployable release
are using traditional methods? * Explore initial scope * Improve quality
To address these concerns, it is essential to have a
framework that supports a coherent, end-to-end strategy for * Identify initial technical strategy * Prove architecture early
improvement; this includes a solid understanding of how agile * Develop initial release plan
solution delivery succeeds in practice.
Our experience with large enterprises has led to the * Form work environment
development of the Disciplined Agile Delivery (DAD) process
* Secure funding
framework [31]. The DAD process framework has several
important characteristics. These characteristics are presented * Identify risks
in priority order from the point of view of enabling the scaling
of your agile process.
 For example, usage modeling can be addressed via user Agile. Our process framework adheres to, and enhances, the
stories, use cases, usage scenarios, feature statements, and values and principles of the Agile Manifesto [1]. Teams
more. What elicitation strategies (formal, informal, interviews) following either iterative or agile processes have demonstrated
will you use? What change management strategy (formal, higher quality, provide greater return on investment, provide
product backlog, work item list, work item pool) will you greater stakeholder satisfaction, and deliver more quickly than
adopt? How will you capture (through technical stories, teams using either a traditional/waterfall approach or an ad hoc
acceptance criteria, an explicit list, or not at all) the pertinent (no defined process) approach.
nonfunctional requirements? Although identifying initial Hybrid. The approach is hybrid in that it adopts and tailors
requirements is a straightforward goal, satisfying this goal in a proven strategies from methods such as Scrum, Extreme
way that reflects the actual situation often requires more than Programming (XP), Agile Modeling (AM), Unified Process
simply creating a stack of index cards. (UP), Lean Software Development, Kanban, and Agile Data
This goal-driven approach avoids the tailoring challenges (AD). Consequently, DAD is a second-generation agile
associated with more prescriptive methods such as Scrum or process framework that leverages what has come before it.
UP. Where Scrum starts with a small base from which you Learning-oriented. There are three key aspects of an
need to add process, or UP starts with a large base from which effective learning environment. The first is domain learning:
you remove process, the DAD approach starts in the middle How are you exploring and identifying your stakeholder
ground and provides sensible tailoring advice. Your tailoring needs? Perhaps more importantly, how are you helping your
choices will be driven both by the scaling factors faced by a team do so? The second aspect is improving your process at
team as well as the skills and culture of that team. the individual, team, and enterprise levels. The third aspect is
Enterprise awareness. Agile delivery teams generally live technical learning, understanding how to work effectively with
within a larger context. There are often systems currently in the tools and technologies being used to craft the solution.
production, and minimally your solution shouldn’t impact
them, although your solution could leverage existing VI. CONCLUSIONS
functionality and data available in production. There may be Many organizations are now moving from small-scale agile
teams working in parallel to your team; you might take development success toward larger-scale agile delivery
advantage of some of what they’re doing, and vice versa. Your excellence. Our transformational approach led us to focus on
organization may be working toward a common vision which three key areas: economic governance, measured improvement,
your team should contribute to. There will be a governance and disciplined agile delivery.
strategy in place, although it may not be obvious to you, that Economic governance is not a new idea. The principles of
enhances what your team is doing. Effective teams work in a the spiral model [3], iterative development [4], and risk
way that reflects these realities. management [32], and the foundations of modern agile
Risk and value driven. Our process framework adopts a methods such as test-driven development all exemplify a
risk/value lifecycle, which is a lightweight version of the process spirit that emphasizes continuous and early integration.
strategy promoted by UP. Teams strive to address common To translate these principles into better outcomes, project
project risks, such as coming to stakeholder consensus around teams need to plan their activities and early releases to drive
the vision and proving the architecture, early in the lifecycle. integration testing priorities before unit testing.
Explicit checkpoints are defined for continued project viability In the most standout software successes we have observed,
and for assessing whether sufficient functionality has been where software product releases are straightforward to
produced and the solution is production ready. It is also value- maintain over a long period, teams prioritize continuous
driven, a strategy that reduces delivery risk, in that teams integration testing of the forest over detailed unit testing of the
produce potentially consumable solutions on a regular basis, trees. The two strongest, industrial-strength examples of
which can be deployed with positive results ahead of project integration-first results are well-documented [24, Appendix D],
schedule, if necessary. [33]. This integration-first spirit has a natural parallel in
Delivery-focused. The lifecycle context ranges from managing systems and software development teams:
initiation through construction to releasing your solution into Optimizing collaborative teamwork is more important than
production. It also shows some pre-initiation portfolio optimizing individual productivity.
management activities as well as post-delivery production Measurable improvements in defect profiles and reduced
activities. This expands on first-generation agile methods, cycle times for change are the key agile attributes that
which typically focus on the construction aspects of the differentiate the winners from the also-rans in improving
lifecycle. DAD also supports a lean, or advanced, lifecycle, to software economics. In this context, measured improvement
which many agile teams evolve after applying process means measuring the progressions and digressions in
improvements over time. continuous integration milestones. These are the important
IT solution focused. IT professionals do far more than just windows into improved economic governance that improve
develop software. While software is clearly important, in collaboration and resolve uncertainties in their process and
addressing the needs of our stakeholders we often provide new product earlier.
or upgraded hardware, change the business/operational Agility means quick to react. Change speed must be an
processes that stakeholders follow, and even help change the asset, not an anchor. Agility is best achieved by demanding
organizational structure in which our stakeholders work. that integration testing be performed earlier and continuously.
Agility is best measured by quantifying the costs of change 7. H. Glazer, “CMMi and agile: why not embrace both?,” SEI Technical
over time. Teams that have improved their turnaround time for Report, CMU/SEI-2008-TN-003, 2009.
8. S.A. Ambler, “The agile scaling model (ASM): adapting agile methods
software changes from a few weeks to a few days have clearly for complex environments,” IBM Developer Works, December 2009.
become more agile. Instead of being mired in late scrap and 9. P. Kruchten, “Contextualizing agile software development,” J Softw
rework, and consumed playing defense, they can go on the Maint Evol-R, 2011.
offensive by innovating more, adding more features or more 10. D. Leffingwell, Agile Software Requirements: Lean Requirements
quality, improving performance, or delivering earlier. Practices for Teams, Programs, and the Enterprise. Pearson Education,
The topics of discipline and control in agile software Inc., 2011.
delivery can be sensitive for many organizations. Too many 11. E. Woodward, A Guide to Distributed Scrum. IBM Press, Pearson
Publishing, 2010.
process owners advocate extreme positions on both ends of the 12. C. Larman and B. Vodde, Scaling Lean and Agile Development:
process rigor spectrum. The traditional extreme uses overly Thinking and Organizational Tools for Large-Scale Scrum. Addison-
comprehensive process descriptions: the IBM RUP, Team Wesley, 2009.
Software Process (TSP), and CMMI, for example. The 13. A.W. Brown, “A case study in agility-at-scale delivery,” Proc 12th Int
challenge with RUP was usually that teams didn’t scale it Conf on Agile Software Development, XP2011, Springer Verlag, May
down appropriately, which often resulted in an excess of 2011.
14. G Benefield, “Rolling out agile in a large enterprise,” Proc IEEE Comput
artifacts and too much overhead. The progressive extreme uses Soc HICSS08, 2008.
an overly lightweight process description, with Scrum being 15. I. Oshri, J. Kotlarsky, J.W. Rottman, and L.P. Willcocks, “Global
the exemplar. Many Scrum teams could not scale up sourcing: recent trends and issues,” Inform Technol People, vol. 22, no.
appropriately, resulting in significant effort reinventing 3, pp. 192-200.
techniques to address the myriad issues beyond core software 16. D. Leffingwell. Agile Software Requirements, Addison-Wesley, 2010.
development which Scrum doesn’t cover. A lot of waste could 17. A. Highsmith. Agile Project Management: Creating Innovative Products.
have been avoided if there had been an option between these Addison-Wesley, 2009.
two extremes. Disciplined Agile Delivery is a more balanced 18. D. West, “Water-scrum-fall is the reality for most organizations,”
Forrester Analyst Report, July 26, 2011.
option.
19. J. Gorman, “The Wagile software development lifecycle,” 2008.
There are probably more books on agile methods than http://www.codemanship.co.uk/parlezuml/blog/?postid=708
successful projects with well-documented agile results. 20. J. Sutherland, “Ten year agile retrospective: how can agile improve in the
Writing a book on agility or project management, where the next ten years?” 2011. http://msdn.microsoft.com/en-
decision-making process is laid out in the abstract, is easy, us/library/hh350860.aspx
compared to managing a real project where you must steer 21. W.E. Royce, K. Bittner, and M. Perrow. The Economics of Software
through a minefield of uncertainties and the consequences of Development. Reading, Massachusetts: Addison-Wesley, 2009.
decisions under game conditions are very real. We should 22. W. Royce, “Measuring agility and architectural integrity,” Int J Softw
Info, vol. 5, no. 3, 2011, ISCAS.
glorify the practice leaders in our industry and the people who 23. W.W. Royce, “Managing the development of large software systems,”
publish measured improvement case studies. They are critical Proc. IEEE Wescon, pp. 1-9, August 1970.
to accelerating the innovation delivered in software. Practice 24. W. Royce, Software Project Management, A Unified Approach.
leaders resolve uncertainties and steer projects from the Addison-Wesley, 1998.
abstract to the real. The demand for such practice leadership is 25. M. Kennaley, SDLC 3.0, Beyond a Tacit Understanding of Agile. Fourth
much higher than the demand for thought leadership. Medium Press, 2010.
26. S.A. Ambler, “IT governance and project management survey results,”
July 2009.
REFERENCES http://www.ambysoft.com/surveys/stateOfITUnion200907.html
1. The Agile Manifesto, http://agilemanifesto.org/ 27. S.A. Ambler, “How agile are you? 2010 survey results,” 2010.
2. P. Naur and B. Randell (eds.), "Software engineering: report of a http://www.ambysoft.com/surveys/howAgileAreYou2010.html
conference sponsored by the NATO Science Committee,” Garmisch, 28. C. Jones, Software Engineering Best Practices. McGraw Hill, 2010.
Germany, 7-11 October 1968. 29. S.M.R. Covey, The Speed of Trust: The One Thing That Changes
http://homepages.cs.ncl.ac.uk/brian.randell/NATO/nato1968.PDF Everything. Free Press, 2006.
3. B. Randell and J.N. Buxton (eds.), “Software engineering techniques: 30. A.W. Brown, Global Software Delivery: Bringing Efficiency and Agility
report of a conference sponsored by the NATO Science Committee,” to the Enterprise. Addison-Wesley, 2012.
Rome, Italy, 27-31 October 1969. 31. S.W. Ambler and M.J. Lines, Disciplined Agile Delivery: A
http://homepages.cs.ncl.ac.uk/brian.randell/NATO/nato1969.PDF Practitioner’s Guide to Agile Solution Delivery in the Enterprise. IBM
4. B. Boehm, “A spiral model of software development and enhancement,” Press, 2012.
ACM Sigsoft Softw Eng Notes, ACM, vol. 11, no. 4, pp. 14-24, August 32. B. Boehm, Software Risk Management. IEEE Computer Society Press,
1988. 1989.
5. P. Kruchten, Rational Unified Process. Addison-Wesley, 2002. 33. C. Berggren, J. Jarkvik, and J. Soderlund, “Lagomizing, organic
6. CMMi for development, version 1.3, CMU/SEI-2010-TR-033, integration, and systems emergency wards: innovative practices in
November 2010. managing complex systems development projects,” Project Manage J,
vol. 39 (supplement), 2008, pp. S111-S122.