CCPDS-R Case Study and

Software Process Tailoring:

Walker Royce Insights
Barry Boehm
CS 577a Lecture

Version 2.3 1
 Outline
• CCPDS-R Case Study
– Project Overview
– Plans and Schedules
– Results and Critical Success Factors
• Process Tailoring
• General Best Practices

Version 2.3 2
 Managing Successful Iterative
Development Projects
A Seminar on Software Best Practices
Version 2.3

2800 San Tomas Expressway

Santa Clara, Ca 95051
(408) 496-3600

Version 2.3 3
Case Study: CCPDS-R Project Overview

Characteristic CCPDS-R
Domain Ground based C3 development
Size/language 1.15M SLOCAda
Average number of 75
people
Schedule 75 months
Process/standards DOD-STD-2167A
Iterative development
Environment Rational host
DEC host
DEC VMS targets
Contractor TRW
Customer USAF
Current status Delivered On-budget, On-schedule

Version 2.3 4
CCPDS-R Case Study Overview
• Organization
• Subsystems/schedules
• Iteration plan/content
• Core metrics
è Development progress (size with respect to time)
è Test progress (evaluation criteria with respect to time)
è Software architecture stability (numbers of objects with respect to time)
è Subsystem stability (SCOs with respect to time)
è Modularity (SLOC: scrap with respect to time)
è Modularity (SLOC %of total: scrap with respect to time)
è Adaptability (rework with respect to time)
è Maturity (reliability with respect to time)
è SCO volumes (raw data by CSCI)
è Cost/effort partitions by activity
è Conclusions

Version 2.3 5
CCPDS-R Organization/Responsibilities
Software Project Manager
2 people

Chief Engineer Admin

3 people 5 people

Major milestones WBS, contract

Specifications Cost/schedule

Software Engineering Applications Software Testing

10 people 10->40 people 5->20 people

Demonstrations SSV CSCI CM, CCB, testbeds

Metrics Architecture Environment/resources
NAS CSCI DCO CSCI Formal testing
Process (SDP/SSPM) CCO CSCI String testing
Architecture CMP CSCI Tools/scenarios
TAS CSCI
Standalone testing

Version 2.3 6
CCPDS-R Overview
Common Subsystem

PDS Subsystem

STRATCOM Subsystem

Project Size Schedule

Common 353 KSLOC 60

months
PDS 254 KSLOC 32
months
STRATCOM 552 KSLOC 36
months ___________

Total 1159 KSLOC 75

months

Software effort: 75 people average

Version 2.3 7
Common Subsystem Build Content
Primitives and support software

Architecture, test scenarios, models

Critical thread applications, displays

Mission algorithms, non-critical thread applications, displays

Communications interfaces, final test scenarios

Associate contractor interface

Reliability and performance-critical Reliability and performance-critical

component development component testing and maturation

Version 2.3 8
Ada Process Model for Large-System RAD

Version 2.3 9
Common Subsystem Macroprocess

Development Life Cycle

Inception Elaboration Construction
Architecture Iterations Release Iterations
{
SSR IPD PDR CDR
R

0 5 10 15 20 25

Contract award

Architecture baseline
under change control

Competitive design phase:

• Architectural prototypes
• Planning
• Requirements analysis Early delivery of
“alpha”
capability to user

Version 2.3 10
Common Subsystem Microprocesses
7.9 PDW: Preliminary Design Walkthrough
KSLOC PD CDW TOR TOR
CDW: Critical Design Walkthrough
W TOR: Turnover Review
43.5 A1
PD CDW TOR TOR
W Development Test

60.3 A2 Turnover for

configuration
CDW TOR control
PD TOR
W
172.1 A3
PD CDW TOR TOR
W
62.7 A4
PD CDW TOR
W

6.5 A5
PD CDW TOR
W

353
SSR IPD PDR CDR
R

0 5 10 15 20 25 30 34

Version 2.3 11
Common Subsystem Progress
Plan
%
10
Designed/coded (% complete) % of Actual
SLOC
0
90 KSLOC under baseline
80 configuration control
100
70
60
75
50
40 50
30
20 25
10

5 10 15 20 25 30 35 10 15 20 25 30 35

Contract month Contract month

IPDR PDR CDR

• CDR progress:
Traditional Approach CCPDS-R Approach
Software design Complete Complete
Code development 10% 94%
Baseline under change control Negligible 47%
Formal test 0% 12%
Performance assessment Modeling 80% of operational
software demonstrated

Version 2.3 12
Common Subsystem Adaptability

n Architecture first
– Integration during the design phase
– Demonstration-based evaluation

n Configuration baseline change metrics:

40
Design
Hours 30 Changes Maintenance
Changes and ECPs
Change 20

10 Implementation
Changes

Project Development Schedule

15 20 25 30 35 40

Version 2.3 13
Some General Conclusions
n Common subsystem subsidized:
è Internal reuse 1

è Common process/tools
Normalized $/SLOC

è Overall systems engineering

è Cost/SLOC: 0.5

Common $X/line
PDS $.40X/line
STRATCOM $.33X/line
0
Common PDS STRATCOM

n Productivity/quality improved with each subsystem.

Common PDS STRATCOM
– This is the real indicator of a mature (level 3/level 4) process.

n Personnel volatility was low on the common

subsystem.
– It was much higher on the PDS and STRATCOM subsystems.

Version 2.3 14
General Conclusions (Continued)
• CCPDS-R adhered to 2167A
– Process could have been much more efficient
– Early (initially unplanned) delivery (month 30) of a useful
increment
• Moderate contractual requirements volatility
– Heavy non-ECP requirements/design volatility
– Only 1 major ECP (complete overhaul of the input message
set)
– Contractor/customer/user rapport avoided an inefficient
contracts bureaucracy
• Personnel skill:
– Strong project management, planning, and architecture
teams
– Cooperative customer/user
– Average application/test team

Version 2.3 15
CCPDS-R Software Technology Thrusts
Current Thrust CCPDS-R Approach
· ·
E ·
Integrated tools
·
DEC/Rational/custom tools
Open systems VAX/DEC dependent
· Hardware performance · Several VAX family upgrades
· Automation · Custom-developed change management system,
metrics tools, code auditors
· Reuse, COTS ·
Size Common architecture primitives, tools,
processes across all subsystems
· Object-oriented · Message-based, object-oriented architecture
· Higher level languages · 100%Ada
· CASE tools · Custom automatic code generators for
architecture, message I/O, display format source
code generation
· Distributedmiddleware · Early investment in NAS development for reuse
across multiple subsystems
· ·
P ·
Iterative development
·
Demonstration, multiple builds, early delivery
Process maturity models Level 3 process prior to SEI CMM definition
· Architecture-first · Executable architecture baseline with CM at PDR
· Acquisition reform · Excellent customer/contractor/user teamwork,
highly tailored 2167A for iterative development
· Training · Mostly OJT and internal mentoring

Version 2.3 16
CCPDS-R MBASE Models

• Success Models
– Reinterpreted DOD-STD-2167a; users involved
– Award fee flowdown to performers
• Product Models
– Domain model and architecture
– Message-passing middleware (UNAS)
• Process Models
– Ada process model and toolset
– Incremental builds; early delivery
• Property Models
– COCOMO cost & schedule
– UNAS - based performance modeling
– Extensive progress and quality metric tools

Version 2.3 17
Agend
a
• Motivation
– The search for software best
practices
– Software economics
• Technology overviews
– Iterative development Tailoring the process
– Architectures â Present the range of
– SEI CMM and ISO 9000 applicability
• Software process
â Same invariant spirit/policy
– The Rational Objectory process
– An organizational process â Different priorities
– Tailoring the process â Different implementations
– The acquisition process
• Experience and results
– General observations
– Case Studies and real-world
metrics
• Conclusions

Version 2.3 18
Classifying Systems
An average software project: Higher Technical Complexity
5-10 people - Embedded, real-time, distributed, fault-tolerant
10-15 month duration
3-5 external interfaces - Custom, unprecedented, architecture reengineering
Some unknowns, risks - High performance
DoD Weapon System

Telecom National ATC System

Switch
Commercial
Compiler
Embedded
Automotive
Software Case Tool
Lower (Rose, SoDA) Higher
Management Large-Scale Management
Complexity Organizational/Entity
Simulation
Complexity
- Small Scale Small Scientific - Large Scale
Simulation
- Informal Enterprise IS - Contractual
IS Application DoD MIS System
- Single stakeholder Distributed Objects (Family of IS - Many stake holders
Applications)
- “Products” (Order Entry) - “Projects”

IS Application
GUI/RDB
(Order Entry)
Business
Spreadsheet Lower Technical
-
Complexity
Mostly 4GL, or component-based
- Application reengineering
- Interactive performance

Version 2.3 19
Process Tailoring
Higher Technical Complexity
• More emphasis on domain experience
• Longer inception/elaboration phase
• More iterations, risk management
• Less predictable costs/schedules

Lower Higher
Management Management
Complexity Complexity
• More emphasis on management perspective
• More process formality/ceremony
• More emphasis on teamwork and win/win
• Longer Inception/elaboration
Lower Technical Complexity
• More emphasis on existing assets/knowledge base
• Shorter inception/elaboration phase
• Fewer iterations
• More predictable costs/schedules

Version 2.3 20
UNIX vs.
Windows
Higher Technical Complexity

Generally DoD Weapon System

Microsoft Telecom
National ATC System
Switch
Windows or Commercial

Windows NT Compiler
Embedded
Automotive
Software Case Tool

Lower (Rose, SoDA) Higher

Management Management
Complexity Large-Scale Complexity
Organizational/Entity
Simulation
Small Scientific
Simulation
Enterprise IS DoD MIS System
IS Application
(Family of IS
Distributed Objects
(Order Entry)
Applications)
Generally
UNIX and other
IS Application
GUI/RDB non-Windows
(Order Entry)
Business platforms
Spreadsheet

Lower Technical Complexity

Version 2.3 21
Products vs. Projects

• Emphasis and focus are different.

– Products: Market-constrained, small teams, driven
by technical challenges
• Usability Varying levels are freely
• Reusability traded off with cost and
• Portability time-to-market
• Adaptability
– Projects: Contract constrained, large teams, driven
by management challenges
• Completeness Fixed levels are required
• Performance at fixed costs and
• Reliability schedules
• The same process themes apply.
– However, priorities are different.

Version 2.3 22
Products vs.
Projects
• Top 10 discriminations of success/failure:

Products Projects

1. Architecture team skill 1. Iterative development

2. Object-oriented technology 2. Architecture team skill
3. Programming team skill 3. Management team skill
4. Iterative development 4. Acquisition process/rapport
5. Change management rigor 5. Megaprogramming
6. User/customer focus 6. Change management rigor
7. Environment integration 7. Environment integration
8. Languages/standards 8. Languages/standards
9. Management team skill 9. Object-oriented technology
10. Megaprogramming 10. Programming team skill

Version 2.3 23
Product Discriminators
• Architecture/programming team skills are paramount.
– Innovation and heroic efforts are critical
• Management skill is less important
– There are few constraints. Time-to-market, function,
quality, and cost are negotiable.
– Tradeoffs can be decided upon within common corporate
goals.
• Object-oriented technology is key to technical success.
– Reuse, adaptability and portability are necessities
• Acquisition process/rapport is generally unimportant.
– User/customer focus is still vital
– The market is moldable; customers appears after
development.

Version 2.3 24
Project
Discriminators
• Iterative development and acquisition process are key.
– R&D (resolving unknowns in requirements, design, and
technology) is separate from production (development of
useful increments).
• Architecture team skill is very important.
– #1 driver of technical quality, feasibility, and application
production
• Management skill is very important.
– Many constraints (cost, schedule, function, quality);
continuous negotiation and team building
• Applications team skill is less important.
– A good architecture can be implemented by an average
team.

Version 2.3 25
Best Practices Cross
Reference
1. Make Quality #1.
2. High-quality software is possible.
3. Give products to customers early.
4. Determine the problem before writing the requirements.
5. Evaluate design alternatives.
6. Use an appropriate process model.
7. Use different languages for different phases.
8. Minimize intellectual distance.
9. Put techniques before tools.
10. Get it right before you make it faster.
11. Inspect code.
12. Good management is more important than good technology.
13. People are the hey to success.
17. Plan to throw one away.
19. Design for change.
20. Design without documentation is not design.
26. Don’t test your own software.

Version 2.3 26
Our P referred Wording
1. Define quality commensurate with your objectives
2. Make quality assurance a team goal, not a separate discipline
3. Give products to customers early.
4. Determine the problem before writing the requirements.
5. Evaluate design alternatives.
6. Tailor the process to the scale/domain/complexity of the project
7. Minimize the number of life-cycle representation formats.
8. Minimize intellectual distance (use an object oriented method).
9. Put techniques before tools.
10. Get it working before you make it faster.
11. Use inspections judiciously
12. Good management is more important than good technology
13. People are the key to success.
17. Encourage experimentation and exposure of design issues.
19. Design for change.
20. Build self-documenting products rather than self-serving documents.
26. Plan and execute a test program from day 1.

Version 2.3 27
Recurring Themes of Success
n Customer-user-contractor teamwork is essential
– Relationships are non-adversarial.
n Managers are performers.
– They author their own plans.
n Requirements/designs are fluid and tangible.
– Maturity evolves where designs are “guilty until proven innocent.
– Real issues are surfaced and resolved systematically.
– Change management overrides change avoidance.
n CM/QA is everyone’s job, not a separate discipline.
n Performance issues arise early in the life cycle.
– “Something” is performing.
n Low attrition of good people is a sign of success.

Version 2.3 28
Recommendations to Buyers
n Try to acquire a good product and have your contractor
make a good profit

n Install an on-line acquisition environment:

– Demand delivery of executable evolving product versions.
Ù Get users involved early and continuously.
– Add value through your own engineering support.
Ù Do your own metrics analysis.
Ù Construct scenarios.
n Avoid attrition and adversarial relationships.

n Strive for 80%solution early, then evolve toward 100%.

– Separate the “need” from the “want.”

Version 2.3 29
Recommendations to Contractors
n Iterative development exposes important trends early.
– Good early performance breeds continuous success.
– Poor early performance is almost impossible to turn around.
Ù Get the right management team.
Ù Get the right architecture team.
n Provide adequate capital environments.
– Iterative development is much more (computer) resource
intensive
– Opportunities for automation must be exploited.
n Enable round-trip engineering.
– Integrated environments and compatible tools.

Version 2.3 30
Rational Process Summary
Development Life Cycle
Inception Elaboration Construction
Transition
Objective Identify Risk Development User
Revenue resolution
satisfaction
opportunity

Product Business Architecture Useful increments Version

case and updates
Production
plan

Cost estimate Prototyping Early design Post-architecture Maintenance

Object points Function Source lines Annual change
points traffic

Risk Focus Technical and Schedule Cost Deployment

economic perturbations
feasibility

Version 2.3 31
The Importance of Automation
n Process maturity depends on capable environments.
– Metrics must be automatically collected by the environments.
Ù It’s the best way software engineers will accept them.
– Change management must be automated and enforced.
Ù It’s the best way to mange multiple iterations.
Ù It’s the best way to enable change freedom.
Ù Change is the fundamental primitive of iterative development.
– An architecture-first and demonstration-based process is
enabled by off-the-shelf architecture components and
middleware.
Ù It’s the best way to remain hardware/topology-independent.
– Tools must be pre-integrated.
Ù It’s the best way to maintain consistency and traceability.
– Testing/documentation must be mostly automated.
Ù It’s the best way to attain change freedom
Version 2.3 32
Making a Positive Economic Impact
Cost = (Environment)* (Size)(Process)

n Environments are the key to reduced “coefficients.”

– More automation and integration
– Change management and change freedom

n Objects technology is the key to reduce “Size.”

– Reuse, off-the-shelf products, automatic code generation
– Line-of-business architectures

n Iterative development is the key to reduced “exponents.”

– Architecture-driven, continuous integration, tangible quality/progress

Maintain a balance of emphasis on all 3 thrusts.

Version 2.3 33
Software Management Balance is Key
Too Too
little/few much/many
Burnout and Inefficiency and
attrition People bureaucracy

Inadequate Inefficiency of
vision Requirements change
detail
Inadequate Inefficiency of
ROI Institutionalization change

Inadequate Inefficiency
quality Schedule/cost of resources

Inadequate Diluted
legacy
Documentation product focus

Version 2.3 34
Next-Generation Environments

R&D R&D
separate from with
All R&D construction automated
•Custom tools •Off-the-shelf tools construction
•Off-the-shelf tools
•Ad hoc methods •Defined methods •Defined methods
and processes and processes and processes
•Numerous languages •Few languages •Few languages
and formats and formats and formats
•Manual integration •Some integration •Highly integrated
•Programming focus •Design/integration focus •Architecture focus

Version 2.3 35