Agile Testing PDF

Agile Testing
Elisabeth Hendrickson
Quality Tree Software, Inc.
www.qualitytree.com
Copyright (c) 2004 - 2005, Quality Tree Software, Inc.
Poll: How Agile Is Your Testing Now?

How many of you
Would call your test practices Agile?
Participate in defining the acceptance criteria?
Routinely test incomplete software to provide feedback to
your stakeholders sooner?
Have adopted a lightweight documentation style?
Have found that your test artifacts are relatively easy to
update even when there are radical changes to the code?
Have access to the source code and to the same set of
tools as the programmers?
Have ensured the programmers have full access to all the
artifacts and tools you use?
Are co-located with the project team?
Whats Agile?
Agile is more than a buzzword. It is a relentless
focus on providing business value, usually by
adopting one or more Agile methodologies such as
Scrum or XP.
See the Agile Manifesto:
http://www.agilemanifesto.org
And the Agile Alliance:
http://www.agilealliance.org
Examples of Agile Methodologies

Lean
Lean manufacturing
concepts applied to
software
development.
Scrum
Lightweight
management
framework.
Crystal
Lightweight set of
development
practices.
Extreme
Programming (XP)
Rigorous set of
practices designed to
keep both the code
and team agile.
Agile Synthesized
Communication and
collaboration
Visible indicators
Disciplined development
practices
Feedback
Whole team thinking
Short iterations
Low overhead, high
productivity
Calling It Agile Doesnt Make It So

This is NOT Agile:
1. Compress the schedule

2. Toss out the
documentation
3. Code up to the last minute
The organization may gain
short term speed but at the cost of
long term pain.
How Traditional Test Practices Evolved

With great optimism and the best of
intentions, The Project Plan is announced:
Analyze
Design
Requirements
handed off to Dev
Code
Test/Bug Fix
Completed Code
handed off to Test
Release
How Traditional Test Practices Evolved
Inevitably, The Project Plan is revised:

Analyze, Design, & Code
Test/Bug Fix
Completed Code
handed off to Test
Release
The Result: Practices Intended to Control Chaos

Traditional test practices attempt to manage the chaos (or
at least avoid the blame):
Last Defender of Quality stance
Strict change management
Detailed preparation and up front planning
Heavyweight documentation suitable for outsourcing
the test effort
Strict entrance and exit criteria with signoffs
Heavyweight test automation focused on regression
Becoming Agile: Shifting Roles

Fear not!
Ill protect
you!
from last line of

defense
Hey, would
this help?
to team support
10
Testing Directions: Maricks Model
Critique Product
Support Programming
Business Facing
Technology Facing
Source:
http://www.testing.com/cgi-bin/blog/2003/08/21#agile-testing-project-1
11
Traditional & Agile Testing Contrasted

Traditional Testing
Agile Testing
Change
Manage & control it.
Accept it.
Planning
Comprehensive up front test

design.
Plan as you go.
Documentation
Verbose.
Only as much as absolutely

necessary.
Handoffs
Formal entrance and exit criteria

with signoffs.
Its not a relay race.

Collaborate.
Automation
System-level, built by tool

specialists, created after the code
is done.
All levels, built by anyone,

an integral part of the project.
12
Planning
When creating test artifacts
Handoffs
Automation
Documentation
Change
Embrace Change: Plan Tests for Maintainability

Minimize duplication.
Thought exercise: if a feature were
removed from your software, how many
test artifacts would have to be updated?
Use tools designed for change.
Hint: if the vendor says stabilize the
interface first, run away!
13
Planning
Handoffs
Automation
Documentation
Change
How Not to Plan a Test Effort

Unfortunately, this is common practice:
Download a generic, standard, formal test plan
template from the Internet.
Fill in every section in the template even if you
dont understand it and have to make stuff up.
Kill the maximum possible number of trees by
distributing copies to every team member.
The result is a large and mostly irrelevant plan
that must now be maintained. Yuck.
14
Planning
Handoffs
Automation
Documentation
Change
Plan as Little as Possible

A little planning is good. More is not better.
Know who you serve (primarily).
Technology-facing or business-facing?
Plan for the current iteration.
Speculative planning means rework.
Have a strategy that fits on one page.
If it is still relevant in 6 months, its probably at
the right level of detail.
Keep an up-to-date, prioritized risks list.
What kind of information are the testers looking
for? The risks list covers it.
15
Planning
Informal
Whiteboards
Sticky Notes
Index Cards
Wikis
Formal
Databases
Gantt/PERT Charts
Polished Documents
Handoffs
Automation
Documentation
Change
Favor Informal, Collaborative Tools
16
# Testers Reporting
Planning
How Much Time Are We Spending on Test Documentation?

40
35
30
25
20
15
10
5
0
0
10
20
30
40
50
60
70
80
90
100
% Time
Handoffs
Automation
Documentation
Change
Monitor Documentation Costs
How Much Does Documentation Cost?

Informal polling of 162 software testers from
65 companies revealed that most spend more
than 33% of their time documenting tests.
17
Planning
Handoffs
Automation
Documentation
Change
Keep the Documentation Simple

Capture the essence, not the details.
Step-by-step instructions cost time without
providing value (usually).
Point to other project documents.
If its in the user guide, requirements,
specs, etc., leave it there.
Centralize generic tests in a checklist.
Try this: count the number of times you find
a common condition, like invalid dates or
null strings, in the test docs. More than
once is too many.
18
Planning
Which would you rather maintain?
Handoffs
Automation
Documentation
Change
Watch Verbose Wording

Choose the Import option from the File menu.
A dialog titled Import File appears. Navigate
to \\x\y\z\long.dat and click Open. A dialog
titled Importing appears with the current
status, a progress bar, and a button labeled
Cancel. Click on the Cancel button. Choose
OK on the confirmation dialog. Verify that the
import stops.
Or:
Start a long import. Cancel it in the
middle. Verify it stops.
19
Planning
Handoffs
Automation
Documentation
Change
A Lightweight Example
20
Planning
Hint: when people joke about

signing the handoff form in
blood, the signoff process is
more about blame than about
producing good software.
Handoffs
Automation
Documentation
Change
Do Away with the Signoff Sheet
21
Planning
Handoffs
Automation
Documentation
Change
Integrate Testing
Testing is not a phase. Its a way of life.
Agile teams are test infected.
The question, How will we test it? is as
important as How will we build it?
Co-locate testers and programmers. But
sitting side by side does not ensure
communication.
Track testing status and programming
status all together.
Show tests run-passed-failed together with
features/stories done and left to do.
22
Planning
Handoffs
Automation
Documentation
Change
Leverage Automation Investments

Automate everything you can, but invest wisely.
Collaborate with programmers on test
infrastructure code.
The programmers have already automated the
unit tests. Why not reuse the investment where
possible?
Use different types of automated tests for
different purposes.
Automated system tests should cover end-to-end
sequences. Unit tests detect unintended change.
Dont substitute one for the other.
23
Planning
Use test automation to support early

exploratory testing.
Traditional test wisdom says we cant start
testing a feature until its accessible from
an external interface (like a GUI). But we
dont have to wait. Test automation can
facilitate manual exploration.
Handoffs
Automation
Documentation
Change
Automation & Exploratory Testing
24
Agile Testing Synthesized

Communication and
collaboration
Whole team thinking
Low overhead, high
productivity
Early involvement
Leveraged automation
Focus on providing rapid
feedback to key stakeholders
25
Necessary Conditions for Agile Testing

1.
2.
3.
4.
The organization is willing to embrace agility as

defined by the Agile Manifesto.
Saying Be More Agile or Test Faster isnt enough.
The whole team is responsible for quality, not just the
testers or people with Quality in their title.
Which are you more likely to hear: How did you miss
that bug?!? or How did we not catch that?
Everyone tests, not just designated testers.
Agile teams are test infected.
Managers focus on fixing problems, not blame. Agile
practices dont provide CYA paper trails and are
unlikely to succeed in a high-blame, high-fear
environment.
26
Agile Testers Wanted: A Job Description

Responsibilities:
Perform manual/exploratory tests on early-stage code
Create automated acceptance tests
Advise the team about overall risks and trends
Assist the business stakeholders define acceptance criteria
Facilitate communication between the technical & business
stakeholders
Qualifications:
Experience designing and executing tests
Scripting skills in at least one of the following languages:
Ruby, Perl, Python, or JavaScript. Experience
programming in Java, C#, etc. a plus.
Strong analysis skills
Ability to work in a team (bullpen) environment
27
Test Teams that Embrace Agility

Focus on providing value to our key
stakeholders (both business-facing and
technology-facing)
Shift from being the last line of defense to
providing an information service.
Aggressively reduce time and resources
spent on anything that does not directly
contribute to providing information.
Collaborate with programmers to improve
testability and leverage test automation
efforts.
28
Further Reading
Beck, K. (1999). Extreme Programming Explained:
Embrace Change. Addison-Wesley.
Cockburn, A. (2004). Crystal Clear: A HumanPowered Methodology for Small Teams. AddisonWesley.
Crispin, L., & House, T. (2002). Testing Extreme
Programming. Addison-Wesley.
Poppendieck, M. & Poppendieck, T. (2003). Lean
Software Development. Addison-Wesley.
Schwaber, K. & Beedle, M. (2001). Agile Software
Development with SCRUM. Prentice Hall.
29
Acknowledgements
Many thanks to early reviewers of the ideas
presented here: Brian Marick, William Wake,
Jonathan Kohl, Jeffrey Fredrick, Daniel Knierim,
Marc Kellogg, Danny Faught, Ron Jeffries,
Hubert Smits, Rob Mee, Sherry Erskine, Amy Jo
Esser, Gunjan Doshi, Dave Liebreich, Janet
Gregory, Chris McMahon
30

Agile Testing PDF

Uploaded by

Copyright:

Available Formats

Agile Testing PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Agile Testing PDF

Uploaded by

Copyright:

Available Formats

Agile Testing

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

Poll: How Agile Is Your Testing Now?

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

Examples of Agile Methodologies

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

Calling It Agile Doesnt Make It So

1. Compress the schedule

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

How Traditional Test Practices Evolved

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

How Traditional Test Practices Evolved

Inevitably, The Project Plan is revised:

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

The Result: Practices Intended to Control Chaos

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

Becoming Agile: Shifting Roles

from last line of

Testing Directions: Maricks Model

Traditional & Agile Testing Contrasted

Manage & control it.

Comprehensive up front test

Plan as you go.

Only as much as absolutely

Formal entrance and exit criteria

Its not a relay race.

System-level, built by tool

All levels, built by anyone,

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

When creating test artifacts

Embrace Change: Plan Tests for Maintainability

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

How Not to Plan a Test Effort

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

Plan as Little as Possible

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

Favor Informal, Collaborative Tools

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

How Much Time Are We Spending on Test Documentation?

Monitor Documentation Costs

How Much Does Documentation Cost?

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

Keep the Documentation Simple

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

Which would you rather maintain?

Watch Verbose Wording

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

Hint: when people joke about

Do Away with the Signoff Sheet

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

Leverage Automation Investments

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

Use test automation to support early

Automation & Exploratory Testing

Copyright (c) 2004 - 2005, Quality Tree Software, Inc.

Agile Testing Synthesized