DYNAMIC TESTING - TECHNIQUES

Specification-based

Dynamic techniques Structure-based

Experience - based

• Specification-based techniques are black-box, also known as behavioral

techniques

• Structure-based techniques are white-box , also known as structural

techniques
Specification-based (black-box) testing techniques

• Specification-based techniques include both functional and nonfunctional

techniques (i.e. quality characteristics).

• Also known as 'black-box' or input/output driven testing techniques

• View the software as a black-box with inputs and outputs

• Requires no knowledge of how the system or component is structured

inside the box
BLACK BOX testing techniques:

• Equivalence Partitioning

• Boundary Value Analysis

• Decision tables

• State Transition testing

• Use Case Testing

 Equivalence partitioning

• Specification-based black-box technique

• Can be applied at any level of testing

• Often a good technique to use first

• Divides or partitions a set of test conditions into groups or sets that

can be considered the same
• Assumes that all the conditions in one partition will be treated in
the same way by the software

• Both valid and invalid partitions should be tested at least once

• Applied to specific situations or inputs

 Boundary value analysis

• Testing at the boundaries between partitions

• Can be applied at any level of testing

• Have both valid boundaries (in the valid partitions) and

invalid boundaries (in the invalid partitions).

• Six tests for boundary values.

• Applied to specific situations or inputs

 Decision table testing

• Applicable where different combinations of inputs result in

different actions.

• More focused on business logic or business rules.

• Good way to deal with combinations of inputs

• Help testers explore the effects of combinations of

different inputs
 State transition testing

• Used where some aspect of the system is described in a

(finite) number of different states.

• Basic parts of the state transition diagram

 the states that the software may occupy

 the transitions from one state to another
 the events that cause a transition
 the actions that result from a transition

• One event can cause only one action, but that the same
event - from a different state - may cause a different action
and a different end state.
 Use case testing

• Helps identify test cases that exercise the whole system on a

transaction by transaction basis from start to finish.
• Serves as the foundation for developing test cases mostly at the
system and acceptance testing levels.

 Use Cases
o Introduced by Ivar Jacobson

o Use Cases are the sequence of actions a system performs that

yields an observable result of value to a particular actor or user

• Use Case diagram is a set of scenarios that describe the interactions

between a user and a system.
• Use cases focus on what a system should do rather than
how it should do it.
 Use case testing

• Helps identify test cases that exercise the whole system on a

transaction by transaction basis from start to finish.
• Serves as the foundation for developing test cases mostly at the
system and acceptance testing levels.

 Use Cases
o Introduced by Ivar Jacobson

o Use Cases are the sequence of actions a system performs that

yields an observable result of value to a particular actor or user

• Use Case diagram is a set of scenarios that describe the interactions

between a user and a system.
• Use cases focus on what a system should do rather than
how it should do it.
• Are purely functional descriptions written in a formalism that
is completely separate from software design
• Describes what the actor does and what the actor sees – not what
inputs the system expects and what the system outputs are

 Use Case Diagram

o two main components of a use case diagram:

o Actor - a person, organization, or external system that plays a role in

one or more interactions with the system

o Use Case - Use cases describe the behavior of the system when the
sends one particular stimulus
o Associations - Associations between actors and use cases are indicated
in use case diagrams by solid lines.

o System boundary - the system boundary defines the limits of the system,
any functionality within the scope of the system,
represented by a rectangle drawn around the use cases

 Use case diagrams help exposing requirements and planning the project

 Use cases are powerful tools for analysts to use when partitioning
the functionality of a system.
 o Example of Use Case Diagram

The diagram can easily be expanded until a complete

description of the ordering system is derived capturing all
of the requirements that the system will need to perform

• Use cases can uncover integration defects, that is, defects caused by
the incorrect interaction between different components.

• Particularly good for finding defects that the users are most likely to
come across when first using the system
Experience-Based Techniques

• Non-Systematic techniques

• Tests based on a person's knowledge, experience, and creativity.

• Helps find those elusive defects that are hard to find using more
systematic approaches

Error guessing

Experience based techniques

Exploratory testing
 Error guessing

• Used as a complement to other more formal techniques

• Very much dependent on the skill of the tester

• No rules

• Think of situations in which the software may not be

able to cope
• Can list possible defects or failures and design
tests that attempt to produce them

• Can be built based on the tester's own experience or that

of other people, available defect and failure data, and
from common knowledge about why software fails.
 Exploratory testing

• Exploring, finding out about the software, what it does, what it doesn't do,
what works and what doesn't work

• Most useful when there are no or poor specifications and when time is
severely limited

• Also serve to complement other, more formal testing, helping to establish

greater confidence in the software

• Hands-on approach in which testers are involved in minimum planning

and maximum test execution

• Test design and test execution activities are performed in parallel typically
without formally documenting the test conditions, test cases or test scripts

• Test logging is undertaken as test execution is performed, documenting

the key aspects of what is tested, any defects found and any thoughts
about possible further testing.
EXERCISES

1. Equivalence Partitioning/Boundary Value Analysis exercise

Scenario: If you take the train before 9:30 am or in the afternoon after
4:00 pm until 7:30 pm ('the rush hour'), you must pay full fare.
A saver ticket is available for trains between 9:30 am and 4:00 pm, and
after 7:30 pm.
What are the partitions and boundary values to test the train times for
ticket types? Which are valid partitions and which are invalid partitions?
What are the boundary values? (A table may be helpful to organize
your partitions and boundaries.)
Derive test cases for the partitions and boundaries.
Are there any questions you have about this 'requirement'? Is anything
unclear?
2. Decision table exercise

Scenario: If you hold an 'over 60s' rail card, you get a 34% discount on
whatever ticket you buy. If you are traveling with a child (under 16), you can
get a 50% discount on any ticket if you hold a family rail card, otherwise you
get a 10% discount. You can only hold one type of rail card.
Produce a decision table showing all the combinations of fare types and
resulting discounts and derive test cases from the decision table.

3. State transition exercise

Scenario: A website shopping basket starts out as empty. As purchases are

selected, they are added to the shopping basket. Items can also be removed
from the shopping basket. When the customer decides to
check out, a summary of the items in the basket and the total cost are shown,
for the customer to say whether this is OK or not. If the contents and price are
OK, then you leave the summary display and go to the payment system.
Otherwise you go back to shopping (so you can remove items if you want).
a. Produce a state diagram showing the different states and transitions. Define
a test, in terms of the
sequence of states, to cover all transitions.
b. Produce a state table. Give an example test for an invalid transition.
STRUCTURE-BASED OR WHITE-BOX TECHNIQUES

• Structure-based testing techniques are dynamic testing techniques

that use the internal structure of the software to derive test cases.
• Commonly called 'white-box' or 'glass-box' techniques
• Requires knowledge of how the software is implemented / how it
works
• Serve two purposes: test coverage measurement and structural
test case design

Test coverage - the amount of testing performed by a set of test

cases (tests derived from specification-based techniques)

• Good way of generating additional test cases that are different

from existing tests
• Helps ensure more breadth of testing, in the sense that test
cases that achieve 100% coverage
Test Coverage:

Number of coverage items exercised

Coverage = --------------------------------------------------- x 100%
Total number of coverage items

‘Coverage item' is whatever we have been able to count and see whether
a test has exercised or used this item

Drawbacks

• 100% coverage does not mean 100% tested!

• Structure-based techniques can only look at a structure which is already
there
• Code coverage measures coverage of what has been written, i.e.
the code itself; it cannot say anything about the software that has
not been written.
Types of coverage

Coverage can be measured at component testing level, integration-testing

level or at system- or acceptance-testing levels.

• At System or Acceptance level, the coverage items may be requirements,

menu options, screens, or typical business transactions.
• Other coverage measures include things such as database structural
elements (records, fields and sub-fields) and files.
• At integration level, we could measure coverage of interfaces or specific
interactions that have been tested.

Coverage for specification-based techniques

• EP: percentage of equivalence partitions exercised (we could measure

valid and invalid partition coverage separately if this makes sense);

• BVA: percentage of boundaries exercised (we could also separate

valid and invalid boundaries if we wished);
 • Decision tables: percentage of business rules or decision table
columns tested;

• State transition testing: there are a number of possible coverage

measures:
- Percentage of states visited
- Percentage of (valid) transitions exercised (this is known as

Chow's 0 - switch coverage)

- Percentage of pairs of valid transitions exercised
('transition pairs' or Chow's 1-switch coverage)
- And longer series of transitions, such as transition triples,
quadruples, etc.
- Percentage of invalid transitions exercised (from the state

• The coverage measures table)

for specification-based techniques would apply at
whichever test level the technique has been used (e.g. system or component level)

• Coverage refers to the percentage of requirements that have been tested by a set
of tests when discussed by business analysts, system testers or users and is
measured by requirements management tool or a test management tool.
• Coverage from a programmers’ perspective refers to the coverage of code, where
the structural elements would be identified using a tool

• Statements and decision outcomes are both structures that can be measured in
code and there is good tool support for these coverage measures

• Code coverage is normally done in component and component integration testing

• In common with all structure-based testing techniques, code coverage

techniques are best used on areas of software code where more thorough
testing is required

• Safety-critical code; code that is vital to the correct operation of a system, and
complex pieces of code are all examples of where structure-based testing is
carried out

• Structural coverage techniques should always be used in addition to specification

-based and experience-based testing techniques rather than as an alternative to
them.
Measuring coverage:

• Coverage measurement often requires a tool support

• Coverage measurement of code - best done using tools

• Number of coverage measurement tools on the market

• Tools help increase quality and productivity of testing

- increase quality by ensuring that more structural aspects are
tested, so defects on those structural paths can be found
- increase productivity and efficiency by highlighting tests that
may be redundant,
Some examples of structure-based coverage

Statement coverage

Number of statements exercised

Statement coverage = --------------------------------------------- x 100%
Total number of statements

• We first check the coverage and then design test cases to achieve 100%
coverage

EXAMPLE:
1 READ A
2 READ B
3 C =A + 2*B
4 IF C> 50 THEN
5 PRINT large C
6 ENDIF
Analyze the coverage of a set of tests on our six-statement program:

TEST SET : Test 1: A = 3, B = 1

Test 2: A = 0, B = 20
Test 3: A = 47, B = 2

 We have covered five out of six statements, we have 83% statement

coverage (with three tests).
 We now need to design a test to cover statement 5

Test 4: A = 32, B = 26

 With TEST 4, we will have exercised all six of the statements

 Now statement coverage = 100%.

Notice that we measured coverage first, and then designed a test to cover
the statement that we had not yet covered
Decision coverage

• Applicable to IF statement, a loop control statement (e.g. DO-WHILE

or REPEAT-UNTIL), or a CASE statement, the outcome is either to
perform the code within the loop or not

Number of decision outcomes exercised

Decision coverage = -------------------------------------------------------- x 100%
Total number of decision outcomes

• Functional testing may achieve only 40% to 60% decision

coverage while a typical ad hoc testing may cover only 20% of
the decisions, leaving 80% of the possible outcomes untested.

• Decision coverage is stronger than statement coverage.

• Subsumes statement coverage - this means that 100% decision

coverage always guarantees 100% statement coverage
• 100% decision coverage guarantees 100% statement coverage, but not
the other way around!

Example:

READ A
READB
I F A>BTHENC = 0
ENDIF

• To achieve 100% statement coverage of this code segment just one

test case is required, one which ensures that variable A contains a
value that is greater than the value of variable B, for example, A = 15
and B = 6
• One test is sufficient for 100% statement coverage, but two tests are
needed for 100% decision coverage
• Decision coverage requires each decision to have had both a True
and False outcome
• To achieve 100% decision coverage, a second test case is necessary
where A is less than or equal to B to ensure that the decision statement
'IF A > B' has a False outcome.
• Decision structure is easier to see in a control flow diagram

For the same example:

READ A
READ B
I F A>B THEN C = 0
ENDIF

the control flow diagram READ A

would be

READ B

A>B C=0

ENDIF
Other structure-based techniques

• Branch Coverage

• Condition coverage

• Path coverage

• LCSAJ (linear code sequence and jump ) coverage

• Multiple condition coverage (condition combination coverage)

• MCDC (multiple condition decision coverage or modified

condition decision coverage)/ Condition determination coverage