UnitTesting Lecture s05
UnitTesting Lecture s05
UnitTesting Lecture s05
Unit Testing
Adapted from the SWENET Module
(http://www.swenet.org)
Unit Testing - 1
Overview
• Testing Fundamentals
• Unit Test Planning
• Test Implementation
• Object Oriented Testing
• References
Unit Testing - 2
Terms for Quality Problems
• error - a mistake made by a human (in a
software development activity)
• defect (or fault) - the result of introducing
an error into a software artifact (SRS,
SDS, code, etc.)
• failure - a departure from the required
behavior for a system
Unit Testing - 3
Testing Fundamentals (1)
• Software Testing is a critical element of
Software Quality Assurance
• It represents the ultimate review of the
requirements specification, the design, and
the code.
• It is the most widely used method to insure
software quality.
• Many organizations spend 40-50% of
development time in testing.
Unit Testing - 4
Testing Fundamentals (2)
• Testing is concerned with establishing the
presence of program defects.
• Debugging is concerned with finding where
defects occur (in code, design or
requirements) and removing them. (fault
identification and removal)
• Even if the best review methods are used
(throughout the entire development of
software), testing is necessary.
Unit Testing - 5
Testing Fundamentals (3)
• Testing is the one step in software
engineering process that could be viewed as
destructive rather than constructive.
– “A successful test is one that breaks the
software.” [McConnell 1993]
• A successful test is one that uncovers an as
yet undiscovered defect.
• Testing can not show the absence of defects,
it can only show that software defects are
present.
• For most software exhaustive testing is not
possible.
Unit Testing - 6
Testing Steps
Design System Other Customer User
Unit code
Unit
test specifications functional software requirements environment
requirements requirements specification
Unit code
Unit
test
.
. Integration Function Performance Acceptance Installation
test test test test test
.
test SYSTEM
IN USE!
[Pfleeger 2001]
Unit Testing - 7
Unit Testing
• Unit Testing checks that an individual program unit
(subprogram, object class, package, module)
behaves correctly.
– Static Testing
• testing a unit without executing the unit code
– Dynamic Testing
• testing a unit by executing a program unit using test data
• There is some debate about what constitutes a
“unit”. Here some common definitions of a unit:
– the smallest chunk that can be compiled by itself
– a stand alone procedure of function
– something so small that it would be developed by a
single person
Unit Testing - 8
• First, unit testing in general
• Second, the role of unit testing in test-
driven design (TDD)
– Spring 05: We’ve done this already!
Unit Testing - 9
Unit Testing Techniques
T e stin g T e c hn iqu es
P ro gram P ro v ing
E qu iv ale nc e P a rtitio ning B a sic P a th T e sting
C a us e -E ffec t G ra ph ing
R a nd om T e sting
Unit Testing - 10
Static Testing
• Symbolic Execution
– works well for small programs
• Program Proving
– difficult and costly to apply
– does not guarantee correctness
• Anomaly Analysis
– unexecutable code (island code)
– uninitialized variables
– unused variables
– unstructured loop entry and exit
Unit Testing - 11
Dynamic Testing
• Black Box Techniques
– design of software tests relies on module
description to devise test data
– uses inputs, functionality, outputs in the
architectural design
– treats module like a “black box”
• White Box Techniques
– design of software tests relies on module source
code to devise test data
– analyze the module algorithm in the detailed
design
– treats module like a “ white box” or "glass box"
Unit Testing - 12
Common Testing Terms (1)
• test class
– a set of input data that is identified for testing
– Note difference from JUnit term
• test case
– specific data that is chosen for testing a test class.
• test Suite
– a set of test cases that serve a particular testing goal
• exhaustive testing
– test all logical paths in a software module
Unit Testing - 13
Example: Exhaustive Testing
Unit Testing - 14
Basic Path Testing
• Basic Path Testing is a white box testing technique
that consists of the following steps:
– convert the unit into a “flow graph”
• A flow graph is a directed graph (for an algorithm) with a
"start node" and a "terminal node" (could have multiple
terminal nodes, but not ideal).
– compute a measure of the unit's logical
complexity
– use the measure to derive a “basic” set of
execution paths for which test cases are
determined.
Unit Testing - 15
Flow Graph Example
procedure XYZ is
A,B,C: INTEGER; 1
begin
1. GET(A); GET(B); 2
2. if A > 15 then
3. if B < 10 then
3 9
4. B := A + 5;
5. else
6. B := A - 5;
4 6
7. end if
8. else
9. A := B + 5;
7
10. end if;
end XYZ;
10
Unit Testing - 16
Cyclomatic Complexity
• The cyclomatic complexity (McCabe complexity) is a metric,
V(G), that describes the logical complexity of a flow graph, G.
V(G) can be computed using any of the following formulae:
– V(G) = E - N + 2
• where E = number of edges in G and N = number of
nodes in G
– V(G) = R
• where R = number of bounded regions in G
– V(G) = P + 1
• where P = number of predicates
• Studies have shown:
– V(G) is directly related to the number of errors in source
code
– V(G) = 10 is a practical upper limit for testing
Unit Testing - 17
Basic Path Set
• An execution path is a set of nodes and directed
edges in a flow graph that connects (in a directed
fashion) the start node to a terminal node.
• Two execution paths are said to be independent if
they do not include the same set of nodes and
edges.
• A basic set of execution paths for a flow graph is an
independent set of paths in which all nodes and
edges of the graph are included at least once. V(G)
provides an upper bound on the number of
independent paths needed.
Unit Testing - 18
Equivalence Partitioning
• Equivalence partitioning is an approach to black
box testing that divides the input domain of a
program into classes of data from which test cases
can be derived.
• Example: Suppose a program computes the value
of the function ( X 1) ( X 2) . This function
defines the following valid and invalid equivalence
classes:
X < = -2 valid
-2 < X < 1 invalid
X >= 1 valid
• Test cases would be selected from each
equivalence class.
Unit Testing - 19
Boundary Value Analysis (1)
• Boundary Value Analysis is a black box
testing technique that where test cases
are designed to test the boundary of an
input domain. Studies have shown that
more errors occur on the "boundary" of an
input domain rather than on the "center".
• Boundary value analysis complements
and can be used in conjunction with
equivalence partitioning.
Unit Testing - 20
Boundary Value Analysis (2)
• In previous example, after using equivalence
partitioning to generate equivalence classes,
boundary value analysis would dictate that
the boundary values of the three ranges be
included in the test data. That is, we might
choose the following test cases (for a 32 bit
system):
Unit Testing - 23
Right-BICEP
• Right: Are the results right?
– Validate results
– Does the expected result match what the method
does?
• If you don’t know what “right” would be, then
how can you test? How do you know if your
code works?
– Perhaps requirements not known or stable
– Make a decision. Your tests document what you
decided. Reassess if it changes later.
Unit Testing - 24
Right-BICEP (cont’d)
• Boundary Conditions
– See earlier slides, and also consider:
– Garbage input values
– Badly formatted data
– Empty or missing values (0, null, etc.)
– Values out of reasonable range
– Duplicates if they’re not allowed
– Unexpected orderings
Unit Testing - 26
Right-BICEP
• Use this acronym, CORRECT, to remember:
– Conformance
– Ordering
– Range
– Reference
• Does code reference anything external outside of its control?
– Existence
– Cardinality
– Time (absolute and relative)
• Are things happening in order? On time? In time?
Unit Testing - 27
Right-BICEP
Unit Testing - 28
Right-BICEP
Unit Testing - 29
Right-BICEP
• Force Error Conditions
• Some are easy:
– Invalid parameters, out of range values, etc.
• Others not so easy
– See JUnit slide on how to force exceptions
• Failures outside your code:
– Out of memory, disk full, network down, etc.
– Can simulate such failures
• Example: use Mock Objects
Unit Testing - 30
Right-BICEP
• Performance
– Perhaps absolute performance, or
– Perhaps how performance changes as input grows.
– Perhaps separate test suite in JUnit
Unit Testing - 31
Unit Testing - 32
Example follows…
Unit Testing - 33
GCD Test Planning (1)
• Let’s look at an example of testing a unit
designed to compute the “greatest common
divisor” (GCS) of a pair of integers (not both
zero).
– GCD(a,b) = c where
• c is a positive integer
• c is a common divisor of a and b (e.g., c divides a and c
divides b)
• c is greater than all other common divisors of a and b.
– For example
• GCD(45, 27) = 9
• GCD (7,13) = 1
• GCD(-12, 15) = 3
• GCD(13, 0) = 13
• GCD(0, 0) undefined Unit Testing - 34
GCD Test Planning (2)
• How do we proceed to determine the tests
cases?
1. Design an algorithm for the GCD function.
2. Analyze the algorithm using basic path analysis.
3. Determine appropriate equivalence classes for the
input data.
4. Determine the boundaries of the equivalence
classes.
5. Then, choose tests cases that include the basic
path set, data form each equivalence class, and
data at and near the boundaries.
Unit Testing - 35
GCD Algorithm
note: Based on Euclid’s algorithm
1. function gcd (int a, int b) { 1
2. int temp, value;
3. a := abs(a);
4. b := abs(b);
5. if (a = 0) then 5
6. value := b; // b is the GCD
7. else if (b = 0) then 7
8. raise exception;
6
9. else 9
10. loop
11. temp := b;
12. b := a mod b; 1
13. a := temp; 0
14. until (b = 0) 17
15. value := a;
16. end if;
17. return value;
18
18. end gcd Unit Testing - 36
GCD Test Planning (3)
• Basic Path Set
– V(G) = 4
– (1,5,6,17,18), (1,5,7,18), (1,5,7,9,10,17,18),
(1,5,7,9,10,9,10,17,18)
• Equivalence Classes
– Although the the GCD algorithm should accept any integers as
input, one could consider 0, positive integers and negative
integers as “special” values. This yields the following classes:
• a < 0 and b < 0, a < 0 and b > 0, a > 0 and b < 0
• a > 0 and b > 0, a = 0 and b < 0, a = 0 and b > 0
• a > 0 and b = 0, a > 0 and b = 0, a = 0 and b = 0
• Boundary Values
– a = -231, -1, 0, 1, 231-1 and b = -231, -1, 0, 1, 231-1
Unit Testing - 37
GCD Test Plan
Test Description / Data Expected Results Test Experience / Actual Results
Basic Path Set
path (1,5,6,17,18) (0, 15) 15
path (1,5,7,18) (15, 0) 15
path (1,5,7,9,10,17,18) (30, 15) 15
path (1,5,7,9,10,9,10,17,18) (15, 30) 15
Equivalence Classes
a < 0 and b < 0 (-27, -45) 9
a < 0 and b > 0 (-72, 100) 4
a > 0 and b < 0 (121, -45) 1
a > 0 and b > 0 (420, 252) 28
a = 0 and b < 0 (0, -45) 45
a = 0 and b > 0 (0 , 45) 45
a > 0 and b = 0 (-27, 0) 27
a > 0 and b = 0 (27, 0) 27
a = 0 and b = 0 (0 , 0) exception raised
Boundary Points
(1 , 0) 1
(-1 , 0) 1
(0 , 1) 1
(0 , -1) 1
(0 , 0) (redundant) exception raised
(1, 1) 1
(1, -1) 1
(-1, 1) 1
(-1, -1) 1
Unit Testing - 41
Unit Testing in TDD
• Motto: “Clean code that works.” (Ron
Jeffries)
• Unit testing has “broader goals” that just
insuring quality
– Improve developers lives (coping,
confidence)
– Support design flexibility and change
– Allow iterative development with working
code early
Unit Testing - 42
Unit Testing Benefits
Unit Testing - 44
Assigned Readings
• http://junit.sourceforge.net/doc/testinfected/testing.htm
• http://junit.sourceforge.net/doc/cookstour/cookbook.htm
Unit Testing - 45
Object-Oriented Testing Issues
• The object class is the basic testing unit for
system developed with OO techniques.
– This is especially appropriate if strong cohesion and
loose coupling is applied effectively in the class
design.
• An incremental testing approach is dictated by
– The package/class design decomposition.
– The incremental nature of the development process.
• Information hiding restricts/constrains testing to
using a white-box approach.
• Encapsulation motivates testing based on the
class interface.
Unit Testing - 46
Class Testing (1)
• Class test cases are created by examining the specification of the
class.
– This is more difficult if the class is a subclass and inherits data
and behavior from a super class. A complicated class hierarchy
can be pose significant testing problems.
• If you have a state model for the class, test each transition - devise
a driver that sets an object of the class in the source state of the
transition and generates the transition event.
• Class Constraints/Invariants should be incorporated into the class
test.
• All class methods should be tested, but testing a method in
isolation from the rest of the class is usually meaningless
– In a given increment, you may not implement all methods; if so,
create stubs for such methods.
Unit Testing - 47
Class Testing (2)
• To test a class, you may need instances
of other classes.
• Interaction diagrams provide useful
guidance in constructing class test cases:
– They provide more specific knowledge about
how objects of one class interact with
instances of other classes.
– Messages sent to a class object provide a
guidance on which test cases are most
critical for that class.
Unit Testing - 48
Method Testing (1)
• A public method in a class is typically tested
using a black-box approach.
– Start from the specification, no need to look at the
method body.
– Consider each parameter in the method signature,
and identify its equivalence classes.
– Incorporate pre-conditions and post-conditions in
your test of a method.
– Test exceptions.
• For complex logic, also use white-box testing or
static testing.
Unit Testing - 49
Method Testing (2)
• For private methods, either
– modify the class (temporarily) so that it can
be tested externally:
• change the access to public
• or incorporate a test driver within the class
– or use static test methods, such as program
tracing or symbolic execution
Unit Testing - 50
Incremental Testing
• Incremental testing indicates that we test each
unit in isolation, and then integrate each unit,
one at a time, into the system, testing the
overall system as we go.
• Classes that are dependent on each other
called class clusters, are good candidates for
an increment integration.
• Candidate class clusters:
– Classes in a package
– Classes in a class hierarchy.
– Classes associated with the interaction diagram for a
use case.
Unit Testing - 51
Integration Test Plan
• An Integration Test checks and verifies that the
various components in a class cluster
communicate properly with each other.
• With this plan, the emphasis is not on whether
system functionality is correctly implemented,
but rather do all the cluster classes “fit” together
properly.
• A single test plan, for each cluster, which tests
the communication between all cluster
components is typically sufficient.
Unit Testing - 52
Increment Test Planning
• Determine the classes to be tested in an
increment.
• Determine the appropriate “class clusters”.
• Develop a unit test plan and test driver for each
class.
• Develop an integration test plan and test driver
for each class cluster.
• Develop a test script that details the
components to be tested and the order in which
the plans will be executed.
Unit Testing - 53
Testing Tools
• There are a number of tools that have been
developed to support the testing of a unit or
system.
– googling “Software Testing Tools” will yield
thousands of results.
• JUnit testing (http://www.junit.org/index.htm) is
a popular tool/technique that can be integrated
into the development process for a unit coded
in Java.
Unit Testing - 54
References
• [Beck 2004] Beck, K., and Gamma, E. Test Infected: Programmers
Love Writing Tests, http://members.pingnet.ch/gamma/junit.htm,
accessed July 2004.
• [Binder 1999] Binder, R.V., Testing Object-Oriented Systems,
Addison-Wesley, 1999.
• [Humphrey 1995] Humphrey, Watts S., A Discipline for Software
Engineering, Addison Wesley, 1995.
• [McConnell 1993] McConnell, Steve, Code Complete, A Practical
Handbook of Software Construction, Microsoft Press, 1993.
• [Jorgensen 2002] Jorgensen, Paul C., Software Testing: A
Craftsman’s Approach, 2nd edition, CRC Press, 2002.
• [Pfleeger 2001] Pfleeger, S., Software Engineering Theory and
Practice, 2nd Edition, Prentice-Hall, 2001.
• [Pressman 2005] Pressman, R.S., Software Engineering: A
Practitioner’s Approach, 6th edition, McGraw-Hill, 2005.
Unit Testing - 55