Lecturer: Sebastian Coope

Ashton Building, Room G.18

E-mail: coopes@liverpool.ac.uk

COMP 201 web-page:

http://www.csc.liv.ac.uk/~coopes/comp201
Verification and Validation

COMP201 - Software Engineering 1

 Verification and Validation

Ensuring that a software system

meets a user's needs

COMP201 - Software Engineering 2

 Objectives
 To introduce software verification and validation and to
discuss the distinction between them
 To describe the program inspection process and its role
in V & V
 To explain static analysis as a verification technique
 To describe the Cleanroom software development
process

COMP201 - Software Engineering 3

 Verification vs Validation
 Verification:
"Are we building the product right"
 The software should conform to its specification

 Validation:
"Are we building the right product"
 The software should do what the user really requires

COMP201 - Software Engineering 4

 Verification vs Validation
 Verification should check the program meets its
specification as written in the requirements document
for example.
 This may involve checking that it meets it functional
and non-functional requirements
 Validation ensures that the product meets the
customers expectations
 This goes beyond checking it meets its specification; as
we have seen, system specifications don’t always
accurately reflect the real needs of users

COMP201 - Software Engineering 5

 The Verification & Validation Process
 As a whole life-cycle process - V & V must be
applied at each stage in the software process.
 Has two principal objectives
 The discovery of defects in a system
 The assessment of whether or not the system is usable in
an operational situation.

COMP201 - Software Engineering 6

 Static and Dynamic Verification
 Software inspections Concerned with analysis of the
static system representation to discover problems
(static verification)
 May be supplement by tool-based document and code
analysis
 Software testing Concerned with exercising and
observing product behaviour (dynamic verification)
 The system is executed with test data and its operational
behaviour is observed

COMP201 - Software Engineering 7

 An Example of Bad Code
public class Temperature {
// constructor
public Temperature(double initTemp) {
x = initTemp;
}
// calcTGrd function to calc. the value of a T gradiant
public double calcTGrd(float ZVAL) {
float a = x * x;
a = a * ZVAL * 3.8883;
return a;
}
public double x;
}
COMP201 - Software Engineering 8
 Static and Dynamic Verification
 System testing is only possible when an executable
version of the program is available
 This is therefore an advantage of incremental
development since a testable version of the system is
available at a fairly early stage
 New functionality can be checked as it is added to the
system and we can perform regression testing (we will
talk about this in a few slides time)
 Real data can be used as input to the system and we try
to observe any anomalies in the output

COMP201 - Software Engineering 9

 Static and Dynamic V&V
Static
verification

Requirements High-level Formal Detailed

specification Program
specification design design

Dynamic
Prototype
validation

COMP201 - Software Engineering 10

 Program Testing
 Can reveal the presence of errors NOT their
absence !!!
 A successful test is a test which discovers one or more
errors
 Program testing is the only validation technique for
non-functional requirements
 Should be used in conjunction with static verification
to provide full V&V coverage

COMP201 - Software Engineering 11

 Types of Testing
 Defect testing
 Tests designed to discover system defects.
 A successful defect test is one which reveals the presence
of defects in a system.

 Statistical testing
 Tests designed to reflect the frequency of user inputs.
Used for reliability estimation.

COMP201 - Software Engineering 12

 Verification & Validation Goals
Verification and validation should establish a degree
of confidence that the software is fit for purpose

 This does NOT mean completely free of defects

 The degree of confidence required depends upon several
different factors as we see on the next slide

COMP201 - Software Engineering 13

 Verification & Validation Goals
 Software function – A much higher level of confidence
that the system is fit for purpose is required for safety
critical systems that for prototype systems for example
 User expectations – Users sometimes have a low
expectation of software and are willing to tolerate some
system failures (although this is decreasing)
 Marketing environment – Competing programs must be
taken into account and the required schedule for
introducing the product to market. Cheaper products may
be expected to have more faults.

COMP201 - Software Engineering 14

 Testing and Debugging
Defect testing and debugging are distinct processes
 (!) Verification and validation is concerned with
establishing the existence of defects in a program
Debugging is concerned with
- locating and
- repairing these errors
 (!!) Debugging involves
 formulating a hypothesis about program behaviour
 then testing these hypotheses to find the system error

COMP201 - Software Engineering 15

 Testing and Debugging
 There is no simple process for debugging and it often
involves looking for patterns in test outputs with
defects and using a programmers skill to locate the
error
 Question: Recall the programs you have written in Java
for example so far. Were there errors in early versions?
How did you discover them and fix them? Were they
syntactic or semantic errors?
 Interactive debuggers provide a special run-time
environment with access to the symbol table and
program variables to aid error location. You can also
“step-through” the program line by line
COMP201 - Software Engineering 16
 More Incorrect Code!
public class Temperature {
// calcTGrd function to calc. the value of a T gradient
public double calcTGrd(float ZVAL) {
int a = (int) x * x Syntax error
(missing semicolon)
if(a = 1)
x = ZVAL * 3.8883; Semantic error (should
use double equals)
return a;
}
public double x;
}
COMP201 - Software Engineering 17
 Syntax and Semantic Errors
 A syntax error should be caught by the compiler which
will (usually) indicate the location the error occurred
in and the type of error.
 A semantic error (also called a logical error) can occur
in a program which compiles and runs, but produces
incorrect output on some (or all) input (e.g. An
incorrect algorithm or mistake in a formulae etc.)
 Semantic errors are often harder to detect since the
compiler may not be able to indicate where/what the
problem is.

COMP201 - Software Engineering 18

 Testing and Debugging
 Once errors are located, it is necessary to correct the
program code and re-test the program
 Regression testing – after fixing a defect, it is
advisable to retest the program with all previous test
data to make sure the “fix” has not introduced new
problems
 This is not always feasible due to costs
 Experience has shown that a large proportion of fault
repairs introduce new errors or are incomplete

COMP201 - Software Engineering 19

 The Debugging Process

Test Test
results Specification
cases

Locate Design Repair Re-test

error error repair error program

COMP201 - Software Engineering 20

 V & V Planning
 Careful planning is required to get the most out of
testing and inspection processes
 Planning should start early in the development process
 The plan should identify the balance between
static verification and testing
 Test planning is about defining standards for the testing
process rather than describing product tests

COMP201 - Software Engineering 21

 The V-model of Development

Requir ements System System Detailed

specification specification design design

System Sub-system Module and

Acceptance
integration integration unit code
test plan
test plan test plan and tess

Acceptance System Sub-system

Service
test integration test integration test

This diagram shows how test plans should be derived from

the system specification and design.
COMP201 - Software Engineering 22
 The Structure of a Software Test Plan
 The testing process - a description of the major phases of
the testing process
 Requirements traceability – testing should ensure that all
requirements are individually tested
 Tested items – Specify the products of the software process
to be tested
 Testing schedule – An overall schedule for the testing of the
software is required and resources (time and personnel) must
be allocated as part of the general project schedule

COMP201 - Software Engineering 23

 The Structure of a Software Test Plan
 Test recording procedures – The results of tests must be
systematically recorded, it is not enough to simply run the
tests. This allows an audit of the testing process to check it
has been carried out correctly (imagine a safety critical
system; procedures for auditing the tests are often necessary)
 Hardware and software requirements – This part of the
document sets out a list of software tools required and the
estimated hardware utilisation
 Constraints – Any constraints affecting the testing process
should be anticipated in this section

COMP201 - Software Engineering 24

 Software Inspections
 Involve people examining the source representation
with the aim of discovering anomalies and defects
 Does not require execution of a system so it may be
used before the implementation phase
 May be applied to any representation of the system
(requirements, design, test data, etc.)
 Very effective technique for discovering errors

COMP201 - Software Engineering 25

 Software Inspections
 Incomplete versions of the system can be inspected
without additional costs – specialised test harnesses that
work on only a part of the program are not required
 As well as program defects, inspections can consider
broader quality attributes such as compliance with
standards, portability and maintainability
 Poor programming style and inefficiencies can be found
which could make the system difficult to maintain and
update

COMP201 - Software Engineering 26

 Inspection Success
 Many different defects may be discovered in a single
inspection, there is no “interaction” between errors to
be concerned with. In testing, one defect, may mask
another, so several executions are required
 They reuse domain and programming knowledge so
reviewers are likely to have seen the types of error that
commonly arise

COMP201 - Software Engineering 27

 Inspections and Testing
 Inspections and testing are complementary and not
opposing verification techniques
 Both should be used during the V & V process

 Inspections can check conformance with a specification

but not conformance with the customer’s real
requirements
 Also inspections cannot check non-functional
characteristics such as performance, usability, etc.
(Emergent properties).

COMP201 - Software Engineering 28

 Lecture Key Points
 Verification and validation are not the same thing.
 Verification shows conformance with specification;
 Validation shows that the program meets the customer’s
needs
 Test plans should be drawn up to guide the testing
process.
 Program inspections are very effective in discovering
errors
 Different types of systems and software development
processes require different levels of verification and
validation

COMP201 - Software Engineering 29