CHAPTER 3.

SOFTWARE
TESTING
METHODS AND TECHNIQUES
I. SOFTWARE TESTING APPROACHES
 STATIC TESTING
• Is a type of software testing in which software application is tested
without code execution.

• This is the process of testing some documentation products

• The primary goal of Static Testing is to reduce defects in the software by

reducing defects in the documentation from which the software is
developed
 Static Techniques
• Do not execute code
• People techniques
• Specific techniques
• Desk checking
• Developer tests his/her own documentation
• Walkthroughs
• Developer “walks” the participants through the documentation
• Peer Reviews/Inspections
• More formal – documentation is distributed beforehand
 Benefits of reviews
• Development productivity improvement
• Reduced development timescales
• Reduced testing time and cost
• Lifetime cost reductions
• Reduced fault levels
• Improved customer relations
•…
 Reviews are cost-effective
• 10 times reduction in faults reaching test, testing cost reduced by 50% to
80%
• Freedman & Weinberg, Handbook of Walkthroughs, Inspections & Technical
Reviews
• reduce faults by a factor of 10
• Yourdon, Structured Walkthroughs
• 25% reduction in schedules, remove 80% - 95% of faults at each stage, 28
times reduction in maintenance cost, many others
• Gilb & Graham, Software Inspection
 What can be Inspected?
• Anything written down can be Inspected
• policy, strategy, business plans, marketing or advertising material, contracts
• system requirements, feasibility studies, acceptance test plans
• test plans, test designs, test cases, test results
• system designs, logical & physical
• software code
• user manuals, procedures, training material
 What can be reviewed?
• anything which could be Inspected
• i.e. anything written down
• plans, visions, “big picture”, strategic directions, ideas
• project progress
• work completed to schedule, etc.
• “Should we develop this” marketing options
What to review / Inspect?
Tests

Requirements Accept. Test

Tests

Functions System Test

Tests

Design Integration T
Tests

Code Unit Test

9
 Costs of reviews
• Rough guide: 5%-15% of development effort
• half day a week is 10%
• Effort required for reviews
• planning (by leader / moderator)
• preparation / self-study checking
• meeting
• fixing / editing / follow-up
• recording & analysis of statistics / metrics
• process improvement (should!)
TYPES OF REVIEW
 Types of review of documents
• Informal Review (undocumented)
• widely viewed as useful and cheap (but no one can prove it!). A helpful first step for chaotic organisations.
• Technical Review (or peer review)
• includes peer and technical experts, no management participation. Normally documented, fault-finding.
Can be rather subjective.
• Decision-making Review
• group discusses document and makes a decision about the content, e.g. how something should be done,
go or no-go decision, or technical comments
• Walkthrough
• author guides the group through a document and his or her thought processes, so all understand the
same thing, consensus on changes to make
• Inspection
• formal individual and group checking, using sources and standards, according to generic and specific rules
and checklists, using entry and exit criteria, Leader must be trained & certified, metrics required
 Reviews in general 1
• Objectives / goals
• validation & verification against specifications & standards
• achieve consensus (excluding Inspection)
• process improvement (ideal, included in Inspection)
 Reviews in general 2
• Activities
• planning
• overview / kickoff meeting (Inspection)
• preparation / individual checking
• review meeting (not always)
• follow-up (for some types)
• metrics recording & analysis (Inspections and sometimes reviews)
 Reviews in general 3
• Roles and responsibilities
• Leader / moderator - plans the review / Inspection, chooses participants, helps &
encourages, conducts the meeting, performs follow-up, manages metrics
• Author of the document being reviewed / Inspected
• Reviewers / Inspectors - specialised fault-finding roles for Inspection
• Managers - excluded from some types of review, need to plan project time for review /
Inspection
• Others: e.g. Inspection/ review Co-ordinator
 Reviews in general 4
• Deliverables
• Changes (edits) in review product
• Change requests for source documents (predecessor documents to product being
reviewed / Inspected)
• Process improvement suggestions
• to the review / Inspection process
• to the development process which produced the product just reviewed / Inspected
• Metrics (Inspection and some types of review)
 Reviews in general 5
• Pitfalls (they don’t always work!)
• Lack of training in the technique (especially Inspection, the most formal)
• Lack of or quality of documentation - what is being reviewed / Inspected
• Lack of management support
• Failure to improve processes
 Inspection is different

• the document to be • not just product,

reviewed is given out in sources
advance
• chunk or sample
• typically dozens of pages to
review
• instructions are "please • training, roles
review this"
Inspection is different

• some people have time to  entry criteria to

look through it and make meeting, may not be
comments before the worth holding
meeting (which is difficult to
arrange)
• the meeting often lasts for
hours  2 max., often much
shorter

19
STATIC ANALYSIS
 What can static analysis do?
• Remember: static techniques do not execute the code

• A form of automated testing

• check for violations of standards
• check for things which may be a fault
• Descended from compiler technology
• a compiler statically analyses code, and “knows” a lot about it, e.g. variable usage; finds
syntax faults
• static analysis tools extend this knowledge
• can find unreachable code, undeclared variables, parameter type mis-matches, uncalled
functions & procedures, array bound violations, etc.
 Data flow analysis
• This is the study of program variables
• variable defined* where a value is stored into it
• variable used where the stored value is accessed
• variable is undefined before it is defined or when it goes out of
scope

x is defined, y and z are used

x=y+z
IF a > b THEN read(S)

a and b are used, S is defined

*defined should not be confused with declared
22
Data flow analysis faults

n := 0
read (x) Data flow anomaly: n is
n := 1 re-defined without being used
while x > y do Data flow fault: y is used
begin before it has been defined
read (y) (first time around the loop)
write( n*y)
x := x - n
end

23
 Control flow analysis
• Highlights:
• nodes not accessible from start node
• infinite loops
• multiple entry to loops
• whether code is well structured, i.e. reducible
• whether code conforms to a flowchart grammar
• any jumps to undefined labels
• any labels not jumped to
• cyclomatic complexity and other metrics

24
 Cyclomatic complexity
• cyclomatic complexity is a measure of the complexity of a flow graph
• (and therefore the code that the flow graph represents)
• the more complex the flow graph, the greater the measure
• it can most easily be calculated as:
• complexity = number of decisions + 1

25
Which flow graph is most complex?
1 What is the cyclomatic complexity?

2 3 5
26
Example of control flow graph init

do
Pseudo-code:
Result = 0 if r=r+1
Right = 0
DO WHILE more Questions
IF Answer = Correct THEN end
Right = Right + 1
ENDIF
res
END DO
Result = (Right / Questions)
IF Result > 60% THEN if pass
Print "pass"
ELSE fail
Print "fail”
ENDIF
end
27
 Other static metrics
• lines of code (LOC)
• operands & operators (Halstead’s metrics)
• fan-in & fan-out
• nesting levels
• function calls
• OO metrics: inheritance tree depth, number of methods, coupling &
cohesion
• symbolic evaluation

28
 Limitations and advantages
• Limitations
• cannot distinguish "fail-safe" code from programming faults or anomalies (often
creates overload of spurious error messages)
• does not execute the code, so not related to operating conditions
• Advantages
• can find faults difficult to "see"
• gives objective quality assessment of code

29
 Summary
• Reviews help to find faults in development and test documentation, and
should be applied early
• Types of review: informal, walkthrough, technical / peer review,
Inspection
• Static analysis can find faults and give information about code without
executing

30
DYNAMIC TESTING
 DYNAMIC TESTING
• Dynamic Testing is a software testing method used to test the dynamic
behaviour of software code.
• Under Dynamic Testing, a code is executed.
• It checks for functional behavior of software system, memory/cpu usage
and overall performance of the system.
 Dynamic Testing Example

• Suppose we are testing a Login Page where we have two fields say
"Username" and "Password" and the Username is restricted to
Alphanumeric.
• When the user enters Username as "Guru99", the system accepts the
same. Where as when the user enters as Guru99@123 then the
application throws an error message. This result shows that the code is
acting dynamically based on the user input.
 DYNAMIC TESTING
• Dynamic testing is when you are working with the actual system by
providing an input and comparing the actual behavior of the application
to the expected behavior. In other words, working with the system with
the intent of finding errors.

• So based on the above statements we can say or conclude that dynamic

testing is a process of validating software applications as an end user
under different environments to build the right software.
 What does dynamic testing do?
•  ensure that software works properly during and after the installation of
the software ensuring a stable application without any major flaws( this
statement is made because no software is error free, testing only can
show presence of defects and not absence)
•  ensure consistency to the software
• Example: Let's say My Accounts field displays amount as 25,000 and
Funds Transfer as $25,000 and Bill pay screen as $25000 though the
amount is the same, the way amount is displayed is not the same hence
making the software nonconsistent.
Types of Dynamic Testing
 Dynamic Testing Techniques
• In STLC, we can say that actual Dynamic Testing Process starts from Test
Case Design, let's discuss each activity in details.
• What is Test design and Implementation: In this phase we identify the,
• Features to be tested
• Derive the Test Conditions
• Derive the coverage Items
• Derive the Test Cases
• Test Environment Setup
• We have to ensure that Testing Environment should always be similar to the Production
environment, in this phase we have to install the build and manage the test machines.
• Test Execution
• During this phase, test cases are actually executed.
• Bug report captured
• Based on the Execution if the Expected and Actual Results are not same then the Test case has to
be marked as Fail and a Bug should be logged.
 Advantages of Dynamic Testing
• Dynamic Testing can reveal the uncovered defects that are considered to
be too difficult or complicated and which cannot be covered through
static Analysis
• In Dynamic Testing, we execute the software, end to end, ensuring error
free software which in turn increases the quality of a product and project.
• Dynamic Testing becomes an essential Tool for detecting any security
Threats
 Disadvantages of Dynamic Testing
• Dynamic Testing is Time Consuming because it executes the
application/software or code which requires huge amount of Resources
• Dynamic Testing increases the cost of project/product because it does not
start early in the software lifecycle and hence any issues fixed in later
stages can result in an increase of cost.