Software Testing

Unit No.1
In general, testing is finding out how well
something works. In terms of human beings,
testing tells what level of knowledge or skill has
been acquired. In computer hardware and
software development, testing is used at key
checkpoints in the overall process to determine
whether objectives are being met.
Software Testing:
is the act of examining the artifacts and the
behavior of the software under test by validation
and verification? Software testing can also
provide an objective, independent view of the
software to allow the business to appreciate and
understand the risks of software
implementation.
Software Testing is a method to check whether
the actual software product matches expected
requirements and to ensure that software
product is Defect free.
It involves execution of software/system
components using manual or automated tools to
evaluate one or more properties of interest. The
purpose of software testing is to identify errors,
gaps or missing requirements in contrast to
actual requirements.
Software Errors:
An error is a deviation from accuracy or
correctness” and “A software bug is an error,
flaw, failure, or fault in a computer program or
system that causes it to produce an incorrect or
unexpected result, or to behave in unintended
ways “.
• Reasons:-
Error is a variance of the actual result from the
expected result.
Errors are a category of software bugs.
Errors can be introduced as result of incomplete
or inaccurate requirements or due to human data
entry problems.
Software Errors:-
1) Functionality Errors:
Functionality is a way the software is intended
to behave. Software has a functionality error if
something that you expect it to do is hard,
awkward, confusing, or impossible.
E.x.
Expected Functionality for Cancel button is that
the ‘Create new project’ window should close
and none of the changes should be saved (i.e. no
new project must be created). If the Cancel
button is not clickable then it is a functionality
error
2) Communication Errors:
These errors occur in communication from
software to end-user. Anything that the end user
needs to know in order to use the software
should be made available on screen.
Few examples of communication errors are –
No Help instructions/menu provided, features
that are part of the release but are not
documented in the help menu, a button named
‘Save’ should not erase a file etc.
3) Missing command errors:
This happens to occur when an expected
command is missing. See this screenshot:
E.X.:-Cancel’ option/button is not provided to
the user, this is a missing command error.
4) Syntactic Error:
Syntactic errors are misspelled words or
grammatically incorrect sentences and are very
evident while testing software GUI. Please note
that we are NOT referring to syntax errors in
code. The compiler will warn the developer
about any syntax errors that occur in the code
5) Error handling errors:
Any errors that occur while the user is
interacting with the software needs to be
handled in a clear and meaningful manner. If
not, it is called as an Error Handling Error.
Take a look at this image. The error message
gives no indication of what the error actually is.
Is it missing mandatory field, saving error, page
loading error or is it a system error? Hence, this
is an ‘Error Handing Error’.
6) Calculation Errors:
These errors occur due to any of the following
reasons:
• Bad logic
• Incorrect formulae
• Data type mismatch
• Coding errors
• Function call issues , etc.
In 1999, NASA lost its Mars climate orbiter
because one of the subcontractors NASA
employed had used English units instead of the
intended metric system, which caused the
orbiter’s thrusters to work incorrectly. Due to
this bug, the orbiter crashed almost immediately
when it arrived at Mars.
7) Control flow errors:
The control flow of a software describes what it
will do next and on what condition.
For example, consider a system where user has
to fill in a form and the options available to user
are: Save, Save and Close, and Cancel. If a user
clicks on ‘Save and Close’ button, the user
information in the form should be saved and the
form should close. If clicking on the button does
not close the form, then it is a control flow error.
Testing Objectives:
• To find any defects or bugs that may have
been created when the software was being
developed
• To increase confidence in the quality of the
software
• To prevent defects in the final product
• To ensure the end product meets customer
requirements as well as the company
specifications
• To provide customers with a quality product
and increase their confidence in the
company
• Testing Principles:
• Optimal testing – it's not possible to test
everything so it's important to determine the
 optimal amount. The decision is made using
a risk assessment. This assessment will
uncover the area that is most likely to fail
and this is where testing should take place.
• Pareto Principle – this principle states that
approximately 80% of problems will be
found in 20% of tests. However, there is a
flaw in this principle in that repeating the
same tests over and over again will mean no
new bugs will be found.
• Review and Revise – repeating the same
tests will mean that the methods will
eventually become useless for uncovering
new defects. To prevent this from happens
only requires the tests to be reviewed and
 revised on a regular basis. Adding new tests
will help to find more defects.
• Defects that are present – testing reduces the
probability of the being a defect in the final
product but does not guarantee that a defect
won't be there. And even if you manage to
make a product that's 99% bug free, the
testing won't have shown whether the
software meets the needs of clients.
• Meeting customer needs – testing a product
for the wrong requirements is foolhardy.
Even if it is bug free it may still fail to meet
customer requirements.
• Test early – it's imperative that testing starts
as soon as possible in the development of a
product.
• Test in context – test a product in
accordance with how it will be used
Software is not identical and will be
developed to meet a certain need rather than
a general one. Different techniques,
methodologies, approach and type of testing
can be used depending on the applications
planned use.
Software Testing Fundamentals (STF):
STF: testing: The practice of investigating a
software / system under test so as to ensure that
it is of the highest quality.
Why Software Testing?
Software that does not work correctly can lead
to many problems such as:
Delay / Loss of time
Futility / Loss of effort
Wastage / Loss of money
Shame / Loss of business reputation
Injury or death
The three main goals of Software Testing are:
Defect Detection: Find defects / bugs in the
software during all stages of its development
(earlier, the better).
Defect Prevention: As a consequence of defect
detection, help anticipate and prevent defects
from occurring at later stages of development or
from recurring in the future.
User Satisfaction: Ensure customers / users are
satisfied that their requirements (explicit or
implicit) are met.
The three main goals of Software Testing are:
Defect Detection: Find defects / bugs in the
software during all stages of its development
(earlier, the better).
Defect Prevention: As a consequence of defect
detection, help anticipate and prevent defects
from occurring at later stages of development or
from recurring in the future.
User Satisfaction: Ensure customers / users are
satisfied that their requirements (explicit or
implicit) are met.

THE PATH THE

 SAMPLE

“SOFTWA
RE
QUALITY i
Get started by going
s the degree
through Software Testing
of
Basics like Quality
conformanc
Assurance, Quality Control,
e to explicit
Software Development Life
or implicit
Cycle and Software Testing
requirement
Life Cycle.
s and
expectation
s.”

And, level up by “UNIT

 TESTING i
s a level of
software
understanding Software
testing
Testing Levels like Unit
where
Testing, Integration Testing,
individual
Syste
units/
m Testing and Acceptance
components
Testing.
of a
software
are tested.”

Then, go through the “BLACK

various Software Testing BOX
Methods like Black Box TESTING i
Testing, White Box Testing, s a software
 testing
method in
which the
internal
Gray Box Testing, Agile
structure of
Testing and Ad Hoc
the item
Testing.
being tested
is not know
n to the
tester.”

Further, check out the “SMOKE

various Software Testing TESTING
Types like Smoke Testing, is a type of
Functional Testing, software
Usability Testing, Security testing that
 comprises
of a non-
exhaustive
set of tests
Testing, Performance that aim at
Testing, Regression Testing ensuring
and Compliance Testing. that the
most
important
functions
work.”

Also, learn about Software “A TEST

Testing PLAN is a
Artifacts, Defects and Soft document
ware Testing Metrics. describing
 software
testing
scope and
activities.”

Software review:
• A software review is an effective way of
filtering errors in a software product.
• Reviews conducted at each of these phases
i.e., analysis, design, coding, and testing
reveal areas of improvement in the product.
• Reviews also indicate those areas that do not
need any improvement.
• We can use software reviews to achieve
consistency and uniformity across products.
 • Reviews also make the task of product
creation more manageable.
• Software review:
• Some of the most common software review
techniques are:
i. Inspection
ii. Walkthrough
iii. Code review
iv. Formal Technical Reviews (FTR)
v. Pair programming
Formal Technical Review:
Software review:
A software review is an effective way of
filtering errors in a software product. Reviews
conducted at each of these phases i.e., analysis,
design, coding, and testing reveal areas of
improvement in the product. Reviews also
indicate those areas that do not need any
improvement. We can use software reviews to
achieve consistency and uniformity across
products. Reviews also make the task of product
creation more manageable. Some of the most
common software review techniques are:
i. Inspection
ii. Walkthrough
iii. Code review
iv. Formal Technical Reviews (FTR)
v. Pair programming
Formal technical review (FTR):
1. A formal technical review is a software
quality assurance activity performed by
software engineers.
2. In addition, the FTR serves as a training
ground, enabling junior engineers to
observe the different approaches to
software analysis, design, and
implementation.
3. The objectives of FTR are
i. To uncover errors in function, logic, or
implementation for representation of
software.
ii. To verify that software under review
meets its requirements.
iii. To ensure that the software has been
 represented according to predefined
standards.
iv. To achieve software that is developed
in a uniform manner. v. To make projects
more manageable.
Steps required to conduct a successful FTR:
1. The review meeting
 Every review meeting should be conducted
by considering the following constraints:
a. Short duration Advance preparation
b. Involvement of people
c. walkthrough
 Rather than attempting to review the entire
design; is conducted for modules or for a
small group of modules.
  The focus of the FTR is on the work
product (a software component to be
review). The review meeting is attend by
the review leader., all reviewers, and the
producer.
 One of the reviewers becomes a recorder
who records all the important issues raised
during the review. When errors are
discovere, the recorder notes each error.
 At the end of the review, the attendees
decide whether to accept the product or
not, with or without modification.
2. Review reporting and record-keeping
 During the FTR, the reviewer actively
records all the issues that have been raised.
  At the end of the meeting, these all raised
issues are consolidating and a review
issues list is prepare.
Finally, a formal technical review
summary report is produce.
3. Review guidelines
 Guidelines for the conducting of formal
technical review must be establish in
advance.
 These guidelines must be distribute to all
reviewers, agree upon, and then followed.
Software Testing Life Cycle:
Software Testing Life Cycle (STLC) is a
sequence of specific activities conducted during
the testing process to ensure software quality
goals are met. STLC involves both verification
and validation activities. Contrary to popular
belief, Software Testing is not just a
single/isolate activity, i.e. testing. It consists of a
series of activities carried out methodologically
to help certify your software product. STLC
stands for Software Testing Life Cycle.

Requirement Phase Testing

Requirement Phase Testing also known as
Requirement Analysis in which test team studies
the requirements from a testing point of view to
identify testable requirements and the QA team
may interact with various stakeholders to
understand requirements in detail. Requirements
could be either functional or non-functional.
Automation feasibility for the testing project is
also done in this stage.
Activities in Requirement Phase Testing
 Identify types of tests to be performed.
 Gather details about testing priorities and
focus.
 Prepare Requirement Traceability Matrix
(RTM).
 Identify test environment details where
testing is supposed to be carried out.
  Automation feasibility analysis (if required).
Deliverables of Requirement Phase Testing
 RTM
 Automation feasibility report. (if applicable)
Test Planning in STLC
Test Planning in STLC is a phase in which a
Senior QA manager determines the test plan
strategy along with efforts and cost estimates for
the project. Moreover, the resources, test
environment, test limitations and the testing
schedule are also determined. The Test Plan gets
prepared and finalized in the same phase.
Test Planning Activities
 Preparation of test plan/strategy document
for various types of testing
  Test tool selection
 Test effort estimation
 Resource planning and determining roles
and responsibilities.
 Training requirement
Deliverables of Test Planning
 Test plan /strategy document.
 Effort estimation document.
Test Case Development Phase
The Test Case Development Phase involves
the creation, verification and rework of test
cases & test scripts after the test plan is ready.
Initially, the Test data is identified then created
and reviewed and then reworked based on the
preconditions. Then the QA team starts the
development process of test cases for individual
units.
Test Case Development Activities
 Create test cases, automation scripts (if
applicable)
 Review and baseline test cases and scripts
 Create test data (If Test Environment is
available)
Deliverables of Test Case Development
 Test cases/scripts
 Test data
Test Environment Setup
Test Environment Setup decides the software
and hardware conditions under which a work
product is tested. It is one of the critical aspects
of the testing process and can be done in parallel
with the Test Case Development Phase. Test
team may not be involved in this activity if the
development team provides the test
environment. The test team is required to do a
readiness check (smoke testing) of the given
environment.
Test Environment Setup Activities
 Understand the required architecture,
environment set-up and prepare hardware
and software requirement list for the Test
Environment.
 Setup test Environment and test data
 Perform smoke test on the build
Deliverables of Test Environment Setup
  Environment ready with test data set up
 Smoke Test Results.
Test Execution Phase
Test Execution Phase is carried out by the
testers in which testing of the software build is
done based on test plans and test cases prepared.
The process consists of test script execution, test
script maintenance and bug reporting. If bugs
are reported then it is reverted back to
development team for correction and retesting
will be performed.
Test Execution Activities
 Execute tests as per plan
 Document test results, and log defects for
failed cases
  Map defects to test cases in RTM
 Retest the Defect fixes
 Track the defects to closure
Deliverables of Test Execution
 Completed RTM with the execution status
 Test cases updated with results
 Defect reports
Test Cycle Closure
Test Cycle Closure phase is completion of test
execution which involves several activities like
test completion reporting, collection of test
completion matrices and test results. Testing
team members meet, discuss and analyze testing
artifacts to identify strategies that have to be
implemented in future, taking lessons from
current test cycle. The idea is to remove process
bottlenecks for future test cycles.
Test Cycle Closure Activities
 Evaluate cycle completion criteria based on
Time, Test coverage, Cost,Software, Critical
Business Objectives, Quality
 Prepare test metrics based on the above
parameters.
 Document the learning out of the project
 Prepare Test closure report
 Qualitative and quantitative reporting of
quality of the work product to the customer.
 Test result analysis to find out the defect
distribution by type and severity.
Deliverables of Test Cycle Closure
 Test Closure report
 Test metrics