SOFTWARE

QUALITY
ENGINEERING
(SQE)
Chapter 22: Software Testing Strategies
1
 WHAT IS
2
SOFTWARE
TESTING?
 Software Testing: A set of activities performed with the objective of
identifying failures in a software and to evaluate its level of quality, to obtain
user satisfaction. It has:
1. an objective (characteristic we want to learn)
2. a way to determine objective (defined procedure)
3. an activity (execution of the procedure for results)
4. result (presence/absence of expected characteristic)

 The impact of human “error” on a product is called a “defect”, which will

eventually produce a “failure”

 Debugging is fixing software defects to avoid failure.

3
 Software Testing is defined as an activity to:
1. check whether the actual results match the expected results.
2. ensure that the software system is defect free.
 It involves execution of a software component or system component to
evaluate one or more properties of interest.
 Software testing also helps to identify errors, gaps or missing requirements in
contrary to the actual requirements.
 It can be either done manually or using automated tools. Some prefer saying
Software testing as a White Box and Black Box Testing.
 In simple terms, Software Testing means Verification & Validation of
Application Under Test (AUT).

4
 WHY IS
5
SOFTWARE
TESTING
IMPORTANT?
 Testing is important because software bugs could be expensive or even
dangerous. Software bugs can potentially cause monetary and human loss, and
history is full of such examples.
1. Nissan cars have to recall over 1 million cars from the market due to software
failure in the airbag sensory detectors. There has been reported two accident due
to this software failure.
2. In 2015 fighter plane F-35 fell victim to a software bug, making it unable to
detect targets correctly.
3. China Airlines Airbus A300 crashed due to a software bug on April 26, 1994,
killing 264 innocent lives.
4. In may of 1996, a software bug caused the bank accounts of 823 customers of a
major U.S. bank to be credited with 920 million US dollars.

6
CLASS ACTIVITY 01: WHY
TESTING IS IMPORTANT?
 Make a group of 3-4 people.
 Tear a page out and your complete ID’s and full name on it.
 Statement: As per the importance of testing discussed in the previous slide,
give an example of a scenarios where lack of testing can cause serious
damage/loss.
 Solution: In case of ATMs, if testing would not be done properly it may cause
a serious loss of money to the bank as well as customers.

7
8
VERIFICATION
AND VALIDATION
 Software testing is one element of a broader topic that is often referred to as
verification and validation (V&V) – The two V’s of Testing.
 Verification refers to the set of tasks that ensure that software correctly implements
specific functionalities.
 Validation refers to the set of tasks that ensure that the software that has been built is
traceable to customer requirements.
 Boehm states this another way:
 Verification: “Are we building the product right?”
 Validation: “Are we building the right product?”

9
 Verification and validation includes a wide array of Software Quality
Assurance (SQA) activities:
 Technical reviews
 Quality and configuration audits
 Performance monitoring
 Simulation
 Feasibility study
 Documentation review
 Database review
 Algorithm analysis
 Development testing
 Usability testing
 Qualification testing
 Acceptance testing
 Installation testing

10
 It should be noted that there is a strong divergence of opinion about what types of
testing constitute “validation.”

 Some people believe that all testing is verification and that validation is conducted
when requirements are reviewed and approved, and later, by the user when the system
is operational.

 Other people view unit and integration testing as verification and higher-order testing
as validation.

11
 CATEGORIES OF
12
SOFTWARE
TESTING
 Typically Testing is classified into two categories.

Testing Category Types of Testing

Functional Testing Unit Testing
Integration Testing
Smoke Testing
UAT ( User Acceptance Testing)
So on…
Non-Functional Testing Performance Testing
Load Testing
Stress Testing
So on…

 This is not the complete list as there are more than 150 types of testing and still
adding. Also, note that not all testing types are applicable to all projects but
depend on the nature & scope of the project.
13
 SOFTWARE
14
TESTING
STRATEGY—THE
BIG PICTURE
 The software process may be viewed as the spiral illustrated in Figure 22.1

15
 Unit testing begins at the vortex of the spiral and concentrates on each unit
(e.g., component or class) of the software as implemented in source code.
 Testing progresses by moving outward along the spiral to integration testing,
where the focus is on design and the construction of the software architecture.
 Taking another turn outward on the spiral, you encounter validation testing,
where client’s requirements are validated against the software that has been
constructed.
 Finally, you arrive at system testing, where the software and other system
elements are tested as a whole along with hardware requirements.

16
 Testing within the context of software engineering is actually a series of four
steps that are implemented sequentially. The steps are shown in Figure 22.2

17
 Initially, tests focus on each component individually, ensuring that it functions
properly as a unit. Hence, the name unit testing.
 Integration testing addresses the issues associated with the dual problems of
verification and program construction.
 After the software has been integrated (constructed), a set of high-order tests is
conducted. Validation testing provides final assurance that software meets all
functional, behavioral, and performance requirements given by the client.
 Software, once validated, must be combined with other system elements (e.g.,
hardware, people, databases). System testing verifies that all elements mesh
properly and that overall system function/performance is achieved.

18
 CRITERIA FOR
19
COMPLETION OF
TESTING AND ITS
STRATEGIC ISSUES
 A classic question arises every time software testing is discussed: “When are
we done testing—how do we know that we’ve tested enough?”
 Sadly, there is no definitive answer to this question 
 Following could be the responses to this question:
 “You're never done testing; the burden simply shifts from you (the software
engineer) to the end user.”
 “You’re done testing when you run out of time or you run out of money.”

20
 SOFTWARE
21
TESTING
STRATEGY—THE
BIG PICTURE
Unit Testing
 Unit Testing: Unit testing focuses verification effort on the smallest unit of
software design— the software component or module.
 Unit Test Considerations: Unit tests are illustrated schematically in Figure
22.3

22
 Unit-Test Procedures. Unit testing is normally considered as an adjunct to the
coding step. The design of unit tests can occur before coding begins or after
source code has been generated.
 The unit test environment is illustrated in Figure 22.4

Stubs and drivers both are dummy modules

and are only created for test purposes.

23
 Stubs are used in top down testing approach, when one has the major module
ready to test, but the sub modules are still not ready yet. So in a simple
language stubs are "called" programs, which are called in to test the major
module's functionality.
 For example, in a situation where one webpage has three different modules:
Login, Home, User. Suppose login module is ready for test, but the two minor
modules Home and User, which are called by Login module are not ready yet
for testing. At this time, a piece of dummy code is written, which simulates the
called methods of Home and User. These dummy pieces of code are the stubs.

24
 On the other hand, Drivers are the ones, which are the "calling" programs.
Drivers are used in bottom up testing approach. Drivers are dummy code,
which is used when the sub modules are ready but the main module is still not
ready.
 Taking the same example. Suppose this time, the User and Home modules are
ready, but the Login module is not ready to test. Now since Home and User
return values from Login module, so a dummy piece of code is written, which
simulates the Login module. This dummy code is then called Driver.

25
CLASS ACTIVITY 02:
PRACTICE UNIT TESTING
 Make a group of 3-4 people.
 Tear a page out and your complete ID’s and full name on it.
 Statement: Take into consideration any online software application and
identify its at-least five units to perform unit testing.
 Solution: Discussed on the board.

26
27
INTEGRATION
TESTING
 Integration Testing: The objective is to take unit-tested components and build
a program structure that has been dictated by design.
 Incremental Integration: The program is constructed and tested in small
increments, where errors are easier to isolate and correct.
 Top-Down Integration. Top-down integration testing is an incremental
approach for construction of the software architecture.

28
 The top down integration process is performed in a series of five steps:
1. The main control module is used as a test driver and stubs are substituted for
all components directly subordinate to the main control module.
2. Subordinate stubs are replaced one at a time with actual components.
3. Tests are conducted as each component is integrated.
4. On completion of each set of tests, another stub is replaced with the real
component.
5. Regression testing (discussed later in this section) is conducted to ensure
that new errors have not been introduced

29
 Bottom-Up Integration: Begins construction and testing with atomic modules
(i.e., components at the lowest levels in the program structure).

30
 A bottom-up integration strategy may be implemented with the following
steps:
1. Low-level components are combined into clusters (sometimes called builds )
that perform a specific software sub-function.
2. A driver (a control program for testing) is written to coordinate test-case
input and output.
3. The cluster is tested.
4. Drivers are removed and clusters are combined moving upward in the
program structure.

31
 TYPES OF
32
INTEGRATION
TESTING
 Regression Testing. Each time a new module is added as part of integration
testing, the software changes. Regression testing helps to ensure that changes
(due to testing or for other reasons) do not introduce unintended behavior or
additional errors.
 Regression testing may be conducted manually by re-executing a subset of all
test cases, or using automated tools.
 Smoke testing is an integration testing approach that is commonly used when
software product is developed. Smoke tests are used to evaluate the stability of
the initial or newly developed software builds
 It is designed as a pacing mechanism for time-critical projects, allowing the
software team to assess the project on frequent basis. The purpose is to reject a
badly broken application so that the QA team does not waste time installing
and testing the software application.

33
34
VALIDATION
TESTING
 Validation testing begins after the execution of integration testing, when:
 individual components have been exercised,
 the software is completely assembled as a package, and
 interfacing errors have been uncovered and corrected.

 Validation succeeds when software functions in a manner that can be

reasonably expected by the customer.
 Validation testing needs a Test Plan and Test Procedure for its execution.

35
 A test plan outlines the classes of tests to be conducted.
 A test procedure defines specific test cases that are designed to ensure that all
types of requirements are satisfied and documentation is correct, and usability
and other requirements are met.
 An important element of the validation process is a configuration review
(a.k.a audit). The intent of the review is to ensure that all elements of the
software configuration have been properly developed.

36
37
ALPHA AND BETA
TESTING
 Alpha testing is performed to identify all possible issues/bugs before releasing
the product to everyday users or the public in a controlled environment.
 Beta Testing of a product is performed by "real users" of the software
application in a "real environment" 

38
 Alpha Testing Beta Testing

Alpha testing is performed by the Testers who are Beta testing is performed by Clients or End Users
usually internal employees of the organization who are not employees of the organization

Alpha Testing is performed on developer's and Beta testing is performed at a client’s location or
tester’s side   end user of the product  

Beta testing doesn't require any lab environment

Alpha testing requires a lab environment or or testing environment. The software is made
testing environment   available to the public and is said to be real time
environment  

Critical issues or fixes can be addressed by the Most of the issues or feedback is collected from
developers and testers immediately in Alpha the client and real users. Beta testing is
testing   implemented in future versions of the product 

Beta testing also concentrates on the quality of

Alpha testing is done to ensure the quality of the the product, but gathers users input on the
product before moving to Beta testing product and ensures that the product is ready for
real time users.   39
 A variation of beta testing is called customer acceptance testing or simply
acceptance testing which is sometimes performed when custom software is
delivered to a customer under contract.
 The customer performs a series of specific tests in an attempt to uncover errors
before accepting the software from the development team.

40
41
SYSTEM TESTING
 System Testing is the testing of a complete and fully integrated software
product.
 Usually, software is only one element of a larger computer-based system.
Ultimately, software is interfaced with other software/hardware systems.
 System Testing is actually a series of different tests whose sole purpose is to
exercise the full computer-based system.

42
43
TYPES OF SYSTEM
TESTING
 Recovery Testing is a system test that forces the software to fail in a
variety of ways and verifies that recovery is properly performed.
 Example: In case of an online application, unplug the connecting
cable while application is receiving data from the network. After
some time, plug the cable back in and analyze the application’s
ability to continue receiving data from the point at which the
network connection was broken.

44
 Security Testing attempts to verify that protection mechanisms built
into a system will, in fact, protect it from improper penetration.
 Example: In case of a financial application , the Browser back
button should not work.

45
 Stress Testing executes a system in a manner that demands
resources in abnormal quantity, frequency, or volume.
 Example: Let’s say a text editor application is able to store data less
than 5GB. Under stress testing it will be stressed by copying and
pasting 5GB data from the website. Text editor should go under
stress and gives 'Not Responded' error message.

46
 Performance Testing is designed to test the run-time performance
of software within the context of an integrated system.
 Example: In context of a online application, let’s say that it can
entertain 1000 clicks per second. The application will be tested by
doing 1000 clicks per second to check its performance.

47
 Deployment Testing, sometimes called configuration testing,
exercises the software in each environment in which it is to operate.
 Example: Let’s say a desktop application named “Writing Pad” is
deployed on the operating system of the client to ensure its proper
functioning.

48
BOOK READING
 Contents covered so far.

49
QUIZ 1 ANNOUNCEMENT
 Quiz 1 will be conducted in the next lecture.
 Contents: Chapter 22 (Topics covered so far)
 P.S. Book reading is a must.

50
CLASS ACTIVITY 3: SYSTEM TESTING ON AN APPLICATION

 Make a group of 3-4 people.

 Tear a page out and your complete ID’s and full name on it.
 Now, take into consideration the GMAIL application and give one example
test case for Stress and Performance Testing
 Help:
 Stress testing executes a system in a manner that demands resources in
abnormal quantity, frequency, or volume
 Performance testing is designed to test the run-time performance of software
within the context of an integrated system.
 Solution: Discussed on board in the class
51