Chapter One

Software Testing and Quality Assurance

| Fozia Abako
1
 Introduction

• Quality assurance (QA) is any systematic process of determining whether a

product or service meets specified requirements.
• QA establishes and maintains set requirements for developing or
manufacturing reliable products.
• A quality assurance system is meant to increase customer confidence and a
company's credibility (reliability/integrity), while also improving work
processes and efficiency, and it enables a company to better compete with
others.
• guidance for recognizing, defining, analysing, and improving the production
process.

2
 Cont.…

The role of Quality Assurance may cover aspects such as the design and
monitoring of documentation systems,
• approval and monitoring of written procedures to produce the product,
• approval of written records of the processing operations,
• approval and monitoring of cleaning systems,
• regulatory control,
• batch or lot review,
• release of product. ..etc.

3
 Quality Control
• Simplistically, quality is an attribute of software that implies the software meets its
specification
• This definition is too simple for ensuring quality in software systems
• Software specifications are often incomplete or ambiguous
• Some quality attributes are difficult to specify
• Tension exists between some quality attributes, e.g. efficiency vs. reliability
• Software requirements are the foundation from which quality is measured.
• Lack of conformance to requirements is lack of quality.
• Specified standards define a set of development criteria that guide the manner in which
software is engineered.
• If the criteria are not met, lack of quality will almost surely result.
• There is a set of implicit requirements that often goes unmentioned.
• If software conforms to its explicit requirements but fails to meet its implicit
requirements, software quality is suspect.
4
 Cont.…
• Explicit requirements: is special requirements given by customer.
• Implicit requirements: that are added(analyzed) by the business analyst which will increase
the value of application without disturbing the original customer requirement.
• Without quality, your company will not survive.
• Use a written quality program to ensure you can offer your customers consistent products.
• Provide consistent products to keep production costs down and increase revenue.

What makes quality software?

There is no clear-cut answer
It depends on:
Stakeholders
Type of system
Type of users
Different ideas about a quality car
5
Introduction Software Quality Attributes
Safety Modularity
Security Complexity
Reliability Portability (compatible with different OS
Resilience(Flexibility) Usability(Both Expert/beginner)
Robustness Reusability(easily can be used for new one
Understandability Efficiency
Testability Learnability
Correctness(According SRS) Accuracy (of result)
Maintainability (New Function
easily added)

6
7
 Effective quality process
An effective quality process must focus on:
• Paying much attention to customer’s requirements
• Making efforts to continuously improve quality
• Integrating measurement processes with product design and development
• Pushing the quality concept down to the lowest level of the organization
• Developing a system-level perspective with an emphasis on methodology and process
• Eliminating waste through continuous improvement.
From the different quality scenarios Online banking system(Security , correctness, reliability ),
Air traffic control system (Robustness, real time responses) Educational Game for Children (
User-friendliness) 8
 QA versus QC
• Some people may confuse the term quality assurance with quality
control (QC).
• Although the two concepts share similarities, there are important distinctions
between them.
• In effect, QA provides the overall guidelines used anywhere, and QC is a
production-focused process – for things such as inspections.
• QA is any systematic process for making sure a product meets specified
requirements, whereas QC addresses other issues, such as individual
inspections or defects.
• In terms of software development, QA practices seek to prevent
malfunctioning code or products, while QC implements testing and
troubleshooting and fixes code.
9
 Quality assurance in software
• Software quality assurance (SQA) systematically finds patterns and the
actions needed to improve development cycles.
• Finding and fixing coding errors can carry unintended consequences; it is
possible to fix one thing, yet break other features and functionality at the
same time.
• SQA has become important for developers as a means of avoiding errors
before they occur, saving development time and expenses.
• Even with SQA processes in place, an update to software can break other
features and cause defects -- commonly known as bugs.
• There have been numerous SQA strategies.

10
 Cont....

• For example, Capability Maturity Model Integration (CMMI) is a

performance improvement-focused SQA model.
• CMMI works by ranking maturity levels of areas within an organization, and
it identifies optimizations that can be used for improvement.
• Rank levels range from being disorganized to being fully optimal.

11
 Cont....
• Software development methodologies have developed over time that rely on SQA, such as
Waterfall, Agile and Scrum.
• Each development process seeks to optimize work efficiency.
• Waterfall is the traditional linear approach to software development.
• It's a step-by-step process that typically involves gathering requirements, formalizing a
design, implementing code, code testing and remediation, and release.
• It is often seen as too slow, which is why alternative development methods were
constructed.
• Agile is a team-oriented software development methodology where each step in the work
process is approached as a sprint (race/run).
• Agile software development is highly adaptive, but it is less predictive because the scope of
the project can easily change.
• Scrum is a combination of both processes where developers are split into teams to handle
specific tasks, and each task is separated into multiple sprints.
12
 Software Quality Assurance
• To ensure quality in a software product, an organization must having
following approach to quality management:
• 1. Organization-wide policies, procedures and standards must be established.
• 2. Project-specific policies, procedures and standards must be tailored from
the organization-wide templates.
• 3. Quality must be controlled; that is, the organization must ensure that the
appropriate procedures are followed for each project
• Standards exist to help an organization draft an appropriate software quality
assurance plan.
• External entities can be contracted to verify that an organization is standard-
compliant.

13
 What is Testing?
• Testing is a process of executing a software application with the intent of finding errors and
to verify that it satisfies specified requirements .
• Testing is the process of exercising or evaluating a system or a system component by manual
or automated means to verify that it satisfies specified requirements or to identify differences
between expected and actual results.
• Testing is a measurement of software quality in terms of defects found, for both functional
and non-functional software requirements and characteristics.
• Testing is the process of executing the program with the intent of finding faults.
❑Who should do this testing and when should it start are very important questions.
❑As we know software testing is the fourth phase of the software development life cycle
(SDLC). About 70% of development time is spent on testing. 14
 What is Software Testing?

• Testers used to be relied upon to test everything, but as the whole software delivery life cycle
has evolved, code-specific tests (unit tests, which test individual units of computer code, and
integration tests, which test multiple modules of code) have been taken on by the software
developer and the software tester has been allowed to focus on adding value into the process
with:
Functional Testing - testing the function of a system, or part of an overall system, to ensure
that the specification is met, usually based on acceptance criteria.
Integration Testing - testing a number of software modules in an integrated fashion as a group.
User Experience Testing - testing the main graphical user interface between the software
system under development and the end user. 15
 Cont…
• Exploratory Testing - manually exploring the developed system while simultaneously
learning about its functionality, designing tests as you go and observing any testable or
unexpected behavior.
• Regression Testing - testing designed to uncover any new bugs introduced to existing
functionality as a result of changes to the code currently being deployed or environmental
changes.
• Acceptance Testing - testing usually, although not always, performed on the customer side to
ensure that the code/system being delivered meets acceptable standards and is as defined
prior to commencing the project.
• Smoke & Sanity(rational/stability) Testing - testing performed at a high level to ensure that
the main functionality of a system is as required prior to moving onto more granular tests.

16
 Cont…
• Performance Testing - the process of determining the performance of a system (speed, load,
stress points, bottlenecks, etc.) based on a number of metrics and quite often measured
against a baseline measurement prior to development.
• Security Testing - ensuring that the system security and data integrity are of the highest
possible standard and any weaknesses or vulnerabilities are spotted prior to release.
• Internationalization and Localization Testing - testing that all local languages, symbols,
currencies and other regional variations are correctly applied to each local site.

17
IO Mapping

Subset of inputs
causing erroneous
Input Set
outputs

Software

Output Set

Erroneous
outputs

18
 Software Faults and Failures?
• A failure corresponds to erroneous/unexpected runtime behavior observed by a user.
• A fault is a static software characteristic that can cause a failure to occur.
• The presence of a fault doesn’t necessarily imply the occurrence of a failure.
Input Set

Erroneous
User A Inputs
Inputs

User B User C
Inputs Inputs

19
 What is Software Testing?
❑ The concept of software testing has evolved from simple program “check-out” to a broad set of activities that
cover the entire software life-cycle.

There are five distinct levels of testing that are given below:

1. Debug: It is defined as the successful correction of a failure.

2. Demonstrate: The process of showing that major features work with typical input.

3. Verify: The process of finding as many faults in the application under test (AUT) as possible.

4. Validate: The process of finding as many faults in requirements, design, and AUT.

5. Prevent: To avoid errors in development of requirements, design, and implementation by self-checking

techniques, including “test before design.”

20
 Cont.…

❑ Various definitions of testing

➢ “Testing is the process of exercising or evaluating a system or system components by manual or automated
means to verify that it satisfies specified requirements.” [IEEE]

➢ “Software testing is the process of executing a program or system with the intent of finding errors.”
[Myers]

➢ “It involves any activity aimed at evaluating an attribute or capability of a program or system and
determining that it meets its required results.” [Hetzel]

Note: Testing is basically a task of locating errors.

21
 What is NOT Software Testing?
❑ The process of demonstrating that errors are not present.

❑ The process of showing that a program performs its intended functions correctly.

❑ The process of establishing confidence that a program does what it is supposed to do.

Note: Testing is basically a task of locating errors.

22
 What is Positive testing?
❑ Operate the application as it should be operated(normal/common Workflows).

❑ Does it behave normally? Use a proper variety of legal test data, including data values at the boundaries to
test if it fails. Check actual test results with the expected.

❑ Are results correct? Does the application function correctly?

❑ Testing as per the requirements (functional testing)

23
 What is Negative testing?
❑ Test for abnormal operations. Does the system fail/crash?

❑ Testing NOT as per the requirement (quality/non functional)

❑ Test with illegal or abnormal data or invalid values. Intentionally attempt to make things go wrong and to
discover/detect.

❑ Does the program do what it should not do?

❑ Does it fail to do what it should?

24
 What is Positive View of Negative Testing?
❑ Positive view of negative testing: The job of testing is to discover errors before the user does.

❑ A good tester is one who is successful in making the system fail.

❑ Mentality of the tester has to be destructive--opposite to that of the creator /author /developer, which should
be constructive

❑ Tester and Developer should be collaborated to improve quality

25
 What is Software Testing?
❑ One very popular equation of software testing is:

Software Testing = Software Verification + Software Validation

❑ As per IEEE definition(s):

Software verification:
“It is the process of evaluating a system or component to determine whether the products of a given development
phase satisfy the conditions imposed at the start of that phase.”

(OR)

“It is the process of evaluating, reviewing, inspecting and doing desk checks of work products such as
requirement specifications, design specifications and code.”

(OR)

“It is a human testing activity as it involves looking at the documents on paper.”

26
 What is Software Testing?
❑ As per IEEE definition(s):

❑ Software validation:

❑ “It is defined as the process of evaluating a system or component during or at the end of development
process to determine whether it satisfies the specified requirements.

❑ Software validation involves executing the actual software.

❑ Both verification and validation (V&V) are complementary to each other.

❑ As mentioned earlier, good testing expects more than just running a program.

27
 Why Should We Test? What is the Purpose?
Testing is necessary- Reasons from various prospective
1. The Technical Case:
a. Competent developers are not always effective.
b. The implications of requirements are not always predictible.
c. The behaviour of a system is not necessarily predictable from its components.
d. Languages, databases, user interfaces, and operating systems have bugs that can cause application failures.
e. Reusable classes and objects must be trustworthy.
2. The Business Case:
a. If you don’t find bugs your customers or users will.
b. Post-release debugging is the most expensive form of development.
c. Buggy software hurts operations, sales, and reputation.
d. Buggy software can be hazardous to life and property.
3. The Professional Case:
a. Test case design is a challenging and rewarding task.
b. Good testing allows confidence in your work.
c. Systematic testing allows you to be most effective.
d. Your credibility is increased and you have pride in your efforts.
28
 Why Should We Test? What is the Purpose?
4.The Economics Case: Practically speaking, defects get introduced in every phase of SDLC. Pressman has
described a defect amplification model wherein he says that errors get amplified by a certain factor if that error is
not removed in that phase only.
❑ This may increase the cost of defect removal. This principle of detecting errors as close to their point of
introduction as possible is known as phase containment of errors.

29
Efforts During SDLC
 Why Should We Test? What is the Purpose?
5. To Improve Quality: As computers and software are used in critical applications, the outcome of a bug can be
severe.
❑ Bugs can cause huge losses.
❑ Bugs in critical systems have caused airplane crashes, allowed space shuttle systems to go awry, and halted
trading on the stock market. Bugs can kill.
❑ Bugs can cause disasters.
❑ In a computerized embedded world, the quality and reliability of software is a matter of life and death. This
can be achieved only if thorough testing is done.

6. For Verification and Validation (V&V): Testing can serve as metrics.

❑ It is heavily used as a tool in the V&V process.
❑ We can compare the quality among different products under the same specification based on results from the
same test.
❑ Good testing can provide measures for all relevant quality factors.

30