Software Testing For Dummies (ADITI Edition)

Aditi Technologies Basic Concepts of Software Testing
Software Testing
Basic Concepts and Industry awareness
Page 1 of 62
Table of Contents
1. Introduction.................................................................................................4
2. Types of Testing............................................................................................4
2.1. White Box Testing...................................................................................4
2.2. Black Box Testing....................................................................................4
2.3. Unit Testing............................................................................................5
2.3.1. Benefits ...........................................................................................5
2.3.2. Encourages change ...........................................................................5
2.3.3. Simplifies Integration ........................................................................5
2.3.4. Documents the code .........................................................................5
2.3.5. Separation of Interface from Implementation .......................................6
2.3.6. Limitations .......................................................................................6
2.4. Integration testing .................................................................................6
2.4.1. Purpose ...........................................................................................6
2.5. Performance Testing................................................................................7
2.5.1. Technology ......................................................................................7
2.5.2. Performance specifications .................................................................7
2.5.3. Tasks to undertake ...........................................................................8
2.6. Stress Testing ........................................................................................8
2.7. Security Testing......................................................................................9
2.7.1. Security Testing Techniques ...............................................................9
2.8. Usability Testing......................................................................................9
2.9. Stability Testing....................................................................................10
2.10. Acceptance Testing..............................................................................10
2.11. Installation Testing..............................................................................11
2.12. Alfa Testing.........................................................................................11
2.13. Beta Testing........................................................................................11
2.14. Product Testing...................................................................................11
2.15. System Testing....................................................................................12
2.16. Regression Testing...............................................................................12
2.17. Compatibility Testing............................................................................13
2.18. Test Cases, Suits, Scripts and Scenario...................................................13
2.19. Defect Tracking...................................................................................14
2.20. Formal Verification...............................................................................14
2.20.1. Validation and Verification ..............................................................14
2.21. Fuzz Testing........................................................................................15
2.21.1. Uses ............................................................................................15
2.21.2. Fuzz testing methods .....................................................................15
2.21.3. Event-driven fuzz ..........................................................................16
2.21.4. Character-driven fuzz ....................................................................16
2.21.5. Database fuzz ...............................................................................16
3. Manual Testing...........................................................................................17
3.1. Facts....................................................................................................17
3.2. Software Crisis .....................................................................................17
3.3. Software Myths ....................................................................................17
3.3.1. Management Myths .........................................................................17
3.3.2. Developers Myths ...........................................................................17
3.3.3. Customer’s Myth .............................................................................18
3.3.4. What do we do? ..............................................................................18
3.4. Software Quality Assurance: ..................................................................18
3.4.1. Verification: ...................................................................................18
3.4.2. Validation: .....................................................................................18
3.5. Software Life Cycle Models: ...................................................................18
Page 2 of 62
3.6. What makes a good Software QA engineer? .............................................19

3.7. Testing: ...............................................................................................19
3.7.1. Why Testing? ..................................................................................19
3.8. Test Life Cycle ......................................................................................19
3.9. Testing Techniques ...............................................................................19
3.10. Test Plan: ..........................................................................................20
3.10.1. Test Specification: .........................................................................20
4. Testing Procedure.......................................................................................20
4.1. Bug Tracking ........................................................................................21
5. Testing Tools and Software...........................................................................23
5.1. Load and Performance Test Tools ............................................................23
5.2. Java test Tools......................................................................................23
5.3. Link Checking Tools...............................................................................27
5.4. Perl Testing Tools...................................................................................28
5.5. Web Functional and Regression Testing Tools............................................29
5.6. Web Site Security Test Tools...................................................................33
5.7. Web Site Management Tools...................................................................37
5.8. Other Web Testing Tools.........................................................................45
6. Testing FAQ ...............................................................................................51
Page 3 of 62
1. Introduction
Testing is a process used to help identify the correctness, completeness and quality
of developed computer software.
With that in mind, testing can never completely establish the correctness of
computer software. In other words Testing is nothing but CRITICISM or
COMPARISION. Here comparison in the sense comparing the actual value with
expected one.
There are many approaches to software testing, but effective testing of complex
products is essentially a process of investigation, not merely a matter of creating and
following rote procedure. One definition of testing is "the process of questioning a
product in order to evaluate it", where the "questions" are things the tester tries to
do with the product, and the product answers with its behavior in reaction to the
probing of the tester. Although most of the intellectual processes of testing are nearly
identical to that of review or inspection, the word testing is connoted to mean the
dynamic analysis of the product—putting the product through its paces.
The quality of the application can and normally does vary widely from system to
system but some of the common quality attributes include reliability, stability,
portability, maintainability and usability.
Refer to the ISO standard ISO 9126 for a more complete list of attributes and criteria.
2. Types of Testing
2.1.White Box Testing
White box testing is also known as glass box, structural, clear box and open box
testing. This is a software testing technique whereby explicit knowledge of the
internal workings of the item being tested are used to select the test data.
Unlike black box testing, white box testing uses specific knowledge of
programming code to examine outputs. The test is accurate only if the tester
knows what the program is supposed to do. He or she can then see if the
program diverges from its intended goal. White box testing does not account for
errors caused by omission, and all visible code must also be readable.
2.2.Black Box Testing

Testing of a function without knowing internal structure of the program.
Black-box and white-box are test design methods. Black-box test design treats
the system as a "black-box", so it doesn't explicitly use knowledge of the internal
structure. Black-box test design is usually described as focusing on testing
functional requirements. Synonyms for black-box include: behavioral, functional,
opaque-box, and closed-box. White-box test design allows one to peek inside the
"box", and it focuses specifically on using internal knowledge of the software to
guide the selection of test data. Synonyms for white-box include: structural,
glass-box and clear-box.
Page 4 of 62
While black-box and white-box are terms that are still in popular use, many
people prefer the terms "behavioral" and "structural". Behavioral test design is
slightly different from black-box test design because the use of internal
knowledge isn't strictly forbidden, but it's still discouraged. In practice, it hasn't
proven useful to use a single test design method. One has to use a mixture of
different methods so that they aren't hindered by the limitations of a particular
one. Some call this "gray-box" or "translucent-box" test design, but others wish
we'd stop talking about boxes altogether.
It is important to understand that these methods are used during the test design
phase, and their influence is hard to see in the tests once they're implemented.
Note that any level of testing (unit testing, system testing, etc.) can use any test
design methods. Unit testing is usually associated with structural test design, but
this is because testers usually don't have well-defined requirements at the unit
level to validate.
2.3.Unit Testing
In computer programming, a unit test is a method of testing the correctness of
a particular module of source code.
The idea is to write test cases for every non-trivial function or method in the
module so that each test case is separate from the others if possible. This type of
testing is mostly done by the developers.
2.3.1.Benefits
The goal of unit testing is to isolate each part of the program and show that
the individual parts are correct. It provides a written contract that the piece
must satisfy. This isolated testing provides four main benefits:
2.3.2.Encourages change
Unit testing allows the programmer to re-factor code at a later date, and
make sure the module still works correctly (regression testing). This provides
the benefit of encouraging programmers to make changes to the code since it
is easy for the programmer to check if the piece is still working properly.
2.3.3.Simplifies Integration
Unit testing helps eliminate uncertainty in the pieces themselves and can be
used in a bottom-up testing style approach. By testing the parts of a program
first and then testing the sum of its parts will make integration testing easier.
2.3.4.Documents the code

Unit testing provides a sort of "living document" for the class being tested.
Clients looking to learn how to use the class can look at the unit tests to
determine how to use the class to fit their needs.
Page 5 of 62
2.3.5.Separation of Interface from Implementation

Because some classes may have references to other classes, testing a class
can frequently spill over into testing another class. A common example of this
is classes that depend on a database; in order to test the class, the tester
finds herself writing code that interacts with the database. This is a mistake,
because a unit test should never go outside of its own class boundary. As a
result, the software developer abstracts an interface around the database
connection, and then implements that interface with their own Mock Object.
This results in loosely coupled code, thus minimizing dependencies in the
system.
2.3.6.Limitations
It is important to realize that unit-testing will not catch every error in the
program. By definition, it only tests the functionality of the units themselves.
Therefore, it will not catch integration errors, performance problems and any
other system-wide issues. In addition, it may not be trivial to anticipate all
special cases of input the program unit under study may receive in reality.
Unit testing is only effective if it is used in conjunction with other software
testing activities.
2.4.Integration testing
Integration Testing is the phase of software testing in which individual software

modules are combined and tested as a group.
It follows unit testing and precedes system testing. takes as its input modules
that have been checked out by unit testing, groups them in larger aggregates,
applies tests defined in an Integration test plan to those aggregates, and delivers
as its output the integrated system ready for system testing.
2.4.1.Purpose
The purpose of Integration testing is to verify functional, performance and
reliability requirements placed on major design items. These "design items",
i.e. assemblages (or groups of units), are exercised through their interfaces
using Black box testing, success and error cases being simulated via
appropriate parameter and data inputs. Simulated usage of shared data areas
and inter-process communication is tested; individual subsystems are
exercised through their input interface. All test cases are constructed to test
that all components within assemblages interact correctly, for example, across
procedure calls or process activations.
The overall idea is the "building block" approach in which verified

assemblages are added to a verified base which is then used to support the
Integration testing of further assemblages.
Page 6 of 62
2.5.Performance Testing
In software engineering, performance testing is testing that is performed to
determine how fast some aspect of a system performs under a particular
workload.
Performance testing can serve different purposes. It can demonstrate that the
system meets performance criteria. It can compare two systems to find which
performs better. Or it can measure what parts of the system or workload cause
the system to perform badly. In the diagnostic case, software engineers use tools
such as profilers to measure what parts of a device or software contribute most
to the poor performance or to establish throughput levels (and thresholds) for
maintained acceptable response time.
In performance testing, it is often crucial (and often difficult to arrange) for the
test conditions to be similar to the expected actual use.
2.5.1.Technology
Performance testing technology employs one or more PCs to act as injectors –
each emulating the presence or numbers of users and each running an
automated sequence of interactions (recorded as a script, or as a series of
scripts to emulate different types of user interaction) with the host whose
performance is being tested. Usually, a separate PC acts as a test conductor,
coordinating and gathering metrics from each of the injectors and collating
performance data for reporting purposes. The usual sequence is to ramp up
the load – starting with a small number of virtual users and increasing the
number over a period to some maximum.
The test result shows how the performance varies with the load, given as
number of users vs. response time. Various tools, including Compuware
Corporation's QACenter Performance Edition, are available to perform such
tests. Tools in this category usually execute a suite of tests which will emulate
real users against the system. Sometimes the results can reveal oddities,
e.g., that while the average response time might be acceptable, there are
outliers of a few key transactions that take considerably longer to complete –
something that might be caused by inefficient database queries, etc.
Performance testing can be combined with stress testing, in order to see what
happens when an acceptable load is exceeded –does the system crash? How
long does it take to recover if a large load is reduced? Does it fail in a way
that causes collateral damage?
2.5.2.Performance specifications
Performance testing is frequently not performed against a specification, i.e.
no one will have expressed what the maximum acceptable response time for a
given population of users is. However, performance testing is frequently used
as part of the process of performance profile tuning. The idea is to identify the
“weakest link” – there is inevitably a part of the system which, if it is made to
respond faster, will result in the overall system running faster. It is sometimes
a difficult task to identify which part of the system represents this critical
path, and some test tools come provided with (or can have add-ons that
Page 7 of 62
provide) instrumentation that runs on the server and reports transaction

times, database access times, network overhead, etc. which can be analyzed
together with the raw performance statistics. Without such instrumentation
one might have to have someone crouched over Windows Task Manager at
the server to see how much CPU load the performance tests are generating.
There is an apocryphal story of a company that spent a large amount
optimizing their software without having performed a proper analysis of the
problem. They ended up rewriting the system’s ‘idle loop’, where they had
found the system spent most of its time, but even having the most efficient
idle loop in the world obviously didn’t improve overall performance one iota!
Performance testing almost invariably identifies that it is parts of the software

(rather than hardware) that contribute most to delays in processing users’
requests.
Performance testing can be performed across the web, and even done in
different parts of the country, since it is known that the response times of the
internet itself vary regionally. It can also be done in-house, although routers
would then need to be configured to introduce the lag what would typically
occur on public networks.
It is always helpful to have a statement of the likely peak numbers of users

that might be expected to use the system at peak times. If there can also be
a statement of what constitutes the maximum allowable 95 percentile
response time, then an injector configuration could be used to test whether
the proposed system met that specification.
2.5.3.Tasks to undertake
Tasks to perform such a test would include:
• Analysis of the types of interaction that should be emulated and the

production of scripts to do those emulations
• Decision whether to use internal or external resources to perform the
tests.
• Set up of a configuration of injectors/controller
• Set up of the test configuration (ideally identical hardware to the
production platform), router configuration, quiet network (we don’t want
results upset by other users), deployment of server instrumentation.
• Running the tests – probably repeatedly in order to see whether any
unaccounted for factor might affect the results.
• Analyzing the results, either pass/fail, or investigation of critical path and
recommendation of corrective action.
2.6.Stress Testing
Stress Testing is a form of testing that is used to determine the stability of a
given system or entity.
It involves testing beyond normal operational capacity, often to a breaking point,

in order to observe the results. For example, a web server may be stress tested
using scripts, bots, and various denial of service tools to observe the performance
Page 8 of 62
of a web site during peak loads. Stress testing a subset of load testing. Also see
testing, software testing, performance testing.
2.7.Security Testing
Application vulnerabilities leave your system open to attacks, Downtime, Data
theft, Data corruption and application Defacement. Security within an
application or web service is crucial to avoid such vulnerabilities and new
threats.
While automated tools can help to eliminate many generic security issues, the
detection of application vulnerabilities requires independent evaluation of your
specific application's features and functions by experts. An external security
vulnerability review by Third Eye Testing will give you the best possible
confidence that your application is as secure as possible.
2.7.1.Security Testing Techniques

• Vulnerability Scanning
• Network Scanning
• Password Cracking
• Log Views
• Virus Detect
• Penetration Testing
• File Integrity Checkers
• War Dialing
2.8.Usability Testing
Usability testing is a means for measuring how well people can use some
human-made object (such as a web page, a computer interface, a document, or
a device) for its intended purpose, i.e. usability testing measures the usability
of the object.
Usability testing focuses on a particular object or a small set of objects, whereas

general human-computer interaction studies attempt to formulate universal
principles.
If usability testing uncovers difficulties, such as people having difficulty

understanding instructions, manipulating parts, or interpreting feedback, then
developers should improve the design and test it again. During usability testing,
the aim is to observe people using the product in as realistic a situation as
possible, to discover errors and areas of improvement. Designers commonly
focus excessively on creating designs that look "cool", compromising usability and
functionality. This is often caused by pressure from the people in charge, forcing
designers to develop systems based on management expectations instead of
people's needs. A designers' primary function should be more than appearance,
including making things work with people.
"Caution: simply gathering opinions is not usability testing -- you must arrange
an experiment that measures a subject's ability to use your document."
Page 9 of 62
Rather than showing users a rough draft and asking, "Do you understand this?",
usability testing involves watching people trying to use something for its intended
purpose. For example, when testing instructions for assembling a toy, the test
subjects should be given the instructions and a box of parts. Instruction phrasing,
illustration quality, and the toy's design all affect the assembly process.
Setting up a usability test involves carefully creating a scenario, or realistic

situation, wherein the person performs a list of tasks using the product being
tested while observers watch and take notes. Several other test instruments such
as scripted instructions, paper prototypes, and pre- and post-test questionnaires
are also used to gather feedback on the product being tested. For example, to
test the attachment function of an e-mail program, a scenario would describe a
situation where a person needs to send an e-mail attachment, and ask him or her
to undertake this task. The aim is to observe how people function in a realistic
manner, so that developers can see problem areas, and what people like. The
technique popularly used to gather data during a usability test is called a think
aloud protocol.
2.9.Stability Testing
In software testing, stability testing is an attempt to determine if an application
will crash.
In the pharmaceutical field, it refers to a period of time during which a multi-dose

product retains its quality after the container is opened.
2.10.Acceptance Testing
User acceptance testing (UAT) is one of the final stages of a software project and
will often occur before the customer accepts a new system.
Users of the system will perform these tests which, ideally, developers have
derived from the User Requirements Specification, to which the system should
conform.
Test designers will draw up a formal test plan and devise a range of severity
levels. The focus in this type of testing is less on simple problems (spelling
mistakes, cosmetic problems) and show stoppers (major problems like the
software crashing, software will not run etc.). Developers should have worked out
these issues during unit testing and integration testing. Rather, the focus is on a
final verification of the required business function and flow of the system. The
test scripts will emulate real-world usage of the system. The idea is that if the
software works as intended and without issues during a simulation of normal use,
it will work just the same in production.
Results of these tests will allow both the customers and the developers to be
confident that the system will work as intended.
Page 10 of 62
2.11.Installation Testing
Installation testing (in software engineering) can simply be defined as any
testing that occurs outside of the development environment.
Such testing will frequently occur on the computer system the software product
will eventually be installed on.
Whilst the ideal installation might simply appear to be to run a setup program,
the generation of that setup program itself and its efficacy in a variety of machine
and operating system environments can require extensive testing before it can be
used with confidence.
In distributed systems, particularly where software is to be released into an

already live target environment (such as an operational web site) installation (or
deployment as it is sometimes called) can involve database schema changes as
well as the installation of new software. Deployment plans in such circumstances
may include back-out procedures whose use is intended to roll the target
environment back in the event that the deployment is unsuccessful. Ideally, the
deployment plan itself should be tested in an environment that is a replica of the
live environment. A factor that can increase the organizational requirements of
such an exercise is the need to synchronize the data in the test deployment
environment with that in the live environment with minimum disruption to live
operation.
2.12.Alfa Testing
In software development, testing is usually required before release to the general
public.
In-house developers often test the software in what is known as 'ALPHA' testing
which is often performed under a debugger or with hardware-assisted
debugging to catch bugs quickly.
It can then be handed over to testing staff for additional inspection in an

environment similar to how it was intended to be used. This technique is known
as black box testing. This is often known as the second stage of alpha testing.
2.13.Beta Testing
Many a time, the software is released to a limited audience who would finally
form the end users, to use it / test it and come back with feedback or bugs.
This process helps in determining whether the final software meets its intended
purpose and whether the end users would accept the same.
The product handed out as a Beta Release is not bug free, however no serious or
critical bugs would exist. A beta release is very close to the final release.
2.14.Product Testing
Software Product development companies face unique challenges in testing. Only
suitably organized and executed test process can contribute to the success of a
software product.
Page 11 of 62
Product testing experts design the test process to take advantage of the
economies of scope and scale that are present in a software product.
These activities are sequenced and scheduled so that a test activity occurs
immediately following the construction activity whose output the test is
intended to validate.
2.15.System Testing
According to the IEEE Standard Computer Dictionary, System testing is testing
conducted on a complete, integrated system to evaluate the system's
compliance with its specified requirements.
System testing falls within the scope of Black box testing, and as such, should
require no knowledge of the inner design of the code or logic (IEEE. IEEE
Standard Computer Dictionary: A Compilation of IEEE Standard Computer
Glossaries. New York, NY. 1990.).
Alpha testing and Beta testing are sub-categories of System testing.
As a rule, System testing takes, as its input, all of the "integrated" software
components that have successfully passed Integration testing and also the
software system itself integrated with any applicable hardware system(s). The
purpose of Integration testing is to detect any inconsistencies between the
software units that are integrated together called assemblages or between any of
the assemblages and hardware. System testing is more of a limiting type of
testing, where it seeks to detect both defects within the "inter-assemblages" and
also the system as a whole.
2.16.Regression Testing
Regression Testing is typically carried out at the end of the development cycle.
During this testing, all bug previously identified and fixed is tested along with
it's impacted areas to confirm the fix and it's impact if any.
According to the IEEE Standard Computer Dictionary, Regression testing is testing

conducted on a complete, integrated system to evaluate the system's compliance
with its specified requirements.
Regression testing falls within the scope of Black box testing, and as such, should
require no knowledge of the inner design of the code or logic (IEEE. IEEE
Standard Computer Dictionary: A Compilation of IEEE Standard Computer
Glossaries. New York, NY. 1990.).
Alpha testing and Beta testing are sub-categories of Regression testing.
As a rule, Regression testing takes, as its input, all of the "integrated" software
components that have successfully passed Integration testing and also the
software Regression itself integrated with any applicable hardware Regression(s).
The purpose of Integration testing is to detect any inconsistencies between the
software units that are integrated together called assemblages or between any of
the assemblages and hardware. Regression testing is more of a limiting type of
Page 12 of 62
testing, where it seeks to detect both defects within the "inter-assemblages" and
also the system as a whole.
2.17.Compatibility Testing
One of the challenges of software development is ensuring that the application
works properly on the different platforms and operating systems on the market
and also with the applications and devices in its environment.
Compatibility testing service aims at locating application problems by running

them in real environments, thus ensuring you that the application is compatible
with various hardware, operating system and browser versions.
2.18.Test Cases, Suits, Scripts and Scenario

Black box testers usually write test cases for the majority of their testing
activities.
A test case is usually a single step, and its expected result, along with various
additional pieces of information.
It can occasionally be a series of steps but with one expected result or expected
outcome. The optional fields are a test case ID, test step or order of execution
number, related requirement(s), depth, test category, author, and check boxes for
whether the test is automatable and has been automated. Larger test cases may
also contain prerequisite states or steps, and descriptions. A test case should also
contain a place for the actual result. These steps can be stored in a word
processor document, spreadsheet, database or other common repository. In a
database system, you may also be able to see past test results and who
generated the results and the system configuration used to generate those
results. These past results would usually be stored in a separate table.
The most common term for a collection of test cases is a test suite.
The test suite often also contains more detailed instructions or goals for each
collection of test cases. It definitely contains a section where the tester identifies
the system configuration used during testing. A group of test cases may also
contain prerequisite states or steps, and descriptions of the following tests.
Collections of test cases are sometimes incorrectly termed a test plan. They may
also be called a test script, or even a test scenario.
Most white box tester write and use test scripts in unit, system, and regression
testing. Test scripts should be written for modules with the highest risk of failure
and the highest impact if the risk becomes an issue. Most companies that use
automated testing will call the code that is used their test scripts.
A scenario test is a test based on a hypothetical story used to help a person

think through a complex problem or system.
They can be as simple as a diagram for a testing environment or they could be a

description written in prose. The ideal scenario test has five key characteristics. It
is (a) a story that is (b) motivating, (c) credible, (d) complex, and (e) easy to
Page 13 of 62
evaluate. They are usually different from test cases in that test cases are single
steps and scenarios cover a number of steps. Test suites and scenarios can be
used in concert for complete system tests.
Scenario testing is similar to, but not the same as session-based testing, which is
more closely related to exploratory testing, but the two concepts can be used in
conjunction.
2.19.Defect Tracking
In engineering, defect tracking is the process of finding defects in a product,
(by inspection, testing, or recording feedback from customers), and tracking
them to closure.
Defect tracking is important in software engineering as complex software systems

typically have tens or hundreds of thousands of defects: managing, evaluating
and prioritizing these defects is a difficult task. Defect tracking systems are
computer database systems that store defects and help people to manage them.
2.20.Formal Verification
In the context of hardware and software systems, formal verification is the act
of proving or disproving the correctness of a system with respect to a certain
formal specification or property, using formal methods.
System types that are considered in the literature for formal verification include
finite state machines (FSM), labeled transition systems (LTS) and their
compositions, Petri nets, timed automata and hybrid automata, cryptographic
protocols, combinatorial circuits, digital circuits with internal memory, and
abstractions of general software components.
The properties to be verified are often described in temporal logics, such as linear
temporal logic (LTL) or computational tree logic (CTL).
Usually formal verification is carried out algorithmically. The main approaches to

implementing formal verification include state space enumeration, symbolic state
space enumeration, abstract interpretation, abstraction refinement, process-
algebraic methods, and reasoning with the aid of automatic theorem provers such
as HOL or Isabelle.
2.20.1.Validation and Verification

Verification is one aspect of testing a product's fitness for purpose. Validation
is the complementary aspect. Often one refers to the overall checking process
as V & V.
Validation: "Are we building the right product?” i.e., does the product do what
the user really requires.
Verification: "Are we building the product right?” i.e., does the product
conform to the specifications.
Page 14 of 62
The verification process consists of static and dynamic parts. E.g., for a
software product one can inspect the source code (static) and run against
specific test cases (dynamic).
Validation usually can only be done dynamically, i.e., the product is tested by
putting it through typical usages and atypical usages ("Can we break it?").
2.21.Fuzz Testing
Fuzz testing is a software testing technique. The basic idea is to attach the
inputs of a program to a source of random data. If the program fails (for
example, by crashing, or by failing in-built code assertions), then there are
defects to correct.
The great advantage of fuzz testing is that the test design is extremely simple,
and free of preconceptions about system behavior.
2.21.1.Uses
Fuzz testing is often used in large software development projects that perform
black box testing. These usually have a budget to develop test tools, and fuzz
testing is one of the techniques which offer a high benefit to cost ratio.
Fuzz testing is also used as a gross measurement of a large software system's

quality. The advantage here is that the cost of generating the tests is
relatively low. For example, third party testers have used fuzz testing to
evaluate the relative merits of different operating systems and application
programs.
Fuzz testing is thought to enhance software security and software safety

because it often finds odd oversights and defects which human testers would
fail to find, and even careful human test designers would fail to create tests
for.
However, fuzz testing is not a substitute for exhaustive testing or formal

methods: it can only provide a random sample of the system's behavior, and
in many cases passing a fuzz test may only demonstrate that a piece of
software handles exceptions without crashing, rather than behaving correctly.
Thus, fuzz testing can only be regarded as a proxy for program correctness,
rather than a direct measure, with fuzz test failures actually being more
useful as a bug-finding tool than fuzz test passes as an assurance of quality.
2.21.2.Fuzz testing methods

As a practical matter, developers need to reproduce errors in order to fix
them. For this reason, almost all fuzz testing makes a record of the data it
manufactures, usually before applying it to the software, so that if the
computer fails dramatically, the test data is preserved.
Modern software has several different types of inputs:

• Event driven inputs are usually from a graphical user interface, or possibly
from a mechanism in an embedded system.
Page 15 of 62
• Character driven inputs are from files or data streams.

• Database inputs are from tabular data, such as relational databases.
There are at least two different forms of fuzz testing:

• Valid fuzz attempts to assure that the random input is reasonable, or
conforms to actual production data.
• Simple fuzz usually uses a pseudo random number generator to provide
input.
• A combined approach uses valid test data with some proportion of totally
random input injected.
By using all of these techniques in combination, fuzz-generated randomness

can test the un-designed behavior surrounding a wider range of designed
system states.
Fuzz testing may use tools to simulate all of these domains.
2.21.3.Event-driven fuzz
Normally this is provided as a queue of data-structures. The queue is filled
with data structures that have random values.
The most common problem with an event-driven program is that it will often
simply use the data in the queue, without even crude validation. To succeed in
a fuzz-tested environment, software must validate all fields of every queue
entry, decode every possible binary value, and then ignore impossible
requests.
One of the more interesting issues with real-time event handling is that if
error reporting is too verbose, simply providing error status can cause
resource problems or a crash. Robust error detection systems will report only
the most significant or most recent error over a period of time.
2.21.4.Character-driven fuzz
Normally this is provided as a stream of random data. The classic source in
UNIX is the random data generator.
One common problem with a character driven program is a buffer overrun,

when the character data exceeds the available buffer space. This problem
tends to recur in every instance in which a string or number is parsed from
the data stream and placed in a limited-size area.
Another is that decode tables or logic may be incomplete, not handling every
possible binary value.
2.21.5.Database fuzz
The standard database scheme is usually filled with fuzz that is random data
of random sizes. Some IT shops use software tools to migrate and manipulate
such databases. Often the same schema descriptions can be used to
automatically generate fuzz databases.
Page 16 of 62
Database fuzz is controversial, because input and comparison constraints

reduce the invalid data in a database. However, often the database is more
tolerant of odd data than its client software, and a general-purpose interface
is available to users. Since major customer and enterprise management
software is starting to be open-source, database-based security attacks are
becoming more credible.
A common problem with fuzz databases is buffer overrun. A common data

dictionary, with some form of automated enforcement is quite helpful and
entirely possible. To enforce this, normally all the database clients need to be
recompiled and retested at the same time. Another common problem is that
database clients may not understand the binary possibilities of the database
field type, or, legacy software might have been ported to a new database
system with different possible binary values. A normal, inexpensive solution is
to have each program validate database inputs in the same fashion as user
inputs. The normal way to achieve this is to periodically "clean" production
databases with automated verifiers.
3. Manual Testing
3.1.Facts
• In India itself, Software industry growth has been phenomenal.
• IT field has enormously grown in the past 50 years.
• IT industry in India is expected to touch 10,000 crores of which software
share is dramatically increasing.
3.2.Software Crisis
• Software cost/schedules are grossly inaccurate. Cost overruns of several
times, schedule slippage’s by months, or even years are common.
• Productivity of people has not kept pace with demand. Added to it is the
shortage of skilled people.
• Productivity of people has not kept pace with demand Added to it is the
shortage of skilled people.
3.3.Software Myths
3.3.1.Management Myths
• Software Management is different.
• Why change or approach to development?
• We have provided the state-of-the-art hardware.
• Problems are technical
• If project is late, add more engineers.
• We need better people.
3.3.2.Developers Myths
• We must start with firm requirements
Page 17 of 62
• Why bother about Software Engineering techniques, I will go to terminal

and code it.
• Once coding is complete, my job is done.
• How can you measure the quality...it is so intangible.
3.3.3.Customer’s Myth
• A general statement of objective is good enough to produce software.
• Anyway software is “Flex-ware”, it can accommodate my changing needs.
3.3.4.What do we do?
• Use Software Engineering techniques/processes.
• Institutionalize them and make them as part of your development culture.
• Adopt Quality Assurance Frameworks : ISO, CMM
• Choose the one that meets your requirements and adopt where necessary.
3.4.Software Quality Assurance:

The purpose of Software Quality Assurance is to provide management with
appropriate visibility into the process being used by the software project
and of the products being built.
• Software Quality Assurance involves reviewing and auditing the software

products and activities to verify that they comply with the applicable
procedures and standards and providing the software project and other
appropriate managers with the results of these reviews and audits.
3.4.1.Verification:
• Verification typically involves reviews and meetings to evaluate
documents, plans, code, requirements, and specifications.
• The determination of consistency, correctness & completeness of a
program at each stage.
3.4.2.Validation:
• Validation typically involves actual testing and takes place after
verifications are completed
• The determination of correctness of a final program with respect to its
requirements.
3.5.Software Life Cycle Models:

• Prototyping Model
• Waterfall Model – Sequential
• Spiral Model
• V Model - Sequential
Page 18 of 62
3.6.What makes a good Software QA engineer?

The same qualities a good tester has are useful for a QA engineer.
Additionally, they must be able to understand the entire software
development process and how it can fit into the business approach and
goals of the organization.
Communication skills and the ability to understand various sides of issues are
important. In organizations in the early stages of implementing QA processes,
patience and diplomacy are especially needed. An ability to find problems as
well as to see 'what's missing' is important for inspections and reviews.
3.7.Testing:
• An examination of the behavior of a program by executing on sample data
sets.
• Testing comprises of set of activities to detect defects in a produced
material.
• To unearth & correct defects.
• To detect defects early & to reduce cost of defect fixing.
• To avoid user detecting problems.
• To ensure that product works as users expected it to.
3.7.1.Why Testing?
• To unearth and correct defects.
• To detect defects early and to reduce cost of defect fixing.
• To ensure that product works as user expected it to.
• To avoid user detecting problems.
3.8.Test Life Cycle

• Identify Test Candidates
• Test Plan
• Design Test Cases
• Execute Tests
• Evaluate Results
• Document Test Results
• Casual Analysis/ Preparation of Validation Reports
• Regression Testing / Follow up on reported bugs.
3.9.Testing Techniques
• Black Box Testing
• White Box Testing
• Regression Testing
These principles & techniques can be applied to any type of testing.
Page 19 of 62
3.10.Test Plan:
A Test Plan is a detailed project plan for testing, covering the scope of
testing, the methodology to be used, the tasks to be performed, resources,
schedules, risks, and dependencies.
A Test Plan is developed prior to the implementation of a project to provide a

well defined and understood project roadmap.
3.10.1.Test Specification:
A Test Specification defines exactly what tests will be performed and what
their scope and objectives will be.
A Test Specification is produced as the first step in implementing a Test Plan,

prior to the onset of manual testing and/or automated test suite
development. It provides a repeatable, comprehensive definition of a testing
campaign.
4. Testing Procedure
The following are some of the steps to consider:
• Obtain requirements, functional design, and internal design specifications

and other necessary documents.
• Obtain budget and schedule requirements. Determine project-related
personnel and their responsibilities, reporting requirements, required
standards and processes (such as release processes, change processes,
etc.)
• Identify application's higher-risk aspects, set priorities, and determine
scope and limitations of tests.
• Determine test approaches and methods - unit, integration, functional,
system, load, usability tests, etc.
• Determine test environment requirements (hardware, software,
communications, etc.)
• Determine test-ware requirements (record/playback tools, coverage
analyzers, test tracking, problem/bug tracking, etc.)
• Determine test input data requirements
• Identify tasks, those responsible for tasks, and labor requirements
• Set schedule estimates, timelines, milestones
• Determine input equivalence classes, boundary value analyses, error
classes
• Prepare test plan document and have needed reviews/approvals
• Write test cases
• Have needed reviews/inspections/approvals of test cases
• Prepare test environment and test-ware, obtain needed user
manuals/reference documents/configuration guides/installation guides, set
up test tracking processes, set up logging and archiving processes, set up
or obtain test input data
• Obtain and install software releases
• Perform tests
Page 20 of 62
• Evaluate and report results

• Track problems/bugs and fixes
• Retest as needed
• Maintain and update test plans, test cases, test environment, and test
ware through life cycle
4.1.Bug Tracking
What's a 'test case'?
• A test case is a document that describes an input, action, or event and an

expected response, to determine if a feature of an application is working
correctly. A test case should contain particulars such as test case identifier,
test case name, objective, test conditions/setup, input data requirements,
steps, and expected results.
• Note that the process of developing test cases can help find problems in
the requirements or design of an application, since it requires completely
thinking through the operation of the application. For this reason, it's
useful to prepare test cases early in the development cycle if possible.
What should be done after a bug is found?
• The bug needs to be communicated and assigned to developers that can

fix it. After the problem is resolved, fixes should be re-tested, and
determinations made regarding requirements for regression testing to
check that fixes didn't create problems elsewhere. If a problem-tracking
system is in place, it should encapsulate these processes. A variety of
commercial problem-tracking/management software tools are available
(see the 'Tools' section for web resources with listings of such tools). The
following are items to consider in the tracking process:
• Complete information such that developers can understand the bug, get
an idea of it's severity, and reproduce it if necessary.
• Bug identifier (number, ID, etc.)
• Current bug status (e.g., 'Released for Retest', 'New', etc.)
• The application name or identifier and version
• The function, module, feature, object, screen, etc. where the bug occurred
• Environment specifics, system, platform, relevant hardware specifics
• Test case name/number/identifier
• One-line bug description
• Full bug description
• Description of steps needed to reproduce the bug if not covered by a test
case or if the developer doesn't have easy access to the test case/test
script/test tool
• Names and/or descriptions of file/data/messages/etc. used in test
• File excerpts/error messages/log file excerpts/screen shots/test tool logs
that would be helpful in finding the cause of the problem
• Severity estimate (a 5-level range such as 1-5 or 'critical'-to-'low' is
common
• Was the bug reproducible?
• Tester name
Page 21 of 62
• Test date
• Bug reporting date
• Name of developer/group/organization the problem is assigned to
• Description of problem cause
• Description of fix
• Code section/file/module/class/method that was fixed
• Date of fix
• Application version that contains the fix
• Tester responsible for retest
• Retest date
• Retest results
• Regression testing requirements
• Tester responsible for regression tests
• Regression testing results
• A reporting or tracking process should enable notification of appropriate
personnel at various stages. For instance, testers need to know when
retesting is needed, developers need to know when bugs are found and
how to get the needed information, and reporting/summary capabilities
are needed for managers.
Why does software have bugs?
• Miscommunication or no communication - as to specifics of what an

application should or shouldn't do (the application's requirements).
• Software complexity - the complexity of current software applications can
be difficult to comprehend for anyone without experience in modern-day
software development. Windows-type interfaces, client-server and
distributed applications, data communications, enormous relational
databases, and sheer size of applications have all contributed to the
exponential growth in software/system complexity. And the use of object-
oriented techniques can complicate instead of simplify a project unless it is
well engineered.
• Programming errors - programmers, like anyone else, can make mistakes.
• Changing requirements - the customer may not understand the effects of
changes, or may understand and request them anyway - redesign,
rescheduling of engineers, effects on other projects, work already
completed that may have to be redone or thrown out, hardware
requirements that may be affected, etc. If there are many minor changes
or any major changes, known and unknown dependencies among parts of
the project are likely to interact and cause problems, and the complexity
of keeping track of changes may result in errors. Enthusiasm of
engineering staff may be affected. In some fast-changing business
environments, continuously modified requirements may be a fact of life. In
this case, management must understand the resulting risks, and QA and
test engineers must adapt and plan for continuous extensive testing to
keep the inevitable bugs from running out of control.
• Time pressures - scheduling of software projects is difficult at best, often
requiring a lot of guesswork. When deadlines loom and the crunch comes,
mistakes will be made.
• Egos - people prefer to say things like:
o 'no problem'
Page 22 of 62
o 'piece of cake'
o 'I can whip that out in a few hours'
o 'it should be easy to update that old code'
• Instead of:
o 'that adds a lot of complexity and we could end up
o making a lot of mistakes'
o 'we have no idea if we can do that; we'll wing it'
o 'I can't estimate how long it will take, until I take a close look at it'
o 'we can't figure out what that old spaghetti code did in the first place'
• If there are too many unrealistic 'no problems', the result is bugs.
• Poorly documented code - it's tough to maintain and modify code that is
badly written or poorly documented; the result is bugs. In many
organizations management provides no incentive for programmers to
document their code or write clear, understandable code. In fact, it's
usually the opposite: they get points mostly for quickly turning out code,
and there's job security if nobody else can understand it ('if it was hard to
write, it should be hard to read').
• Software development tools - visual tools, class libraries, compilers,
scripting tools, etc. often introduce their own bugs or are poorly
documented, resulting in added bugs.
5. Testing Tools and Software

5.1.Load and Performance Test Tools
AppPerfect DevSuite
5.2.Java test Tools
Java Development Tools
Java coverage, metrics, profiler, and clone detection tools from Semantic Designs.
AppPerfect DevSuite
Suite of testing, tuning, and monitoring products for java development from AppPerfect
Corp. Includes: Unit Tester, Code Analyzer, Java/J2EE Profiler and other modules.
Introscope
Performance monitoring tool from Wily Technology; presents data in easy-to-use
customizable dashboards which enable deep, intuitive views of interrelation between
system components and application infrastructure. Monitors applications as soon as
installed no coding is needed. Included 'LeakHunter'identifies potential memory leaks.
'Transaction Tracer' can provide detailed tracing of execution paths and component
response times for individual transactions in production systems.
GJTester
Java unit, regression, and contract (black box) test tool from TreborSoft. Enables test case
and test script development without programming. Test private and protected functions,
and server application's modules, without implementing test clients, regression testing for
JAVA VM upgrades. Useful for testing CORBA, RMI, and other server technologies as well.
GUI interface emphasizing ease of use.
Page 23 of 62
qftestJUI
Record/playback test tool from Quality First Software for creation, execution and
management of automated Java/Swing application tests. Includes a natural user interface,
scripting capabilities, and a component recognition algorithm that takes into account a
variety of attributes. Recorded GUI elements, user actions and associated data are
automatically integrated into an editable tree view reflecting the hierarchical structure of
the application's GUI. Extensive documentation.
Cactus
A simple open-source test framework for unit testing server-side java code (Servlets,
EJBs, Tag Libs, Filters, etc.). Intent is to allow fine-grained continuous testing of all files
making up an application: source code but also meta-data files (such as deployment
descriptors, etc) through an in-container approach. It uses JUnit and extends it. Typically
use within your IDE, or from the command line, using Ant. From Apache Software
Foundation.
JUnitPerf
Allows performance testing to be dynamically added to existing JUnit tests. Enables quick
composition of a performance test suite, which can then be run automatically and
independent of other JUnit tests. Intended for use where there are performance/scalability
requirements that need re-checking while refactoring code. By Mike Clark/Clarkware
Consulting, licensed under the BSD License.
QStudio for Java
Java code inspection tool from QA Systems allows automation of a major portion of code
inspection process, for early detection of software defects and automatic assessment of
code quality. Couples advanced static analysis capabilities to ISO 9126 quality standard
framework. Integrates with leading Java Development Environments and platforms.
Supports customizing existing rules and defining custom rules.
Koalog Code Coverage
Code coverage analyzer for Java applications from Koalog SARL. Includes: in-process or
remote coverage computation, capability of working directly on Java method binaries (no
recompilation), predefined (XML, HTML, LaTex, CSV, TEXT) or custom report generation,
and session merging to allow compilation of overall results for distinct executions.
Integrates with Ant and JUnit.
Abbot Java GUI Test Framework
Testing framework by Timothy Wall provides automated event generation and validation of
Java GUI components, improving upon the very basic functions provided by the
java.awt.Robot class. (Abbot = "A Better 'Bot'). The framework may be invoked directly
from Java code or accessed without programming through the use of scripts via 'Costello',
a script editor/recorder. Suitable for use both by developers for unit tests and QA for
functional testing. Free - available under the GNU Lesser General Public License.
JUnit - Framework to write repeatable java unit tests
A regression testing framework written by Erich Gamma and Kent Beck. For use by
developers implementing unit tests in Java. Free Open Source Software released under the
Page 24 of 62
IBM Public License and hosted on SourceForge. Site includes a large collection of
extensions and documentation.
jfcUnit
Framework for developing automated testing of Java Swing-based applications at the UI
layer (as opposed to testing at lower layers, for which JUnit may be sufficient). Provides
recording and playback capabilities. Also available as plugins for JBuilder and Eclipse. Free
Open Source Software from SourceForge site.
Jemmy
A Java library that is used to create automated tests for Java GUI applications. Contains
methods to reproduce all user actions which can be performed on Swing/AWT components
(i.e. button pushing, text typing, tree node expanding, ...). JemmyTest is a program
written in Java which uses the Jemmy API to test applications; it can be used separately as
well as together with the NetBeans IDE.
JBench
Freeware Java benchmarking framework to compare algorithms, virtual machines, etc. for
speed. Available as binary distribution (including documentation), source distribution, or
jar file.
Clover
Code coverage tool for Java from Cenqua. Fully integrated plugin for NetBeans, JBuilder,
and other IDE's. Seamless integration with projects using Apache ANT. View coverage data
in XML, HTML, PDF, or via a Swing GUI.
TrueJ
Source code audit and metrics tool from BlueBay systems. Fifty different audits and
metrics, compiler-style output, integrates with a variety of editors/IDE's, configurable,
integrates with build tools for quality gate and reporting, highly scalable.
JCover
Java code test coverage analysis tool from Codework Limited. Works with source or
compiled files. Gathers coverage measures of branches, statements, methods, classes,
file, package and produces reports in multiple formats. Coverage difference comparison
between runs. Coverage API provided.
reView
Java source code visualization tool from Headway Software. Reverse engineer and
automatically lay out and view code, components, and dependencies for Java, C, and C++
applications. Shows all dependencies, at all levels and between all levels; method, class,
package, application.
Panorama for Java
Visual environment containing six integrated java tools from ISA, Inc. J_SQA for Object-
Oriented software quality measurement; J_DocGen for Java code static analysis;
J_Structure for Java code structure analysis and diagramming; J_Diagrammer for Java
code logic analysis, control flow analysis and diagramming; J_Test for test coverage
analysis and test case minimization, etc.; and J_Playback for GUI operation capture and
automatic playback.
Java Tool Suite from Man Machine Systems
Page 25 of 62
Includes JStyle, a Java source analyzer to generate code comments and metrics such as
inheritance depth, Cyclomatic Number, Halstead Measures, etc; JPretty reformats Java
code according to specified options; JCover test coverage analyzer; JVerify Java class/API
testing tool uses an invasive testing model allowing access to internals of Java objects
from within a test script and utilizes a proprietary OO scripting language; JMSAssert, a tool
and technique for writing reliable software; JEvolve, an intelligent Java code evolution
analyzer that automatically analyzes multiple versions of a Java program and shows how
various classes have evolved across versions; can 'reason' about selective need for
regression testing Java classes; JBrowser class browser; JSynTest, a syntax testing tool
that automatically builds a Java-based test data generator.
PerformaSure
Low-overhead, user-friendly performance diagnosis tool from Quest Software for
distributed J2EE applications. Traces and reconstructs execution path of end-user
transactions across all components of a clustered multi-tieer J2EE system, to diagnose and
resolve performance bottlenecks. Hundreds of easily-confugured run-time, OS, and
network metrics.
JProbe Developer Suite
Collection of Java debugging tools from Quest Software; includes JProbe Profiler and
JProbe Memory Debugger for finding performance bottlenecks and memory leaks, LProbe
Coverage code coverage tool, and JProbe Threadalyzer for finding deadlocks, stalls, and
race conditions. JProfiler freeware version available.
Krakatau Metrics for Java
Software metrics tool from Power Software includes more than 70 OO, procedural,
complexity, and size metrics related to reusability, maintainability, testability, and clarity.
Includes Cyclomatic Complexity, Enhanced Cyclomatic Complexity, Halstead Software
Science metrics, LOC metrics and MOOD metrics. Has online advisor for quality
improvement.
OptimizeIt
Profiler, thread debugger, and code coverage tool suite from Borland (formerly from
VMGear).
Jtest
ParaSoft's Jtest is an integrated, automatic unit testing and standards compliance tool for
Java. It automatically generates and executes JUnit tests and checks whether code follows
400 coding standards and can automatically correct for many.
DevPartner Java Edition
Compuware's (formerly NuMega) debugging/productivity tool to detect and diagnose Java

bugs and memory and performance problems; thread and event analysis, coverage
analysis. Integrates with several Java IDE's.
VTune
Intel's performance tuning tool for applications running on Intel processors; includes Java
support. Includes suggestions for optimization techniques.
Sun's Java Test Tools
Page 26 of 62
As of February 4, 2000 Sun discontinued accepting orders for these products.
TCAT for Java
Part of Software Research's TestWorks suite of test tools; code coverage analyzer and code
analysis for Java; written in Java.
(Note: some other tools in these listings also handle testing, management, or load testing
of java applets, servlets, and applications, or are planning to add such capabilities. Check
listed web sites for current information.)
5.3.Link Checking Tools
HiSoftware Link Validation Utility
Link validation tool; free version or low-cost pro version.
SiteAnalysis
Hosted service from Webmetrics, used to test and validate critical website components,
such as internal and external links, domain names, DNS servers and SSL certificates. Runs
as often as every hour, or as infrequent as once a week. Ideal for dynamic sites requiring
frequent link checking.
ChangeAgent
Link checking and repair tool from Expandable Language. Identifies orphan files and
broken links when browsing files; employs a simple, familiar interface for managing files;
previews files when fixing broken links and before orphan removal; updates links to moved
and renamed files; fixes broken links with an easy, 3-click process; provides multiple-level
undo/redo for all operations; replaces links but does not reformat or restructure HTML
code. For Windows.
Link Checker Pro
Link check tool from KyoSoft; can also produce a graphical site map of entire web site.
Handles HTTP, HTTPS, and FTP protocols; several report formats available. For Windows
platforms.
Web Link Validator
Link checker from REL Software checks links for accuracy and availability, finds broken
links or paths and links with syntactic errors. Export to text, HTML, CSV, RTF, Excel.
Freeware 'REL Link Checker Lite' version available for small sites. For Windows.
Site Audit
Low-cost on-the-web link-checking service from Blossom Software.
Xenu's Link Sleuth
Freeware link checker by Tilman Hausherr; supports SSL websites; partial testing of ftp
and gopher sites; detects and reports redirected URL; Site Map; for Windows.
Linkalarm
Page 27 of 62
Low cost on-the-web link checker from Link Alarm Inc.; free trial period available.
Automatically-scheduled reporting by e-mail.
Alert Linkrunner
Link check tool from Viable Software Alternatives; evaluation version available. For
Windows.
InfoLink
Link checker program from BiggByte Software; can be automatically scheduled; includes
FTP link checking; multiple page list and site list capabilities; customizable reports;
changed-link checking; results can be exported to database. For Windows. Discontinued,
but old versions still available as freeware.
LinkScan
Electronic Software Publishing Co.'s link checker/site mapping tool; capabilities include
automated retesting of problem links, randomized order checking; can check for bad links
due to specified problems such as server-not-found, unauthorized-access, doc-not-found,
relocations, timeouts. Includes capabilities for central management of large multiple
intranet/internet sites. Results stored in database, allowing for customizable queries and
reports. Validates hyperlinks for all major protocols; HTML syntax error checking. For all
UNIX flavors, Windows, Mac.
CyberSpyder Link Test
Shareware link checker by Aman Software; capabilities include specified URL exclusions,
ID/Password entries, test resumption at interruption point, page size analysis, 'what's new'
reporting. For Windows.
5.4.Perl Testing Tools

W3C Link Checker
Link checker PERL source code, via the WWW Consortium (the folks who set web
standards); configurable. Handles one URL at a time.
HTML TIDY
Free utility available from SourceForget.net; originally by Dave Raggett. For automatic
fixing of HTML errors, formatting disorganized editing, and finding problem HTML areas.
Available as source code or binaries.
Big Brother
Freeware command-line link checker for Unix, Windows, by Francois Pottier. Available as
source code; binary avaialable for Linux.
LinkLint
Open source Perl program checks local/remote HTML links. Includes cross referenced and
hyperlinked output reports, ability to check password-protected areas, support for all
standard server-side image maps, reports of orphan files and files with mismatching case,
reports URLs changed since last checked, support of proxy servers for remote URL
checking. Distributed under Gnu General Public License. Has not been updated in recent
years.
MOMspider
Multi-Owner Maintenance Spider; link checker. PERL script for a web spider for web site
maintenance; for UNIX and PERL. Utilizes the HTTP 'HEAD' request instead of the 'GET'
request so that it does not require retreival of the entire html page. This site contains an
interesting discussion on the use of META tags. Not updated in recent years.
Page 28 of 62
HTMLchek for awk or perl

Old but still useful HTML 2.0 or 3.0 validator programs for AWK or PERL by H. Churchyard;
site has much documentation and related info. Not updated in recent years.
5.5.Web Functional and Regression Testing Tools

IeUnit
IeUnit is an open-source simple framework to test logical behaviors of web pages, released
under IBM's Common Public License. It helps users to create, organize and execute
functional unit tests. Includes a test runner with GUI interface. Implemented in JavaScript
for the Windows XP platform with Internet Explorer.
QEngine Web Test Studio

Web functional test tool from AdventNet. Scripting uses Jython; records using page
elements controls symbolically rather than with raw screen coordinate. Secure recording
on password fields; data-driven Test wizard to fetch script data from external source;
provision to add GUI, Database and File checkpoints and verify database tables, files, page
titles and HTML element properties. Supports keyword-driven testing, built-in exception
handling and reporting facility. Works with a variety of browsers and OS's. Free and
professional versions available.
AppPerfect DevSuite
Suite of testing, tuning, and monitoring products from AppPefect Corp. that includes a web
functional testing module. Records browser interaction by element instead of screen co-
ordinates. Supports handling dynamic content created by JavaScript; supports ASP, JSP,
HTML, cookies, SSL. For Windows and MSIE; integrates with a variety of IDE's.
JStudio SiteWalker
Test tool from Jarsch Software Studio allows capture/replay recording; fail definitions can
be specified for each step of the automated workflow via JavaScript. JavaScript's
Document Object Model enables full access to all document elements. Test data from any
database or Excel spreadsheet can be mapped to enter values automatically into HTML
form controls. HTML-based test result reports can be generated. Shareware for
Windows/MSIE.
Test Complete Enterprise

Automated test tool from AutomatedQA Corp. includes web functional testing capabilities.
Works with Internet Explorer.
QEngine
Test tool from AdventNet enables functional testing of Web sites and Web-based
applications. Record and playback capability; automatic recording of any Web browser
events and translates into an Python editable scripts. Includes Script Editor, Application
Map Editor to view and edit the map object properties. Supports multiple OS's and
browsers.
actiWate
Java-based Web application testing environment from Actimind Inc. Advanced framework
for writing test scripts in Java (similar to open-source frameworks like HttpUnit, HtmlUnit
etc. but with extended API), and Test Writing Assistant - Web browser plug-in module to
assist the test writing process. Freeware.
KUMO Editor
Toolset from Softmorning LTD for creation and editing of web macros and automated web
tests. Includes syntax-coloring editor with intellisense, autocomplete, run-time debugging
features. Macro recorder transforms any click to a C# directive. Page objects navigator
allows browsing of hierarchy of web objects in a page. Enables creation of scenarios from
Page 29 of 62
spreadsheets; and loop, retry on error, robust handling of page modifications. Can export
created .DLL and .EXE files to enable running web macros on demand and integration into
other software frameworks. Multilingual for Asian, eastern and western European
languages.
WebInject
Open source tool in PERL for automated testing of web applications and services. Can be
used to unit test any individual component with an HTTP interface (JSP, ASP, CGI, PHP,
servlets, HTML forms, etc.) or it can be used to create a suite of HTTP level functional or
regression tests.
Site Test Center

Functional and performance test tool from Alliance Software Engineering. Has an XML-
based scripting capability to enable modifying captured scripts or creating new scripts.
Utilizes a distributed testing model and consists of three parts: STC Administrator, STC
Master and STC Master Service.
jWebUnit
Open source Java framework that facilitates creation of acceptance tests for web
applications. Provides a high-level API for navigating a web application combined with a
set of assertions to verify the application's correctness including navigation via links, form
entry and submission, validation of table contents, and other typical business web
application features. Utilizes HttpUnit behind the scenes. The simple navigation methods
and ready-to-use assertions allow for more rapid test creation than using only JUnit and
HttpUnit.
SimpleTest
Open source unit testing framework which aims to be a complete PHP developer test
solution. Includes all of the typical functions that would be expected from JUnit and the
PHPUnit ports, but also adds mock objects; has some JWebUnit functionality as well. This
includes web page navigation, cookie testing and form submission.
WinTask
Macro recorder from TaskWare, automates repetitive tasks for Web site testing (and
standard Windows applications), with its HTML objects recognition. Includes capability to
expand scope of macros by editing and adding loops, branching statements, etc. (300+
commands); ensure robustness of scripts with Synchronization commands. Includes a
WinTask Scheduler.
TestCaseMaker/Runner
Test case document driven functional test tool for web applications from Agile Web
Development. Maker creates test case documents, and Runner executes the test case
document; test case documents are always synchronized with the application. Free
including source code.
Canoo WebTest
Free Java Open Source tool for automatic functional testing of web applications. XML-
based test script code is editable with user's preferred XML editor; until recording
capabilities are added, scripts have to be developed manually. Can group tests into a
testsuite that again can be part of a bigger testsuite. Test results are reported in either
plain text or XML format for later presentation via XSLT. Standard reporting XSLT
stylesheets included, and can be adapted to any reporting style or requirements.
TestSmith
Functional/Regression test tool from Quality Forge. Includes an Intelligent, HTML/DOM-
Aware and Object Mode Recording Engine, and a Data-Driven, Adaptable and Multi-
Threaded Playback Engine. Handles Applets, Flash, Active-X controls, animated bitmaps,
etc. Controls are recorded as individual objects independent of screen positions or
resolution; playback window/size can be different than in capture. Special validation
Page 30 of 62
points, such as bitmap or text matching, can be inserted during a recording, but all
recorded items are validated and logged 'on the fly'. Fuzzy matching capabilities. Editable
scripts can be recorded in SmithSript language or in Java, C++ or C++/MFC. 90-day
evaluation copy available.
TestAgent
Capture/playback tool for user acceptance testing from Strenuus, LLC. Key features
besides capture/playback include automatically detecting and capturing standard and
custom content errors. Reports information needed to troubleshoot problems. Enables
'Persistent Acceptance Testing' that activates tests each time a web application is used.
MITS.GUI
Unique test automation tool from Omsphere LLC; has an intelligent state machine engine
that makes real-time decisions for navigating through the GUI portion of an application. It
can test thousands of test scenarios without use of any scripts. Allows creation of
completely new test scenarios without ever having performed that test before, all without
changing tool, testware architecture (object names, screen names, etc), or logic
associated with the engine. Testers enter test data into a spreadsheet used to populate
objects that appear for the particular test scenario defined.
Badboy
Tool from Bradley Software to aid in building and testing dynamic web based applications.
Combines sophisticated capture/replay ability with performance testing and regression
features. Free for most uses; source code avalable.
SAMIE
Free tool designed for QA engineers - 'Simple Automated Module For Internet Explorer'.
Perl module that allows a user to automate use of IE via Perl scripts; Written in ActivePerl,
allowing inheritance of all Perl functionality including regular expressions, Perl dbi database
access, many Perl cpan library functions. Uses IE's built in COM object which provides a
reference to the DOM for each browser window or frame. Easy development and
maintenance - no need to keep track of GUI maps for each window. For Windows.
PAMIE
Free open-source 'Python Automated Module For Internet Explorer' Allows control of an
instance of MSIE and access to it's methods though OLE automation . Utilizes Collections,
Methods, Events and Properties exposed by the DHTML Object Model.
PureTest
Free tool from Minq Software AB, includes an HTTP Recorder and Web Crawler. Create
scenarios using the point and click interface. Includes a scenario debugger including single
step, break points and response introspection. Supports HTTPS/SSL, dynamic Web
applications, data driven scenarios, and parsing of response codes or parsing page content
for expected or unexpected strings. Includes a Task API for building custom test tasks. The
Web Crawler is useful for verifying consistency of a static web structure, reporting various
metrics, broken links and the structure of the crawled web. Multi-platform - written in
Java.
Solex
Web application testing tool built as a plug-in for the Eclipse IDE (an open, extensible
IDE). Records HTTP messages by acting as a Web proxy; recorded sessions can be saved
as XML and reopened later. HTTP requests and responses are fully displayed in order to
inspect and customize their content. Allows the attachment of extraction or replacement
rules to any HTTP message content, and assertions to responses in order to validate a
scenario during its playback.
QA Wizard
Automated functional web test tool from Seapine Software. Advanced object binding
reduces script changes when Web-based apps change. Next-generation scripting language
Page 31 of 62
eliminates problems created by syntax or other language errors. Includes capability for
automated scripting, allowing creation of more scripts in less time. Supports unlimited set
of ODBC-compatible data sources as well as MS Excel, tab/comma delimited file formats,
and more. Free Demo and Test Script available. For Windows platforms.
HTTP-WebTest
A Perl module which runs tests on remote URLs or local Web files containing
Perl/JSP/HTML/JavaScript/etc., and generates a detailed test report. This module can be
used "as-is" or its functionality can be extended using plugins. Plugins can define test
types and provide additional report capabilities. This module comes with a set of default
plugins, but can be easily extended with third-party plugins. Open-source project
maintained by Ilya Martynov.
HttpUnit
Open source Java program for accessing web sites without a browser, from
SourceForge.net/Open Source Development Network, designed and implemented by
Russell Gold. Ideally suited for automated unit testing of web sites when combined with a
Java unit test framework such as JUnit. Emulates the relevant portions of browser
behavior, including form submission, basic http authentication, cookies and automatic
page redirection, and allows Java test code to examine returned pages as text, an XML
DOM, or containers of forms, tables, and links. Includes ServletUnit to test servlets
without a servlet container.
iOpus Internet Macros

Macro recorder utility from iOpus Inc. automates repetitious aspects of web site testing.
Records any combination of browsing, form filling, clicking, script testing and information
gathering; assists user during the recording with visual feedback. Power users can
manually edit a recorded macro. A command line interface allows for easy integration with
other test software. Works by remote controlling the browser, thus automatically supports
advanced features such as SSL, HTTP-Redirects and cookies. Can handle data input from
text files, databases, or XML. Can extract web data and save as CSV file or process the
data via a script. For Windows and MSIE.
MaxQ
Free open-source web functional testing tool from Tigris.org, written in Java. Works as a
proxy server; includes an HTTP proxy recorder to automate test script generation, and a
mechanism for playing tests back from the GUI and command line. Jython is used as the
scripting language, and JUnit is used as the testing library.
TestWeb
Test tool from Original Software Group Ltd. utilizes a new approach to recording/playback
of web browser scripts. It analyses the underlying intentions of the script and executes it
by direct communication with web page elements. IntelliScripting logic removes the
reliance on specific browser window sizes, component location and mouse movements for
accurate replay, for easier script maintenance; supports hyperlinks targeted at new
instances of browser. Playback can run in background while other tasks are performed on
the same machine.
Compuware TestPartner
Automated software testing tool from Compuware designed specifically to validate
Windows, Java, and web-based applications. The 'TestPartner Visual Navigator' can create
visual-based tests, or MS VBA can be used for customized scripting.
WebKing
Web site functional, load, and static analysis test suite from ParaSoft. Maps and tests all
possible paths through a dynamic site; can enforce over 200 HTML, CSS, JavaScript, 508
compliance, WML and XHTML coding standards or customized standards. Allows creation of
rules for automatic monitoring of dynamic page content. Can run load tests based on the
tool's analysis of web server log files. For Windows, Linux, Solaris.
Page 32 of 62
eValid
Web test tool from Software Research, Inc that uses a 'Test Enabled Web Browser' test
engine that provides browser-based client side quality checking, dynamic testing, content
validation, page performance tuning, and webserver load and capacity analysis. Utilizes
multiple validation methods.
Rational Functional Tester

IBM's (formerly Rational's) automated tool for testing of Java, .NET, and web-based
applications. Enables data-driven testing, choice of scripting languages and editors. For
Windows and Linux.
e-Test Suite
Integrated functional/regression test tool from Empirix for web applications and services
and .NET and J2EE applications; includes site monitoring and load testing capabilities, and
record/playback, scripting language, test process management capabilities. Includes full
VBA script development environment and options such as javascript, C++, etc. DOM-
based testing and validation; 'Data Bank Wizard' simplifies creation of data-driven tests.
Evaluation version available.
QuickTest Pro
Functional/regression test tool from Mercury; includes support for testing Web, Java, ERP,
etc.
Winrunner
Functional/regression test tool from Mercury; includes support for testing Web, Java, ERP,
etc.
Compuware's QARun
QARun for functional/regression testing of web, Java, and other applications. Handles
ActiveX, HTML, DHTML, XML, Java beans, and more.
SilkTest
Functional test tool from Segue for Web, Java or traditional client/server-based
applications. Features include: test creation and customization, test planning and
management, direct database access and validation, recovery system for unattended
testing, and IDE for developing, editing, compiling, running, and debugging scripts, test
plans, etc.
5.6.Web Site Security Test Tools
Watchfire's AppScan
Tool suite from Watchfire automates web application security testing, produces
defect analyses, and offers recommendations for fixing detected security flaws.
Assessment module can be used by auditors and compliance officers to conduct
comprehensive audits, and to validate compliance with security requirements.
Codenomicon HTTP Test Tool
Tool for discovering and eradicating security-related flaws in HTTP

implementations via robustness testing. Tool systematically generates very large
numbers of protocol messages containing exceptional elements simulating
malicious attacks, in order to induce component crashes, hanging, and denial-of-
Page 33 of 62
service situations which may affect component/application security. (HTTP

implementations may be utilized in web servers, browsers, network appliances,
proxies, protocol analyzers, PDAs and cell phones.)
SecurityMetrics Appliance
Integrated software and hardware device includes Intrusion Detection and

Prevention Systems and Vulnerability Assessment. Operates as a Layer 2 Bridge -
no network configuration needed. Automatically downloads latest IDS attack
signatures, vulnerability assessment scripts and program enhancements nightly.
Lightning Console
Security management tool from Tenable Network Security for multiple security
and network administrators across multiple organizations. Scheduled vulnerability
scanning, real-time IDS event analysis, asset management, remediation
management of vulnerabilities, network topology discovery and organizational
executive reporting for hundreds of administrators via easy-to-use web interface.
SARA
'Security Auditor's Research Assistant' Unix-based security analysis tool from
Advanced Research Corp. Supports the FBI/SANS Top 20 Consensus; remote self
scan and API facilities; plug-in facility for third party apps; SANS/ISTS certified,
updated bi-monthly; CVE standards support; based on the SATAN model.
Freeware. Also available is 'Tiger Analytical Research Assistant' (TARA), an
upgrade to the TAMU 'tiger' program - a set of scripts that scan a Unix system for
security problems.
Qualys Free Security Scans
Several free security scan services from Qualys, Inc. including SANS/FBI Top 20
Vulnerabilities Scan, network security scan, and browser checkup tool.
Qualys Guard
Online service that does remote network security assessments; provides

proactive 'Managed Vulnerability Assessment', inside and outside the firewall.
Perimeter Check
SecurityMetrics 'Perimeter Check' service analyzes external network devices like

servers, websites, firewalls, routers, and more for security vulnerabilities which
may lead to interrupted service, data theft or system destruction. Includes
instructions to help immediately remedy security problems. Can automatically
schedule vulnerability assessment of designated IP addresses during low traffic
times.
MacAnalysis
Page 34 of 62
Low cost Network Security audit tool from Lagoon Software. Provides detailed
reports and vulnerability fix recommendations. MacAnalysis database is updated
regularly with a "Vulnerability Update" function to update database of more than
100 new holes per month. Runs on Macintosh, can audit UNIX, Mac, Win servers
STAT Scanner
Tool from Harris Corp. for security analysis of Windows/UNIX/Linux and other
resources. Utilizes comprehensive updatabale vulnerability databases to
automatically detect vulnerabilities. Capabilities include: scan and analyze an
entire network domain and/or a single machine, select or ignore specific
vulnerabilities via configuration files, analysis reports of vulnerabilities with
detailed information relating to the name/description/risk level of each
vulnerability, eliminate vulnerabilities using recommended solutions with links to
related websites and knowledge-base articles, correct vulnerabilities across the
network with 'AutoFix' function, retest corrected vulnerabilities immediately, track
vulnerability trends via analyses comparing current and previous assessments,
customizable security reports for management and technical personnel.
Nessus Security Scanner
Free, open-source remote network security auditing tool, by Renaud Deraison,

based on 'never trust the version number' and 'never trust that a given service is
listening on the good port'. Nessus is made up of two parts : a server and a
client; the server (nessusd) manages the 'attacks', whereas the client is a
frontend designed to collect the results. Includes more than 1000 tests in 23
vulnerability categories, and Nessus Attack Scripting Language. Works with a
variety of operating systems.
NetRecon
Network vulnerability assessment scanner from Symantec. Scans systems and
services on the network and safely simulates common intrusion or attack
scenarios. Utilizes a root-cause and path-analysis engine to illustrate exact
sequence of steps taken to uncover vulnerabilities, enabling identification of
exactly where to correct vulnerabilities to enforce corporate security policies.
Learns about network as it scans, adapting penetration strategy based on
previous results. Customizable reporting. Can scan a variety of operating
systems; product runs on Windows.
Secure-Me
Automated security test scanning service from Broadbandreports.com for
individual machines. Port scans, denial-of-service checks, 45 common web server
vulnerability checks, web server requests-per-second benchmark, and a wide
variety of other tests. Limited free or full licensed versions available.
SAINT
Security Administrator's Integrated Network Tool - Security testing tool from
SAINT Corporation. An updated and enhanced version of the SATAN network
Page 35 of 62
security testing tool. Updated regularly; CVE compatible. Includes DoS testing,
reports specify severity levels of problems. Single machine or full network scans.
Also available is 'WebSAINT' self-guided scanning service, and SAINTbox scanner
appliance. Runs on many UNIX flavors.
NMap Network Mapper
Free open source utility for network exploration or security auditing; designed to
rapidly scan large networks or single hosts. Uses raw IP packets in novel ways to
determine what hosts are available on the network, what services (ports) they
are offering, what operating system (and OS version) they are running, what
type of packet filters/firewalls are in use, and many other characteristics. Runs on
most flavors of UNIX as well as Windows.
bv-Control for Internet Security
Secutiry scanning tool from BindView, includes capabilities for checking and
remediation of security problems of servers, workstations, routers, hubs,
printers, and any other devices with an IP address on a network. Scans any
device on network regardless of platform.
NetIQ Security Analyzer
Multi-platform vulnerability scanning and assessment product. Systems are

analyzed on demand or at scheduled intervals. Automatic update service allows
updating with latest security tests. Includes a Software Developer's Kit to allow
custom security test additions. For Windows/Solaris/Linux.
Foundstone
Vulnerability management software tools from McAfee/Network Associates can
provide comprehensive enterprise vulnerability assessments, remediation
information, etc. Available as a hardware appliance, software product, or
managed service.
CyberCop ASaP
Managed vulnerability assessment service from McAfee/Network Associates.

Remotely evaluates security of network perimeter, DMZ and externally visible
assets.
CERIAS Security Archive
Purdue University's 'Center for Education and Research in Information Assurance

and Security' site; 'hotlist' section includes extensive collection of links, organized
by subject, to hundreds of security information resources and tools, intrusion
detection resources, electronic law, publications, etc. Also includes an FTP site
Page 36 of 62
with a large collection of (mostly older) security-related utilities, scanners,

intrusion detection tools, etc.
InternetScanner
Tool from Internet Security Systems, provides automated vulnerability
assessment for measuring online security risks. Performs scheduled and selective
probes of network services, servers, desktops, operating systems, routers,
firewalls, etc. to uncover and report systems vulnerabilities that might be open to
attack. Provides flexible risk management reports, prepares remediation advice,
trend analyses and comprehensive data sets to support policy enforcement.
Similar managed service also available.
Top 75 Security Tools
Listing of 'top 75' network security tools from survey by Insecure.org. (Includes
various types of security tools, not just for testing.)
5.7.Web Site Management Tools
(This section includes products that contain: site version control tools, combined
utilities/tools, server management and optimization tools, and
authoring/publishing/deployment tools that include significant site management
or testing capabilities.)
COAST WebCentral
Site management tool from Coast Software, Inc. for monitoring compliance standards
related to Privacy, Information Assurance, Accessibility, and Web Governance. Capabilities
include: high-level summary reports, site scores, and trend analyses using intuitive
browser-based reports;roll-up of results from different business units or global Web sites
for overall views of site compliance; complete inventory of Web sites and Comparison
Inventory Reports to compare different versions of a Web site.
Maxamine Knowledge Platform
Integrated site managment tool from Maxamine Inc., combining site content, structure
and visitor traffic data into one searchable data base, to enable detailed analysis of what is
going on behind the scenes of Web operations. Desktop product for multiple platforms,
and online service available.
WebCEO
Tool from Radiocom Ltd. for maintaining, promoting, analyzing web sites. Limited-
capability free version also available. Includes link checker, WYSIWYG editor,
FPT/publishing, traffic analysis, and site monitoring capabilities.
ManageEngine Applications Manager
Site management tool from AdventNet; works with a variety of web servers, database
servers, service types, and OS's. Free and professional versions available.
SilkCentral Performance Monitor
Page 37 of 62
Tool from Segue Software, Inc., actively monitors and evaluates the health of a site's live
applications utilizing synthetic business transactions. Passive monitoring records real user
sessions and reliably detects and logs all error situations encountered by real-users.
Recorded transactions can then be analyzed when production errors occur and fed back
into SilkPerformer for real-world performance testing scenarios.
webSensor
Integrated site measurement and analysis tool from Moniforce BV; Integrates performance
and availability data with site traffic data, including server errors, response errors, network
errors, page aborts, response times, page sizes, bandwidth usage. Other capabilities
include search engine optimization; click path analysis; visitor segmentation; real-time
executive dash board.
RealiTea
Web application management solution that provides detailed visibility into availability and
functionality issues to enable efficient problem identification, isolation, and repair.
Captures and monitors real user sessions, providing context and correlation data for
application failure analysis. Add-on capabilities include a 'Dashboard' to provide real-time,
customizable views of success/failure rates for key online business processes and other
critical metrics, and 'Real Scripts' automatically generated from recorded user sessions for
use in specified other load testing tools.
PROGNOSIS
Comprehensive tool from Integrated Research Ltd. for performance and availability
monitoring, network management, and diagnostics; suited to large systems.
RedDot CMS
Web content managment system from RedDot Solutions includes modules such as
SmartEdit; Asset Manager to securely centralize images; Site Manager to create and
manage your web site; Web Compliance Manager to manage integrity and accessibility,
and more.
Cuevision Network Monitor
Monitoring tool from Cuevision for monitoring website, server, services, applications, and
network; capabilities include notifications via email, net send, and popup, restart apps and
services, etc. For Windows.
GFI Network Server Monitor
Server management tool from GFI Software Ltd. checks network and servers for failures
and fixes them automatically. Alerts via email, pagers, SMS; automatically reboot servers,
restart services, run scripts, etc. Freeware version of GFI Network Server Monitor is also
available; includes modules to check HTTP and ICMP/ping for checking availability of HTTP
and HTTPS sites.
Web Site Monitoring
Page 38 of 62
Performance Monitoring - Free open-source website performance monitoring and uptime

notification application in PERL, from AllScoop; sends email notification if site is slow or
down.
ContentStudio
E-catalog management tool from TechniCon Systems with Win Explorer-type interface with
drag and drop functionality; eliminates need for programmers and special production staff
to maintain catalogs. Legacy-to-Web Tools can "bulk-load" online catalog from legacy
product data. Capabilities include defining intra-configuration rules, such as option
compatibilities on a single product; spatial relationships between products, etc.
FTP Assistant for Webmasters
FTP client from Byteway.com for web site updating; additional features include manual or
customizable automated control, link verification, HTML and GIF optimization, site cleanup,
drag-and-drop, upload resuming, scheduler, and more.
TestWEB Monitor
Web site monitoring tool from Original Software checks 5 or more URL's up to every 30
seconds. Receive instant notification by email whenever one of the sites is unavailable.
SpinPike
Flexible and scalable content management system from SavvyBox Systems, based on
database-driven, template-based dynamically-created content. Installer easily installs
system on your server, high-level functions save template coding time; WYSIWYG editor.
Constructioner
Website development software with integrated content management system from Artware
Multimedia GmbH. Design/administrate database connected PHP web applications in
combination with individual webdesign. Includes: Ready-to-use Backoffice, Content and
Table Management (WYSIWYG-Editor), User Administration, Multilingualism, Dynamic
Menu, Message Board, PHP-Code Insertion, Statistical Reports, Database Backup, Search.
All can be integrated without writing code. Constructioner Light Edition available as
Freeware.
CrownPeak CMS
Content management service from CrownPeak Technology, which hosts the management
system application and the client's administrative interfaces and pushes the final
assembled pages to client Web servers. Provides complete software developers
environment, comprehensive Communications Gateway for inbound and outbound data,
and a robust API.
WebLight
HTML validator and link checking tool from Illumit LLC. Free for use on small sites, low
cost for large sites. Works on multiple platforms.
Tonic
Page 39 of 62
Suite of web site management/monitoring/test tools from Tonic Software; can manage all
components across complex Web-enabled applications by measuring and monitoring
performance, verifying content and functional integrity, load testing, detecting, reporting
and fixing problems before users see them.
Trellian InternetStudio
Suite of web site management utilities from Trellian including site upload/publishing tools,
text editor, HTML editor, link checker, site mapper, spell checker, site spider, image
handling, HTML encryptor/optimizer, HTML validator, image mapper, e-commerce site
designer/generator. For Windows.
Watchfire WebXM
Content analysis tool from Watchfire analyzes sites for slow loading pages, security,
searchability, broken links, accessibility, privacy compliance, and website standards
compliance.
Documentum
Enterprise content management product from EMC Corp. - capabilites/support include
scalability, security, business process automation, globalization, XML-content-based multi-
channel delivery, support for more than 50 document formats, integration with a variety of
servers, authoring tools, etc.
Serena Collage
Content management tool from Serena; browser-based, scalable content management

platform for content contributors distributed across an organization. Works with content
from any platform or application. Enables collaboration, version control, activity tracking,
administration, templates, styles, approval workflow, multi-lingual support, more. Runs
with a variety of platforms, web servers, and DB servers.
FlexWindow
Tool from Digital Architects B.V., enables users to update their web site via e-mail. Update
news flashes, notifications, advertisements, product info, stories, prices, and more. Use
any e-mail client capable of producing HTML to format your content or use HTML tags in a
plain text e-mail. Easy to install, simply create an account and paste one line of javascript
into your pages. Basic accounts are free.
Alchemy Eye
System management tool from Alchemy Lab continuously monitors server availability and
performance. Alerts by cell phone, pager, e-mail, etc. Can automatically run external
programs, and log events.
Web500 CMS
Page 40 of 62
Web content management and site maintenance solution from Web500. Add-on modules
allow capabilities such as WAP, e-commerce, payment processing, customer relationship
management, and more.
HTML Rename
Site Migration/Batch processing tool from Expandable Language that enforces file naming
conventions (case, length, invalid chars), renaming the files to match the convention, then
correcting the links to those files automatically. Eliminates problems encountered when
moving files between Windows, Mac, and UNIX systems and publishing to CD-ROM. For
Mac or Windows.
IP Check
Server monitoring tool from Paessler GmbH. Alerts webmasters if a webserver is not
working correctly because the web service is down, CGI scripts return errors or the SQL
server is not reachable. It can constantly monitor any number of servers and ports. For
Windows.
Stellent Universal Content Management System
Content management tool from Stellent, Inc. 'Stellent Content Server' uses a web-based
repository, where all content and content types are stored for management, reuse and
access. Enables services such as library services, security, conversion services, workflow,
personalization, index/search, replication and administration. Other modules provide
additional services such as: services for creating, managing and publishing Web content
and supporting from one to thousands of Web sites; services for capturing, securing and
sharing digital and paper-based documents and reports; and services for collaborative
environments and for digital asset and records management.
Rhythmyx Content Manager
Web content management product from Percussion Software; based on native XML and
XSL technologies; content development, publishing, version control, and customizable
workflow. Manages Web content, documents, digital assets, portals and scanned images.
Content Management Server
Windows based content mgmt tool from Microsoft (formerly 'nResolution' from nCompass
Labs). Enterprise web content management system that enables quickly and efficiently
building, deploying, and maintaining highly dynamic web sites. Enables scheduling of
content refreshes, management of workflow, tracking of revisions, and indexing content by
means of a browser window or via MS Word.
Broadvision
Suite of content and publishing management tools from Broadvision Inc.; allows a
distributed team of non-technical content experts to manage every aspect of site content,
including creation, editing, staging, production, and archiving.
Page 41 of 62
HP OpenView Internet Services
Internet services monitoring/management tool from HP; integrates with other OpenView
products to provide a variety of management and monitoring services and capabilities.
Enables end-user emulation of major business-critical applications as well as a single
integrated view of the complete Internet infrastructure. Designed to help IT staff efficiently
predict, isolate, diagnose and troubleshoot problem occurrences, anticipate capacity
shortfalls, and manage and report on service level agreements.
HTML-Kit
Free, full-featured editor from Chami.com designed to help HTML, XHTML and XML authors
to edit, format, lookup help, validate, preview and publish web pages. Uses a highly
customizable and extensible integrated development environment while maintaining full
control over multiple file types including HTML, XHTML, XML, CSS, XSL, JavaScript, Perl,
Python, Ruby, Java, and much more. Finds errors and provides suggestions on how to
create standards compliant pages. Includes internal, external, server-side and live preview
modes; FTP Workspace for uploading, downloading and online editing of files; and the
ability to use hundreds of optional free add-ins through its open plugins interface. GUI
support of W3C's HTML Tidy; seamless integration with the CSE HTML Validator. Validate
XML documents using its DTD and/or check for well-formedness. Over 400 free plugins
available for extending and customizing HTML-Kit. Pro plugins available to paid registered
users.
IBM Workplace Web Content Management
IBM's web content management product for Internet, intranet, extranet and portal sites;
runs on both Lotus Domino and IBM WebSphere.
WebCheck
Windows application that runs in background and periodically checks a site for availability
and correctness; searches for keywords; provides notification by displaying a message or
sending an e-mail. Requires MSIE.
WS_FTP Pro
FTP/web publishing tool from Ipswitch; manage, upload, and update websites;
automatically resume interrupted transfers; support more than 50 host file systems; drag-
and-drop files; for Windows.
A1Monitor
Utility from A1Tech for monitoring availability of web servers. Capabilities include
notification by email and automatic reboot of web server. For Windows.
WebLog Manager
In addition to logging capabilities, checks search engine placement including position,

page, rank on page, click paths. Enhanced logging utilizing custom generated script
uploaded to host; includes 'anti-caching technology' ensuring more reliable logging by
blocking caching of site's pages by visitor's browsers; 'Active Protect MetaTags' allowing
only search engine spiders to see front-page meta-tags; 'Active Protect Images' to prevent
other sites from "borrowing" images; and 'Active Deny Access' to prevent particular IP
addresses from accessing web site. From Monocle Solutions. Requires UNIX, PERL, SSI, on
web server and Windows client.
Page 42 of 62
AgentWebRanking
Freeware tool from AADSoft to monitor site's search engine position, improve search
engine ranks, submit URL's. Searches top engines for keywords; can specify search depth.
Also has keyword count for pages vs competitor's pages; auto or manual submit of URL's
to search engines, meta tag creator. Requires MSIE and Windows.
WebSite Director
Web-content workflow management system with browser-based interface includes

configurable workflow management, e-mail submission of web content, and e-mail
notifications; allows defining and applying existing workflow and approval rules to web
content management process. For Windows, UNIX.
Equalizer
Load balancing server appliance and site management tool from Coyote Point Systems.
Web based interface for load balancing administration, server failure detection, real-time
server monitoring of server response time, number of pending requests, etc.
WebTrends Analysis Suite
Web site management tool from NetIQ includes log analysis, link analysis and quality
control, content management and site visualization, alerting, monitoring and recovery,
proxy server traffic analysis and reporting. For Windows.
XMetal
XML development tool from BlastRadius for XML-based web site authoring and validation.
Includes a 'Database Import Wizard', and can automatically convert output to CALS or
HTML table models or to XML; For Windows.
Unicenter TNG w/Web Management Option
Site management application from Computer Associates includes access and security
control, monitoring, logging, metrics, server management, network management. For MS
and Netscape web servers.
Interwoven Team Site
Web development, version control, access control, and publishing control tool; works with
many servers, OS's, and platforms.
Macromedia Web Publishing System
Macromedia's complete, scalable web content management solution that enables

collaboration, management, and publishing to websites within a centrally controlled,
standards-based environment. Supports multiple platforms and browsers.
Site/C
Page 43 of 62
'Set-and-forget' utility from Robomagic Software; for periodic server monitoring for web
server connection problems, link problems. E-mail/pager notifications, logging capabilities.
For Windows.
PowerMapper
From Electrum Multimedia; for customizable automated site mapping, HTML validation,
link checking. Evaluation copy available; requires Windows and MSIE.
SiteScope
Mercury's product for agentless site monitoring and maintenance. Runs on servers and
monitors server performance, links, connections, logs, etc.; scheduled and on-demand
reporting; provides notifications of problems. Includes published API for creating custom
monitors. Monitors mimic users' end-to-end actions. For Windows or Unix.
HTML PowerTools
HTML validator, global search-and-replace. Date stamper, spell checker, Meta manager,
image tag checker, HTML-to-Text converter, customizable reports. Link checker. Validates
against various HTML versions, browser extensions; has updateable rulebase. From
Talicom. For Windows.
OpenDeploy
Interwoven's configurable control system for deploying from development to production
environments. Includes automated deployment, security, and encryption capabilities. For
Windows and Unix.
TeamSite
Interwoven's collaborative web site production control, administration, and management
product for enterprise-wide internet and intranet projects. Includes version control,
browser interface, comparison capabilitie, file edit merging, variable lock controls. Server
side compatible with many available web servers.
Content Management Server
Vignette Corporation's product for web site collaborative content, publishing,

management, and maintenance. Support for managing content stored in databases, XML
repositories, and static files. Supports a wide variety of web attributes, databases, API's,
and servers.
Microsoft FrontPage
Microsoft's web site authoring and site management tool; includes site management
capabilities, link checking, etc.
HomeSite
Macromeida's web site project management/authoring/validator tool; page in screen is
validated with results displayed below; error info is linked to error in html; file uploading;
link checker; thumbnail viewer. For Windows.
Page 44 of 62
NetObjects Fusion
Site authoring/management tool. Visual site structure editor, layout editor, graphics
management, staging/publishing control. For Windows.
5.8.Other Web Testing Tools
Morae
Usability test tool for web sites and software, from TechSmith Corp. for automated
recording, analyzing and sharing of usability data. Consists of 3 components. A Recorder
records and synchronizes video and data, creating a digital record of system activity and
user interaction. A Remote Viewer enables geographically dispersed observers to watch
usability tests from any location; it displays test user's computer screen along with a
picture-in-picture window displaying the test participant's face and audio; Remote Viewer
observers can set markers and add text notes. The Manager component includes
integrated editing functionality for assembly of important video clips to share with
stakeholders.
Repro
Manual testing 'helper' tool that records desktop video, system operations in 7 different
categories, system resource usage, and system configuration information. Allows user to
save and review relevant information for bug reports, and compress the result into a very
small file to replay, upload to a bug tracking system, and share with others. Instruments in
memory the target application at runtime so no changes are required to application under
test. For Windows.
Extreme Web Analyzer
Web site performance analysis tool by Dale Feiste for use in a development or controlled
test environment, where network traffic is captured over a LAN. The predictable network
performance of this environment enables reliable modeling of trace data. Free; requires a
system capable of running Ethereal and/or Sniffer, Perl, and MS Excel for charting and
modeling.
URL2image.com
Service from HREF Tools to check web page appearance in different Browser/OS
combinations. For anyone interested in css, web standards and elastic design; can specify
the screen width(s), font magnification(s) and page position(s) for the proofs. Enter URL
and receive back report with screenshots taken in real time on real hardware.
TestGen
Free open-source web test data generation program that allows developers to quickly
generate test data for their web-services before publicly or internally releasing the web
service for production.
EngineViewer and SiteTimer
Free basic services: EngineViewer - reports on how a search engine may view a webpage,
from how it breaks down the HTML, to which links it extracts, how it interprets page's
robot exclusion rules and more. SiteTimer service - Find out how long it takes various
Page 45 of 62
connection types to get a page, check all the graphical links to ensure they're correct,
examine server's HTTP headers, more.
Fiddler
An HTTP Debugging tool by Eric Lawrence. Acts as an HTTP Proxy running on port 8888 of
local PC. Any application which accepts an HTTP Proxy can be configured to run through
Fiddler. Logs all HTTP traffic between between computer and the Internet, and allows
inspection of the HTTP data, set breakpoints, and "fiddle" with incoming or outgoing data.
Designed to be much simpler than using NetMon or Achilles, and includes a simple but
powerful JScript.NET event-based scripting subsystem. Free, for Windows.
FREEping
Free ping software utility from Tools4ever which will ping all your Windows-based servers
(or any other IP address) in freely-definable intervals. Will send a popup when one of the
servers stops responding.
IP Traffic Test and Measure
Network traffic simulation and test tool from Omnicor Corp. can generate TCP/UDP
connections using different IP addresses; data creation or capture and replay; manage and
monitor throughput, loss, and delay.
SetSim Pro
Northern Webs' free on-the-web meta tag checker; includes additional search-engine-
related capabilities.
VisitorVille
Site traffic monitoring tool from World Market Watch Inc. that depicts website visitors as
animated characters in a virtual village; users can watch their web traffic as if they're
watching a movie.
FunnelIT
Free utility from ByStorm Software helps streamline test reporting and analysis by
gathering Windows platform environment/configuration information (running tasks, loaded
modules, virtual memory settings, OS version, component details, processor information,
physical memory information, etc.) in small text files. Available as an executable intended
to leave no footprint. Full-featured version (not free) has configurable reporting, and also
allows gathering TCP/IP Address/Network Information, Network Adaptor Information,
Services information, Installed Software and Installed XP Updates, Drive Free
Space/Size/Used info, etc.). Also allows comparisons among results from multiple
systems.
Sandra
'System ANalyser, Diagnostic and Reporting Assistant' utility from SiSoftware. Provides
large variety of information about a Windows system's hardware and software. Includes
CPU, mainboard, drives, ports, processes, modules, services, device drivers, ODBC
sources, memory details, environment settings, system file listings, and much more.
Provides performance enhancing tips, tune-up wizard, file system and memory bandwidth
Page 46 of 62
benchmarking, more. Reporting via save/print/fax/email in text, html, XML, etc. Free,
Professional, and other versions available in multiple languages.
Path Application Manager
Application Monitoring and management tool from Winmoore, Inc. Uses pattern
recognition technology to peer deep inside customized or COTS applications, analogous to
running an MRI scan. Enables enhancement of QA, testing, and troubleshooting with code
coverage capabilities.
RAMP
Section 508 and W3C Accessibility Guidelines tool from Deque Systems that automates
analysis and remediation of non-compliant web functionality.
Browser Cam
Service for web developers and testers that creates screen captures of web pages loaded
in any browser, any version, any operating system. Allows viewing of web page
appearance on Windows, Linux, Macintosh, in most versions of every browser ever
released.
Dummynet
Flexible tool developed by Luigi Rizzo, originally designed for testing networking protocols,
can be used in testing to simulate queue and bandwidth limitations, delays, packet losses,
and multipath effects. Can be used on user's workstations, or on FreeBSD machines acting
as routers or bridges.
HTTP Interceptor
A real-time HTTP protocol analysis and troubleshooting tool from AllHTTP.com. View all
headers and data that travel between your browser and the server. Split-screen display
and dual logs for request and response data. Interceptor also allows changing of select
request headers on-the-fly, such as "Referrer" and "User Agent".
SpySmith
Simple but powerful diagnostic tool from Quality Forge; especially useful when testing web
sites and web-based applications. It allows the user to peek inside I.E. Browser-based
Documents (including those without a 'view source' command) to extract precise
information about the DOM elements in an HTML source. SpySmith can also spy on
Windows objects. For Windows. Free 90-day trial.
Co-Advisor
Tool from The Measurement Factory for testing quality of protocol implementations. Co-
Advisor can test for protocol compatibility, compliance, robustness, security, and other
quality factors. Co-Advisor's current focus is on HTTP intermediaries such as firewalls,
filters, caching proxies, and XML switches. Other info: runs on FreeBSD packages, Linux
RPMs, Windows (on-demand); available as on-line service, binaries, or source code.
Page 47 of 62
PocketSOAP
Packet-capture tool by Simon Fell, with GUI; captures and displays packet data between
local client and specified web server. Can log captures to disk. For Windows; binaries and
source available; freeware. Also available is PocketXML-RPC and PocketHTTP.
TcpTrace
Tool by Simon Fell acts as a relay between client and server for monitoring packet data.
Works with all text-based IP protocols. For windows; freeware.
ProxyTrace
Tool by Simon Fell acts as a proxy server to allow tracing of HTTP data; can be used by
setting browser to use it as a proxy server and then can monitor all traffic to and from
browser. Freeware.
tcptrace
Tool written by Shawn Ostermann for analysis of TCP dumpfiles, such as those produced
by tcpdump, snoop, etherpeek, HP Net Metrix, or WinDump. Can produce various types of
output with info on each connection seen such as elapsed time, bytes, and segments sent
and received, retransmissions, round trip times, window advertisements, throughput, and
various graphs. Available for various UNIX flavors, for Windows, and as source code;
freeware.
MITs.comm
Tool from Omsphere LLC for simulating virtually any software interface (internal or
external). Allows testing without pitfalls associated with live connections to other systems
(TCP/IP, Ethernet, FTP, etc). Allows developers to test down to the unit level by simulating
the internal software interfaces (message queues, mailboxes, etc.) Tool can learn what
request/response scenarios are being tested for future tests and can work with any
protocol, any message definitions, and any network. Also available: MITS.GUI
XML Conformance Test Suite
XML conformance test suites from W3C and NIST; contains over 2000 test files and an
associated test report (also in XML). The test report contains background information on
conformance testing for XML as well as test descriptions for each of the test files. This is a
set of metrics for determining conformance to the listed W3C XML Recommendation.
Certify
Test automation management tool from WorkSoft, Inc. For managing and developing test
cases and scripts, and generating test scripts. For automated testing of Web, client/server,
and mainframe applications. Runs on Windows platforms.
HiSoftware AccVerify
Tool for testing site Accessibility & Usability, Searchability, Privacy and Intellectual Property
policy verification, Overall Site Quality, Custom Checks and Test Suites to meet
organization's standards. Can crawl a site and report errors; can also programmatically fix
most common errors found. Runs on Windows.
Page 48 of 62
HiSoftware Web Site Monitor
Tool allows user to monitor your server and send alerts, allows monitoring web sites for
changes or misuse of your intellectual property in metadata or in the presented document;
link validation.
Web Optimizer
Web page optimizing tool from Visionary Technologies intelligently compresses web pages
to accelerate web sites without changing site's appearance. Removes unnecessary
information in HTML, XML, XHTML, CSS, and Javascript and includes GIF and JPEG
optimizer techniques.
HTML2TXT
Conversion utility that converts HTML as rendered in MS Internet Explorer into ASCII text
while accurately preserving the layout of the text. Included with software are examples of
using the control from within Visual Basic, Visual C++, and HTML.
Team Remote Debugger
Debugging tool from Spline Technologies allows tracing of any number of code units of any
kind ( ASP, MTS, T-SQL, COM+, ActiveX Exe, DLL, COM, Thread, CFML ), written in any
language ( ASP, VB, VC++, Delphi, T-SQL, VJ, CFML ) residing on multiple shared and
dedicated servers at the same time, without ever attaching to process. Remote code can
pass messages and dialogs directly to your local machine via Team Remote Debugger
component, and developers can then debug their respective code independently of one
another no matter if the code units reside on the same servers or on different servers or
on any combination thereof.
Datatect
Test data generator from Banner Software generates data to a flat file or ODBC-compliant
database; includes capabilities such as scripting support that allows user to write
VBScripts that modify data to create XML output, data generation interface to Segue
SilkTest, capability to read in existing database table structures to aid in data generation,
wide variety of data types and capabilities for custom data types. For Windows.
Hypertrak
Suite of software protocol analyzers from Triometric accurately calculates end-to-end
download speeds for each transaction, not just samples; produces a range of configurable
reports that breaks down info into network and server speeds, errors, comparison to
SLA's, performance for each server, client, URL, time period, etc. Runs on Solaris or Linux.
WebBug
Debugging tool from Aman Software for monitoring HTTP protocol sends and receives;
handles HTTP 0.9/1.0/1.1; allows for entry of custom headers. Freeware.
WebMetrics
Page 49 of 62
Web usability testing and evaluation tool suite from U.S. Govt. NIST. Source code
available. For UNIX, Windows.
MRTG
Multi Router Traffic Grapher - free tool utilizing SNMP to monitoring traffic loads on
network links; generates reports as web pages with GIF graphics on inbound and
outbound traffic. For UNIX, Windows.
Page 50 of 62
6. Testing FAQ
1. What is 'Software Quality Assurance'?
2. What is 'Software Testing'?
3. What are some recent major computer system failures caused by software
bugs?
4. Why is it often hard for management to get serious about quality assurance?
5. Why does software have bugs?
6. How can new Software QA processes be introduced in an existing organization?
7. What is verification and Validation?
8. What is a 'walkthrough'?
9. What's an 'inspection'?
10. What kinds of testing should be considered?
11. What are 5 common problems in the software development process?
12. What is software 'quality'?
13. What is 'good code'?
14. What is 'good design'?Top of the Document
15. What is SEI? CMM? CMMI? ISO? Will it help?
16. What is the 'software life cycle'?
17. Will automated testing tools make testing easier?
1. What is 'Software Quality Assurance'?
Software QA involves the entire software development Process - monitoring and

improving the process, making sure that any agreed-upon standards and
procedures are followed, and ensuring that problems are found and dealt with. It is
oriented to 'prevention'. (See the Books section for a list of useful books on
Software Quality Assurance.)
2. What is 'Software Testing'?
Testing involves operation of a system or application under controlled conditions

and evaluating the results (eg, 'if the user is in interface A of the application while
using hardware B, and does C, then D should happen'). The controlled conditions
should include both normal and abnormal conditions. Testing should intentionally
attempt to make things go wrong to determine if things happen when they
shouldn't or things don't happen when they should. It is oriented to 'detection'.
Organizations vary considerably in how they assign responsibility for QA and

testing. Sometimes they're the combined responsibility of one group or individual.
Also common are project teams that include a mix of testers and developers who
work closely together, with overall QA processes monitored by project managers. It
will depend on what best fits an organization's size and business structure.
3. What are some recent major computer system failures caused by

software bugs?
Page 51 of 62
• Media reports in January of 2005 detailed severe problems with a $170

million high-profile U.S. government IT systems project. Software testing
was one of the five major problem areas according to a report of the
commission reviewing the project. Studies were under way to determine
which, if any, portions of the project could be salvaged.
• In July 2004 newspapers reported that a new government welfare
management system in Canada costing several hundred million dollars was
unable to handle a simple benefits rate increase after being put into live
operation. Reportedly the original contract allowed for only 6 weeks of
acceptance testing and the system was never tested for its ability to handle
a rate increase.
• Millions of bank accounts were impacted by errors due to installation of
inadequately tested software code in the transaction processing system of a
major North American bank, according to mid-2004 news reports. Articles
about the incident stated that it took two weeks to fix all the resulting
errors, that additional problems resulted when the incident drew a large
number of e-mail phishing attacks against the bank's customers, and that
the total cost of the incident could exceed $100 million.
• A bug in site management software utilized by companies with a significant
percentage of worldwide web traffic was reported in May of 2004. The bug
resulted in performance problems for many of the sites simultaneously and
required disabling of the software until the bug was fixed.
• According to news reports in April of 2004, a software bug was determined
to be a major contributor to the 2003 Northeast blackout, the worst power
system failure in North American history. The failure involved loss of
electrical power to 50 million customers, forced shutdown of 100 power
plants, and economic losses estimated at $6 billion. The bug was reportedly
in one utility company's vendor-supplied power monitoring and management
system, which was unable to correctly handle and report on an unusual
confluence of initially localized events. The error was found and corrected
after examining millions of lines of code.
• In early 2004, news reports revealed the intentional use of a software bug
as a counter-espionage tool. According to the report, in the early 1980's one
nation surreptitiously allowed a hostile nation's espionage service to steal a
version of sophisticated industrial software that had intentionally-added
flaws. This eventually resulted in major industrial disruption in the country
that used the stolen flawed software.
• A major U.S. retailer was reportedly hit with a large government fine in
October of 2003 due to web site errors that enabled customers to view one
anothers' online orders.
• News stories in the fall of 2003 stated that a manufacturing company
recalled all their transportation products in order to fix a software problem
causing instability in certain circumstances. The company found and
reported the bug itself and initiated the recall procedure in which a software
upgrade fixed the problems.
• In January of 2001 newspapers reported that a major European railroad was
hit by the aftereffects of the Y2K bug. The company found that many of their
newer trains would not run due to their inability to recognize the date
'31/12/2000'; the trains were started by altering the control system's date
settings.
• News reports in September of 2000 told of a software vendor settling a
lawsuit with a large mortgage lender; the vendor had reportedly delivered
Page 52 of 62
an online mortgage processing system that did not meet specifications, was
delivered late, and didn't work.
• In early 2000, major problems were reported with a new computer system
in a large suburban U.S. public school district with 100,000+ students;
problems included 10,000 erroneous report cards and students left stranded
by failed class registration systems; the district's CIO was fired. The school
district decided to reinstate it's original 25-year old system for at least a
year until the bugs were worked out of the new system by the software
vendors.
• In October of 1999 the $125 million NASA Mars Climate Orbiter spacecraft
was believed to be lost in space due to a simple data conversion error. It
was determined that spacecraft software used certain data in English units
that should have been in metric units. Among other tasks, the orbiter was to
serve as a communications relay for the Mars Polar Lander mission, which
failed for unknown reasons in December 1999. Several investigating panels
were convened to determine the process failures that allowed the error to go
undetected.
• Bugs in software supporting a large commercial high-speed data network
affected 70,000 business customers over a period of 8 days in August of
1999. Among those affected was the electronic trading system of the largest
U.S. futures exchange, which was shut down for most of a week as a result
of the outages.
• January 1998 news reports told of software problems at a major U.S.
telecommunications company that resulted in no charges for long distance
calls for a month for 400,000 customers. The problem went undetected until
customers called up with questions about their bills.
4. Why is it often hard for management to get serious about quality

assurance?
• Solving problems is a high-visibility process; preventing problems is low-

visibility. This is illustrated by an old parable: In ancient China there was a
family of healers, one of whom was known throughout the land and
employed as a physician to a great lord.
5. Why does software have bugs?
• Miscommunication or no communication - as to specifics of what an

application should or shouldn't do (the application's requirements).
• Software complexity - the complexity of current software applications can be
difficult to comprehend for anyone without experience in modern-day
software development. Multi-tiered applications, client-server and
distributed applications, data communications, enormous relational
databases, and sheer size of applications have all contributed to the
exponential growth in software/system complexity.
• Programming errors - programmers, like anyone else, can make mistakes.
• Changing requirements (whether documented or undocumented) - the end-
user may not understand the effects of changes, or may understand and
request them anyway - redesign, rescheduling of engineers, effects on other
projects, work already completed that may have to be redone or thrown out,
hardware requirements that may be affected, etc. If there are many minor
changes or any major changes, known and unknown dependencies among
Page 53 of 62
parts of the project are likely to interact and cause problems, and the
complexity of coordinating changes may result in errors. Enthusiasm of
engineering staff may be affected. In some fast-changing business
environments, continuously modified requirements may be a fact of life. In
this case, management must understand the resulting risks, and QA and
test engineers must adapt and plan for continuous extensive testing to keep
the inevitable bugs from running out of control - see 'What can be done if
requirements are changing continuously?' in Part 2 of the FAQ. Also see
information about 'agile' approaches such as XP, also in Part 2 of the FAQ.
• Time pressures - scheduling of software projects is difficult at best, often
requiring a lot of guesswork. When deadlines loom and the crunch comes,
mistakes will be made.
• egos - people prefer to say things like:
o 'no problem'
o 'piece of cake'
o 'I can whip that out in a few hours'
o 'it should be easy to update that old code'
• instead of:
o 'that adds a lot of complexity and we could end up making a lot of
mistakes'
o 'we have no idea if we can do that; we'll wing it'
o 'I can't estimate how long it will take, until I take a close look at it'
o 'we can't figure out what that old spaghetti code did in the first place'
• If there are too many unrealistic 'no problem's', the result is bugs.
• Poorly documented code - it's tough to maintain and modify code that is
badly written or poorly documented; the result is bugs. In many
organizations management provides no incentive for programmers to
document their code or write clear, understandable, maintainable code. In
fact, it's usually the opposite: they get points mostly for quickly turning out
code, and there's job security if nobody else can understand it ('if it was
hard to write, it should be hard to read').
• Software development tools - visual tools, class libraries, compilers,
scripting tools, etc. often introduce their own bugs or are poorly
documented, resulting in added bugs.
6. How can new Software QA processes be introduced in an existing

organization?
• A lot depends on the size of the organization and the risks involved. For
large organizations with high-risk (in terms of lives or property) projects,
serious management buy-in is required and a formalized QA process is
necessary.
• Where the risk is lower, management and organizational buy-in and QA
implementation may be a slower, step-at-a-time process. QA processes
should be balanced with productivity so as to keep bureaucracy from getting
out of hand.
• For small groups or projects, a more ad-hoc process may be appropriate,
depending on the type of customers and projects. A lot will depend on team
leads or managers, feedback to developers, and ensuring adequate
communications among customers, managers, developers, and testers.
Page 54 of 62
• The most value for effort will often be in (a) requirements management
processes, with a goal of clear, complete, testable requirement specifications
embodied in requirements or design documentation, or in 'agile'-type
environments extensive continuous coordination with end-users, (b) design
inspections and code inspections, and (c) post-mortems/retrospectives.
7. What is verification and validation?
• Verification typically involves reviews and meetings to evaluate documents,

plans, code, requirements, and specifications. This can be done with
checklists, issues lists, walkthroughs, and inspection meetings. Validation
typically involves actual testing and takes place after verifications are
completed. The term 'IV & V' refers to Independent Verification and
Validation.
8. What is a 'walkthrough'?
• A 'walkthrough' is an informal meeting for evaluation or informational

purposes. Little or no preparation is usually required.
9. What's an 'inspection'?
• An inspection is more formalized than a 'walkthrough', typically with 3-8

people including a moderator, reader, and a recorder to take notes. The
subject of the inspection is typically a document such as a requirements
spec or a test plan, and the purpose is to find problems and see what's
missing, not to fix anything. Attendees should prepare for this type of
meeting by reading thru the document; most problems will be found during
this preparation. The result of the inspection meeting should be a written
report.
10. What kinds of testing should be considered?
• Black box testing - not based on any knowledge of internal design or code.
Tests are based on requirements and functionality.
• White box testing - based on knowledge of the internal logic of an
application's code. Tests are based on coverage of code statements,
branches, paths, conditions.
• Unit testing - the most 'micro' scale of testing; to test particular functions or
code modules. Typically done by the programmer and not by testers, as it
requires detailed knowledge of the internal program design and code. Not
always easily done unless the application has a well-designed architecture
with tight code; may require developing test driver modules or test
harnesses.
• Incremental integration testing - continuous testing of an application as new
functionality is added; requires that various aspects of an application's
functionality be independent enough to work separately before all parts of
the program are completed, or that test drivers be developed as needed;
done by programmers or by testers.
• Integration testing - testing of combined parts of an application to
determine if they function together correctly. The 'parts' can be code
Page 55 of 62
modules, individual applications, client and server applications on a network,

etc. This type of testing is especially relevant to client/server and distributed
systems.
• Functional testing - black-box type testing geared to functional requirements
of an application; this type of testing should be done by testers. This doesn't
mean that the programmers shouldn't check that their code works before
releasing it (which of course applies to any stage of testing.)
• System testing - black-box type testing that is based on overall
requirements specifications; covers all combined parts of a system.
• End-to-end testing - similar to system testing; the 'macro' end of the test
scale; involves testing of a complete application environment in a situation
that mimics real-world use, such as interacting with a database, using
network communications, or interacting with other hardware, applications,
or systems if appropriate.
• Sanity testing or smoke testing - typically an initial testing effort to
determine if a new software version is performing well enough to accept it
for a major testing effort. For example, if the new software is crashing
systems every 5 minutes, bogging down systems to a crawl, or corrupting
databases, the software may not be in a 'sane' enough condition to warrant
further testing in its current state.
• Regression testing - re-testing after fixes or modifications of the software or
its environment. It can be difficult to determine how much re-testing is
needed, especially near the end of the development cycle. Automated
testing tools can be especially useful for this type of testing.
• Acceptance testing - final testing based on specifications of the end-user or
customer, or based on use by end-users/customers over some limited period
of time.
• Load testing - testing an application under heavy loads, such as testing of a
web site under a range of loads to determine at what point the system's
response time degrades or fails.
• Stress testing - term often used interchangeably with 'load' and
'performance' testing. Also used to describe such tests as system functional
testing while under unusually heavy loads, heavy repetition of certain
actions or inputs, input of large numerical values, large complex queries to a
database system, etc.
• Performance testing - term often used interchangeably with 'stress' and
'load' testing. Ideally 'performance' testing (and any other 'type' of testing)
is defined in requirements documentation or QA or Test Plans.
• Usability testing - testing for 'user-friendliness'. Clearly this is subjective,
and will depend on the targeted end-user or customer. User interviews,
surveys, video recording of user sessions, and other techniques can be
used. Programmers and testers are usually not appropriate as usability
testers.
• Install/uninstall testing - testing of full, partial, or upgrade install/uninstall
processes.
• Recovery testing - testing how well a system recovers from crashes,
hardware failures, or other catastrophic problems.
• Failover testing - typically used interchangeably with 'recovery testing'
• Security testing - testing how well the system protects against unauthorized
internal or external access, willful damage, etc; may require sophisticated
testing techniques.
Page 56 of 62
• Compatability testing - testing how well software performs in a particular

hardware/software/operating system/network/etc. environment.
• Exploratory testing - often taken to mean a creative, informal software test
that is not based on formal test plans or test cases; testers may be learning
the software as they test it.
• Ad-hoc testing - similar to exploratory testing, but often taken to mean that
the testers have significant understanding of the software before testing it.
• Context-driven testing - testing driven by an understanding of the
environment, culture, and intended use of software. For example, the
testing approach for life-critical medical equipment software would be
completely different than that for a low-cost computer game.
• User acceptance testing - determining if software is satisfactory to an end-
user or customer.
• Comparison testing - comparing software weaknesses and strengths to
competing products.
• Alpha testing - testing of an application when development is nearing
completion; minor design changes may still be made as a result of such
testing. Typically done by end-users or others, not by programmers or
testers.
• Beta testing - testing when development and testing are essentially
completed and final bugs and problems need to be found before final
release. Typically done by end-users or others, not by programmers or
testers.
• Mutation testing - a method for determining if a set of test data or test
cases is useful, by deliberately introducing various code changes ('bugs')
and retesting with the original test data/cases to determine if the 'bugs' are
detected. Proper implementation requires large computational resources.
11. What are 5 common problems in the software development process?
• Solid requirements - clear, complete, detailed, cohesive, attainable, testable

requirements that are agreed to by all players. Use prototypes to help nail
down requirements. In 'agile'-type environments, continuous coordination
with customers/end-users is necessary.
• Realistic schedules - allow adequate time for planning, design, testing, bug
fixing, re-testing, changes, and documentation; personnel should be able to
complete the project without burning out.
• Adequate testing - start testing early on, re-test after fixes or changes, plan
for adequate time for testing and bug-fixing. 'Early' testing ideally includes
unit testing by developers and built-in testing and diagnostic capabilities.
• Stick to initial requirements as much as possible - be prepared to defend
against excessive changes and additions once development has begun, and
be prepared to explain consequences. If changes are necessary, they should
be adequately reflected in related schedule changes. If possible, work
closely with customers/end-users to manage expectations. This will provide
them a higher comfort level with their requirements decisions and minimize
excessive changes later on.
• Communication - require walkthroughs and inspections when appropriate;
make extensive use of group communication tools - e-mail, groupware,
networked bug-tracking tools and change management tools, intranet
capabilities, etc.; insure that information/documentation is available and up-
Page 57 of 62
to-date - preferably electronic, not paper; promote teamwork and

cooperation; use protoypes if possible to clarify customers' expectations.
12. What is software 'quality'?
• Quality software is reasonably bug-free, delivered on time and within

budget, meets requirements and/or expectations, and is maintainable.
However, quality is obviously a subjective term. It will depend on who the
'customer' is and their overall influence in the scheme of things. A wide-
angle view of the 'customers' of a software development project might
include end-users, customer acceptance testers, customer contract officers,
customer management, the development organization's.
• Management/accountants/testers/salespeople, future software maintenance
engineers, stockholders, magazine columnists, etc. Each type of 'customer'
will have their own slant on 'quality' - the accounting department might
define quality in terms of profits while an end-user might define quality as
user-friendly and bug-free.
13. What is 'good code'?
• 'Good code' is code that works, is bug free, and is readable and
maintainable. Some organizations have coding 'standards' that all
developers are supposed to adhere to, but everyone has different ideas
about what's best, or what is too many or too few rules. There are also
various theories and metrics, such as McCabe Complexity metrics. It should
be kept in mind that excessive use of standards and rules can stifle
productivity and creativity. 'Peer reviews', 'buddy checks' code analysis
tools, etc. can be used to check for problems and enforce standards. For C
and C++ coding, here are some typical ideas to consider in setting
rules/standards; these may or may not apply to a particular situation:
• Minimize or eliminate use of global variables.
• Use descriptive function and method names - use both upper and lower
case, avoid abbreviations, use as many characters as necessary to be
adequately descriptive (use of more than 20 characters is not out of line);
be consistent in naming conventions.
• Use descriptive variable names - use both upper and lower case, avoid
abbreviations, use as many characters as necessary to be adequately
descriptive (use of more than 20 characters is not out of line); be consistent
in naming conventions.
• Function and method sizes should be minimized; less than 100 lines of code
is good, less than 50 lines is preferable.
• Function descriptions should be clearly spelled out in comments preceding a
function's code.
• Organize code for readability.
• Use whitespace generously - vertically and horizontally.
• Each line of code should contain 70 characters max.
• One code statement per line.
• Coding style should be consistent throught a program (eg, use of brackets,
indentations, naming conventions, etc.)
Page 58 of 62
• In adding comments, err on the side of too many rather than too few
comments; a common rule of thumb is that there should be at least as
many lines of comments (including header blocks) as lines of code.
• No matter how small, an application should include documentaion of the
overall program function and flow (even a few paragraphs is better than
nothing); or if possible a separate flow chart and detailed program
documentation.
• Make extensive use of error handling procedures and status and error
logging.
• For C++, to minimize complexity and increase maintainability, avoid too
many levels of inheritance in class heirarchies (relative to the size and
complexity of the application). Minimize use of multiple inheritance, and
minimize use of operator overloading (note that the Java programming
language eliminates multiple inheritance and operator overloading.)
• For C++, keep class methods small, less than 50 lines of code per method is
preferable.
• For C++, make liberal use of exception handlers.
14. What is 'good design'?
• 'Design' could refer to many things, but often refers to 'functional design' or
'internal design'. Good internal design is indicated by software code whose
overall structure is clear, understandable, easily modifiable, and
maintainable; is robust with sufficient error-handling and status logging
capability; and works correctly when implemented. Good functional design is
indicated by an application whose functionality can be traced back to
customer and end-user requirements.For programs that have a user
interface, it's often a good idea to assume that the end user will have little
computer knowledge and may not read a user manual or even the on-line
help; some common rules-of-thumb include:
• The program should act in a way that least surprises the user
• It should always be evident to the user what can be done next and how to
exit
• The program shouldn't let the users do something stupid without warning
them.
15. What is SEI? CMM? CMMI? ISO? IEEE? ANSI? Will it help?
• SEI = 'Software Engineering Institute' at Carnegie-Mellon University;

initiated by the U.S. Defense Department to help improve software
development processes.
• CMM = 'Capability Maturity Model', now called the CMMI ('Capability Maturity
Model Integration'), developed by the SEI. It's a model of 5 levels of process
'maturity' that determine effectiveness in delivering quality software. It is
geared to large organizations such as large U.S. Defense Department
contractors. However, many of the QA processes involved are appropriate to
any organization, and if reasonably applied can be helpful. Organizations can
receive CMMI ratings by undergoing assessments by qualified auditors.
• Level 1 - characterized by chaos, periodic panics, and heroic efforts required
by individuals to successfully complete projects. Few if any processes in
place; successes may not be repeatable.
Page 59 of 62
• Level 2 - software project tracking, requirements management, realistic

planning, and configuration management processes are in place; successful
practices can be repeated.
• Level 3 - standard software development and maintenance processes are
integrated throughout an organization; a Software Engineering Process
Group is is in place to oversee software processes, and training programs
are used to ensure understanding and compliance.
• Level 4 - metrics are used to track productivity, processes, and products.
Project performance is predictable, and quality is consistently high.
• Level 5 - the focus is on continuous process improvement. The impact of
new processes and technologies can be predicted and effectively
implemented when required.
• Perspective on CMM ratings: During 1997-2001, 1018 organizations were
assessed. Of those, 27% were rated at Level 1, 39% at 2, 23% at 3, 6% at
4, and 5% at 5. (For ratings during the period 1992-96, 62% were at Level
1, 23% at 2, 13% at 3, 2% at 4, and 0.4% at 5.) The median size of
organizations was 100 software engineering/maintenance personnel; 32% of
organizations were U.S. federal contractors or agencies. For those rated at
Level 1, the most problematical key process area was in Software Quality
Assurance.
• ISO = 'International Organization for Standardization' - The ISO 9001:2000
standard (which replaces the previous standard of 1994) concerns quality
systems that are assessed by outside auditors, and it applies to many kinds
of production and manufacturing organizations, not just software. It covers
documentation, design, development, production, testing, installation,
servicing, and other processes. The full set of standards consists of:
(a)Q9001-2000 - Quality Management Systems: Requirements; (b)Q9000-
2000 - Quality Management Systems: Fundamentals and Vocabulary;
(c)Q9004-2000 - Quality Management Systems: Guidelines for Performance
Improvements. To be ISO 9001 certified, a third-party auditor assesses an
organization, and certification is typically good for about 3 years, after which
a complete reassessment is required. Note that ISO certification does not
necessarily indicate quality products - it indicates only that documented
processes are followed. Also see http://www.iso.ch/ for the latest
information. In the U.S. the standards can be purchased via the ASQ web
site at http://e-standards.asq.org/
• IEEE = 'Institute of Electrical and Electronics Engineers' - among other
things, creates standards such as 'IEEE Standard for Software Test
Documentation' (IEEE/ANSI Standard 829), 'IEEE Standard of Software Unit
Testing (IEEE/ANSI Standard 1008), 'IEEE Standard for Software Quality
Assurance Plans' (IEEE/ANSI Standard 730), and others.
• ANSI = 'American National Standards Institute', the primary industrial
standards body in the U.S.; publishes some software-related standards in
conjunction with the IEEE and ASQ (American Society for Quality).
• Other software development/IT management process assessment methods
besides CMMI and ISO 9000 include SPICE, Trillium, TickIT, Bootstrap, ITIL,
MOF, and CobiT.
16. What is the 'software life cycle'?
• The life cycle begins when an application is first conceived and ends when it
is no longer in use. It includes aspects such as initial concept, requirements
Page 60 of 62
analysis, functional design, internal design, documentation planning, test

planning, coding, document preparation, integration, testing, maintenance,
updates, retesting, phase-out, and other aspects.
17. Will automated testing tools make testing easier?
• Possibly For small projects, the time needed to learn and implement them
may not be worth it. For larger projects, or on-going long-term projects they
can be valuable.
• A common type of automated tool is the 'record/playback' type. For
example, a tester could click through all combinations of menu choices,
dialog box choices, buttons, etc. in an application GUI and have them
'recorded' and the results logged by a tool. The 'recording' is typically in the
form of text based on a scripting language that is interpretable by the
testing tool. If new buttons are added, or some underlying code in the
application is changed, etc. the application might then be retested by just
'playing back' the 'recorded' actions, and comparing the logging results to
check effects of the changes. The problem with such tools is that if there are
continual changes to the system being tested, the 'recordings' may have to
be changed so much that it becomes very time-consuming to continuously
update the scripts. Additionally, interpretation and analysis of results
(screens, data, logs, etc.) can be a difficult task. Note that there are
record/playback tools for text-based interfaces also, and for all types of
platforms.
• Another common type of approach for automation of functional testing is
'data-driven' or 'keyword-driven' automated testing, in which the test
drivers are separated from the data and/or actions utilized in testing (an
'action' would be something like 'enter a value in a text box'). Test drivers
can be in the form of automated test tools or custom-written testing
software. The data and actions can be more easily maintained - such as via
a spreadsheet - since they are separate from the test drivers. The test
drivers 'read' the data/action information to perform specified tests. This
approach can enable more efficient control, development, documentation,
and maintenance of automated tests/test cases.
• Other automated tools can include:
• Code analyzers - monitor code complexity, adherence to standards, etc.
• Coverage analyzers - these tools check which parts of the code have been
exercised by a test, and may be oriented to code statement coverage,
condition coverage, path coverage, etc.
• Memory analyzers - such as bounds-checkers and leak detectors.
• Load/performance test tools - for testing client/server and web applications
under various load levels.
• Web test tools - to check that links are valid, HTML code usage is correct,
client-side and server-side programs work, a web site's interactions are
secure.
• Other tools - for test case management, documentation management, bug
reporting, and configuration management.
Page 61 of 62
This document is for internal purposes only. All contents of this document have been
collated from various resources, the primary being: http://www.vyomworld.com/testing/.
The same has then been reviewed and restructured for better reading and understanding.
Page 62 of 62

Software Testing For Dummies (ADITI Edition)

Uploaded by

Copyright:

Available Formats

Software Testing For Dummies (ADITI Edition)

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Software Testing For Dummies (ADITI Edition)

Uploaded by

Copyright:

Available Formats

Aditi Technologies Basic Concepts of Software Testing

3.6. What makes a good Software QA engineer? .............................................19

2.2.Black Box Testing

2.3.4.Documents the code

2.3.5.Separation of Interface from Implementation

Integration Testing is the phase of software testing in which individual software

The overall idea is the "building block" approach in which verified

provide) instrumentation that runs on the server and reports transaction

Performance testing almost invariably identifies that it is parts of the software

It is always helpful to have a statement of the likely peak numbers of users

• Analysis of the types of interaction that should be emulated and the

It involves testing beyond normal operational capacity, often to a breaking point,

2.7.1.Security Testing Techniques

Usability testing focuses on a particular object or a small set of objects, whereas

If usability testing uncovers difficulties, such as people having difficulty

Setting up a usability test involves carefully creating a scenario, or realistic

In the pharmaceutical field, it refers to a period of time during which a multi-dose

In distributed systems, particularly where software is to be released into an

It can then be handed over to testing staff for additional inspection in an

Alpha testing and Beta testing are sub-categories of System testing.

According to the IEEE Standard Computer Dictionary, Regression testing is testing

Alpha testing and Beta testing are sub-categories of Regression testing.

Compatibility testing service aims at locating application problems by running

2.18.Test Cases, Suits, Scripts and Scenario

A scenario test is a test based on a hypothetical story used to help a person

They can be as simple as a diagram for a testing environment or they could be a

Defect tracking is important in software engineering as complex software systems

Usually formal verification is carried out algorithmically. The main approaches to

2.20.1.Validation and Verification

Fuzz testing is also used as a gross measurement of a large software system's

Fuzz testing is thought to enhance software security and software safety

However, fuzz testing is not a substitute for exhaustive testing or formal

2.21.2.Fuzz testing methods

Modern software has several different types of inputs:

• Character driven inputs are from files or data streams.

There are at least two different forms of fuzz testing:

By using all of these techniques in combination, fuzz-generated randomness

Fuzz testing may use tools to simulate all of these domains.

One common problem with a character driven program is a buffer overrun,

Database fuzz is controversial, because input and comparison constraints

A common problem with fuzz databases is buffer overrun. A common data

• Why bother about Software Engineering techniques, I will go to terminal

3.4.Software Quality Assurance:

• Software Quality Assurance involves reviewing and auditing the software

3.5.Software Life Cycle Models:

3.6.What makes a good Software QA engineer?

3.8.Test Life Cycle

These principles & techniques can be applied to any type of testing.

A Test Plan is developed prior to the implementation of a project to provide a

A Test Specification is produced as the first step in implementing a Test Plan,

• Obtain requirements, functional design, and internal design specifications

• Evaluate and report results

What's a 'test case'?

• A test case is a document that describes an input, action, or event and an

What should be done after a bug is found?

• The bug needs to be communicated and assigned to developers that can

Why does software have bugs?

• Miscommunication or no communication - as to specifics of what an

5. Testing Tools and Software