Sagar Software Testing and Validation Lab
Sagar Software Testing and Validation Lab
On
Software Testing and Validation Lab
List of Experiments
1. (a) Write a program that calculates the area and perimeter of the circle. And find the
Coverage & Test Cases of that program using JaButi Tool.
(b) Write a program which read the first name and last name from console and matching with
expected result by using JaBuTi.
(c) Write a program that takes three double numbers from the java console representing ,
respectively, the three coefficients a ,b, and c of a quadratic equation.
(d) Write a program that reads commercial website URL from a url from file. you should
expect that the URL starts with www and ends with .com. retrieve the name of the site and
output it. For instance, if the user inputs www.yahoo.com, you should output yahoo. After
that find the test cases and coverage using JaBuTi.
(e) Write a program for a calculator and find the test case and coverage and Def-use-graph.
(f) Write a program that reads two words representing passwords from the java console and
outputs the number of character in the smaller of the two. For example, if the words
are open and sesame, then the output should be 4, the length of the shorter word, open. And
test this program using JaBuTi.
3. Calculate the mutation score of programs given in 1(a) to 1 (f) using jumble Tool.
4. Calculate the coverage analysis of programs given in 1 (a) to 1 (f) using Eclemma Free open
source Tool.
5. Generate Test sequences and validate using Selenium tool for given websites below:
lOMoARcPSD|25
lOMoARcPS
EXPERIMENT 1(a)
Name of Experiment: Write a program that calculates the area and perimeter of the circle.And
find the Coverage & Test Cases of that program using JaButi Tool.
Aim :
Program Code:AreaOfCircle.java
import java.util.Scanner;
import java.lang.Math;
public class AreaOfCircle
{
public static void main(String[] args)
{
int option;
double radius, circumference, diameter, area;
//object of the Scanner class
Scanner sc=new Scanner (System.in);
//options available
System.out.println("1. If the radius is known");
System.out.println("2. If the diameter is known");
System.out.println("3. If the circumference is known");
System.out.print("Enter your choice: ");
//taking an option as input from the user
option=sc.nextInt();
switch(option)
{
//Case statements
case 1:
}
}
// BYTE CODE
Now assign this ByteCode to JaBUTi Tool for Coverage Testing by following steps
EXPERIMENT : 1(b)
Name of Experiment: Write a program which read the first name and last name from console
and matching with expected result by using JaBuTi.
/**
* An example program to read a Name from console input in Java */
Public class Example {
23
EXPERIMENT : 1(c)
Name of Experiment: Write a program which read the first name and last name from console
and matching with expected result by using JaBuTi.
EXPERIMENT : 1(D)
Name of Experiment: Write a program that reads commercial website URL from a url from
file. You should expect that the URL starts with www and ends with .com. Retrieve the name
of the site and output it. For instance, if the user inputs www.yahoo.com, you should output
yahoo. After that find the test cases and coverage using JaButi.
Program Code:
package mypkg;
import java.io.*;
import javax.servlet.*;
import javax.servlet.http.*;
@Override
response.setContentType("text/html;charset=UTF-8");
// Allocate a output writer to write the response message into the network
socket PrintWriter out = response.getWriter()
// Write the response message, in an HTML page
try {
out.println("<!DOCTYPE html>");
out.println("<html><head>");
out.println("<meta http-equiv='Content-Type' content='text/html; charset=UTF-8'>");
information
out.println("</body>");
out.println("</html>");
} finally {
Name of Experiment: Write a program for a calculator and find the test case and coverage and Def-use-
graph.
Program Code:
import java.util.Scanner;
class Main {
char operator;
Double number1, number2, result;
= input.nextDouble();
case '-':
case '/':
result = number1 / number2;
System.out.println(number1 + " / " + number2 + " = " + result);
break;
default:
System.out.println("Invalid operator!");
break;
input.close();
Name of Experiment: Write a program that reads two words representing passwords from the
java console and outputs the number of character in the smaller of the two. For example, if the
words are open and sesame, then the output should be 4, the length of the shorter word, open.
And test this program using JaButi.
Program Code:
import java.util.Scanner;
if (is_Valid_Password(s)) {
System.out.println("Password is valid: " + s);
} else {
System.out.println("Not a valid password: " + s);
}
int charCount = 0;
int numCount = 0;
for (int i = 0; i < password.length(); i++) {
char ch = password.charAt(i);
if (is_Numeric(ch)) numCount++;
else if (is_Letter(ch)) charCount++;
else return false;
}
EXPERIMENT : 2
JMeter Introduction
JMeter is a software that can perform load test, performance-oriented business (functional) test,
regression test, etc., on different protocols or technologies. Stefano Mazzocchi of the Apache
Software Foundation was the original developer of JMeter. He wrote it primarily to test the
performance of Apache JServ (now called as Apache Tomcat project). Apache later redesigned
JMeter to enhance the GUI and to add functional testing capabilities.
JMeter is a Java desktop application with a graphical interface that uses the Swing
graphical API. It can therefore run on any environment / workstation that accepts a Java virtual
machine, for example − Windows, Linux, Mac, etc.
The protocols supported by JMeter are −
• Web − HTTP, HTTPS sites 'web 1.0' web 2.0 (ajax, flex and flex-ws-amf)
• Web Services − SOAP / XML-RPC
• Database via JDBC drivers
• Directory − LDAP
• Messaging Oriented service via JMS
• Service − POP3, IMAP, SMTP
• FTP Service
JMeter Features
Following are some of the features of JMeter −
• Being an open source software, it is freely available.
• It has a simple and intuitive GUI.
• JMeter can conduct load and performance test for many different server types −
Web - HTTP, HTTPS, SOAP, Database via JDBC, LDAP, JMS, Mail - POP3, etc.
• It is a platform-independent tool. On Linux/Unix, JMeter can be invoked by clicking on
JMeter shell script. On Windows, it can be invoked by starting the jmeter.bat file.
• It has full Swing and lightweight component support (precompiled JAR uses packages
javax.swing.* ).
• JMeter store its test plans in XML format. This means you can generate a test plan using
a text editor.
• Its full multi-threading framework allows concurrent sampling by many threads and
simultaneous sampling of different functions by separate thread groups.
• It is highly extensible.
• It can also be used to perform automated and functional testing of the applications.
JMeter Working Techniques
JMeter simulates a group of users sending requests to a target server, and returns statistics that
show the performance/functionality of the target server/application via tables, graphs, etc.
Take a look at the following figure that depicts how JMeter works −
JMeter is a framework for Java, so the very first requirement is to have JDK installed in your
machine.
System Requirement
OS Task Command
If you have Java installed in your system, you would get an appropriate output based on the OS
you are working on.
OS Output
If you do not have Java installed, install the Java Software Development Kit (SDK)
from www.oracle.com/technetwork/java/javase/downloads/index.html. We are assuming Java
1.7.0_25 as the installed version for this tutorial.
Step 2: Set Java Environment
Set the JAVA_HOME environment variable to point to the base directory location, where
Java is installed on your machine. For example −
OS Output
Windows SettheenvironmentvariableJAVA_HOMEtoC:\Program
Files\Java\jdk1.7.0_25
OS Output
You can rename the parent directory (i.e. apache-jmeter-2.9) if you want, but do not change any
of the sub-directory names.
Step 4: Run JMeter
After downloading JMeter, go to the bin directory. In this case, it is /home/manisha/apache-
jmeter-2.9/bin.
After a short pause, the JMeter GUI should appear, which is a Swing application, as seen in the
following screenshot −
This is the main page and the default page of the tool.
Apache JMeter - User's Manual - https://jmeter.apache.org/usermanual/index.html
EXPERIMENT : 3
Name of Experiment: Calculate the mutation score of programs given in 1(a) to 1 (f) using
jumble Tool.
Introduction to Jumble :
The primary entry point to Jumble is a single Java class that takes as parameters a class to be
mutation tested and one or more JUnit test classes. The output is a simple textual summary of
running the tests. This is in the style of JUnit test output and includes an overall score of how
many mutations were successfully caught as well as details about those mutations that are not.
This includes the source line and the mutation performed. Variants of the output include a
version compatible with Java development in emacs where it is possible to click on the line and
go to the source line containing the mutation point. A plugin for the Eclipse IDE [10] has been
provided. Currently it permits only jumbling of a single class. When running the mutation tests a
separate JVM [11] is used to prevent runaway or system-exiting code disturbing the testing
process. It is given a number which specifies the mutation to start at and it continues until all
mutations have been tested or some catastrophic failure occurs. It has been very important to
separate the test execution in this way for reasons discussed below. If a failure occurs in one
mutation then the child JVM can be restarted and testing continues with the next mutation.
Before running the mutation tests, a full run is done of the unit tests both to ensure that they all
pass, to ensure that there are no environmental problems when running the tests in this way and
to collect timing information to later detect mutations that lead to infinite loops. At Reel Two,
Jumble is integrated with an internal continual integration system, on several source code
repositories. Every fifteen minutes this checks out all the source code, clean compiles it and then
runs all the unit tests for packages that have been modified. It also places all modified classes on
a queue to be mutation tested. After the unit testing has been done, Jumble is used to test classes
until the fifteen minute time limit is exceeded. Overnight more time is dedicated to mutation
testing so that any tests that have been accumulated during the day can be cleared. The results of
the Jumble tests for a project are presented in a web interface. These give results for individual
classes and accumulate results over the package hierarchy. The web interface permits a manual
override of the test queuing so that classes which have not been modified can be retested. This is
sometimes necessary when the Jumble code itself has been modified or when a class needs to be
retested because of changes in other classes that it interacts with.
In summary we typically use three different ways of running Jumble:
• directly from the command line where the class com.reeltwo.jumble.Jumble is provided
with the name of the class to be tested and the class(es) which test it (among many other
options). Executing directly from the distributed jumble.jar will achieve the same effect.
• from the Eclipse IDE where there is a menu item for a class which runs Jumble on that
class (it is possible to configure other options for such runs)
• as part of a web based system where all classes are tested incrementally as they are
committed and results accumulated in a set of webpages.
Jumble System
In this section we discuss individual parts of the Jumble system and the issues that arose in
implementing them.
mutations is straightforward. Care needs to be taken to avoid mutating instructions in some parts
of the code. For example, it makes no sense to mutate assertion statements. Detection of
assertion statements is done by their reference to the class level flag which indicates when
assertions are enabled. The pattern of code generated by the compiler is then used to detect the
end of the assertion. Unfortunately such patterns are compiler dependent and care is needed
when moving to new compilers and new JDK releases. Other code that needs to be excluded are
conditionals generated when class constants are accessed, lengths for certain array allocations,
and switch code generated for enumerations. A facility is provided to globally exclude certain
named methods from mutation. In practice this is used to exclude main methods (this is coupled
with a coding standard that main methods should be as short as possible and that they should call
another method with such things as input and output files as parameters). Also excluded are
“integrity” methods which can be called on a class to check that its internal state is currently
consistent. These effectively function as post conditions and like assertions should be excluded.
Jumble also mutates constants, both those that occur inline and those that occur in the constant
pool. An integer constant x is transformed to (-x + 1) which works correctly even when the
integer represents a boolean. In Java boolean, short, and char data types, are implemented in
bytecode as 32-bit ints. Consequently, Jumble cannot readily discriminate among these types.
Constants in the constant pool (such as String literals) are also mutated. The constant pool for a
class often contains entries not referenced by any line of code or only referenced by immutable
code such as assertions. Care is taken not to modify such literals.
Class Loader
One feature of the BCEL which has been extremely useful in the development of Jumble is its
customizable class loader. It allows classes to be modified as they are loaded into the JVM. The
process is much simpler than creating a class loader from scratch, as the processing of the actual
class file is done by the BCEL and the user only has to implement a modify Class method to
perform bytecode modifications on the fly. The Jumble class loader is derived from the BCEL
class loader. Given the name of the class to modify (the target class) and the mutation number, it
loads the class with the mutation inserted. All classes other than the target class are loaded
normally with no modifications. The Jumble test suite is an extension of the JUnit test suite. It
runs a JUnit test in a way suitable for Jumble testing, by running the tests until one fails.
A “PASS” message is returned at that stage and no further tests are run. If no tests fail, a
“FAIL” message is returned. The Jumble test suite is given the name of the test to run as a string
and loads the test class dynamically (using Class.forName). The Jumble test suite is intended to
be loaded in the Jumble class loader so that the tests are run with a mutated class. When running
several mutations inside a single JVM, a separate class loader must be used for each mutated
version of the class being tested. To ensure independence between mutation tests, the entire set
of (non-system) classes are reloaded in each class loader, and the classes being tested are not
available directly from the system class loader. Heavy usage of class loaders in this way can use
a lot of memory in the permgen space in the JVM, particularly for classes that make use of many
thirdparty libraries. In practice it has been difficult to prevent increasing usage of the permgen
space as the tests are run. This is dealt with in two ways. Firstly the child JVM that is used to run
the tests is given additional permgen space when it is started. Secondly after each test is run a
check is done to see if the available permgen space is nearly exhausted, if so the child process is
terminated and restarted (it will then start at the next mutation). Some code remains difficult to
run within the Jumble environment. Typically this involves code that attempts to directly access
the system class loader (such code is usually incorrectly written, and unlikely to function when
run in similar environments, such as within the Tomcat servlet container).
Name of Experiment: Calculate the coverage analysis of programs given in 1 (a) to 1 (f)
using Eclemma Free open source Tool.
Steps to perform Experiment:
Eclemma is a free Java code coverage tool for Eclipse, available under the Eclipse Public
License. It brings code coverage analysis directly into the Eclipse workbench:
• Fast develop/test cycle: Launches from within the workbench like JUnit test runs can
directly be analyzed for code coverage.
• Rich coverage analysis: Coverage results are immediately summarized and highlighted in
the Java source code editors.
• Non-invasive: EclEmma does not require modifying your projects or performing any
other setup.
Since version 2.0 EclEmma is based on the JaCoCo code coverage library. The Eclipse
integration has its focus on supporting the individual developer in an highly interactive way. For
automated builds please refer to JaCoCo documentation for integrations with other tools.
Originally EclEmma was inspired by and technically based on the great EMMA library
developed by Vlad Roubtsov.
The update site for EclEmma is http://update.eclemma.org/. EclEmma is also available via the
Eclipse Marketplace Client, simply search for "EclEmma".
Features
Launching
EclEmma adds a so called launch mode to the Eclipse workbench. It is called Coverage mode
and works exactly like the existing Run and Debug modes. The Coverage launch mode can be
activated from the Run menu or the workbench's toolbar:
Simply launch your applications or unit tests in the Coverage mode to collect coverage
information. Currently the following launch types are supported:
• Local Java application
• Eclipse/RCP application
• Equinox OSGi framework
• JUnit test
• TestNG test
• JUnit plug-in test
• JUnit RAP test
• SWTBot test
• Scala application
Analysis
On request or after your target application has terminated code coverage information is automatically available in the
Eclipse workbench:
• Coverage overview: The Coverage view lists coverage summaries for your Java projects, allowing drill-
down to method level.
• Source highlighting: The result of a coverage session is also directly visible in the Java source editors. A
customizable color code highlights fully, partly and not covered lines. This works for your own source code
as well as for source attached to instrumented external libraries.
Additional features support analysis for your test coverage:
• Different counters: Select whether instructions, branches, lines, methods, types or cyclomatic complexity
should be summarized.
• Multiple coverage sessions: Switching between coverage data from multiple sessions is possible.
• Merge Sessions: If multiple different test runs should be considered for analysis coverage sessions can
easily be merged.
Import/Export
While Eclemma is primarily designed for test runs and analysis within the Eclipse workbench, it provides some import/export
features.
• Execution data import: A wizard allows to import JaCoCo *.exec execution data files from external
launches.
• Coverage report export: Coverage data can be exported in HTML, XML or CSV format or as JaCoCo
execution data files (*.exec).
lOMoARcPSD|25743713
EXPERIMENT : 5
Name of Experiment: Generate Test sequences and validate using Selenium tool for given
websites below:
Selenium is an open-source and a portable automated software testing tool for testing web
applications. It has capabilities to operate across different browsers and operating systems.
Selenium is not just a single tool but a set of tools that helps testers to automate web-based
applications more efficiently.
Creating Selenium IDE Tests
The following steps are involved in creating Selenium tests using IDE:
Recording and adding commands in a test
Saving the recorded test
Saving the test suite
Executing the recorded test
Recording and Adding Commands in a Test
We will use www.ncalculators.com to demonstrate the features of Selenium.
Step 1 : Launch the Firefox browser and navigate to the website –
http://www.ncalculators.com/
Step 2 : Open Selenium IDE from the Tools menu and press the record button that is on the top-
right corner
Step 3 : Navigate to "Math Calculator" >> "Percent Calculator >> enter "10" as number1 and 50
as number2 and click "calculate".
lOMoARcPSD|25743713
Step 4 : The recorded script is generated and the script is displayed as shown below.
lOMoARcPSD|25743713
Step 2 : The tests can be recorded one by one by choosing the option "New Test Case" from the
"File" Menu.
Step 3 : The individual tests are saved with a name along with saving a "Test Suite"
Executing the Recorded Test
The recorded scripts can then be executed either by clicking "Play entire suite" or "Play current
test" button in the toolbar.
Step 1 : The Run status can be seen in the status pane that displays the number of tests passed
and failed.
Step 2 : Once a step is executed, the user can see the result in the "Log" Pane.
Step 3 : After executing each step, the background of the test step turns "Green" if passed and
"Red" if failed as shown below