Mutation Testing Using

JUnit
CST458 SOFTWARE TESTING
What is mutation testing?

● Mutation testing is a white box method in software testing where we insert errors purposely into a
program (under test) to verify whether the existing test case can detect the error or not.
● In this testing, the mutant of the program is created by making some modifications to the original
program.
● The primary objective of mutation testing is to check whether each mutant created an output,
which means that it is different from the output of the original program. We will make slight
modifications in the mutant program because if we change it on a massive scale than it will affect
the overall plan.
What is mutation?

● The mutation is a small modification in a program; these minor modifications are planned to typical
low-level errors which are happened at the time of coding process.
● Generally, we deliberate the mutation operators in the form of rules which match the data and also
generate some efficient environment to produce the mutant.
Types of mutation testing

3 types :

1. Decision mutations
2. Value mutations
3. Statement mutations
Decision mutations

In this type of mutation testing, we will check the design errors. And here, we will do the modification in
arithmetic and logical operator to detect the errors in the program.

● plus(+)→ minus(-)
● asterisk(*)→ double asterisk(**)
● plus(+)→incremental operator(i++)
● Exchange P > → P<, OR P>=
Decision mutations
Value mutations

In this, the values will modify to identify the errors in the program, and generally, we will change the
following:

● Small value → higher value

● Higher value → Small value.
Value mutations
Statement Mutations

Statement mutations means that we can do the modifications into the statements by removing or
replacing the line as we
How to perform mutation testing

● In this, firstly, we will add the errors into the source code of the program by producing various
versions, which are known mutants.
● Here every mutant having the one error, which leads the mutant kinds unsuccessful and also
validates the efficiency of the test cases.
● After that, we will take the help of the test cases in the mutant program and the actual application
will find the errors in the code.
● Once we identify the faults, we will match the output of the actual code and mutant code.
● After comparing the output of both actual and mutant programs, if the results are not matched,
then the mutant is executed by the test cases. Therefore the test case has to be sufficient for
identifying the modification between the actual program and the mutant program.
How to perform mutation testing

● And if the actual program and the mutant program produced the exact result, then the mutant is
saved. And those cases are more active test cases because it helps us to execute all the mutants.
How to perform mutation testing
JUnit

JUnit is an open source framework, which is used for writing and running tests.

● Provides annotations to identify test methods.

● Provides assertions for testing expected results.
● Provides test runners for running tests.
● JUnit tests allow you to write codes faster, which increases quality.
● JUnit is elegantly simple. It is less complex and takes less time.
● JUnit tests can be run automatically and they check their own results and provide immediate
feedback. There's no need to manually comb through a report of test results.
Types of unit testing

● Manual Testing

If you execute the test cases manually without any tool support, it is known as manual testing. It
is time consuming and less reliable.

● Automated Testing

If you execute the test cases by tool support, it is known as automated testing. It is fast and
more reliable.
Unit test case

● A Unit Test Case is a part of code, which ensures that another part of code (method) works as
expected. To achieve the desired results quickly, a test framework is required.
● A test suite bundles a few unit test cases and runs them together.
JUnit - Steps

● Create a test class: In JUnit, each test is represented by a test method within a test class. Create a
new Java class that will contain your unit tests. Typically, the test class name follows the
convention of appending Test to the class being tested. For example, if you have a class named
Calculator, the corresponding test class could be named CalculatorTest.
● Add necessary annotations: Annotate your test class and test methods with JUnit annotations to
define their purpose and behavior. Some commonly used annotations are Test, Before, After
● Write test methods: Define test methods within your test class. Each test method should be
annotated with @Test and contain the assertions that verify the expected behavior of the code
being tested. Use JUnit's assertion methods (e.g., assertEquals, assertTrue, assertFalse, etc.) to
perform these checks.
JUnit - Steps

● Run the tests: Execute your unit tests. You can run the tests within your IDE by right-clicking the
test class or test methods and selecting the "Run" option. Alternatively, you can use build tools like
Maven or Gradle to run the tests from the command line.
● Analyze test results: After the tests complete, JUnit provides feedback on the results. You'll see
output indicating which tests passed and which failed. If any tests fail, review the error messages
and stack traces to identify the issues.
How to conduct mutation testing using JUnit

● Write a unit test.

● Write code to go through the test.
● Run the mutation test with the code, ensuring the code and test work.
● Make some mutations to the code and run the new mutated code through the test. Every mutant
should have one error to validate the test's efficiency.
How to conduct mutation testing using JUnit

● Make some mutations to the code and run the new mutated code through the test. Every mutant
should have one error to validate the test's efficiency.
● Compare the results from the original code and the mutated version.
○ If the results don't match, the test successfully identified and executed the mutant.
○ If the test case produced the same result between the original and mutant code, the test
failed and the mutant is saved.
● A mutation score can also be calculated. The score is given as a percentage that's determined using
the formula noted above.
Performance parameters

Mutation operators: Mutation testing tools apply specific mutation operators to introduce faults into the
program.

Mutation coverage criteria: Different mutation coverage criteria measure the effectiveness of the test
suite in detecting mutations.

Mutation score : A mutation score represents the proportion of mutations detected by the test suite