International Journal of Latest Technology in Engineering, Management & Applied Science (IJLTEMAS)

Volume VI, Issue IV, April 2017 | ISSN 2278-2540

Mutation Testing Vs. Regression Testing

Mahesh Kumar Tiwari1, Shalini Lamba 2
1, 2
Assistant Professor, Computer Science Department, National P.G. College, Lucknow, U.P.

Abstract:- Testing is the process of finding as many errors development process. It basically consists of two techniques-
as possible before software is delivered to customer. Since, Functional testing or black box testing and Structural testing
there are various testing techniques available to establish or white box testing. As the name suggests, in functional
quality, performance and reliability of software but testing the focus is on what is the output produced not how
Mutation Testing and Regression Testing is focused in this it is produced while in structural testing, focus is on both i.e.
paper. Mutation testing involves manipulating program what is the output produced and how it is produced.
slightly and testing it with intention to find effectiveness of Therefore, in structural testing, source code is reviewed
test suite selected. Regression testing intends to find bugs thoroughly and test cases are derived from it. Functional
in software, if software is modified after delivery either testing includes Boundary Value Analysis (BVA),
due to result of fixes or due to new or enhanced Equivalence Class Testing, Decision Based Table Testing,
functionality. The use of regression testing is to check that Cause Effect Graphing Technique etc. while Structural testing
enhancements have not affected previous functionality as includes Control Flow Testing, Data Flow Testing, Slice
well as working correctly. Based Testing, Mutation Testing etc[1].
Keywords: - Mutation Testing, Regression Testing
II. MUTATION TESTING
I. INTRODUCTION
Mutation testing involves changing the program and testing it.

T esting can be defined as verifying and validating

something (any software in terms of software testing).
These two activities play major role in software testing
In this, a copy of program is created and one or more changes
are introduced to this copy. The changed program is termed as
mutant of actual program and process of changing program
process. Before software is delivered to customer, it has to be is known as mutation. This changed program is tested and
sure that all requirements are met. The process of checking result produced is compared with expected output. The
whether the software is fulfilling all the customer purpose of mutation testing is to find effectiveness of test
requirements is termed as software verification. It basically suite selected. While creating the mutants of program,
checks that all expectations are properly met or not. While the following precautions should be considered-
process of checking that is software is satisfying all user
Mutants should be syntactically correct,
requirements or not is termed as software validation. The
aim of validating software is to remove all the bugs form They should compile correctly,
software. Example: Consider a small program of Calculator. Changes should be small so that objective of
In this calculator, checking if it has options of addition, program remains unchanged,
deletion, multiplication and division is regarded as its Each mutant should differ from original program by
verification while checking whether it is producing correct one and only one mutation.
result for every operation (including worst case scenario) is Mutants can be created by changing access modifier, static
termed as its validation. modifier, argument order, operator, any numeric value etc[2].
Testing applies to real world scenarios also. Suppose that a The mutants of programs are executed and their result is
customer want to purchase TV of any particular brand (say tested against original program. If test cases are able to find
XYZ), so prior to billing he would like to verify TVs brand changes made i.e. if actual output and expected output is
by seeing label on it and then customer would see its different, then it is regarded as Killed Mutant. Otherwise if
performance i.e. customer will validate it. Thus, testing is an actual output and expected output is same, then it is termed as
essential part before purchasing any product. Testing gives an Alive Mutant[3].
idea about efficiency and quality of product under test and at Mutation testing is based on the assumption that a program
the same time minimizes the future risks. will be well tested if all simple faults are detected and
Software testing is very important activity of Software removed[4].
Development Life Cycle. It is very vast and time consuming
activity which consists of one third to one half of total

www.ijltemas.in Page 91
 International Journal of Latest Technology in Engineering, Management & Applied Science (IJLTEMAS)
Volume VI, Issue IV, April 2017 | ISSN 2278-2540

15) tax=10.0/100*(income-40000);
16) System.out.println("Tax\t: "+tax);
17) }
18) else if(income<=150000)
19) {
20) tax=2000+20.0/100*(income-60000);
21) System.out.println("Tax\t: "+tax);
22) }
23) else
Figure 1 Process of Mutation Testing
24) {
25) tax=20000+30.0/100*(income-150000);
After mutation testing is done mutation score is calculated as- 26) System.out.println("Tax\t: "+tax);
27) }
Mutation Score = Number of Mutants Killed / Total 28) }
Number of Mutants 29) }

Mutation score determines the accuracy and sensitivity of Let the test suite selected is
program towards changes. Mutation score is always lies
between 0 and 1. A higher value of mutation score indicates ID Income Tax (Expected Output)
the effectiveness of test suite. 1 37000 No Tax
2 57000 1700.0
Mutation was originally proposed in 1971 but being very
3 100000 10000.0
expensive, it lost its importance but now, again it is being
widely used for languages like Java and XML. 4 200000 35000.0

Let the mutants of program are-

Mutation Testing Example
To understand the concept of Mutation testing consider a
program that calculates the income tax based on following
criteria-
Income <= 40000 No Tax
Income >40000 and <=60000 tax=10% of income
exceeding 40000
Income>60000 and <=150000 tax=2000+20% of income
exceeding 60000
Income >150000 tax=20000+30% of Test cases for Mutant M1
income exceeding 150000
Tax Tax
The program for above code will be- ID Income (Expected (Original
Output) Output)
1) import java.io.*; 1 37000 No Tax -300.0
2) public class Income_Tax 2 57000 1700.0 1700.0
3) { 3 100000 10000.0 10000.0
4) public static void main(String agrs[])throws 4 200000 35000.0 35000.0
IOException
5) { Test cases for Mutant M2
6) int income;
7) double tax; Tax Tax
ID Income (Expected (Original
8) BufferedReader br=new BufferedReader(new Output) Output)
InputStreamReader(System.in)); 1 37000 No Tax No Tax
9) System.out.print("Income\t: "); 2 57000 1700.0 1700.0
10) income=Integer.parseInt(br.readLine()); 3 100000 10000.0 10000.0
11) if(income<=40000) 4 200000 35000.0 35000.0
12) System.out.println("Tax\t: No Tax");
13) else if(income<=60000) Test cases for Mutant M3
14) {

www.ijltemas.in Page 92
 International Journal of Latest Technology in Engineering, Management & Applied Science (IJLTEMAS)
Volume VI, Issue IV, April 2017 | ISSN 2278-2540

Tax Tax Regression Testing Example

ID Income (Expected (Original
Output) Output) To understand the concept more clearly, consider the previous
1 37000 No Tax No Tax program of computing income tax. Suppose that now
2 57000 1700.0 1700.0 customer wants to add clause that if income is above 200000
3 100000 10000.0 6000.0 then surcharge of 2% on tax is calculated as per previous
4 200000 35000.0 35000.0 rates. So, the programmer will modify the program according
to additional requirements and the modified program will be
Test cases for Mutant M4 as-
Tax Tax
ID Income (Expected (Original
1) import java.io.*;
Output) Output) 2) public class Income_Tax
1 37000 No Tax No Tax 3) {
2 57000 1700.0 1700.0 4) public static void main(String agrs[])throws IOException
3 100000 10000.0 6000.0 5) {
4 200000 35000.0 75500.0 6) int income;
7) double tax;
As we can see that Mutation M2 is producing same expected 8) BufferedReader br=new
and desired output hence it will be alive while Mutation M1, BufferedReader(new
M3, M4 will be killed. Hence, InputStreamReader(System.in));
Mutation score=3/4 9) System.out.print("Income\t: ");
=0.75 10) income=Integer.parseInt(br.readLine());
Higher the mutation score, better the effectiveness of test 11) if(income<=40000)
suite. 12) System.out.println("Tax\t: No Tax");
13) else if(income<=60000)
III. REGRESSION TESTING 14) {
Once software is tested, it is delivered to customer. Since 15) tax=10.0/100*(income-40000);
testing shows presence of errors not absence of errors, 16) System.out.println("Tax\t: "+tax);
therefore there may be possibility that any error may arise 17) }
after delivery of software. In this case software will be 18) else if(income<=150000)
modified. The process of testing the modified portion of 19) {
program and portion likely to be affected by modification is 20) tax=2000+20.0/100*(income-60000);
termed as Regression Testing. Modification in program or 21) System.out.println("Tax\t: "+tax);
software is not always carried out due to errors. It may also be 22) }
carried out if customer wants to add certain functionality to 23) else if(income>150000&&income<=200000)
existing software. 24) {
25) tax=20000+30.0/100*(income-150000);
Regression testing is part of system maintenance activity, 26) System.out.println("Tax\t: "+tax);
carried out to ensure that modified portion has not introduced 27) }
errors in previously tested software and is working properly. 28) else
Thus, regression testing is carried out when any changes are 29) {
made to software after delivery. 30) tax=1.02*(20000+30.0/100*(income-150000));
31) System.out.println("Tax\t: "+tax);
For regression testing, already available test cases are used.
32) }
There are three possibilities of selecting test cases-
33) }
1) Select all test cases, which is very simple but at same 34) }
time very time consuming,
2) Select random test cases, which is less time As one can see that, previous code is modified from line 23.
consuming but there is possibility that it may escape So test cases will be chosen such that it covers all statements
test cases related to modified portion of code, from line 23, to ensure that the modified code is correct and
3) Select test cases that traverse the modified portion of properly implemented. Therefore test cases for this will be as
code. follows-
The idea of regression testing is to save time and money. As
there is no logic in testing completely tested program again.

www.ijltemas.in Page 93
 International Journal of Latest Technology in Engineering, Management & Applied Science (IJLTEMAS)
Volume VI, Issue IV, April 2017 | ISSN 2278-2540

Tax Tax effectiveness of test suite selected, while in regression testing

S. No. Income (Expected (Original effectiveness of modified application is tested. Since in both
Output) Output) process code is modified, but in regression testing
1 160000 23000.0 23000.0 modification is requested by client and is system maintenance
2 200000 35000.0 35000.0
activity while in mutation testing modification is part of
testing and is fault based testing technique as faults are
3 200001 35700.306 35700.306
forcefully introduced by modification. Thus, purpose of both
4 250000 51000.0 51000.0 testing is to provide a bug free software but the techniques
used for same are totally non identical.
Thus, modified program is also correct; also it has not
disturbed working of previous program. Hence regression REFERENCES
testing is successful.
[1]. Megha Jhamb, Abhishek Singhal, And AbhayBansal, A Survey
on Different Approaches for Efficient Mutation Testing
IV. CONCLUSION International Journal of Scientific and Research Publications,
Volume 3, Issue 4, April 2013.
Testing is an essential phase of SDLC. Out of various testing [2]. Lingming Zhang, Darko Marinov, Lu Zhang, Sarfraz Khurshid
techniques available mutation testing and regression technique Regression Mutation Testing, ISSTA-2012.
is highlighted in this paper. Mutation testing and Regression [3]. Y. Jia and M. Harman. An analysis and survey of the
development of mutation testing, IEEE TSE, 37(5):649678,
testing are two completely different testing techniques but 2011.
often resembled same by beginners due to modification [4]. A Jefferson Offutt, Kanupriya Tewary, Experiments with Data
associated with it. In mutation testing focus is on finding flow and Mutation Testing , Feburary 1994.

www.ijltemas.in Page 94