Papers by Michaël Marcozzi
ACM SIGPLAN conference on Systems, Programming, Languages, and Applications, Software for Humanity, OOPSLA (SPLASH 2019 OOPSLA), 2019
Despite much recent interest in randomised testing (fuzzing) of compilers, the practical impact o... more Despite much recent interest in randomised testing (fuzzing) of compilers, the practical impact of fuzzer-found compiler bugs on real-world applications has barely been assessed. We present the first quantitative and qualitative study of the tangible impact of miscompilation bugs in a mature compiler. We follow a rigorous methodology where the bug impact over the compiled application is evaluated based on (1) whether the bug appears to trigger during compilation; (2) the extent to which generated assembly code changes syntactically due to triggering of the bug; and (3) whether such changes cause regression test suite failures, or whether we can manually find application inputs that trigger execution divergence due to such changes. The study is conducted with respect to the compilation of more than 10 million lines of C/C++ code from 309 Debian packages, using 12% of the historical and now fixed miscompilation bugs found by four state-of-the-art fuzzers in the Clang/LLVM compiler, as well as 18 bugs found by human users compiling real code or as a by-product of formal verification efforts. The results show that almost half of the fuzzer-found bugs propagate to the generated binaries for at least one package, in which case only a very small part of the binary is typically affected, yet causing two failures when running the test suites of all the impacted packages. User-reported and formal verification bugs do not exhibit a higher impact, with a lower rate of triggered bugs and one test failure. The manual analysis of a selection of the syntactic changes caused by some of our bugs (fuzzer-found and non fuzzer-found) in package assembly code, shows that either these changes have no semantic impact or that they would require very specific runtime circumstances to trigger execution divergence.
Testing is the primary approach for detecting software defects. A major challenge faced by tester... more Testing is the primary approach for detecting software defects. A major challenge faced by testers lies in crafting eecient test suites, able to detect a maximum number of bugs with manageable eeort. To do so, they rely on coverage criteria, which deene some precise test objectives to be covered. However, many common criteria specify a signiicant number of objectives that occur to be infeasible or redundant in practice, like covering dead code or semantically equal mutants. Such objectives are well-known to be harmful to the design of test suites, impacting both the eeciency and precision of the tester's eeort. This work introduces a sound and scalable technique to prune out a signiicant part of the infeasible and redundant objectives produced by a panel of white-box criteria. In a nutshell, we reduce this task to proving the validity of logical assertions in the code under test. The technique is implemented in a tool that relies on weakest-precondition calculus and SMT solving for proving the assertions. The tool is built on top of the Frama-C veriication platform, which we carefully tune for our speciic scalability needs. The experiments reveal that the pruning capabilities of the tool can reduce the number of targeted test objectives in a program by up to 27% and scale to real programs of 200K lines, making it possible to automate a painstaking part of their current testing process.
A large amount of research has been carried out to automate white-box testing. While a wide range... more A large amount of research has been carried out to automate white-box testing. While a wide range of different and sometimes heterogeneous code-coverage criteria have been proposed, there exists no generic formalism to describe them all, and available test automation tools usually support only a small subset of them. We introduce a new specification language, called HTOL (Hyperlabel Test Objectives Language), providing a powerful generic mechanism to define a wide range of test objectives. HTOL comes with a formal semantics, and can encode all standard criteria but full mutations. Besides specification, HTOL is appealing in the context of test automation as it allows to handle criteria in a unified way.
— Automated white-box testing is a major issue in software engineering. In previous work, we intr... more — Automated white-box testing is a major issue in software engineering. In previous work, we introduced LTest, a generic and integrated toolkit for automated white-box testing of C programs. LTest supports a broad class of coverage criteria in a unified way (through the label specification mechanism) and covers most major parts of the testing process – including coverage measurement, test generation and detection of infeasible test objectives. However, the original version of LTest was unable to handle several major classes of coverage criteria, such as MCDC or dataflow criteria. Moreover, its practical applicability remained barely assessed. In this work, we present a significantly extended version of LTest that supports almost all existing testing criteria, including MCDC and some software security properties, through a native support of recently proposed hyperlabels. We also provide a more realistic view on the practical applicability of the extended tool, with experiments assessing its efficiency and scalability on real-world programs.
Symbolic execution is a technique which enables automatically generating test inputs (and outputs... more Symbolic execution is a technique which enables automatically generating test inputs (and outputs) exercising a set of execution paths within a program to be tested. If the paths cover a sufficient part of the code under test, the test data offer a representative view of the program's actual behaviour, which notably enables detecting errors and correcting faults. Relational databases are ubiquitous in software, but symbolic execution of pieces of code that manipulate them remains a non-trivial problem, particularly because of the complex structure of such databases and the complex behaviour of SQL statements. In this work, we define a direct symbolic execution for database manipulation code and integrate it with a more traditional symbolic execution of normal program code. The database tables are represented by relational symbols and the SQL statements by relational constraints over these symbols and the symbols representing the normal variables of the program. An algorithm base...
Symbolic execution is a technique enabling the automatic generation of test inputs that exercise ... more Symbolic execution is a technique enabling the automatic generation of test inputs that exercise a set of execution paths within a code unit to be tested. If the paths cover a sufficient part of the code under test, the test data offer a representative view of the actual behaviour of this code. This notably enables detecting errors and correcting faults. Relational databases are ubiquitous in software, but symbolic execution of code units that manipulate them remains a non-trivial problem, particularly because of the complex structure of such databases and the complex behaviour of SQL statements. Finding errors in such code units is yet critical, as it can avoid corrupting important data. In this work, we define a symbolic execution translating database manipulation code directly into constraints and integrate it with a more traditional symbolic execution of normal program code. The database tables are represented by relational symbols and the SQL statements by relational constraints over these symbols. An algorithm based on these principles is presented for the symbolic execution of simple Java methods that implement transactional use cases by reading and writing in a relational database, the latter subject to data integrity constraints. The algorithm is integrated in a test generation tool and experimented over sample code. The target language for the constraints produced by the tool is the SMT-Lib standard and the used solver is Microsoft Z3. The results show that the proposed approach enables generating meaningful test data, including valid database content, in reasonable time. In particular, the Z3 solver is shown to be more scalable than the Alloy solver, used in our previous work, for solving relational constraints.
Constraint-based testing is an automatic test case generation approach where the tested applicati... more Constraint-based testing is an automatic test case generation approach where the tested application is transformed into constraints whose solutions are adequate test data. In previous work, we have shown that this technique is particularly well-suited for testing SQL applications, as the semantics of SQL can be naturally transformed into standard SMT constraints, using so-called relational symbolic execution. In particular, we have demonstrated such testing to be possible in practice with current solver techniques for small-scale applications. In this work, we identify the main challenges and provide research directions towards constraint-based testing of full-scale SQL applications. We investigate the additional research work needed to integrate relational and dynamic symbolic execution, handle properly dynamic SQL, generate tractable SMT constraints for most SQL applications, detect SQL runtime errors and deal with non-deterministic SQL.
In constraint-based program testing, symbolic execution is a technique which allows to generate t... more In constraint-based program testing, symbolic execution is a technique which allows to generate test data exercising a given execution path, selected within the program to be tested. Applied to a set of paths covering a sufficient part of the code under test, this technique permits to generate automatically adequate test sets for units of code. As databases are ubiquitous in software, generalizing such a technique for efficient testing of programs manipulating databases is an interesting approach to enhance the reliability of software. In this work, we propose a relational symbolic execution algorithm to be used for testing of simple Java methods, reading and writing with transactional SQL in a relational database, subject to integrity constraints. This algorithm considers the Java method under test as a sequence of operations over a set of constrained relational variables, modeling both the database tables and the method variables. By integrating this relational model of the method and database with the classical symbolic execution process, the algorithm can generate a set of Alloy constraints for any finite path to test in the control-flow graph of the method. Solutions of these constraints are data which constitute a test case, including valid content for the database, which exercises the selected path in the method. A tool implementing the proposed algorithm is demonstrated over a number of examples.
Symbolic execution is a technique which allows to automatically generate test inputs (and outputs... more Symbolic execution is a technique which allows to automatically generate test inputs (and outputs) exercising a set of execution paths within a program to be tested. If the paths cover a sufficient part of the code under test, the test data offer a representative view of the program's actual behaviour, allowing to detect failures and correct faults. Relational databases are ubiquitous in software, but symbolic execution of the programs that manipulate them remains a non-trivial problem, because of the complex structure of such databases and the complex behaviour of the SQL statements. In this work, we define a symbolic execution for SQL code and integrate it with a more traditional symbolic execution of normal program code. The database tables are represented by relational symbols and the SQL statements by relational constraints over these symbols and the symbols representing the normal variables of the program. An algorithm based on these principles is presented for the symbolic execution of simple Java methods, reading and writing with transactional SQL in a relational database, the latter subject to data integrity constraints. The algorithm is integrated in a test data generation tool and experimentally evaluated over thousands of program paths including real application code. The target language for the constraints produced by the tool is the SMTv2 standard and the used solver is Microsoft Z3. The results show that the proposed approach allow to generate meaningful test data for database programs, including valid database content, in reasonable time.
In this paper we propose a novel representation scheme, called probabilistic encoding. In this re... more In this paper we propose a novel representation scheme, called probabilistic encoding. In this representation, each gene of an individual represents the probability that a certain trait of a given problem has to belong to the solution. This allows to deal with uncertainty that can be present in an optimization problem, and grant more exploration capability to an evolutionary algorithm. With this encoding, the search is not restricted to points of the search space. Instead, whole regions are searched, with the aim of individuating a promising region, i.e., a region that contains the optimal solution. This implies that a strategy for searching the individuated region has to be adopted. In this paper we incorporate the probabilistic encoding into a multi-objective and multi-modal evolutionary algorithm. The algorithm returns a promising region, which is then searched by using simulated annealing. We apply our proposal to the problem of discovering biclusters in microarray data. Results confirm the validity of our proposal.
Databases are ubiquitous in software and testing of programs manipulating databases is thus essen... more Databases are ubiquitous in software and testing of programs manipulating databases is thus essential to enhance the reliability of software. In this paper, we describe a clean and unified approach to automatically generate test inputs for such database programs. First, we propose a formal language, called ImperDB, to model database programs. ImperDB allows to model common program behaviors and data structures, as well as typical interaction scenarios between programs and databases. Secondly, we present a static analysis technique to generate test inputs for ImperDB programs, according to any chosen structural adequacy criterion. The technique considers an ImperDB program as a sequence of operations over a set of relational variables, modeling both the database original content and the program inputs. The problem of finding test inputs forcing the execution of a given path can then be transformed into the problem of solving constraints over the relational variables associated to the program. These constraints are expressed with the Alloy language and solved by the Alloy analyzer.
Surface and Interface Analysis, 2007
In this work, aluminium (Alclad 2024-T3) substrates were cleaned by an r.f. (13.56 MHz) plasma, u... more In this work, aluminium (Alclad 2024-T3) substrates were cleaned by an r.f. (13.56 MHz) plasma, using argon (Ar), oxygen (O2) and a mixture of O2/Ar (50:50) gases. The effectiveness of plasma cleaning was checked in situ using X-ray photoelectron spectroscopy (XPS) and ex situ using water contact angle measurements. XPS O/Al surface atomic ratios are in excellent agreement with those of the crystalline boehmite and the pseudoboehmite. Oxygen O 1s peak-fitting was used to quantify the proportion of hydroxyl ions and the functional composition on the aluminium surface: the surface cleaned with O2 plasma contains 50% of aluminium hydroxides, the ones cleaned with Ar plasma and with Ar/O2 plasma contain, respectively, 25 and 37% hydroxyl ions. The binding energy separation between Al 2p and O 1s is characteristic of AlO(OH). Thin SiOx films were subsequently deposited from a mixture of hexamethyldisiloxane (HMDSO) and oxygen. In the absence of oxygen, a hydrophobic (Θ≥ 100° ) film characteristic of polydimethylsiloxane (PDMS) is formed: polysiloxane-like thinner films (SiOx) are obtained with the introduction of oxygen. XPS and contact angle measurements confirmed both the composition and the structure of these films. More importantly, contact angle measurements using different liquids and interpreted with the van Oss-Good-Chaudhury theory allowed determination of the surface free energy of the deposited films: the calculated values of surface tension of the film formed from HMDSO/O2: (50/50) are in excellent agreement with those of reference silica-based materials such as a silicon wafer and cleaned glass. Copyright © 2007 John Wiley & Sons, Ltd.
Drafts by Michaël Marcozzi
Testing is the primary approach for detecting software defects. A major challenge faced by tester... more Testing is the primary approach for detecting software defects. A major challenge faced by testers lies in crafting efficient test suites, able to detect a maximum number of bugs with manageable effort. To do so, they rely on coverage criteria, which define some precise test objectives to be covered. However, many common criteria specify a significant number of objectives that occur to be infeasible or redundant in practice, like covering dead code or semantically equal mutants. Such objectives are well-known to be harmful to the design of test suites, impacting both the efficiency and precision of testers' effort. This work introduces a sound and scalable formal technique able to prune out a significant part of the infeasible and redundant objectives produced by a large panel of white-box criteria. In a nutshell, we reduce this challenging problem to proving the validity of logical assertions in the code under test. This technique is implemented in a tool that relies on weakest-precondition calculus and SMT solving for proving the assertions. The tool is built on top of the Frama-C verification platform, which we carefully tune for our specific scalability needs. The experiments reveal that the tool can prune out up to 27% of test objectives in a program and scale to applications of 200K lines of code.
Uploads
Papers by Michaël Marcozzi
Drafts by Michaël Marcozzi