Quantum Mutants

Eñaut Mendiluze Usandizaga, Thomas Laurent, Paolo Arcaini +1 more

Quantum mutation analysis is emerging as an essential technique for evaluating test suites due to the limited availability of real faulty quantum programs. However, existing quantum mutation analysis tools use fixed gate-based mutations, resulting in easy-to-detect mutants, which reduces their effectiveness in assessing the quality of test suites. We propose QUMUG, a search-based approach for generating challenging mutants by utilising parameterisable quantum gates. QUMUG employs search algorithms to optimise mutation parameters and find non-equivalent mutants passing a given test suite. In our evaluation over 30 quantum programs, QUMUG produced mutants that are three times more challenging than the mutants generated by existing tools. Among the four evaluated search algorithms, the genetic algorithm was the most effective, generating an average of 494 undetected mutants per program with a 99.67% success rate and 94.3% non-equivalent ratio. The generated mutants demonstrated their effectiveness by requiring the addition of five times more test cases to the test suite than the mutants generated by existing tools. We also analysed the behaviour of higher order mutants in quantum circuits, and showed that while first order mutations are more effective for enhancing the test suite, higher order mutants highlight the need for new unique test cases.

Arthur Zajonc, L. J. WANG, X. Y. Zou +1 more

Jose Campos, Andriy V. Miranskyy

Mutation testing evaluates test suite quality by introducing artificial faults (mutants) and checking whether tests detect (kill) them. A central challenge is the equivalent mutant problem: some mutants are syntactically different from the original program but semantically identical to it and therefore cannot be killed by any test. If left unidentified, such mutants waste testing effort and distort mutation scores. In quantum software, mutation testing is increasingly used, but the equivalent mutant problem remains unsolved. A recent study generated more than 700,000 quantum circuit mutants and found that roughly half survived the available tests, making it unclear whether these survivors reflect weak tests or semantic equivalence. We propose Transpiler-Based Equivalence (TBE), a lightweight approach that identifies equivalent quantum mutants by transpiling original and mutated circuits under the same configuration and comparing their resulting OpenQASM code. We evaluate TBE on 348,299 surviving mutants, 92,011 of which are equivalent; TBE identifies 29,536 of them (32.1%) as equivalent while achieving 100% precision and 82% accuracy.

Basil J. Hiley

Frederik F. Flöther

Eñaut Mendiluze Usandizaga, Thomas Laurent, Paolo Arcaini +1 more

Quantum software testing has attracted interest in recent years, prompting the development of various techniques to automate the testing of quantum software. These techniques generate test cases that must be assessed for their effectiveness in detecting faults. Such an assessment requires benchmarks of faulty programs. However, there is a lack of benchmarks containing faults. In this data showcase, we propose QMutBench, a dataset that contains over 700,000 quantum circuit mutants representing different faults. The dataset is accessible via an online interface with selection criteria, such as the original quantum circuit(s) from which mutants are generated, the desired survival rate of the selected mutants, and other mutation characteristics (e.g., the type of faulty quantum gate). QMutBench provides quantum software developers and testers with an accessible online dataset to obtain benchmarks of mutants necessary to assess either the quality of the test cases generated by their testing technique or to compare different testing techniques. It also enables the development of new mutation-guided quantum software testing techniques.