Mutation testing is an important technique for assessing the quality of test cases and software systems. It involves introducing changes into the code and identifying those changes. In the context of quantum computing, where programs are shown as quantum circuits composed of quantum gates, the traditional mutation testing techniques face unique challenges due to the inherent complexity of quantum algorithms and operations. We propose a novel strategy for higher-order mutation testing of quantum programs. Our main objective is to enhance the reliability of the quantum program. We propose a mutator to replace quantum gates in a quantum program. It identifies the gate combinations and their syntactic equivalence. We compare circuit state vectors to identify equivalent gate combinations and present the results for efficient mutation generation.
Mutation testing is an important technique for assessing the quality of test cases and software systems. It involves introducing changes into the code and identifying those changes. In the context of quantum computing, where programs are shown as quantum circuits composed of quantum gates, the traditional mutation testing techniques face unique challenges due to the inherent complexity of quantum algorithms and operations. We propose a novel strategy for higher-order mutation testing of quantum programs. Our main objective is to enhance the reliability of the quantum program. We propose a mutator to replace quantum gates in a quantum program. It identifies the gate combinations and their syntactic equivalence. We compare circuit state vectors to identify equivalent gate combinations and present the results for efficient mutation generation.
As quantum computing is still in its infancy, there is an inherent lack of knowledge and technology to test a quantum program properly. In the classical realm, mutation testing has been successfully used to evaluate how well a program's test suite detects seeded faults (i.e., mutants). In this paper, building on the definition of syntactically equivalent quantum operations, we propose a novel set of mutation operators to generate mutants based on qubit measurements and quantum gates. To ease the adoption of quantum mutation testing, we further propose QMUTPY, an extension of the well-known and fully automated open-source mutation tool MUTPY. To evaluate QMUTPY's performance, we conducted a case study on 24 real quantum programs written in IBM's QISKIT library. Furthermore, we show how better test suite coverage and improvements to test assertions can increase the test suites' mutation score and quality. QMUTPY has proven to be an effective quantum mutation tool, providing insight into the current state of quantum tests and how to improve them.
Quantum mutation analysis is emerging as an essential technique for evaluating test suites due to the limited availability of real faulty quantum programs. However, existing quantum mutation analysis tools use fixed gate-based mutations, resulting in easy-to-detect mutants, which reduces their effectiveness in assessing the quality of test suites. We propose QUMUG, a search-based approach for generating challenging mutants by utilising parameterisable quantum gates. QUMUG employs search algorithms to optimise mutation parameters and find non-equivalent mutants passing a given test suite. In our evaluation over 30 quantum programs, QUMUG produced mutants that are three times more challenging than the mutants generated by existing tools. Among the four evaluated search algorithms, the genetic algorithm was the most effective, generating an average of 494 undetected mutants per program with a 99.67% success rate and 94.3% non-equivalent ratio. The generated mutants demonstrated their effectiveness by requiring the addition of five times more test cases to the test suite than the mutants generated by existing tools. We also analysed the behaviour of higher order mutants in quantum circuits, and showed that while first order mutations are more effective for enhancing the test suite, higher order mutants highlight the need for new unique test cases.
Abstract Software testing is an important activity to ensure the software's quality. Mutation testing is a fault-based technique for checking the effectiveness of test cases for the verification and validation of software. Testing quantum programs is challenging due to the involvement of quantum mechanics concepts (such as superposition and entanglement ) and the probabilistic nature of the outcome. In this paper, We will propose the modified mutation testing approach for quantum program. The proposed approach will help to reduce the cost, time, and effort of performing the mutation testing. Moreover, we also propose a method for determining the equivalent mutant and developing a tool to check whether the mutated and original quantum program behave semantically equivalent or differently.
Abstract Software testing is an important activity to ensure the software's quality. Mutation testing is a fault-based technique for checking the effectiveness of test cases for the verification and validation of software. Testing quantum programs is challenging due to the involvement of quantum mechanics concepts (such as superposition and entanglement ) and the probabilistic nature of the outcome. In this paper, We will propose the modified mutation testing approach for quantum program. The proposed approach will help to reduce the cost, time, and effort of performing the mutation testing. Moreover, we also propose a method for determining the equivalent mutant and developing a tool to check whether the mutated and original quantum program behave semantically equivalent or differently.
Given that quantum software testing is a new area of research, there is a lack of benchmark programs and bugs repositories to assess the effectiveness of testing techniques. To this end, quantum mutation analysis focuses on systematically generating faulty versions of Quantum Programs (QPs), called mutants, using mutation operators. Such mutants can be used as benchmarks to assess the quality of test cases in a test suite. Thus, we present Muskit - a quantum mutation analysis tool for QPs coded in IBM's Qiskit language. Muskit defines mutation operators on gates of QPs and selection criteria to reduce the number of mutants to generate. Moreover, it allows for the execution of test cases on mutants and generation of results for test analyses. Muskit is provided as command line interface, GUI, and web application. We validated Muskit by using it to generate and execute mutants for four QPs. Muskit code: https://github.com/Simula-COMPLEX/muskitWeb app: https://qiskitmutantcreatorsrl.pythonanywhere.com/YouTube Video: EbPHJOK_AEA Artifact Available: https://doi.org/10.5281/zenodo.5288917
Mutation testing assesses test suites by introducing program changes and checking whether tests detect them. In quantum programs, a surviving mutant may leave program behavior unchanged, produce a change hidden by the chosen input or measurement, or remain undetected because statistical evidence is insufficient. We develop a circuit-aware framework to distinguish these causes and guide execution planning and test changes. The framework covers ideal, noiseless programs and mutations that replace one gate or subcircuit. The method traces how a mutation can affect measured outputs and combines established mathematical checks with statistical estimates. For supported cases, we establish conditions under which a test cannot reveal a mutation and exact formulas for its effect when the required state information is available. Under stated assumptions, we bound the mutation score while retaining surviving and unresolved mutants, and identify a restricted circuit family whose mutation effects admit efficient classical classification. The deliverables include a decision procedure, mathematical guarantees, worked examples of test changes and their implementation requirements, and a small-circuit reference implementation for differential expectation tests. These results help testers decide what to execute, what to change, and what remains unknown. The evidence is limited to proofs and executable examples under the ideal model.