Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA)

QMutPy: a mutation testing tool for Quantum algorithms and applications in Qiskit

Daniel Fortunato, José Campos, Rui Abreu

Published Jul 15, 2022• 48 citations
Publisher

Abstract

There is an inherent lack of knowledge and technology to test a quantum program properly. In this paper, building on the definition of syntactically equivalent quantum gates, we describe our efforts in developing a tool, coined QMutPy, leveraging the well-known open-source mutation tool MutPy. We further discuss the design and implementation of QMutPy, and the usage of a novel set of mutation operators that generate mutants for qubit measurements and gates. To evaluate QMutPy’s performance, we conducted a preliminary study on 11 real quantum programs written in the IBM’s Qiskit library. QMutPy has proven to be an effective quantum mutation tool, providing insight into the current state of quantum tests. QMutPy is publicly available at https://github.com/danielfobooss/mutpy. Tool demo: https://youtu.be/fC4tOY5trqc.

DOI
10.1145/3533767.3543296

Related papers

Mutation testing of quantum programs written in QISKit

Daniel Fortunato, José Campos, Rui Abreu

There is an inherent lack of knowledge and technology to test a quantum program properly. In this paper, building on the definition of syntactically equivalent quantum operations, we investigated a novel set of mutation operators to generate mutants based on qubit measurements and quantum gates. To ease the adoption of quantum mutation testing, we further discuss QMutPy, an extension of the well-known and fully automated open-source mutation tool MutPy. To evaluate QMutPy's performance we conducted a case study on 11 real quantum programs written in the IBM's QISKit library. QMutPy has proven to be an effective quantum mutation tool, providing insight on the current state of quantum tests.

ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion)•May 21, 2022•28 citations

Mutation Testing of Quantum Programs Written in QISKit

Daniel Fortunato, José Campos, Rui Abreu

There is an inherent lack of knowledge and technology to test a quantum program properly. In this paper, building on the definition of syntactically equivalent quantum operations, we investigated a novel set of mutation operators to generate mutants based on qubit measurements and quantum gates. To ease the adoption of quantum mutation testing, we further discuss QMutPy, an extension of the well-known and fully automated open-source mutation tool MutPy. To evaluate QMutPy’s performance we conducted a case study on 11 real quantum programs written in the IBM’s QISKit library. QMutPy has proven to be an effective quantum mutation tool, providing insight on the current state of quantum tests.

ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion)•May 1, 2022•5 citations

Mutation Testing of Quantum Programs: A Case Study With Qiskit

Daniel Fortunato, José Campos, Rui Abreu

As quantum computing is still in its infancy, there is an inherent lack of knowledge and technology to test a quantum program properly. In the classical realm, mutation testing has been successfully used to evaluate how well a program's test suite detects seeded faults (i.e., mutants). In this paper, building on the definition of syntactically equivalent quantum operations, we propose a novel set of mutation operators to generate mutants based on qubit measurements and quantum gates. To ease the adoption of quantum mutation testing, we further propose QMUTPY, an extension of the well-known and fully automated open-source mutation tool MUTPY. To evaluate QMUTPY's performance, we conducted a case study on 24 real quantum programs written in IBM's QISKIT library. Furthermore, we show how better test suite coverage and improvements to test assertions can increase the test suites' mutation score and quality. QMUTPY has proven to be an effective quantum mutation tool, providing insight into the current state of quantum tests and how to improve them.

IEEE Transactions on Quantum Engineering•Jan 1, 2022•62 citations

Muskit: A Mutation Analysis Tool for Quantum Software Testing

Eñaut Mendiluze Usandizaga, Shaukat Ali, Paolo Arcaini +1 more

Given that quantum software testing is a new area of research, there is a lack of benchmark programs and bugs repositories to assess the effectiveness of testing techniques. To this end, quantum mutation analysis focuses on systematically generating faulty versions of Quantum Programs (QPs), called mutants, using mutation operators. Such mutants can be used as benchmarks to assess the quality of test cases in a test suite. Thus, we present Muskit - a quantum mutation analysis tool for QPs coded in IBM's Qiskit language. Muskit defines mutation operators on gates of QPs and selection criteria to reduce the number of mutants to generate. Moreover, it allows for the execution of test cases on mutants and generation of results for test analyses. Muskit is provided as command line interface, GUI, and web application. We validated Muskit by using it to generate and execute mutants for four QPs. Muskit code: https://github.com/Simula-COMPLEX/muskitWeb app: https://qiskitmutantcreatorsrl.pythonanywhere.com/YouTube Video: EbPHJOK_AEA Artifact Available: https://doi.org/10.5281/zenodo.5288917

IEEE/ACM International Conference on Automated Software Engineering (ASE)•Nov 1, 2021•69 citations

Development of a Tool for Finding Equivalent Mutants in Quantum Program: A Perspective to Measure the Quality of Quantum Software

Ajay Kumar, Prisha Kwatra, Sunita Garhwal

Abstract Software testing is an important activity to ensure the software's quality. Mutation testing is a fault-based technique for checking the effectiveness of test cases for the verification and validation of software. Testing quantum programs is challenging due to the involvement of quantum mechanics concepts (such as superposition and entanglement ) and the probabilistic nature of the outcome. In this paper, We will propose the modified mutation testing approach for quantum program. The proposed approach will help to reduce the cost, time, and effort of performing the mutation testing. Moreover, we also propose a method for determining the equivalent mutant and developing a tool to check whether the mutated and original quantum program behave semantically equivalent or differently.

•May 30, 2023•1 citation

Development of a Tool for Finding Equivalent Mutants in Quantum Program: A Perspective to Measure the Quality of Quantum Software

Ajay Kumar, Prisha Kwatra, Sunita Garhwal

Abstract Software testing is an important activity to ensure the software's quality. Mutation testing is a fault-based technique for checking the effectiveness of test cases for the verification and validation of software. Testing quantum programs is challenging due to the involvement of quantum mechanics concepts (such as superposition and entanglement ) and the probabilistic nature of the outcome. In this paper, We will propose the modified mutation testing approach for quantum program. The proposed approach will help to reduce the cost, time, and effort of performing the mutation testing. Moreover, we also propose a method for determining the equivalent mutant and developing a tool to check whether the mutated and original quantum program behave semantically equivalent or differently.

•Dec 9, 2022•2 citations
There is an inherent lack of knowledge and technology to test a quantum program properly. In this paper, building on the definition of syntactically equivalent quantum gates, we describe our efforts in developing a tool, coined QMutPy, leveraging the well-known open-source mutation tool MutPy. We further discuss the design and implementation of QMutPy, and the usage of a novel set of mutation operators that generate mutants for qubit measurements and gates. To evaluate QMutPy’s performance, we conducted a preliminary study on 11 real quantum programs written in the IBM’s Qiskit library. QMutPy has proven to be an effective quantum mutation tool, providing insight into the current state of quantum tests. QMutPy is publicly available at https://github.com/danielfobooss/mutpy. Tool demo: https://youtu.be/fC4tOY5trqc.
There is an inherent lack of knowledge and technology to test a quantum program properly. In this paper, building on the definition of syntactically equivalent quantum operations, we investigated a novel set of mutation operators to generate mutants based on qubit measurements and quantum gates. To ease the adoption of quantum mutation testing, we further discuss QMutPy, an extension of the well-known and fully automated open-source mutation tool MutPy. To evaluate QMutPy's performance we conducted a case study on 11 real quantum programs written in the IBM's QISKit library. QMutPy has proven to be an effective quantum mutation tool, providing insight on the current state of quantum tests.
There is an inherent lack of knowledge and technology to test a quantum program properly. In this paper, building on the definition of syntactically equivalent quantum operations, we investigated a novel set of mutation operators to generate mutants based on qubit measurements and quantum gates. To ease the adoption of quantum mutation testing, we further discuss QMutPy, an extension of the well-known and fully automated open-source mutation tool MutPy. To evaluate QMutPy’s performance we conducted a case study on 11 real quantum programs written in the IBM’s QISKit library. QMutPy has proven to be an effective quantum mutation tool, providing insight on the current state of quantum tests.
As quantum computing is still in its infancy, there is an inherent lack of knowledge and technology to test a quantum program properly. In the classical realm, mutation testing has been successfully used to evaluate how well a program's test suite detects seeded faults (i.e., mutants). In this paper, building on the definition of syntactically equivalent quantum operations, we propose a novel set of mutation operators to generate mutants based on qubit measurements and quantum gates. To ease the adoption of quantum mutation testing, we further propose QMUTPY, an extension of the well-known and fully automated open-source mutation tool MUTPY. To evaluate QMUTPY's performance, we conducted a case study on 24 real quantum programs written in IBM's QISKIT library. Furthermore, we show how better test suite coverage and improvements to test assertions can increase the test suites' mutation score and quality. QMUTPY has proven to be an effective quantum mutation tool, providing insight into the current state of quantum tests and how to improve them.
Given that quantum software testing is a new area of research, there is a lack of benchmark programs and bugs repositories to assess the effectiveness of testing techniques. To this end, quantum mutation analysis focuses on systematically generating faulty versions of Quantum Programs (QPs), called mutants, using mutation operators. Such mutants can be used as benchmarks to assess the quality of test cases in a test suite. Thus, we present Muskit - a quantum mutation analysis tool for QPs coded in IBM's Qiskit language. Muskit defines mutation operators on gates of QPs and selection criteria to reduce the number of mutants to generate. Moreover, it allows for the execution of test cases on mutants and generation of results for test analyses. Muskit is provided as command line interface, GUI, and web application. We validated Muskit by using it to generate and execute mutants for four QPs. Muskit code: https://github.com/Simula-COMPLEX/muskitWeb app: https://qiskitmutantcreatorsrl.pythonanywhere.com/YouTube Video: EbPHJOK_AEA Artifact Available: https://doi.org/10.5281/zenodo.5288917
Abstract Software testing is an important activity to ensure the software's quality. Mutation testing is a fault-based technique for checking the effectiveness of test cases for the verification and validation of software. Testing quantum programs is challenging due to the involvement of quantum mechanics concepts (such as superposition and entanglement ) and the probabilistic nature of the outcome. In this paper, We will propose the modified mutation testing approach for quantum program. The proposed approach will help to reduce the cost, time, and effort of performing the mutation testing. Moreover, we also propose a method for determining the equivalent mutant and developing a tool to check whether the mutated and original quantum program behave semantically equivalent or differently.
Abstract Software testing is an important activity to ensure the software's quality. Mutation testing is a fault-based technique for checking the effectiveness of test cases for the verification and validation of software. Testing quantum programs is challenging due to the involvement of quantum mechanics concepts (such as superposition and entanglement ) and the probabilistic nature of the outcome. In this paper, We will propose the modified mutation testing approach for quantum program. The proposed approach will help to reduce the cost, time, and effort of performing the mutation testing. Moreover, we also propose a method for determining the equivalent mutant and developing a tool to check whether the mutated and original quantum program behave semantically equivalent or differently.