Manually debugging quantum programs is a difficult and time-intensive process. In this paper, we introduce an automated debugging technique, based on delta debugging and property-based testing, for quantum programs. Our technique automatically identifies the changes made within an update to a quantum program that cause a property-based regression test to fail. To evaluate our technique, we inject faults and semantic preserving changes into three quantum algorithms. We discuss the viability and efficacy of our approach after measuring the percentage of faults and semantic preserving changes. Our results indicate that our method has a high true positive (called sensitivity) and true negative rate (called specificity) and is robust in terms of the amount of changes introduced to the program. Moreover, the sensitivity of the method increases significantly with the number of properties. While the specificity remains stable when increasing the number of properties and inputs.
Manually debugging quantum programs is a difficult and time-intensive process. In this paper, we introduce an automated debugging technique, based on delta debugging and property-based testing, for quantum programs. Our technique automatically identifies the changes made within an update to a quantum program that cause a property-based regression test to fail. To evaluate our technique, we inject faults and semantic preserving changes into three quantum algorithms. We discuss the viability and efficacy of our approach after measuring the percentage of faults and semantic preserving changes. Our results indicate that our method has a high true positive (called sensitivity) and true negative rate (called specificity) and is robust in terms of the amount of changes introduced to the program. Moreover, the sensitivity of the method increases significantly with the number of properties. While the specificity remains stable when increasing the number of properties and inputs.
Recent advancements in quantum computing software are gradually increasing the scope and size of quantum programs being developed. At the same time, however, these larger programs provide more possibilities for functional errors that are harder to detect and resolve. Meanwhile, debugging tools that could aid developers in resolving these errors are still barely existent and far from what we take for granted in classical design automation and software engineering. As a result, even if one manages to identify the incorrect behavior of a developed quantum program, detecting and resolving the underlying errors in the program remains a time-consuming and tedious task. Moreover, the exponential growth of the state space in quantum programs makes the efficient manual investigation of errors radically difficult even for respectively simple algorithms, and almost impossible as the number of qubits increases. To address this problem, this work proposes a debugging framework, available as an open-source implementation at https://github.com/cda-tum/mqt-debugger. It assists developers in debugging errors in quantum programs, allowing them to efficiently identify the existence of errors and diagnose their causes. Users are given the ability to place assertions in the code that test for the correctness of a given algorithm and are evaluated using classical simulations of the underlying quantum program. Once an assertion fails, the proposed framework employs different diagnostic methods to point towards possible error causes. This way, the debugging workload for quantum programs is drastically reduced.
As new advancements in the field of quantum computing lead to the development of increasingly complex programs, approaches to validate and debug these programs are becoming more important. To this end, methods employed in classical debugging, such as assertions for testing specific properties of a program's state, have been adapted for quantum programs. However, to efficiently debug quantum programs, it is key to properly place these assertions. This usually requires a deep understanding of the program's underlying mathematical properties, constituting a time-consuming manual task for developers. To address this problem, this work proposes methods for automatically refining assertions in quantum programs by moving them to more favorable positions in the program or by placing new assertions that help to further narrow down potential error locations. This allows developers to take advantage of rich and expressive assertions that greatly improve the debugging experience without requiring them to place these assertions manually in an otherwise tedious manner. An opensource implementation of the proposed methods is available at https://github.com/cda-tum/mqt-debugger.
Property-based testing has been previously proposed for quantum programs in Q# with QSharpCheck; however, this implementation was limited in functionality, lacked extensibility, and was evaluated on a narrow range of programs using a single property. To address these limitations, we propose QuCheck, an improved property-based testing framework for Qiskit. By leveraging Qiskit and the broader Python ecosystem, QuCheck facilitates property construction, introduces flexible input generators and assertions, and supports expressive preconditions. We assessed its effectiveness through mutation analysis on five quantum programs (2-10 qubits), varying the number of properties, inputs, and measurement shots to assess their impact on fault detection and demonstrate the effectiveness of property-based testing across a range of conditions. Results show a strong positive correlation between the mutation score (a measure of fault detection) and number of properties evaluated, with a moderate negative correlation between the false positive rate and number of measurement shots. The most thorough test configurations achieved a mean mutation score of 0.90 averaged across all five algorithms, with a false positive rate between 0 and 0.06. QuCheck identified 47.8% more faults than QSharpCheck, with execution time reduced by 67.4%. These findings highlight the viability of property-based testing for verifying quantum systems.
Quantum computing has existed in the theoretical realm for several decades. Recently, quantum computing has re-emerged as a promising technology to solve problems that a classical computer could take hundreds of years to solve. However, there are challenges and opportunities for academics and practitioners regarding software engineering practices for testing and debugging quantum programs. This article presents a roadmap for addressing these challenges, pointing out the existing gaps in the literature and suggesting research directions. We discuss the limitations caused by noise, the no-cloning theorem, the lack of a standard architecture for quantum computers, among others. Regarding testing, we highlight gaps and opportunities related to transpilation, mutation analysis, input states with hybrid interfaces, program analysis, and coverage. For debugging, we present the current strategies, including classical techniques applied to quantum programs, quantum-specific assertions, and quantum-related bug patterns. We introduce a conceptual model to illustrate concepts regarding the testing and debugging of quantum programs and the relationship between them. Those concepts are used to identify and discuss research challenges to cope with quantum programs through 2030, focusing on the interfaces between classical and quantum computing and on creating testing and debugging techniques that take advantage of the unique quantum computing characteristics.
Testing quantum programs in a black-box setting is difficult because bug-relevant behavioral differences may be missed at two levels: the chosen inputs may fail to trigger faulty behavior, and even when faults are triggered, the chosen measurements may fail to make the resulting differences observable. Existing methods have mainly strengthened either input generation or measurement design in isolation, leaving their coupling insufficiently addressed. In this paper, we present QDoctor, a probe-based black-box testing framework that wraps the program under test with a variety of property-based probes, thereby effectively extracting program information and exploring its behavior. A probe integrates two components: test-case generation and the specification of the measurement basis. To reduce false positive rate, QDoctor also supports semantic-aware partial equivalence testing through constrained probe-oracle design. Experimental results on benchmark suites show that QDoctor achieves the best balanced performance among representative black-box baselines, reaching 99.89% recall, 98.23% precision, 99.06% F1, and 0.9988 AUC. Ablation shows that jointly strengthening inputs and measurements raises recall from 85.280% to 99.728% with zero false positives, while semantic probes drive the false-positive rate to zero on SWAP-induced equivalent mutants.
In this paper, we propose a novel method for locating and diagnosing bugs in quantum circuits. Debugging in the quantum domain is especially challenging due to the inherent inability of assessing the quantum state of a program. Moreover, explaining the root cause behind unexpected outcomes is hard due to the limited information gain provided by measurements. Our approach aims to address both of these issue: Firstly, the bug site is identified using a standard circuit slicing technique combined with an associated measurement strategy. Secondly, we provide information about the nature of the bug, generated through repeated measurements. To minimize the number of measurements, we introduce a notion of equivalence classes based on unitary operations. This allows us to partition the gate library into classes that produce indistinguishable results under certain measurements. Finally, We assess the effectiveness and measurement complexity of our method by applying it to relevant primitive gate components and well-known quantum algorithms. Our empirical results shows that in 95.79% of all cases, our approach reveals the correct location of the bug along with a valid set of fault candidates. Furthermore, we demonstrate that the required number of circuit executions scales logarithmically with the circuit depth or linearly with the number of qubits.