Testing can be key to software quality assurance. Automated verification may increase throughput and reduce human fallibility errors. Test scripts supply inputs, run programs and check their outputs mechanically using test oracles. In software engineering implicit oracles automatically check for universally undesirable behaviour, such as the software under test crashing. We propose 4 properties (probability distributions, fixed qubit width, reversibility and entropy conservation) which all quantum computing must have and suggest they could be implicit test oracles for automatic, random, or fuzz testing of quantum circuits and simulators of quantum programs.
Testing can be key to software quality assurance. Automated verification may increase throughput and reduce human fallibility errors. Test scripts supply inputs, run programs and check their outputs mechanically using test oracles. In software engineering implicit oracles automatically check for universally undesirable behaviour, such as the software under test crashing. We propose 4 properties (probability distributions, fixed qubit width, reversibility and entropy conservation) which all quantum computing must have and suggest they could be implicit test oracles for automatic, random, or fuzz testing of quantum circuits and simulators of quantum programs.
As quantum computing transitions from theoretical concepts to commercially available quantum processing units, ensuring the quality, reliability, and correctness of quantum software systems has become paramount. The unique challenges of quantum software testing have necessitated the adaptation of classical software engineering techniques to the quantum domain. One such technique is fuzz testing, or fuzzing, an automated software testing technique that generates large volumes of inputs to identify bugs and unexpected behaviors. In this work we provide a comprehensive examination of how fuzz testing has been successfully applied to quantum computing. We analyze the application of fuzzing across the quantum ecosystem, from testing quantum programs to verifying quantum software stacks, quantum compilers, and quantum simulators. This survey establishes the foundation for understanding how fuzz testing can ensure the reliability and correctness of quantum software systems and outlines promising future research directions.
Quantum computing will make it possible to exponentially accelerate the performance of a wide range of computational problems, such as cryptography, machine learning or chemical simulation. However, the quantum potential is not only a matter of hardware, but also of software. Therefore, this new paradigm has an impact yet to be explored on software development processes and techniques, and the adaptation of classical software engineering to the new classical/quantum hybrid systems raises a number of important challenges: a new Quantum Software Engineering is therefore needed. Specifically, and focusing on quantum software quality, software verification remains an open research question, as its novelty and complexity make quantum software development a particularly error-prone process. Most current approaches to test-driven verification rely heavily on simulations, which is a problem due to the lack of scalability of simulators running on classical computers. To address this shortcoming, we define the concept of a "Quantum Test Case", and then present a method to test quantum circuits on real machines, without using simulation test functionalities such as amplitude calculation or non-destructive measurement. This is achieved by automatically generating a Quantum Test Case, which wraps the circuit under test and performs the verification. We also present the process to run a set of tests on a circuit with this method, along with an example to illustrate the technique.
Quantum software has entered production use, but its reliability remains sensitive. Bugs in compilers, simulators, and algorithm-layer source codes do not lead to program crashes; rather, they emerge as faulty probability distributions, successfully compiled but incorrect circuits, and differences between platforms, which classical testing oracles are incapable of noticing. This paper surveys quantum fuzzing as a response to that gap. This paper is organize in the field along the classical grey-box, white-box, and black-box axis and trace, for each category, the sequence of tools that have advanced the state of the art QuanTest, QSolver, and NovaQ representing the current frontiers respectively. Characterized the defect landscape these tools have collectively exposed, showing that bugs concentrate disproportionately in compilers and transpilation passes, that simulator and algorithm-layer faults form a distinct oracle-evading class, and that roughly 40% of confirmed quantum bugs are quantum-specific in symptom. Across all categories mentioned the same that crash-based detection is structurally inadequate for quantum software, and that the most productive fuzzers replace it with statistical, metamorphic, entanglement-guided, or formal-invariant oracles. We close by identifying the unification of these oracle strategies within a single campaign as the principal open challenge for the field.
Quantum software stacks (QSSs), which provide quantum circuit transformers and simulators, enable circuit transformations and the execution of circuits on classical computers. Despite their importance, they have not been effectively tested yet, leaving the correctness in question. The main obstacle to testing is the absence of a testing oracle, which checks the semantics-preservation of circuit transformations and the correctness of simulation results. While previous studies have employed differential and metamorphic testing to circumvent the necessity for an oracle, they have detected very few non-crash bugs. In this work, we address this gap by introducing QASMInfer, an exact inference system for quantum circuits, which computes the probability distribution of possible circuit outcomes. By supporting circuits written in OpenQASM, the de facto standard quantum assembly language used by most QSSs, QASMInfer acts as a unified testing oracle for multiple QSSs. Our design of QASMInfer achieves three key goals: (1) support for dynamic circuits, an important class of quantum circuits, (2) efficiency, and (3) reliability. For efficiency, we introduce two optimizations and an efficient matrix representation. For reliability, we prove physical consistency, ensuring that QASMInfer’s inference results adhere to the physical principles of quantum computing. To simplify the proof, we introduce OpenQASMCore, a core language for OpenQASM, and perform exact inference for OpenQASM by desugaring it to OpenQASMCore. Our implementation and proof are fully mechanized in the Coq proof assistant. Testing six real-world QSSs using QASMInfer revealed 31 bugs, including 20 non-crash bugs, demonstrating QASMInfer’s effectiveness as a testing oracle.
As quantum software engineering advances, testing techniques are required to assess the quality of quantum programs (QPs). In the test process, the test oracle is vital for determining whether the test result indicates a success or a failure. Most related works directly measure the output states and acquire the corresponding test results by comparing the output distribution with the expected one. While attention has been paid to the capability of fault detection, the guarantee for the correctness of the produced test results remains limited. Unlike classical programs (CPs), the output quantum states of QPs should be transformed into probabilistic classical outcomes through quantum measurement. This additional operation of measurement could cause a test oracle to yield the wrong test results. Especially for high-dimensional output spaces, numerous measurement outcomes are required to capture the distribution characteristics, threatening the effectiveness and cost-efficiency of test oracles. Hence, this paper proposes a novel specified test oracle DOSS employing a dynamic scheme to integrate a quantum algorithm (i.e., swap test) with the direct measurement mode. This innovative approach enables the validation of individual outputs rather than their distribution during the testing phase. Considering acceptable cost, DOSS decomposes the fully or partially separable output states to lower the dimensionality and simplify the quantum circuit for testing. Empirical studies demonstrate that DOSS generally gives more correct test results than baselines, and maintains reasonable cost on an ideal simulator. Besides, DOSS’s effectiveness with quantum noise involved is validated via three noisy simulators.
A typical oracle problem is finding which software program is installed on a computer, by running the computer and testing its input-output behaviour. The program is randomly chosen from a set of programs known to the problem solver. As well known, some oracle problems are solved more efficiently by using quantum algorithms; this naturally implies changing the computer to quantum, while the choice of the software program remains sharp. In order to highlight the non-mechanistic origin of this higher efficiency, also the uncertainty about which program is installed must be represented in a quantum way.