Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2604.26609

Probabilistic Condition, Decision and Path Coverage of Circuit-based Quantum Programs

Daniel Fortunato, José Campos, Rui Abreu

quant-phcs.SESubmitted Apr 29, 2026• Announced Apr 30, 2026• 0 citations
arXivHTMLPDF

Abstract

Coverage criteria play a central role in assessing test adequacy in classical software, yet their effectiveness for quantum programs remains poorly understood and largely unexplored. In this paper, we propose six quantum-tailored criteria - condition, decision, and path coverage, and their probabilistic variants - adapted from their classical counterparts. We present QaCoCo, a tool that computes these criteria for circuit-based quantum programs. We empirically evaluate these criteria on a large and diverse set of 540 circuits and analyze the coverage achieved. Our results show that while circuits frequently achieve high condition and decision coverage (97.56% and 97.63%, on average), path coverage remains limited (71.84%), particularly in the presence of multi-controlled gates, which induce extreme path explosion and coverage imbalance. Moreover, to account for the probabilistic nature of quantum circuits, we introduce probabilistic coverage, which augments structural coverage with a confidence measure (88.87%, 88.65%, and 37.18% for condition, decision, and path coverage, respectively, on average). Finally, through mutation testing, we find weak or no correlation between fault detection and structural coverage, consistent with observations in classical computing.

Comments
12 pages, 4 figures, 1 table

Related papers

Assessing the Effectiveness of Input and Output Coverage Criteria for Testing Quantum Programs

Shaukat Ali, Paolo Arcaini, Xinyi Wang +1 more

Quantum programs implement quantum algorithms solving complex computational problems. Testing such programs is challenging due to the inherent characteristics of Quantum Computing (QC), such as the probabilistic nature and computations in superposition. However, automated and systematic testing is needed to ensure the correct behavior of quantum programs. To this end, we present an approach called Quito (QUantum InpuT Output coverage) consisting of three coverage criteria defined on the inputs and outputs of a quantum program, together with their test generation strategies. Moreover, we define two types of test oracles, together with a procedure to determine the passing and failing of test suites with statistical analyses. To evaluate the cost-effectiveness of the three coverage criteria, we conducted experiments with five quantum programs. We used mutation analysis to determine the coverage criteria' effectiveness and cost in terms of the number of test cases. Based on the results of mutation analysis, we also identified equivalent mutants for quantum programs.

IEEE International Conference on Software Testing, Verification and Validation (ICST)•Apr 1, 2021•82 citations

Quantum circuit mutants: Empirical analysis and recommendations

Eñaut Mendiluze Usandizaga, Shaukat Ali, Tao Yue +1 more

Abstract As a new research area, quantum software testing lacks systematic testing benchmarks to assess testing techniques’ effectiveness. Recently, some open-source benchmarks and mutation analysis tools have emerged. However, there is insufficient evidence on how various quantum circuit characteristics (e.g., circuit depth, number of quantum gates), algorithms (e.g., Quantum Approximate Optimization Algorithm), and mutation characteristics (e.g., mutation operators) affect the detection of mutants in quantum circuits. Studying such relations is important to systematically design faulty benchmarks with varied attributes (e.g., the difficulty in detecting a seeded fault) to facilitate assessing the cost-effectiveness of quantum software testing techniques efficiently. To this end, we present a large-scale empirical evaluation with more than 700K faulty benchmarks (quantum circuits) generated by mutating 382 real-world quantum circuits. Based on the results, we provide valuable insights for researchers to define systematic quantum mutation analysis techniques. We also provide a tool to recommend mutants to users based on chosen characteristics (e.g., a quantum algorithm type) and the required difficulty of detecting mutants. Finally, we also provide faulty benchmarks that can already be used to assess the cost-effectiveness of quantum software testing techniques.

arXiv:2311.16913•Empirical Software Engineering•Apr 16, 2025•9 citations

Detection and diagnosis of faulty quantum circuits

Alexandru Paler, Ilia Polian, John P. Hayes

A new approach to detecting and diagnosing faults in quantum circuits is introduced. In order to account for the probabilistic nature of quantum circuits, collections of test experiments, called binary tomographic tests (BTTs), are generated. A BTT can identify a fault with respect to some user-defined confidence threshold τ. We present an algorithm to generate BTTs that either detect, or ensure the absence of, all modeled faults in a given circuit. We also present an adaptive diagnostic method to locate quantum faults. While classical circuits, even probabilistic ones, only handle ordinary probabilities, quantum circuits deal with quantum states, which have phase as an extra probabilistic parameter. The tomographic testing methods introduced previously for probabilistic circuits are unable to detect differences in phase, and therefore leave many quantum faults undetected. In contrast, we develop a design-for-test method which is specifically intended to detect faults that only affect the phase of a quantum state. We give experimental results for benchmark and random circuits which show high coverage of quantum faults by BTTs, and good resolution in the case of the adaptive diagnosis method.

Asia and South Pacific Design Automation Conference (ASP-DAC)•Jan 1, 2012•16 citations

Toward Circuit-Aware Mutation Testing for Quantum Programs: Exact Visibility and Execution Planning

Andriy Miranskyy, Jose Campos

Mutation testing assesses test suites by introducing program changes and checking whether tests detect them. In quantum programs, a surviving mutant may leave program behavior unchanged, produce a change hidden by the chosen input or measurement, or remain undetected because statistical evidence is insufficient. We develop a circuit-aware framework to distinguish these causes and guide execution planning and test changes. The framework covers ideal, noiseless programs and mutations that replace one gate or subcircuit. The method traces how a mutation can affect measured outputs and combines established mathematical checks with statistical estimates. For supported cases, we establish conditions under which a test cannot reveal a mutation and exact formulas for its effect when the required state information is available. Under stated assumptions, we bound the mutation score while retaining surviving and unresolved mutants, and identify a restricted circuit family whose mutation effects admit efficient classical classification. The deliverables include a decision procedure, mathematical guarantees, worked examples of test changes and their implementation requirements, and a small-circuit reference implementation for differential expectation tests. These results help testers decide what to execute, what to change, and what remains unknown. The evidence is limited to proofs and executable examples under the ideal model.

cs.SECross-listedarXiv:2610.36976•Oct 28, 2026•0 citations

A Survey of Functional Testing and Validation of Quantum Circuits

Emma Andrews, Aruna Jayasena, Prabhat Mishra

Quantum computing has the potential to solve a wide variety of classically intractable problems. However, the inherent complexity, probabilistic behavior, and fragility of quantum systems pose significant challenges to ensuring the correctness and reliability of quantum programs. In this survey, we present a comprehensive overview of functional testing and validation techniques aimed at addressing these challenges for quantum circuits. We begin by outlining the unique characteristics of quantum systems that complicate traditional testing methodologies, including the absence of classical oracles, non-determinism due to quantum measurement, and the exponential growth of the state space. We then categorize and analyze a wide range of functional testing and validation techniques that have emerged to meet these challenges, including mutation testing, fuzz testing, and coverage testing. We further explore advanced semantic and white-box methods such as property-based testing, metamorphic testing, equivalence checking, fault testing, assertions, and concolic testing. Specifically, we discuss the applicability of these testing techniques to quantum circuits and examine their limitations in the current NISQ (Noisy Intermediate-Scale Quantum) era.

IEEE Design and Test•May 29, 2026•0 citations

QMutBench: A Dataset of Quantum Circuit Mutants

Eñaut Mendiluze Usandizaga, Thomas Laurent, Paolo Arcaini +1 more

Quantum software testing has attracted interest in recent years, prompting the development of various techniques to automate the testing of quantum software. These techniques generate test cases that must be assessed for their effectiveness in detecting faults. Such an assessment requires benchmarks of faulty programs. However, there is a lack of benchmarks containing faults. In this data showcase, we propose QMutBench, a dataset that contains over 700,000 quantum circuit mutants representing different faults. The dataset is accessible via an online interface with selection criteria, such as the original quantum circuit(s) from which mutants are generated, the desired survival rate of the selected mutants, and other mutation characteristics (e.g., the type of faulty quantum gate). QMutBench provides quantum software developers and testers with an accessible online dataset to obtain benchmarks of mutants necessary to assess either the quality of the test cases generated by their testing technique or to compare different testing techniques. It also enables the development of new mutation-guided quantum software testing techniques.

arXiv:2604.15870•Apr 17, 2026•0 citations
Coverage criteria play a central role in assessing test adequacy in classical software, yet their effectiveness for quantum programs remains poorly understood and largely unexplored. In this paper, we propose six quantum-tailored criteria - condition, decision, and path coverage, and their probabilistic variants - adapted from their classical counterparts. We present QaCoCo, a tool that computes these criteria for circuit-based quantum programs. We empirically evaluate these criteria on a large and diverse set of 540 circuits and analyze the coverage achieved. Our results show that while circuits frequently achieve high condition and decision coverage (97.56% and 97.63%, on average), path coverage remains limited (71.84%), particularly in the presence of multi-controlled gates, which induce extreme path explosion and coverage imbalance. Moreover, to account for the probabilistic nature of quantum circuits, we introduce probabilistic coverage, which augments structural coverage with a confidence measure (88.87%, 88.65%, and 37.18% for condition, decision, and path coverage, respectively, on average). Finally, through mutation testing, we find weak or no correlation between fault detection and structural coverage, consistent with observations in classical computing.
Quantum programs implement quantum algorithms solving complex computational problems. Testing such programs is challenging due to the inherent characteristics of Quantum Computing (QC), such as the probabilistic nature and computations in superposition. However, automated and systematic testing is needed to ensure the correct behavior of quantum programs. To this end, we present an approach called Quito (QUantum InpuT Output coverage) consisting of three coverage criteria defined on the inputs and outputs of a quantum program, together with their test generation strategies. Moreover, we define two types of test oracles, together with a procedure to determine the passing and failing of test suites with statistical analyses. To evaluate the cost-effectiveness of the three coverage criteria, we conducted experiments with five quantum programs. We used mutation analysis to determine the coverage criteria' effectiveness and cost in terms of the number of test cases. Based on the results of mutation analysis, we also identified equivalent mutants for quantum programs.
Abstract As a new research area, quantum software testing lacks systematic testing benchmarks to assess testing techniques’ effectiveness. Recently, some open-source benchmarks and mutation analysis tools have emerged. However, there is insufficient evidence on how various quantum circuit characteristics (e.g., circuit depth, number of quantum gates), algorithms (e.g., Quantum Approximate Optimization Algorithm), and mutation characteristics (e.g., mutation operators) affect the detection of mutants in quantum circuits. Studying such relations is important to systematically design faulty benchmarks with varied attributes (e.g., the difficulty in detecting a seeded fault) to facilitate assessing the cost-effectiveness of quantum software testing techniques efficiently. To this end, we present a large-scale empirical evaluation with more than 700K faulty benchmarks (quantum circuits) generated by mutating 382 real-world quantum circuits. Based on the results, we provide valuable insights for researchers to define systematic quantum mutation analysis techniques. We also provide a tool to recommend mutants to users based on chosen characteristics (e.g., a quantum algorithm type) and the required difficulty of detecting mutants. Finally, we also provide faulty benchmarks that can already be used to assess the cost-effectiveness of quantum software testing techniques.
A new approach to detecting and diagnosing faults in quantum circuits is introduced. In order to account for the probabilistic nature of quantum circuits, collections of test experiments, called binary tomographic tests (BTTs), are generated. A BTT can identify a fault with respect to some user-defined confidence threshold τ. We present an algorithm to generate BTTs that either detect, or ensure the absence of, all modeled faults in a given circuit. We also present an adaptive diagnostic method to locate quantum faults. While classical circuits, even probabilistic ones, only handle ordinary probabilities, quantum circuits deal with quantum states, which have phase as an extra probabilistic parameter. The tomographic testing methods introduced previously for probabilistic circuits are unable to detect differences in phase, and therefore leave many quantum faults undetected. In contrast, we develop a design-for-test method which is specifically intended to detect faults that only affect the phase of a quantum state. We give experimental results for benchmark and random circuits which show high coverage of quantum faults by BTTs, and good resolution in the case of the adaptive diagnosis method.
Mutation testing assesses test suites by introducing program changes and checking whether tests detect them. In quantum programs, a surviving mutant may leave program behavior unchanged, produce a change hidden by the chosen input or measurement, or remain undetected because statistical evidence is insufficient. We develop a circuit-aware framework to distinguish these causes and guide execution planning and test changes. The framework covers ideal, noiseless programs and mutations that replace one gate or subcircuit. The method traces how a mutation can affect measured outputs and combines established mathematical checks with statistical estimates. For supported cases, we establish conditions under which a test cannot reveal a mutation and exact formulas for its effect when the required state information is available. Under stated assumptions, we bound the mutation score while retaining surviving and unresolved mutants, and identify a restricted circuit family whose mutation effects admit efficient classical classification. The deliverables include a decision procedure, mathematical guarantees, worked examples of test changes and their implementation requirements, and a small-circuit reference implementation for differential expectation tests. These results help testers decide what to execute, what to change, and what remains unknown. The evidence is limited to proofs and executable examples under the ideal model.
Quantum computing has the potential to solve a wide variety of classically intractable problems. However, the inherent complexity, probabilistic behavior, and fragility of quantum systems pose significant challenges to ensuring the correctness and reliability of quantum programs. In this survey, we present a comprehensive overview of functional testing and validation techniques aimed at addressing these challenges for quantum circuits. We begin by outlining the unique characteristics of quantum systems that complicate traditional testing methodologies, including the absence of classical oracles, non-determinism due to quantum measurement, and the exponential growth of the state space. We then categorize and analyze a wide range of functional testing and validation techniques that have emerged to meet these challenges, including mutation testing, fuzz testing, and coverage testing. We further explore advanced semantic and white-box methods such as property-based testing, metamorphic testing, equivalence checking, fault testing, assertions, and concolic testing. Specifically, we discuss the applicability of these testing techniques to quantum circuits and examine their limitations in the current NISQ (Noisy Intermediate-Scale Quantum) era.
Quantum software testing has attracted interest in recent years, prompting the development of various techniques to automate the testing of quantum software. These techniques generate test cases that must be assessed for their effectiveness in detecting faults. Such an assessment requires benchmarks of faulty programs. However, there is a lack of benchmarks containing faults. In this data showcase, we propose QMutBench, a dataset that contains over 700,000 quantum circuit mutants representing different faults. The dataset is accessible via an online interface with selection criteria, such as the original quantum circuit(s) from which mutants are generated, the desired survival rate of the selected mutants, and other mutation characteristics (e.g., the type of faulty quantum gate). QMutBench provides quantum software developers and testers with an accessible online dataset to obtain benchmarks of mutants necessary to assess either the quality of the test cases generated by their testing technique or to compare different testing techniques. It also enables the development of new mutation-guided quantum software testing techniques.