Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Artifact of Quantum Squeeziness: Information-Theoretic Foundations of Quantum Software Testability

Avner Bensoussan, Héctor D. Menéndez, Mohammad Reza Mousavi

Published Dec 4, 2025• 1 citation
Publisher

Abstract

Abstract Testability—the degree to which faults can be revealed and diagnosed—has long been fundamental in Software Engineering but remains unexplored for quantum software. This work presents the first formalisation of testability for quantum programs, adapting classical concepts to the unique properties of quantum computation. We introduce Quantum Squeeziness, an information-theoretic metric inspired by the classical squeeziness measure, to quantify how effectively faults propagate to observable failures. We provide an efficient computational method for calculating Quantum Squeeziness and validate it through large-scale quantum mutation testing. Across more than 50k mutants executed over several months, we observe a strong negative correlation between squeeziness and mutation score, demonstrating that higher squeeziness reliably predicts reduced fault detectability. Performance results show that the standard squeeziness computation is already 15% faster than baseline methods, while our optimised variant, FastSqueeziness, reduces mean runtime from 3200 s to 189 s, achieving a 16.9× speedup with 95% accuracy. By establishing a rigorous notion of testability for quantum software, this study opens a new line of research at the intersection of software engineering, quantum information theory, and testing. Beyond its immediate implications for testing and debugging, the proposed framework lays the foundation for future applications in quantum error mitigation and error correction, providing a principled means to assess and improve the robustness of quantum programs. Artifact Contents The archive Quantum_Squeeziness_AE_Kit.zip contains all materials required for ASE 2026 Artifact Evaluation. Specifically, it includes: Complete source code Benchmark quantum circuits Precomputed experimental results Docker environment Dependency specifications Documentation Replication scripts for RQ1–RQ3 A copy of the accepted paper (ase26-paper151.pdf) The artifact is self-contained and can be executed without access to external services. Precomputed results are provided for experiments whose full recomputation would require weeks or months of execution time, while all scripts necessary to regenerate those results are also included. Consequently, Quantum_Squeeziness_AE_Kit.zip constitutes the complete artifact-evaluation package and contains everything required by reviewers to assess the Available, Functional, and Reusable badges, including software, data, documentation, execution environments, and reproduction instructions. Replication Steps Setup Create a virtual environment and install the packages listed in requirements.txt. RQ1 Run: python3 testabilityBenchmark_RQ1/run_benchmark.py In line 885, you can modify folder_path to select the desired benchmark folder and run larger experiments. RQ2 Compute Quantum Squeeziness Run: python3 definingSqueeziness_RQ2/squeeziness.py to produce the Squeeziness results for a given QASM folder. Line 931: choose the origin folder. Line 932: choose the target folder. Compute FastSqueeziness Run: python3 definingSqueeziness_RQ2/fast_squeeziness.py to produce the FastSqueeziness results for a given QASM folder. Line 937: choose the origin folder. Line 938: choose the target folder. Note: Running either script on the complete mutant dataset requires several months of computation. For testing purposes, we recommend using smaller subsets such as the MQT or chosen QASM folders. Analysis Run: python3 definingSqueeziness_RQ2/analyse_dqcSqueeziness.py python3 definingSqueeziness_RQ2/evaluate_fastSqueeziness.py python3 definingSqueeziness_RQ2/statistical_analysisSqueezinessR2.py to reproduce the analysis results. RQ3 Prepare Mutation Results Unzip Results_mutants_chosen.zip and place the extracted folder inside: validatingSqueeziness_RQ3/ Alternatively, the results can be reproduced manually using: python3 validatingSqueeziness_RQ3/mutantsGeneratorBasic.py python3 validatingSqueeziness_RQ3/executeCircuit.py These scripts originate from Muskit (see the reference in line 1 of the source file). Reproducing the complete dataset manually may require several weeks of execution. Reproduce FastSqueeziness Validation Results Generate and organise result folders: python3 results/combine_folder.py Select mutants within the desired Squeeziness range: python3 validatingSqueeziness_RQ3/select_mutants.py \ --origin_csv definingSqueeziness_RQ2/correlation_analysis_chosen/Fastsqueeziness_metrics_summary.csv \ --mutant_info_folder validatingSqueeziness_RQ3/resultsFastSqueezinessMutantsChosen100Iterations \ --mutants_results_root validatingSqueeziness_RQ3/Results_mutants_chosen \ --output_folder validatingSqueeziness_RQ3/selectedMutantsChosen_FastSqueeziness \ --qasm_original_root qasm_circuits/chosen \ --qasm_mutant_root validatingSqueeziness_RQ3/Mutants_chosen Execute mutation testing: python3 validatingSqueeziness_RQ3/analyse.py \ --oracle validatingSqueeziness_RQ3/chosen_oracle \ --mutants validatingSqueeziness_RQ3/selectedMutantsChosen_FastSqueeziness/ \ --output validatingSqueeziness_RQ3/Test_killingMutants_FastSqueeziness_Chosen Produce the final correlation plots: python3 validatingSqueeziness_RQ3/correlation_squeeziness_kill_ratio.py Running these scripts recreates all necessary intermediate directories and the complete set of results. The configuration can be modified to reproduce alternative experiments (e.g., MQT datasets or standard Squeeziness results). Performance Comparison Run: python3 validatingSqueeziness_RQ3/compare_squeezinessFast.py to analyse the performance of FastSqueeziness compared with Squeeziness. Additional Materials Additional experiments, exploratory analyses, and investigations are available in the: .experimentations/ folder.

DOI
10.5281/zenodo.17815824

Related papers

Quantum Squeeziness: Information-Theoretic Foundations of Quantum Software Testability

Avner Bensoussan, Héctor D. Menéndez, Mohammadreza Ahmadi Mousavi

Testability—the degree to which faults can be revealed and diagnosed—has long been fundamental in software engineering but remains largely unexplored for quantum software. This work presents the first formalisation of testability for quantum programs, adapting classical concepts to quantum computation. We introduce Quantum Squeeziness, an information-theoretic metric inspired by classical squeeziness, to quantify how faults are masked and prevented from propagating to observable outcomes. We treat it as an inverse measure of testability: higher squeeziness implies stronger fault masking and reduced detectability. We propose an efficient method to compute Quantum Squeeziness and validate it through large-scale quantum mutation analysis. Across more than 50k mutants, we observe a strong negative correlation between squeeziness and mutation score, showing that higher squeeziness predicts lower testability. We further introduce a technique that constrains where and how perturbations are injected, reducing cost while preserving accuracy. The computation is 15% faster than a standard error-propagation baseline, while the optimised variant, FastSqueeziness, reduces mean runtime from 3200 s to 189 s (16.9×speedup) with 95% accuracy. By establishing a rigorous foundation of testability for quantum software, this work opens new directions at the intersection of software engineering, quantum information theory, and testing. Beyond testing and debugging, it also supports applications in quantum error mitigation and correction, enabling principled assessment and improvement of program robustness.

•Jul 16, 2026•0 citations

Quantum Squeeziness: An Information Theoretical Metric for Quantum Software Testability

Avner Bensoussan, Héctor D. Menéndez, Mohammad Reza Mousavi

Testability—the degree to which faults can be revealed and diagnosed—has long been fundamental in software engineering but remains largely unexplored for quantum software. This work presents the first formalisation of testability for quantum programs, adapting classical concepts to quantum computation. We introduce Quantum Squeeziness, an information-theoretic metric inspired by classical squeeziness, to quantify how faults are masked and prevented from propagating to observable outcomes. We treat it as an inverse measure of testability: higher squeeziness implies stronger fault masking and reduced detectability.

IEEE/ACM International Conference on Automated Software Engineering (ASE)•Oct 7, 2026•0 citations

Muskit: A Mutation Analysis Tool for Quantum Software Testing

Eñaut Mendiluze Usandizaga, Shaukat Ali, Paolo Arcaini +1 more

Given that quantum software testing is a new area of research, there is a lack of benchmark programs and bugs repositories to assess the effectiveness of testing techniques. To this end, quantum mutation analysis focuses on systematically generating faulty versions of Quantum Programs (QPs), called mutants, using mutation operators. Such mutants can be used as benchmarks to assess the quality of test cases in a test suite. Thus, we present Muskit - a quantum mutation analysis tool for QPs coded in IBM's Qiskit language. Muskit defines mutation operators on gates of QPs and selection criteria to reduce the number of mutants to generate. Moreover, it allows for the execution of test cases on mutants and generation of results for test analyses. Muskit is provided as command line interface, GUI, and web application. We validated Muskit by using it to generate and execute mutants for four QPs. Muskit code: https://github.com/Simula-COMPLEX/muskitWeb app: https://qiskitmutantcreatorsrl.pythonanywhere.com/YouTube Video: EbPHJOK_AEA Artifact Available: https://doi.org/10.5281/zenodo.5288917

IEEE/ACM International Conference on Automated Software Engineering (ASE)•Nov 1, 2021•69 citations

Works on My QPU: Reproducibility in Quantum Computing Research

Dominik Köster, Maja Franz, Benjamin Zec +3 more

Quantum computing research increasingly depends on complex software stacks, yet the reproducibility of published results does not receive the priority and longevity mandated by recommendations of large international scientific bodies and best practices in software-centric systems research. In this paper, we present a combined manual and automated large-scale analysis of the reproducibility landscape in quantum computing research, quantify shortcomings, and derive actionable steps forward. We manually evaluate a curated sample of 127 papers using a five-question framework that covers code availability, environment specification, documentation, hardware description, and executability. To place these findings in a broader context, we conduct an automated large-scale screening of nearly 5000 quantum computing papers for the same reproducibility indicators. Our manual analysis reveals that only 24.4% of the sampled papers provide code artefacts, and among those, 64.5% fail to execute successfully in a clean environment. This assessment is corroborated by a large-scale automated analysis that yields a consistent code availability rate of 26.8%. Further, it shows that approximately one-third of the papers with accessible code lack machine-readable environment specifications. The results in this paper indicate that reproducibility is not yet consistently achieved in quantum computing research. In response, we outline a set of practical recommendations that address the observed failure modes and illustrate how reproducibility can be improved in practice.

quant-pharXiv:2607.08348•Jul 10, 2026•0 citations

On the Reproducibility of Quantum Software Defect Datasets: A Case Study of Bugs4Q

Haruto Ohto, Yuta Ishimoto, Shinsuke Matsumoto +1 more

The reproducibility of software defect datasets is essential for obtaining reliable and comparable research results. Zhu et al. have shown that defect datasets such as Defects4J suffer from reproduction failures (i.e., reported bugs become non-reproducible) as time passes since their creation. However, it remains unclear whether these findings generalize to quantum software defect datasets. We therefore conduct a replication study of the prior work using Bugs4Q, a widely used dataset of real-world bugs in quantum programs. Our analysis includes 77,700 quantum program executions of 37 Bugs4Q artifacts across 21 core-library versions. The experimental results showed that the reproducibility of Bugs4Q dropped from 62.2% on Qiskit v0.20.1 to 16.2% on v2.3.1, the latest version as of April 1, 2026. A manual inspection of the root causes further indicated that 93.6% of the failures were dependency-related. While these findings are consistent with those of the prior work, we also observed differences. In particular, most reproduction failures in Bugs4Q cannot be resolved merely by adjusting dependency versions; instead, they require source-code modifications such as migrating import paths and API invocations. Based on this observation, we curated Bugs4Q-Robust, a patched version of Bugs4Q to restore reproducibility. Bugs4Q-Robust increases reproducibility from 16.2% to 78.4% on Qiskit v2.3.1. Our findings highlight the importance of continuous dataset maintenance in the rapidly evolving quantum software ecosystem.

arXiv:2606.27124•Jun 25, 2026•1 citation

Quantum circuit mutants: Empirical analysis and recommendations

Eñaut Mendiluze Usandizaga, Shaukat Ali, Tao Yue +1 more

Abstract As a new research area, quantum software testing lacks systematic testing benchmarks to assess testing techniques’ effectiveness. Recently, some open-source benchmarks and mutation analysis tools have emerged. However, there is insufficient evidence on how various quantum circuit characteristics (e.g., circuit depth, number of quantum gates), algorithms (e.g., Quantum Approximate Optimization Algorithm), and mutation characteristics (e.g., mutation operators) affect the detection of mutants in quantum circuits. Studying such relations is important to systematically design faulty benchmarks with varied attributes (e.g., the difficulty in detecting a seeded fault) to facilitate assessing the cost-effectiveness of quantum software testing techniques efficiently. To this end, we present a large-scale empirical evaluation with more than 700K faulty benchmarks (quantum circuits) generated by mutating 382 real-world quantum circuits. Based on the results, we provide valuable insights for researchers to define systematic quantum mutation analysis techniques. We also provide a tool to recommend mutants to users based on chosen characteristics (e.g., a quantum algorithm type) and the required difficulty of detecting mutants. Finally, we also provide faulty benchmarks that can already be used to assess the cost-effectiveness of quantum software testing techniques.

arXiv:2311.16913•Empirical Software Engineering•Apr 16, 2025•9 citations
Abstract Testability—the degree to which faults can be revealed and diagnosed—has long been fundamental in Software Engineering but remains unexplored for quantum software. This work presents the first formalisation of testability for quantum programs, adapting classical concepts to the unique properties of quantum computation. We introduce Quantum Squeeziness, an information-theoretic metric inspired by the classical squeeziness measure, to quantify how effectively faults propagate to observable failures. We provide an efficient computational method for calculating Quantum Squeeziness and validate it through large-scale quantum mutation testing. Across more than 50k mutants executed over several months, we observe a strong negative correlation between squeeziness and mutation score, demonstrating that higher squeeziness reliably predicts reduced fault detectability. Performance results show that the standard squeeziness computation is already 15% faster than baseline methods, while our optimised variant, FastSqueeziness, reduces mean runtime from 3200 s to 189 s, achieving a 16.9× speedup with 95% accuracy. By establishing a rigorous notion of testability for quantum software, this study opens a new line of research at the intersection of software engineering, quantum information theory, and testing. Beyond its immediate implications for testing and debugging, the proposed framework lays the foundation for future applications in quantum error mitigation and error correction, providing a principled means to assess and improve the robustness of quantum programs. Artifact Contents The archive Quantum_Squeeziness_AE_Kit.zip contains all materials required for ASE 2026 Artifact Evaluation. Specifically, it includes: Complete source code Benchmark quantum circuits Precomputed experimental results Docker environment Dependency specifications Documentation Replication scripts for RQ1–RQ3 A copy of the accepted paper (ase26-paper151.pdf) The artifact is self-contained and can be executed without access to external services. Precomputed results are provided for experiments whose full recomputation would require weeks or months of execution time, while all scripts necessary to regenerate those results are also included. Consequently, Quantum_Squeeziness_AE_Kit.zip constitutes the complete artifact-evaluation package and contains everything required by reviewers to assess the Available, Functional, and Reusable badges, including software, data, documentation, execution environments, and reproduction instructions. Replication Steps Setup Create a virtual environment and install the packages listed in requirements.txt. RQ1 Run: python3 testabilityBenchmark_RQ1/run_benchmark.py In line 885, you can modify folder_path to select the desired benchmark folder and run larger experiments. RQ2 Compute Quantum Squeeziness Run: python3 definingSqueeziness_RQ2/squeeziness.py to produce the Squeeziness results for a given QASM folder. Line 931: choose the origin folder. Line 932: choose the target folder. Compute FastSqueeziness Run: python3 definingSqueeziness_RQ2/fast_squeeziness.py to produce the FastSqueeziness results for a given QASM folder. Line 937: choose the origin folder. Line 938: choose the target folder. Note: Running either script on the complete mutant dataset requires several months of computation. For testing purposes, we recommend using smaller subsets such as the MQT or chosen QASM folders. Analysis Run: python3 definingSqueeziness_RQ2/analyse_dqcSqueeziness.py python3 definingSqueeziness_RQ2/evaluate_fastSqueeziness.py python3 definingSqueeziness_RQ2/statistical_analysisSqueezinessR2.py to reproduce the analysis results. RQ3 Prepare Mutation Results Unzip Results_mutants_chosen.zip and place the extracted folder inside: validatingSqueeziness_RQ3/ Alternatively, the results can be reproduced manually using: python3 validatingSqueeziness_RQ3/mutantsGeneratorBasic.py python3 validatingSqueeziness_RQ3/executeCircuit.py These scripts originate from Muskit (see the reference in line 1 of the source file). Reproducing the complete dataset manually may require several weeks of execution. Reproduce FastSqueeziness Validation Results Generate and organise result folders: python3 results/combine_folder.py Select mutants within the desired Squeeziness range: python3 validatingSqueeziness_RQ3/select_mutants.py \ --origin_csv definingSqueeziness_RQ2/correlation_analysis_chosen/Fastsqueeziness_metrics_summary.csv \ --mutant_info_folder validatingSqueeziness_RQ3/resultsFastSqueezinessMutantsChosen100Iterations \ --mutants_results_root validatingSqueeziness_RQ3/Results_mutants_chosen \ --output_folder validatingSqueeziness_RQ3/selectedMutantsChosen_FastSqueeziness \ --qasm_original_root qasm_circuits/chosen \ --qasm_mutant_root validatingSqueeziness_RQ3/Mutants_chosen Execute mutation testing: python3 validatingSqueeziness_RQ3/analyse.py \ --oracle validatingSqueeziness_RQ3/chosen_oracle \ --mutants validatingSqueeziness_RQ3/selectedMutantsChosen_FastSqueeziness/ \ --output validatingSqueeziness_RQ3/Test_killingMutants_FastSqueeziness_Chosen Produce the final correlation plots: python3 validatingSqueeziness_RQ3/correlation_squeeziness_kill_ratio.py Running these scripts recreates all necessary intermediate directories and the complete set of results. The configuration can be modified to reproduce alternative experiments (e.g., MQT datasets or standard Squeeziness results). Performance Comparison Run: python3 validatingSqueeziness_RQ3/compare_squeezinessFast.py to analyse the performance of FastSqueeziness compared with Squeeziness. Additional Materials Additional experiments, exploratory analyses, and investigations are available in the: .experimentations/ folder.
Testability—the degree to which faults can be revealed and diagnosed—has long been fundamental in software engineering but remains largely unexplored for quantum software. This work presents the first formalisation of testability for quantum programs, adapting classical concepts to quantum computation. We introduce Quantum Squeeziness, an information-theoretic metric inspired by classical squeeziness, to quantify how faults are masked and prevented from propagating to observable outcomes. We treat it as an inverse measure of testability: higher squeeziness implies stronger fault masking and reduced detectability. We propose an efficient method to compute Quantum Squeeziness and validate it through large-scale quantum mutation analysis. Across more than 50k mutants, we observe a strong negative correlation between squeeziness and mutation score, showing that higher squeeziness predicts lower testability. We further introduce a technique that constrains where and how perturbations are injected, reducing cost while preserving accuracy. The computation is 15% faster than a standard error-propagation baseline, while the optimised variant, FastSqueeziness, reduces mean runtime from 3200 s to 189 s (16.9×speedup) with 95% accuracy. By establishing a rigorous foundation of testability for quantum software, this work opens new directions at the intersection of software engineering, quantum information theory, and testing. Beyond testing and debugging, it also supports applications in quantum error mitigation and correction, enabling principled assessment and improvement of program robustness.
Testability—the degree to which faults can be revealed and diagnosed—has long been fundamental in software engineering but remains largely unexplored for quantum software. This work presents the first formalisation of testability for quantum programs, adapting classical concepts to quantum computation. We introduce Quantum Squeeziness, an information-theoretic metric inspired by classical squeeziness, to quantify how faults are masked and prevented from propagating to observable outcomes. We treat it as an inverse measure of testability: higher squeeziness implies stronger fault masking and reduced detectability.
Given that quantum software testing is a new area of research, there is a lack of benchmark programs and bugs repositories to assess the effectiveness of testing techniques. To this end, quantum mutation analysis focuses on systematically generating faulty versions of Quantum Programs (QPs), called mutants, using mutation operators. Such mutants can be used as benchmarks to assess the quality of test cases in a test suite. Thus, we present Muskit - a quantum mutation analysis tool for QPs coded in IBM's Qiskit language. Muskit defines mutation operators on gates of QPs and selection criteria to reduce the number of mutants to generate. Moreover, it allows for the execution of test cases on mutants and generation of results for test analyses. Muskit is provided as command line interface, GUI, and web application. We validated Muskit by using it to generate and execute mutants for four QPs. Muskit code: https://github.com/Simula-COMPLEX/muskitWeb app: https://qiskitmutantcreatorsrl.pythonanywhere.com/YouTube Video: EbPHJOK_AEA Artifact Available: https://doi.org/10.5281/zenodo.5288917
Quantum computing research increasingly depends on complex software stacks, yet the reproducibility of published results does not receive the priority and longevity mandated by recommendations of large international scientific bodies and best practices in software-centric systems research. In this paper, we present a combined manual and automated large-scale analysis of the reproducibility landscape in quantum computing research, quantify shortcomings, and derive actionable steps forward. We manually evaluate a curated sample of 127 papers using a five-question framework that covers code availability, environment specification, documentation, hardware description, and executability. To place these findings in a broader context, we conduct an automated large-scale screening of nearly 5000 quantum computing papers for the same reproducibility indicators. Our manual analysis reveals that only 24.4% of the sampled papers provide code artefacts, and among those, 64.5% fail to execute successfully in a clean environment. This assessment is corroborated by a large-scale automated analysis that yields a consistent code availability rate of 26.8%. Further, it shows that approximately one-third of the papers with accessible code lack machine-readable environment specifications. The results in this paper indicate that reproducibility is not yet consistently achieved in quantum computing research. In response, we outline a set of practical recommendations that address the observed failure modes and illustrate how reproducibility can be improved in practice.
The reproducibility of software defect datasets is essential for obtaining reliable and comparable research results. Zhu et al. have shown that defect datasets such as Defects4J suffer from reproduction failures (i.e., reported bugs become non-reproducible) as time passes since their creation. However, it remains unclear whether these findings generalize to quantum software defect datasets. We therefore conduct a replication study of the prior work using Bugs4Q, a widely used dataset of real-world bugs in quantum programs. Our analysis includes 77,700 quantum program executions of 37 Bugs4Q artifacts across 21 core-library versions. The experimental results showed that the reproducibility of Bugs4Q dropped from 62.2% on Qiskit v0.20.1 to 16.2% on v2.3.1, the latest version as of April 1, 2026. A manual inspection of the root causes further indicated that 93.6% of the failures were dependency-related. While these findings are consistent with those of the prior work, we also observed differences. In particular, most reproduction failures in Bugs4Q cannot be resolved merely by adjusting dependency versions; instead, they require source-code modifications such as migrating import paths and API invocations. Based on this observation, we curated Bugs4Q-Robust, a patched version of Bugs4Q to restore reproducibility. Bugs4Q-Robust increases reproducibility from 16.2% to 78.4% on Qiskit v2.3.1. Our findings highlight the importance of continuous dataset maintenance in the rapidly evolving quantum software ecosystem.
Abstract As a new research area, quantum software testing lacks systematic testing benchmarks to assess testing techniques’ effectiveness. Recently, some open-source benchmarks and mutation analysis tools have emerged. However, there is insufficient evidence on how various quantum circuit characteristics (e.g., circuit depth, number of quantum gates), algorithms (e.g., Quantum Approximate Optimization Algorithm), and mutation characteristics (e.g., mutation operators) affect the detection of mutants in quantum circuits. Studying such relations is important to systematically design faulty benchmarks with varied attributes (e.g., the difficulty in detecting a seeded fault) to facilitate assessing the cost-effectiveness of quantum software testing techniques efficiently. To this end, we present a large-scale empirical evaluation with more than 700K faulty benchmarks (quantum circuits) generated by mutating 382 real-world quantum circuits. Based on the results, we provide valuable insights for researchers to define systematic quantum mutation analysis techniques. We also provide a tool to recommend mutants to users based on chosen characteristics (e.g., a quantum algorithm type) and the required difficulty of detecting mutants. Finally, we also provide faulty benchmarks that can already be used to assess the cost-effectiveness of quantum software testing techniques.