Bug-fix pattern detection has been investigated in the past in the context of classical software.However, while quantum software is developing rapidly, the literature is still lacking automated methods and tools to identify, analyze, and detect bug-fix patterns.To the best of our knowledge, our work is the first to leverage classical techniques to detect bug-fix patterns in quantum code.In this paper, we propose an automated framework, called Q-Diff, for detecting bug-fix patterns in IBM Qiskit quantum code.In the framework, we develop a proof-of-concept tool based on Abstract Syntax Trees.To validate our method, we test Q-Diff with a variety of quantum bug-fix patterns using examples.We hope our work will attract the attention of the quantum software engineering community to improve the quality of quantum software.
Bug-fix pattern detection has been investigated in the past in the context of classical software.However, while quantum software is developing rapidly, the literature is still lacking automated methods and tools to identify, analyze, and detect bug-fix patterns.To the best of our knowledge, our work is the first to leverage classical techniques to detect bug-fix patterns in quantum code.In this paper, we propose an automated framework, called Q-Diff, for detecting bug-fix patterns in IBM Qiskit quantum code.In the framework, we develop a proof-of-concept tool based on Abstract Syntax Trees.To validate our method, we test Q-Diff with a variety of quantum bug-fix patterns using examples.We hope our work will attract the attention of the quantum software engineering community to improve the quality of quantum software.
Context: Bug-fix pattern detection has been investigated in the past in the context of classical software. However, while quantum software is developing rapidly, the literature still lacks automated methods and tools to identify, analyze, and detect bug-fix patterns. To the best of our knowledge, our work previously published in SEKE'23 was the first to leverage classical techniques to detect bug-fix patterns in quantum code. Objective: To extend our previous effort, we present a research agenda (Q-Repair), including a series of testing and debugging methodologies, to improve the quality of quantum software. The ultimate goal is to utilize machine learning techniques to automatically predict fix patterns for existing quantum bugs. Method: As part of the first stage of the agenda, we extend our initial study and propose a more comprehensive automated framework, called Q-PAC, for detecting bug-fix patterns in IBM Qiskit quantum code. In the framework, we develop seven bug-fix pattern detectors using abstract syntax trees, syntactic filters, and semantic checks. Results: To demonstrate our method, we run Q-PAC on a variety of quantum bug-fix patterns using both real-world and handcrafted examples of bugs and fixes. The experimental results show that Q-PAC can effectively identify bug-fix patterns in IBM Qiskit. Conclusion: We hope our initial study on quantum bug-fix detection can bring awareness of quantum software engineering to both researchers and practitioners. Thus, we also publish Q-PAC as an open-source software on GitHub. We would like to encourage other researchers to work on research directions (such as Q-Repair) to improve the quality of the quantum programming.
Static analysis is the process of analyzing software code without executing the software. It can help find bugs and potential problems in software that may only appear at runtime. Although many static analysis tools have been developed for classical software, due to the nature of quantum programs, these existing tools are unsuitable for analyzing quantum programs. This paper presents QChecker, a static analysis tool that supports finding bugs in quantum programs in Qiskit. QChecker consists of two main modules: a module for extracting program information based on abstract syntax tree (AST), and a module for detecting bugs based on patterns. We evaluate the performance of QChecker using the Bugs4Q benchmark. The evaluation results show that QChecker can effectively detect various bugs in quantum programs.
Quantum programming involves designing and constructing executable quantum programs to achieve specific computational results.Several quantum programming languages, such as Scaffold, Quipper, Qiskit, Q#, and Cirq, enable researchers and developers to implement and experiment with quantum computing techniques efficiently.However, ensuring the correctness of quantum programs is challenging due to unique features like superposition, entanglement, and no-cloning, which make bug detection difficult.Although some approaches for debugging and testing quantum software have been proposed, the research in identifying and detecting bugs in quantum programs is still in its early stages.Existing research has shortcomings in the following areas: (1) Redundancy in Bug Patterns: Many publicly reported bugs follow similar patterns, necessitating proper bug pattern classification metrics to assist developers in understanding and avoiding these bugs effectively.(2) Lack of Suitable Bug Benchmark Suites: The absence of comprehensive bug benchmark suites restricts the systematic evaluation of debugging and testing methods for quantum programs, hindering the progress of quantum software development.(3) Challenges of Dynamic Techniques and Classic Static Analysis: Dynamic techniques relying on the execution of quantum programs can be cumbersome and expensive, while classic static analysis tools designed for traditional programs are unsuitable for analyzing quantum programs due to their unique mechanisms.To address these challenges and improve the quality assurance of quantum software, this thesis primarily focuses on identifying and detecting bugs in quantum programs.The work is divided into three phases: (1) Identification and Categorization of Bug Patterns: The thesis identifies and categorizes bug patterns in the Qiskit quantum programming language.This provides researchers and programmers with a clear understanding of the types of bugs that can occur in quantum programs and how to detect them.The bug patterns cover quantum-related constructs, focusing on symptoms, root causes, cures, and preventions.(2) Bugs4Q Benchmark Suite: As an initial step towards evaluating debugging and testing tools for quantum software, the thesis introduces Bugs4Q.This benchmark suite consists of forty-two real Qiskit bugs obtained from popular platforms like GitHub, StackOverflow, and Stack Exchange.Bugs4Q includes a bug database and test cases to reproduce faulty behaviors.Researchers have access to the actual bugs and corresponding fixes for further study.The i I would like to express my sincere thanks to the members of the defense committee:
Quantum software engineering faces unique challenges due to the interaction of classical and quantum components, which produce complex and often poorly understood bug patterns. Characterizing these bugs is essential for advancing testing, debugging, and quality assurance in quantum ecosystems. This paper presents a comparative study of 12,910 issues from Qiskit and 4,613 issues from 11 additional repositories, including Cirq and PyQuil. Using a rule-based classification framework, we analyze bugs by type, category, severity, quality attributes, and quantum-specific subtypes. Results show that classical bugs consistently dominate (67%) across ecosystems, while quantum-specific bugs account for 27-30%. Ecosystem-specific trends emerge: Qiskit repositories exhibit more compatibility related bugs, whereas other ecosystems show higher syntax and quantum-specific bug rates. Across both ecosystems, gate and circuit issues dominate quantum-specific bugs, though non-Qiskit projects reveal broader diversity, including algorithmic, resource, and hybrid-interface issues. Statistical validation confirms that the framework generalizes at the bug-type level while detecting significant variations at finer levels. Benchmarking against four supervised machine-learning baselines further shows that the rule-based framework consistently outperforms data-driven models, particularly for fine-grained quantum-specific subtypes, while longitudinal analysis (2017-2025) indicates that quantum- specific bugs remain relatively stable over time rather than exhibiting a steady increase. This study provides the first cross- ecosystem comparison of bug distributions in quantum software, demonstrating the utility of an interpretable, automation-ready, rule-based framework for guiding testing, debugging, and quality assurance.
Quantum computing is getting increasing interest from both academia and industry, and the quantum software landscape has been growing rapidly. The quantum software stack comprises quantum programs, implementing algorithms, and platforms like IBM Qiskit, Google Cirq, and Microsoft Q#, enabling their development. To ensure the reliability and performance of quantum software, various techniques for testing and analyzing it have been proposed, such as test generation, bug pattern detection, and circuit optimization. However, the large amount of work and the fact that work on quantum software is performed by several research communities, make it difficult to get a comprehensive overview of the existing techniques. In this work, we provide an extensive survey of the state of the art in testing and analysis of quantum software. We discuss literature from several research communities, including quantum computing, software engineering, programming languages, and formal methods. Our survey covers a wide range of topics, including expected and unexpected behavior of quantum programs, testing techniques, program analysis approaches, optimizations, and benchmarks for testing and analyzing quantum software. We create novel connections between the discussed topics and present them in an accessible way. Finally, we discuss key challenges and open problems to inspire future research.
Quantum simulators are a foundational component of the quantum software ecosystem. They are widely used to develop and debug quantum programs, validate compiler transformations, and support empirical claims about correctness and performance. In the absence of large-scale quantum hardware, simulator outputs are often treated as ground truth for algorithm development and system evaluation. However, quantum simulators also introduce unique implementation challenges. They must faithfully emulate quantum behavior while executing on classical hardware, requiring complex representations of quantum state evolution, operator composition, and noise modeling. Yet, we still lack a large-scale and in-depth study of failures in quantum simulators.
To bridge this gap, this work presents a comprehensive empirical study of bugs in widely used open-source quantum simulators. We analyze 394 confirmed bugs from 12 simulators and manually categorize them based on root causes, failure manifestations, affected components, and discovery mechanisms. Our study reveals several key findings. First, bug discovery is largely user-driven, with most crashes, exceptions, and resource-related failures not detected by automated testing and identified after deployment. Second, logical correctness failures are widespread and often silent, producing plausible but incorrect outputs without triggering crashes or explicit error signals. Third, many critical failures originate in classical simulator infrastructure, such as memory management, indexing, configuration, and dependency compatibility, rather than in core quantum execution logic. These findings provide new insights into the reliability challenges of quantum simulators and highlight opportunities to improve testing and validation practices in the quantum software ecosystem.