Quantum software engineering faces unique challenges due to the interaction of classical and quantum components, which produce complex and often poorly understood bug patterns. Characterizing these bugs is essential for advancing testing, debugging, and quality assurance in quantum ecosystems. This paper presents a comparative study of 12,910 issues from Qiskit and 4,613 issues from 11 additional repositories, including Cirq and PyQuil. Using a rule-based classification framework, we analyze bugs by type, category, severity, quality attributes, and quantum-specific subtypes. Results show that classical bugs consistently dominate (67%) across ecosystems, while quantum-specific bugs account for 27-30%. Ecosystem-specific trends emerge: Qiskit repositories exhibit more compatibility related bugs, whereas other ecosystems show higher syntax and quantum-specific bug rates. Across both ecosystems, gate and circuit issues dominate quantum-specific bugs, though non-Qiskit projects reveal broader diversity, including algorithmic, resource, and hybrid-interface issues. Statistical validation confirms that the framework generalizes at the bug-type level while detecting significant variations at finer levels. Benchmarking against four supervised machine-learning baselines further shows that the rule-based framework consistently outperforms data-driven models, particularly for fine-grained quantum-specific subtypes, while longitudinal analysis (2017-2025) indicates that quantum- specific bugs remain relatively stable over time rather than exhibiting a steady increase. This study provides the first cross- ecosystem comparison of bug distributions in quantum software, demonstrating the utility of an interpretable, automation-ready, rule-based framework for guiding testing, debugging, and quality assurance.
Quantum software engineering faces unique challenges due to the interaction of classical and quantum components, which produce complex and often poorly understood bug patterns. Characterizing these bugs is essential for advancing testing, debugging, and quality assurance in quantum ecosystems. This paper presents a comparative study of 12,910 issues from Qiskit and 4,613 issues from 11 additional repositories, including Cirq and PyQuil. Using a rule-based classification framework, we analyze bugs by type, category, severity, quality attributes, and quantum-specific subtypes. Results show that classical bugs consistently dominate (67%) across ecosystems, while quantum-specific bugs account for 27-30%. Ecosystem-specific trends emerge: Qiskit repositories exhibit more compatibility related bugs, whereas other ecosystems show higher syntax and quantum-specific bug rates. Across both ecosystems, gate and circuit issues dominate quantum-specific bugs, though non-Qiskit projects reveal broader diversity, including algorithmic, resource, and hybrid-interface issues. Statistical validation confirms that the framework generalizes at the bug-type level while detecting significant variations at finer levels. Benchmarking against four supervised machine-learning baselines further shows that the rule-based framework consistently outperforms data-driven models, particularly for fine-grained quantum-specific subtypes, while longitudinal analysis (2017-2025) indicates that quantum- specific bugs remain relatively stable over time rather than exhibiting a steady increase. This study provides the first cross- ecosystem comparison of bug distributions in quantum software, demonstrating the utility of an interpretable, automation-ready, rule-based framework for guiding testing, debugging, and quality assurance.
Quantum Software Engineering (QSE) is essential for ensuring the reliability and maintainability of hybrid quantum-classical systems, yet empirical evidence on how bugs emerge and affect quality in real-world quantum projects remains limited. This study presents the first ecosystem-scale longitudinal analysis of software bugs across 123 open source quantum repositories from 2012 to 2024, spanning eight functional categories, including full-stack libraries, simulators, annealing, algorithms, compilers, assembly, cryptography, and experimental computing. Using a mixed method approach combining repository mining, static code analysis, issue metadata extraction, and a validated rule-based classification framework, we analyze 32,296 verified bug reports. Results show that full-stack libraries and compilers are the most bug-prone categories due to circuit, gate, and transpilation-related issues, while simulators are mainly affected by measurement and noise modeling errors. Classical bugs primarily impact usability and interoperability, whereas quantum-specific bugs disproportionately degrade performance, maintainability, and reliability. Longitudinal analysis indicates ecosystem maturation, with bug densities peaking between 2017 and 2021 and declining thereafter. High-severity bugs cluster in cryptography, experimental computing, and compiler toolchains. Repositories employing automated testing detect more bugs and resolve issues faster. A negative binomial regression further shows that automated testing is associated with an approximate 60 percent reduction in expected bug incidence. Overall, this work provides the first large-scale data-driven characterization of quantum software bugs and offers empirical guidance for improving testing, documentation, and maintainability practices in QSE.
Quantum simulators are a foundational component of the quantum software ecosystem. They are widely used to develop and debug quantum programs, validate compiler transformations, and support empirical claims about correctness and performance. In the absence of large-scale quantum hardware, simulator outputs are often treated as ground truth for algorithm development and system evaluation. However, quantum simulators also introduce unique implementation challenges. They must faithfully emulate quantum behavior while executing on classical hardware, requiring complex representations of quantum state evolution, operator composition, and noise modeling. Yet, we still lack a large-scale and in-depth study of failures in quantum simulators.
To bridge this gap, this work presents a comprehensive empirical study of bugs in widely used open-source quantum simulators. We analyze 394 confirmed bugs from 12 simulators and manually categorize them based on root causes, failure manifestations, affected components, and discovery mechanisms. Our study reveals several key findings. First, bug discovery is largely user-driven, with most crashes, exceptions, and resource-related failures not detected by automated testing and identified after deployment. Second, logical correctness failures are widespread and often silent, producing plausible but incorrect outputs without triggering crashes or explicit error signals. Third, many critical failures originate in classical simulator infrastructure, such as memory management, indexing, configuration, and dependency compatibility, rather than in core quantum execution logic. These findings provide new insights into the reliability challenges of quantum simulators and highlight opportunities to improve testing and validation practices in the quantum software ecosystem.
Abstract As quantum computers continue to improve in quality and scale, there is a growing need for accessible software frameworks for programming them. However, the unique behavior of quantum systems means specialized approaches, beyond traditional software development, are required. This is particularly true for debugging due to quantum bugs, i.e., bugs that occur precisely because an algorithm is a quantum algorithm. Pinpointing a quantum bug’s root cause often requires significant developer time, as there is little established guidance for quantum debugging techniques. Developing such guidance is the main challenge we sought to address. In this work, we describe a set of 14 quantum bugs, sourced primarily from our experience as quantum software developers, and supplemented by analysis of open-source GitHub repositories. We detail their context, symptoms, and the techniques applied to identify and fix them. While classifying these bugs based on existing schemes, we observed that most emerged due to unique interactions between multiple aspects of an algorithm or workflow. In other words, they occurred because more than one thing went wrong, which provided important insight into why quantum debugging is more challenging. Furthermore, based on this clustering, we found that—unexpectedly—there is no clear relationship between debugging strategies and bug classes. Further research is needed to develop effective and systematic quantum debugging strategies.
Bug-fix pattern detection has been investigated in the past in the context of classical software.However, while quantum software is developing rapidly, the literature is still lacking automated methods and tools to identify, analyze, and detect bug-fix patterns.To the best of our knowledge, our work is the first to leverage classical techniques to detect bug-fix patterns in quantum code.In this paper, we propose an automated framework, called Q-Diff, for detecting bug-fix patterns in IBM Qiskit quantum code.In the framework, we develop a proof-of-concept tool based on Abstract Syntax Trees.To validate our method, we test Q-Diff with a variety of quantum bug-fix patterns using examples.We hope our work will attract the attention of the quantum software engineering community to improve the quality of quantum software.
This research addresses the unique challenges faced by quantum and hybrid quantum-classical (HQC) software developers, focusing on recurring bugs and workflow inefficiencies. Through an empirical analysis of developer forums, a comprehensive taxonomy of issues was developed, categorizing previously undocumented errors in areas such as quantum state manipulation, gate operations, and hybrid quantum-classical (HQC) integration. Building on these insights, this work proposes the development of specialized tools and methodologies informed by practitioners-such as a static analysis tool and an automated testing framework-to support effective debugging and testing in quantum applications. The ultimate goal is to enhance productivity and reliability in quantum software engineering, contributing to a robust quantum development ecosystem.
Quantum programming involves designing and constructing executable quantum programs to achieve specific computational results.Several quantum programming languages, such as Scaffold, Quipper, Qiskit, Q#, and Cirq, enable researchers and developers to implement and experiment with quantum computing techniques efficiently.However, ensuring the correctness of quantum programs is challenging due to unique features like superposition, entanglement, and no-cloning, which make bug detection difficult.Although some approaches for debugging and testing quantum software have been proposed, the research in identifying and detecting bugs in quantum programs is still in its early stages.Existing research has shortcomings in the following areas: (1) Redundancy in Bug Patterns: Many publicly reported bugs follow similar patterns, necessitating proper bug pattern classification metrics to assist developers in understanding and avoiding these bugs effectively.(2) Lack of Suitable Bug Benchmark Suites: The absence of comprehensive bug benchmark suites restricts the systematic evaluation of debugging and testing methods for quantum programs, hindering the progress of quantum software development.(3) Challenges of Dynamic Techniques and Classic Static Analysis: Dynamic techniques relying on the execution of quantum programs can be cumbersome and expensive, while classic static analysis tools designed for traditional programs are unsuitable for analyzing quantum programs due to their unique mechanisms.To address these challenges and improve the quality assurance of quantum software, this thesis primarily focuses on identifying and detecting bugs in quantum programs.The work is divided into three phases: (1) Identification and Categorization of Bug Patterns: The thesis identifies and categorizes bug patterns in the Qiskit quantum programming language.This provides researchers and programmers with a clear understanding of the types of bugs that can occur in quantum programs and how to detect them.The bug patterns cover quantum-related constructs, focusing on symptoms, root causes, cures, and preventions.(2) Bugs4Q Benchmark Suite: As an initial step towards evaluating debugging and testing tools for quantum software, the thesis introduces Bugs4Q.This benchmark suite consists of forty-two real Qiskit bugs obtained from popular platforms like GitHub, StackOverflow, and Stack Exchange.Bugs4Q includes a bug database and test cases to reproduce faulty behaviors.Researchers have access to the actual bugs and corresponding fixes for further study.The i I would like to express my sincere thanks to the members of the defense committee: