Quantum programming involves designing and constructing executable quantum programs to achieve specific computational results.Several quantum programming languages, such as Scaffold, Quipper, Qiskit, Q#, and Cirq, enable researchers and developers to implement and experiment with quantum computing techniques efficiently.However, ensuring the correctness of quantum programs is challenging due to unique features like superposition, entanglement, and no-cloning, which make bug detection difficult.Although some approaches for debugging and testing quantum software have been proposed, the research in identifying and detecting bugs in quantum programs is still in its early stages.Existing research has shortcomings in the following areas: (1) Redundancy in Bug Patterns: Many publicly reported bugs follow similar patterns, necessitating proper bug pattern classification metrics to assist developers in understanding and avoiding these bugs effectively.(2) Lack of Suitable Bug Benchmark Suites: The absence of comprehensive bug benchmark suites restricts the systematic evaluation of debugging and testing methods for quantum programs, hindering the progress of quantum software development.(3) Challenges of Dynamic Techniques and Classic Static Analysis: Dynamic techniques relying on the execution of quantum programs can be cumbersome and expensive, while classic static analysis tools designed for traditional programs are unsuitable for analyzing quantum programs due to their unique mechanisms.To address these challenges and improve the quality assurance of quantum software, this thesis primarily focuses on identifying and detecting bugs in quantum programs.The work is divided into three phases: (1) Identification and Categorization of Bug Patterns: The thesis identifies and categorizes bug patterns in the Qiskit quantum programming language.This provides researchers and programmers with a clear understanding of the types of bugs that can occur in quantum programs and how to detect them.The bug patterns cover quantum-related constructs, focusing on symptoms, root causes, cures, and preventions.(2) Bugs4Q Benchmark Suite: As an initial step towards evaluating debugging and testing tools for quantum software, the thesis introduces Bugs4Q.This benchmark suite consists of forty-two real Qiskit bugs obtained from popular platforms like GitHub, StackOverflow, and Stack Exchange.Bugs4Q includes a bug database and test cases to reproduce faulty behaviors.Researchers have access to the actual bugs and corresponding fixes for further study.The i I would like to express my sincere thanks to the members of the defense committee:
Quantum programming involves designing and constructing executable quantum programs to achieve specific computational results.Several quantum programming languages, such as Scaffold, Quipper, Qiskit, Q#, and Cirq, enable researchers and developers to implement and experiment with quantum computing techniques efficiently.However, ensuring the correctness of quantum programs is challenging due to unique features like superposition, entanglement, and no-cloning, which make bug detection difficult.Although some approaches for debugging and testing quantum software have been proposed, the research in identifying and detecting bugs in quantum programs is still in its early stages.Existing research has shortcomings in the following areas: (1) Redundancy in Bug Patterns: Many publicly reported bugs follow similar patterns, necessitating proper bug pattern classification metrics to assist developers in understanding and avoiding these bugs effectively.(2) Lack of Suitable Bug Benchmark Suites: The absence of comprehensive bug benchmark suites restricts the systematic evaluation of debugging and testing methods for quantum programs, hindering the progress of quantum software development.(3) Challenges of Dynamic Techniques and Classic Static Analysis: Dynamic techniques relying on the execution of quantum programs can be cumbersome and expensive, while classic static analysis tools designed for traditional programs are unsuitable for analyzing quantum programs due to their unique mechanisms.To address these challenges and improve the quality assurance of quantum software, this thesis primarily focuses on identifying and detecting bugs in quantum programs.The work is divided into three phases: (1) Identification and Categorization of Bug Patterns: The thesis identifies and categorizes bug patterns in the Qiskit quantum programming language.This provides researchers and programmers with a clear understanding of the types of bugs that can occur in quantum programs and how to detect them.The bug patterns cover quantum-related constructs, focusing on symptoms, root causes, cures, and preventions.(2) Bugs4Q Benchmark Suite: As an initial step towards evaluating debugging and testing tools for quantum software, the thesis introduces Bugs4Q.This benchmark suite consists of forty-two real Qiskit bugs obtained from popular platforms like GitHub, StackOverflow, and Stack Exchange.Bugs4Q includes a bug database and test cases to reproduce faulty behaviors.Researchers have access to the actual bugs and corresponding fixes for further study.The i I would like to express my sincere thanks to the members of the defense committee:
Abstract As quantum computers continue to improve in quality and scale, there is a growing need for accessible software frameworks for programming them. However, the unique behavior of quantum systems means specialized approaches, beyond traditional software development, are required. This is particularly true for debugging due to quantum bugs, i.e., bugs that occur precisely because an algorithm is a quantum algorithm. Pinpointing a quantum bug’s root cause often requires significant developer time, as there is little established guidance for quantum debugging techniques. Developing such guidance is the main challenge we sought to address. In this work, we describe a set of 14 quantum bugs, sourced primarily from our experience as quantum software developers, and supplemented by analysis of open-source GitHub repositories. We detail their context, symptoms, and the techniques applied to identify and fix them. While classifying these bugs based on existing schemes, we observed that most emerged due to unique interactions between multiple aspects of an algorithm or workflow. In other words, they occurred because more than one thing went wrong, which provided important insight into why quantum debugging is more challenging. Furthermore, based on this clustering, we found that—unexpectedly—there is no clear relationship between debugging strategies and bug classes. Further research is needed to develop effective and systematic quantum debugging strategies.
Realistic benchmarks of reproducible bugs and fixes are vital to good experimental evaluation of debugging and testing approaches. However, there is no suitable benchmark suite that can systematically evaluate the debugging and testing methods of quantum programs until now. This paper proposes Bugs4Q, a benchmark of thirty-six real, manually validated Qiskit bugs from four popular Qiskit elements (Terra, Aer, Ignis, and Aqua), supplemented with the test cases for reproducing buggy behaviors. Bugs4Q also provides interfaces for accessing the buggy and fixed versions of the Qiskit programs and executing the corresponding test cases, facilitating the reproducible empirical studies and comparisons of Qiskit program debugging and testing tools. Bugs4Q is publicly available at https://github.com/Z-928/Bugs4Q
Quantum computing is getting increasing interest from both academia and industry, and the quantum software landscape has been growing rapidly. The quantum software stack comprises quantum programs, implementing algorithms, and platforms like IBM Qiskit, Google Cirq, and Microsoft Q#, enabling their development. To ensure the reliability and performance of quantum software, various techniques for testing and analyzing it have been proposed, such as test generation, bug pattern detection, and circuit optimization. However, the large amount of work and the fact that work on quantum software is performed by several research communities, make it difficult to get a comprehensive overview of the existing techniques. In this work, we provide an extensive survey of the state of the art in testing and analysis of quantum software. We discuss literature from several research communities, including quantum computing, software engineering, programming languages, and formal methods. Our survey covers a wide range of topics, including expected and unexpected behavior of quantum programs, testing techniques, program analysis approaches, optimizations, and benchmarks for testing and analyzing quantum software. We create novel connections between the discussed topics and present them in an accessible way. Finally, we discuss key challenges and open problems to inspire future research.
As quantum programming evolves, more and more quantum programming languages are being developed. As a result, debugging and testing quantum programs have become increasingly important. While bug fixing in classical programs has come a long way, there is a lack of research in quantum programs. To this end, this paper presents a comprehensive study on bug fixing in quantum programs. We collect and investigate 96 real-world bugs and their fixes from four popular quantum programming languages (Qiskit, Cirq, Q#, and ProjectQ). Our study shows that a high proportion of bugs in quantum programs are quantum-specific bugs (over 80%), which requires further research in the bug fixing domain. We also summarize and extend the bug patterns in quantum programs and subdivide the most critical part, math-related bugs, to make it more applicable to the study of quantum programs. Our findings summarize the characteristics of bugs in quantum programs and provide a basis for studying testing and debugging quantum programs.
Advances in NISQ devices have allowed quantum researchers and enthusiasts to design and develop quantum programs for different applications. As the hardware progresses from noisy to fault-tolerant, the size of the programs implemented will increase. As the circuits get larger, testing and debugging these circuits will become more complex. To overcome the counterintuitive nature of quantum circuits and help developers better understand their programs and locate bugs during the development process's testing and debugging stages, we must understand the different types of bugs that occur in quantum circuits and how they affect the state of the qubits and propagate through the circuit. In this work, we categorize some bugs that could occur when implementing quantum circuits. This poster is part of a bigger work that aims to assist developers in debugging their circuits. This is done by bridging some testing and debugging techniques from the classical world, such as hypothesis and binary-like testing, and debugging-assistance functions, such as tracking gates and detecting qubits' light cones. In addition, we collected and studied bugs from Stack Exchange, Stack Overflow, and GitHub and categorized these bugs based on their source to provide developers with tools and information for an efficient and systematic debugging process for their programs.
Static analysis is the process of analyzing software code without executing the software. It can help find bugs and potential problems in software that may only appear at runtime. Although many static analysis tools have been developed for classical software, due to the nature of quantum programs, these existing tools are unsuitable for analyzing quantum programs. This paper presents QChecker, a static analysis tool that supports finding bugs in quantum programs in Qiskit. QChecker consists of two main modules: a module for extracting program information based on abstract syntax tree (AST), and a module for detecting bugs based on patterns. We evaluate the performance of QChecker using the Bugs4Q benchmark. The evaluation results show that QChecker can effectively detect various bugs in quantum programs.