DescriptionThis replication package contains the tools, datasets, and experimental evaluation results for the research paper "Beyond Rules: LLM-Powered Linting for Quantum Programs". It introduces LLM-powered linting techniques designed to detect 10 quantum-specific programming problems across Qiskit applications.Package Structurecode/: Python implementations of the LLM-powered linters, including single-prompt (LintQ-LLM), Chain-of-Thought (LintQ-LLM+CoT), and Retrieval-Augmented Generation (LintQ-LLM+RAG) architectures, along with pre-built FAISS vector databases.samples/: An evaluation corpus of 55 Qiskit files, comprising 43 verified real-world programs and 12 fault-injected programs.xlsx/: Spreadsheets containing manual ground-truth labeling data and complete metric breakdown reports (Precision, Recall, F1-Score)README.MD: Details on how to reproduce the experimentation
DescriptionThis replication package contains the tools, datasets, and experimental evaluation results for the research paper "Beyond Rules: LLM-Powered Linting for Quantum Programs". It introduces LLM-powered linting techniques designed to detect 10 quantum-specific programming problems across Qiskit applications.Package Structurecode/: Python implementations of the LLM-powered linters, including single-prompt (LintQ-LLM), Chain-of-Thought (LintQ-LLM+CoT), and Retrieval-Augmented Generation (LintQ-LLM+RAG) architectures, along with pre-built FAISS vector databases.samples/: An evaluation corpus of 55 Qiskit files, comprising 43 verified real-world programs and 12 fault-injected programs.xlsx/: Spreadsheets containing manual ground-truth labeling data and complete metric breakdown reports (Precision, Recall, F1-Score)README.MD: Details on how to reproduce the experimentation
As quantum computing transitions from theoretical experimentation to its practical application, the reliability of quantum software has become a critical bottleneck. Traditional static analysis techniques for quantum programs, primarily rule-based linters, are increasingly inadequate; they struggle to keep pace with rapidly evolving APIs and fail to capture complex, context-dependent quantum programming problems. This results in high maintenance overhead and limited detection capabilities. In this paper, we introduce LintQ-LLM+CoT and LintQ-LLM+RAG, novel approaches that redefine the detection of quantum programming problems by employing Large Language Models (LLMs) specialized, respectively, via Chain-of-Thought (CoT) prompting and a Retrieval-Augmented Generation (RAG) system that grounds the model's reasoning in a curated knowledge base of verified quantum programming problems and best practices. We conducted a rigorous and manual comparative evaluation against the state-of-the-art rule-based tool, LintQ, using a corpus of 55 Qiskit programs. Our results show that LLM-based approaches, with and without RAG, outperform LintQ in terms of quantum programming problems detection correctness (precision) and completeness (recall). Overall, LLM-based approaches were more effective than LintQ (F1-score equal to 0.70 and 0.68 vs. 0.41). Furthermore, the RAG-enhanced variant demonstrated a slightly superior precision, effectively reducing false positives. Our findings suggest that LLMs provide a scalable and adaptive foundation for the next generation of linters in quantum software engineering.
Ensuring the quality of quantum programs is increasingly important; however, traditional static analysis techniques are insufficient due to the unique characteristics of quantum computing. Quantum-specific linting tools, such as LintQ, have been developed to detect quantum-specific programming problems; however, they typically rely on manually crafted analysis queries. The manual effort required to update these tools limits their adaptability to evolving quantum programming practices. To address this challenge, this study investigates the feasibility of employing Large Language Models (LLMs) to develop a novel linting technique for quantum software development and explores potential avenues to advance linting approaches. We introduce LintQ-LLM, an LLM-based linting tool designed to detect quantum-specific problems comparable to those identified by LintQ. Through an empirical comparative study using realworld Qiskit programs, our results show that LintQ-LLM is a viable solution that complements LintQ, with particular strengths in problem localization, explanation clarity, and adaptability potential for emerging quantum programming frameworks, thus providing a basis for further research. Furthermore, this study discusses several research opportunities for developing more advanced, adaptable, and feedback-aware quantum software quality assurance methods by leveraging LLMs.
Large Language Models (LLMs) offer powerful capabilities in code generation, natural language understanding, and domain-specific reasoning. Their application to quantum software development remains limited, in part because of the lack of high-quality datasets both for LLM training and as dependable knowledge sources. To bridge this gap, we introduce \textit{PennyLang}, an off-the-shelf, high-quality dataset of 3,347 PennyLane-specific quantum code samples with contextual descriptions, curated from textbooks, official documentation, and open-source repositories. Our contributions are threefold: (1) the creation and open-source release of PennyLang, a purpose-built dataset for quantum programming with PennyLane; (2) a framework for automated quantum code dataset construction that systematizes curation, annotation, and formatting to maximize downstream LLM usability; and (3) a baseline evaluation of the dataset across multiple open-source and commercial models, including ablation studies, all conducted within a retrieval-augmented generation (RAG) pipeline. Using PennyLang with RAG substantially improves performance: for example, Qwen 7B's success rate rises from 8.7% without retrieval to 41.7% with full-context augmentation, and LLaMa 4 improves from 78.8% to 84.8%, while also reducing hallucinations and enhancing quantum code correctness. Moving beyond Qiskit-focused studies, we bring LLM-based tools and reproducible methods to PennyLane for advancing AI-assisted quantum development.
As quantum computing is rising in popularity, the amount of quantum programs and the number of developers writing them are increasing rapidly. Unfortunately, writing correct quantum programs is challenging due to various subtle rules developers need to be aware of. Empirical studies show that 40–82% of all bugs in quantum software are specific to the quantum domain. Yet, existing static bug detection frameworks are mostly unaware of quantum-specific concepts, such as circuits, gates, and qubits, and hence miss many bugs. This paper presents LintQ, a comprehensive static analysis framework for detecting bugs in quantum programs. Our approach is enabled by a set of abstractions designed to reason about common concepts in quantum computing without referring to the details of the underlying quantum computing platform. Built on top of these abstractions, LintQ offers an extensible set of ten analyses that detect likely bugs, such as operating on corrupted quantum states, redundant measurements, and incorrect compositions of sub-circuits. We apply the approach to a newly collected dataset of 7,568 real-world Qiskit-based quantum programs, showing that LintQ effectively identifies various programming problems, with a precision of 91.0% in its default configuration with the six best performing analyses. Comparing to a general-purpose linter and two existing quantum-aware techniques shows that almost all problems (92.1%) found by LintQ during our evaluation are missed by prior work. LintQ hence takes an important step toward reliable software in the growing field of quantum computing.
Generating correct and executable quantum circuit code remains a significant barrier for both newcomers and experienced practitioners in quantum computing. While Large Language Models (LLMs) have shown promise in classical code generation, their application to quantum programming is hampered by limited domain-specific training data and the rapidly evolving nature of quantum software frameworks. In this work, we present Quantum-RAG, an extended Retrieval-Augmented Generation framework that couples a domain-specialized language model with a curated, multi-source knowledge corpus to produce Qiskit-based quantum circuit code. Our three-stage pipeline ingests documentation, educational notebooks, and research literature into a dense retrieval index and augments the generation model with the most relevant context at inference time. Evaluated on the Qiskit HumanEval benchmark, QuantumRAG achieves a Pass@1 of 0.4132, a Pass@3 of 0.6291, and a Pass@5 of 0.7020, surpassing the domain-adapted GraniteQiskit baseline by 101%, 72%, and 47 % at each respective threshold. These results demonstrate that retrieval-augmented approaches can substantially improve quantum code generation without expensive model retraining.
Large Language Models (LLMs) are increasingly deployed as quantum coding copilots, assisting researchers by generating complex programs for frameworks such as Qiskit, PennyLane, and OpenQASM. However, accurately evaluating the practical utility of these models remains a significant challenge. Existing benchmarks are highly fragmented, ranging from syntax-centric unit tests to simulator- and hardware-aware metrics, making results difficult to compare across different languages and hardware targets. To address this fragmentation, this paper systematically reviews recent evaluation suites and post-training methodologies for quantum code generation. We distill current evaluation criteria into four practical dimensions: syntax and API validity, semantic and physical correctness, resource and hardware feasibility, and execution-based feedback. Building upon these dimensions, we present a unified taxonomy that maps current benchmarks to emerging model adaptation techniques, specifically retrieval-augmented generation (RAG), fine-tuning, multi-agent workflows, and reinforcement learning (RL). Our analysis highlights a clear shift: moving away from prompt-only baselines and toward agentic workflows driven by quantum-verifiable rewards. This shift directly targets the semantic gap between superficially compilable code and physically executable quantum programs. In addition, we identify key research opportunities-such as standardized semantic metrics and integration with quantum error correction-aimed at fostering a more reproducible, verifiable, and hardware-grounded ecosystem for automated quantum software development.