Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2603.22184

Revisiting Quantum Code Generation: Where Should Domain Knowledge Live?

Oscar Novo, Oscar Bastidas-Jossa, Alberto Calvo, Antonio Peris, Carlos Kuchkovsky

cs.LGquant-phSubmitted Mar 23, 2026• Announced Mar 24, 2026• 0 citations
arXivHTMLPDF

Abstract

Recent advances in large language models (LLMs) have enabled the automation of an increasing number of programming tasks, including code generation for scientific and engineering domains. In rapidly evolving software ecosystems such as quantum software development, where frameworks expose complex abstractions, a central question is how best to incorporate domain knowledge into LLM-based assistants while preserving maintainability as libraries evolve. In this work, we study specialization strategies for Qiskit code generation using the Qiskit-HumanEval benchmark. We compare a parameter-specialized fine-tuned baseline introduced in prior work against a range of recent general-purpose LLMs enhanced with retrieval-augmented generation (RAG) and agent-based inference with execution feedback. Our results show that modern general-purpose LLMs consistently outperform the parameter-specialized baseline. While the fine-tuned model achieves approximately 47% pass@1 on Qiskit-HumanEval, recent general-purpose models reach 60-65% under zero-shot and retrieval-augmented settings, and up to 85% for the strongest evaluated model when combined with iterative execution-feedback agents -representing an improvement of more than 20% over zero-shot general-purpose performance and more than 35% over the parameter-specialized baseline. Agentic execution feedback yields the most consistent improvements, albeit at increased runtime cost, while RAG provides modest and model-dependent gains. These findings indicate that performance gains can be achieved without domain-specific fine-tuning, instead relying on inference-time augmentation, thereby enabling a more flexible and maintainable approach to LLM-assisted quantum software development.

Comments
Submitted to Quantum Machine Intelligence

Related papers

Benchmarking the Benchmarks for Quantum Code Generation with LLMs

Yunhe Feng, Xiaokun Yang, Kewei Sha +1 more

Large Language Models (LLMs) are increasingly deployed as quantum coding copilots, assisting researchers by generating complex programs for frameworks such as Qiskit, PennyLane, and OpenQASM. However, accurately evaluating the practical utility of these models remains a significant challenge. Existing benchmarks are highly fragmented, ranging from syntax-centric unit tests to simulator- and hardware-aware metrics, making results difficult to compare across different languages and hardware targets. To address this fragmentation, this paper systematically reviews recent evaluation suites and post-training methodologies for quantum code generation. We distill current evaluation criteria into four practical dimensions: syntax and API validity, semantic and physical correctness, resource and hardware feasibility, and execution-based feedback. Building upon these dimensions, we present a unified taxonomy that maps current benchmarks to emerging model adaptation techniques, specifically retrieval-augmented generation (RAG), fine-tuning, multi-agent workflows, and reinforcement learning (RL). Our analysis highlights a clear shift: moving away from prompt-only baselines and toward agentic workflows driven by quantum-verifiable rewards. This shift directly targets the semantic gap between superficially compilable code and physically executable quantum programs. In addition, we identify key research opportunities-such as standardized semantic metrics and integration with quantum error correction-aimed at fostering a more reproducible, verifiable, and hardware-grounded ecosystem for automated quantum software development.

IEEE Dallas Circuits and Systems Conference (DCAS)•Apr 10, 2026•0 citations

Quantum-Rag: Retrieval-Augmented Generation Framework for Quantum Circuit Code Generation

Nicholas Stapleton, Sharmin Afrose, Ethan Seefried +3 more

Generating correct and executable quantum circuit code remains a significant barrier for both newcomers and experienced practitioners in quantum computing. While Large Language Models (LLMs) have shown promise in classical code generation, their application to quantum programming is hampered by limited domain-specific training data and the rapidly evolving nature of quantum software frameworks. In this work, we present Quantum-RAG, an extended Retrieval-Augmented Generation framework that couples a domain-specialized language model with a curated, multi-source knowledge corpus to produce Qiskit-based quantum circuit code. Our three-stage pipeline ingests documentation, educational notebooks, and research literature into a dense retrieval index and augments the generation model with the most relevant context at inference time. Evaluated on the Qiskit HumanEval benchmark, QuantumRAG achieves a Pass@1 of 0.4132, a Pass@3 of 0.6291, and a Pass@5 of 0.7020, surpassing the domain-adapted GraniteQiskit baseline by 101%, 72%, and 47 % at each respective threshold. These results demonstrate that retrieval-augmented approaches can substantially improve quantum code generation without expensive model retraining.

IEEE Dallas Circuits and Systems Conference (DCAS)•Apr 10, 2026•0 citations

Qiskit Code Migration with LLMs

Jose Manuel Suarez, Luís Mariano Bibbó, Joaquín Bogado +1 more

The rapid evolution of Quantum Development Kits (QDKs) introduces a specific form of technical debt that compromises code maintainability and hinders software reuse. In the specialized domain of Quantum Software Engineering (QSE), this challenge is intensified by the scarcity of high-quality training data and the high volatility of emerging frameworks, which often lead general-purpose Large Language Models (LLMs) to produce unreliable or hallucinated results. This paper proposes a hybrid approach integrating LLMs with Retrieval-Augmented Generation (RAG) to automate the migration of Qiskit code across versions. The proposed methodology enhances the precision and reliability of migration suggestions by leveraging an automatically generated taxonomy of migration scenarios as the structured, version-specific knowledge source to guide the models. The approach is implemented through an automated, extensible workflow evaluating LLMs (Google Gemini Flash-2.5 and OpenAI Gpt-oss-20b) under different retrieval schemes (unconstrained and restrictive). Results demonstrate that the taxonomy-based RAG architecture, particularly under the restrictive scheme, significantly reduces hallucinations and improves descriptive quality, with Google Gemini Flash-2.5 showing superior performance in detecting complex refactoring scenarios. These findings confirm the potential of this data-centric methodology to foster technological independence and provide robust, intelligent assistants that mitigate API obsolescence, ensuring the long-term availability of quantum algorithms within a rapidly shifting ecosystem and flattening the learning curve within Quantum Software Engineering (QSE).

arXiv:2606.20173•JAIIO, Jornadas Argentinas de Informática•Jun 18, 2026•0 citations

Model-Driven Quantum Code Generation Using Large Language Models and Retrieval-Augmented Generation

Nazanin Siavash, Armin Moin

This paper introduces a novel research direction for model-to-text/code transformations by leveraging Large Language Models (LLMs) that can be enhanced with Retrieval-Augmented Generation (RAG) pipelines. The focus is on quantum and hybrid quantum-classical software systems, where model-driven approaches can help reduce the costs and mitigate the risks associated with the heterogeneous platform landscape and lack of developers' skills. We validate one of the proposed ideas regarding generating code out of UML model instances of software systems. This Python code uses a well-established library, called Qiskit, to execute on gate-based or circuit-based quantum computers. The RAG pipeline that we deploy incorporates sample Qiskit code from public GitHub repositories. Experimental results show that well-engineered prompts can improve CodeBLEU scores by up to a factor of four, yielding more accurate and consistent quantum code. However, the proposed research direction can go beyond this through further investigation in the future by conducting experiments to address our other research questions and ideas proposed here, such as deploying software system model instances as the source of information in the RAG pipelines, or deploying LLMs for code-to-code transformations, for instance, for transpilation use cases.

arXiv:2508.21097•ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MODELS)•Oct 5, 2025•4 citations

QiboAgent: a practitioner's guideline to open source assistants for Quantum Computing code development

Lorenzo Esposito, Andrea Papaluca, Stefano Carrazza

We introduce QiboAgent, a reference implementation designed to serve as a practitioner's guideline for developing specialized coding assistants in Quantum Computing middleware. Addressing the limitations in scientific software development of general-purpose proprietary models, we explore how lightweight, open-source Large Language Models (LLMs) provided with a custom workflow architecture compare. In detail, we experiment with two complementary paradigms: a Retrieval-Augmented Generation pipeline for high-precision information retrieval, and an autonomous agentic workflow for complex software engineering tasks. We observe that this hybrid approach significantly reduces hallucination rates in code generation compared to a proprietary baseline, achieving a peak accuracy of 90.2% with relatively small open-source models of size up to 30B parameters. Furthermore, the agentic framework exhibits advanced coding capabilities, automating the resolution of maintenance issues and new features requests, or by prototyping larger-scale refactors of the codebase, such as producing a compiled Rust module with bindings of an original pure python package, Qibo in our case. The LLM workflows used for our analysis are integrated into a user interface and a Model Context Protocol server, providing an accessible tool for Qibo developers.

quant-pharXiv:2603.15538•Mar 17, 2026•0 citations

Enhancing LLM-based Quantum Code Generation with Multi-Agent Optimization and Quantum Error Correction

Charlie Campbell, Hao Chen, Wayne Luk +1 more

Multi-agent frameworks with Large Language Models (LLMs) have become promising tools for generating generalpurpose programming languages using test-driven development, allowing developers to create more accurate and robust code. However, their potential has not been fully unleashed for domainspecific programming languages, where specific domain exhibits unique optimization opportunities for customized improvement. In this paper, we take the first step in exploring multi-agent code generation for quantum programs. By identifying the unique optimizations in quantum designs such as quantum error correction, we introduce a novel multi-agent framework tailored to generating accurate, fault-tolerant quantum code. Each agent in the framework focuses on distinct optimizations, iteratively refining the code using a semantic analyzer with multi-pass inference, alongside an error correction code decoder. We also examine the effectiveness of traditional techniques, like Chain-ofThought (CoT) and Retrieval-Augmented Generation (RAG) in the context of quantum programming, uncovering observations that are different from general-purpose code generation. To evaluate our approach, we develop a test suite to measure the impact each optimization has on the accuracy of the generated code. Our findings indicate that techniques such as structured CoT significantly improve the generation of quantum algorithms by up to $50 \%$. In contrast, we have also found that certain techniques such as RAG show limited improvement, yielding an accuracy increase of only $4 \%$. Moreover, we showcase examples of AIassisted quantum error prediction and correction, demonstrating the effectiveness of our multi-agent framework in reducing the errors of generated quantum programs.

ACM/IEEE Design Automation Conference (DAC)•Jun 22, 2025•8 citations
Recent advances in large language models (LLMs) have enabled the automation of an increasing number of programming tasks, including code generation for scientific and engineering domains. In rapidly evolving software ecosystems such as quantum software development, where frameworks expose complex abstractions, a central question is how best to incorporate domain knowledge into LLM-based assistants while preserving maintainability as libraries evolve.
In this work, we study specialization strategies for Qiskit code generation using the Qiskit-HumanEval benchmark. We compare a parameter-specialized fine-tuned baseline introduced in prior work against a range of recent general-purpose LLMs enhanced with retrieval-augmented generation (RAG) and agent-based inference with execution feedback.
Our results show that modern general-purpose LLMs consistently outperform the parameter-specialized baseline. While the fine-tuned model achieves approximately 47% pass@1 on Qiskit-HumanEval, recent general-purpose models reach 60-65% under zero-shot and retrieval-augmented settings, and up to 85% for the strongest evaluated model when combined with iterative execution-feedback agents -representing an improvement of more than 20% over zero-shot general-purpose performance and more than 35% over the parameter-specialized baseline.
Agentic execution feedback yields the most consistent improvements, albeit at increased runtime cost, while RAG provides modest and model-dependent gains. These findings indicate that performance gains can be achieved without domain-specific fine-tuning, instead relying on inference-time augmentation, thereby enabling a more flexible and maintainable approach to LLM-assisted quantum software development.
Large Language Models (LLMs) are increasingly deployed as quantum coding copilots, assisting researchers by generating complex programs for frameworks such as Qiskit, PennyLane, and OpenQASM. However, accurately evaluating the practical utility of these models remains a significant challenge. Existing benchmarks are highly fragmented, ranging from syntax-centric unit tests to simulator- and hardware-aware metrics, making results difficult to compare across different languages and hardware targets. To address this fragmentation, this paper systematically reviews recent evaluation suites and post-training methodologies for quantum code generation. We distill current evaluation criteria into four practical dimensions: syntax and API validity, semantic and physical correctness, resource and hardware feasibility, and execution-based feedback. Building upon these dimensions, we present a unified taxonomy that maps current benchmarks to emerging model adaptation techniques, specifically retrieval-augmented generation (RAG), fine-tuning, multi-agent workflows, and reinforcement learning (RL). Our analysis highlights a clear shift: moving away from prompt-only baselines and toward agentic workflows driven by quantum-verifiable rewards. This shift directly targets the semantic gap between superficially compilable code and physically executable quantum programs. In addition, we identify key research opportunities-such as standardized semantic metrics and integration with quantum error correction-aimed at fostering a more reproducible, verifiable, and hardware-grounded ecosystem for automated quantum software development.
Generating correct and executable quantum circuit code remains a significant barrier for both newcomers and experienced practitioners in quantum computing. While Large Language Models (LLMs) have shown promise in classical code generation, their application to quantum programming is hampered by limited domain-specific training data and the rapidly evolving nature of quantum software frameworks. In this work, we present Quantum-RAG, an extended Retrieval-Augmented Generation framework that couples a domain-specialized language model with a curated, multi-source knowledge corpus to produce Qiskit-based quantum circuit code. Our three-stage pipeline ingests documentation, educational notebooks, and research literature into a dense retrieval index and augments the generation model with the most relevant context at inference time. Evaluated on the Qiskit HumanEval benchmark, QuantumRAG achieves a Pass@1 of 0.4132, a Pass@3 of 0.6291, and a Pass@5 of 0.7020, surpassing the domain-adapted GraniteQiskit baseline by 101%, 72%, and 47 % at each respective threshold. These results demonstrate that retrieval-augmented approaches can substantially improve quantum code generation without expensive model retraining.
The rapid evolution of Quantum Development Kits (QDKs) introduces a specific form of technical debt that compromises code maintainability and hinders software reuse. In the specialized domain of Quantum Software Engineering (QSE), this challenge is intensified by the scarcity of high-quality training data and the high volatility of emerging frameworks, which often lead general-purpose Large Language Models (LLMs) to produce unreliable or hallucinated results. This paper proposes a hybrid approach integrating LLMs with Retrieval-Augmented Generation (RAG) to automate the migration of Qiskit code across versions. The proposed methodology enhances the precision and reliability of migration suggestions by leveraging an automatically generated taxonomy of migration scenarios as the structured, version-specific knowledge source to guide the models. The approach is implemented through an automated, extensible workflow evaluating LLMs (Google Gemini Flash-2.5 and OpenAI Gpt-oss-20b) under different retrieval schemes (unconstrained and restrictive). Results demonstrate that the taxonomy-based RAG architecture, particularly under the restrictive scheme, significantly reduces hallucinations and improves descriptive quality, with Google Gemini Flash-2.5 showing superior performance in detecting complex refactoring scenarios. These findings confirm the potential of this data-centric methodology to foster technological independence and provide robust, intelligent assistants that mitigate API obsolescence, ensuring the long-term availability of quantum algorithms within a rapidly shifting ecosystem and flattening the learning curve within Quantum Software Engineering (QSE).
This paper introduces a novel research direction for model-to-text/code transformations by leveraging Large Language Models (LLMs) that can be enhanced with Retrieval-Augmented Generation (RAG) pipelines. The focus is on quantum and hybrid quantum-classical software systems, where model-driven approaches can help reduce the costs and mitigate the risks associated with the heterogeneous platform landscape and lack of developers' skills. We validate one of the proposed ideas regarding generating code out of UML model instances of software systems. This Python code uses a well-established library, called Qiskit, to execute on gate-based or circuit-based quantum computers. The RAG pipeline that we deploy incorporates sample Qiskit code from public GitHub repositories. Experimental results show that well-engineered prompts can improve CodeBLEU scores by up to a factor of four, yielding more accurate and consistent quantum code. However, the proposed research direction can go beyond this through further investigation in the future by conducting experiments to address our other research questions and ideas proposed here, such as deploying software system model instances as the source of information in the RAG pipelines, or deploying LLMs for code-to-code transformations, for instance, for transpilation use cases.
We introduce QiboAgent, a reference implementation designed to serve as a practitioner's guideline for developing specialized coding assistants in Quantum Computing middleware. Addressing the limitations in scientific software development of general-purpose proprietary models, we explore how lightweight, open-source Large Language Models (LLMs) provided with a custom workflow architecture compare. In detail, we experiment with two complementary paradigms: a Retrieval-Augmented Generation pipeline for high-precision information retrieval, and an autonomous agentic workflow for complex software engineering tasks. We observe that this hybrid approach significantly reduces hallucination rates in code generation compared to a proprietary baseline, achieving a peak accuracy of 90.2% with relatively small open-source models of size up to 30B parameters. Furthermore, the agentic framework exhibits advanced coding capabilities, automating the resolution of maintenance issues and new features requests, or by prototyping larger-scale refactors of the codebase, such as producing a compiled Rust module with bindings of an original pure python package, Qibo in our case. The LLM workflows used for our analysis are integrated into a user interface and a Model Context Protocol server, providing an accessible tool for Qibo developers.
Multi-agent frameworks with Large Language Models (LLMs) have become promising tools for generating generalpurpose programming languages using test-driven development, allowing developers to create more accurate and robust code. However, their potential has not been fully unleashed for domainspecific programming languages, where specific domain exhibits unique optimization opportunities for customized improvement. In this paper, we take the first step in exploring multi-agent code generation for quantum programs. By identifying the unique optimizations in quantum designs such as quantum error correction, we introduce a novel multi-agent framework tailored to generating accurate, fault-tolerant quantum code. Each agent in the framework focuses on distinct optimizations, iteratively refining the code using a semantic analyzer with multi-pass inference, alongside an error correction code decoder. We also examine the effectiveness of traditional techniques, like Chain-ofThought (CoT) and Retrieval-Augmented Generation (RAG) in the context of quantum programming, uncovering observations that are different from general-purpose code generation. To evaluate our approach, we develop a test suite to measure the impact each optimization has on the accuracy of the generated code. Our findings indicate that techniques such as structured CoT significantly improve the generation of quantum algorithms by up to 50%50 \%50%. In contrast, we have also found that certain techniques such as RAG show limited improvement, yielding an accuracy increase of only 4%4 \%4%. Moreover, we showcase examples of AIassisted quantum error prediction and correction, demonstrating the effectiveness of our multi-agent framework in reducing the errors of generated quantum programs.