Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Adaptive Agents and Multi-Agents Systems

Quantum Circuit Design: A Reinforcement Learning Challenge

Philipp J. Altmann, Adelina Bärligea, Jonas Stein, Michael Kölle, Thomas Gabor, Thomy Phan, Claudia Linnhof-Popien

Published May 6, 2024• 0 citations
Publisher

Abstract

To assess the prospects of using reinforcement learning (RL) for selecting and parameterizing quantum gates to build viable circuit architectures, we introduce the quantum circuit designer (QCD). By considering quantum control a decision-making problem, we strive to profit from advanced RL exploration mechanisms to overcome the need for granular specification and hand-crafted architectures. To evaluate current state-of-the-art RL algorithms, we define generic objectives that arise from quantum architecture search and circuit optimization. Those evaluation results reveal challenges inherent to learning optimal quantum control.

DOI
10.65109/jvfi3998

Related papers

Challenges for Reinforcement Learning in Quantum Circuit Design

Philipp J. Altmann, Jonas Stein, Michael Kölle +6 more

Quantum computing (QC) in the current NISQ era is still limited in size and precision. Hybrid applications mitigating those shortcomings are prevalent to gain early insight and advantages. Hybrid quantum machine learning (QML) comprises both the application of QC to improve machine learning (ML) and ML to improve QC architectures. This work considers the latter, leveraging reinforcement learning (RL) to improve quantum circuit design (QCD), which we formalize by a set of generic objectives. Furthermore, we propose qcd-gym, a concrete framework formalized as a Markov decision process, to enable learning policies capable of controlling a universal set of continuously parameterized quantum gates. Finally, we provide benchmark comparisons to assess the shortcomings and strengths of current state-of-the-art RL algorithms.

arXiv:2312.11337•IEEE International Conference on Quantum Computing and Engineering (QCE)•Sep 15, 2024•11 citations

Reinforcement Learning for Quantum Technology

Marin Bukov, Florian Marquardt

Many challenges arising in Quantum Technology can be successfully addressed using a set of machine learning algorithms collectively known as reinforcement learning (RL), based on adaptive decision-making through interaction with the quantum device. After a concise and intuitive introduction to RL aimed at a broad physics readership, we discuss the key ideas and core concepts in reinforcement learning with a particular focus on quantum systems. We then survey recent progress in RL in all relevant areas. We discuss state preparation in few- and many-body quantum systems, the design and optimization of high-fidelity quantum gates, and the automated construction of quantum circuits, including applications to variational quantum eigensolvers and architecture search. We further highlight the interactive capabilities of RL agents, emphasizing recent progress in quantum feedback control and quantum error correction, and briefly discuss quantum reinforcement learning as well as applications to quantum metrology. The review concludes with a discussion of open challenges -- such as scalability, interpretability, and integration with experimental platforms -- and outlines promising directions for future research. Throughout, we highlight experimental implementations that exemplify the increasing role of reinforcement learning in shaping the development of quantum technologies.

quant-pharXiv:2601.18953•Jan 28, 2026•0 citations

Quantum Architecture Search for Solving Quantum Machine Learning Tasks

Michael Kölle, Simon Salfer, Tobias Rohe +2 more

Quantum computing leverages quantum mechanics to address computational problems in ways that differ fundamentally from classical approaches. While current quantum hardware remains error-prone and limited in scale, Variational Quantum Circuits offer a noise-resilient framework suitable for today's devices. The performance of these circuits strongly depends on the underlying architecture of their parameterized quantum components. Identifying efficient, hardware-compatible quantum circuit architectures -- known as Quantum Architecture Search (QAS) -- is therefore essential. Manual QAS is complex and error-prone, motivating efforts to automate it. Among various automated strategies, Reinforcement Learning (RL) remains underexplored, particularly in Quantum Machine Learning contexts. This work introduces RL-QAS, a framework that applies RL to discover effective circuit architectures for classification tasks. We evaluate RL-QAS using the Iris and binary MNIST datasets. The agent autonomously discovers low-complexity circuit designs that achieve high test accuracy. Our results show that RL is a viable approach for automated architecture search in quantum machine learning. However, applying RL-QAS to more complex tasks will require further refinement of the search strategy and performance evaluation mechanisms.

arXiv:2509.11198•International Conference on Agents and Artificial Intelligence (ICAART)•Jan 1, 2026•0 citations

Quantum Architecture Search via Continual Reinforcement Learning

Esther Ye, Samuel Yen-Chi Chen

Quantum computing has promised significant improvement in solving difficult computational tasks over classical computers. Designing quantum circuits for practical use, however, is not a trivial objective and requires expert-level knowledge. To aid this endeavor, this paper proposes a machine learning-based method to construct quantum circuit architectures. Previous works have demonstrated that classical deep reinforcement learning (DRL) algorithms can successfully construct quantum circuit architectures without encoded physics knowledge. However, these DRL-based works are not generalizable to settings with changing device noises, thus requiring considerable amounts of training resources to keep the RL models up-to-date. With this in mind, we incorporated continual learning to enhance the performance of our algorithm. In this paper, we present the Probabilistic Policy Reuse with deep Q-learning (PPR-DQL) framework to tackle this circuit design challenge. By conducting numerical simulations over various noise patterns, we demonstrate that the RL agent with PPR was able to find the quantum gate sequence to generate the two-qubit Bell state faster than the agent that was trained from scratch. The proposed framework is general and can be applied to other quantum gate synthesis or control problems -- including the automatic calibration of quantum devices.

arXiv:2112.05779•Dec 10, 2021•15 citations

QAS-QTNs: Curriculum Reinforcement Learning-Driven Quantum Architecture Search for Quantum Tensor Networks

Siddhant Dutta, Nouhaila Innan, Sadok Ben Yahia +1 more

Quantum Architecture Search (QAS) is an emerging field aimed at automating the design of quantum circuits for optimal performance. This paper introduces a novel QAS framework employing hybrid quantum reinforcement learning with quantum curriculum learning strategies, enabling learning agents to tackle increasingly complex quantum circuit design tasks. We benchmark four state-of-the-art classical reinforcement learning algorithms (A2C, PPO, DDQN, TD3) against their quantum-enhanced counterparts (QA2C, QPPO, QDDQN, QTD3) for optimizing variational quantum circuits (VQCs). Our approach progressively increases circuit depth and gate complexity during training, leveraging parameterized quantum circuits as function approximations. To improve learning efficiency and stability, all algorithms, both classical and quantum, are augmented with Prioritized Experience Replay (PER). Experimental results show that quantum-enhanced RL significantly outperforms classical methods. In a 2-qubit environment, PERQDDQN achieves a success probability of 0.46 with 3,000 optimal successes, surpassing classical PERDDQN ($\mathbf{0. 4 2}, \mathbf{2, 4 0 0}$). In the more complex 3-qubit setting, PERQDDQN and PERQTD3 reach success probabilities of 0.47, with optimal success counts of 3,800 and 3,600, respectively, outperforming their classical counterparts. Additionally, we apply our QAS-QTN approach to a classification problem, where the optimized quantum circuit achieves an accuracy of 90.33%, outperforming quantum models consisting of random ansatz. This hybrid classical-quantum approach leads to faster convergence and more efficient quantum circuit designs, demonstrating its potential for advancing automated quantum architecture search.

arXiv:2507.12013•IEEE International Conference on Quantum Computing and Engineering (QCE)•Aug 30, 2025•1 citation

Reinforcement Learning for Quantum Circuit Design: Using Matrix Representations

Zhiyuan Wang, Chunlin Feng, Poon, Christopher +5 more

Quantum computing promises advantages over classical computing. The manufacturing of quantum hardware is in the infancy stage, called the Noisy Intermediate-Scale Quantum (NISQ) era. A major challenge is automated quantum circuit design that map a quantum circuit to gates in a universal gate set. In this paper, we present a generic MDP modeling and employ Q-learning and DQN algorithms for quantum circuit design. By leveraging the power of deep reinforcement learning, we aim to provide an automatic and scalable approach over traditional hand-crafted heuristic methods.

arXiv:2501.16509•Jan 27, 2025•0 citations
To assess the prospects of using reinforcement learning (RL) for selecting and parameterizing quantum gates to build viable circuit architectures, we introduce the quantum circuit designer (QCD). By considering quantum control a decision-making problem, we strive to profit from advanced RL exploration mechanisms to overcome the need for granular specification and hand-crafted architectures. To evaluate current state-of-the-art RL algorithms, we define generic objectives that arise from quantum architecture search and circuit optimization. Those evaluation results reveal challenges inherent to learning optimal quantum control.
Quantum computing (QC) in the current NISQ era is still limited in size and precision. Hybrid applications mitigating those shortcomings are prevalent to gain early insight and advantages. Hybrid quantum machine learning (QML) comprises both the application of QC to improve machine learning (ML) and ML to improve QC architectures. This work considers the latter, leveraging reinforcement learning (RL) to improve quantum circuit design (QCD), which we formalize by a set of generic objectives. Furthermore, we propose qcd-gym, a concrete framework formalized as a Markov decision process, to enable learning policies capable of controlling a universal set of continuously parameterized quantum gates. Finally, we provide benchmark comparisons to assess the shortcomings and strengths of current state-of-the-art RL algorithms.
Many challenges arising in Quantum Technology can be successfully addressed using a set of machine learning algorithms collectively known as reinforcement learning (RL), based on adaptive decision-making through interaction with the quantum device. After a concise and intuitive introduction to RL aimed at a broad physics readership, we discuss the key ideas and core concepts in reinforcement learning with a particular focus on quantum systems. We then survey recent progress in RL in all relevant areas. We discuss state preparation in few- and many-body quantum systems, the design and optimization of high-fidelity quantum gates, and the automated construction of quantum circuits, including applications to variational quantum eigensolvers and architecture search. We further highlight the interactive capabilities of RL agents, emphasizing recent progress in quantum feedback control and quantum error correction, and briefly discuss quantum reinforcement learning as well as applications to quantum metrology. The review concludes with a discussion of open challenges -- such as scalability, interpretability, and integration with experimental platforms -- and outlines promising directions for future research. Throughout, we highlight experimental implementations that exemplify the increasing role of reinforcement learning in shaping the development of quantum technologies.
Quantum computing leverages quantum mechanics to address computational problems in ways that differ fundamentally from classical approaches. While current quantum hardware remains error-prone and limited in scale, Variational Quantum Circuits offer a noise-resilient framework suitable for today's devices. The performance of these circuits strongly depends on the underlying architecture of their parameterized quantum components. Identifying efficient, hardware-compatible quantum circuit architectures -- known as Quantum Architecture Search (QAS) -- is therefore essential. Manual QAS is complex and error-prone, motivating efforts to automate it. Among various automated strategies, Reinforcement Learning (RL) remains underexplored, particularly in Quantum Machine Learning contexts. This work introduces RL-QAS, a framework that applies RL to discover effective circuit architectures for classification tasks. We evaluate RL-QAS using the Iris and binary MNIST datasets. The agent autonomously discovers low-complexity circuit designs that achieve high test accuracy. Our results show that RL is a viable approach for automated architecture search in quantum machine learning. However, applying RL-QAS to more complex tasks will require further refinement of the search strategy and performance evaluation mechanisms.
Quantum computing has promised significant improvement in solving difficult computational tasks over classical computers. Designing quantum circuits for practical use, however, is not a trivial objective and requires expert-level knowledge. To aid this endeavor, this paper proposes a machine learning-based method to construct quantum circuit architectures. Previous works have demonstrated that classical deep reinforcement learning (DRL) algorithms can successfully construct quantum circuit architectures without encoded physics knowledge. However, these DRL-based works are not generalizable to settings with changing device noises, thus requiring considerable amounts of training resources to keep the RL models up-to-date. With this in mind, we incorporated continual learning to enhance the performance of our algorithm. In this paper, we present the Probabilistic Policy Reuse with deep Q-learning (PPR-DQL) framework to tackle this circuit design challenge. By conducting numerical simulations over various noise patterns, we demonstrate that the RL agent with PPR was able to find the quantum gate sequence to generate the two-qubit Bell state faster than the agent that was trained from scratch. The proposed framework is general and can be applied to other quantum gate synthesis or control problems -- including the automatic calibration of quantum devices.
Quantum Architecture Search (QAS) is an emerging field aimed at automating the design of quantum circuits for optimal performance. This paper introduces a novel QAS framework employing hybrid quantum reinforcement learning with quantum curriculum learning strategies, enabling learning agents to tackle increasingly complex quantum circuit design tasks. We benchmark four state-of-the-art classical reinforcement learning algorithms (A2C, PPO, DDQN, TD3) against their quantum-enhanced counterparts (QA2C, QPPO, QDDQN, QTD3) for optimizing variational quantum circuits (VQCs). Our approach progressively increases circuit depth and gate complexity during training, leveraging parameterized quantum circuits as function approximations. To improve learning efficiency and stability, all algorithms, both classical and quantum, are augmented with Prioritized Experience Replay (PER). Experimental results show that quantum-enhanced RL significantly outperforms classical methods. In a 2-qubit environment, PERQDDQN achieves a success probability of 0.46 with 3,000 optimal successes, surpassing classical PERDDQN (0.42,2,400\mathbf{0. 4 2}, \mathbf{2, 4 0 0}0.42,2,400). In the more complex 3-qubit setting, PERQDDQN and PERQTD3 reach success probabilities of 0.47, with optimal success counts of 3,800 and 3,600, respectively, outperforming their classical counterparts. Additionally, we apply our QAS-QTN approach to a classification problem, where the optimized quantum circuit achieves an accuracy of 90.33%, outperforming quantum models consisting of random ansatz. This hybrid classical-quantum approach leads to faster convergence and more efficient quantum circuit designs, demonstrating its potential for advancing automated quantum architecture search.
Quantum computing promises advantages over classical computing. The manufacturing of quantum hardware is in the infancy stage, called the Noisy Intermediate-Scale Quantum (NISQ) era. A major challenge is automated quantum circuit design that map a quantum circuit to gates in a universal gate set. In this paper, we present a generic MDP modeling and employ Q-learning and DQN algorithms for quantum circuit design. By leveraging the power of deep reinforcement learning, we aim to provide an automatic and scalable approach over traditional hand-crafted heuristic methods.