Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2308.02536 · IEEE International Conference on Quantum Computing and Engineering (QCE)

qgym: A Gym for Training and Benchmarking RL-Based Quantum Compilation

Stan van der Linde, Willem de Kok, Tariq Bontekoe, Sebastian Feld

Published Sep 17, 2023• 16 citations
arXivHTMLPDF

Abstract

Compiling a quantum circuit for specific quantum hardware is a challenging task. Moreover, current quantum computers have severe hardware limitations. To make the most use of the limited resources, the compilation process should be optimized. To improve currents methods, Reinforcement Learning (RL), a technique in which an agent interacts with an environment to learn complex policies to attain a specific goal, can be used. In this work, we present qgym, a software framework derived from the OpenAI gym, together with environments that are specifically tailored towards quantum compilation. The goal of qgym is to connect the research fields of Artificial Intelligence (AI) with quantum compilation by abstracting parts of the process that are irrelevant to either domain. It can be used to train and benchmark RL agents and algorithms in highly customizable environments.

DOI
10.1109/qce57702.2023.10179

Related papers

Dynamic Quantum Circuit Compilation via Reinforcement Learning

Jincheng Zhang

Quantum circuit compilation is a crucial step in realizing quantum algorithms on real quantum hardware. Traditionally, compilation methods are largely static, relying on pre-defined mappings and optimizations that fail to account for the inherent noise and variability present in quantum devices. This paper proposes a novel approach leveraging Reinforcement Learning (RL) to dynamically generate and optimize quantum circuits. An RL agent is trained to learn a policy for circuit design, adapting to specific hardware characteristics and real-time measurement data. The agent's goal is to minimize the overall error rate of the compiled circuit. This dynamic compilation framework offers a feedback loop between hardware performance and circuit design, resulting in adaptive and robust quantum computation. The core claim of this work is that traditional static compilation methods are insufficient, and dynamic compilation driven by RL offers a superior solution. The approach presented here represents a significant advancement towards realizing the full potential of quantum computing by mitigating the effects of hardware imperfections. The framework includes a detailed description of the RL agent architecture, the reward function design, and the exploration strategy employed. The effectiveness of the proposed method is demonstrated through simulations, highlighting its ability to outperform static compilation strategies in noisy environments.

•Sep 5, 2026•0 citations

Unentangled quantum reinforcement learning agents in the OpenAI Gym

Jen-Yueh Hsiao, Yuxuan Du, Wei-Yin Chiang +2 more

Classical reinforcement learning (RL) has generated excellent results in different regions; however, its sample inefficiency remains a critical issue. In this paper, we provide concrete numerical evidence that the sample efficiency (the speed of convergence) of quantum RL could be better than that of classical RL, and for achieving comparable learning performance, quantum RL could use much (at least one order of magnitude) fewer trainable parameters than classical RL. Specifically, we employ the popular benchmarking environments of RL in the OpenAI Gym, and show that our quantum RL agent converges faster than classical fully-connected neural networks (FCNs) in the tasks of CartPole and Acrobot under the same optimization process. We also successfully train the first quantum RL agent that can complete the task of LunarLander in the OpenAI Gym. Our quantum RL agent only requires a single-qubit-based variational quantum circuit without entangling gates, followed by a classical neural network (NN) to post-process the measurement output. Finally, we could accomplish the aforementioned tasks on the real IBM quantum machines. To the best of our knowledge, none of the earlier quantum RL agents could do that.

arXiv:2203.14348•Mar 27, 2022•17 citations

Adaptive Quantum Circuit Compilation via Reinforcement Learning

Jincheng Zhang

Quantum circuit compilation is a critical step in realizing quantum algorithms on near-term quantum hardware. The process of translating high-level quantum algorithms into low-level gate sequences, while minimizing gate count and circuit depth, is often performed using deterministic algorithms that struggle to adapt to the diverse and often imperfect characteristics of available quantum devices. This paper proposes a novel approach to quantum circuit compilation utilizing reinforcement learning (RL). An RL agent is trained to dynamically adapt to hardware constraints, such as qubit connectivity and gate decomposition costs, optimizing circuit fidelity and execution time. The core claim is that RL can significantly improve quantum circuit compilation by introducing a dynamic and adaptive compilation process. The agent learns a policy that maps high-level quantum algorithms to optimized low-level gate sequences, effectively bridging the gap between theoretical algorithms and practical hardware execution. This work demonstrates the potential of RL to overcome the limitations of traditional deterministic compilation methods and paves the way for more efficient and robust quantum algorithm implementation.

•Aug 28, 2026•0 citations

Adaptive Quantum Circuit Compilation with Reinforcement Learning

Jincheng Zhang

This paper presents a novel approach to quantum circuit compilation leveraging reinforcement learning (RL). The core idea is to train an RL agent to dynamically optimize quantum circuit structures, aiming to minimize gate count and enhance overall circuit performance. Traditional quantum circuit compilation methods often rely on static heuristics and pre-defined optimization strategies, which may not be optimal for all quantum algorithms or hardware platforms. Our approach addresses this limitation by allowing the RL agent to learn a policy that adapts to the specific characteristics of the algorithm and the target quantum hardware. The agent learns by interacting with a simulated quantum circuit environment, receiving rewards based on circuit fidelity, depth, and gate count. The results demonstrate the potential of RL to significantly improve the efficiency and robustness of quantum computations. The key components of the system are: a high-level quantum algorithm representation, an RL agent, a quantum circuit simulator, and a reward function. The agent's policy is represented as a neural network, enabling it to generalize across different algorithms and hardware. We demonstrate the effectiveness of this approach through simulations, showcasing its ability to outperform conventional compilation methods in terms of reducing circuit depth and gate count while maintaining high fidelity.

•Sep 5, 2026•0 citations

Quantum circuit optimization using deep reinforcement learning: Applications across multiple gate sets

Khoa Dang Tao, Sumin Jin, Muhammad Raza +1 more

The practical implementation of quantum algorithms on noisy intermediate-scale quantum devices encounters operational limitations due to decoherence and other sources of noise inherent in real hardware. To mitigate these errors while preserving the original functionality of the algorithm, shorter quantum circuits are therefore preferred. This motivates the development of effective quantum circuit optimization algorithms. Learning-based approaches have emerged as a leading candidate, yet existing autonomous agents remain inefficient, spending most of their training capacity rediscovering elementary reductions that deterministic rule-based methods already handle reliably. To address this challenge, we propose a reinforcement learning framework that embeds a deterministic Commutation-and-Reduction (CR) algorithm directly into the training environment. After every agent action, the CR algorithm automatically resolves elementary commutations and cancellations, enabling the agent to focus its learning capacity on the non-trivial optimizations where reinforcement learning adds real value. Empirical evaluation across two gate sets, the universal Clifford+T basis and the CNOT+Pauli basis, shows that RL+CR produces shorter circuits than a standard RL agent at all tested scales. We demonstrate that RL trained on smaller quantum circuits can be applied to larger quantum circuits. On 20-qubit Clifford+T circuits, five times larger than the training circuits, RL+CR removes twice as many gates as standard RL. This work provides a robust approach that could accelerate the compilation and optimization processes for future fault-tolerant and utility-scale quantum systems.

quant-pharXiv:2608.19103•Aug 20, 2026•0 citations

Reinforcement Learning for Quantum Phase Estimation Using Deep Q-Network

B Sundharess, Vikash Krishna R, Gopika SS +3 more

Quantum Phase Estimation(QPE) is a fundamental quantum algorithm that is used for the estimation of eigenphases of unitary operators. Its main goal is to determine the phase associated with each eigenstate. Usually, it take steps such as prepare quantum states, apply controlled unitaries, inverse quantum fourier transformation, and measurement. This study uses the OpenAI Gym framework to build a customized QPE environment. Here, the phase of a randomly generated target unitary operator is estimated using a quantum circuit. Through interaction with this environment, the DQN agent learns the best course of action to increase phase estimation accuracy. It exhibits more flexibility in noisy environments and reduces estimating mistakes. With its insights and approaches for further study in this area, this effort represents a significant advancement in the use of Deep Reinforcement Learning in quantum computing. A Comparative analysis between IBM Quantum(ibm kyiv) and the Aer Simulator on the OpenAI Gym environment using RL agents has been done.

IEEE Madhya Pradesh Section Conference (MPCON)•Aug 29, 2025•0 citations
Compiling a quantum circuit for specific quantum hardware is a challenging task. Moreover, current quantum computers have severe hardware limitations. To make the most use of the limited resources, the compilation process should be optimized. To improve currents methods, Reinforcement Learning (RL), a technique in which an agent interacts with an environment to learn complex policies to attain a specific goal, can be used. In this work, we present qgym, a software framework derived from the OpenAI gym, together with environments that are specifically tailored towards quantum compilation. The goal of qgym is to connect the research fields of Artificial Intelligence (AI) with quantum compilation by abstracting parts of the process that are irrelevant to either domain. It can be used to train and benchmark RL agents and algorithms in highly customizable environments.
Quantum circuit compilation is a crucial step in realizing quantum algorithms on real quantum hardware. Traditionally, compilation methods are largely static, relying on pre-defined mappings and optimizations that fail to account for the inherent noise and variability present in quantum devices. This paper proposes a novel approach leveraging Reinforcement Learning (RL) to dynamically generate and optimize quantum circuits. An RL agent is trained to learn a policy for circuit design, adapting to specific hardware characteristics and real-time measurement data. The agent's goal is to minimize the overall error rate of the compiled circuit. This dynamic compilation framework offers a feedback loop between hardware performance and circuit design, resulting in adaptive and robust quantum computation. The core claim of this work is that traditional static compilation methods are insufficient, and dynamic compilation driven by RL offers a superior solution. The approach presented here represents a significant advancement towards realizing the full potential of quantum computing by mitigating the effects of hardware imperfections. The framework includes a detailed description of the RL agent architecture, the reward function design, and the exploration strategy employed. The effectiveness of the proposed method is demonstrated through simulations, highlighting its ability to outperform static compilation strategies in noisy environments.
Classical reinforcement learning (RL) has generated excellent results in different regions; however, its sample inefficiency remains a critical issue. In this paper, we provide concrete numerical evidence that the sample efficiency (the speed of convergence) of quantum RL could be better than that of classical RL, and for achieving comparable learning performance, quantum RL could use much (at least one order of magnitude) fewer trainable parameters than classical RL. Specifically, we employ the popular benchmarking environments of RL in the OpenAI Gym, and show that our quantum RL agent converges faster than classical fully-connected neural networks (FCNs) in the tasks of CartPole and Acrobot under the same optimization process. We also successfully train the first quantum RL agent that can complete the task of LunarLander in the OpenAI Gym. Our quantum RL agent only requires a single-qubit-based variational quantum circuit without entangling gates, followed by a classical neural network (NN) to post-process the measurement output. Finally, we could accomplish the aforementioned tasks on the real IBM quantum machines. To the best of our knowledge, none of the earlier quantum RL agents could do that.
Quantum circuit compilation is a critical step in realizing quantum algorithms on near-term quantum hardware. The process of translating high-level quantum algorithms into low-level gate sequences, while minimizing gate count and circuit depth, is often performed using deterministic algorithms that struggle to adapt to the diverse and often imperfect characteristics of available quantum devices. This paper proposes a novel approach to quantum circuit compilation utilizing reinforcement learning (RL). An RL agent is trained to dynamically adapt to hardware constraints, such as qubit connectivity and gate decomposition costs, optimizing circuit fidelity and execution time. The core claim is that RL can significantly improve quantum circuit compilation by introducing a dynamic and adaptive compilation process. The agent learns a policy that maps high-level quantum algorithms to optimized low-level gate sequences, effectively bridging the gap between theoretical algorithms and practical hardware execution. This work demonstrates the potential of RL to overcome the limitations of traditional deterministic compilation methods and paves the way for more efficient and robust quantum algorithm implementation.
This paper presents a novel approach to quantum circuit compilation leveraging reinforcement learning (RL). The core idea is to train an RL agent to dynamically optimize quantum circuit structures, aiming to minimize gate count and enhance overall circuit performance. Traditional quantum circuit compilation methods often rely on static heuristics and pre-defined optimization strategies, which may not be optimal for all quantum algorithms or hardware platforms. Our approach addresses this limitation by allowing the RL agent to learn a policy that adapts to the specific characteristics of the algorithm and the target quantum hardware. The agent learns by interacting with a simulated quantum circuit environment, receiving rewards based on circuit fidelity, depth, and gate count. The results demonstrate the potential of RL to significantly improve the efficiency and robustness of quantum computations. The key components of the system are: a high-level quantum algorithm representation, an RL agent, a quantum circuit simulator, and a reward function. The agent's policy is represented as a neural network, enabling it to generalize across different algorithms and hardware. We demonstrate the effectiveness of this approach through simulations, showcasing its ability to outperform conventional compilation methods in terms of reducing circuit depth and gate count while maintaining high fidelity.
The practical implementation of quantum algorithms on noisy intermediate-scale quantum devices encounters operational limitations due to decoherence and other sources of noise inherent in real hardware. To mitigate these errors while preserving the original functionality of the algorithm, shorter quantum circuits are therefore preferred. This motivates the development of effective quantum circuit optimization algorithms. Learning-based approaches have emerged as a leading candidate, yet existing autonomous agents remain inefficient, spending most of their training capacity rediscovering elementary reductions that deterministic rule-based methods already handle reliably. To address this challenge, we propose a reinforcement learning framework that embeds a deterministic Commutation-and-Reduction (CR) algorithm directly into the training environment. After every agent action, the CR algorithm automatically resolves elementary commutations and cancellations, enabling the agent to focus its learning capacity on the non-trivial optimizations where reinforcement learning adds real value. Empirical evaluation across two gate sets, the universal Clifford+T basis and the CNOT+Pauli basis, shows that RL+CR produces shorter circuits than a standard RL agent at all tested scales. We demonstrate that RL trained on smaller quantum circuits can be applied to larger quantum circuits. On 20-qubit Clifford+T circuits, five times larger than the training circuits, RL+CR removes twice as many gates as standard RL. This work provides a robust approach that could accelerate the compilation and optimization processes for future fault-tolerant and utility-scale quantum systems.
Quantum Phase Estimation(QPE) is a fundamental quantum algorithm that is used for the estimation of eigenphases of unitary operators. Its main goal is to determine the phase associated with each eigenstate. Usually, it take steps such as prepare quantum states, apply controlled unitaries, inverse quantum fourier transformation, and measurement. This study uses the OpenAI Gym framework to build a customized QPE environment. Here, the phase of a randomly generated target unitary operator is estimated using a quantum circuit. Through interaction with this environment, the DQN agent learns the best course of action to increase phase estimation accuracy. It exhibits more flexibility in noisy environments and reduces estimating mistakes. With its insights and approaches for further study in this area, this effort represents a significant advancement in the use of Deep Reinforcement Learning in quantum computing. A Comparative analysis between IBM Quantum(ibm kyiv) and the Aer Simulator on the OpenAI Gym environment using RL agents has been done.