Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2511.07076 · Quantum Science and Technology

Achieving fast and robust perfect entangling gates via reinforcement learning

Leander Grech, Matthias G. Krauss, Mirko Consiglio, Tony J. G. Apollaro, Christiane P. Koch, Simon Hirlaender, Gianluca Valentino

quant-phSubmitted Nov 10, 2025• Announced Nov 11, 2025• 2 citations
arXivHTMLPDF

Abstract

Noisy intermediate-scale quantum computers hold the promise of tackling complex and otherwise intractable computational challenges through the massive parallelism offered by qubits. Central to realizing the potential of quantum computing are perfect entangling (PE) two-qubit gates, which serve as a critical building block for universal quantum computation. In the context of quantum optimal control, shaping electromagnetic pulses to drive quantum gates is crucial for pushing gate performance toward theoretical limits. In this work, we leverage reinforcement learning (RL) techniques to discover near-optimal pulse shapes that yield PE gates. A collection of RL agents is trained within robust simulation environments, enabling the identification of effective control strategies even under noisy conditions. Selected agents are then validated on higher-fidelity simulations, illustrating how RL-based methods can reduce calibration overhead when compared to quantum optimal control techniques. Furthermore, the RL approach is hardware agnostic with the potential for broad applicability across various quantum computing platforms.

Journal reference
Quantum Sci. Technol. 11, 015030 (2026)
DOI
10.1088/2058-9565/ae2c16
Comments
23 pages, 20 figures, accepted in Quantum Science and Technology

Related papers

Robust quantum control using reinforcement learning from demonstration

Shengyong Li, Yidian Fan, Xiangyang Li +6 more

Quantum control requires high-precision and robust control pulses to ensure optimal system performance. However, control sequences generated with a system model may suffer from model bias, leading to low fidelity. While model-free reinforcement learning (RL) methods have been developed to avoid such biases, training an RL agent from scratch can be time-consuming, often taking hours to gather enough samples for convergence. This challenge has hindered the broad application of RL techniques to larger and more complex quantum control issues, limiting their adaptability. In this work, we use Reinforcement Learning from Demonstration (RLfD) to leverage the control sequences generated with system models and further optimize them with RL to avoid model bias. By avoiding learning from scratch and starting with reasonable control pulse shapes, this approach can increase sample efficiency by reducing the number of samples, which can significantly reduce the training time. Thus, this method can effectively handle pulse shapes that are discretized into more than 1000 pieces without compromising final fidelity. We have simulated the preparation of several high-fidelity non-classical states using the RLfD method. We also find that the training process is more stable when using RLfD. In addition, this method is suitable for fast gate calibration using reinforcement learning.

arXiv:2503.21085•npj Quantum Information•Jul 25, 2025•12 citations

Adaptive Quantum Control via Reinforcement Learning

Jincheng Zhang

This paper investigates the application of reinforcement learning (RL) for adaptive quantum control. Traditional quantum control methods often rely on pre-designed pulse shapes optimized for specific scenarios, lacking adaptability to variations in system parameters or environmental noise. This work proposes a novel framework where an RL agent learns to dynamically adjust quantum control pulses in real-time, maximizing the fidelity of quantum operations. The agent receives feedback from the quantum system, typically measured through state variables, and utilizes this information to optimize the control pulse sequence. This adaptive approach addresses the limitations of static control schemes and promises enhanced robustness and performance in complex quantum systems. The core claim centers on utilizing reinforcement learning to design optimal quantum control pulses. The central mechanism involves a trained controller dynamically adjusting pulses based on system feedback. The presented methodology offers a new approach to quantum control, particularly relevant in noisy environments and systems with parameter variations. The research demonstrates the potential of RL for achieving superior quantum control outcomes compared to conventional techniques. ---

•Sep 4, 2026•0 citations

Designing fast quantum gates using optimal control with a reinforcement-learning ansatz

Bijita Sarma, Michael J. Hartmann

Fast quantum gates are crucial not only for the contemporary era of noisy intermediate-scale quantum devices but also for the prospective development of practical fault-tolerant quantum computing. Leakage errors, which arise from data qubits jumping beyond the confines of the computational subspace, are the main challenges in realizing nonadiabatically driven, fast gates. In this work, we propose and illustrate the usefulness of reinforcement learning (RL) to generate fast two-qubit gates in practical multilevel superconducting qubits. In particular, we show that the RL controller offers great effectiveness in finding piecewise constant gate-pulse sequences that act on two transmon data qubits coupled by a tunable coupler to generate a controlled-$Z$ (cz) gate with a gate time of 10 ns and an error rate of approximately $4\ifmmode\times\else\texttimes\fi{}{10}^{\ensuremath{-}3}$. Using a gradient-based method to solve the same optimization problem often does not achieve high fidelity for such fast gates. However, we show that using the gate pulses discovered by RL as an ansatz for the gradient-based controller can substantially enhance fidelity compared to using RL alone. While for a 10-ns pulse, this improvement is marginal, the combined RL + gradient approach decreases the gate errors below ${10}^{\ensuremath{-}4}$ for a gate of length 20 ns.

arXiv:2312.16358•Physical Review Applied•Jan 6, 2025•7 citations

Universal Quantum Control through Deep Reinforcement Learning

Murphy Yuezhen Niu, Sergio Boixo, Vadim Smelyanskiy +1 more

Abstract Emerging reinforcement learning techniques using deep neural networks have shown great promise in control optimization. They harness non-local regularities of noisy control trajectories and facilitate transfer learning between tasks. To leverage these powerful capabilities for quantum control optimization, we propose a new control framework to simultaneously optimize the speed and fidelity of quantum computation against both leakage and stochastic control errors. For a broad family of two-qubit unitary gates that are important for quantum simulation of many-electron systems, we improve the control robustness by adding control noise into training environments for reinforcement learning agents trained with trusted-region-policy-optimization. The agent control solutions demonstrate a two-order-of-magnitude reduction in average-gate-error over baseline stochastic-gradient-descent solutions and up to a one-order-of-magnitude reduction in gate time from optimal gate synthesis counterparts. These significant improvements in both fidelity and runtime are achieved by combining new physical understandings and state-of-the-art machine learning techniques. Our results open a venue for wider applications in quantum simulation, quantum chemistry and quantum supremacy tests using near-term quantum devices.

arXiv:1803.01857•npj Quantum Information•Jan 6, 2019•387 citations

Quantum circuit optimization using deep reinforcement learning: Applications across multiple gate sets

Khoa Dang Tao, Sumin Jin, Muhammad Raza +1 more

The practical implementation of quantum algorithms on noisy intermediate-scale quantum devices encounters operational limitations due to decoherence and other sources of noise inherent in real hardware. To mitigate these errors while preserving the original functionality of the algorithm, shorter quantum circuits are therefore preferred. This motivates the development of effective quantum circuit optimization algorithms. Learning-based approaches have emerged as a leading candidate, yet existing autonomous agents remain inefficient, spending most of their training capacity rediscovering elementary reductions that deterministic rule-based methods already handle reliably. To address this challenge, we propose a reinforcement learning framework that embeds a deterministic Commutation-and-Reduction (CR) algorithm directly into the training environment. After every agent action, the CR algorithm automatically resolves elementary commutations and cancellations, enabling the agent to focus its learning capacity on the non-trivial optimizations where reinforcement learning adds real value. Empirical evaluation across two gate sets, the universal Clifford+T basis and the CNOT+Pauli basis, shows that RL+CR produces shorter circuits than a standard RL agent at all tested scales. We demonstrate that RL trained on smaller quantum circuits can be applied to larger quantum circuits. On 20-qubit Clifford+T circuits, five times larger than the training circuits, RL+CR removes twice as many gates as standard RL. This work provides a robust approach that could accelerate the compilation and optimization processes for future fault-tolerant and utility-scale quantum systems.

quant-pharXiv:2608.19103•Aug 20, 2026•0 citations

Reinforcement learning entangling operations on spin qubits

Abedi, Mohammad, Markus Schmitt

High-fidelity control of one- and two-qubit gates past the error correction threshold is an essential ingredient for scalable quantum computing. We present a reinforcement learning (RL) approach to find entangling protocols for semiconductor-based singlet-triplet qubits in a double quantum dot. Despite the presence of realistically modelled experimental constraints, such as various noise contributions and finite rise-time effects, we demonstrate that an RL agent can yield performative protocols, while avoiding the model-biases of traditional gradient-based methods. We optimise our RL approach for different regimes and tasks, including training from simulated process tomography reconstruction of unitary gates, and investigate the nuances of RL agent design.

arXiv:2508.14761•Aug 20, 2025•0 citations
Noisy intermediate-scale quantum computers hold the promise of tackling complex and otherwise intractable computational challenges through the massive parallelism offered by qubits. Central to realizing the potential of quantum computing are perfect entangling (PE) two-qubit gates, which serve as a critical building block for universal quantum computation. In the context of quantum optimal control, shaping electromagnetic pulses to drive quantum gates is crucial for pushing gate performance toward theoretical limits. In this work, we leverage reinforcement learning (RL) techniques to discover near-optimal pulse shapes that yield PE gates. A collection of RL agents is trained within robust simulation environments, enabling the identification of effective control strategies even under noisy conditions. Selected agents are then validated on higher-fidelity simulations, illustrating how RL-based methods can reduce calibration overhead when compared to quantum optimal control techniques. Furthermore, the RL approach is hardware agnostic with the potential for broad applicability across various quantum computing platforms.
Quantum control requires high-precision and robust control pulses to ensure optimal system performance. However, control sequences generated with a system model may suffer from model bias, leading to low fidelity. While model-free reinforcement learning (RL) methods have been developed to avoid such biases, training an RL agent from scratch can be time-consuming, often taking hours to gather enough samples for convergence. This challenge has hindered the broad application of RL techniques to larger and more complex quantum control issues, limiting their adaptability. In this work, we use Reinforcement Learning from Demonstration (RLfD) to leverage the control sequences generated with system models and further optimize them with RL to avoid model bias. By avoiding learning from scratch and starting with reasonable control pulse shapes, this approach can increase sample efficiency by reducing the number of samples, which can significantly reduce the training time. Thus, this method can effectively handle pulse shapes that are discretized into more than 1000 pieces without compromising final fidelity. We have simulated the preparation of several high-fidelity non-classical states using the RLfD method. We also find that the training process is more stable when using RLfD. In addition, this method is suitable for fast gate calibration using reinforcement learning.
This paper investigates the application of reinforcement learning (RL) for adaptive quantum control. Traditional quantum control methods often rely on pre-designed pulse shapes optimized for specific scenarios, lacking adaptability to variations in system parameters or environmental noise. This work proposes a novel framework where an RL agent learns to dynamically adjust quantum control pulses in real-time, maximizing the fidelity of quantum operations. The agent receives feedback from the quantum system, typically measured through state variables, and utilizes this information to optimize the control pulse sequence. This adaptive approach addresses the limitations of static control schemes and promises enhanced robustness and performance in complex quantum systems. The core claim centers on utilizing reinforcement learning to design optimal quantum control pulses. The central mechanism involves a trained controller dynamically adjusting pulses based on system feedback. The presented methodology offers a new approach to quantum control, particularly relevant in noisy environments and systems with parameter variations. The research demonstrates the potential of RL for achieving superior quantum control outcomes compared to conventional techniques. ---
Fast quantum gates are crucial not only for the contemporary era of noisy intermediate-scale quantum devices but also for the prospective development of practical fault-tolerant quantum computing. Leakage errors, which arise from data qubits jumping beyond the confines of the computational subspace, are the main challenges in realizing nonadiabatically driven, fast gates. In this work, we propose and illustrate the usefulness of reinforcement learning (RL) to generate fast two-qubit gates in practical multilevel superconducting qubits. In particular, we show that the RL controller offers great effectiveness in finding piecewise constant gate-pulse sequences that act on two transmon data qubits coupled by a tunable coupler to generate a controlled-ZZZ (cz) gate with a gate time of 10 ns and an error rate of approximately $4\ifmmode\times\else\texttimes\fi{}{10}^{\ensuremath{-}3}$. Using a gradient-based method to solve the same optimization problem often does not achieve high fidelity for such fast gates. However, we show that using the gate pulses discovered by RL as an ansatz for the gradient-based controller can substantially enhance fidelity compared to using RL alone. While for a 10-ns pulse, this improvement is marginal, the combined RL + gradient approach decreases the gate errors below ${10}^{\ensuremath{-}4}$ for a gate of length 20 ns.
Abstract Emerging reinforcement learning techniques using deep neural networks have shown great promise in control optimization. They harness non-local regularities of noisy control trajectories and facilitate transfer learning between tasks. To leverage these powerful capabilities for quantum control optimization, we propose a new control framework to simultaneously optimize the speed and fidelity of quantum computation against both leakage and stochastic control errors. For a broad family of two-qubit unitary gates that are important for quantum simulation of many-electron systems, we improve the control robustness by adding control noise into training environments for reinforcement learning agents trained with trusted-region-policy-optimization. The agent control solutions demonstrate a two-order-of-magnitude reduction in average-gate-error over baseline stochastic-gradient-descent solutions and up to a one-order-of-magnitude reduction in gate time from optimal gate synthesis counterparts. These significant improvements in both fidelity and runtime are achieved by combining new physical understandings and state-of-the-art machine learning techniques. Our results open a venue for wider applications in quantum simulation, quantum chemistry and quantum supremacy tests using near-term quantum devices.
The practical implementation of quantum algorithms on noisy intermediate-scale quantum devices encounters operational limitations due to decoherence and other sources of noise inherent in real hardware. To mitigate these errors while preserving the original functionality of the algorithm, shorter quantum circuits are therefore preferred. This motivates the development of effective quantum circuit optimization algorithms. Learning-based approaches have emerged as a leading candidate, yet existing autonomous agents remain inefficient, spending most of their training capacity rediscovering elementary reductions that deterministic rule-based methods already handle reliably. To address this challenge, we propose a reinforcement learning framework that embeds a deterministic Commutation-and-Reduction (CR) algorithm directly into the training environment. After every agent action, the CR algorithm automatically resolves elementary commutations and cancellations, enabling the agent to focus its learning capacity on the non-trivial optimizations where reinforcement learning adds real value. Empirical evaluation across two gate sets, the universal Clifford+T basis and the CNOT+Pauli basis, shows that RL+CR produces shorter circuits than a standard RL agent at all tested scales. We demonstrate that RL trained on smaller quantum circuits can be applied to larger quantum circuits. On 20-qubit Clifford+T circuits, five times larger than the training circuits, RL+CR removes twice as many gates as standard RL. This work provides a robust approach that could accelerate the compilation and optimization processes for future fault-tolerant and utility-scale quantum systems.
High-fidelity control of one- and two-qubit gates past the error correction threshold is an essential ingredient for scalable quantum computing. We present a reinforcement learning (RL) approach to find entangling protocols for semiconductor-based singlet-triplet qubits in a double quantum dot. Despite the presence of realistically modelled experimental constraints, such as various noise contributions and finite rise-time effects, we demonstrate that an RL agent can yield performative protocols, while avoiding the model-biases of traditional gradient-based methods. We optimise our RL approach for different regimes and tasks, including training from simulated process tomography reconstruction of unitary gates, and investigate the nuances of RL agent design.