Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2605.04628

Intelligent Optimal Control of Rydberg Gates with Incremental-Update Deep Reinforcement Learning

Yue Cai, Hanlin Zhang, Keye Zhang, Jing Qian

quant-phSubmitted May 6, 2026• Announced May 7, 2026• 0 citations
arXivHTMLPDF

Abstract

Deep reinforcement learning (DRL), acting as a novel and powerful paradigm for quantum optimal control, offers transformative opportunities for advancing neutral-atom quantum computing. In this work, we theoretically demonstrate a DRL-based framework for realizing Rydberg controlled-NOT gates that achieve both high speed and high fidelity through the synchronous modulation of multiple pulse parameters without any prior heuristic ansatz. By introducing an incremental-update learning policy, our framework effectively regularizes the exploration of the control landscape, ensuring the generation of smooth, experimentally feasible pulse profiles while significantly reducing computational overhead compared to conventional schemes. Crucially, the framework autonomously discovers an early-cutoff policy by optimally reconciling operation speed with high-precision coherent control. Our optimized protocol achieves a peak average fidelity of 0.9991, significantly outperforming conventional methods and surpassing the critical fault-tolerant threshold. This work establishes a generalizable, AI-driven pathway for designing high-performance quantum gates and provides a robust paradigm for autonomous control field optimization across diverse qubit platforms.

Related papers

Deep Reinforcement Learning Quantum Control on IBMQ Platforms and Qiskit Pulse

Rudy Semola, Lorenzo Del Moro, Davide Bacciu +1 more

Quantum optimal control theory (QOCT) typically addresses the control of physical qubits. Finding explicit pulse control sequences in such a framework is challenging, especially when an underlying physical model is unknown. We propose a deep reinforcement learning (DRL) method, which doesn't require any underlying gate model or qubit pre-calibration, capable of controlling a superconductive qubit via analog pulses acting in the IBM Qiskit Pulse environment. We applied the method to build a single-qubit gate with high fidelity and short duration at pulse level. In particular, the DRL agent approximated the X90 gate at the physical layer on the IBM Armonk transmon superconductive qubit simulated by the Qiskit Pulse simulator. The learned sequence has an average gate fidelity greater than 0.978 and a duration of 58 ns only, faster than the default X90 pulse IBM implementation, which has a runtime of 140 ns. Without prior knowledge and gate model knowledge, the agent learned a non-traditional shaped microwave pulse, providing an alternative strategy for controlling noisy quantum states.

IEEE International Conference on Quantum Computing and Engineering (QCE)•Sep 1, 2022•4 citations

Reinforcement Learning for Robust Calibration of Multi-Qudit Quantum Gates

Amine Jaouadi, Sahel Ashhab

Higher-dimensional quantum systems, such as qudits, offer architectural and algorithmic advantages over qubits, but their increased spectral crowding and limited controllability render high-fidelity quantum gates particularly challenging. We propose a hybrid optimization framework that integrates optimal control theory methods with contextual deep reinforcement learning to achieve robust controlled-phase gates on two qutrits. Optimal control is first used to design high-fidelity control pulses for a nominal system model. Reinforcement learning is then employed as a calibration stage that learns small residual corrections to these pulses in the presence of static model mismatch, thereby preserving good gate performance under realistic parameter uncertainties. By learning structured, low-dimensional residual corrections conditioned on device-specific parameter variations, reinforcement learning enhances the transfer robustness of nominally optimal but parameter-sensitive control solutions across ensembles of devices. Crucially, the reinforcement learning step in our framework does not compete with the optimal control step but provides the adaptability required for realistic hardware, systematically reducing the sensitivity to parameter fluctuations. Our results establish reinforcement learning as a practical and scalable ingredient for robust calibration of quantum gates in high-dimensional systems.

quant-pharXiv:2604.19990•Physical Review A•Apr 23, 2026•1 citation

Designing fast quantum gates using optimal control with a reinforcement-learning ansatz

Bijita Sarma, Michael J. Hartmann

Fast quantum gates are crucial not only for the contemporary era of noisy intermediate-scale quantum devices but also for the prospective development of practical fault-tolerant quantum computing. Leakage errors, which arise from data qubits jumping beyond the confines of the computational subspace, are the main challenges in realizing nonadiabatically driven, fast gates. In this work, we propose and illustrate the usefulness of reinforcement learning (RL) to generate fast two-qubit gates in practical multilevel superconducting qubits. In particular, we show that the RL controller offers great effectiveness in finding piecewise constant gate-pulse sequences that act on two transmon data qubits coupled by a tunable coupler to generate a controlled-$Z$ (cz) gate with a gate time of 10 ns and an error rate of approximately $4\ifmmode\times\else\texttimes\fi{}{10}^{\ensuremath{-}3}$. Using a gradient-based method to solve the same optimization problem often does not achieve high fidelity for such fast gates. However, we show that using the gate pulses discovered by RL as an ansatz for the gradient-based controller can substantially enhance fidelity compared to using RL alone. While for a 10-ns pulse, this improvement is marginal, the combined RL + gradient approach decreases the gate errors below ${10}^{\ensuremath{-}4}$ for a gate of length 20 ns.

arXiv:2312.16358•Physical Review Applied•Jan 6, 2025•7 citations

Auxiliary Task-Based Deep Reinforcement Learning for Quantum Control

Shumin Zhou, Hailan Ma, Sen Kuang +1 more

Due to its property of not requiring prior knowledge of the environment, reinforcement learning (RL) has significant potential for solving quantum control problems. In this work, we investigate the effectiveness of continuous control policies based on deep deterministic policy gradient. To achieve good control of quantum systems with high fidelity, we propose an auxiliary task-based deep RL (AT-DRL) for quantum control. In particular, we design an auxiliary task to predict the fidelity value, sharing partial parameters with the main network (from the main RL task). The auxiliary task learns synchronously with the main task, allowing one to extract intrinsic features of the environment, thus aiding the agent to achieve the desired state with high fidelity. To further enhance the control performance, we also design a guided reward function based on the fidelity of quantum states that enables gradual fidelity improvement. Numerical simulations demonstrate that the proposed AT-DRL can provide a good solution to the exploration of quantum dynamics. It not only achieves high task fidelities but also demonstrates fast learning rates. Moreover, AT-DRL has great potential in designing control pulses that achieve effective quantum state preparation.

arXiv:2302.14312•IEEE Transactions on Cybernetics•Jan 7, 2025•6 citations

Achieving fast and robust perfect entangling gates via reinforcement learning

Leander Grech, Matthias G. Krauss, Mirko Consiglio +4 more

Noisy intermediate-scale quantum computers hold the promise of tackling complex and otherwise intractable computational challenges through the massive parallelism offered by qubits. Central to realizing the potential of quantum computing are perfect entangling (PE) two-qubit gates, which serve as a critical building block for universal quantum computation. In the context of quantum optimal control, shaping electromagnetic pulses to drive quantum gates is crucial for pushing gate performance toward theoretical limits. In this work, we leverage reinforcement learning (RL) techniques to discover near-optimal pulse shapes that yield PE gates. A collection of RL agents is trained within robust simulation environments, enabling the identification of effective control strategies even under noisy conditions. Selected agents are then validated on higher-fidelity simulations, illustrating how RL-based methods can reduce calibration overhead when compared to quantum optimal control techniques. Furthermore, the RL approach is hardware agnostic with the potential for broad applicability across various quantum computing platforms.

quant-pharXiv:2511.07076•Quantum Science and Technology•Nov 11, 2025•2 citations

Hardware Co-Designed Optimal Control for Programmable Atomic Quantum Processors via Reinforcement Learning

Ding, Qian, Dirk Englund

Developing scalable, fault-tolerant atomic quantum processors requires precise control over large arrays of optical beams. This remains a major challenge due to inherent imperfections in classical control hardware, such as inter-channel crosstalk and beam leakage. In this work, we introduce a hardware co-designed intelligent quantum control framework to address these limitations. We construct a mathematical model of the photonic control hardware, integrate it into the quantum optimal control (QOC) framework, and apply reinforcement learning (RL) techniques to discover optimal control strategies. We demonstrate that the proposed framework enables robust, high-fidelity parallel single-qubit gate operations under realistic control conditions, where each atom is individually addressed by an optical beam. Specifically, we implement and benchmark three optimization strategies: a classical hybrid Self-Adaptive Differential Evolution-Adam (SADE-Adam) optimizer, a conventional RL approach based on Proximal Policy Optimization (PPO), and a novel end-to-end differentiable RL method. Using SADE-Adam as a baseline, we find that while PPO performance degrades as system complexity increases, the end-to-end differentiable RL consistently achieves gate fidelities above 99.9$\%$, exhibits faster convergence, and maintains robustness under varied channel crosstalk strength and randomized dynamic control imperfections.

arXiv:2504.11737•Apr 16, 2025•0 citations
Deep reinforcement learning (DRL), acting as a novel and powerful paradigm for quantum optimal control, offers transformative opportunities for advancing neutral-atom quantum computing. In this work, we theoretically demonstrate a DRL-based framework for realizing Rydberg controlled-NOT gates that achieve both high speed and high fidelity through the synchronous modulation of multiple pulse parameters without any prior heuristic ansatz. By introducing an incremental-update learning policy, our framework effectively regularizes the exploration of the control landscape, ensuring the generation of smooth, experimentally feasible pulse profiles while significantly reducing computational overhead compared to conventional schemes. Crucially, the framework autonomously discovers an early-cutoff policy by optimally reconciling operation speed with high-precision coherent control. Our optimized protocol achieves a peak average fidelity of 0.9991, significantly outperforming conventional methods and surpassing the critical fault-tolerant threshold. This work establishes a generalizable, AI-driven pathway for designing high-performance quantum gates and provides a robust paradigm for autonomous control field optimization across diverse qubit platforms.
Quantum optimal control theory (QOCT) typically addresses the control of physical qubits. Finding explicit pulse control sequences in such a framework is challenging, especially when an underlying physical model is unknown. We propose a deep reinforcement learning (DRL) method, which doesn't require any underlying gate model or qubit pre-calibration, capable of controlling a superconductive qubit via analog pulses acting in the IBM Qiskit Pulse environment. We applied the method to build a single-qubit gate with high fidelity and short duration at pulse level. In particular, the DRL agent approximated the X90 gate at the physical layer on the IBM Armonk transmon superconductive qubit simulated by the Qiskit Pulse simulator. The learned sequence has an average gate fidelity greater than 0.978 and a duration of 58 ns only, faster than the default X90 pulse IBM implementation, which has a runtime of 140 ns. Without prior knowledge and gate model knowledge, the agent learned a non-traditional shaped microwave pulse, providing an alternative strategy for controlling noisy quantum states.
Higher-dimensional quantum systems, such as qudits, offer architectural and algorithmic advantages over qubits, but their increased spectral crowding and limited controllability render high-fidelity quantum gates particularly challenging. We propose a hybrid optimization framework that integrates optimal control theory methods with contextual deep reinforcement learning to achieve robust controlled-phase gates on two qutrits. Optimal control is first used to design high-fidelity control pulses for a nominal system model. Reinforcement learning is then employed as a calibration stage that learns small residual corrections to these pulses in the presence of static model mismatch, thereby preserving good gate performance under realistic parameter uncertainties. By learning structured, low-dimensional residual corrections conditioned on device-specific parameter variations, reinforcement learning enhances the transfer robustness of nominally optimal but parameter-sensitive control solutions across ensembles of devices. Crucially, the reinforcement learning step in our framework does not compete with the optimal control step but provides the adaptability required for realistic hardware, systematically reducing the sensitivity to parameter fluctuations. Our results establish reinforcement learning as a practical and scalable ingredient for robust calibration of quantum gates in high-dimensional systems.
Fast quantum gates are crucial not only for the contemporary era of noisy intermediate-scale quantum devices but also for the prospective development of practical fault-tolerant quantum computing. Leakage errors, which arise from data qubits jumping beyond the confines of the computational subspace, are the main challenges in realizing nonadiabatically driven, fast gates. In this work, we propose and illustrate the usefulness of reinforcement learning (RL) to generate fast two-qubit gates in practical multilevel superconducting qubits. In particular, we show that the RL controller offers great effectiveness in finding piecewise constant gate-pulse sequences that act on two transmon data qubits coupled by a tunable coupler to generate a controlled-ZZZ (cz) gate with a gate time of 10 ns and an error rate of approximately $4\ifmmode\times\else\texttimes\fi{}{10}^{\ensuremath{-}3}$. Using a gradient-based method to solve the same optimization problem often does not achieve high fidelity for such fast gates. However, we show that using the gate pulses discovered by RL as an ansatz for the gradient-based controller can substantially enhance fidelity compared to using RL alone. While for a 10-ns pulse, this improvement is marginal, the combined RL + gradient approach decreases the gate errors below ${10}^{\ensuremath{-}4}$ for a gate of length 20 ns.
Due to its property of not requiring prior knowledge of the environment, reinforcement learning (RL) has significant potential for solving quantum control problems. In this work, we investigate the effectiveness of continuous control policies based on deep deterministic policy gradient. To achieve good control of quantum systems with high fidelity, we propose an auxiliary task-based deep RL (AT-DRL) for quantum control. In particular, we design an auxiliary task to predict the fidelity value, sharing partial parameters with the main network (from the main RL task). The auxiliary task learns synchronously with the main task, allowing one to extract intrinsic features of the environment, thus aiding the agent to achieve the desired state with high fidelity. To further enhance the control performance, we also design a guided reward function based on the fidelity of quantum states that enables gradual fidelity improvement. Numerical simulations demonstrate that the proposed AT-DRL can provide a good solution to the exploration of quantum dynamics. It not only achieves high task fidelities but also demonstrates fast learning rates. Moreover, AT-DRL has great potential in designing control pulses that achieve effective quantum state preparation.
Noisy intermediate-scale quantum computers hold the promise of tackling complex and otherwise intractable computational challenges through the massive parallelism offered by qubits. Central to realizing the potential of quantum computing are perfect entangling (PE) two-qubit gates, which serve as a critical building block for universal quantum computation. In the context of quantum optimal control, shaping electromagnetic pulses to drive quantum gates is crucial for pushing gate performance toward theoretical limits. In this work, we leverage reinforcement learning (RL) techniques to discover near-optimal pulse shapes that yield PE gates. A collection of RL agents is trained within robust simulation environments, enabling the identification of effective control strategies even under noisy conditions. Selected agents are then validated on higher-fidelity simulations, illustrating how RL-based methods can reduce calibration overhead when compared to quantum optimal control techniques. Furthermore, the RL approach is hardware agnostic with the potential for broad applicability across various quantum computing platforms.
Developing scalable, fault-tolerant atomic quantum processors requires precise control over large arrays of optical beams. This remains a major challenge due to inherent imperfections in classical control hardware, such as inter-channel crosstalk and beam leakage. In this work, we introduce a hardware co-designed intelligent quantum control framework to address these limitations. We construct a mathematical model of the photonic control hardware, integrate it into the quantum optimal control (QOC) framework, and apply reinforcement learning (RL) techniques to discover optimal control strategies. We demonstrate that the proposed framework enables robust, high-fidelity parallel single-qubit gate operations under realistic control conditions, where each atom is individually addressed by an optical beam. Specifically, we implement and benchmark three optimization strategies: a classical hybrid Self-Adaptive Differential Evolution-Adam (SADE-Adam) optimizer, a conventional RL approach based on Proximal Policy Optimization (PPO), and a novel end-to-end differentiable RL method. Using SADE-Adam as a baseline, we find that while PPO performance degrades as system complexity increases, the end-to-end differentiable RL consistently achieves gate fidelities above 99.9%\%%, exhibits faster convergence, and maintains robustness under varied channel crosstalk strength and randomized dynamic control imperfections.