Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2305.01169 · IEEE International Conference on Quantum Computing and Engineering (QCE)

Fast Quantum Gate Design with Deep Reinforcement Learning Using Real-Time Feedback on Readout Signals

Emily Wright, Rogério de Sousa

Published Sep 17, 2023• 4 citations
arXivHTMLPDF

Abstract

The design of high-fidelity quantum gates is difficult because it requires the optimization of two competing effects, namely maximizing gate speed and minimizing leakage out of the qubit subspace. We propose a deep reinforcement learning algorithm that uses two agents to address the speed and leakage challenges simultaneously. The first agent constructs the qubit in-phase control pulse using a policy learned from rewards that compensate short gate times. The rewards are obtained at intermediate time steps throughout the construction of a full-length pulse, allowing the agent to explore the landscape of shorter pulses. The second agent determines an out-of-phase pulse to target leakage. Both agents are trained on real-time data from noisy hardware, thus providing model-free gate design that adapts to unpredictable hardware noise. To reduce the effect of measurement classification errors, the agents are trained directly on the readout signal from probing the qubit. We present proof-of-concept experiments by designing X and square root of X gates of various durations on IBM hardware. After just 200 training iterations, our algorithm is able to construct novel control pulses up to two times faster than the default IBM gates, while matching their performance in terms of state fidelity and leakage rate. As the length of our custom control pulses increases, they begin to outperform the default gates. Improvements to the speed and fidelity of gate operations open the way for higher circuit depth in quantum simulation, quantum chemistry and other algorithms on near-term and future quantum devices.

DOI
10.1109/qce57702.2023.00146

Related papers

Universal Quantum Control through Deep Reinforcement Learning

Murphy Yuezhen Niu, Sergio Boixo, Vadim Smelyanskiy +1 more

Abstract Emerging reinforcement learning techniques using deep neural networks have shown great promise in control optimization. They harness non-local regularities of noisy control trajectories and facilitate transfer learning between tasks. To leverage these powerful capabilities for quantum control optimization, we propose a new control framework to simultaneously optimize the speed and fidelity of quantum computation against both leakage and stochastic control errors. For a broad family of two-qubit unitary gates that are important for quantum simulation of many-electron systems, we improve the control robustness by adding control noise into training environments for reinforcement learning agents trained with trusted-region-policy-optimization. The agent control solutions demonstrate a two-order-of-magnitude reduction in average-gate-error over baseline stochastic-gradient-descent solutions and up to a one-order-of-magnitude reduction in gate time from optimal gate synthesis counterparts. These significant improvements in both fidelity and runtime are achieved by combining new physical understandings and state-of-the-art machine learning techniques. Our results open a venue for wider applications in quantum simulation, quantum chemistry and quantum supremacy tests using near-term quantum devices.

arXiv:1803.01857•npj Quantum Information•Jan 6, 2019•387 citations

Realizing a deep reinforcement learning agent discovering real-time feedback control strategies for a quantum system

Kevin Reuer, Jonas Landgraf, Thomas Fösel +10 more

To realize the full potential of quantum technologies, finding good strategies to control quantum information processing devices in real time becomes increasingly important. Usually these strategies require a precise understanding of the device itself, which is generally not available. Model-free reinforcement learning circumvents this need by discovering control strategies from scratch without relying on an accurate description of the quantum system. Furthermore, important tasks like state preparation, gate teleportation and error correction need feedback at time scales much shorter than the coherence time, which for superconducting circuits is in the microsecond range. Developing and training a deep reinforcement learning agent able to operate in this real-time feedback regime has been an open challenge. Here, we have implemented such an agent in the form of a latency-optimized deep neural network on a field-programmable gate array (FPGA). We demonstrate its use to efficiently initialize a superconducting qubit into a target state. To train the agent, we use model-free reinforcement learning that is based solely on measurement data. We study the agent's performance for strong and weak measurements, and for three-level readout, and compare with simple strategies based on thresholding. This demonstration motivates further research towards adoption of reinforcement learning for real-time feedback control of quantum devices and more generally any physical system requiring learnable low-latency feedback control.

arXiv:2210.16715•Oct 30, 2022•3 citations

Deep Reinforcement Learning Quantum Control on IBMQ Platforms and Qiskit Pulse

Rudy Semola, Lorenzo Del Moro, Davide Bacciu +1 more

Quantum optimal control theory (QOCT) typically addresses the control of physical qubits. Finding explicit pulse control sequences in such a framework is challenging, especially when an underlying physical model is unknown. We propose a deep reinforcement learning (DRL) method, which doesn't require any underlying gate model or qubit pre-calibration, capable of controlling a superconductive qubit via analog pulses acting in the IBM Qiskit Pulse environment. We applied the method to build a single-qubit gate with high fidelity and short duration at pulse level. In particular, the DRL agent approximated the X90 gate at the physical layer on the IBM Armonk transmon superconductive qubit simulated by the Qiskit Pulse simulator. The learned sequence has an average gate fidelity greater than 0.978 and a duration of 58 ns only, faster than the default X90 pulse IBM implementation, which has a runtime of 140 ns. Without prior knowledge and gate model knowledge, the agent learned a non-traditional shaped microwave pulse, providing an alternative strategy for controlling noisy quantum states.

IEEE International Conference on Quantum Computing and Engineering (QCE)•Sep 1, 2022•4 citations

Realizing a deep reinforcement learning agent for real-time quantum feedback

Kevin Reuer, Jonas Landgraf, Thomas Fösel +10 more

Realizing the full potential of quantum technologies requires precise real-time control on time scales much shorter than the coherence time. Model-free reinforcement learning promises to discover efficient feedback strategies from scratch without relying on a description of the quantum system. However, developing and training a reinforcement learning agent able to operate in real-time using feedback has been an open challenge. Here, we have implemented such an agent for a single qubit as a sub-microsecond-latency neural network on a field-programmable gate array (FPGA). We demonstrate its use to efficiently initialize a superconducting qubit and train the agent based solely on measurements. Our work is a first step towards adoption of reinforcement learning for the control of quantum devices and more generally any physical device requiring low-latency feedback.

Nature Communications•Nov 6, 2023•54 citations

Designing fast quantum gates using optimal control with a reinforcement-learning ansatz

Bijita Sarma, Michael J. Hartmann

Fast quantum gates are crucial not only for the contemporary era of noisy intermediate-scale quantum devices but also for the prospective development of practical fault-tolerant quantum computing. Leakage errors, which arise from data qubits jumping beyond the confines of the computational subspace, are the main challenges in realizing nonadiabatically driven, fast gates. In this work, we propose and illustrate the usefulness of reinforcement learning (RL) to generate fast two-qubit gates in practical multilevel superconducting qubits. In particular, we show that the RL controller offers great effectiveness in finding piecewise constant gate-pulse sequences that act on two transmon data qubits coupled by a tunable coupler to generate a controlled-$Z$ (cz) gate with a gate time of 10 ns and an error rate of approximately $4\ifmmode\times\else\texttimes\fi{}{10}^{\ensuremath{-}3}$. Using a gradient-based method to solve the same optimization problem often does not achieve high fidelity for such fast gates. However, we show that using the gate pulses discovered by RL as an ansatz for the gradient-based controller can substantially enhance fidelity compared to using RL alone. While for a 10-ns pulse, this improvement is marginal, the combined RL + gradient approach decreases the gate errors below ${10}^{\ensuremath{-}4}$ for a gate of length 20 ns.

arXiv:2312.16358•Physical Review Applied•Jan 6, 2025•7 citations

Reinforcement Learning for Robust Calibration of Multi-Qudit Quantum Gates

Amine Jaouadi, Sahel Ashhab

Higher-dimensional quantum systems, such as qudits, offer architectural and algorithmic advantages over qubits, but their increased spectral crowding and limited controllability render high-fidelity quantum gates particularly challenging. We propose a hybrid optimization framework that integrates optimal control theory methods with contextual deep reinforcement learning to achieve robust controlled-phase gates on two qutrits. Optimal control is first used to design high-fidelity control pulses for a nominal system model. Reinforcement learning is then employed as a calibration stage that learns small residual corrections to these pulses in the presence of static model mismatch, thereby preserving good gate performance under realistic parameter uncertainties. By learning structured, low-dimensional residual corrections conditioned on device-specific parameter variations, reinforcement learning enhances the transfer robustness of nominally optimal but parameter-sensitive control solutions across ensembles of devices. Crucially, the reinforcement learning step in our framework does not compete with the optimal control step but provides the adaptability required for realistic hardware, systematically reducing the sensitivity to parameter fluctuations. Our results establish reinforcement learning as a practical and scalable ingredient for robust calibration of quantum gates in high-dimensional systems.

quant-pharXiv:2604.19990•Physical Review A•Apr 23, 2026•1 citation
The design of high-fidelity quantum gates is difficult because it requires the optimization of two competing effects, namely maximizing gate speed and minimizing leakage out of the qubit subspace. We propose a deep reinforcement learning algorithm that uses two agents to address the speed and leakage challenges simultaneously. The first agent constructs the qubit in-phase control pulse using a policy learned from rewards that compensate short gate times. The rewards are obtained at intermediate time steps throughout the construction of a full-length pulse, allowing the agent to explore the landscape of shorter pulses. The second agent determines an out-of-phase pulse to target leakage. Both agents are trained on real-time data from noisy hardware, thus providing model-free gate design that adapts to unpredictable hardware noise. To reduce the effect of measurement classification errors, the agents are trained directly on the readout signal from probing the qubit. We present proof-of-concept experiments by designing X and square root of X gates of various durations on IBM hardware. After just 200 training iterations, our algorithm is able to construct novel control pulses up to two times faster than the default IBM gates, while matching their performance in terms of state fidelity and leakage rate. As the length of our custom control pulses increases, they begin to outperform the default gates. Improvements to the speed and fidelity of gate operations open the way for higher circuit depth in quantum simulation, quantum chemistry and other algorithms on near-term and future quantum devices.
Abstract Emerging reinforcement learning techniques using deep neural networks have shown great promise in control optimization. They harness non-local regularities of noisy control trajectories and facilitate transfer learning between tasks. To leverage these powerful capabilities for quantum control optimization, we propose a new control framework to simultaneously optimize the speed and fidelity of quantum computation against both leakage and stochastic control errors. For a broad family of two-qubit unitary gates that are important for quantum simulation of many-electron systems, we improve the control robustness by adding control noise into training environments for reinforcement learning agents trained with trusted-region-policy-optimization. The agent control solutions demonstrate a two-order-of-magnitude reduction in average-gate-error over baseline stochastic-gradient-descent solutions and up to a one-order-of-magnitude reduction in gate time from optimal gate synthesis counterparts. These significant improvements in both fidelity and runtime are achieved by combining new physical understandings and state-of-the-art machine learning techniques. Our results open a venue for wider applications in quantum simulation, quantum chemistry and quantum supremacy tests using near-term quantum devices.
To realize the full potential of quantum technologies, finding good strategies to control quantum information processing devices in real time becomes increasingly important. Usually these strategies require a precise understanding of the device itself, which is generally not available. Model-free reinforcement learning circumvents this need by discovering control strategies from scratch without relying on an accurate description of the quantum system. Furthermore, important tasks like state preparation, gate teleportation and error correction need feedback at time scales much shorter than the coherence time, which for superconducting circuits is in the microsecond range. Developing and training a deep reinforcement learning agent able to operate in this real-time feedback regime has been an open challenge. Here, we have implemented such an agent in the form of a latency-optimized deep neural network on a field-programmable gate array (FPGA). We demonstrate its use to efficiently initialize a superconducting qubit into a target state. To train the agent, we use model-free reinforcement learning that is based solely on measurement data. We study the agent's performance for strong and weak measurements, and for three-level readout, and compare with simple strategies based on thresholding. This demonstration motivates further research towards adoption of reinforcement learning for real-time feedback control of quantum devices and more generally any physical system requiring learnable low-latency feedback control.
Quantum optimal control theory (QOCT) typically addresses the control of physical qubits. Finding explicit pulse control sequences in such a framework is challenging, especially when an underlying physical model is unknown. We propose a deep reinforcement learning (DRL) method, which doesn't require any underlying gate model or qubit pre-calibration, capable of controlling a superconductive qubit via analog pulses acting in the IBM Qiskit Pulse environment. We applied the method to build a single-qubit gate with high fidelity and short duration at pulse level. In particular, the DRL agent approximated the X90 gate at the physical layer on the IBM Armonk transmon superconductive qubit simulated by the Qiskit Pulse simulator. The learned sequence has an average gate fidelity greater than 0.978 and a duration of 58 ns only, faster than the default X90 pulse IBM implementation, which has a runtime of 140 ns. Without prior knowledge and gate model knowledge, the agent learned a non-traditional shaped microwave pulse, providing an alternative strategy for controlling noisy quantum states.
Realizing the full potential of quantum technologies requires precise real-time control on time scales much shorter than the coherence time. Model-free reinforcement learning promises to discover efficient feedback strategies from scratch without relying on a description of the quantum system. However, developing and training a reinforcement learning agent able to operate in real-time using feedback has been an open challenge. Here, we have implemented such an agent for a single qubit as a sub-microsecond-latency neural network on a field-programmable gate array (FPGA). We demonstrate its use to efficiently initialize a superconducting qubit and train the agent based solely on measurements. Our work is a first step towards adoption of reinforcement learning for the control of quantum devices and more generally any physical device requiring low-latency feedback.
Fast quantum gates are crucial not only for the contemporary era of noisy intermediate-scale quantum devices but also for the prospective development of practical fault-tolerant quantum computing. Leakage errors, which arise from data qubits jumping beyond the confines of the computational subspace, are the main challenges in realizing nonadiabatically driven, fast gates. In this work, we propose and illustrate the usefulness of reinforcement learning (RL) to generate fast two-qubit gates in practical multilevel superconducting qubits. In particular, we show that the RL controller offers great effectiveness in finding piecewise constant gate-pulse sequences that act on two transmon data qubits coupled by a tunable coupler to generate a controlled-ZZZ (cz) gate with a gate time of 10 ns and an error rate of approximately $4\ifmmode\times\else\texttimes\fi{}{10}^{\ensuremath{-}3}$. Using a gradient-based method to solve the same optimization problem often does not achieve high fidelity for such fast gates. However, we show that using the gate pulses discovered by RL as an ansatz for the gradient-based controller can substantially enhance fidelity compared to using RL alone. While for a 10-ns pulse, this improvement is marginal, the combined RL + gradient approach decreases the gate errors below ${10}^{\ensuremath{-}4}$ for a gate of length 20 ns.
Higher-dimensional quantum systems, such as qudits, offer architectural and algorithmic advantages over qubits, but their increased spectral crowding and limited controllability render high-fidelity quantum gates particularly challenging. We propose a hybrid optimization framework that integrates optimal control theory methods with contextual deep reinforcement learning to achieve robust controlled-phase gates on two qutrits. Optimal control is first used to design high-fidelity control pulses for a nominal system model. Reinforcement learning is then employed as a calibration stage that learns small residual corrections to these pulses in the presence of static model mismatch, thereby preserving good gate performance under realistic parameter uncertainties. By learning structured, low-dimensional residual corrections conditioned on device-specific parameter variations, reinforcement learning enhances the transfer robustness of nominally optimal but parameter-sensitive control solutions across ensembles of devices. Crucially, the reinforcement learning step in our framework does not compete with the optimal control step but provides the adaptability required for realistic hardware, systematically reducing the sensitivity to parameter fluctuations. Our results establish reinforcement learning as a practical and scalable ingredient for robust calibration of quantum gates in high-dimensional systems.