Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2311.03684 · Machine Learning Science and Technology

Reinforcement learning pulses for transmon qubit entangling gates

Hồ Nam Nguyễn, Felix Motzoi, Mekena Metcalf, K. Birgitta Whaley, Marin Bukov, Markus Schmitt

Published May 22, 2024• 14 citations
arXivHTMLPDF

Abstract

Abstract The utility of a quantum computer is highly dependent on the ability to reliably perform accurate quantum logic operations. For finding optimal control solutions, it is of particular interest to explore model-free approaches, since their quality is not constrained by the limited accuracy of theoretical models for the quantum processor—in contrast to many established gate implementation strategies. In this work, we utilize a continuous control reinforcement learning algorithm to design entangling two-qubit gates for superconducting qubits; specifically, our agent constructs cross-resonance and CNOT gates without any prior information about the physical system. Using a simulated environment of fixed-frequency fixed-coupling transmon qubits, we demonstrate the capability to generate novel pulse sequences that outperform the standard cross-resonance gates in both fidelity and gate duration, while maintaining a comparable susceptibility to stochastic unitary noise. We further showcase an augmentation in training and input information that allows our agent to adapt its pulse design abilities to drifting hardware characteristics, importantly, with little to no additional optimization. Our results exhibit clearly the advantages of unbiased adaptive-feedback learning-based optimization methods for transmon gate design.

DOI
10.1088/2632-2153/ad4f4d

Related papers

Fast Quantum Gate Design with Deep Reinforcement Learning Using Real-Time Feedback on Readout Signals

Emily Wright, Rogério de Sousa

The design of high-fidelity quantum gates is difficult because it requires the optimization of two competing effects, namely maximizing gate speed and minimizing leakage out of the qubit subspace. We propose a deep reinforcement learning algorithm that uses two agents to address the speed and leakage challenges simultaneously. The first agent constructs the qubit in-phase control pulse using a policy learned from rewards that compensate short gate times. The rewards are obtained at intermediate time steps throughout the construction of a full-length pulse, allowing the agent to explore the landscape of shorter pulses. The second agent determines an out-of-phase pulse to target leakage. Both agents are trained on real-time data from noisy hardware, thus providing model-free gate design that adapts to unpredictable hardware noise. To reduce the effect of measurement classification errors, the agents are trained directly on the readout signal from probing the qubit. We present proof-of-concept experiments by designing X and square root of X gates of various durations on IBM hardware. After just 200 training iterations, our algorithm is able to construct novel control pulses up to two times faster than the default IBM gates, while matching their performance in terms of state fidelity and leakage rate. As the length of our custom control pulses increases, they begin to outperform the default gates. Improvements to the speed and fidelity of gate operations open the way for higher circuit depth in quantum simulation, quantum chemistry and other algorithms on near-term and future quantum devices.

arXiv:2305.01169•IEEE International Conference on Quantum Computing and Engineering (QCE)•Sep 17, 2023•4 citations

Achieving fast and robust perfect entangling gates via reinforcement learning

Leander Grech, Matthias G. Krauss, Mirko Consiglio +4 more

Noisy intermediate-scale quantum computers hold the promise of tackling complex and otherwise intractable computational challenges through the massive parallelism offered by qubits. Central to realizing the potential of quantum computing are perfect entangling (PE) two-qubit gates, which serve as a critical building block for universal quantum computation. In the context of quantum optimal control, shaping electromagnetic pulses to drive quantum gates is crucial for pushing gate performance toward theoretical limits. In this work, we leverage reinforcement learning (RL) techniques to discover near-optimal pulse shapes that yield PE gates. A collection of RL agents is trained within robust simulation environments, enabling the identification of effective control strategies even under noisy conditions. Selected agents are then validated on higher-fidelity simulations, illustrating how RL-based methods can reduce calibration overhead when compared to quantum optimal control techniques. Furthermore, the RL approach is hardware agnostic with the potential for broad applicability across various quantum computing platforms.

quant-pharXiv:2511.07076•Quantum Science and Technology•Nov 11, 2025•2 citations

Deep Reinforcement Learning Quantum Control on IBMQ Platforms and Qiskit Pulse

Rudy Semola, Lorenzo Del Moro, Davide Bacciu +1 more

Quantum optimal control theory (QOCT) typically addresses the control of physical qubits. Finding explicit pulse control sequences in such a framework is challenging, especially when an underlying physical model is unknown. We propose a deep reinforcement learning (DRL) method, which doesn't require any underlying gate model or qubit pre-calibration, capable of controlling a superconductive qubit via analog pulses acting in the IBM Qiskit Pulse environment. We applied the method to build a single-qubit gate with high fidelity and short duration at pulse level. In particular, the DRL agent approximated the X90 gate at the physical layer on the IBM Armonk transmon superconductive qubit simulated by the Qiskit Pulse simulator. The learned sequence has an average gate fidelity greater than 0.978 and a duration of 58 ns only, faster than the default X90 pulse IBM implementation, which has a runtime of 140 ns. Without prior knowledge and gate model knowledge, the agent learned a non-traditional shaped microwave pulse, providing an alternative strategy for controlling noisy quantum states.

IEEE International Conference on Quantum Computing and Engineering (QCE)•Sep 1, 2022•4 citations

Optimized cross-resonance gate for coupled transmon systems

Susanna Kirchhoff, Torsten Keßler, Per Liebermann +4 more

The cross-resonance (CR) gate is an entangling gate for fixed-frequency superconducting qubits. While being simple and extensible, it is comparatively slow, at 160 ns, and thus of limited fidelity due to on-going incoherent processes. Using two different optimal control algorithms, we estimate the quantum speed limit for a controlled-not cnot gate in this system to be 10 ns, indicating a potential for great improvements. We show that the ability to approach this limit depends strongly on the choice of ansatz used to describe optimized control pulses and limitations placed on their complexity. Using a piecewise-constant ansatz, with a single carrier and bandwidth constraints, we identify an experimentally feasible 70-ns pulse shape. Further, an ansatz based on the two dominant frequencies involved in the optimal control problem allows for an optimal solution more than twice as fast again, at under 30 ns, with smooth features and limited complexity. This is twice as fast as gate realizations using tunable-frequency, resonantly coupled qubits. Compared to current CR-gate implementations, we project our scheme will provide a sixfold speed-up and thus a sixfold reduction in fidelity loss due to incoherent effects.

arXiv:1701.01841•Physical Review A•Apr 30, 2018•56 citations

Robust high-fidelity three-qubit gates design for superconducting quantum computers via model-free reinforcement learning

Xinxin Zhu, Yangyang Fei, Liu FuDong +1 more

Abstract Multi-qubit gates are crucial to realizing quantum algorithms. Direct implementation of these gates will facilitate a reduction in circuit depth. However, existing model-based methods rely on precise characterization of the system Hamiltonian as well as execution of the control fields, which present challenges in obtaining high fidelity multi-qubit gates. In this paper, we exploit the bang–bang control protocol and model-free reinforcement learning to realize several kinds of three-qubit gates for superconducting quantum computers with tunable/fixed frequency qubits, including the iToffoli gate, the parity check gate and the CCZ gate. The simulation results show that the fidelity of each gate can achieve 99.2% with the Adam gradient descent optimizer. Moreover, the proposed method is robust against random noise in the control pulse. Gate fidelity can be maintained over 95% even in the case of Gaussian noise with variance of 0.3. Besides, the impacts of parameter iterative optimization strategies are also analyzed.

Machine Learning Science and Technology•Oct 1, 2026•0 citations

Adaptive Quantum Control via Reinforcement Learning

Jincheng Zhang

This paper investigates the application of reinforcement learning (RL) for adaptive quantum control. Traditional quantum control methods often rely on pre-designed pulse shapes optimized for specific scenarios, lacking adaptability to variations in system parameters or environmental noise. This work proposes a novel framework where an RL agent learns to dynamically adjust quantum control pulses in real-time, maximizing the fidelity of quantum operations. The agent receives feedback from the quantum system, typically measured through state variables, and utilizes this information to optimize the control pulse sequence. This adaptive approach addresses the limitations of static control schemes and promises enhanced robustness and performance in complex quantum systems. The core claim centers on utilizing reinforcement learning to design optimal quantum control pulses. The central mechanism involves a trained controller dynamically adjusting pulses based on system feedback. The presented methodology offers a new approach to quantum control, particularly relevant in noisy environments and systems with parameter variations. The research demonstrates the potential of RL for achieving superior quantum control outcomes compared to conventional techniques. ---

•Sep 4, 2026•0 citations
Abstract The utility of a quantum computer is highly dependent on the ability to reliably perform accurate quantum logic operations. For finding optimal control solutions, it is of particular interest to explore model-free approaches, since their quality is not constrained by the limited accuracy of theoretical models for the quantum processor—in contrast to many established gate implementation strategies. In this work, we utilize a continuous control reinforcement learning algorithm to design entangling two-qubit gates for superconducting qubits; specifically, our agent constructs cross-resonance and CNOT gates without any prior information about the physical system. Using a simulated environment of fixed-frequency fixed-coupling transmon qubits, we demonstrate the capability to generate novel pulse sequences that outperform the standard cross-resonance gates in both fidelity and gate duration, while maintaining a comparable susceptibility to stochastic unitary noise. We further showcase an augmentation in training and input information that allows our agent to adapt its pulse design abilities to drifting hardware characteristics, importantly, with little to no additional optimization. Our results exhibit clearly the advantages of unbiased adaptive-feedback learning-based optimization methods for transmon gate design.
The design of high-fidelity quantum gates is difficult because it requires the optimization of two competing effects, namely maximizing gate speed and minimizing leakage out of the qubit subspace. We propose a deep reinforcement learning algorithm that uses two agents to address the speed and leakage challenges simultaneously. The first agent constructs the qubit in-phase control pulse using a policy learned from rewards that compensate short gate times. The rewards are obtained at intermediate time steps throughout the construction of a full-length pulse, allowing the agent to explore the landscape of shorter pulses. The second agent determines an out-of-phase pulse to target leakage. Both agents are trained on real-time data from noisy hardware, thus providing model-free gate design that adapts to unpredictable hardware noise. To reduce the effect of measurement classification errors, the agents are trained directly on the readout signal from probing the qubit. We present proof-of-concept experiments by designing X and square root of X gates of various durations on IBM hardware. After just 200 training iterations, our algorithm is able to construct novel control pulses up to two times faster than the default IBM gates, while matching their performance in terms of state fidelity and leakage rate. As the length of our custom control pulses increases, they begin to outperform the default gates. Improvements to the speed and fidelity of gate operations open the way for higher circuit depth in quantum simulation, quantum chemistry and other algorithms on near-term and future quantum devices.
Noisy intermediate-scale quantum computers hold the promise of tackling complex and otherwise intractable computational challenges through the massive parallelism offered by qubits. Central to realizing the potential of quantum computing are perfect entangling (PE) two-qubit gates, which serve as a critical building block for universal quantum computation. In the context of quantum optimal control, shaping electromagnetic pulses to drive quantum gates is crucial for pushing gate performance toward theoretical limits. In this work, we leverage reinforcement learning (RL) techniques to discover near-optimal pulse shapes that yield PE gates. A collection of RL agents is trained within robust simulation environments, enabling the identification of effective control strategies even under noisy conditions. Selected agents are then validated on higher-fidelity simulations, illustrating how RL-based methods can reduce calibration overhead when compared to quantum optimal control techniques. Furthermore, the RL approach is hardware agnostic with the potential for broad applicability across various quantum computing platforms.
Quantum optimal control theory (QOCT) typically addresses the control of physical qubits. Finding explicit pulse control sequences in such a framework is challenging, especially when an underlying physical model is unknown. We propose a deep reinforcement learning (DRL) method, which doesn't require any underlying gate model or qubit pre-calibration, capable of controlling a superconductive qubit via analog pulses acting in the IBM Qiskit Pulse environment. We applied the method to build a single-qubit gate with high fidelity and short duration at pulse level. In particular, the DRL agent approximated the X90 gate at the physical layer on the IBM Armonk transmon superconductive qubit simulated by the Qiskit Pulse simulator. The learned sequence has an average gate fidelity greater than 0.978 and a duration of 58 ns only, faster than the default X90 pulse IBM implementation, which has a runtime of 140 ns. Without prior knowledge and gate model knowledge, the agent learned a non-traditional shaped microwave pulse, providing an alternative strategy for controlling noisy quantum states.
The cross-resonance (CR) gate is an entangling gate for fixed-frequency superconducting qubits. While being simple and extensible, it is comparatively slow, at 160 ns, and thus of limited fidelity due to on-going incoherent processes. Using two different optimal control algorithms, we estimate the quantum speed limit for a controlled-not cnot gate in this system to be 10 ns, indicating a potential for great improvements. We show that the ability to approach this limit depends strongly on the choice of ansatz used to describe optimized control pulses and limitations placed on their complexity. Using a piecewise-constant ansatz, with a single carrier and bandwidth constraints, we identify an experimentally feasible 70-ns pulse shape. Further, an ansatz based on the two dominant frequencies involved in the optimal control problem allows for an optimal solution more than twice as fast again, at under 30 ns, with smooth features and limited complexity. This is twice as fast as gate realizations using tunable-frequency, resonantly coupled qubits. Compared to current CR-gate implementations, we project our scheme will provide a sixfold speed-up and thus a sixfold reduction in fidelity loss due to incoherent effects.
Abstract Multi-qubit gates are crucial to realizing quantum algorithms. Direct implementation of these gates will facilitate a reduction in circuit depth. However, existing model-based methods rely on precise characterization of the system Hamiltonian as well as execution of the control fields, which present challenges in obtaining high fidelity multi-qubit gates. In this paper, we exploit the bang–bang control protocol and model-free reinforcement learning to realize several kinds of three-qubit gates for superconducting quantum computers with tunable/fixed frequency qubits, including the iToffoli gate, the parity check gate and the CCZ gate. The simulation results show that the fidelity of each gate can achieve 99.2% with the Adam gradient descent optimizer. Moreover, the proposed method is robust against random noise in the control pulse. Gate fidelity can be maintained over 95% even in the case of Gaussian noise with variance of 0.3. Besides, the impacts of parameter iterative optimization strategies are also analyzed.
This paper investigates the application of reinforcement learning (RL) for adaptive quantum control. Traditional quantum control methods often rely on pre-designed pulse shapes optimized for specific scenarios, lacking adaptability to variations in system parameters or environmental noise. This work proposes a novel framework where an RL agent learns to dynamically adjust quantum control pulses in real-time, maximizing the fidelity of quantum operations. The agent receives feedback from the quantum system, typically measured through state variables, and utilizes this information to optimize the control pulse sequence. This adaptive approach addresses the limitations of static control schemes and promises enhanced robustness and performance in complex quantum systems. The core claim centers on utilizing reinforcement learning to design optimal quantum control pulses. The central mechanism involves a trained controller dynamically adjusting pulses based on system feedback. The presented methodology offers a new approach to quantum control, particularly relevant in noisy environments and systems with parameter variations. The research demonstrates the potential of RL for achieving superior quantum control outcomes compared to conventional techniques. ---