Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Advanced Physics Research

Pulse‐Level Quantum Robust Control with Diffusion‐Based Reinforcement Learning

Yuanjing Zhang, Tao Shang, Chenyi Zhang, Xueyi Guo

Published Mar 19, 2025• 2 citations
Publisher

Abstract

Abstract The pulse‐level quantum control presents a large range of external parameter dependencies, including control field noise, frequency detuning, nonlinearities, and uncertainty of Hamiltonian parameters, which can lead to significant deviation from the target quantum gate. These terms are not usually considered directly in standard optimization scenarios for robustness, but are often found in analytical solutions. The latter are often difficult to emerge and generalize to different settings. This paper proposes a diffusion‐based reinforcement learning method for pulse‐level quantum robust control (PQC‐DBRL) to enhance the robustness of pulse‐level quantum gate control. PQC‐DBRL does not require an accurate Hamiltonian model of the underlying system, effectively mitigating deviations from target quantum gates caused by control field noise and parameter uncertainties. The quantum pulse control problem is formulated as a conditional generative modeling task, leveraging diffusion reinforcement learning to capture unobserved system information. Furthermore, the results show that PQC‐DBRL pulses maintain a fidelity greater than 0.95 for 100% of the cases and greater than 0.999 for 32.16% of the cases, outperforming GRAPE, which achieves 0.999 fidelity for only 12.48% of the cases under the same noise conditions. In large‐scale experiments with repeated gate operations, PQC‐DBRL demonstrates significantly higher resilience to cumulative errors, maintaining fidelity advantages even after 200 gate repetitions. Additionally, when evaluated across different Hamiltonian variations, PQC‐DBRL shows smaller fidelity variance compared to GRAPE, indicating higher robustness against system parameter fluctuations. This paper offers a promising solution to scalable, noise‐resilient quantum control in practical quantum computing applications.

DOI
10.1002/apxr.202400159

Related papers

Uncovering Latent Structures in Robust Pulse Sequences: A Model-Based Reinforcement Learning Approach for Adaptable Quantum Control

Tobias Kiermeyer, Thomas Heydenreich, Léo Van Damme +3 more

Real-time adaptive control of quantum systems requires rapid generation of robust, high-fidelity pulses across a continuous range of operating conditions. Standard optimization algorithms such as gradient-ascent pulse engineering (GRAPE) solve each instance independently, discarding information between runs and requiring costly reinitialization when parameters change. We present an approach to robust optimal quantum control based on model-based reinforcement learning, in which a single neural network -- embedding the Hamiltonian directly into the training pipeline -- generates robust gates across an entire family of gate configurations, without pre-computed training data. Demonstrated on a single-spin (two-level) system, the trained networks produce pulses for arbitrary rotation angles over a range of pulse durations, detunings, and field inhomogeneities in milliseconds, at fidelities comparable to multi-seed GRAPE. The framework is inherently adaptable: any parameter entering the Hamiltonian can serve as a network input, extending the approach to different systems and control settings. Beyond speed, the network reveals structure in the control landscape: it discovers the same structured phase profiles that appear in GRAPE solutions -- made identifiable through fidelity-invariant symmetry transformations -- but more consistently than independent optimization. This consistency enables smooth interpolation across the entire trained parameter space.

quant-pharXiv:2606.24507•Jun 24, 2026•0 citations

Reinforcement Learning for Robust Calibration of Multi-Qudit Quantum Gates

Amine Jaouadi, Sahel Ashhab

Higher-dimensional quantum systems, such as qudits, offer architectural and algorithmic advantages over qubits, but their increased spectral crowding and limited controllability render high-fidelity quantum gates particularly challenging. We propose a hybrid optimization framework that integrates optimal control theory methods with contextual deep reinforcement learning to achieve robust controlled-phase gates on two qutrits. Optimal control is first used to design high-fidelity control pulses for a nominal system model. Reinforcement learning is then employed as a calibration stage that learns small residual corrections to these pulses in the presence of static model mismatch, thereby preserving good gate performance under realistic parameter uncertainties. By learning structured, low-dimensional residual corrections conditioned on device-specific parameter variations, reinforcement learning enhances the transfer robustness of nominally optimal but parameter-sensitive control solutions across ensembles of devices. Crucially, the reinforcement learning step in our framework does not compete with the optimal control step but provides the adaptability required for realistic hardware, systematically reducing the sensitivity to parameter fluctuations. Our results establish reinforcement learning as a practical and scalable ingredient for robust calibration of quantum gates in high-dimensional systems.

quant-pharXiv:2604.19990•Physical Review A•Apr 23, 2026•1 citation

Quantum Gate Control with State Representation for Deep Reinforcement Learning

Yuanjing Zhang, Tao Shang, Chenyi Zhang +1 more

The extreme sensitivity of contemporary hardware to noise, manufacturing variability, and imperfect quantum logic gates are key factors that limit hardware to reliably perform quantum computation at scale. The use of reinforcement learning assisted control techniques for system dynamics optimization in quantum gate control can significantly improve hardware performance and downstream computing power. However, con-trol field noise and Hamiltonian parameter uncertainty can lead to significant deviations from the target quantum gate. As a result, quantum gates lack the generalization ability to the quantum environment, and existing reinforcement learning assisted for quantum state preparation is not suitable for quantum gate control. In this paper, we design QcontrolSR, a state representation for reinforcement learning method, enabling quantum gate control with enhanced generalization ability. It requires no knowledge of a specific Hamiltonian model of the system, or its underlying unknown process. We introduce the state representation of quantum observations and show through demonstrations that QcontrolSR-optimized quantum gate control is insensitive to driving noise of the typical strength in practice. We show that$R_{x}(\pi/2)$gate implemented using QcontrolSR can maintain fidelity greater than 0.95 and 0.99 on dynamical quantum stochastic system. The regions with fidelity greater than 0.99 and 0.999 are 2 and 3 times higher than the model-optimized Quantum gate control, respectively.

International Conference on Quantum Communications, Networking, and Computing (QCNC)•Jul 1, 2024•2 citations

Adaptive Reinforcement Learning for Robust Open Quantum System Control: A Multi-Task Framework with Temporal Optimization

Haftu W. Fentaw, Steve Campbell, Simon Caton

We present a Multi-task Soft Actor-Critic (SAC) Reinforcement Learning framework designed for open-system quantum control across diverse Hamiltonians, which learns optimal pulse sequences while simultaneously discovering problem-specific evolution time T and number of control pulse segments N. Experimental results across 51 Hamiltonian variations demonstrate that the multi-task SAC model is able to generate control pulses that can drive a system, under environment noise, from its initial state to its target state with high fidelities, establishing essential foundations for universal quantum control applicable to realistic noisy quantum devices. Through progressive expansion of the training Hamiltonian set, we investigate if a single multi-task model trained using a given number of sample Hamiltonians can successfully accomplish state-transfer tasks for Hamiltonians drawn from the same Hamiltonian space but not encountered during training. In addition, our Robustness Infidelity Measure (RIM) analysis reveals that SAC trained policies exhibit superior robustness to pulse amplitude perturbations and decoherence rate variations compared to GRAPE-optimized controls.

quant-pharXiv:2605.26925•May 27, 2026•0 citations

Sample-efficient model-based reinforcement learning for quantum control

Irtaza Khalid, Carrie Ann Weidner, EDMOND A. JONCKHEERE +2 more

We propose a model-based reinforcement learning (RL) approach for noisy time-dependent gate optimization with reduced sample complexity over model-free RL. Sample complexity is defined as the number of controller interactions with the physical system. Leveraging an inductive bias, inspired by recent advances in neural ordinary differential equations (ODEs), we use an autodifferentiable ODE, parametrized by a learnable Hamiltonian ansatz, to represent the model approximating the environment, whose time-dependent part, including the control, is fully known. Control alongside Hamiltonian learning of continuous time-independent parameters is addressed through interactions with the system. We demonstrate an order of magnitude advantage in sample complexity of our method over standard model-free RL in preparing some standard unitary gates with closed and open system dynamics, in realistic computational experiments incorporating single-shot measurements, arbitrary Hilbert space truncations, and uncertainty in Hamiltonian parameters. Also, the learned Hamiltonian can be leveraged by existing control methods like GRAPE (gradient ascent pulse engineering) for further gradient-based optimization with the controllers found by RL as initializations. Our algorithm, which we apply to nitrogen vacancy (NV) centers and transmons, is well suited for controlling partially characterized one- and two-qubit systems.

arXiv:2304.09718•Physical Review Research•Oct 2, 2023•15 citations

Meta-learning assisted robust control of universal quantum gates with uncertainties

Shihui Zhang, Zibo Miao, Yu Pan +2 more

Achieving high-fidelity quantum gates is crucial for reliable quantum computing. However, decoherence and control pulse imperfections pose significant challenges in realizing the theoretical fidelity of quantum gates in practical systems. To address these challenges, we propose the meta-reinforcement learning quantum control algorithm (metaQctrl), which leverages a two-layer learning framework to enhance robustness and fidelity. The inner reinforcement learning network focuses on decision making for specific optimization problems, while the outer meta-learning network adapts to varying environments and provides feedback to the inner network. Our comparative analysis regarding the realization of universal quantum gates demonstrates that metaQctrl achieves higher fidelity with fewer control pulses than conventional methods in the presence of uncertainties. These results can contribute to the exploration of the quantum speed limit and facilitate the implementation of quantum circuits with system imperfections involved.

arXiv:2406.07225•npj Quantum Information•May 19, 2025•5 citations
Abstract The pulse‐level quantum control presents a large range of external parameter dependencies, including control field noise, frequency detuning, nonlinearities, and uncertainty of Hamiltonian parameters, which can lead to significant deviation from the target quantum gate. These terms are not usually considered directly in standard optimization scenarios for robustness, but are often found in analytical solutions. The latter are often difficult to emerge and generalize to different settings. This paper proposes a diffusion‐based reinforcement learning method for pulse‐level quantum robust control (PQC‐DBRL) to enhance the robustness of pulse‐level quantum gate control. PQC‐DBRL does not require an accurate Hamiltonian model of the underlying system, effectively mitigating deviations from target quantum gates caused by control field noise and parameter uncertainties. The quantum pulse control problem is formulated as a conditional generative modeling task, leveraging diffusion reinforcement learning to capture unobserved system information. Furthermore, the results show that PQC‐DBRL pulses maintain a fidelity greater than 0.95 for 100% of the cases and greater than 0.999 for 32.16% of the cases, outperforming GRAPE, which achieves 0.999 fidelity for only 12.48% of the cases under the same noise conditions. In large‐scale experiments with repeated gate operations, PQC‐DBRL demonstrates significantly higher resilience to cumulative errors, maintaining fidelity advantages even after 200 gate repetitions. Additionally, when evaluated across different Hamiltonian variations, PQC‐DBRL shows smaller fidelity variance compared to GRAPE, indicating higher robustness against system parameter fluctuations. This paper offers a promising solution to scalable, noise‐resilient quantum control in practical quantum computing applications.
Real-time adaptive control of quantum systems requires rapid generation of robust, high-fidelity pulses across a continuous range of operating conditions. Standard optimization algorithms such as gradient-ascent pulse engineering (GRAPE) solve each instance independently, discarding information between runs and requiring costly reinitialization when parameters change. We present an approach to robust optimal quantum control based on model-based reinforcement learning, in which a single neural network -- embedding the Hamiltonian directly into the training pipeline -- generates robust gates across an entire family of gate configurations, without pre-computed training data. Demonstrated on a single-spin (two-level) system, the trained networks produce pulses for arbitrary rotation angles over a range of pulse durations, detunings, and field inhomogeneities in milliseconds, at fidelities comparable to multi-seed GRAPE. The framework is inherently adaptable: any parameter entering the Hamiltonian can serve as a network input, extending the approach to different systems and control settings. Beyond speed, the network reveals structure in the control landscape: it discovers the same structured phase profiles that appear in GRAPE solutions -- made identifiable through fidelity-invariant symmetry transformations -- but more consistently than independent optimization. This consistency enables smooth interpolation across the entire trained parameter space.
Higher-dimensional quantum systems, such as qudits, offer architectural and algorithmic advantages over qubits, but their increased spectral crowding and limited controllability render high-fidelity quantum gates particularly challenging. We propose a hybrid optimization framework that integrates optimal control theory methods with contextual deep reinforcement learning to achieve robust controlled-phase gates on two qutrits. Optimal control is first used to design high-fidelity control pulses for a nominal system model. Reinforcement learning is then employed as a calibration stage that learns small residual corrections to these pulses in the presence of static model mismatch, thereby preserving good gate performance under realistic parameter uncertainties. By learning structured, low-dimensional residual corrections conditioned on device-specific parameter variations, reinforcement learning enhances the transfer robustness of nominally optimal but parameter-sensitive control solutions across ensembles of devices. Crucially, the reinforcement learning step in our framework does not compete with the optimal control step but provides the adaptability required for realistic hardware, systematically reducing the sensitivity to parameter fluctuations. Our results establish reinforcement learning as a practical and scalable ingredient for robust calibration of quantum gates in high-dimensional systems.
The extreme sensitivity of contemporary hardware to noise, manufacturing variability, and imperfect quantum logic gates are key factors that limit hardware to reliably perform quantum computation at scale. The use of reinforcement learning assisted control techniques for system dynamics optimization in quantum gate control can significantly improve hardware performance and downstream computing power. However, con-trol field noise and Hamiltonian parameter uncertainty can lead to significant deviations from the target quantum gate. As a result, quantum gates lack the generalization ability to the quantum environment, and existing reinforcement learning assisted for quantum state preparation is not suitable for quantum gate control. In this paper, we design QcontrolSR, a state representation for reinforcement learning method, enabling quantum gate control with enhanced generalization ability. It requires no knowledge of a specific Hamiltonian model of the system, or its underlying unknown process. We introduce the state representation of quantum observations and show through demonstrations that QcontrolSR-optimized quantum gate control is insensitive to driving noise of the typical strength in practice. We show thatRx(π/2)R_{x}(\pi/2)Rx​(π/2)gate implemented using QcontrolSR can maintain fidelity greater than 0.95 and 0.99 on dynamical quantum stochastic system. The regions with fidelity greater than 0.99 and 0.999 are 2 and 3 times higher than the model-optimized Quantum gate control, respectively.
We present a Multi-task Soft Actor-Critic (SAC) Reinforcement Learning framework designed for open-system quantum control across diverse Hamiltonians, which learns optimal pulse sequences while simultaneously discovering problem-specific evolution time T and number of control pulse segments N. Experimental results across 51 Hamiltonian variations demonstrate that the multi-task SAC model is able to generate control pulses that can drive a system, under environment noise, from its initial state to its target state with high fidelities, establishing essential foundations for universal quantum control applicable to realistic noisy quantum devices. Through progressive expansion of the training Hamiltonian set, we investigate if a single multi-task model trained using a given number of sample Hamiltonians can successfully accomplish state-transfer tasks for Hamiltonians drawn from the same Hamiltonian space but not encountered during training. In addition, our Robustness Infidelity Measure (RIM) analysis reveals that SAC trained policies exhibit superior robustness to pulse amplitude perturbations and decoherence rate variations compared to GRAPE-optimized controls.
We propose a model-based reinforcement learning (RL) approach for noisy time-dependent gate optimization with reduced sample complexity over model-free RL. Sample complexity is defined as the number of controller interactions with the physical system. Leveraging an inductive bias, inspired by recent advances in neural ordinary differential equations (ODEs), we use an autodifferentiable ODE, parametrized by a learnable Hamiltonian ansatz, to represent the model approximating the environment, whose time-dependent part, including the control, is fully known. Control alongside Hamiltonian learning of continuous time-independent parameters is addressed through interactions with the system. We demonstrate an order of magnitude advantage in sample complexity of our method over standard model-free RL in preparing some standard unitary gates with closed and open system dynamics, in realistic computational experiments incorporating single-shot measurements, arbitrary Hilbert space truncations, and uncertainty in Hamiltonian parameters. Also, the learned Hamiltonian can be leveraged by existing control methods like GRAPE (gradient ascent pulse engineering) for further gradient-based optimization with the controllers found by RL as initializations. Our algorithm, which we apply to nitrogen vacancy (NV) centers and transmons, is well suited for controlling partially characterized one- and two-qubit systems.
Achieving high-fidelity quantum gates is crucial for reliable quantum computing. However, decoherence and control pulse imperfections pose significant challenges in realizing the theoretical fidelity of quantum gates in practical systems. To address these challenges, we propose the meta-reinforcement learning quantum control algorithm (metaQctrl), which leverages a two-layer learning framework to enhance robustness and fidelity. The inner reinforcement learning network focuses on decision making for specific optimization problems, while the outer meta-learning network adapts to varying environments and provides feedback to the inner network. Our comparative analysis regarding the realization of universal quantum gates demonstrates that metaQctrl achieves higher fidelity with fewer control pulses than conventional methods in the presence of uncertainties. These results can contribute to the exploration of the quantum speed limit and facilitate the implementation of quantum circuits with system imperfections involved.