Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

International Conference on Quantum Communications, Networking, and Computing (QCNC)

Quantum Gate Control with State Representation for Deep Reinforcement Learning

Yuanjing Zhang, Tao Shang, Chenyi Zhang, Xueyi Guo

Published Jul 1, 2024• 2 citations
Publisher

Abstract

The extreme sensitivity of contemporary hardware to noise, manufacturing variability, and imperfect quantum logic gates are key factors that limit hardware to reliably perform quantum computation at scale. The use of reinforcement learning assisted control techniques for system dynamics optimization in quantum gate control can significantly improve hardware performance and downstream computing power. However, con-trol field noise and Hamiltonian parameter uncertainty can lead to significant deviations from the target quantum gate. As a result, quantum gates lack the generalization ability to the quantum environment, and existing reinforcement learning assisted for quantum state preparation is not suitable for quantum gate control. In this paper, we design QcontrolSR, a state representation for reinforcement learning method, enabling quantum gate control with enhanced generalization ability. It requires no knowledge of a specific Hamiltonian model of the system, or its underlying unknown process. We introduce the state representation of quantum observations and show through demonstrations that QcontrolSR-optimized quantum gate control is insensitive to driving noise of the typical strength in practice. We show that$R_{x}(\pi/2)$gate implemented using QcontrolSR can maintain fidelity greater than 0.95 and 0.99 on dynamical quantum stochastic system. The regions with fidelity greater than 0.99 and 0.999 are 2 and 3 times higher than the model-optimized Quantum gate control, respectively.

DOI
10.1109/qcnc62729.2024.00028

Related papers

Model-Free Quantum Gate Design and Calibration Using Deep Reinforcement Learning

Omar Shindi, Qi Yu, Parth Girdhar +1 more

High-fidelity quantum gate design is important for various quantum technologies, such as quantum computation and quantum communication. Numerous control policies for quantum gate design have been proposed given a dynamical model of the quantum system of interest. However, a quantum system is often highly sensitive to noise, and obtaining its accurate modeling can be difficult for many practical applications. Thus, the control policy based on a quantum system model may be unpractical for quantum gate design. Also, quantum measurements collapse quantum states, which makes it challenging to obtain information through measurements during the control process. In this article, we propose a novel training framework using deep reinforcement learning for model-free quantum control. The proposed framework relies only on the measurement at the end of the control process and offers the ability to find the optimal control policy without access to quantum systems during the learning process. The effectiveness of the proposed technique is numerically demonstrated for model-free quantum gate design and quantum gate calibration using off-policy reinforcement learning algorithms.

arXiv:2302.02371•IEEE Transactions on Artificial Intelligence•Feb 8, 2023•16 citations

Reinforcement Learning for Robust Calibration of Multi-Qudit Quantum Gates

Amine Jaouadi, Sahel Ashhab

Higher-dimensional quantum systems, such as qudits, offer architectural and algorithmic advantages over qubits, but their increased spectral crowding and limited controllability render high-fidelity quantum gates particularly challenging. We propose a hybrid optimization framework that integrates optimal control theory methods with contextual deep reinforcement learning to achieve robust controlled-phase gates on two qutrits. Optimal control is first used to design high-fidelity control pulses for a nominal system model. Reinforcement learning is then employed as a calibration stage that learns small residual corrections to these pulses in the presence of static model mismatch, thereby preserving good gate performance under realistic parameter uncertainties. By learning structured, low-dimensional residual corrections conditioned on device-specific parameter variations, reinforcement learning enhances the transfer robustness of nominally optimal but parameter-sensitive control solutions across ensembles of devices. Crucially, the reinforcement learning step in our framework does not compete with the optimal control step but provides the adaptability required for realistic hardware, systematically reducing the sensitivity to parameter fluctuations. Our results establish reinforcement learning as a practical and scalable ingredient for robust calibration of quantum gates in high-dimensional systems.

quant-pharXiv:2604.19990•Physical Review A•Apr 23, 2026•1 citation

Universal Quantum Control through Deep Reinforcement Learning

Murphy Yuezhen Niu, Sergio Boixo, Vadim Smelyanskiy +1 more

Abstract Emerging reinforcement learning techniques using deep neural networks have shown great promise in control optimization. They harness non-local regularities of noisy control trajectories and facilitate transfer learning between tasks. To leverage these powerful capabilities for quantum control optimization, we propose a new control framework to simultaneously optimize the speed and fidelity of quantum computation against both leakage and stochastic control errors. For a broad family of two-qubit unitary gates that are important for quantum simulation of many-electron systems, we improve the control robustness by adding control noise into training environments for reinforcement learning agents trained with trusted-region-policy-optimization. The agent control solutions demonstrate a two-order-of-magnitude reduction in average-gate-error over baseline stochastic-gradient-descent solutions and up to a one-order-of-magnitude reduction in gate time from optimal gate synthesis counterparts. These significant improvements in both fidelity and runtime are achieved by combining new physical understandings and state-of-the-art machine learning techniques. Our results open a venue for wider applications in quantum simulation, quantum chemistry and quantum supremacy tests using near-term quantum devices.

arXiv:1803.01857•npj Quantum Information•Jan 6, 2019•387 citations

Deep Reinforcement Learning for quantum state preparation with noisy gates in the NISQ-era

Narvaez Hernandez, Roberto

Quantum computers promise significant computational advantages over classical computers for specific tasks. However, one of the challenges in the noisy intermediate-scale quantum (NISQ) era is the efficient compilation of quantum states and gates, which is crucial for the implementation of quantum algorithms. Given the high dimensionality of quantum systems, this compilation task is non-trivial. Reinforcement learning (RL) is a machine learning subfield focused on developing algorithms for decision-making in dynamic environments. An agent learns by interacting with its environment, how to act to optimize its performance on a certain task. Recently, the integration of deep learning has given rise to deep reinforcement learning (DRL), where deep neural networks approximate functions like value or policy, allowing agents to tackle high-dimensional and complex tasks with greater efficiency and success. Recently, deep reinforcement learning (DRL) has been proposed as a method to automate quantum compiling. In this thesis, we explore the application of RL for quantum state and unitary matrix synthesis, focusing on the use of two reinforcement learning algorithms Deep Q-Networks (DQN) and Proximal Policy Optimization (PPO). Our results indicate that DRL shows promise in generalizing the task of quantum unitary synthesis and quantum state preparation. However, the current state-of-the-art is still limited to working in the no-volume law entangled part of the Hilbert space.

•Sep 9, 2024•0 citations

Adaptive Quantum Control via Reinforcement Learning

Jincheng Zhang

This paper investigates the application of reinforcement learning (RL) for adaptive quantum control. Traditional quantum control methods often rely on pre-designed pulse shapes optimized for specific scenarios, lacking adaptability to variations in system parameters or environmental noise. This work proposes a novel framework where an RL agent learns to dynamically adjust quantum control pulses in real-time, maximizing the fidelity of quantum operations. The agent receives feedback from the quantum system, typically measured through state variables, and utilizes this information to optimize the control pulse sequence. This adaptive approach addresses the limitations of static control schemes and promises enhanced robustness and performance in complex quantum systems. The core claim centers on utilizing reinforcement learning to design optimal quantum control pulses. The central mechanism involves a trained controller dynamically adjusting pulses based on system feedback. The presented methodology offers a new approach to quantum control, particularly relevant in noisy environments and systems with parameter variations. The research demonstrates the potential of RL for achieving superior quantum control outcomes compared to conventional techniques. ---

•Sep 4, 2026•0 citations

Meta-learning assisted robust control of universal quantum gates with uncertainties

Shihui Zhang, Zibo Miao, Yu Pan +2 more

Achieving high-fidelity quantum gates is crucial for reliable quantum computing. However, decoherence and control pulse imperfections pose significant challenges in realizing the theoretical fidelity of quantum gates in practical systems. To address these challenges, we propose the meta-reinforcement learning quantum control algorithm (metaQctrl), which leverages a two-layer learning framework to enhance robustness and fidelity. The inner reinforcement learning network focuses on decision making for specific optimization problems, while the outer meta-learning network adapts to varying environments and provides feedback to the inner network. Our comparative analysis regarding the realization of universal quantum gates demonstrates that metaQctrl achieves higher fidelity with fewer control pulses than conventional methods in the presence of uncertainties. These results can contribute to the exploration of the quantum speed limit and facilitate the implementation of quantum circuits with system imperfections involved.

arXiv:2406.07225•npj Quantum Information•May 19, 2025•5 citations
The extreme sensitivity of contemporary hardware to noise, manufacturing variability, and imperfect quantum logic gates are key factors that limit hardware to reliably perform quantum computation at scale. The use of reinforcement learning assisted control techniques for system dynamics optimization in quantum gate control can significantly improve hardware performance and downstream computing power. However, con-trol field noise and Hamiltonian parameter uncertainty can lead to significant deviations from the target quantum gate. As a result, quantum gates lack the generalization ability to the quantum environment, and existing reinforcement learning assisted for quantum state preparation is not suitable for quantum gate control. In this paper, we design QcontrolSR, a state representation for reinforcement learning method, enabling quantum gate control with enhanced generalization ability. It requires no knowledge of a specific Hamiltonian model of the system, or its underlying unknown process. We introduce the state representation of quantum observations and show through demonstrations that QcontrolSR-optimized quantum gate control is insensitive to driving noise of the typical strength in practice. We show thatRx(π/2)R_{x}(\pi/2)Rx​(π/2)gate implemented using QcontrolSR can maintain fidelity greater than 0.95 and 0.99 on dynamical quantum stochastic system. The regions with fidelity greater than 0.99 and 0.999 are 2 and 3 times higher than the model-optimized Quantum gate control, respectively.
High-fidelity quantum gate design is important for various quantum technologies, such as quantum computation and quantum communication. Numerous control policies for quantum gate design have been proposed given a dynamical model of the quantum system of interest. However, a quantum system is often highly sensitive to noise, and obtaining its accurate modeling can be difficult for many practical applications. Thus, the control policy based on a quantum system model may be unpractical for quantum gate design. Also, quantum measurements collapse quantum states, which makes it challenging to obtain information through measurements during the control process. In this article, we propose a novel training framework using deep reinforcement learning for model-free quantum control. The proposed framework relies only on the measurement at the end of the control process and offers the ability to find the optimal control policy without access to quantum systems during the learning process. The effectiveness of the proposed technique is numerically demonstrated for model-free quantum gate design and quantum gate calibration using off-policy reinforcement learning algorithms.
Higher-dimensional quantum systems, such as qudits, offer architectural and algorithmic advantages over qubits, but their increased spectral crowding and limited controllability render high-fidelity quantum gates particularly challenging. We propose a hybrid optimization framework that integrates optimal control theory methods with contextual deep reinforcement learning to achieve robust controlled-phase gates on two qutrits. Optimal control is first used to design high-fidelity control pulses for a nominal system model. Reinforcement learning is then employed as a calibration stage that learns small residual corrections to these pulses in the presence of static model mismatch, thereby preserving good gate performance under realistic parameter uncertainties. By learning structured, low-dimensional residual corrections conditioned on device-specific parameter variations, reinforcement learning enhances the transfer robustness of nominally optimal but parameter-sensitive control solutions across ensembles of devices. Crucially, the reinforcement learning step in our framework does not compete with the optimal control step but provides the adaptability required for realistic hardware, systematically reducing the sensitivity to parameter fluctuations. Our results establish reinforcement learning as a practical and scalable ingredient for robust calibration of quantum gates in high-dimensional systems.
Abstract Emerging reinforcement learning techniques using deep neural networks have shown great promise in control optimization. They harness non-local regularities of noisy control trajectories and facilitate transfer learning between tasks. To leverage these powerful capabilities for quantum control optimization, we propose a new control framework to simultaneously optimize the speed and fidelity of quantum computation against both leakage and stochastic control errors. For a broad family of two-qubit unitary gates that are important for quantum simulation of many-electron systems, we improve the control robustness by adding control noise into training environments for reinforcement learning agents trained with trusted-region-policy-optimization. The agent control solutions demonstrate a two-order-of-magnitude reduction in average-gate-error over baseline stochastic-gradient-descent solutions and up to a one-order-of-magnitude reduction in gate time from optimal gate synthesis counterparts. These significant improvements in both fidelity and runtime are achieved by combining new physical understandings and state-of-the-art machine learning techniques. Our results open a venue for wider applications in quantum simulation, quantum chemistry and quantum supremacy tests using near-term quantum devices.
Quantum computers promise significant computational advantages over classical computers for specific tasks. However, one of the challenges in the noisy intermediate-scale quantum (NISQ) era is the efficient compilation of quantum states and gates, which is crucial for the implementation of quantum algorithms. Given the high dimensionality of quantum systems, this compilation task is non-trivial. Reinforcement learning (RL) is a machine learning subfield focused on developing algorithms for decision-making in dynamic environments. An agent learns by interacting with its environment, how to act to optimize its performance on a certain task. Recently, the integration of deep learning has given rise to deep reinforcement learning (DRL), where deep neural networks approximate functions like value or policy, allowing agents to tackle high-dimensional and complex tasks with greater efficiency and success. Recently, deep reinforcement learning (DRL) has been proposed as a method to automate quantum compiling. In this thesis, we explore the application of RL for quantum state and unitary matrix synthesis, focusing on the use of two reinforcement learning algorithms Deep Q-Networks (DQN) and Proximal Policy Optimization (PPO). Our results indicate that DRL shows promise in generalizing the task of quantum unitary synthesis and quantum state preparation. However, the current state-of-the-art is still limited to working in the no-volume law entangled part of the Hilbert space.
This paper investigates the application of reinforcement learning (RL) for adaptive quantum control. Traditional quantum control methods often rely on pre-designed pulse shapes optimized for specific scenarios, lacking adaptability to variations in system parameters or environmental noise. This work proposes a novel framework where an RL agent learns to dynamically adjust quantum control pulses in real-time, maximizing the fidelity of quantum operations. The agent receives feedback from the quantum system, typically measured through state variables, and utilizes this information to optimize the control pulse sequence. This adaptive approach addresses the limitations of static control schemes and promises enhanced robustness and performance in complex quantum systems. The core claim centers on utilizing reinforcement learning to design optimal quantum control pulses. The central mechanism involves a trained controller dynamically adjusting pulses based on system feedback. The presented methodology offers a new approach to quantum control, particularly relevant in noisy environments and systems with parameter variations. The research demonstrates the potential of RL for achieving superior quantum control outcomes compared to conventional techniques. ---
Achieving high-fidelity quantum gates is crucial for reliable quantum computing. However, decoherence and control pulse imperfections pose significant challenges in realizing the theoretical fidelity of quantum gates in practical systems. To address these challenges, we propose the meta-reinforcement learning quantum control algorithm (metaQctrl), which leverages a two-layer learning framework to enhance robustness and fidelity. The inner reinforcement learning network focuses on decision making for specific optimization problems, while the outer meta-learning network adapts to varying environments and provides feedback to the inner network. Our comparative analysis regarding the realization of universal quantum gates demonstrates that metaQctrl achieves higher fidelity with fewer control pulses than conventional methods in the presence of uncertainties. These results can contribute to the exploration of the quantum speed limit and facilitate the implementation of quantum circuits with system imperfections involved.