Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics

A Modified Deep Q-Learning Algorithm for Optimal and Robust Quantum Gate Design of a Single Qubit System

Omar Shindi, Qi Yu, Parth Girdhar, Daoyi Dong

Published Oct 9, 2022• 0 citations
Publisher

Abstract

Precise and resilient quantum gate design is important for the building of quantum devices. In this paper, we consider the optimal and robust quantum gate design problem for three classes of two-level quantum systems. The aim is to construct quantum gates in a given fixed time with limited control resources. A modified dueling deep Q-learning (MDuDQL) is employed for the optimal and robust gate design problem. To improve the performance of the classical DuDQL method, we propose a unique semi-Markov DuDQL algorithm based on a modified action selection procedure, modified replay memory, and soft update procedure. The proposed algorithm outperforms ordinary DuDQL in terms of discovering global optimal or near-global optimal control protocols and faster convergence to a better policy. Moreover, the modified DuDQL agent shows improved performance in finding robust control protocols which achieve high-fidelity quantum gate design for varying uncertainties in a certain range. The effectiveness of the proposed algorithm for the optimal and robust gate design problems has been illustrated by numerical results.

DOI
10.1109/smc53654.2022.9945165

Related papers

A Modified Deep Q-Learning Algorithm for Control of Two-qubit Systems

Omar Shindi, Qi Yu, Daoyi Dong

Quantum control refers to the manipulation of dynamical quantum systems to force them to complete given tasks such as preparing a desired state and tracking a designed trajectory. We consider the state preparation problem of a two-qubit closed quantum system from an initial to a desired state. The aim is to achieve a high fidelity in a given fixed time with limited control resources. The Deep Q-learning (DQL) for solving the quantum state preparation problem is explored in this paper. We propose a novel semi-Markov DQL algorithm based on a modified action selection procedure and improved replay memory to enhance performance of standard DQL algorithm. The proposed algorithm shows high performance for discovering high-fidelity control protocols and for converging to a good policy, compared with standard DQL. The proposed modifications enhance the exploration-exploitation ability for DQL agent and the robustness for solving quantum control problem with high-fidelity at different numbers of control steps. Numerical results on a two-qubit closed system show effectiveness of the proposed algorithm.

Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics•Oct 17, 2021•3 citations

Fast Quantum Gate Design with Deep Reinforcement Learning Using Real-Time Feedback on Readout Signals

Emily Wright, Rogério de Sousa

The design of high-fidelity quantum gates is difficult because it requires the optimization of two competing effects, namely maximizing gate speed and minimizing leakage out of the qubit subspace. We propose a deep reinforcement learning algorithm that uses two agents to address the speed and leakage challenges simultaneously. The first agent constructs the qubit in-phase control pulse using a policy learned from rewards that compensate short gate times. The rewards are obtained at intermediate time steps throughout the construction of a full-length pulse, allowing the agent to explore the landscape of shorter pulses. The second agent determines an out-of-phase pulse to target leakage. Both agents are trained on real-time data from noisy hardware, thus providing model-free gate design that adapts to unpredictable hardware noise. To reduce the effect of measurement classification errors, the agents are trained directly on the readout signal from probing the qubit. We present proof-of-concept experiments by designing X and square root of X gates of various durations on IBM hardware. After just 200 training iterations, our algorithm is able to construct novel control pulses up to two times faster than the default IBM gates, while matching their performance in terms of state fidelity and leakage rate. As the length of our custom control pulses increases, they begin to outperform the default gates. Improvements to the speed and fidelity of gate operations open the way for higher circuit depth in quantum simulation, quantum chemistry and other algorithms on near-term and future quantum devices.

arXiv:2305.01169•IEEE International Conference on Quantum Computing and Engineering (QCE)•Sep 17, 2023•4 citations

Reinforcement Learning for Robust Calibration of Multi-Qudit Quantum Gates

Amine Jaouadi, Sahel Ashhab

Higher-dimensional quantum systems, such as qudits, offer architectural and algorithmic advantages over qubits, but their increased spectral crowding and limited controllability render high-fidelity quantum gates particularly challenging. We propose a hybrid optimization framework that integrates optimal control theory methods with contextual deep reinforcement learning to achieve robust controlled-phase gates on two qutrits. Optimal control is first used to design high-fidelity control pulses for a nominal system model. Reinforcement learning is then employed as a calibration stage that learns small residual corrections to these pulses in the presence of static model mismatch, thereby preserving good gate performance under realistic parameter uncertainties. By learning structured, low-dimensional residual corrections conditioned on device-specific parameter variations, reinforcement learning enhances the transfer robustness of nominally optimal but parameter-sensitive control solutions across ensembles of devices. Crucially, the reinforcement learning step in our framework does not compete with the optimal control step but provides the adaptability required for realistic hardware, systematically reducing the sensitivity to parameter fluctuations. Our results establish reinforcement learning as a practical and scalable ingredient for robust calibration of quantum gates in high-dimensional systems.

quant-pharXiv:2604.19990•Physical Review A•Apr 23, 2026•1 citation

Deep Reinforcement Learning for Control Design of Quantum Gates

Shouliang Hu, Chunlin Chen, Daoyi Dong

This paper investigates quantum gate control problems using the deep reinforcement learning algorithm, i.e., a model-free machine learning method. We implement the twin delayed deep deterministic policy gradient (TD3) algorithm to search for piece-wise constant control pulses for quantum gates through the trail interaction with the quantum system. Simulation results on four typical gates, including three one-qubit gates and a two-qubit CNOT gate, demonstrate that deep reinforcement learning exhibits improved performance for quantum gate control tasks. By setting punishment for steps in the reward function, DRL can automatically find a shorter control sequence than the traditional gradient-based algorithm (e.g., GRAPE algorithm) and the evolutionary algorithm (e.g., DE algorithm) while maintaining high control precision.

Asian Control Conference (ASCC)•May 4, 2022•4 citations

Machine Learning for Designing Fast Quantum Gates

Ehsan Zahedinejad

Fault-tolerant quantum computing requires encoding the quantum information into logical qubits and performing the quantum information processing in a code-space. Quantum error correction codes, then, can be employed to diagnose and remove the possible errors in the quantum information, thereby avoiding the loss of information. Although a series of single- and two-qubit gates can be employed to construct a quantum error correcting circuit, however this decomposition approach is not practically desirable because it leads to circuits with long operation times. An alternative approach to designing a fast quantum circuit is to design quantum gates that act on a multi-qubit gate. Here I devise quantum control schemes to design high-fidelity single-shot multi-qubit (up to three) quantum gates. Quantum control task is to steer quantum dynamics towards closely realizing specific quantum operation by varying the external control parameters (external field) such that the resultant evolution closely approximates the desired evolution. A set of instructions that determines the control parameters, and hence the effectiveness of the control scheme, is called a policy. Machine learning algorithms can be employed to find successful policies for designing quantum gates. In particular, we employ supervised machine learning techniques to generate these successful policies. Finding successful policies is a feasibility problem for which optimization algorithms can be employed. Greedy algorithms are at the heart of machine learning techniques. They converge faster onto a successful policy and require less-computational resource than non-greedy algorithms. However, there is no guarantee that greedy algorithms succeed to a feasible solution when there exist constraints on i) gate operation time ii) computational resources, and iii) experimental resources. Our results show the failure of standard greedy machine learning algorithms and the superiority of non-greedy machine learning algorithms over greedy ones for designing quantum logic gates, when there exist constraints on the quantum system. We have also observed the failure of existing greedy and non-greedy techniques for designing high-fidelity three-qubit gates. Hence, we devised our machine learning technique called Subspace-Selective Self-adaptive Differential Evolution (SuSSADE). Each three-qubit gate designed by SuSSADE operates as fast as an entangling two-qubit gate under the same experimental constraints.

•Jan 1, 2016•0 citations

Deep reinforcement learning for quantum gate control

Zheng An, D. L. Zhou

How to implement multi-qubit gates efficiently with high precision is essential for realizing universal fault-tolerant computing. For a physical system with some external controllable parameters, it is a great challenge to control the time dependence of these parameters to achieve a target multi-qubit gate efficiently and precisely. Here we construct a dueling double deep Q-learning neural network (DDDQN) to find out the optimized time dependence of controllable parameters to implement two typical quantum gates: a single-qubit Hadamard gate and a two-qubit CNOT gate. Compared with traditional optimal control methods, this deep reinforcement learning method can realize efficient and precise gate control without requiring any gradient information during the learning process. This work attempts to pave the way to investigate more quantum control problems with deep reinforcement learning techniques.

arXiv:1902.08418•Europhysics Letters (EPL)•Jul 22, 2019•123 citations
Precise and resilient quantum gate design is important for the building of quantum devices. In this paper, we consider the optimal and robust quantum gate design problem for three classes of two-level quantum systems. The aim is to construct quantum gates in a given fixed time with limited control resources. A modified dueling deep Q-learning (MDuDQL) is employed for the optimal and robust gate design problem. To improve the performance of the classical DuDQL method, we propose a unique semi-Markov DuDQL algorithm based on a modified action selection procedure, modified replay memory, and soft update procedure. The proposed algorithm outperforms ordinary DuDQL in terms of discovering global optimal or near-global optimal control protocols and faster convergence to a better policy. Moreover, the modified DuDQL agent shows improved performance in finding robust control protocols which achieve high-fidelity quantum gate design for varying uncertainties in a certain range. The effectiveness of the proposed algorithm for the optimal and robust gate design problems has been illustrated by numerical results.
Quantum control refers to the manipulation of dynamical quantum systems to force them to complete given tasks such as preparing a desired state and tracking a designed trajectory. We consider the state preparation problem of a two-qubit closed quantum system from an initial to a desired state. The aim is to achieve a high fidelity in a given fixed time with limited control resources. The Deep Q-learning (DQL) for solving the quantum state preparation problem is explored in this paper. We propose a novel semi-Markov DQL algorithm based on a modified action selection procedure and improved replay memory to enhance performance of standard DQL algorithm. The proposed algorithm shows high performance for discovering high-fidelity control protocols and for converging to a good policy, compared with standard DQL. The proposed modifications enhance the exploration-exploitation ability for DQL agent and the robustness for solving quantum control problem with high-fidelity at different numbers of control steps. Numerical results on a two-qubit closed system show effectiveness of the proposed algorithm.
The design of high-fidelity quantum gates is difficult because it requires the optimization of two competing effects, namely maximizing gate speed and minimizing leakage out of the qubit subspace. We propose a deep reinforcement learning algorithm that uses two agents to address the speed and leakage challenges simultaneously. The first agent constructs the qubit in-phase control pulse using a policy learned from rewards that compensate short gate times. The rewards are obtained at intermediate time steps throughout the construction of a full-length pulse, allowing the agent to explore the landscape of shorter pulses. The second agent determines an out-of-phase pulse to target leakage. Both agents are trained on real-time data from noisy hardware, thus providing model-free gate design that adapts to unpredictable hardware noise. To reduce the effect of measurement classification errors, the agents are trained directly on the readout signal from probing the qubit. We present proof-of-concept experiments by designing X and square root of X gates of various durations on IBM hardware. After just 200 training iterations, our algorithm is able to construct novel control pulses up to two times faster than the default IBM gates, while matching their performance in terms of state fidelity and leakage rate. As the length of our custom control pulses increases, they begin to outperform the default gates. Improvements to the speed and fidelity of gate operations open the way for higher circuit depth in quantum simulation, quantum chemistry and other algorithms on near-term and future quantum devices.
Higher-dimensional quantum systems, such as qudits, offer architectural and algorithmic advantages over qubits, but their increased spectral crowding and limited controllability render high-fidelity quantum gates particularly challenging. We propose a hybrid optimization framework that integrates optimal control theory methods with contextual deep reinforcement learning to achieve robust controlled-phase gates on two qutrits. Optimal control is first used to design high-fidelity control pulses for a nominal system model. Reinforcement learning is then employed as a calibration stage that learns small residual corrections to these pulses in the presence of static model mismatch, thereby preserving good gate performance under realistic parameter uncertainties. By learning structured, low-dimensional residual corrections conditioned on device-specific parameter variations, reinforcement learning enhances the transfer robustness of nominally optimal but parameter-sensitive control solutions across ensembles of devices. Crucially, the reinforcement learning step in our framework does not compete with the optimal control step but provides the adaptability required for realistic hardware, systematically reducing the sensitivity to parameter fluctuations. Our results establish reinforcement learning as a practical and scalable ingredient for robust calibration of quantum gates in high-dimensional systems.
This paper investigates quantum gate control problems using the deep reinforcement learning algorithm, i.e., a model-free machine learning method. We implement the twin delayed deep deterministic policy gradient (TD3) algorithm to search for piece-wise constant control pulses for quantum gates through the trail interaction with the quantum system. Simulation results on four typical gates, including three one-qubit gates and a two-qubit CNOT gate, demonstrate that deep reinforcement learning exhibits improved performance for quantum gate control tasks. By setting punishment for steps in the reward function, DRL can automatically find a shorter control sequence than the traditional gradient-based algorithm (e.g., GRAPE algorithm) and the evolutionary algorithm (e.g., DE algorithm) while maintaining high control precision.
Fault-tolerant quantum computing requires encoding the quantum information into logical qubits and performing the quantum information processing in a code-space. Quantum error correction codes, then, can be employed to diagnose and remove the possible errors in the quantum information, thereby avoiding the loss of information. Although a series of single- and two-qubit gates can be employed to construct a quantum error correcting circuit, however this decomposition approach is not practically desirable because it leads to circuits with long operation times. An alternative approach to designing a fast quantum circuit is to design quantum gates that act on a multi-qubit gate. Here I devise quantum control schemes to design high-fidelity single-shot multi-qubit (up to three) quantum gates. Quantum control task is to steer quantum dynamics towards closely realizing specific quantum operation by varying the external control parameters (external field) such that the resultant evolution closely approximates the desired evolution. A set of instructions that determines the control parameters, and hence the effectiveness of the control scheme, is called a policy. Machine learning algorithms can be employed to find successful policies for designing quantum gates. In particular, we employ supervised machine learning techniques to generate these successful policies. Finding successful policies is a feasibility problem for which optimization algorithms can be employed. Greedy algorithms are at the heart of machine learning techniques. They converge faster onto a successful policy and require less-computational resource than non-greedy algorithms. However, there is no guarantee that greedy algorithms succeed to a feasible solution when there exist constraints on i) gate operation time ii) computational resources, and iii) experimental resources. Our results show the failure of standard greedy machine learning algorithms and the superiority of non-greedy machine learning algorithms over greedy ones for designing quantum logic gates, when there exist constraints on the quantum system. We have also observed the failure of existing greedy and non-greedy techniques for designing high-fidelity three-qubit gates. Hence, we devised our machine learning technique called Subspace-Selective Self-adaptive Differential Evolution (SuSSADE). Each three-qubit gate designed by SuSSADE operates as fast as an entangling two-qubit gate under the same experimental constraints.
How to implement multi-qubit gates efficiently with high precision is essential for realizing universal fault-tolerant computing. For a physical system with some external controllable parameters, it is a great challenge to control the time dependence of these parameters to achieve a target multi-qubit gate efficiently and precisely. Here we construct a dueling double deep Q-learning neural network (DDDQN) to find out the optimized time dependence of controllable parameters to implement two typical quantum gates: a single-qubit Hadamard gate and a two-qubit CNOT gate. Compared with traditional optimal control methods, this deep reinforcement learning method can realize efficient and precise gate control without requiring any gradient information during the learning process. This work attempts to pave the way to investigate more quantum control problems with deep reinforcement learning techniques.