Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2302.14312 · IEEE Transactions on Cybernetics

Auxiliary Task-Based Deep Reinforcement Learning for Quantum Control

Shumin Zhou, Hailan Ma, Sen Kuang, Daoyi Dong

Published Jan 7, 2025• 6 citations
arXivHTMLPDF

Abstract

Due to its property of not requiring prior knowledge of the environment, reinforcement learning (RL) has significant potential for solving quantum control problems. In this work, we investigate the effectiveness of continuous control policies based on deep deterministic policy gradient. To achieve good control of quantum systems with high fidelity, we propose an auxiliary task-based deep RL (AT-DRL) for quantum control. In particular, we design an auxiliary task to predict the fidelity value, sharing partial parameters with the main network (from the main RL task). The auxiliary task learns synchronously with the main task, allowing one to extract intrinsic features of the environment, thus aiding the agent to achieve the desired state with high fidelity. To further enhance the control performance, we also design a guided reward function based on the fidelity of quantum states that enables gradual fidelity improvement. Numerical simulations demonstrate that the proposed AT-DRL can provide a good solution to the exploration of quantum dynamics. It not only achieves high task fidelities but also demonstrates fast learning rates. Moreover, AT-DRL has great potential in designing control pulses that achieve effective quantum state preparation.

DOI
10.1109/tcyb.2024.3521300

Related papers

Event-Based Deep Reinforcement Learning for Quantum Control

Haixu Yu, Xudong Zhao

Deep reinforcement learning (DRL) has been widely used in many quantum control tasks, in which rewards are used to guide the agent in acquiring an optimal control strategy. However, designing suitable rewards is usually challenging, which seriously affects the DRL's application performance, especially for complex quantum control tasks. In this article, we propose a simple and general reward design method by constructing some events, where events are defined by fidelity intervals. In the proposed event-based DRL (EDRL), events are used to describe the evolution of a quantum system. Rewards are designed according to several defined events rather than a large number of specific evolved quantum states, thereby resulting in a reduction in dimensions to be considered. With designed event-attributed rewards, the agent can capture the main evolution characteristics of a quantum system, and then obtain an effective control strategy. The effectiveness of the proposed EDRL method is verified by three quantum systems: i.e., 1) the one-qubit closed quantum system, 2) the two-qubit closed quantum system, and 3) the two-level open quantum system. Numerical comparisons demonstrate that the proposed EDRL outperforms the traditional DRL algorithms (deep Q-network and proximal policy optimization) with some representative reward methods for solving quantum control problems.

IEEE Transactions on Emerging Topics in Computational Intelligence•Sep 26, 2023•9 citations

Curriculum-Based Deep Reinforcement Learning for Quantum Control

Hailan Ma, Daoyi Dong, Steven X. Ding +1 more

Deep reinforcement learning (DRL) has been recognized as an efficient technique to design optimal strategies for different complex systems without prior knowledge of the control landscape. To achieve a fast and precise control for quantum systems, we propose a novel DRL approach by constructing a curriculum consisting of a set of intermediate tasks defined by fidelity thresholds, where the tasks among a curriculum can be statically determined before the learning process or dynamically generated during the learning process. By transferring knowledge between two successive tasks and sequencing tasks according to their difficulties, the proposed curriculum-based DRL (CDRL) method enables the agent to focus on easy tasks in the early stage, then move onto difficult tasks, and eventually approaches the final task. Numerical comparison with the traditional methods [gradient method (GD), genetic algorithm (GA), and several other DRL methods] demonstrates that CDRL exhibits improved control performance for quantum systems and also provides an efficient way to identify optimal strategies with few control pulses.

arXiv:2012.15427•IEEE Transactions on Neural Networks and Learning Systems•Mar 9, 2022•54 citations

Application of Deep Reinforcement Learning in Quantum Control

Yiming Cheng, Yixi Yin, Jiaying Lin +1 more

Abstract Machine learning technology based on artificial neural network has been successfully applied to solve many scientific problems. One of the most interesting areas of machine learning is reinforcement learning, which has natural applicability to optimization problems in physics. In the quantum control task, it is necessary to find a set of optimal control functions to transfer a quantum system from the initial state to the target state with the highest fidelity possible, which is essentially an optimization task. In this paper, we use Deep Deterministic Policy Gradient algorithm (DDPG) to study the quantum control tasks. We use the algorithm to control the transfer of several quantum systems from one state to another. The results show that DDPG algorithm can find a control strategy to make the fidelity of the final state and the target state of the quantum system be maximum value 1. The results show the potential of DDPG in quantum control.

Journal of Physics Conference Series•May 1, 2023•1 citation

Deep Reinforcement Learning With Reward Design for Quantum Control

Haixu Yu, Xudong Zhao

Deep reinforcement learning (DRL) has been recognized as a powerful tool in quantum physics, where DRL's reward design is nontrivial but crucial for quantum control tasks. To address the problem of over-reliance on human empirical knowledge to design DRL's rewards, we propose a DRL with a novel reward paradigm designed by the learning process information (DRL-LPI), where the learning process information (LPI) comprises the state information and the experiences. In DRL-LPI, the state information after being classified by a fidelity threshold, and the experiences are first stored simultaneously in the respective sequences, and this process is repeated until a similar-segment ends. Then, the stored state information is converted to the real value and used to design the reward value by applying a self-amplitude function. Next, the designed reward values are integrated with the stored experiences to compose transitions for DRL's training. Through comparisons to five representative reward schemes, the proposed DRL-LPI is evaluated on two typical quantum control tasks, i.e., the spin-(1/2) quantum state control and many-coupled qubits state control, and the experimental results show the superior learning efficiency and control performance of the proposed approach. More results show that DRL-LPI exhibits the ability to learn the control strategy with few control actions compared to stochastic gradient descent (SGD) and genetic algorithm (GA).

IEEE Transactions on Artificial Intelligence•Nov 28, 2022•27 citations

Guided Reward Design in Continuous Reinforcement Learning for Quantum Control

Shumin Zhou, Hailan Ma, Sen Kuang +1 more

Reinforcement learning has been intensively applied to tackle complex quantum control problems owing to its adaptability in dynamic environments. However, commonly used reinforcement learning algorithms are restricted to selecting actions from a discrete action space, which may result in inaccurate control. In order to surpass this constraint, we propose an improved continuous reinforcement learning algorithm that can generate control policies in a continuous action space, enabling precise control of quantum systems. Moreover, a guided reward function design method is proposed to guide the learning process toward higher fidelity. Numerical results demonstrate that the proposed continuous reinforcement learning algorithm with a guided reward function can capably prepare states on one-qubit and two-qubit systems.

Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics•Oct 1, 2023•0 citations

Deep Reinforcement Learning for Control Design of Quantum Gates

Shouliang Hu, Chunlin Chen, Daoyi Dong

This paper investigates quantum gate control problems using the deep reinforcement learning algorithm, i.e., a model-free machine learning method. We implement the twin delayed deep deterministic policy gradient (TD3) algorithm to search for piece-wise constant control pulses for quantum gates through the trail interaction with the quantum system. Simulation results on four typical gates, including three one-qubit gates and a two-qubit CNOT gate, demonstrate that deep reinforcement learning exhibits improved performance for quantum gate control tasks. By setting punishment for steps in the reward function, DRL can automatically find a shorter control sequence than the traditional gradient-based algorithm (e.g., GRAPE algorithm) and the evolutionary algorithm (e.g., DE algorithm) while maintaining high control precision.

Asian Control Conference (ASCC)•May 4, 2022•4 citations
Due to its property of not requiring prior knowledge of the environment, reinforcement learning (RL) has significant potential for solving quantum control problems. In this work, we investigate the effectiveness of continuous control policies based on deep deterministic policy gradient. To achieve good control of quantum systems with high fidelity, we propose an auxiliary task-based deep RL (AT-DRL) for quantum control. In particular, we design an auxiliary task to predict the fidelity value, sharing partial parameters with the main network (from the main RL task). The auxiliary task learns synchronously with the main task, allowing one to extract intrinsic features of the environment, thus aiding the agent to achieve the desired state with high fidelity. To further enhance the control performance, we also design a guided reward function based on the fidelity of quantum states that enables gradual fidelity improvement. Numerical simulations demonstrate that the proposed AT-DRL can provide a good solution to the exploration of quantum dynamics. It not only achieves high task fidelities but also demonstrates fast learning rates. Moreover, AT-DRL has great potential in designing control pulses that achieve effective quantum state preparation.
Deep reinforcement learning (DRL) has been widely used in many quantum control tasks, in which rewards are used to guide the agent in acquiring an optimal control strategy. However, designing suitable rewards is usually challenging, which seriously affects the DRL's application performance, especially for complex quantum control tasks. In this article, we propose a simple and general reward design method by constructing some events, where events are defined by fidelity intervals. In the proposed event-based DRL (EDRL), events are used to describe the evolution of a quantum system. Rewards are designed according to several defined events rather than a large number of specific evolved quantum states, thereby resulting in a reduction in dimensions to be considered. With designed event-attributed rewards, the agent can capture the main evolution characteristics of a quantum system, and then obtain an effective control strategy. The effectiveness of the proposed EDRL method is verified by three quantum systems: i.e., 1) the one-qubit closed quantum system, 2) the two-qubit closed quantum system, and 3) the two-level open quantum system. Numerical comparisons demonstrate that the proposed EDRL outperforms the traditional DRL algorithms (deep Q-network and proximal policy optimization) with some representative reward methods for solving quantum control problems.
Deep reinforcement learning (DRL) has been recognized as an efficient technique to design optimal strategies for different complex systems without prior knowledge of the control landscape. To achieve a fast and precise control for quantum systems, we propose a novel DRL approach by constructing a curriculum consisting of a set of intermediate tasks defined by fidelity thresholds, where the tasks among a curriculum can be statically determined before the learning process or dynamically generated during the learning process. By transferring knowledge between two successive tasks and sequencing tasks according to their difficulties, the proposed curriculum-based DRL (CDRL) method enables the agent to focus on easy tasks in the early stage, then move onto difficult tasks, and eventually approaches the final task. Numerical comparison with the traditional methods [gradient method (GD), genetic algorithm (GA), and several other DRL methods] demonstrates that CDRL exhibits improved control performance for quantum systems and also provides an efficient way to identify optimal strategies with few control pulses.
Abstract Machine learning technology based on artificial neural network has been successfully applied to solve many scientific problems. One of the most interesting areas of machine learning is reinforcement learning, which has natural applicability to optimization problems in physics. In the quantum control task, it is necessary to find a set of optimal control functions to transfer a quantum system from the initial state to the target state with the highest fidelity possible, which is essentially an optimization task. In this paper, we use Deep Deterministic Policy Gradient algorithm (DDPG) to study the quantum control tasks. We use the algorithm to control the transfer of several quantum systems from one state to another. The results show that DDPG algorithm can find a control strategy to make the fidelity of the final state and the target state of the quantum system be maximum value 1. The results show the potential of DDPG in quantum control.
Deep reinforcement learning (DRL) has been recognized as a powerful tool in quantum physics, where DRL's reward design is nontrivial but crucial for quantum control tasks. To address the problem of over-reliance on human empirical knowledge to design DRL's rewards, we propose a DRL with a novel reward paradigm designed by the learning process information (DRL-LPI), where the learning process information (LPI) comprises the state information and the experiences. In DRL-LPI, the state information after being classified by a fidelity threshold, and the experiences are first stored simultaneously in the respective sequences, and this process is repeated until a similar-segment ends. Then, the stored state information is converted to the real value and used to design the reward value by applying a self-amplitude function. Next, the designed reward values are integrated with the stored experiences to compose transitions for DRL's training. Through comparisons to five representative reward schemes, the proposed DRL-LPI is evaluated on two typical quantum control tasks, i.e., the spin-(1/2) quantum state control and many-coupled qubits state control, and the experimental results show the superior learning efficiency and control performance of the proposed approach. More results show that DRL-LPI exhibits the ability to learn the control strategy with few control actions compared to stochastic gradient descent (SGD) and genetic algorithm (GA).
Reinforcement learning has been intensively applied to tackle complex quantum control problems owing to its adaptability in dynamic environments. However, commonly used reinforcement learning algorithms are restricted to selecting actions from a discrete action space, which may result in inaccurate control. In order to surpass this constraint, we propose an improved continuous reinforcement learning algorithm that can generate control policies in a continuous action space, enabling precise control of quantum systems. Moreover, a guided reward function design method is proposed to guide the learning process toward higher fidelity. Numerical results demonstrate that the proposed continuous reinforcement learning algorithm with a guided reward function can capably prepare states on one-qubit and two-qubit systems.
This paper investigates quantum gate control problems using the deep reinforcement learning algorithm, i.e., a model-free machine learning method. We implement the twin delayed deep deterministic policy gradient (TD3) algorithm to search for piece-wise constant control pulses for quantum gates through the trail interaction with the quantum system. Simulation results on four typical gates, including three one-qubit gates and a two-qubit CNOT gate, demonstrate that deep reinforcement learning exhibits improved performance for quantum gate control tasks. By setting punishment for steps in the reward function, DRL can automatically find a shorter control sequence than the traditional gradient-based algorithm (e.g., GRAPE algorithm) and the evolutionary algorithm (e.g., DE algorithm) while maintaining high control precision.