Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2012.10711 · Quantum

Quantum reinforcement learning in continuous action space

Shaojun Wu, Shan Jin, Dingding Wen, Donghong Han, Xiaoting Wang

Published Mar 12, 2025• 45 citations
arXivHTMLPDF

Abstract

Quantum reinforcement learning (QRL) is a promising paradigm for near-term quantum devices. While existing QRL methods have shown success in discrete action spaces, extending these techniques to continuous domains is challenging due to the curse of dimensionality introduced by discretization. To overcome this limitation, we introduce a quantum Deep Deterministic Policy Gradient (DDPG) algorithm that efficiently addresses both classical and quantum sequential decision problems in continuous action spaces. Moreover, our approach facilitates single-shot quantum state generation: a one-time optimization produces a model that outputs the control sequence required to drive a fixed initial state to any desired target state. In contrast, conventional quantum control methods demand separate optimization for each target state. We demonstrate the effectiveness of our method through simulations and discuss its potential applications in quantum control.

DOI
10.22331/q-2025-03-12-1660

Related papers

Application of Deep Reinforcement Learning in Quantum Control

Yiming Cheng, Yixi Yin, Jiaying Lin +1 more

Abstract Machine learning technology based on artificial neural network has been successfully applied to solve many scientific problems. One of the most interesting areas of machine learning is reinforcement learning, which has natural applicability to optimization problems in physics. In the quantum control task, it is necessary to find a set of optimal control functions to transfer a quantum system from the initial state to the target state with the highest fidelity possible, which is essentially an optimization task. In this paper, we use Deep Deterministic Policy Gradient algorithm (DDPG) to study the quantum control tasks. We use the algorithm to control the transfer of several quantum systems from one state to another. The results show that DDPG algorithm can find a control strategy to make the fidelity of the final state and the target state of the quantum system be maximum value 1. The results show the potential of DDPG in quantum control.

Journal of Physics Conference Series•May 1, 2023•1 citation

Continuous Quantum Reinforcement Learning for Robot Navigation

Theodora-Augustina Drăgan, Alexander Künzner, Robert Wille +1 more

One of the multiple facets of quantum reinforcement learning (QRL) is enhancing reinforcement learning (RL) algorithms with quantum submodules, namely with variational quantum circuits (VQC) as function approx-imators. QRL solutions are empirically proven to require fewer training iterations or adjustable parameters than their classical counterparts, but are usually restricted to applications that have a discrete action space and thus limited industrial relevance. We propose a hybrid quantum-classical (HQC) deep deterministic policy gradient (DDPG) approach for a robot to navigate through a maze using continuous states, continuous actions and using local observations from the robot’s LiDAR sensors. We show that this HQC method can lead to models of comparable test results to the neural network (NN)-based DDPG algorithm, that need around 200 times fewer weights. We also study the scalability of our solution with respect to the number of VQC layers and qubits, and find that in general results improve as the layer and qubit counts increase. The best rewards among all similarly sized HQC and classical DDPG methods correspond to a VQC of 8 qubits and 5 layers with no other NN. This work is another step towards continuous QRL, where literature has been sparse.

International Conference on Agents and Artificial Intelligence (ICAART)•Jan 1, 2025•1 citation

Reinforcement Learning for Gate Synthesis in Noisy Quantum Systems

Amara Katabarwa, Collin Farquhar, Hyeongrak Choi +4 more

We present a gate synthesis application of the Deep Deterministic Policy Gradient (DDPG) algorithm for continuous control in a noisy quantum environment. Gate synthesis plays a crucial role in quantum computing, and optimizing control strategies for gate synthesis is of significant interest. The proposed approach leverages a DNN-based actor-critic architecture to optimize gate synthesis performance. The effectiveness of the algorithm is evaluated through simulations in a noiseless and noisy quantum environment, demonstrating its ability to achieve high-fidelity gate synthesis,$\mathcal{O}(10^{-3})$, despite the presence of Markovian damping terms. The results highlight the potential of DDPG and reinforcement learning techniques for addressing continuous control challenges in quantum computing applications.

IEEE International Conference on Quantum Computing and Engineering (QCE)•Sep 17, 2023•0 citations

Quantum State Generation Via Deep Reinforcement Learning

Shaojun Wu, Shan Jin, Xiaoting Wang

The quantum state generation problem is a major research goal of quantum control and quantum variational algorithms, which use iterative optimization methods to evolve the initial state to the target state. The Twin Delayed Deep Deterministic Policy (TD3) algorithm in reinforcement learning achieves high learning efficiency and better stability in continuous control tasks. Here, using the TD3, we propose a new quantum state preparation method that does not require case-by-case optimization, to find a suitable evolution path to obtain the desired state. Specifically, we input the initial state into the trained actor-network, which can output the parameters of the unitary gates step by step, thus gradually evolving the initial state to the fixed quantum state. According to the reversibility of the unitary transformation, we can obtain a sequence of unitary gates to evolve the fixed state to the desired state. To verify the effectiveness of the algorithm, we perform simulations for one-qubit, two-qubit, and four-qubit cases, and the results show that the trained actor-network can provide appropriate unitary transformations to obtain the fixed state.

Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics•Oct 1, 2023•1 citation

Guided Reward Design in Continuous Reinforcement Learning for Quantum Control

Shumin Zhou, Hailan Ma, Sen Kuang +1 more

Reinforcement learning has been intensively applied to tackle complex quantum control problems owing to its adaptability in dynamic environments. However, commonly used reinforcement learning algorithms are restricted to selecting actions from a discrete action space, which may result in inaccurate control. In order to surpass this constraint, we propose an improved continuous reinforcement learning algorithm that can generate control policies in a continuous action space, enabling precise control of quantum systems. Moreover, a guided reward function design method is proposed to guide the learning process toward higher fidelity. Numerical results demonstrate that the proposed continuous reinforcement learning algorithm with a guided reward function can capably prepare states on one-qubit and two-qubit systems.

Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics•Oct 1, 2023•0 citations

UAV Coverage Path Planning with Quantum-based Deep Deterministic Policy Gradient

Silvirianti Silvirianti, Bhaskara Narottama, Soo Young Shin

This study proposes quantum-based deep deterministic policy gradient (Q-DDPG) and quantum-based recurrent DDPG (Q-RDDPG) schemes for time-series optimization in UAV communications. Herein, Q-DDPG based actor-critic reinforcement learning is utilized to optimize action selections in a large state and continuous action space. In this scheme, quantum models are exploited to reduce computational complexity and training loss. As a particular case, Q-DDPG and Q-RDDPG are employed for trajectory optimization and dynamic resource allocation in UAV communications. Quantum circuits of the Q-DDPG schemes are described to showcase their implementation in noisy intermediate-scale quantum (NISQ) computers. The results demonstrate that Q-DDPG and Q-RDDPG schemes achieved higher rewards with lower training losses compared to classical DDPG.

•Feb 6, 2023•1 citation
Quantum reinforcement learning (QRL) is a promising paradigm for near-term quantum devices. While existing QRL methods have shown success in discrete action spaces, extending these techniques to continuous domains is challenging due to the curse of dimensionality introduced by discretization. To overcome this limitation, we introduce a quantum Deep Deterministic Policy Gradient (DDPG) algorithm that efficiently addresses both classical and quantum sequential decision problems in continuous action spaces. Moreover, our approach facilitates single-shot quantum state generation: a one-time optimization produces a model that outputs the control sequence required to drive a fixed initial state to any desired target state. In contrast, conventional quantum control methods demand separate optimization for each target state. We demonstrate the effectiveness of our method through simulations and discuss its potential applications in quantum control.
Abstract Machine learning technology based on artificial neural network has been successfully applied to solve many scientific problems. One of the most interesting areas of machine learning is reinforcement learning, which has natural applicability to optimization problems in physics. In the quantum control task, it is necessary to find a set of optimal control functions to transfer a quantum system from the initial state to the target state with the highest fidelity possible, which is essentially an optimization task. In this paper, we use Deep Deterministic Policy Gradient algorithm (DDPG) to study the quantum control tasks. We use the algorithm to control the transfer of several quantum systems from one state to another. The results show that DDPG algorithm can find a control strategy to make the fidelity of the final state and the target state of the quantum system be maximum value 1. The results show the potential of DDPG in quantum control.
One of the multiple facets of quantum reinforcement learning (QRL) is enhancing reinforcement learning (RL) algorithms with quantum submodules, namely with variational quantum circuits (VQC) as function approx-imators. QRL solutions are empirically proven to require fewer training iterations or adjustable parameters than their classical counterparts, but are usually restricted to applications that have a discrete action space and thus limited industrial relevance. We propose a hybrid quantum-classical (HQC) deep deterministic policy gradient (DDPG) approach for a robot to navigate through a maze using continuous states, continuous actions and using local observations from the robot’s LiDAR sensors. We show that this HQC method can lead to models of comparable test results to the neural network (NN)-based DDPG algorithm, that need around 200 times fewer weights. We also study the scalability of our solution with respect to the number of VQC layers and qubits, and find that in general results improve as the layer and qubit counts increase. The best rewards among all similarly sized HQC and classical DDPG methods correspond to a VQC of 8 qubits and 5 layers with no other NN. This work is another step towards continuous QRL, where literature has been sparse.
We present a gate synthesis application of the Deep Deterministic Policy Gradient (DDPG) algorithm for continuous control in a noisy quantum environment. Gate synthesis plays a crucial role in quantum computing, and optimizing control strategies for gate synthesis is of significant interest. The proposed approach leverages a DNN-based actor-critic architecture to optimize gate synthesis performance. The effectiveness of the algorithm is evaluated through simulations in a noiseless and noisy quantum environment, demonstrating its ability to achieve high-fidelity gate synthesis,O(10−3)\mathcal{O}(10^{-3})O(10−3), despite the presence of Markovian damping terms. The results highlight the potential of DDPG and reinforcement learning techniques for addressing continuous control challenges in quantum computing applications.
The quantum state generation problem is a major research goal of quantum control and quantum variational algorithms, which use iterative optimization methods to evolve the initial state to the target state. The Twin Delayed Deep Deterministic Policy (TD3) algorithm in reinforcement learning achieves high learning efficiency and better stability in continuous control tasks. Here, using the TD3, we propose a new quantum state preparation method that does not require case-by-case optimization, to find a suitable evolution path to obtain the desired state. Specifically, we input the initial state into the trained actor-network, which can output the parameters of the unitary gates step by step, thus gradually evolving the initial state to the fixed quantum state. According to the reversibility of the unitary transformation, we can obtain a sequence of unitary gates to evolve the fixed state to the desired state. To verify the effectiveness of the algorithm, we perform simulations for one-qubit, two-qubit, and four-qubit cases, and the results show that the trained actor-network can provide appropriate unitary transformations to obtain the fixed state.
Reinforcement learning has been intensively applied to tackle complex quantum control problems owing to its adaptability in dynamic environments. However, commonly used reinforcement learning algorithms are restricted to selecting actions from a discrete action space, which may result in inaccurate control. In order to surpass this constraint, we propose an improved continuous reinforcement learning algorithm that can generate control policies in a continuous action space, enabling precise control of quantum systems. Moreover, a guided reward function design method is proposed to guide the learning process toward higher fidelity. Numerical results demonstrate that the proposed continuous reinforcement learning algorithm with a guided reward function can capably prepare states on one-qubit and two-qubit systems.
This study proposes quantum-based deep deterministic policy gradient (Q-DDPG) and quantum-based recurrent DDPG (Q-RDDPG) schemes for time-series optimization in UAV communications. Herein, Q-DDPG based actor-critic reinforcement learning is utilized to optimize action selections in a large state and continuous action space. In this scheme, quantum models are exploited to reduce computational complexity and training loss. As a particular case, Q-DDPG and Q-RDDPG are employed for trajectory optimization and dynamic resource allocation in UAV communications. Quantum circuits of the Q-DDPG schemes are described to showcase their implementation in noisy intermediate-scale quantum (NISQ) computers. The results demonstrate that Q-DDPG and Q-RDDPG schemes achieved higher rewards with lower training losses compared to classical DDPG.