Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

UAV Coverage Path Planning with Quantum-based Deep Deterministic Policy Gradient

Silvirianti Silvirianti, Bhaskara Narottama, Soo Young Shin

Published Feb 6, 2023• 1 citation
Publisher

Abstract

This study proposes quantum-based deep deterministic policy gradient (Q-DDPG) and quantum-based recurrent DDPG (Q-RDDPG) schemes for time-series optimization in UAV communications. Herein, Q-DDPG based actor-critic reinforcement learning is utilized to optimize action selections in a large state and continuous action space. In this scheme, quantum models are exploited to reduce computational complexity and training loss. As a particular case, Q-DDPG and Q-RDDPG are employed for trajectory optimization and dynamic resource allocation in UAV communications. Quantum circuits of the Q-DDPG schemes are described to showcase their implementation in noisy intermediate-scale quantum (NISQ) computers. The results demonstrate that Q-DDPG and Q-RDDPG schemes achieved higher rewards with lower training losses compared to classical DDPG.

DOI
10.36227/techrxiv.21973784

Related papers

Layerwise Quantum Deep Reinforcement Learning for Joint Optimization of UAV Trajectory and Resource Allocation

Silvirianti Silvirianti, Bhaskara Narottama, Soo Young Shin

This study proposes a layerwise quantum-based deep reinforcement learning (LQ-DRL) method for optimizing continuous large space and time series problems using deep-layer training. The actions in LQ-DRL are optimized using a layerwise quantum embedding that leverages the advantages of quantum computing to maximize reward and reduce training loss. Moreover, this study employs a local loss to minimize the occurrence of barren plateaus phenomena and further enhance performance. As a particular case, the proposed scheme is employed to jointly optimize: 1) unmanned aerial vehicle (UAV) trajectory planning; 2) user grouping; and 3) power allocation for higher energy efficiency of a UAV as the reward. The combination of these optimized factors is referred to as action space in the presented LQ-DRL. The LQ-DRL is employed to solve the optimization problem due to its nonconvexity, continuous and large action space, and time-series domain. In a practical view, LQ-DRL aims to solve the issue of energy consumption related to limited-battery energy of a UAV base station (BS) while maintaining Quality of Service (QoS) for users, by gaining maximum energy efficiency as the reward. One of real applications, as an example, LQ-DRL can be employed to maximize the energy efficiency of a UAV BS in UAV empowered disaster recovery networks scenario. The quantum circuits of layerwise quantum embedding are presented to show the practical implementation in noisy intermediate-scale quantum computers. Based on the results, LQ-DRL outperformed the classical DRL by achieving higher effective dimension, rewards, and lower learning losses. In addition, better performances were achieved using more layers.

IEEE Internet of Things Journal•Jun 14, 2023•26 citations

Continuous Quantum Reinforcement Learning for Robot Navigation

Theodora-Augustina Drăgan, Alexander Künzner, Robert Wille +1 more

One of the multiple facets of quantum reinforcement learning (QRL) is enhancing reinforcement learning (RL) algorithms with quantum submodules, namely with variational quantum circuits (VQC) as function approx-imators. QRL solutions are empirically proven to require fewer training iterations or adjustable parameters than their classical counterparts, but are usually restricted to applications that have a discrete action space and thus limited industrial relevance. We propose a hybrid quantum-classical (HQC) deep deterministic policy gradient (DDPG) approach for a robot to navigate through a maze using continuous states, continuous actions and using local observations from the robot’s LiDAR sensors. We show that this HQC method can lead to models of comparable test results to the neural network (NN)-based DDPG algorithm, that need around 200 times fewer weights. We also study the scalability of our solution with respect to the number of VQC layers and qubits, and find that in general results improve as the layer and qubit counts increase. The best rewards among all similarly sized HQC and classical DDPG methods correspond to a VQC of 8 qubits and 5 layers with no other NN. This work is another step towards continuous QRL, where literature has been sparse.

International Conference on Agents and Artificial Intelligence (ICAART)•Jan 1, 2025•1 citation

Reinforcement Learning for Gate Synthesis in Noisy Quantum Systems

Amara Katabarwa, Collin Farquhar, Hyeongrak Choi +4 more

We present a gate synthesis application of the Deep Deterministic Policy Gradient (DDPG) algorithm for continuous control in a noisy quantum environment. Gate synthesis plays a crucial role in quantum computing, and optimizing control strategies for gate synthesis is of significant interest. The proposed approach leverages a DNN-based actor-critic architecture to optimize gate synthesis performance. The effectiveness of the algorithm is evaluated through simulations in a noiseless and noisy quantum environment, demonstrating its ability to achieve high-fidelity gate synthesis,$\mathcal{O}(10^{-3})$, despite the presence of Markovian damping terms. The results highlight the potential of DDPG and reinforcement learning techniques for addressing continuous control challenges in quantum computing applications.

IEEE International Conference on Quantum Computing and Engineering (QCE)•Sep 17, 2023•0 citations

Quantum reinforcement learning in continuous action space

Shaojun Wu, Shan Jin, Dingding Wen +2 more

Quantum reinforcement learning (QRL) is a promising paradigm for near-term quantum devices. While existing QRL methods have shown success in discrete action spaces, extending these techniques to continuous domains is challenging due to the curse of dimensionality introduced by discretization. To overcome this limitation, we introduce a quantum Deep Deterministic Policy Gradient (DDPG) algorithm that efficiently addresses both classical and quantum sequential decision problems in continuous action spaces. Moreover, our approach facilitates single-shot quantum state generation: a one-time optimization produces a model that outputs the control sequence required to drive a fixed initial state to any desired target state. In contrast, conventional quantum control methods demand separate optimization for each target state. We demonstrate the effectiveness of our method through simulations and discuss its potential applications in quantum control.

arXiv:2012.10711•Quantum•Mar 12, 2025•45 citations

Application of Deep Reinforcement Learning in Quantum Control

Yiming Cheng, Yixi Yin, Jiaying Lin +1 more

Abstract Machine learning technology based on artificial neural network has been successfully applied to solve many scientific problems. One of the most interesting areas of machine learning is reinforcement learning, which has natural applicability to optimization problems in physics. In the quantum control task, it is necessary to find a set of optimal control functions to transfer a quantum system from the initial state to the target state with the highest fidelity possible, which is essentially an optimization task. In this paper, we use Deep Deterministic Policy Gradient algorithm (DDPG) to study the quantum control tasks. We use the algorithm to control the transfer of several quantum systems from one state to another. The results show that DDPG algorithm can find a control strategy to make the fidelity of the final state and the target state of the quantum system be maximum value 1. The results show the potential of DDPG in quantum control.

Journal of Physics Conference Series•May 1, 2023•1 citation

Quantum-Enhanced DRL Optimization for DoA Estimation and Task Offloading in ISAC Systems

Anal Paul, Keshav Kumar Singh, Aryan Kaushik +4 more

This work proposes a quantum-aided deep reinforcement learning (DRL) framework designed to enhance the accuracy of direction-of-arrival (DoA) estimation and the efficiency of computational task offloading in integrated sensing and communication systems. Traditional DRL approaches face challenges in handling high-dimensional state spaces and ensuring convergence to optimal policies within complex operational environments. The proposed quantum-aided DRL framework that operates in a military surveillance system exploits quantum computing’s parallel processing capabilities to encode operational states and actions into quantum states, significantly reducing the dimensionality of the decision space. For the very first time in literature, we propose a quantum-enhanced actor-critic method, utilizing quantum circuits for policy representation and optimization. Through comprehensive simulations, we demonstrate that our framework improves DoA estimation accuracy by 91.66% and 82.61% over existing DRL algorithms with faster convergence rate, and effectively manages the trade-off between sensing and communication and optimizing task offloading decisions under stringent ultra-reliable low-latency communication requirements. Comparative analysis also reveals that our approach reduces the overall task offloading latency by 43.09% and 32.35% compared to the DRL-based deep deterministic policy gradient and proximal policy optimization algorithms, respectively.

IEEE Journal on Selected Areas in Communications•Sep 13, 2024•32 citations
This study proposes quantum-based deep deterministic policy gradient (Q-DDPG) and quantum-based recurrent DDPG (Q-RDDPG) schemes for time-series optimization in UAV communications. Herein, Q-DDPG based actor-critic reinforcement learning is utilized to optimize action selections in a large state and continuous action space. In this scheme, quantum models are exploited to reduce computational complexity and training loss. As a particular case, Q-DDPG and Q-RDDPG are employed for trajectory optimization and dynamic resource allocation in UAV communications. Quantum circuits of the Q-DDPG schemes are described to showcase their implementation in noisy intermediate-scale quantum (NISQ) computers. The results demonstrate that Q-DDPG and Q-RDDPG schemes achieved higher rewards with lower training losses compared to classical DDPG.
This study proposes a layerwise quantum-based deep reinforcement learning (LQ-DRL) method for optimizing continuous large space and time series problems using deep-layer training. The actions in LQ-DRL are optimized using a layerwise quantum embedding that leverages the advantages of quantum computing to maximize reward and reduce training loss. Moreover, this study employs a local loss to minimize the occurrence of barren plateaus phenomena and further enhance performance. As a particular case, the proposed scheme is employed to jointly optimize: 1) unmanned aerial vehicle (UAV) trajectory planning; 2) user grouping; and 3) power allocation for higher energy efficiency of a UAV as the reward. The combination of these optimized factors is referred to as action space in the presented LQ-DRL. The LQ-DRL is employed to solve the optimization problem due to its nonconvexity, continuous and large action space, and time-series domain. In a practical view, LQ-DRL aims to solve the issue of energy consumption related to limited-battery energy of a UAV base station (BS) while maintaining Quality of Service (QoS) for users, by gaining maximum energy efficiency as the reward. One of real applications, as an example, LQ-DRL can be employed to maximize the energy efficiency of a UAV BS in UAV empowered disaster recovery networks scenario. The quantum circuits of layerwise quantum embedding are presented to show the practical implementation in noisy intermediate-scale quantum computers. Based on the results, LQ-DRL outperformed the classical DRL by achieving higher effective dimension, rewards, and lower learning losses. In addition, better performances were achieved using more layers.
One of the multiple facets of quantum reinforcement learning (QRL) is enhancing reinforcement learning (RL) algorithms with quantum submodules, namely with variational quantum circuits (VQC) as function approx-imators. QRL solutions are empirically proven to require fewer training iterations or adjustable parameters than their classical counterparts, but are usually restricted to applications that have a discrete action space and thus limited industrial relevance. We propose a hybrid quantum-classical (HQC) deep deterministic policy gradient (DDPG) approach for a robot to navigate through a maze using continuous states, continuous actions and using local observations from the robot’s LiDAR sensors. We show that this HQC method can lead to models of comparable test results to the neural network (NN)-based DDPG algorithm, that need around 200 times fewer weights. We also study the scalability of our solution with respect to the number of VQC layers and qubits, and find that in general results improve as the layer and qubit counts increase. The best rewards among all similarly sized HQC and classical DDPG methods correspond to a VQC of 8 qubits and 5 layers with no other NN. This work is another step towards continuous QRL, where literature has been sparse.
We present a gate synthesis application of the Deep Deterministic Policy Gradient (DDPG) algorithm for continuous control in a noisy quantum environment. Gate synthesis plays a crucial role in quantum computing, and optimizing control strategies for gate synthesis is of significant interest. The proposed approach leverages a DNN-based actor-critic architecture to optimize gate synthesis performance. The effectiveness of the algorithm is evaluated through simulations in a noiseless and noisy quantum environment, demonstrating its ability to achieve high-fidelity gate synthesis,O(10−3)\mathcal{O}(10^{-3})O(10−3), despite the presence of Markovian damping terms. The results highlight the potential of DDPG and reinforcement learning techniques for addressing continuous control challenges in quantum computing applications.
Quantum reinforcement learning (QRL) is a promising paradigm for near-term quantum devices. While existing QRL methods have shown success in discrete action spaces, extending these techniques to continuous domains is challenging due to the curse of dimensionality introduced by discretization. To overcome this limitation, we introduce a quantum Deep Deterministic Policy Gradient (DDPG) algorithm that efficiently addresses both classical and quantum sequential decision problems in continuous action spaces. Moreover, our approach facilitates single-shot quantum state generation: a one-time optimization produces a model that outputs the control sequence required to drive a fixed initial state to any desired target state. In contrast, conventional quantum control methods demand separate optimization for each target state. We demonstrate the effectiveness of our method through simulations and discuss its potential applications in quantum control.
Abstract Machine learning technology based on artificial neural network has been successfully applied to solve many scientific problems. One of the most interesting areas of machine learning is reinforcement learning, which has natural applicability to optimization problems in physics. In the quantum control task, it is necessary to find a set of optimal control functions to transfer a quantum system from the initial state to the target state with the highest fidelity possible, which is essentially an optimization task. In this paper, we use Deep Deterministic Policy Gradient algorithm (DDPG) to study the quantum control tasks. We use the algorithm to control the transfer of several quantum systems from one state to another. The results show that DDPG algorithm can find a control strategy to make the fidelity of the final state and the target state of the quantum system be maximum value 1. The results show the potential of DDPG in quantum control.
This work proposes a quantum-aided deep reinforcement learning (DRL) framework designed to enhance the accuracy of direction-of-arrival (DoA) estimation and the efficiency of computational task offloading in integrated sensing and communication systems. Traditional DRL approaches face challenges in handling high-dimensional state spaces and ensuring convergence to optimal policies within complex operational environments. The proposed quantum-aided DRL framework that operates in a military surveillance system exploits quantum computing’s parallel processing capabilities to encode operational states and actions into quantum states, significantly reducing the dimensionality of the decision space. For the very first time in literature, we propose a quantum-enhanced actor-critic method, utilizing quantum circuits for policy representation and optimization. Through comprehensive simulations, we demonstrate that our framework improves DoA estimation accuracy by 91.66% and 82.61% over existing DRL algorithms with faster convergence rate, and effectively manages the trade-off between sensing and communication and optimizing task offloading decisions under stringent ultra-reliable low-latency communication requirements. Comparative analysis also reveals that our approach reduces the overall task offloading latency by 43.09% and 32.35% compared to the DRL-based deep deterministic policy gradient and proximal policy optimization algorithms, respectively.