IEEE RO-MAN

Quantum Exploration-based Reinforcement Learning for Efficient Robot Path Planning in Sparse-Reward Environment

Chao Huang, Yibei Guo, Zhihui Zhu, Mei Si, Daniel J Blankenberg, Rui Liu

Published Aug 26, 2024• 3 citations

Abstract

With the latest developments in sensors, battery, and Artificial Intelligence (AI) technologies, robots can perform missions in unstructured environments (e.g., disaster sites). However, their adaptation speed is still slow due to their limited onboard computing capability and dynamic disruption from environments, leading to inaccurate planning and delayed emergency reactions. Even though reinforcement learning increases robots’ exploration speed by fusing effective guidance, the numerous interactions make the learning time-consuming and even risky (e.g., collisions). To fundamentally improve robot adaptation speed, this work seeks help from quantum power. A novel Quantum Exploration based Dreamer model (QED) was developed to facilitate reinforcement learning explorations. QED based on stochastic quantum walker quickly explores environments, evaluates action quality, and obtains a global optimal exploration strategy; then, these high-quality exploration samples will be used to facilitate reinforcement learning speed. A theoretical benefit of QED is facilitating reinforcement learning speed without changing the underlying learning architecture, which makes the proposed QED applicable to general robot learning scenarios. To validate QED effectiveness, a robot path-planning task in an obstacle-dense environment was designed. The number of needed training episodes validated the effectiveness. The results show that QED achieves around ten times faster policy learning compared with Monte Carlo tree-based reinforcement learning methods and vanilla reinforcement learning methods.

DOI: 10.1109/ro-man60168.2024.10731199

Related papers

Survey Paper on Quantum Deep Reinforcement Learning for Robot Navigation Tasks

Shruti Doiphode, Prof. Mukul Jagtap

Abstract: Quantum deep reinforcement learning is an integration of the principles of quantum computing with deep reinforcement learning in an effort to advance robot navigation tasks. Due to the explosion of the state and action spaces, traditionally DRL methods are limited in scalability and efficiency. QDRL, however, uses quantum superposition and entanglement for more efficient processing of big amounts of information-it enables robots to learn optimal policies for navigating complex environments. We introduce here a new QDRL framework in which quantum neural networks encode policy functions and value estimates. We demonstrate how quantum circuits can take advantage of multi-dimensional state representations in order to strengthen the power of a robot in changing its environment over time. Benchmark experiments on navigation tasks reflect phenomenal acceleration in learning performance along with better decision-making accuracy compared to classical DRL methods. We then consider integrating such quantum algorithms, as Grover's search, to improve exploration strategies in sparse reward settings. The experiments show that QDRL speeds up convergence rates and provides robots with the possibility to react more rapidly to unexpected obstacles. It discusses new applications for quantum computing of autonomous systems, potentially applied to robotics, autonomous vehicles, and other environments which require complex decision making. Some lines of work in the future are optimization of the resources needed for quantum computers and hybrid architectures for the resolution of classical problems in navigation

International Journal for Research in Applied Science and Engineering TechnologyOct 18, 20240 citations

Quantum-Enhanced Reinforcement Learning (QRL) for Robotics

Jincheng Zhang

This paper explores the application of Quantum-Enhanced Reinforcement Learning (QRL) to robotics, aiming to accelerate learning speeds and enable more intelligent decision-making in complex environments. Traditional reinforcement learning algorithms often struggle with scalability and convergence, particularly when dealing with high-dimensional state spaces and complex environments. Quantum computing offers the potential to overcome these limitations by leveraging quantum mechanics to perform computations that are intractable for classical computers. We propose a novel QRL framework that utilizes quantum-enhanced reinforcement learning to improve exploration, adaptation, and overall performance. The core mechanism involves employing quantum circuits to represent and manipulate the state space, facilitating more efficient learning and potentially uncovering novel strategies. This research investigates the feasibility and potential impact of QRL on robotic control, focusing on scenarios requiring rapid adaptation and robust decision-making in dynamic environments. We present preliminary results demonstrating the efficacy of our approach in a simulated robotic task involving navigation and obstacle avoidance.

Sep 3, 20260 citations

Path Planning for Autonomous Robots using Quantum Reinforcement Learning

L. Tharun, .Swetha Shailaja, G. Aswitha +1 more

ABSTRACT-This research work focuses on the application of Quantum Reinforcement Learning (QRL) to improve path planning for autonomous robots. Traditional path planning techniques are prone to failure in handling the complexity of uncertain and dynamic environments, but QRL leverages the power of quantum computing to exploit the principles of superposition and parallelism to improve decision-making and optimization processes. By integrating QRL with existing reinforcement learning techniques, the proposed technique quickens the pace of learning processes and improves the quality of path planning to enable robots to navigate around obstacles more effectively with shorter travel time and lower energy costs. Simulation results show that QRL outperforms traditional techniques in convergence speed and optimality of path solutions and offers a promising solution for practical autonomous navigation. Keywords: Dynamic Environment, Path Planning, Autonomous Navigation, Obstacles avoidance.

INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENTFeb 26, 20260 citations

Quantum deep reinforcement learning for humanoid robot navigation task

Romerik Lokossou, Birhanu Shimelis Girma, O.K. Tonguz +1 more

Classical reinforcement learning (RL) methods often struggle in complex, high-dimensional environments because of their extensive parameter requirements and challenges posed by stochastic, non-deterministic settings. This study introduces quantum deep reinforcement learning (QDRL) to train humanoid agents efficiently. While previous quantum RL models focused on smaller environments, such as wheeled robots and robotic arms, our work pioneers the application of QDRL to humanoid robotics, specifically in environments with substantial observation and action spaces, such as MuJoCo's Humanoid-v4 and Walker2d-v4. Using parameterized quantum circuits, we explored a hybrid quantum-classical setup to directly navigate high-dimensional state spaces, bypassing traditional mapping and planning. By integrating quantum computing with deep RL, we aim to develop models that can efficiently learn complex navigation tasks in humanoid robots. We evaluated the performance of the Soft Actor-Critic (SAC) in classical RL against its quantum implementation. The results show that the quantum SAC achieves an 8% higher average return (246.40) than the classical SAC (228.36) after 92% fewer steps, highlighting the accelerated learning potential of quantum computing in RL tasks.

arXiv:2509.11388Sep 14, 20250 citations

Quantum-Enhanced Reinforcement Learning for Robotics

Jincheng Zhang

Quantum-enhanced reinforcement learning (QEL) offers the potential to significantly improve the performance of robotic systems, particularly in navigating complex environments and adapting to dynamic situations. This paper explores the integration of quantum computing with reinforcement learning to accelerate learning, enhance exploration, and improve adaptation. We propose a novel approach that leverages quantum annealing and quantum reinforcement learning to overcome the limitations of classical RL in challenging scenarios. The core claim is that combining these two paradigms will lead to enhanced robot navigation and control capabilities. The study focuses on demonstrating the effectiveness of this combined approach through simulations and preliminary results.

Sep 7, 20260 citations

Quantum-Inspired Algorithms for Reinforcement Learning

Jincheng Zhang

This paper investigates the application of quantum-inspired algorithms to the field of reinforcement learning (RL). Traditional RL methods often struggle with exploration challenges and can be computationally expensive, particularly in complex environments. We propose a novel RL algorithm that leverages concepts from quantum mechanics, specifically superposition and entanglement, to address these limitations. The core idea is to represent states, actions, and rewards using quantum-inspired representations, allowing the agent to explore the environment more efficiently and potentially converge to optimal policies faster. This work presents a detailed formulation of the algorithm, including the mathematical framework and key considerations for implementation. The potential benefits of this approach, including improved exploration and reduced computational complexity, are discussed. The algorithm is designed to iteratively refine its policy based on simulated quantum measurements, aiming to minimize the expected return. The exploration strategy is guided by the inherent probabilistic nature of quantum states, leading to a more diverse and potentially faster learning process. Further research is warranted to fully explore the capabilities and limitations of this approach across various RL benchmark problems.

Sep 1, 20260 citations