Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2311.12875 · Quantum Machine Intelligence

Nav-Q: quantum deep reinforcement learning for collision-free navigation of self-driving cars

Akash Sinha, Antonio Macaluso, Matthias Klusch

Published Feb 11, 2025• 17 citations
arXivHTMLPDF

Abstract

Abstract The task of collision-free navigation (CFN) of self-driving cars is an NP-hard problem usually tackled using Deep Reinforcement Learning (DRL). While DRL methods have proven to be effective, their implementation requires substantial computing resources and extended training periods to develop a robust agent. On the other hand, quantum reinforcement learning has recently demonstrated faster convergence and improved stability in simple, non-real-world environments. In this work, we propose Nav-Q, the first quantum-supported DRL algorithm for CFN of self-driving cars, that leverages quantum computation for improving the training performance without the requirement for onboard quantum hardware. Nav-Q is based on the actor-critic approach, where the critic is implemented using a hybrid quantum-classical algorithm suitable for near-term quantum devices. We assess the performance of Nav-Q using the CARLA driving simulator, a de facto standard benchmark for evaluating state-of-the-art DRL methods. Our empirical evaluations showcase that Nav-Q surpasses its classical counterpart in terms of training stability and, in certain instances, with respect to the convergence rate. Furthermore, we assess Nav-Q in relation to effective dimension, unveiling that the incorporation of a quantum component results in a model with greater descriptive power compared to classical baselines. Finally, we evaluate the performance of Nav-Q using noisy quantum simulation, observing that the quantum noise deteriorates the training performances but enhances the exploratory tendencies of the agent during training.

DOI
10.1007/s42484-024-00226-4

Related papers

Quantum deep reinforcement learning for humanoid robot navigation task

Romerik Lokossou, Birhanu Shimelis Girma, O.K. Tonguz +1 more

Classical reinforcement learning (RL) methods often struggle in complex, high-dimensional environments because of their extensive parameter requirements and challenges posed by stochastic, non-deterministic settings. This study introduces quantum deep reinforcement learning (QDRL) to train humanoid agents efficiently. While previous quantum RL models focused on smaller environments, such as wheeled robots and robotic arms, our work pioneers the application of QDRL to humanoid robotics, specifically in environments with substantial observation and action spaces, such as MuJoCo's Humanoid-v4 and Walker2d-v4. Using parameterized quantum circuits, we explored a hybrid quantum-classical setup to directly navigate high-dimensional state spaces, bypassing traditional mapping and planning. By integrating quantum computing with deep RL, we aim to develop models that can efficiently learn complex navigation tasks in humanoid robots. We evaluated the performance of the Soft Actor-Critic (SAC) in classical RL against its quantum implementation. The results show that the quantum SAC achieves an 8% higher average return (246.40) than the classical SAC (228.36) after 92% fewer steps, highlighting the accelerated learning potential of quantum computing in RL tasks.

arXiv:2509.11388•Sep 14, 2025•0 citations

Quantum Deep Reinforcement Learning for Robot Navigation Tasks

Hans Hohenfeld, Dirk Heimann, Felix Wiebe +1 more

We utilize hybrid quantum deep reinforcement learning to learn navigation tasks for a simple, wheeled robot in simulated environments of increasing complexity. For this, we train parameterized quantum circuits (PQCs) with two different encoding strategies in a hybrid quantum-classical setup as well as a classical neural network baseline with the double deep Q network (DDQN) reinforcement learning algorithm. Quantum deep reinforcement learning (QDRL) has previously been studied in several relatively simple benchmark environments, mainly from the OpenAI gym suite. However, scaling behavior and applicability of QDRL to more demanding tasks closer to real-world problems e.g., from the robotics domain, have not been studied previously. Here, we show that quantum circuits in hybrid quantum-classic reinforcement learning setups are capable of learning optimal policies in multiple robotic navigation scenarios with notably fewer trainable parameters compared to a classical baseline. Across a large number of experimental configurations, we find that the employed quantum circuits outperform the classical neural network baselines when equating for the number of trainable parameters. Yet, the classical neural network consistently showed better results concerning training times and stability, with at least one order of magnitude of trainable parameters more than the best-performing quantum circuits. However, validating the robustness of the learning methods in a large and dynamic environment, we find that the classical baseline produces more stable and better performing policies overall. For the two encoding schemes, we observed better results for consecutively encoding the classical state vector on each qubit compared to encoding each component on a separate qubit. Our findings demonstrate that current hybrid quantum machine-learning approaches can be scaled to simple robotic problems while yielding sufficient results, at least in an idealized simulated setting, but there are yet open questions regarding the application to considerably more demanding tasks. We anticipate that our work will contribute to introducing quantum machine learning in general and quantum deep reinforcement learning in particular to more demanding problem domains and emphasize the importance of encoding techniques for classic data in hybrid quantum-classical settings.

arXiv:2202.12180•IEEE Access•Jan 1, 2024•43 citations

Survey Paper on Quantum Deep Reinforcement Learning for Robot Navigation Tasks

Shruti Doiphode, Prof. Mukul Jagtap

Abstract: Quantum deep reinforcement learning is an integration of the principles of quantum computing with deep reinforcement learning in an effort to advance robot navigation tasks. Due to the explosion of the state and action spaces, traditionally DRL methods are limited in scalability and efficiency. QDRL, however, uses quantum superposition and entanglement for more efficient processing of big amounts of information-it enables robots to learn optimal policies for navigating complex environments. We introduce here a new QDRL framework in which quantum neural networks encode policy functions and value estimates. We demonstrate how quantum circuits can take advantage of multi-dimensional state representations in order to strengthen the power of a robot in changing its environment over time. Benchmark experiments on navigation tasks reflect phenomenal acceleration in learning performance along with better decision-making accuracy compared to classical DRL methods. We then consider integrating such quantum algorithms, as Grover's search, to improve exploration strategies in sparse reward settings. The experiments show that QDRL speeds up convergence rates and provides robots with the possibility to react more rapidly to unexpected obstacles. It discusses new applications for quantum computing of autonomous systems, potentially applied to robotics, autonomous vehicles, and other environments which require complex decision making. Some lines of work in the future are optimization of the resources needed for quantum computers and hybrid architectures for the resolution of classical problems in navigation

International Journal for Research in Applied Science and Engineering Technology•Oct 18, 2024•0 citations

An Approach Based on Quantum Reinforcement Learning for Navigation Problems

Níyazí Furkan Bar, Hasan Yetış, Mehmet Karaköse

The power of classical computers is still insufficient for deep reinforcement learning (DRL) problems which has large state space. Thanks to entanglement and superposition, quantum computers have a high computational power. The concept of using this high computing power to solve problems that would be difficult for classical computers is fairly common. In this study, a hybrid approach is proposed to take advantage of the benefits of quantum computers. Deep Q-Network (DQN) algorithm for DRL, optimization operations, and storage operations are performed on the classical computer side of this hybrid approach. In the quantum side, a variational quantum circuit (VQC) is proposed. The proposed method is applied to a navigation problem. The proposed approach is evaluated in terms of target success rate, collision (going out) rate. The proposed approach is compared to DRL solutions in the literature for the navigation problem with classical computers. According to the number of parameters used, the proposed approach appears to be successful. As a result, the performance of the proposed approach has been validated.

International Conference on Data Analytics for Business and Industry (ICDABI)•Oct 25, 2022•11 citations

Universal Quantum Control through Deep Reinforcement Learning

Murphy Yuezhen Niu, Sergio Boixo, Vadim Smelyanskiy +1 more

Abstract Emerging reinforcement learning techniques using deep neural networks have shown great promise in control optimization. They harness non-local regularities of noisy control trajectories and facilitate transfer learning between tasks. To leverage these powerful capabilities for quantum control optimization, we propose a new control framework to simultaneously optimize the speed and fidelity of quantum computation against both leakage and stochastic control errors. For a broad family of two-qubit unitary gates that are important for quantum simulation of many-electron systems, we improve the control robustness by adding control noise into training environments for reinforcement learning agents trained with trusted-region-policy-optimization. The agent control solutions demonstrate a two-order-of-magnitude reduction in average-gate-error over baseline stochastic-gradient-descent solutions and up to a one-order-of-magnitude reduction in gate time from optimal gate synthesis counterparts. These significant improvements in both fidelity and runtime are achieved by combining new physical understandings and state-of-the-art machine learning techniques. Our results open a venue for wider applications in quantum simulation, quantum chemistry and quantum supremacy tests using near-term quantum devices.

arXiv:1803.01857•npj Quantum Information•Jan 6, 2019•387 citations

Quantum-Enhanced Hybrid Reinforcement Learning Framework for Dynamic Path Planning in Autonomous Systems

Tomar, Sahil, Shamshe Alam, Sandeep Kumar +1 more

In this paper, a novel quantum classical hybrid framework is proposed that synergizes quantum with Classical Reinforcement Learning. By leveraging the inherent parallelism of quantum computing, the proposed approach generates robust Q tables and specialized turn cost estimations, which are then integrated with a classical Reinforcement Learning pipeline. The Classical Quantum fusion results in rapid convergence of training, reducing the training time significantly and improved adaptability in scenarios featuring static, dynamic, and moving obstacles. Simulator based evaluations demonstrate significant enhancements in path efficiency, trajectory smoothness, and mission success rates, underscoring the potential of framework for real time, autonomous navigation in complex and unpredictable environments. Furthermore, the proposed framework was tested beyond simulations on practical scenarios, including real world map data such as the IIT Delhi campus, reinforcing its potential for real time, autonomous navigation in complex and unpredictable environments.

arXiv:2504.20660•Apr 29, 2025•1 citation
Abstract The task of collision-free navigation (CFN) of self-driving cars is an NP-hard problem usually tackled using Deep Reinforcement Learning (DRL). While DRL methods have proven to be effective, their implementation requires substantial computing resources and extended training periods to develop a robust agent. On the other hand, quantum reinforcement learning has recently demonstrated faster convergence and improved stability in simple, non-real-world environments. In this work, we propose Nav-Q, the first quantum-supported DRL algorithm for CFN of self-driving cars, that leverages quantum computation for improving the training performance without the requirement for onboard quantum hardware. Nav-Q is based on the actor-critic approach, where the critic is implemented using a hybrid quantum-classical algorithm suitable for near-term quantum devices. We assess the performance of Nav-Q using the CARLA driving simulator, a de facto standard benchmark for evaluating state-of-the-art DRL methods. Our empirical evaluations showcase that Nav-Q surpasses its classical counterpart in terms of training stability and, in certain instances, with respect to the convergence rate. Furthermore, we assess Nav-Q in relation to effective dimension, unveiling that the incorporation of a quantum component results in a model with greater descriptive power compared to classical baselines. Finally, we evaluate the performance of Nav-Q using noisy quantum simulation, observing that the quantum noise deteriorates the training performances but enhances the exploratory tendencies of the agent during training.
Classical reinforcement learning (RL) methods often struggle in complex, high-dimensional environments because of their extensive parameter requirements and challenges posed by stochastic, non-deterministic settings. This study introduces quantum deep reinforcement learning (QDRL) to train humanoid agents efficiently. While previous quantum RL models focused on smaller environments, such as wheeled robots and robotic arms, our work pioneers the application of QDRL to humanoid robotics, specifically in environments with substantial observation and action spaces, such as MuJoCo's Humanoid-v4 and Walker2d-v4. Using parameterized quantum circuits, we explored a hybrid quantum-classical setup to directly navigate high-dimensional state spaces, bypassing traditional mapping and planning. By integrating quantum computing with deep RL, we aim to develop models that can efficiently learn complex navigation tasks in humanoid robots. We evaluated the performance of the Soft Actor-Critic (SAC) in classical RL against its quantum implementation. The results show that the quantum SAC achieves an 8% higher average return (246.40) than the classical SAC (228.36) after 92% fewer steps, highlighting the accelerated learning potential of quantum computing in RL tasks.
We utilize hybrid quantum deep reinforcement learning to learn navigation tasks for a simple, wheeled robot in simulated environments of increasing complexity. For this, we train parameterized quantum circuits (PQCs) with two different encoding strategies in a hybrid quantum-classical setup as well as a classical neural network baseline with the double deep Q network (DDQN) reinforcement learning algorithm. Quantum deep reinforcement learning (QDRL) has previously been studied in several relatively simple benchmark environments, mainly from the OpenAI gym suite. However, scaling behavior and applicability of QDRL to more demanding tasks closer to real-world problems e.g., from the robotics domain, have not been studied previously. Here, we show that quantum circuits in hybrid quantum-classic reinforcement learning setups are capable of learning optimal policies in multiple robotic navigation scenarios with notably fewer trainable parameters compared to a classical baseline. Across a large number of experimental configurations, we find that the employed quantum circuits outperform the classical neural network baselines when equating for the number of trainable parameters. Yet, the classical neural network consistently showed better results concerning training times and stability, with at least one order of magnitude of trainable parameters more than the best-performing quantum circuits. However, validating the robustness of the learning methods in a large and dynamic environment, we find that the classical baseline produces more stable and better performing policies overall. For the two encoding schemes, we observed better results for consecutively encoding the classical state vector on each qubit compared to encoding each component on a separate qubit. Our findings demonstrate that current hybrid quantum machine-learning approaches can be scaled to simple robotic problems while yielding sufficient results, at least in an idealized simulated setting, but there are yet open questions regarding the application to considerably more demanding tasks. We anticipate that our work will contribute to introducing quantum machine learning in general and quantum deep reinforcement learning in particular to more demanding problem domains and emphasize the importance of encoding techniques for classic data in hybrid quantum-classical settings.
Abstract: Quantum deep reinforcement learning is an integration of the principles of quantum computing with deep reinforcement learning in an effort to advance robot navigation tasks. Due to the explosion of the state and action spaces, traditionally DRL methods are limited in scalability and efficiency. QDRL, however, uses quantum superposition and entanglement for more efficient processing of big amounts of information-it enables robots to learn optimal policies for navigating complex environments. We introduce here a new QDRL framework in which quantum neural networks encode policy functions and value estimates. We demonstrate how quantum circuits can take advantage of multi-dimensional state representations in order to strengthen the power of a robot in changing its environment over time. Benchmark experiments on navigation tasks reflect phenomenal acceleration in learning performance along with better decision-making accuracy compared to classical DRL methods. We then consider integrating such quantum algorithms, as Grover's search, to improve exploration strategies in sparse reward settings. The experiments show that QDRL speeds up convergence rates and provides robots with the possibility to react more rapidly to unexpected obstacles. It discusses new applications for quantum computing of autonomous systems, potentially applied to robotics, autonomous vehicles, and other environments which require complex decision making. Some lines of work in the future are optimization of the resources needed for quantum computers and hybrid architectures for the resolution of classical problems in navigation
The power of classical computers is still insufficient for deep reinforcement learning (DRL) problems which has large state space. Thanks to entanglement and superposition, quantum computers have a high computational power. The concept of using this high computing power to solve problems that would be difficult for classical computers is fairly common. In this study, a hybrid approach is proposed to take advantage of the benefits of quantum computers. Deep Q-Network (DQN) algorithm for DRL, optimization operations, and storage operations are performed on the classical computer side of this hybrid approach. In the quantum side, a variational quantum circuit (VQC) is proposed. The proposed method is applied to a navigation problem. The proposed approach is evaluated in terms of target success rate, collision (going out) rate. The proposed approach is compared to DRL solutions in the literature for the navigation problem with classical computers. According to the number of parameters used, the proposed approach appears to be successful. As a result, the performance of the proposed approach has been validated.
Abstract Emerging reinforcement learning techniques using deep neural networks have shown great promise in control optimization. They harness non-local regularities of noisy control trajectories and facilitate transfer learning between tasks. To leverage these powerful capabilities for quantum control optimization, we propose a new control framework to simultaneously optimize the speed and fidelity of quantum computation against both leakage and stochastic control errors. For a broad family of two-qubit unitary gates that are important for quantum simulation of many-electron systems, we improve the control robustness by adding control noise into training environments for reinforcement learning agents trained with trusted-region-policy-optimization. The agent control solutions demonstrate a two-order-of-magnitude reduction in average-gate-error over baseline stochastic-gradient-descent solutions and up to a one-order-of-magnitude reduction in gate time from optimal gate synthesis counterparts. These significant improvements in both fidelity and runtime are achieved by combining new physical understandings and state-of-the-art machine learning techniques. Our results open a venue for wider applications in quantum simulation, quantum chemistry and quantum supremacy tests using near-term quantum devices.
In this paper, a novel quantum classical hybrid framework is proposed that synergizes quantum with Classical Reinforcement Learning. By leveraging the inherent parallelism of quantum computing, the proposed approach generates robust Q tables and specialized turn cost estimations, which are then integrated with a classical Reinforcement Learning pipeline. The Classical Quantum fusion results in rapid convergence of training, reducing the training time significantly and improved adaptability in scenarios featuring static, dynamic, and moving obstacles. Simulator based evaluations demonstrate significant enhancements in path efficiency, trajectory smoothness, and mission success rates, underscoring the potential of framework for real time, autonomous navigation in complex and unpredictable environments. Furthermore, the proposed framework was tested beyond simulations on practical scenarios, including real world map data such as the IIT Delhi campus, reinforcing its potential for real time, autonomous navigation in complex and unpredictable environments.