Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2308.01519 · IEEE Communications Magazine

Quantum Multi-Agent Reinforcement Learning for Autonomous Mobility Cooperation

Soohyun Park, Jae Pyoung Kim, Chanyoung Park, Soyi Jung, Joongheon Kim

Published Aug 28, 2023• 66 citations
arXivHTMLPDF

Abstract

For Industry 4.0 Revolution, cooperative autonomous mobility systems are widely used based on multiagent reinforcement learning (MARL). However, the MARLbased algorithms suffer from huge parameter utilization and convergence difficulties with many agents. To tackle these problems, a quantum MARL (QMARL) algorithm based on the concept of actor-critic network is proposed, which is beneficial in terms of scalability, to deal with the limitations in the noisy intermediatescale quantum (NISQ) era. Additionally, our QMARL is also beneficial in terms of efficient parameter utilization and fast convergence due to quantum supremacy. Note that the reward in our QMARL is defined as task precision over computation time in multiple agents, thus, multi-agent cooperation can be realized. For further improvement, an additional technique for scalability is proposed, which is called projection value measure (PVM). Based on PVM, our proposed QMARL can achieve the highest reward, by reducing the action dimension into a logarithmicscale. Finally, we can conclude that our proposed QMARL with PVM outperforms the other algorithms in terms of efficient parameter utilization, fast convergence, and scalability.

DOI
10.1109/mcom.020.2300199

Related papers

Quantum Reinforcement Learning for Large-Scale Multi-Agent Decision-Making in Autonomous Aerial Networks

Soohyun Park, Joongheon Kim

This paper addresses a new quantum computing-based multi-agent reinforcement learning (QMARL) algorithm which is inspired by the quantum neural network (QNN)-based centralized critic and multiple actor networks. The benefit of the proposed QMARL-based algorithm is in the action control dimension reduction where it can reduce the size into a logarithmic-scale when project value measure (PVM) is utilized. Therefore, our proposed QMARL-based algorithm is beneficial for massive-agent MARL training convergence. Moreover, the various applications of QMARL-based algorithms are presented in massive-scale unmanned aerial vehicle (UAV) networks. Lastly, our performance evaluation results verify that the proposed QMARL-based algorithm can successfully converge when massive-action dimensions should be utilized.

IEEE VTS Asia Pacific Wireless Communications Symposium (APWCS)•Aug 1, 2023•5 citations

Quantum Multiagent Actor–Critic Networks for Cooperative Mobile Access in Multi-UAV Systems

Chanyoung Park, Won Joon Yun, Jae Pyoung Kim +4 more

This article proposes a novel algorithm, named quantum multiagent actor–critic networks (QMACN) for autonomously constructing a robust mobile access system employing multiple unmanned aerial vehicles (UAVs). In the context of facilitating collaboration among multiple UAVs, the application of multiagent reinforcement learning (MARL) techniques is regarded as a promising approach. These methods enable UAVs to learn collectively, optimizing their actions within a shared environment, ultimately leading to more efficient cooperative behavior. Furthermore, the principles of quantum computing (QC) are employed in our study to enhance the training process and inference capabilities of the UAVs involved. By leveraging the unique computational advantages of QC, our approach aims to boost the overall effectiveness of the UAV system. However, employing a QC introduces scalability challenges due to the near intermediate-scale quantum (NISQ) limitation associated with qubit usage. The proposed algorithm addresses this issue by implementing a quantum centralized critic, effectively mitigating the constraints imposed by NISQ limitations. Additionally, the advantages of the QMACN with performance improvements in terms of training speed and wireless service quality are verified via various data-intensive evaluations. Furthermore, this article validates that a noise injection scheme can be used for handling environmental uncertainties in order to realize robust mobile access.

arXiv:2302.04445•IEEE Internet of Things Journal•Jun 5, 2023•66 citations

Quantum Multi - Agent Reinforcement Learning Software Design and Visual Simulations for Multi - Drone Mobility Control

Soohyun Park, Gyu Seon Kim, Sovi Jung +1 more

Quantum machine learning (QML) has received a lot of attention according to its light training parameter numbers and speeds; and the advances of QML lead to active research on quantum multi-agent reinforcement learning (QMARL). Existing classical multi-agent reinforcement learning (MARL) features non-stationarity and uncertain properties. Therefore, this paper presents a simulation software framework for novel QMARL to control autonomous multi-drone mobility, i.e., quantum multi-drone reinforcement learning. Our proposed framework ac-complishes reasonable reward convergence and service quality performance with fewer trainable parameters. Furthermore, it shows more stable training results. Lastly, our proposed software allows us to analyze the training process and results.

IEEE VTS Asia Pacific Wireless Communications Symposium (APWCS)•Aug 21, 2024•2 citations

eQMARL: Entangled Quantum Multi-Agent Reinforcement Learning for Distributed Cooperation over Quantum Channels

Alexander DeRieux, Walid Saad

Collaboration is a key challenge in distributed multi-agent reinforcement learning (MARL) environments. Learning frameworks for these decentralized systems must weigh the benefits of explicit player coordination against the communication overhead and computational cost of sharing local observations and environmental data. Quantum computing has sparked a potential synergy between quantum entanglement and cooperation in multi-agent environments, which could enable more efficient distributed collaboration with minimal information sharing. This relationship is largely unexplored, however, as current state-of-the-art quantum MARL (QMARL) implementations rely on classical information sharing rather than entanglement over a quantum channel as a coordination medium. In contrast, in this paper, a novel framework dubbed entangled QMARL (eQMARL) is proposed. The proposed eQMARL is a distributed actor-critic framework that facilitates cooperation over a quantum channel and eliminates local observation sharing via a quantum entangled split critic. Introducing a quantum critic uniquely spread across the agents allows coupling of local observation encoders through entangled input qubits over a quantum channel, which requires no explicit sharing of local observations and reduces classical communication overhead. Further, agent policies are tuned through joint observation-value function estimation via joint quantum measurements, thereby reducing the centralized computational burden. Experimental results show that eQMARL with $Ψ^{+}$ entanglement converges to a cooperative strategy up to $17.8\%$ faster and with a higher overall score compared to split classical and fully centralized classical and quantum baselines. The results also show that eQMARL achieves this performance with a constant factor of $25$-times fewer centralized parameters compared to the split classical baseline.

arXiv:2405.17486•May 24, 2024•0 citations

Introduction to Quantum Multi-Agent Reinforcement Learning: Concepts and Applications

Yeryeong Cho, Seok Bin Son, Soohyun Park +1 more

Advancements in quantum computer hardware and software have paved the way for the application of quantum computing across various fields. With the development of computers with thousands of qubits, the power of quantum is increasing. Because of efficiency in high action dimensions, quantum computing has advantages in multi-agent fields where the amount of data is huge. At the same time, reinforcement learning (RL) has gained prominence for its potential in unknown environments. Consequently, the integration of quantum computing and multi-agent reinforcement learning (MARL), known as quantum multi-agent reinforcement learning (QMARL), is proposed. This paper introduces the overall concept and applications of quantum computing, MARL, and QMARL. It also shows the current technology trends depending on the application. In conclusion, the paper presents challenges and future research directions for QMARL and shows that QMARL can be utilized in various applications for excellent performance.

International Conference on Information and Communication Technology Convergence (ICTC)•Oct 16, 2024•0 citations

QiMARL: Quantum-Inspired Multi-Agent Reinforcement Learning Strategy for Efficient Resource Energy Distribution in Nodal Power Stations

Sapthak Mohajon Turjya, Anjan Bandyopadhyay, M. Shamim Kaiser +1 more

The coupling of quantum computing with multi-agent reinforcement learning (MARL) provides an exciting direction to tackle intricate decision-making tasks in high-dimensional spaces. This work introduces a new quantum-inspired multi-agent reinforcement learning (QiMARL) model, utilizing quantum parallelism to achieve learning efficiency and scalability improvement. The QiMARL model is tested on an energy distribution task, which optimizes power distribution between generating and demanding nodal power stations. We compare the convergence time, reward performance, and scalability of QiMARL with traditional Multi-Armed Bandit (MAB) and Multi-Agent Reinforcement Learning methods, such as Greedy, Upper Confidence Bound (UCB), Thompson Sampling, MADDPG, QMIX, and PPO methods with a comprehensive ablation study. Our findings show that QiMARL yields better performance in high-dimensional systems, decreasing the number of training epochs needed for convergence while enhancing overall reward maximization. We also compare the algorithm’s computational complexity, indicating that QiMARL is more scalable to high-dimensional quantum environments. This research opens the door to future studies of quantum-enhanced reinforcement learning (RL) with potential applications to energy optimization, traffic management, and other multi-agent coordination problems.

AI•Sep 1, 2025•5 citations
For Industry 4.0 Revolution, cooperative autonomous mobility systems are widely used based on multiagent reinforcement learning (MARL). However, the MARLbased algorithms suffer from huge parameter utilization and convergence difficulties with many agents. To tackle these problems, a quantum MARL (QMARL) algorithm based on the concept of actor-critic network is proposed, which is beneficial in terms of scalability, to deal with the limitations in the noisy intermediatescale quantum (NISQ) era. Additionally, our QMARL is also beneficial in terms of efficient parameter utilization and fast convergence due to quantum supremacy. Note that the reward in our QMARL is defined as task precision over computation time in multiple agents, thus, multi-agent cooperation can be realized. For further improvement, an additional technique for scalability is proposed, which is called projection value measure (PVM). Based on PVM, our proposed QMARL can achieve the highest reward, by reducing the action dimension into a logarithmicscale. Finally, we can conclude that our proposed QMARL with PVM outperforms the other algorithms in terms of efficient parameter utilization, fast convergence, and scalability.
This paper addresses a new quantum computing-based multi-agent reinforcement learning (QMARL) algorithm which is inspired by the quantum neural network (QNN)-based centralized critic and multiple actor networks. The benefit of the proposed QMARL-based algorithm is in the action control dimension reduction where it can reduce the size into a logarithmic-scale when project value measure (PVM) is utilized. Therefore, our proposed QMARL-based algorithm is beneficial for massive-agent MARL training convergence. Moreover, the various applications of QMARL-based algorithms are presented in massive-scale unmanned aerial vehicle (UAV) networks. Lastly, our performance evaluation results verify that the proposed QMARL-based algorithm can successfully converge when massive-action dimensions should be utilized.
This article proposes a novel algorithm, named quantum multiagent actor–critic networks (QMACN) for autonomously constructing a robust mobile access system employing multiple unmanned aerial vehicles (UAVs). In the context of facilitating collaboration among multiple UAVs, the application of multiagent reinforcement learning (MARL) techniques is regarded as a promising approach. These methods enable UAVs to learn collectively, optimizing their actions within a shared environment, ultimately leading to more efficient cooperative behavior. Furthermore, the principles of quantum computing (QC) are employed in our study to enhance the training process and inference capabilities of the UAVs involved. By leveraging the unique computational advantages of QC, our approach aims to boost the overall effectiveness of the UAV system. However, employing a QC introduces scalability challenges due to the near intermediate-scale quantum (NISQ) limitation associated with qubit usage. The proposed algorithm addresses this issue by implementing a quantum centralized critic, effectively mitigating the constraints imposed by NISQ limitations. Additionally, the advantages of the QMACN with performance improvements in terms of training speed and wireless service quality are verified via various data-intensive evaluations. Furthermore, this article validates that a noise injection scheme can be used for handling environmental uncertainties in order to realize robust mobile access.
Quantum machine learning (QML) has received a lot of attention according to its light training parameter numbers and speeds; and the advances of QML lead to active research on quantum multi-agent reinforcement learning (QMARL). Existing classical multi-agent reinforcement learning (MARL) features non-stationarity and uncertain properties. Therefore, this paper presents a simulation software framework for novel QMARL to control autonomous multi-drone mobility, i.e., quantum multi-drone reinforcement learning. Our proposed framework ac-complishes reasonable reward convergence and service quality performance with fewer trainable parameters. Furthermore, it shows more stable training results. Lastly, our proposed software allows us to analyze the training process and results.
Collaboration is a key challenge in distributed multi-agent reinforcement learning (MARL) environments. Learning frameworks for these decentralized systems must weigh the benefits of explicit player coordination against the communication overhead and computational cost of sharing local observations and environmental data. Quantum computing has sparked a potential synergy between quantum entanglement and cooperation in multi-agent environments, which could enable more efficient distributed collaboration with minimal information sharing. This relationship is largely unexplored, however, as current state-of-the-art quantum MARL (QMARL) implementations rely on classical information sharing rather than entanglement over a quantum channel as a coordination medium. In contrast, in this paper, a novel framework dubbed entangled QMARL (eQMARL) is proposed. The proposed eQMARL is a distributed actor-critic framework that facilitates cooperation over a quantum channel and eliminates local observation sharing via a quantum entangled split critic. Introducing a quantum critic uniquely spread across the agents allows coupling of local observation encoders through entangled input qubits over a quantum channel, which requires no explicit sharing of local observations and reduces classical communication overhead. Further, agent policies are tuned through joint observation-value function estimation via joint quantum measurements, thereby reducing the centralized computational burden. Experimental results show that eQMARL with Ψ+Ψ^{+}Ψ+ entanglement converges to a cooperative strategy up to 17.8%17.8\%17.8% faster and with a higher overall score compared to split classical and fully centralized classical and quantum baselines. The results also show that eQMARL achieves this performance with a constant factor of 252525-times fewer centralized parameters compared to the split classical baseline.
Advancements in quantum computer hardware and software have paved the way for the application of quantum computing across various fields. With the development of computers with thousands of qubits, the power of quantum is increasing. Because of efficiency in high action dimensions, quantum computing has advantages in multi-agent fields where the amount of data is huge. At the same time, reinforcement learning (RL) has gained prominence for its potential in unknown environments. Consequently, the integration of quantum computing and multi-agent reinforcement learning (MARL), known as quantum multi-agent reinforcement learning (QMARL), is proposed. This paper introduces the overall concept and applications of quantum computing, MARL, and QMARL. It also shows the current technology trends depending on the application. In conclusion, the paper presents challenges and future research directions for QMARL and shows that QMARL can be utilized in various applications for excellent performance.
The coupling of quantum computing with multi-agent reinforcement learning (MARL) provides an exciting direction to tackle intricate decision-making tasks in high-dimensional spaces. This work introduces a new quantum-inspired multi-agent reinforcement learning (QiMARL) model, utilizing quantum parallelism to achieve learning efficiency and scalability improvement. The QiMARL model is tested on an energy distribution task, which optimizes power distribution between generating and demanding nodal power stations. We compare the convergence time, reward performance, and scalability of QiMARL with traditional Multi-Armed Bandit (MAB) and Multi-Agent Reinforcement Learning methods, such as Greedy, Upper Confidence Bound (UCB), Thompson Sampling, MADDPG, QMIX, and PPO methods with a comprehensive ablation study. Our findings show that QiMARL yields better performance in high-dimensional systems, decreasing the number of training epochs needed for convergence while enhancing overall reward maximization. We also compare the algorithm’s computational complexity, indicating that QiMARL is more scalable to high-dimensional quantum environments. This research opens the door to future studies of quantum-enhanced reinforcement learning (RL) with potential applications to energy optimization, traffic management, and other multi-agent coordination problems.