Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2601.18419

Emergent Cooperation in Quantum Multi-Agent Reinforcement Learning Using Communication

Michael Kölle, Christian Reff, Leo Sünkel, Julian Hager, Gerhard Stenzel, Claudia Linnhoff-Popien

quant-phcs.AIcs.LGcs.MASubmitted Jan 26, 2026• Announced Jan 27, 2026
arXivHTMLPDF

Abstract

Emergent cooperation in classical Multi-Agent Reinforcement Learning has gained significant attention, particularly in the context of Sequential Social Dilemmas (SSDs). While classical reinforcement learning approaches have demonstrated capability for emergent cooperation, research on extending these methods to Quantum Multi-Agent Reinforcement Learning remains limited, particularly through communication. In this paper, we apply communication approaches to quantum Q-Learning agents: the Mutual Acknowledgment Token Exchange (MATE) protocol, its extension Mutually Endorsed Distributed Incentive Acknowledgment Token Exchange (MEDIATE), the peer rewarding mechanism Gifting, and Reinforced Inter-Agent Learning (RIAL). We evaluate these approaches in three SSDs: the Iterated Prisoner's Dilemma, Iterated Stag Hunt, and Iterated Game of Chicken. Our experimental results show that approaches using MATE with temporal-difference measure (MATE\textsubscript{TD}), AutoMATE, MEDIATE-I, and MEDIATE-S achieved high cooperation levels across all dilemmas, demonstrating that communication is a viable mechanism for fostering emergent cooperation in Quantum Multi-Agent Reinforcement Learning.

Comments
Accepted at IEEE ICC 2026

Related papers

eQMARL: Entangled Quantum Multi-Agent Reinforcement Learning for Distributed Cooperation over Quantum Channels

Alexander DeRieux, Walid Saad

Collaboration is a key challenge in distributed multi-agent reinforcement learning (MARL) environments. Learning frameworks for these decentralized systems must weigh the benefits of explicit player coordination against the communication overhead and computational cost of sharing local observations and environmental data. Quantum computing has sparked a potential synergy between quantum entanglement and cooperation in multi-agent environments, which could enable more efficient distributed collaboration with minimal information sharing. This relationship is largely unexplored, however, as current state-of-the-art quantum MARL (QMARL) implementations rely on classical information sharing rather than entanglement over a quantum channel as a coordination medium. In contrast, in this paper, a novel framework dubbed entangled QMARL (eQMARL) is proposed. The proposed eQMARL is a distributed actor-critic framework that facilitates cooperation over a quantum channel and eliminates local observation sharing via a quantum entangled split critic. Introducing a quantum critic uniquely spread across the agents allows coupling of local observation encoders through entangled input qubits over a quantum channel, which requires no explicit sharing of local observations and reduces classical communication overhead. Further, agent policies are tuned through joint observation-value function estimation via joint quantum measurements, thereby reducing the centralized computational burden. Experimental results show that eQMARL with $Ψ^{+}$ entanglement converges to a cooperative strategy up to $17.8\%$ faster and with a higher overall score compared to split classical and fully centralized classical and quantum baselines. The results also show that eQMARL achieves this performance with a constant factor of $25$-times fewer centralized parameters compared to the split classical baseline.

arXiv:2405.17486•May 24, 2024•0 citations

Quantum Advantage in Multi Agent Reinforcement Learning

Simranjeet Singh Dahia, Claudia Szabo

We present an empirical evaluation of quantum entanglement in agent coordination within quantum multi agent reinforcement learning (QMARL). While QMARL has attracted growing interest recently, most prior work evaluates quantum policies without provable baselines, making it impossible to rigorously distinguish quantum advantage from algorithmic coincidence. We address this directly by evaluating a decentralized QMARL framework with variational quantum circuit (VQC) actors with shared entangled states. In the CHSH game, which has a mathematically proven classical performance ceiling of 0.75 win rate, we show that entangled QMARL agents approach the Tsirelson limit of 0.854, providing clear evidence of their quantum advantage. We show that unentangled quantum circuits match the classical baseline, confirming that entanglement and not the quantum circuit itself is the active coordination mechanism. We also explore the effect of specific entanglement structures, as some Bell states enable coordination gains while others actively harm performance. On cooperative navigation (CoopNav), QMARL without entanglement achieves $\sim2\times$ improvement in success rate over classical MAA2C ($\sim$0.85 versus $\sim$0.40), with the hybrid configuration, quantum actor paired with a classical centralised critic, outperforming both fully classical and fully quantum solutions. We present our experimental analysis and discuss future work.

cs.LGCross-listedarXiv:2605.14235•May 15, 2026•0 citations

Learning to Coordinate via Quantum Entanglement in Multi-Agent Reinforcement Learning

John Gardiner, Orlando Romero, Brendan Tivnan +2 more

The inability to communicate poses a major challenge to coordination in multi-agent reinforcement learning (MARL). Prior work has explored correlating local policies via shared randomness, sometimes in the form of a correlation device, as a mechanism to assist in decentralized decision-making. In contrast, this work introduces the first framework for training MARL agents to exploit shared quantum entanglement as a coordination resource, which permits a larger class of communication-free correlated policies than shared randomness alone. This is motivated by well-known results in quantum physics which posit that, for certain single-round cooperative games with no communication, shared quantum entanglement enables strategies that outperform those that only use shared randomness. In such cases, we say that there is quantum advantage. Our framework is based on a novel differentiable policy parameterization that enables optimization over quantum measurements, together with a novel policy architecture that decomposes joint policies into a quantum coordinator and decentralized local actors. To illustrate the effectiveness of our proposed method, we first show that we can learn, purely from experience, strategies that attain quantum advantage in single-round games that are treated as black box oracles. We then demonstrate how our machinery can learn policies with quantum advantage in an illustrative multi-agent sequential decision-making problem formulated as a decentralized partially observable Markov decision process (Dec-POMDP).

cs.MACross-listedarXiv:2602.08965•Feb 10, 2026•0 citations

General Analysis of Incentive Mechanisms for Peer-to-Peer Transmissions: A Quantum Game Perspective

Weiman Sun, Shengling Wang

The peer-to-peer transmission is a mainstream in challenged network environments. Yet, the free rider phenomenon in peer-to peer transmissions presses a need for incentive mechanisms to stimulate contributions of data transmission. As a result, it is imperative to answer the questions: whether and to what extent an incentive mechanism can invoke such contributions? To answer these questions, we employ an n-player continuous quantum game model to analyze extrinsic incentive mechanisms (promoting cooperative behaviors by offering rewards), and use the quantum prisoner's dilemma model to analyze intrinsic incentive mechanisms (encouraging reciprocal cooperation by exploiting internal bounds). To the best of our knowledge, we are the first to analyze incentive mechanisms for peer-to-peer transmissions from a quantum game perspective. Such a perspective is adopted because the extended strategy space in the quantum game broadens the range for searching optimal strategies and the introduction of entanglement makes the proposed analytical frameworks more practical due to the consideration of the peers' relationships in decision-making. Our proposed quantum game-based analytical frameworks are generic because they are compatible with classic game-based schemes. Our analytical results can provide straightforward insights on evaluating the potential of incentive mechanisms and can serve as important references for designing new incentive mechanisms.

IEEE International Conference on Distributed Computing Systems (ICDCS)•Jun 1, 2017•4 citations

Cooperative multi-agent reinforcement learning based on quantum computing

Xiangping Meng

Due to the interactions among the agents in the cooperative multi-agent systems,multi-agent learning problem complexity can rise rapidly with the number of agents or their behavioral sophistication.In order to converge to desirable equilibrium,agents generally require sufficient exploration of strategy space and coordinate their policies to achieve optimal equilibrium.A novel cooperative multi-agent learning method is proposed based on quantum theory and reinforcement learning.This method not only coordinates agents’ behaviors using quantum entanglement and helps agents make action selection under quantum superposition,but also adopts Grover’s searching algorithm which can speed up learning.This method also makes a good tradeoff between exploration and exploitation using probability characteristics of quantum theory.The results of simulated experiments show that quantum theory can be effectively used to cooperative multi-agent reinforcement learning.

Computer Engineering and Applications Journal•Jan 1, 2008•0 citations

Quantum Multi-Agent Reinforcement Learning as an Emerging AI Technology: A Survey and Future Directions

Wenhan Yu, Jun Zhao

This paper surveys Quantum Multi-Agent Reinforcement Learning (QMARL), an emerging fusion of quantum computing and multi-agent systems. It begins by introducing the transformative potential of quantum computing in computational capabilities. Examining the principles of multi-agent reinforcement learning (MARL), the paper explores how quantum computing enhances learning efficiency and decision-making. Focusing on the current state of QMARL, it reviews literature, methodologies, and case studies showcasing the integration of quantum algorithms with MARL frameworks. The survey addresses challenges and opportunities arising from quantum technologies in multi-agent systems, such as entanglement and superposition, exploring their implications for agent coordination and learning dynamics. Practical applications in domains like cybersecurity and finance underscore QMARL’s transformative potential. Concluding, the paper identifies research gaps and proposes future directions, emphasizing the need for scalable quantum algorithms, exploring quantum-resistant strategies in adversarial settings, and integrating quantum principles in agent communication and collaboration. This concise survey serves as a foundational guide for researchers and practitioners, offering insights into the current state and future possibilities of QMARL.

International Conference on Computer and Applications (ICCA)•Nov 28, 2023•6 citations
Emergent cooperation in classical Multi-Agent Reinforcement Learning has gained significant attention, particularly in the context of Sequential Social Dilemmas (SSDs). While classical reinforcement learning approaches have demonstrated capability for emergent cooperation, research on extending these methods to Quantum Multi-Agent Reinforcement Learning remains limited, particularly through communication. In this paper, we apply communication approaches to quantum Q-Learning agents: the Mutual Acknowledgment Token Exchange (MATE) protocol, its extension Mutually Endorsed Distributed Incentive Acknowledgment Token Exchange (MEDIATE), the peer rewarding mechanism Gifting, and Reinforced Inter-Agent Learning (RIAL). We evaluate these approaches in three SSDs: the Iterated Prisoner's Dilemma, Iterated Stag Hunt, and Iterated Game of Chicken. Our experimental results show that approaches using MATE with temporal-difference measure (MATE\textsubscript{TD}), AutoMATE, MEDIATE-I, and MEDIATE-S achieved high cooperation levels across all dilemmas, demonstrating that communication is a viable mechanism for fostering emergent cooperation in Quantum Multi-Agent Reinforcement Learning.
Collaboration is a key challenge in distributed multi-agent reinforcement learning (MARL) environments. Learning frameworks for these decentralized systems must weigh the benefits of explicit player coordination against the communication overhead and computational cost of sharing local observations and environmental data. Quantum computing has sparked a potential synergy between quantum entanglement and cooperation in multi-agent environments, which could enable more efficient distributed collaboration with minimal information sharing. This relationship is largely unexplored, however, as current state-of-the-art quantum MARL (QMARL) implementations rely on classical information sharing rather than entanglement over a quantum channel as a coordination medium. In contrast, in this paper, a novel framework dubbed entangled QMARL (eQMARL) is proposed. The proposed eQMARL is a distributed actor-critic framework that facilitates cooperation over a quantum channel and eliminates local observation sharing via a quantum entangled split critic. Introducing a quantum critic uniquely spread across the agents allows coupling of local observation encoders through entangled input qubits over a quantum channel, which requires no explicit sharing of local observations and reduces classical communication overhead. Further, agent policies are tuned through joint observation-value function estimation via joint quantum measurements, thereby reducing the centralized computational burden. Experimental results show that eQMARL with Ψ+Ψ^{+}Ψ+ entanglement converges to a cooperative strategy up to 17.8%17.8\%17.8% faster and with a higher overall score compared to split classical and fully centralized classical and quantum baselines. The results also show that eQMARL achieves this performance with a constant factor of 252525-times fewer centralized parameters compared to the split classical baseline.
We present an empirical evaluation of quantum entanglement in agent coordination within quantum multi agent reinforcement learning (QMARL). While QMARL has attracted growing interest recently, most prior work evaluates quantum policies without provable baselines, making it impossible to rigorously distinguish quantum advantage from algorithmic coincidence. We address this directly by evaluating a decentralized QMARL framework with variational quantum circuit (VQC) actors with shared entangled states. In the CHSH game, which has a mathematically proven classical performance ceiling of 0.75 win rate, we show that entangled QMARL agents approach the Tsirelson limit of 0.854, providing clear evidence of their quantum advantage. We show that unentangled quantum circuits match the classical baseline, confirming that entanglement and not the quantum circuit itself is the active coordination mechanism. We also explore the effect of specific entanglement structures, as some Bell states enable coordination gains while others actively harm performance. On cooperative navigation (CoopNav), QMARL without entanglement achieves ∼2×\sim2\times∼2× improvement in success rate over classical MAA2C ($\sim$0.85 versus $\sim$0.40), with the hybrid configuration, quantum actor paired with a classical centralised critic, outperforming both fully classical and fully quantum solutions. We present our experimental analysis and discuss future work.
The inability to communicate poses a major challenge to coordination in multi-agent reinforcement learning (MARL). Prior work has explored correlating local policies via shared randomness, sometimes in the form of a correlation device, as a mechanism to assist in decentralized decision-making. In contrast, this work introduces the first framework for training MARL agents to exploit shared quantum entanglement as a coordination resource, which permits a larger class of communication-free correlated policies than shared randomness alone. This is motivated by well-known results in quantum physics which posit that, for certain single-round cooperative games with no communication, shared quantum entanglement enables strategies that outperform those that only use shared randomness. In such cases, we say that there is quantum advantage. Our framework is based on a novel differentiable policy parameterization that enables optimization over quantum measurements, together with a novel policy architecture that decomposes joint policies into a quantum coordinator and decentralized local actors. To illustrate the effectiveness of our proposed method, we first show that we can learn, purely from experience, strategies that attain quantum advantage in single-round games that are treated as black box oracles. We then demonstrate how our machinery can learn policies with quantum advantage in an illustrative multi-agent sequential decision-making problem formulated as a decentralized partially observable Markov decision process (Dec-POMDP).
The peer-to-peer transmission is a mainstream in challenged network environments. Yet, the free rider phenomenon in peer-to peer transmissions presses a need for incentive mechanisms to stimulate contributions of data transmission. As a result, it is imperative to answer the questions: whether and to what extent an incentive mechanism can invoke such contributions? To answer these questions, we employ an n-player continuous quantum game model to analyze extrinsic incentive mechanisms (promoting cooperative behaviors by offering rewards), and use the quantum prisoner's dilemma model to analyze intrinsic incentive mechanisms (encouraging reciprocal cooperation by exploiting internal bounds). To the best of our knowledge, we are the first to analyze incentive mechanisms for peer-to-peer transmissions from a quantum game perspective. Such a perspective is adopted because the extended strategy space in the quantum game broadens the range for searching optimal strategies and the introduction of entanglement makes the proposed analytical frameworks more practical due to the consideration of the peers' relationships in decision-making. Our proposed quantum game-based analytical frameworks are generic because they are compatible with classic game-based schemes. Our analytical results can provide straightforward insights on evaluating the potential of incentive mechanisms and can serve as important references for designing new incentive mechanisms.
Due to the interactions among the agents in the cooperative multi-agent systems,multi-agent learning problem complexity can rise rapidly with the number of agents or their behavioral sophistication.In order to converge to desirable equilibrium,agents generally require sufficient exploration of strategy space and coordinate their policies to achieve optimal equilibrium.A novel cooperative multi-agent learning method is proposed based on quantum theory and reinforcement learning.This method not only coordinates agents’ behaviors using quantum entanglement and helps agents make action selection under quantum superposition,but also adopts Grover’s searching algorithm which can speed up learning.This method also makes a good tradeoff between exploration and exploitation using probability characteristics of quantum theory.The results of simulated experiments show that quantum theory can be effectively used to cooperative multi-agent reinforcement learning.
This paper surveys Quantum Multi-Agent Reinforcement Learning (QMARL), an emerging fusion of quantum computing and multi-agent systems. It begins by introducing the transformative potential of quantum computing in computational capabilities. Examining the principles of multi-agent reinforcement learning (MARL), the paper explores how quantum computing enhances learning efficiency and decision-making. Focusing on the current state of QMARL, it reviews literature, methodologies, and case studies showcasing the integration of quantum algorithms with MARL frameworks. The survey addresses challenges and opportunities arising from quantum technologies in multi-agent systems, such as entanglement and superposition, exploring their implications for agent coordination and learning dynamics. Practical applications in domains like cybersecurity and finance underscore QMARL’s transformative potential. Concluding, the paper identifies research gaps and proposes future directions, emphasizing the need for scalable quantum algorithms, exploring quantum-resistant strategies in adversarial settings, and integrating quantum principles in agent communication and collaboration. This concise survey serves as a foundational guide for researchers and practitioners, offering insights into the current state and future possibilities of QMARL.