Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2412.06938 · Machine Learning Science and Technology

Optimising entanglement distribution policies under classical communication constraints assisted by reinforcement learning

Jan Li, Tim J. Coopmans, Patrick Emonts, Kenneth Goodenough, Jordi Tura, Evert van Nieuwenburg

Published Jul 11, 2025• 1 citation
arXivHTMLPDF

Abstract

Abstract Quantum repeaters play a crucial role in the effective distribution of entanglement over long distances. The nearest-future type of quantum repeater requires two operations: entanglement generation across neighbouring repeaters and entanglement swapping to promote short-range entanglement to long-range. For many hardware setups, these actions are probabilistic, leading to longer distribution times and incurred errors. Significant efforts have been vested in finding the optimal entanglement-distribution policy, i.e. the protocol specifying when a network node needs to generate or swap entanglement, such that the expected time to distribute long-distance entanglement is minimal. This problem is even more intricate in more realistic scenarios, especially when classical communication delays are taken into account. In this work, we formulate our problem as a Markov decision problem and use reinforcement learning (RL) to optimise over centralised strategies, where one designated node instructs other nodes which actions to perform. Contrary to most RL models, ours can be readily interpreted. Additionally, we introduce and evaluate a fixed local policy, the ‘predictive swap-asap’ policy, where nodes only coordinate with nearest neighbours. Compared to the straightforward generalisation of the common swap-asap policy to the scenario with classical communication effects, the ‘wait-for-broadcast swap-asap’ policy, both of the aforementioned entanglement-delivery policies are faster at high success probabilities. Our work showcases the merit of considering policies acting with incomplete information in the realistic case when classical communication effects are significant.

DOI
10.1088/2632-2153/adeefa

Related papers

Optimal entanglement distribution policies in homogeneous repeater chains with cutoffs

Álvaro G. Iñesta, Gayane Vardoyan, Lara Scavuzzo +1 more

We study the limits of bipartite entanglement distribution using a chain of quantum repeaters that have quantum memories. To generate end-to-end entanglement, each node can attempt the generation of an entangled link with a neighbor, or perform an entanglement swapping measurement. A maximum storage time, known as cutoff, is enforced on the memories to ensure high-quality entanglement. Nodes follow a policy that determines when to perform each operation. Global-knowledge policies take into account all the information about the entanglement already produced. Here, we find global-knowledge policies that minimize the expected time to produce end-to-end entanglement. Our methods are based on Markov decision processes and value and policy iteration. We compare optimal policies to a policy in which nodes only use local information. We find that the advantage in expected delivery time provided by an optimal global-knowledge policy increases with increasing number of nodes and decreasing probability of successful swapping.

arXiv:2207.06533•npj Quantum Information•May 6, 2023•54 citations

Reinforcement Learning for Entanglement Swapping in Quantum Networks

Álvaro Troyano Olivas, Andrés Agustí Casado, Marco Pérez González +4 more

Current approaches for entanglement distribution networks face a largely untackled problem: precise timing and management of entanglement generation and swapping attempts is key in order to make entanglement distribution between any two nodes.To address this, we model a quantum network as two graphs, one containing the telecom fibers enabling physically the entanglement, and another containing the Bell pairs stored in its memories. Then, efficient entanglement distribution becomes a task of populating a mix of both graphs with a connection between one Alice and one Bob subject to constraints. Here, machine learning techniques are useful to find efficient policies through reinforcement learning. This way, we manage to consider inhomogeneous probabilities of generation and swapping, as well as complex networks beyond the chain scenario.

International Conference on Quantum Communications, Networking, and Computing (QCNC)•Mar 31, 2025•3 citations

Learning and interpreting policies for simultaneous entanglement requests in quantum networks

Leon Rode, Sumeet Khatri, Supartha Podder

Future quantum networks will make use of entanglement to perform numerous tasks, such as sending quantum information over long distances, distributed quantum computing, and quantum sensing. In general, these tasks will need to be performed simultaneously in various regions of a network, while minimizing resources and latency. We will thus require policies for scheduling link-level entanglement resources, and using the link-level entanglement to create various forms of multipartite entanglement required for every task. In this work, we address this problem using reinforcement learning. We formulate a Markov Decision Process for the problem and use double deep Q-networks (DQN) with Message Passing Neural Networks (MPNNs), experience replay buffers, and curriculum training to obtain policies. The key physical parameter is the probability of link-level entanglement generation, i.e., the link activation probability. We show that our policies maintain 100% success for up to 71% lower link activation probability than the baseline heuristics for a set of physically relevant network topologies. We then examine an additional constraint where experiment (task) placements are restricted to specific hardware types and demonstrate a similar advantage in performance over heuristics, with our policy maintaining at least an 80% success rate for up to a 59% lower link activation probability. Finally, we explore methods to interpret the learned policy by defining metrics enabling conclusions to be drawn about the model's behavior and by tasking a large language model (LLM) to derive a novel heuristic given example actions taken by the DQN-trained policy. We find that the LLM heuristic performs similarly to the DQN-trained policy in performance, indicating a promising method for interpretable policy extraction for large quantum networks, where direct training becomes computationally expensive.

quant-pharXiv:2609.30157•Sep 25, 2026•0 citations

Fast and reliable entanglement distribution with quantum repeaters: Principles for improving protocols using reinforcement learning

Stav Haldar, Pratik J. Barge, Sumeet Khatri +1 more

Tomorrow's quantum technologies for communication, sensing, and distributed computing will rely on networks with entanglement shared between spatially separated nodes. The authors provide improved protocols and policies for entanglement distribution along a chain of nodes, accounting for practical limitations such as photon losses, nonideal measurements, and quantum memories with short coherence times. These policies feature dynamic, state-dependent memory cutoffs and collaboration between nodes, all of which are quantified. Nesting policies for small repeater chains yields policies for large chains that improve upon a swap-as-soon-as-possible approach, and thus pave the way to scaling up.

arXiv:2303.00777•Physical Review Applied•Feb 21, 2024•28 citations

Policies for elementary links in a quantum network

Sumeet Khatri

Distributing entanglement over long distances is one of the central tasks in quantum networks. An important problem, especially for near-term quantum networks, is to develop optimal entanglement distribution protocols that take into account the limitations of current and near-term hardware, such as quantum memories with limited coherence time. We address this problem by initiating the study of quantum network protocols for entanglement distribution using the theory of decision processes, such that optimal protocols (referred to as policies in the context of decision processes) can be found using dynamic programming or reinforcement learning algorithms. As a first step, in this work we focus exclusively on the elementary link level. We start by defining a quantum decision process for elementary links, along with figures of merit for evaluating policies. We then provide two algorithms for determining policies, one of which we prove to be optimal (with respect to fidelity and success probability) among all policies. Then we show that the previously-studied memory-cutoff protocol can be phrased as a policy within our decision process framework, allowing us to obtain several new fundamental results about it. The conceptual developments and results of this work pave the way for the systematic study of the fundamental limitations of near-term quantum networks, and the requirements for physically realizing them.

arXiv:2007.03193•Quantum•Sep 7, 2021•39 citations

Optimal Entanglement Swapping in Quantum Repeaters

Evgeny V Shchukin, Peter van Loock

We formulate the problem of finding the optimal entanglement swapping scheme in a quantum repeater chain as a Markov decision process and present its solution for different repeaters' sizes. Based on this, we are able to demonstrate that the commonly used "doubling" scheme for performing probabilistic entanglement swapping of probabilistically distributed entangled qubit pairs in quantum repeaters does not always produce the best possible raw rate. Focusing on this figure of merit, without considering additional probabilistic elements for error suppression such as entanglement distillation on higher "nesting levels," our approach reveals that a power-of-two number of segments has no privileged position in quantum repeater theory; the best scheme can be constructed for any number of segments. Moreover, classical communication can be included into our scheme, and we show how this influences the raw waiting time for different numbers of segments, confirming again the optimality of "nondoubling" in some relevant parameter regimes. Thus, our approach provides the minimal possible waiting time of quantum repeaters in a fairly general physical setting.

arXiv:2109.00793•Physical Review Letters•Apr 13, 2022•63 citations
Abstract Quantum repeaters play a crucial role in the effective distribution of entanglement over long distances. The nearest-future type of quantum repeater requires two operations: entanglement generation across neighbouring repeaters and entanglement swapping to promote short-range entanglement to long-range. For many hardware setups, these actions are probabilistic, leading to longer distribution times and incurred errors. Significant efforts have been vested in finding the optimal entanglement-distribution policy, i.e. the protocol specifying when a network node needs to generate or swap entanglement, such that the expected time to distribute long-distance entanglement is minimal. This problem is even more intricate in more realistic scenarios, especially when classical communication delays are taken into account. In this work, we formulate our problem as a Markov decision problem and use reinforcement learning (RL) to optimise over centralised strategies, where one designated node instructs other nodes which actions to perform. Contrary to most RL models, ours can be readily interpreted. Additionally, we introduce and evaluate a fixed local policy, the ‘predictive swap-asap’ policy, where nodes only coordinate with nearest neighbours. Compared to the straightforward generalisation of the common swap-asap policy to the scenario with classical communication effects, the ‘wait-for-broadcast swap-asap’ policy, both of the aforementioned entanglement-delivery policies are faster at high success probabilities. Our work showcases the merit of considering policies acting with incomplete information in the realistic case when classical communication effects are significant.
We study the limits of bipartite entanglement distribution using a chain of quantum repeaters that have quantum memories. To generate end-to-end entanglement, each node can attempt the generation of an entangled link with a neighbor, or perform an entanglement swapping measurement. A maximum storage time, known as cutoff, is enforced on the memories to ensure high-quality entanglement. Nodes follow a policy that determines when to perform each operation. Global-knowledge policies take into account all the information about the entanglement already produced. Here, we find global-knowledge policies that minimize the expected time to produce end-to-end entanglement. Our methods are based on Markov decision processes and value and policy iteration. We compare optimal policies to a policy in which nodes only use local information. We find that the advantage in expected delivery time provided by an optimal global-knowledge policy increases with increasing number of nodes and decreasing probability of successful swapping.
Current approaches for entanglement distribution networks face a largely untackled problem: precise timing and management of entanglement generation and swapping attempts is key in order to make entanglement distribution between any two nodes.To address this, we model a quantum network as two graphs, one containing the telecom fibers enabling physically the entanglement, and another containing the Bell pairs stored in its memories. Then, efficient entanglement distribution becomes a task of populating a mix of both graphs with a connection between one Alice and one Bob subject to constraints. Here, machine learning techniques are useful to find efficient policies through reinforcement learning. This way, we manage to consider inhomogeneous probabilities of generation and swapping, as well as complex networks beyond the chain scenario.
Future quantum networks will make use of entanglement to perform numerous tasks, such as sending quantum information over long distances, distributed quantum computing, and quantum sensing. In general, these tasks will need to be performed simultaneously in various regions of a network, while minimizing resources and latency. We will thus require policies for scheduling link-level entanglement resources, and using the link-level entanglement to create various forms of multipartite entanglement required for every task. In this work, we address this problem using reinforcement learning. We formulate a Markov Decision Process for the problem and use double deep Q-networks (DQN) with Message Passing Neural Networks (MPNNs), experience replay buffers, and curriculum training to obtain policies. The key physical parameter is the probability of link-level entanglement generation, i.e., the link activation probability. We show that our policies maintain 100% success for up to 71% lower link activation probability than the baseline heuristics for a set of physically relevant network topologies. We then examine an additional constraint where experiment (task) placements are restricted to specific hardware types and demonstrate a similar advantage in performance over heuristics, with our policy maintaining at least an 80% success rate for up to a 59% lower link activation probability. Finally, we explore methods to interpret the learned policy by defining metrics enabling conclusions to be drawn about the model's behavior and by tasking a large language model (LLM) to derive a novel heuristic given example actions taken by the DQN-trained policy. We find that the LLM heuristic performs similarly to the DQN-trained policy in performance, indicating a promising method for interpretable policy extraction for large quantum networks, where direct training becomes computationally expensive.
Tomorrow's quantum technologies for communication, sensing, and distributed computing will rely on networks with entanglement shared between spatially separated nodes. The authors provide improved protocols and policies for entanglement distribution along a chain of nodes, accounting for practical limitations such as photon losses, nonideal measurements, and quantum memories with short coherence times. These policies feature dynamic, state-dependent memory cutoffs and collaboration between nodes, all of which are quantified. Nesting policies for small repeater chains yields policies for large chains that improve upon a swap-as-soon-as-possible approach, and thus pave the way to scaling up.
Distributing entanglement over long distances is one of the central tasks in quantum networks. An important problem, especially for near-term quantum networks, is to develop optimal entanglement distribution protocols that take into account the limitations of current and near-term hardware, such as quantum memories with limited coherence time. We address this problem by initiating the study of quantum network protocols for entanglement distribution using the theory of decision processes, such that optimal protocols (referred to as policies in the context of decision processes) can be found using dynamic programming or reinforcement learning algorithms. As a first step, in this work we focus exclusively on the elementary link level. We start by defining a quantum decision process for elementary links, along with figures of merit for evaluating policies. We then provide two algorithms for determining policies, one of which we prove to be optimal (with respect to fidelity and success probability) among all policies. Then we show that the previously-studied memory-cutoff protocol can be phrased as a policy within our decision process framework, allowing us to obtain several new fundamental results about it. The conceptual developments and results of this work pave the way for the systematic study of the fundamental limitations of near-term quantum networks, and the requirements for physically realizing them.
We formulate the problem of finding the optimal entanglement swapping scheme in a quantum repeater chain as a Markov decision process and present its solution for different repeaters' sizes. Based on this, we are able to demonstrate that the commonly used "doubling" scheme for performing probabilistic entanglement swapping of probabilistically distributed entangled qubit pairs in quantum repeaters does not always produce the best possible raw rate. Focusing on this figure of merit, without considering additional probabilistic elements for error suppression such as entanglement distillation on higher "nesting levels," our approach reveals that a power-of-two number of segments has no privileged position in quantum repeater theory; the best scheme can be constructed for any number of segments. Moreover, classical communication can be included into our scheme, and we show how this influences the raw waiting time for different numbers of segments, confirming again the optimality of "nondoubling" in some relevant parameter regimes. Thus, our approach provides the minimal possible waiting time of quantum repeaters in a fairly general physical setting.