Future quantum networks will make use of entanglement to perform numerous tasks, such as sending quantum information over long distances, distributed quantum computing, and quantum sensing. In general, these tasks will need to be performed simultaneously in various regions of a network, while minimizing resources and latency. We will thus require policies for scheduling link-level entanglement resources, and using the link-level entanglement to create various forms of multipartite entanglement required for every task. In this work, we address this problem using reinforcement learning. We formulate a Markov Decision Process for the problem and use double deep Q-networks (DQN) with Message Passing Neural Networks (MPNNs), experience replay buffers, and curriculum training to obtain policies. The key physical parameter is the probability of link-level entanglement generation, i.e., the link activation probability. We show that our policies maintain 100% success for up to 71% lower link activation probability than the baseline heuristics for a set of physically relevant network topologies. We then examine an additional constraint where experiment (task) placements are restricted to specific hardware types and demonstrate a similar advantage in performance over heuristics, with our policy maintaining at least an 80% success rate for up to a 59% lower link activation probability. Finally, we explore methods to interpret the learned policy by defining metrics enabling conclusions to be drawn about the model's behavior and by tasking a large language model (LLM) to derive a novel heuristic given example actions taken by the DQN-trained policy. We find that the LLM heuristic performs similarly to the DQN-trained policy in performance, indicating a promising method for interpretable policy extraction for large quantum networks, where direct training becomes computationally expensive.
Future quantum networks will make use of entanglement to perform numerous tasks, such as sending quantum information over long distances, distributed quantum computing, and quantum sensing. In general, these tasks will need to be performed simultaneously in various regions of a network, while minimizing resources and latency. We will thus require policies for scheduling link-level entanglement resources, and using the link-level entanglement to create various forms of multipartite entanglement required for every task. In this work, we address this problem using reinforcement learning. We formulate a Markov Decision Process for the problem and use double deep Q-networks (DQN) with Message Passing Neural Networks (MPNNs), experience replay buffers, and curriculum training to obtain policies. The key physical parameter is the probability of link-level entanglement generation, i.e., the link activation probability. We show that our policies maintain 100% success for up to 71% lower link activation probability than the baseline heuristics for a set of physically relevant network topologies. We then examine an additional constraint where experiment (task) placements are restricted to specific hardware types and demonstrate a similar advantage in performance over heuristics, with our policy maintaining at least an 80% success rate for up to a 59% lower link activation probability. Finally, we explore methods to interpret the learned policy by defining metrics enabling conclusions to be drawn about the model's behavior and by tasking a large language model (LLM) to derive a novel heuristic given example actions taken by the DQN-trained policy. We find that the LLM heuristic performs similarly to the DQN-trained policy in performance, indicating a promising method for interpretable policy extraction for large quantum networks, where direct training becomes computationally expensive.
Abstract Quantum repeaters play a crucial role in the effective distribution of entanglement over long distances. The nearest-future type of quantum repeater requires two operations: entanglement generation across neighbouring repeaters and entanglement swapping to promote short-range entanglement to long-range. For many hardware setups, these actions are probabilistic, leading to longer distribution times and incurred errors. Significant efforts have been vested in finding the optimal entanglement-distribution policy, i.e. the protocol specifying when a network node needs to generate or swap entanglement, such that the expected time to distribute long-distance entanglement is minimal. This problem is even more intricate in more realistic scenarios, especially when classical communication delays are taken into account. In this work, we formulate our problem as a Markov decision problem and use reinforcement learning (RL) to optimise over centralised strategies, where one designated node instructs other nodes which actions to perform. Contrary to most RL models, ours can be readily interpreted. Additionally, we introduce and evaluate a fixed local policy, the ‘predictive swap-asap’ policy, where nodes only coordinate with nearest neighbours. Compared to the straightforward generalisation of the common swap-asap policy to the scenario with classical communication effects, the ‘wait-for-broadcast swap-asap’ policy, both of the aforementioned entanglement-delivery policies are faster at high success probabilities. Our work showcases the merit of considering policies acting with incomplete information in the realistic case when classical communication effects are significant.
Efficient routing in entanglement distribution networks is a challenging task that requires the coordinated allocation of quantum memories, repeaters, and quantum operations such as entanglement swapping and distillation across dynamic network conditions. In this paper, we present a Deep Q-Learning (DQN) Two-Stage Routing strategy designed to improve both path and quantum operations selection. Our approach enables the agent to make decisions over a diverse action space, including request scheduling, multipath routing, and the strategic application of quantum operations to maximize entanglement success and resource efficiency. We further extend DQN to integrate Double DQN, Dueling DQN, and Prioritized Experience Replay (PER) architectures. Simulation results indicate that our approach significantly outperforms baseline methods regarding request success rate, especially in large-scale networks under strict fidelity requirements.
Entanglement generation in long-distance quantum networks is challenging because resources are limited and entanglement swapping is probabilistic. To maximize the rate of successful requests, existing quantum routing algorithms often rely on computationally expensive methods such as Integer Linear Programming (ILP) to determine which links to entangle and use for end-to-end entanglement generation. However, these approaches fail to meet the latency requirements of real-world quantum networks. In this study, we propose a Reinforcement Learning (RL)-based model that determines which links to entangle in each time slot, replacing the slow ILP-based link-selection phase used in prior algorithms. The proposed Deep Q-learning model is up to 19.2× faster than linear programming in link-selection while maintaining comparable routing performance. The RL link-selection model alone matches ILP in request success rate; combining RL link selection with entanglement caching and proactive swapping raises throughput by up to 52.55% over ILP. Overall, our approach achieves success rates exceeding state-of-the-art solutions while reducing execution time by more than an order of magnitude. Experiments on a synthetic 50-node Waxman topology and the real-world 54-node SURFnet core topology confirm that these gains generalize across network structures, providing a practical path toward scalable quantum routing.
This paper investigates resource allocation for entanglement distribution in multi-node, multi-channel quantum networks at the metropolitan scale. A multi-slot quantum network simulation framework is developed across physical and network layers, incorporating heterogeneous link characteristics, limited quantum memories, request queuing, retry mechanisms, and concurrent entanglement distribution. Based on this framework, a centralized scheduling architecture is proposed to coordinate quantum memories, communication channels, and routing paths for multiple simultaneous entanglement requests. Three classes of resource allocation strategies are evaluated: heuristic approaches, a mixed-integer linear programming (MILP) optimization method, and a Proximal Policy Optimization (PPO)-based reinforcement learning approach. The heuristic schemes reveal fundamental trade-offs between request delay and entanglement success rate: Dynamic Efficient minimizes delay, Success Enhancement improves success probability through adaptive multi-path allocation, and Static Efficient provides a balance between these objectives. The MILP approach achieves optimized resource allocation by jointly considering request handling and multi-path assignment, while the PPO-based method learns adaptive scheduling policies to improve overall performance. Simulation results demonstrate that these approaches provide different performance advantages in terms of request delay, entanglement success rate, capacity utilization, and request handling rate, highlighting the trade-offs between efficiency and reliability in metropolitan-scale quantum network scheduling.
Protocols for distributed quantum systems commonly require the simultaneous availability of n entangled states, each with a fidelity above some fixed minimum Fapp relative to the target maximally-entangled state. However, the fidelity of entangled states degrades over time while in memory. Entangled states are therefore rendered useless when their fidelity falls below Fapp. This is problematic when entanglement generation is probabilistic and attempted in a sequential manner, because the expected completion time until n entangled states are available can be large. Motivated by existing entanglement generation schemes, we consider a system where the entanglement generation parameters (the success probability p and fidelity F of the generated entangled state) may be adjusted at each time step. We model the system as a Markov decision process, where the policy dictates which generation parameters (p,F) to use for each attempt. We use dynamic programming to derive optimal policies that minimise the expected time until n entangled states are available with fidelity greater than Fapp. We observe that the advantage of our optimal policies over the selected baselines increases significantly with n. In the parameter regimes explored, which are based closely on current experiments, we find that the optimal policy can provide a speed-up of as much as a factor of twenty over a constant-action policy. In addition, we propose a computationally inexpensive heuristic method to compute policies that perform either optimally or near-optimally in the parameter regimes explored. Our heuristic method can be used to find high-performing policies in parameter regimes where finding an optimal policy is intractable.
Efficient routing in entanglement distribution networks is a challenging task that requires the coordinated allocation of quantum memories, repeaters, and quantum operations such as entanglement swapping and distillation across dynamic network conditions. In this paper, we present a Deep Q-Learning (DQN) Two-Stage Routing strategy designed to improve bothpath and quantum operations selection. Our approach enables the agent to make decisions over a diverse action space, including request scheduling, multipath routing, and the strategic application of quantum operations to maximize entanglementsuccess and resource efficiency. We further extend DQN to integrate Double DQN, Dueling DQN, and Prioritized ExperienceReplay (PER) architectures. Simulation results indicate that our approach significantly outperforms baseline methods regardingrequest success rate, especially in large-scale networks under strict fidelity requirements.