We consider the problem of entanglement routing in heterogeneous quantum networks where each link and node may offer a different entanglement generation quality. We consider that each link offers a base rate and fidelity of entanglement generation. These base values can be altered by performing entanglement purification on each link. Furthermore, we consider that each repeater node offers a different probability of a successful entanglement swapping. With this setting, we design a reinforcement learning-based approach for flexible entanglement routing on future quantum networks. Specifically, we formulate the problem of entanglement routing in the Q-learning framework where the main objective of the agent is to deliver end-to-end entanglement while maximizing a user-specified objective function in the form of a weighted sum of achieve fidelity and rate. We demonstrate the efficacy and flexibility of developed framework by simulating random quantum networks where the random requests in fidelity maximizing, rate maximizing, or balanced mode are generated. We also numerically assess the effects of infrastructure developments and technology enhancements in future quantum networks.
We consider the problem of entanglement routing in heterogeneous quantum networks where each link and node may offer a different entanglement generation quality. We consider that each link offers a base rate and fidelity of entanglement generation. These base values can be altered by performing entanglement purification on each link. Furthermore, we consider that each repeater node offers a different probability of a successful entanglement swapping. With this setting, we design a reinforcement learning-based approach for flexible entanglement routing on future quantum networks. Specifically, we formulate the problem of entanglement routing in the Q-learning framework where the main objective of the agent is to deliver end-to-end entanglement while maximizing a user-specified objective function in the form of a weighted sum of achieve fidelity and rate. We demonstrate the efficacy and flexibility of developed framework by simulating random quantum networks where the random requests in fidelity maximizing, rate maximizing, or balanced mode are generated. We also numerically assess the effects of infrastructure developments and technology enhancements in future quantum networks.
Entanglement generation in long-distance quantum networks is a difficult process due to resource limitations and the probabilistic nature of entanglement swapping. To maximize success probability, existing quantum routing algorithms employ computationally expensive solutions (e.g., linear programming) to determine which links to entangle and use for end-to-end entanglement generation. Such optimization methods, however, cannot meet the delay requirements of real-world quantum networks, necessitating swift yet efficient real-time optimization models. In this paper, we propose reinforcement learning (RL)-based models to determine which links to entangle and proactively swap to meet connection requests. We show that the proposed RL-based approach is 20x faster compared to linear programming. Moreover, we show that one can take advantage of the longevity of entanglements to (i) cache entangled links for future use and (ii) proactively swap entanglement on high-demand path segments, thereby increasing the likelihood of request success. Through comprehensive simulations, we demonstrate that caching unused entanglements leads to a 10-15% improvement in the performance of state-of-the-art quantum routing algorithms. Complementing caching with proactive entanglement swapping further enhances the request success rate by up to 52.55%.
Entanglement generation and swapping is a difficult process due to probabilistic nature of quantum mechanics. To overcome this issue, existing quantum routing algorithms try to create entanglement on multiple paths between source and destination. Although it is possible to save entangled qubits on unused links using quantum memories, the quantum routing algorithms discard them and try creating new entanglement in each time slot. In this work, we leverage the longevity of entanglement and introduce two enhancements to improve the performance of existing routing algorithms: (i) The generation and caching of entanglements across multiple time slots, and (ii) the proactively executing entanglement swapping for frequently utilized links, increasing the success rate of future requests. Since entanglement caching reduces available quantum memory on quantum repeaters, we apply reinforcement learning to identify the optimal set of segments that are likely to be used in the future. Through comprehensive simulations, we demonstrate that caching unused entanglements lead to 10 - 15% improvement on the performance of quantum routing algorithms REPS and SEER. Complementing the caching with reinforcement learning based proactively entanglement swapping leads to further performance improvements. Specifically, using Q-Learning to decide on which segments to proactively swap leads to 52.55% and 41.79% improvement for REPS and SEER algorithms, respectively. Deep Q-Learning outperforms Q-Learning with 61.43% and 48.27% improvements rates.
Entanglement generation in long-distance quantum networks is challenging because resources are limited and entanglement swapping is probabilistic. To maximize the rate of successful requests, existing quantum routing algorithms often rely on computationally expensive methods such as Integer Linear Programming (ILP) to determine which links to entangle and use for end-to-end entanglement generation. However, these approaches fail to meet the latency requirements of real-world quantum networks. In this study, we propose a Reinforcement Learning (RL)-based model that determines which links to entangle in each time slot, replacing the slow ILP-based link-selection phase used in prior algorithms. The proposed Deep Q-learning model is up to 19.2× faster than linear programming in link-selection while maintaining comparable routing performance. The RL link-selection model alone matches ILP in request success rate; combining RL link selection with entanglement caching and proactive swapping raises throughput by up to 52.55% over ILP. Overall, our approach achieves success rates exceeding state-of-the-art solutions while reducing execution time by more than an order of magnitude. Experiments on a synthetic 50-node Waxman topology and the real-world 54-node SURFnet core topology confirm that these gains generalize across network structures, providing a practical path toward scalable quantum routing.
Quantum networks must deliver end-to-end entanglement whose fidelity meets application requirements, which typically requires entanglement purification along multi-hop paths. Existing scheduling approaches either purify at the link level or optimize end-to-end fidelity for a static set of requests, and do not capture concurrent requests competing for network resources over time. This paper studies concurrent entanglement purification scheduling in a time-slotted quantum network with heterogeneous links, in which requests compete for link capacity and quantum memories, and each request may be served by multiple parallel paths for increased reliability. We formulate a mixed-integer linear programming (MILP) scheduler that maximizes the number of scheduled requests, with max-min path allocation as a fairness criterion, and develop reinforcement learning (RL) schedulers based on DQN, PPO, and SAC, whose rewards balance request success against Bell-pair consumption. Simulations under varying network load, topologies and target fidelity thresholds show that the RL schedulers achieve the highest success rates, improving on MILP by up to 6% and on Progressive Filling and Greedy heuristics by 16.5% and 19.5% respectively, at the cost of higher Bell pair consumption and the deferral of a small fraction of requests. MILP, in contrast, achieves the highest fairness and the lowest Bell pair consumption in most scenarios.
Abstract Entanglement-based quantum networks require intelligent routing mechanisms that not only maximize fidelity and connectivity but also ensure service differentiation under varying traffic conditions. In this work, we propose a novel QoS-aware traffic modeling and scheduling framework integrated into a reinforcement learning-based quantum routing agent. Unlike prior approaches that assume deterministic or uniform request arrival patterns, our model introduces stochastic and bursty traffic scenarios using Erlang and ON/OFF distributions to emulate realistic quantum workloads. Each entanglement request is assigned a set of QoS attributes, including fidelity requirements, time-to-live constraints, and priority levels. We then extend the reward function of the agent to consider deadline compliance and resource efficiency in addition to fidelity. Through simulation, we compare the proposed method against state-of-the-art baselines, including DQRA and Proactive RL, under both best-effort and QoS-prioritized scheduling modes. Results demonstrate that our approach maintains higher success rates and lower latency for high-priority requests, while preserving entanglement fidelity and efficient resource allocation across varying network loads. This study highlights the importance of integrating QoS mechanisms into quantum routing to support scalable and differentiated quantum services
Quantum communications are gaining momentum in finding applications in a wide range of domains, especially those require high-security data transmissions. On the other hand, machine learning has achieved numerous breakthrough successes in various application domains including networking. However, currently, machine learning is not as much utilized in quantum networking as in other areas. With such motivation, we propose a machine-learning-powered entanglement routing scheme for quantum networks that aims to accommodate maximum numbers of demands (source-destination pairs) within a time window. More specifically, we present a deep reinforcement routing scheme that is called Deep Quantum Routing Agent (DQRA). In short, DQRA utilizes an empirically designed deep neural network that observes the current network states to schedule the network’s demands which are then routed by a qubit-preserved shortest path algorithm. DQRA is trained towards the goal of maximizing the number of resolved requests in each routing window by using an explicitly designed reward function. Our experiment study shows that, on averse, DQRA is able to maintain a rate of successfully routed requests above 80% in a qubit-limited grid network, and about 60% in extreme conditions i.e. each node can act as a repeater exactly once within a window. Furthermore, we show that the complexity and the computational time of DQRA are polynomial in terms of the sizes of the quantum networks.