With progressive network recovery, operators restore network connectivity after massive failures along multiple stages, by identifying the optimal sequence of repair actions to maximize carried live traffic. Motivated by the initial deployments of quantum-key-distribution (QKD) over optical networks appearing in several locations worldwide, in this work we model and solve the progressive QKD network recovery (PQNR) problem in QKD networks to accelerate the recovery after failures. We formulate an integer linear programming (ILP) model to optimize the achievable accumulative key rates during recovery for four different QKD network architectures, considering different capabilities of using trusted relay and optical bypass. Due to the computational limitations of the ILP model, we propose a deep reinforcement learning (DRL) algorithm based on a twin delayed deep deterministic policy gradients (TD3) framework to solve the PQNR problem for large-scale topologies. Simulation results show that our proposed algorithm approaches well compared to the optimal solution and outperforms several baseline algorithms. Moreover, using optical bypass jointly with trusted relay can improve the performance in terms of the key rate by 14% and 18% compared to the cases where only optical bypass and only trusted relay are applied, respectively.
With progressive network recovery, operators restore network connectivity after massive failures along multiple stages, by identifying the optimal sequence of repair actions to maximize carried live traffic. Motivated by the initial deployments of quantum-key-distribution (QKD) over optical networks appearing in several locations worldwide, in this work we model and solve the progressive QKD network recovery (PQNR) problem in QKD networks to accelerate the recovery after failures. We formulate an integer linear programming (ILP) model to optimize the achievable accumulative key rates during recovery for four different QKD network architectures, considering different capabilities of using trusted relay and optical bypass. Due to the computational limitations of the ILP model, we propose a deep reinforcement learning (DRL) algorithm based on a twin delayed deep deterministic policy gradients (TD3) framework to solve the PQNR problem for large-scale topologies. Simulation results show that our proposed algorithm approaches well compared to the optimal solution and outperforms several baseline algorithms. Moreover, using optical bypass jointly with trusted relay can improve the performance in terms of the key rate by 14% and 18% compared to the cases where only optical bypass and only trusted relay are applied, respectively.
Progressive network recovery is the problem arising when a network is subject to massive failures and the network operator has to identify the optimal sequence of repair actions to maximize carried traffic. As initial deployments of quantum-key-distribution (QKD) over optical networks start appearing in several locations worldwide, in this paper, for the first time, we model and solve the Progressive QKD Network recovery (PQNR) problem in QKD networks to accelerate the QKD network recovery after massive failures. Specifically, we formulate an Integer Linear Programming (ILP) model to model the achievable key rate for different QKD network architectures (w/o trusted relay and optical bypass). Due to the scalability issue of ILP, we also propose a scalable heuristic to solve the PQNR problem for large topologies. Our numerical results show that joint utilization of optical bypass and trusted node technologies leads to significant improvement in performance and that our heuristic solution has an acceptable optimality gap compared with ILP while reducing time.
Routing is a challenging problem in quantum key distribution (QKD)-secured optical networks (QKD-ONs) and involves the selection of an appropriate route that establishes a secure path between the QKD nodes for secret key distribution. Deep reinforcement learning (DRL) is a promising approach for solving decision-making problems in complex networking environments such as QKD-ONs. By leveraging the capabilities of DRL algorithms, the routing decisions can be optimized to enhance network performance. This paper proposes a DRL-based solution for routing in QKD-ONs that enables the routing agent to learn and adapt to changing network conditions by understanding the networking environment. The performance of the proposed scheme is compared with the baseline schemes on NSFNET in terms of blocking probability. Simulation results indicate that compared to the baseline schemes (shortest path (SP) and hop count (HC)), the proposed DRL-based routing scheme reduces the blocking by 14.31% and 8%, respectively.
Trusted relay quantum key distribution networks (TR-QKDNs) have emerged as one of the most practical solutions for implementing large-scale QKDNs. The development of efficient routing algorithms is crucial to ensure adaptability to diverse network parameters and topologies. However, current approaches suffer from multiple limitations, including insufficient consideration of key influencing factors, excessive reliance on manually configured parameters, and scalability bottlenecks caused by the exponential complexity of classical algorithms. In this paper, we propose a quantum reinforcement learning-based routing algorithm, named hybrid quantum deep deterministic policy gradient (HQ-DDPG) that integrates a custom-designed quantum neural network (QNN) with the deep deterministic policy gradient (DDPG), to intelligently balance multiple influencing factors and dynamically optimize routing decisions in TR-QKDNs. To reduce computational complexity, we further utilize a quantum single-source shortest path (QSSP) algorithm to compute the optimal routing path. Training results demonstrate that the proposed HQ-DDPG outperforms DDPG in terms of performance metrics, achieving nearly double the training convergence speed while reducing resource requirements by approximately 45 times and exhibiting superior network expressiveness. In typical network topology tests, the proposed algorithm consistently maintains a quantum key delivery ratio above 91.35% under high-load demand, significantly surpassing both DDPG and optimized link state routing (OLSR). Finally, leveraging distributed quantum computing, the proposed QNN enables efficient solutions for large-scale TR-QKDN problems with fewer quantum resources.
Abstract Quantum key distribution (QKD) optical networks, relying on the information-theoretic security guaranteed by the fundamental laws of quantum physics, have become a core solution for ensuring data transmission security in critical fields such as finance and energy. However, QKD optical networks have inherent characteristics like severe resource constraints, dynamic changes in network environments. The collaborative optimization of routing, wavelength, and time-slot assignment (RWTA) has thus become a key bottleneck restricting their practical deployment. Traditional algorithms suffer from high complexity and poor adaptability in large-scale dynamic networks, while existing deep reinforcement learning (DRL) schemes insufficiently consider network security states, failing multi-objective co-optimization. Therefore, this paper proposes a multi-objective secure proximal policy optimization algorithm (MOS-PPO) based on DRL to solve the routing, wavelength, and time slots assignment problem in QKD optical networks. The MOS-PPO algorithm expands the state space by introducing a network security state vector that includes quantum bit error rate, risk coefficient, node capacity, and processing capability. It designs a multi-objective reward function, and establishes a security constraint mechanism to achieve the collaborative optimization of blocking probability (BP), resource utilization (RU), and security performance. Simulations on NSFNET and UBN24 topologies show MOS-PPO outperforms DRL-PPO algorithm, Security-aware Load balancing, first fit algorithm, and random fit algorithm. On NSFNET, average BP drops by 20.4%, 22.1%,28.4%, 36.7% and RU rises by 4.0%,3.9%, 22.2%, 31.0%, on UBN24, BP decreases by 9.3%, 11.4%, 18.8%, and 27.4% and RU increases by 3.8%, 21.2%, 10.3%, and 29.5%, with higher stable security margins. This study verifies the high adaptability and multi-objective optimization ability of the MOS-PPO algorithm in dynamic network environments, providing an efficient RWTA solution for the practical deployment of QKD optical networks.
Quantum networks attract increasing attention due to their unique advantages over classical networks. However, the challenge of efficiently and economically constructing reliable wide-area quantum networks remains unresolved. This problem, termed network planning, is typically modeled as an Integer Linear Programming (ILP) problem. Existing solver-based solutions for classical networks face scalability issues, while heuristic approaches that sacrifice optimality for scalability heavily rely on human expertise and lack generalizability. Recently, deep reinforcement learning (DRL) has demonstrated potential in adaptive decision-making for classical network planning. However, quantum networks introduce unique constraints due to their end-to-end entanglement distribution requirements. To address this, we propose a two-stage quantum network resource allocation framework, QPLAN, that combines multi-agent DRL with traditional optimization solvers. Our approach uses a novel multi-agent reinforcement learning strategy using graph neural networks (GNNs) and transformers for traffic flow and corresponding hardware resource allocation to minimize hardware cost while cooperatively satisfying problem constraints as much as possible. The output of the DRL algorithm is then used as an initial input to prune the solution space, "warm-starting" the ILP solver and further improving the solution. This hybrid approach effectively reduces manual effort and enhances scalability. Experimental results show that our method yields near-optimal solutions with better scalability than only using ILP solvers.
Abstract In quantum key distribution‐secured optical networks (QKD‐ONs), constrained network resources limit the success probability of QKD lightpath requests (QLRs). Thus, the selection of an appropriate route and the efficient utilisation of network resources for establishment of QLRs are the essential and challenging problems. This work addresses the routing and resource assignment (RRA) problem in the quantum signal channel of QKD‐ONs. The RRA problem of QKD‐ONs is a complex decision making problem, where appropriate solutions depend on understanding the networking environment. Motivated by the recent advances in deep reinforcement learning (DRL) for complex problems and also because of its capability to learn directly from experiences, DRL is exploited to solve the RRA problem and a DRL‐based RRA scheme is proposed. The proposed scheme learns the optimal policy to select an appropriate route and assigns suitable network resources for establishment of QLRs by using deep neural networks. The performance of the proposed scheme is compared with the deep‐Q network (DQN) method and two baseline schemes, namely, first‐fit (FF) and random‐fit (RF) for two different networks, namely The National Science Foundation Network (NSFNET) and UBN24. Simulation results indicate that the proposed scheme reduces blocking by 7.19%, 10.11%, and 33.50% for NSFNET and 2.47%, 3.20%, and 19.60% for UBN24 and improves resource utilisation up to 3.40%, 4.33%, and 7.18% for NSFNET and 1.34%, 1.96%, and 6.44% for UBN24 as compared with DQN, FF, and RF, respectively.