International Conference on Transparent Optical Networks

Quantum Virtual Link Generation via Reinforcement Learning

Ramón Aparicio-Pardo, Antoine Cousson, Redha A. Alliche

Published Jul 2, 2023• 0 citations

Abstract

Quantum networks make use of the quantum entanglement as building block. When two qubits are entangled, their state changes exhibit non-classical correlations used to design new applications not possible with classical communication, such us quantum key distribution or distributed quantum computation. Unfortunately, quantum entanglement is a probabilistic process strongly dependent on the features of involved devices (optical fibers, lasers, quantum memories, …). The management decisions (i.e., the control policy) to set up and keep the entanglement as long as possible with the highest quality constitutes a stochastic control problem. This process can be modelled as Markov Decision Process (MDP) and solved via the Reinforcement Learning (RL) framework (a form of Machine Learning). In this work, we apply this RL framework to learn an entanglement management policy outperforming the State-of-the-Art policy when models characterising precisely the involved quantum devices are not known.

DOI: 10.1109/icton59386.2023.10207249

Related papers

Control of Qubit Dynamics Using Reinforcement Learning

Dimitris Koutromanos, Dionisis Stefanatos, Emmanuel Paspalakis

The progress in machine learning during the last decade has had a considerable impact on many areas of science and technology, including quantum technology. This work explores the application of reinforcement learning (RL) methods to the quantum control problem of state transfer in a single qubit. The goal is to create an RL agent that learns an optimal policy and thus discovers optimal pulses to control the qubit. The most crucial step is to mathematically formulate the problem of interest as a Markov decision process (MDP). This enables the use of RL algorithms to solve the quantum control problem. Deep learning and the use of deep neural networks provide the freedom to employ continuous action and state spaces, offering the expressivity and generalization of the process. This flexibility helps to formulate the quantum state transfer problem as an MDP in several different ways. All the developed methodologies are applied to the fundamental problem of population inversion in a qubit. In most cases, the derived optimal pulses achieve fidelity equal to or higher than 0.9999, as required by quantum computing applications. The present methods can be easily extended to quantum systems with more energy levels and may be used for the efficient control of collections of qubits and to counteract the effect of noise, which are important topics for quantum sensing applications.

InformationMay 11, 202411 citations

Reinforcement Learning for Quantum Network Control with Application-Driven Objectives

Guo Xian Yau, Alexandra Burushkina, Francisco Ferreira da Silva +3 more

Optimized control of quantum networks is essential for enabling distributed quantum applications with strict performance requirements. In near-term architectures with constrained hardware, effective control may determine the feasibility of deploying such applications. Because quantum network dynamics are suitable for being modeled as a Markov decision process, dynamic programming and reinforcement learning (RL) offer promising tools for optimizing control strategies. However, key quantum network performance measures -- such as secret key rate in quantum key distribution -- often involve a non-linear relationship between interdependent variables that describe quantum state quality and generation rate. Such objectives are not easily captured by standard RL approaches based on additive rewards. We propose a novel gradient-based RL framework that directly optimizes non-linear, differentiable objective functions, while accounting for uncertainties introduced by classical communication delays. We evaluate this framework in the context of entanglement distillation between two quantum network nodes equipped with multiplexing capability, and demonstrate up to 20-23% improvement over heuristic baselines in certain parameter regimes. Our work comprises the first step towards non-linear objective function optimization in quantum networks with RL, opening a path towards more advanced use cases.

arXiv:2509.10634Sep 12, 20250 citations

Reinforcement Learning for Entanglement Swapping in Quantum Networks

Álvaro Troyano Olivas, Andrés Agustí Casado, Marco Pérez González +4 more

Current approaches for entanglement distribution networks face a largely untackled problem: precise timing and management of entanglement generation and swapping attempts is key in order to make entanglement distribution between any two nodes.To address this, we model a quantum network as two graphs, one containing the telecom fibers enabling physically the entanglement, and another containing the Bell pairs stored in its memories. Then, efficient entanglement distribution becomes a task of populating a mix of both graphs with a connection between one Alice and one Bob subject to constraints. Here, machine learning techniques are useful to find efficient policies through reinforcement learning. This way, we manage to consider inhomogeneous probabilities of generation and swapping, as well as complex networks beyond the chain scenario.

International Conference on Quantum Communications, Networking, and Computing (QCNC)Mar 31, 20253 citations

Adaptive Quantum Control Policies via Reinforcement Learning

Jincheng Zhang

This paper presents a novel approach to designing adaptive quantum control policies using reinforcement learning (RL). Traditional methods for quantum control often rely on hand-designed pulse shapes or iterative optimization techniques, which can be computationally expensive and may not generalize well to different systems or desired operations. We propose a framework where an RL agent learns to dynamically adjust quantum control parameters in real-time, based on the system's state and the objective of achieving a specific quantum gate with high fidelity. The core of our approach lies in formulating the quantum control problem as a Markov Decision Process (MDP) and utilizing an RL algorithm to train a policy that maps system states to control parameters. We demonstrate the feasibility and effectiveness of this method through a theoretical analysis and a simplified numerical simulation. The resulting adaptive control policies offer a significant advantage over traditional methods, particularly for complex quantum systems and demanding fidelity requirements. The key contributions are the integration of RL for adaptive control and the formalization of the problem as an MDP.

Sep 3, 20260 citations

Adaptive and Efficient Qubit Allocation Using Reinforcement Learning in Quantum Networks

Yanan Gao, Song Yang, Fan Li +1 more

Quantum entanglement brings high-speed and inherently privacy-preserving transmission for information communication in quantum networks. The qubit scarcity is an important issue that cannot be ignored in quantum networks due to the limited storage capacity of quantum devices, the short lifespan of qubits, and so on. In this article, we first formulate the qubit competition problem as the Cooperative-Qubit-Allocation-Problem (CQAP) by taking into account both the waiting time and the fidelity of end-to-end entanglement with the given transmission link set. We then model the CQAP as a Markov Decision Process (MDP) and adopt a Reinforcement Learning (RL) algorithm to self-adaptively and cooperatively allocate qubits among quantum repeaters. Further, we introduce an Active Learning (AL) algorithm to improve the efficiency of the RL algorithm by reducing its trial-error times. Simulation results demonstrate that our proposed algorithm outperforms the benchmark algorithms, with 23.5 ms reduction on the average waiting time and 19.2 improvement on the average path maturity degree, respectively.

IEEE NetworkSep 1, 20223 citations

Learning and interpreting policies for simultaneous entanglement requests in quantum networks

Leon Rode, Sumeet Khatri, Supartha Podder

Future quantum networks will make use of entanglement to perform numerous tasks, such as sending quantum information over long distances, distributed quantum computing, and quantum sensing. In general, these tasks will need to be performed simultaneously in various regions of a network, while minimizing resources and latency. We will thus require policies for scheduling link-level entanglement resources, and using the link-level entanglement to create various forms of multipartite entanglement required for every task. In this work, we address this problem using reinforcement learning. We formulate a Markov Decision Process for the problem and use double deep Q-networks (DQN) with Message Passing Neural Networks (MPNNs), experience replay buffers, and curriculum training to obtain policies. The key physical parameter is the probability of link-level entanglement generation, i.e., the link activation probability. We show that our policies maintain 100% success for up to 71% lower link activation probability than the baseline heuristics for a set of physically relevant network topologies. We then examine an additional constraint where experiment (task) placements are restricted to specific hardware types and demonstrate a similar advantage in performance over heuristics, with our policy maintaining at least an 80% success rate for up to a 59% lower link activation probability. Finally, we explore methods to interpret the learned policy by defining metrics enabling conclusions to be drawn about the model's behavior and by tasking a large language model (LLM) to derive a novel heuristic given example actions taken by the DQN-trained policy. We find that the LLM heuristic performs similarly to the DQN-trained policy in performance, indicating a promising method for interpretable policy extraction for large quantum networks, where direct training becomes computationally expensive.

quant-pharXiv:2609.30157Sep 25, 20260 citations