Distributing entanglement over long distances is one of the central tasks in quantum networks. An important problem, especially for near-term quantum networks, is to develop optimal entanglement distribution protocols that take into account the limitations of current and near-term hardware, such as quantum memories with limited coherence time. We address this problem by initiating the study of quantum network protocols for entanglement distribution using the theory of decision processes, such that optimal protocols (referred to as policies in the context of decision processes) can be found using dynamic programming or reinforcement learning algorithms. As a first step, in this work we focus exclusively on the elementary link level. We start by defining a quantum decision process for elementary links, along with figures of merit for evaluating policies. We then provide two algorithms for determining policies, one of which we prove to be optimal (with respect to fidelity and success probability) among all policies. Then we show that the previously-studied memory-cutoff protocol can be phrased as a policy within our decision process framework, allowing us to obtain several new fundamental results about it. The conceptual developments and results of this work pave the way for the systematic study of the fundamental limitations of near-term quantum networks, and the requirements for physically realizing them.
Distributing entanglement over long distances is one of the central tasks in quantum networks. An important problem, especially for near-term quantum networks, is to develop optimal entanglement distribution protocols that take into account the limitations of current and near-term hardware, such as quantum memories with limited coherence time. We address this problem by initiating the study of quantum network protocols for entanglement distribution using the theory of decision processes, such that optimal protocols (referred to as policies in the context of decision processes) can be found using dynamic programming or reinforcement learning algorithms. As a first step, in this work we focus exclusively on the elementary link level. We start by defining a quantum decision process for elementary links, along with figures of merit for evaluating policies. We then provide two algorithms for determining policies, one of which we prove to be optimal (with respect to fidelity and success probability) among all policies. Then we show that the previously-studied memory-cutoff protocol can be phrased as a policy within our decision process framework, allowing us to obtain several new fundamental results about it. The conceptual developments and results of this work pave the way for the systematic study of the fundamental limitations of near-term quantum networks, and the requirements for physically realizing them.
The quantum internet is one of the frontiers of quantum information science. It will revolutionize the way we communicate and do other tasks, and it will allow for tasks that are not possible using the current, classical internet. The backbone of a quantum internet is entanglement distributed globally in order to allow for such novel applications to be performed over long distances. Experimental progress is currently being made to realize quantum networks on a small scale, but much theoretical work is still needed in order to understand how best to distribute entanglement, especially with the limitations of near-term quantum technologies taken into account. This work provides an initial step toward this goal. In this work, we lay out a theory of near-term quantum networks based on Markov decision processes (MDPs), and we show that MDPs provide a precise and systematic mathematical framework to model protocols for near-term quantum networks that is agnostic to the specific implementation platform. We start by simplifying the MDP for elementary links introduced in prior work and by providing new results on policies for elementary links in the steady-state (infinite-time) limit. Then, we show how the elementary link MDP can be used to analyze a complete quantum network protocol. We then provide an extension of the MDP formalism to two elementary links. Here, as new results, we derive linear programing relaxations that allow us to obtain optimal steady-state policies with respect to the expected fidelity and waiting time of the end-to-end link.
The quantum internet is one of the frontiers of quantum information science. It will revolutionize the way we communicate and do other tasks, and it will allow for tasks that are not possible using the current, classical internet. The backbone of a quantum internet is entanglement distributed globally in order to allow for such novel applications to be performed over long distances. Experimental progress is currently being made to realize quantum networks on a small scale, but much theoretical work is still needed in order to understand how best to distribute entanglement and to guide the realization of large-scale quantum networks, and eventually the quantum internet, especially with the limitations of near-term quantum technologies. This work provides an initial step towards this goal. The main contribution of this thesis is a mathematical framework for entanglement distribution protocols in a quantum network, which allows for discovering optimal protocols using reinforcement learning. We start with a general development of quantum decision processes, which is the theoretical backdrop of reinforcement learning. Then, we define the general task of entanglement distribution in a quantum network, and we present ground- and satellite-based quantum network architectures that incorporate practical aspects of entanglement distribution. We combine the theory of decision processes and the practical quantum network architectures into an overall entanglement distribution protocol. We also define practical figures of merit to evaluate entanglement distribution protocols, which help to guide experimental implementations.
Abstract Quantum repeaters play a crucial role in the effective distribution of entanglement over long distances. The nearest-future type of quantum repeater requires two operations: entanglement generation across neighbouring repeaters and entanglement swapping to promote short-range entanglement to long-range. For many hardware setups, these actions are probabilistic, leading to longer distribution times and incurred errors. Significant efforts have been vested in finding the optimal entanglement-distribution policy, i.e. the protocol specifying when a network node needs to generate or swap entanglement, such that the expected time to distribute long-distance entanglement is minimal. This problem is even more intricate in more realistic scenarios, especially when classical communication delays are taken into account. In this work, we formulate our problem as a Markov decision problem and use reinforcement learning (RL) to optimise over centralised strategies, where one designated node instructs other nodes which actions to perform. Contrary to most RL models, ours can be readily interpreted. Additionally, we introduce and evaluate a fixed local policy, the ‘predictive swap-asap’ policy, where nodes only coordinate with nearest neighbours. Compared to the straightforward generalisation of the common swap-asap policy to the scenario with classical communication effects, the ‘wait-for-broadcast swap-asap’ policy, both of the aforementioned entanglement-delivery policies are faster at high success probabilities. Our work showcases the merit of considering policies acting with incomplete information in the realistic case when classical communication effects are significant.
Future quantum networks will make use of entanglement to perform numerous tasks, such as sending quantum information over long distances, distributed quantum computing, and quantum sensing. In general, these tasks will need to be performed simultaneously in various regions of a network, while minimizing resources and latency. We will thus require policies for scheduling link-level entanglement resources, and using the link-level entanglement to create various forms of multipartite entanglement required for every task. In this work, we address this problem using reinforcement learning. We formulate a Markov Decision Process for the problem and use double deep Q-networks (DQN) with Message Passing Neural Networks (MPNNs), experience replay buffers, and curriculum training to obtain policies. The key physical parameter is the probability of link-level entanglement generation, i.e., the link activation probability. We show that our policies maintain 100% success for up to 71% lower link activation probability than the baseline heuristics for a set of physically relevant network topologies. We then examine an additional constraint where experiment (task) placements are restricted to specific hardware types and demonstrate a similar advantage in performance over heuristics, with our policy maintaining at least an 80% success rate for up to a 59% lower link activation probability. Finally, we explore methods to interpret the learned policy by defining metrics enabling conclusions to be drawn about the model's behavior and by tasking a large language model (LLM) to derive a novel heuristic given example actions taken by the DQN-trained policy. We find that the LLM heuristic performs similarly to the DQN-trained policy in performance, indicating a promising method for interpretable policy extraction for large quantum networks, where direct training becomes computationally expensive.
We study the limits of bipartite entanglement distribution using a chain of quantum repeaters that have quantum memories. To generate end-to-end entanglement, each node can attempt the generation of an entangled link with a neighbor, or perform an entanglement swapping measurement. A maximum storage time, known as cutoff, is enforced on the memories to ensure high-quality entanglement. Nodes follow a policy that determines when to perform each operation. Global-knowledge policies take into account all the information about the entanglement already produced. Here, we find global-knowledge policies that minimize the expected time to produce end-to-end entanglement. Our methods are based on Markov decision processes and value and policy iteration. We compare optimal policies to a policy in which nodes only use local information. We find that the advantage in expected delivery time provided by an optimal global-knowledge policy increases with increasing number of nodes and decreasing probability of successful swapping.
Tomorrow's quantum technologies for communication, sensing, and distributed computing will rely on networks with entanglement shared between spatially separated nodes. The authors provide improved protocols and policies for entanglement distribution along a chain of nodes, accounting for practical limitations such as photon losses, nonideal measurements, and quantum memories with short coherence times. These policies feature dynamic, state-dependent memory cutoffs and collaboration between nodes, all of which are quantified. Nesting policies for small repeater chains yields policies for large chains that improve upon a swap-as-soon-as-possible approach, and thus pave the way to scaling up.