<!--HTML-->Quantum error correction will ultimately empower quantum computers to leverage their full potential. However, substantial device overhead and the need for frequent syndrome measurements, which are themselves error-prone, render the demonstration of logical qubits that significantly surpass break-even still challenging. Autonomous quantum error correction represents a promising alternative, where an engineered environment allows to bypass the syndrome measurements. In this talk, I show how we use reinforcement learning to search for, and find, bosonic code spaces that can surpass break-even under experimentally feasible conditions. Bosonic codes are, for instance, available and utilized in some of the currently most promising and widespread quantum processors based on superconducting qubits. Surprisingly, when we increase the search space by relaxing the constraints on ideal quantum error correction, we find simple and robust code words that significantly surpass break-even while minimizing device overhead. This RL code not only reduces device complexity compared to other proposed encodings, but also outperforms its competitors in terms of its capability to correct errors.
<!--HTML-->Quantum error correction will ultimately empower quantum computers to leverage their full potential. However, substantial device overhead and the need for frequent syndrome measurements, which are themselves error-prone, render the demonstration of logical qubits that significantly surpass break-even still challenging. Autonomous quantum error correction represents a promising alternative, where an engineered environment allows to bypass the syndrome measurements. In this talk, I show how we use reinforcement learning to search for, and find, bosonic code spaces that can surpass break-even under experimentally feasible conditions. Bosonic codes are, for instance, available and utilized in some of the currently most promising and widespread quantum processors based on superconducting qubits. Surprisingly, when we increase the search space by relaxing the constraints on ideal quantum error correction, we find simple and robust code words that significantly surpass break-even while minimizing device overhead. This RL code not only reduces device complexity compared to other proposed encodings, but also outperforms its competitors in terms of its capability to correct errors.
The advent of promising quantum error correction (QEC) codes with efficient resource utilization and high-performance fault-tolerant quantum memories signifies a critical step towards realizing practical quantum computation. While surface codes have been a dominant approach, their limitations have spurred the development of more advanced QEC architectures. These advanced codes often present increased complexity, demanding innovative decoding methodologies. This work investigates the application of reinforcement learning (RL) techniques, including hybrid and multi-agent approaches, to enhance the decoding of various advanced QEC architectures. By leveraging the ability of RL to learn optimal strategies from noisy syndrome measurements, we explore the potential for achieving improved logical error rates and scalability compared to traditional decoding methods. Our approach examines the adaptation of reinforcement learning to exploit the structural properties of these modern QEC models. We also explore the benefits of combining different RL algorithms to address the multifaceted nature of the decoding problem, considering factors such as code degeneracy and real-world noise characteristics. With our proposed method, we are able to demonstrate that an autonomously trained agent can derive decoding schemes for the complex decoding requirement of advanced QEC architectures.
Quantum error correction is essential for fault-tolerant quantum computing. However, standard methods relying on active measurements may introduce additional errors. Autonomous quantum error correction (AQEC) circumvents this by utilizing engineered dissipation and drives in bosonic systems, but identifying practical encoding remains challenging due to stringent Knill-Laflamme conditions. In this work, we utilize curriculum learning enabled deep reinforcement learning to discover Bosonic codes under approximate AQEC framework to resist both single-photon and double-photon losses. We present an analytical solution of solving the master equation under approximation conditions, which can significantly accelerate the training process of reinforcement learning. The agent first identifies an encoded subspace surpassing the breakeven point through rapid exploration within a constrained evolutionary time-frame, then strategically fine-tunes its policy to sustain this performance advantage over extended temporal horizons. We find that the two-phase trained agent can discover the optimal set of codewords, i.e., the Fock states ∣4⟩ and ∣7⟩ considering the effect of both single-photon and double-photon loss. We identify that the discovered code surpasses the breakeven threshold over a longer evolution time and achieve the state-of-art performance. We also analyze the robustness of the code against the phase damping and amplitude damping noise. Our work highlights the potential of curriculum learning enabled deep reinforcement learning in discovering the optimal quantum error correct code especially in early fault-tolerant quantum systems.
Quantum error correction (QEC) is the primary strategy for protecting a quantum computer from the environment. Its prerequisite is that errors must remain sufficiently rare, which requires perpetually adapting the computer's control parameters to the drifting environment conditions. The current solution to this problem is to terminate the entire quantum computation for recalibration, but it is incompatible with the long runtimes of future quantum algorithms. We address this challenge by unifying calibration with computation. We grant the QEC process a dual role: its error detection events are not only used to correct the logical quantum state, but are also repurposed as a learning signal, teaching a reinforcement learning (RL) agent to continuously steer the control parameters and stabilize the quantum system during computation. We experimentally demonstrate this framework on a Willow superconducting processor, improving the logical stability of the surface code 3.5-fold against injected drift. By synthesizing our full suite of technological advances, we achieve record performance of the surface and color codes, with average logical error per cycle of 7.72(9)×10−4 and 8.19(14)×10−3 respectively. Numerical simulations of large codes with tens of thousands of control parameters confirm the scalability of our RL framework, revealing an optimization speed that is independent of system size. This work thus enables a new paradigm: a quantum computer that learns from its errors and never stops computing.
Autonomous quantum error correction (AQEC) protects logical qubits by engineered dissipation and thus circumvents the necessity of frequent, error-prone measurement-feedback loops. Bosonic code spaces, where single-photon loss represents the dominant source of error, are promising candidates for AQEC due to their flexibility and controllability. While existing proposals have demonstrated the in-principle feasibility of AQEC with bosonic code spaces, these schemes are typically based on the exact implementation of the Knill-Laflamme conditions and thus require the realization of Hamiltonian distances d≥2. Implementing such Hamiltonian distances requires multiple nonlinear interactions and control fields, rendering these schemes experimentally challenging. Here, we propose a bosonic code for approximate AQEC by relaxing the Knill-Laflamme conditions. Using reinforcement learning (RL), we identify the optimal bosonic set of code words (denoted here by RL code), which, surprisingly, is composed of the Fock states |2⟩ and |4⟩. As we show, the RL code, despite its approximate nature, successfully suppresses single-photon loss, reducing it to an effective dephasing process that well surpasses the break-even threshold. It may thus provide a valuable building block toward full error protection. The error-correcting Hamiltonian, which includes ancilla systems that emulate the engineered dissipation, is entirely based on the Hamiltonian distance d=1, significantly reducing model complexity. Single-qubit gates are implemented in the RL code with a maximum distance d_{g}=2.
The realization of scalable fault-tolerant quantum computing is expected to hinge on quantum error-correcting codes. In the quest for more efficient quantum fault tolerance, a critical code parameter is the weight of measurements that extract information about errors to enable error correction: as higher measurement weights require higher implementation costs and introduce more errors, it is important in code design to optimize measurement weight. This underlies the surging interest in quantum low-density parity-check (qLDPC) codes, the study of which has primarily focused on the asymptotic (large-code-limit) properties. In this work, we introduce a versatile and computationally efficient approach to stabilizer code weight reduction based on reinforcement learning (RL), which produces new low-weight codes that substantially outperform the state of the art in practically relevant parameter regimes, extending significantly beyond previously accessible small distances. For example, our approach demonstrates savings in physical qubit overhead compared to existing results by 1 to 2 orders of magnitude for weight 6 codes and brings the overhead into a feasible range for near-future experiments. We also investigate the interplay between code parameters using our RL framework, offering new insights into the potential efficiency and power of practically viable coding strategies. Overall, our results demonstrate how RL can effectively advance the crucial yet challenging problem of quantum code discovery and thereby facilitate a faster path to the practical implementation of fault-tolerant quantum technologies.
Abstract Topological error correcting codes, and particularly the surface code, currently provide the most feasible road-map towards large-scale fault-tolerant quantum computation. As such, obtaining fast and flexible decoding algorithms for these codes, within the experimentally realistic and challenging context of faulty syndrome measurements, without requiring any final read-out of the physical qubits, is of critical importance. In this work, we show that the problem of decoding such codes can be naturally reformulated as a process of repeated interactions between a decoding agent and a code environment, to which the machinery of reinforcement learning can be applied to obtain decoding agents. While in principle this framework can be instantiated with environments modelling circuit level noise, we take a first step towards this goal by using deepQ learning to obtain decoding agents for a variety of simplified phenomenological noise models, which yield faulty syndrome measurements without including the propagation of errors which arise in full circuit level noise models.