Quantum error correction is one of the fundamental building blocks of digital quantum computation. The Quantum Lego formalism has introduced a systematic way of constructing new stabilizer codes out of basic lego-like building blocks, which in previous work we have used to generate improved error correcting codes via an automated reinforcement learning process. Here, we take this a step further and show the use of a hybrid classical-quantum algorithm. We combine classical reinforcement learning with calls to two commercial quantum devices to search for a stabilizer code to correct errors specific to the device, as well as an induced photon loss error.
Quantum error correction is one of the fundamental building blocks of digital quantum computation. The Quantum Lego formalism has introduced a systematic way of constructing new stabilizer codes out of basic lego-like building blocks, which in previous work we have used to generate improved error correcting codes via an automated reinforcement learning process. Here, we take this a step further and show the use of a hybrid classical-quantum algorithm. We combine classical reinforcement learning with calls to two commercial quantum devices to search for a stabilizer code to correct errors specific to the device, as well as an induced photon loss error.
Quantum error correction (QEC) is a crucial element in the realization of fault-tolerant quantum computation. However, traditional QEC schemes often rely on pre-determined, static parameters, which may not be optimally suited for the specific noise characteristics present in a given quantum hardware. This work introduces a novel approach to QEC that leverages reinforcement learning (RL) to dynamically adapt the error correction protocol in real-time. The system learns to optimize the parameters of the QEC code, such as measurement schedules and decoding strategies, based on continuous feedback from the quantum hardware. This adaptive approach promises to significantly improve the performance and efficiency of QEC, reducing the overhead associated with error correction and enhancing the fidelity of quantum computations. The core claim of this paper is that quantum error correction is computationally intensive and requires precise control over quantum systems. The central mechanism involves employing reinforcement learning to optimize the parameters of quantum error correction protocols in real-time, adapting to the specific noise characteristics of the quantum hardware. This represents a new automation of the QEC process.
Quantum error correction (QEC) is the primary strategy for protecting a quantum computer from the environment. Its prerequisite is that errors must remain sufficiently rare, which requires perpetually adapting the computer's control parameters to the drifting environment conditions. The current solution to this problem is to terminate the entire quantum computation for recalibration, but it is incompatible with the long runtimes of future quantum algorithms. We address this challenge by unifying calibration with computation. We grant the QEC process a dual role: its error detection events are not only used to correct the logical quantum state, but are also repurposed as a learning signal, teaching a reinforcement learning (RL) agent to continuously steer the control parameters and stabilize the quantum system during computation. We experimentally demonstrate this framework on a Willow superconducting processor, improving the logical stability of the surface code 3.5-fold against injected drift. By synthesizing our full suite of technological advances, we achieve record performance of the surface and color codes, with average logical error per cycle of 7.72(9)×10−4 and 8.19(14)×10−3 respectively. Numerical simulations of large codes with tens of thousands of control parameters confirm the scalability of our RL framework, revealing an optimization speed that is independent of system size. This work thus enables a new paradigm: a quantum computer that learns from its errors and never stops computing.
Major obstacles remain to the implementation of macroscopic quantum computing: hardware problems of noise, decoherence, and scaling; software problems of error correction; and, most important, algorithm construction. Finding truly quantum algorithms is quite difficult, and many of these genuine quantum algorithms, like Shor's prime factoring or phase estimation, require extremely long circuit depth for any practical application, which necessitates error correction. In contrast, we show that machine learning can be used as a systematic method to construct algorithms, that is, to non-algorithmically “program” quantum computers. Quantum machine learning enables us to perform computations without breaking down an algorithm into its gate “building blocks”, eliminating that difficult step and potentially increasing efficiency by simplifying and reducing unnecessary complexity. In addition, our non-algorithmic machine learning approach is robust to both noise and to decoherence, which is ideal for running on inherently noisy NISQ devices which are limited in the number of qubits available for error correction. We demonstrate this using a fundamentally nonclassical calculation: experimentally estimating the entanglement of an unknown quantum state. Results from this have been successfully ported to the IBM hardware and trained using a hybrid reinforcement learning method.
The advent of promising quantum error correction (QEC) codes with efficient resource utilization and high-performance fault-tolerant quantum memories signifies a critical step towards realizing practical quantum computation. While surface codes have been a dominant approach, their limitations have spurred the development of more advanced QEC architectures. These advanced codes often present increased complexity, demanding innovative decoding methodologies. This work investigates the application of reinforcement learning (RL) techniques, including hybrid and multi-agent approaches, to enhance the decoding of various advanced QEC architectures. By leveraging the ability of RL to learn optimal strategies from noisy syndrome measurements, we explore the potential for achieving improved logical error rates and scalability compared to traditional decoding methods. Our approach examines the adaptation of reinforcement learning to exploit the structural properties of these modern QEC models. We also explore the benefits of combining different RL algorithms to address the multifaceted nature of the decoding problem, considering factors such as code degeneracy and real-world noise characteristics. With our proposed method, we are able to demonstrate that an autonomously trained agent can derive decoding schemes for the complex decoding requirement of advanced QEC architectures.
The design and optimization of quantum codes is generally hard because of the prohibitive computational cost. The recently introduced quantum lego framework provides a powerful method for generating complex quantum error-correcting codes (QECCs) out of simple ones. We gamify this process and unlock a new avenue for code design and discovery using reinforcement learning (RL). Already for simple code measures and modest qubit numbers, we produce codes that are both optimal and novel. Moreover, we argue that this framework is both scalable, when combined with efficient tensor contraction methods, and flexible, since we can specify arbitrary properties of the code to be optimized. We train on two such properties, maximizing the code distance, and minimizing the probability of logical error under biased Pauli noise. For the first, we show that the trained agent identifies ways to increase code distance beyond naive concatenation, saturating the linear programming bound for CSS (Calderbank, Shor, Steane) codes on 13 qubits. With a learning objective to minimize the logical error probability under biased Pauli noise, we find the best-known CSS code at this task for $\ensuremath{\lesssim}20$ qubits. Compared to other (locally deformed) CSS codes, including Surface, XZZX, and two-dimensional Color codes, our [[17,1,3]] code construction actually has lower adversarial distance, yet better protects the logical information, highlighting the importance of QECC desiderata. Lastly, we comment on how this RL framework can be used in conjunction with physical quantum devices to tailor a code without explicit characterization of the noise model.
Quantum Error Correction (QEC) codes are a necessary component of utility-scale quantum computers, and the discovery of new codes with desirable properties is one of the most promising avenues for reducing the estimated resources required to perform useful quantum computations. Most current methods for code discovery rely on analytical techniques and construct codes from algebraic or geometric structures, but there also exist some techniques for discovering codes via computational search. In this work we focus on one of the most promising of these methods where a reinforcement learning agent is trained to construct encoding circuits and evaluates the performance of the corresponding code. We seek to improve upon this method by reducing the computational resources required to discover codes of given parameters, primarily by utilizing a probabilistic approximation of the reward function which relies on randomly sampling error operators according to some specified distribution. We investigate the performance of our modified framework and compare the discovered codes and required computational resources to the original work, concluding that probabilistic methods such as our sampling technique are effective strategies for reducing the computational cost of the evaluation component of code searching without excessive penalties to performance.