Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2305.06378 · Physical Review Applied

Discovery of optimal quantum codes via reinforcement learning

Vincent Paul Su, ChunJun Cao, Hong-Ye Hu, Yariv Yanay, Charles Tahan, Brian G. Swingle

Published Mar 20, 2025• 11 citations
arXivHTMLPDF

Abstract

The design and optimization of quantum codes is generally hard because of the prohibitive computational cost. The recently introduced quantum lego framework provides a powerful method for generating complex quantum error-correcting codes (QECCs) out of simple ones. We gamify this process and unlock a new avenue for code design and discovery using reinforcement learning (RL). Already for simple code measures and modest qubit numbers, we produce codes that are both optimal and novel. Moreover, we argue that this framework is both scalable, when combined with efficient tensor contraction methods, and flexible, since we can specify arbitrary properties of the code to be optimized. We train on two such properties, maximizing the code distance, and minimizing the probability of logical error under biased Pauli noise. For the first, we show that the trained agent identifies ways to increase code distance beyond naive concatenation, saturating the linear programming bound for CSS (Calderbank, Shor, Steane) codes on 13 qubits. With a learning objective to minimize the logical error probability under biased Pauli noise, we find the best-known CSS code at this task for $\ensuremath{\lesssim}20$ qubits. Compared to other (locally deformed) CSS codes, including Surface, XZZX, and two-dimensional Color codes, our $[[17,1,3]]$ code construction actually has lower adversarial distance, yet better protects the logical information, highlighting the importance of QECC desiderata. Lastly, we comment on how this RL framework can be used in conjunction with physical quantum devices to tailor a code without explicit characterization of the noise model.

DOI
10.1103/physrevapplied.23.034048

Related papers

Simultaneous discovery of quantum error correction codes and encoders with a noise-aware reinforcement learning agent

Jan Ollé, Remmy A. M. Zen, Matteo Ballerini Puviani +1 more

Abstract In the ongoing race towards experimental implementations of quantum error correction (QEC), finding ways to automatically discover codes and encoding strategies tailored to the qubit hardware platform is emerging as a critical problem. Reinforcement learning (RL) has been identified as a promising approach, but so far it has been severely restricted in terms of scalability. In this work, we significantly expand the power of RL approaches to QEC code discovery. Explicitly, we train an RL agent that automatically discovers both QEC codes and their encoding circuits for a given gate set, qubit connectivity and error model, from scratch. This is enabled by a reward based on the Knill-Laflamme conditions and a vectorized Clifford simulator, showing its effectiveness with up to 25 physical qubits and distance 5 codes, while presenting a roadmap to scale this approach to 100 qubits and distance 10 codes in the near future. We also introduce the concept of a noise-aware meta-agent, which learns to produce encoding strategies simultaneously for a range of noise models, thus leveraging transfer of insights between different situations. Our approach opens the door towards hardware-adapted accelerated discovery of QEC approaches across the full spectrum of quantum hardware platforms of interest.

arXiv:2311.04750•npj Quantum Information•Dec 3, 2024•24 citations

Reinforcement Learning for Enhanced Advanced QEC Architecture Decoding

Yidong Zhou, Lingyi Kong, Yifeng Peng +1 more

The advent of promising quantum error correction (QEC) codes with efficient resource utilization and high-performance fault-tolerant quantum memories signifies a critical step towards realizing practical quantum computation. While surface codes have been a dominant approach, their limitations have spurred the development of more advanced QEC architectures. These advanced codes often present increased complexity, demanding innovative decoding methodologies. This work investigates the application of reinforcement learning (RL) techniques, including hybrid and multi-agent approaches, to enhance the decoding of various advanced QEC architectures. By leveraging the ability of RL to learn optimal strategies from noisy syndrome measurements, we explore the potential for achieving improved logical error rates and scalability compared to traditional decoding methods. Our approach examines the adaptation of reinforcement learning to exploit the structural properties of these modern QEC models. We also explore the benefits of combining different RL algorithms to address the multifaceted nature of the decoding problem, considering factors such as code degeneracy and real-world noise characteristics. With our proposed method, we are able to demonstrate that an autonomously trained agent can derive decoding schemes for the complex decoding requirement of advanced QEC architectures.

quant-pharXiv:2601.19279•Jan 28, 2026•0 citations

Discovering highly efficient low-weight quantum error-correcting codes with reinforcement learning

He, Austin Yubo, Zi-Wen Liu

The realization of scalable fault-tolerant quantum computing is expected to hinge on quantum error-correcting codes. In the quest for more efficient quantum fault tolerance, a critical code parameter is the weight of measurements that extract information about errors to enable error correction: as higher measurement weights require higher implementation costs and introduce more errors, it is important in code design to optimize measurement weight. This underlies the surging interest in quantum low-density parity-check (qLDPC) codes, the study of which has primarily focused on the asymptotic (large-code-limit) properties. In this work, we introduce a versatile and computationally efficient approach to stabilizer code weight reduction based on reinforcement learning (RL), which produces new low-weight codes that substantially outperform the state of the art in practically relevant parameter regimes, extending significantly beyond previously accessible small distances. For example, our approach demonstrates savings in physical qubit overhead compared to existing results by 1 to 2 orders of magnitude for weight 6 codes and brings the overhead into a feasible range for near-future experiments. We also investigate the interplay between code parameters using our RL framework, offering new insights into the potential efficiency and power of practically viable coding strategies. Overall, our results demonstrate how RL can effectively advance the crucial yet challenging problem of quantum code discovery and thereby facilitate a faster path to the practical implementation of fault-tolerant quantum technologies.

arXiv:2502.14372•Feb 20, 2025•0 citations

Designing Quantum Error Correcting Codes to fit decoders via Reinforcement Learning

Omer S. Sella, Robert Pinsler, Thomas Heinis

We present a reinforcement learning (RL) approach to the co-design of stabilizer sets of Quantum Error Correcting Codes (QECCs) and decoders. We show how to produce a generative model that produces Bivariate Bicycle (BB) codes based on the choice of decoder. Specifically, we fix a decoder architecture and use Proximal Policy Optimisation (PPO) to train an agent over BB codes to maximise decoder performance under a depolarising channel noise model.

quant-pharXiv:2608.15754•Aug 18, 2026•0 citations

Optimizing Quantum Error Correction Codes with Reinforcement Learning

Hendrik Poulsen Nautrup, Nicolas Delfosse, Vedran Dunjko +2 more

Quantum error correction is widely thought to be the key to fault-tolerant quantum computation. However, determining the most suited encoding for unknown error channels or specific laboratory setups is highly challenging. Here, we present a reinforcement learning framework for optimizing and fault-tolerantly adapting quantum error correction codes. We consider a reinforcement learning agent tasked with modifying a family of surface code quantum memories until a desired logical error rate is reached. Using efficient simulations with about 70 data qubits with arbitrary connectivity, we demonstrate that such a reinforcement learning agent can determine near-optimal solutions, in terms of the number of data qubits, for various error models of interest. Moreover, we show that agents trained on one setting are able to successfully transfer their experience to different settings. This ability for transfer learning showcases the inherent strengths of reinforcement learning and the applicability of our approach for optimization from off-line simulations to on-line laboratory settings.

arXiv:1812.08451•Quantum•Dec 16, 2019•162 citations

Reinforcement Learning Control of Quantum Error Correction

Volodymyr Sivak, Alexis Morvan, Michael Broughton +296 more

Quantum error correction (QEC) is the primary strategy for protecting a quantum computer from the environment. Its prerequisite is that errors must remain sufficiently rare, which requires perpetually adapting the computer's control parameters to the drifting environment conditions. The current solution to this problem is to terminate the entire quantum computation for recalibration, but it is incompatible with the long runtimes of future quantum algorithms. We address this challenge by unifying calibration with computation. We grant the QEC process a dual role: its error detection events are not only used to correct the logical quantum state, but are also repurposed as a learning signal, teaching a reinforcement learning (RL) agent to continuously steer the control parameters and stabilize the quantum system during computation. We experimentally demonstrate this framework on a Willow superconducting processor, improving the logical stability of the surface code 3.5-fold against injected drift. By synthesizing our full suite of technological advances, we achieve record performance of the surface and color codes, with average logical error per cycle of $7.72(9)\times10^{-4}$ and $8.19(14)\times10^{-3}$ respectively. Numerical simulations of large codes with tens of thousands of control parameters confirm the scalability of our RL framework, revealing an optimization speed that is independent of system size. This work thus enables a new paradigm: a quantum computer that learns from its errors and never stops computing.

quant-pharXiv:2511.08493•Nature•Nov 12, 2025•5 citations
The design and optimization of quantum codes is generally hard because of the prohibitive computational cost. The recently introduced quantum lego framework provides a powerful method for generating complex quantum error-correcting codes (QECCs) out of simple ones. We gamify this process and unlock a new avenue for code design and discovery using reinforcement learning (RL). Already for simple code measures and modest qubit numbers, we produce codes that are both optimal and novel. Moreover, we argue that this framework is both scalable, when combined with efficient tensor contraction methods, and flexible, since we can specify arbitrary properties of the code to be optimized. We train on two such properties, maximizing the code distance, and minimizing the probability of logical error under biased Pauli noise. For the first, we show that the trained agent identifies ways to increase code distance beyond naive concatenation, saturating the linear programming bound for CSS (Calderbank, Shor, Steane) codes on 13 qubits. With a learning objective to minimize the logical error probability under biased Pauli noise, we find the best-known CSS code at this task for \ensuremath≲20\ensuremath{\lesssim}20\ensuremath≲20 qubits. Compared to other (locally deformed) CSS codes, including Surface, XZZX, and two-dimensional Color codes, our [[17,1,3]][[17,1,3]][[17,1,3]] code construction actually has lower adversarial distance, yet better protects the logical information, highlighting the importance of QECC desiderata. Lastly, we comment on how this RL framework can be used in conjunction with physical quantum devices to tailor a code without explicit characterization of the noise model.
Abstract In the ongoing race towards experimental implementations of quantum error correction (QEC), finding ways to automatically discover codes and encoding strategies tailored to the qubit hardware platform is emerging as a critical problem. Reinforcement learning (RL) has been identified as a promising approach, but so far it has been severely restricted in terms of scalability. In this work, we significantly expand the power of RL approaches to QEC code discovery. Explicitly, we train an RL agent that automatically discovers both QEC codes and their encoding circuits for a given gate set, qubit connectivity and error model, from scratch. This is enabled by a reward based on the Knill-Laflamme conditions and a vectorized Clifford simulator, showing its effectiveness with up to 25 physical qubits and distance 5 codes, while presenting a roadmap to scale this approach to 100 qubits and distance 10 codes in the near future. We also introduce the concept of a noise-aware meta-agent, which learns to produce encoding strategies simultaneously for a range of noise models, thus leveraging transfer of insights between different situations. Our approach opens the door towards hardware-adapted accelerated discovery of QEC approaches across the full spectrum of quantum hardware platforms of interest.
The advent of promising quantum error correction (QEC) codes with efficient resource utilization and high-performance fault-tolerant quantum memories signifies a critical step towards realizing practical quantum computation. While surface codes have been a dominant approach, their limitations have spurred the development of more advanced QEC architectures. These advanced codes often present increased complexity, demanding innovative decoding methodologies. This work investigates the application of reinforcement learning (RL) techniques, including hybrid and multi-agent approaches, to enhance the decoding of various advanced QEC architectures. By leveraging the ability of RL to learn optimal strategies from noisy syndrome measurements, we explore the potential for achieving improved logical error rates and scalability compared to traditional decoding methods. Our approach examines the adaptation of reinforcement learning to exploit the structural properties of these modern QEC models. We also explore the benefits of combining different RL algorithms to address the multifaceted nature of the decoding problem, considering factors such as code degeneracy and real-world noise characteristics. With our proposed method, we are able to demonstrate that an autonomously trained agent can derive decoding schemes for the complex decoding requirement of advanced QEC architectures.
The realization of scalable fault-tolerant quantum computing is expected to hinge on quantum error-correcting codes. In the quest for more efficient quantum fault tolerance, a critical code parameter is the weight of measurements that extract information about errors to enable error correction: as higher measurement weights require higher implementation costs and introduce more errors, it is important in code design to optimize measurement weight. This underlies the surging interest in quantum low-density parity-check (qLDPC) codes, the study of which has primarily focused on the asymptotic (large-code-limit) properties. In this work, we introduce a versatile and computationally efficient approach to stabilizer code weight reduction based on reinforcement learning (RL), which produces new low-weight codes that substantially outperform the state of the art in practically relevant parameter regimes, extending significantly beyond previously accessible small distances. For example, our approach demonstrates savings in physical qubit overhead compared to existing results by 1 to 2 orders of magnitude for weight 6 codes and brings the overhead into a feasible range for near-future experiments. We also investigate the interplay between code parameters using our RL framework, offering new insights into the potential efficiency and power of practically viable coding strategies. Overall, our results demonstrate how RL can effectively advance the crucial yet challenging problem of quantum code discovery and thereby facilitate a faster path to the practical implementation of fault-tolerant quantum technologies.
Quantum error correction is widely thought to be the key to fault-tolerant quantum computation. However, determining the most suited encoding for unknown error channels or specific laboratory setups is highly challenging. Here, we present a reinforcement learning framework for optimizing and fault-tolerantly adapting quantum error correction codes. We consider a reinforcement learning agent tasked with modifying a family of surface code quantum memories until a desired logical error rate is reached. Using efficient simulations with about 70 data qubits with arbitrary connectivity, we demonstrate that such a reinforcement learning agent can determine near-optimal solutions, in terms of the number of data qubits, for various error models of interest. Moreover, we show that agents trained on one setting are able to successfully transfer their experience to different settings. This ability for transfer learning showcases the inherent strengths of reinforcement learning and the applicability of our approach for optimization from off-line simulations to on-line laboratory settings.
Quantum error correction (QEC) is the primary strategy for protecting a quantum computer from the environment. Its prerequisite is that errors must remain sufficiently rare, which requires perpetually adapting the computer's control parameters to the drifting environment conditions. The current solution to this problem is to terminate the entire quantum computation for recalibration, but it is incompatible with the long runtimes of future quantum algorithms. We address this challenge by unifying calibration with computation. We grant the QEC process a dual role: its error detection events are not only used to correct the logical quantum state, but are also repurposed as a learning signal, teaching a reinforcement learning (RL) agent to continuously steer the control parameters and stabilize the quantum system during computation. We experimentally demonstrate this framework on a Willow superconducting processor, improving the logical stability of the surface code 3.5-fold against injected drift. By synthesizing our full suite of technological advances, we achieve record performance of the surface and color codes, with average logical error per cycle of 7.72(9)×10−47.72(9)\times10^{-4}7.72(9)×10−4 and 8.19(14)×10−38.19(14)\times10^{-3}8.19(14)×10−3 respectively. Numerical simulations of large codes with tens of thousands of control parameters confirm the scalability of our RL framework, revealing an optimization speed that is independent of system size. This work thus enables a new paradigm: a quantum computer that learns from its errors and never stops computing.