Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:1810.07207 · Machine Learning Science and Technology

Reinforcement learning decoders for fault-tolerant quantum computation

Ryan Sweke, Markus S Kesselring, Evert van Nieuwenburg, Jens Eisert

Published Oct 29, 2020• 92 citations
arXivHTMLPDF

Abstract

Abstract Topological error correcting codes, and particularly the surface code, currently provide the most feasible road-map towards large-scale fault-tolerant quantum computation. As such, obtaining fast and flexible decoding algorithms for these codes, within the experimentally realistic and challenging context of faulty syndrome measurements, without requiring any final read-out of the physical qubits, is of critical importance. In this work, we show that the problem of decoding such codes can be naturally reformulated as a process of repeated interactions between a decoding agent and a code environment, to which the machinery of reinforcement learning can be applied to obtain decoding agents. While in principle this framework can be instantiated with environments modelling circuit level noise, we take a first step towards this goal by using deepQ learning to obtain decoding agents for a variety of simplified phenomenological noise models, which yield faulty syndrome measurements without including the propagation of errors which arise in full circuit level noise models.

DOI
10.1088/2632-2153/abc609

Related papers

Reinforcement Learning for Enhanced Advanced QEC Architecture Decoding

Yidong Zhou, Lingyi Kong, Yifeng Peng +1 more

The advent of promising quantum error correction (QEC) codes with efficient resource utilization and high-performance fault-tolerant quantum memories signifies a critical step towards realizing practical quantum computation. While surface codes have been a dominant approach, their limitations have spurred the development of more advanced QEC architectures. These advanced codes often present increased complexity, demanding innovative decoding methodologies. This work investigates the application of reinforcement learning (RL) techniques, including hybrid and multi-agent approaches, to enhance the decoding of various advanced QEC architectures. By leveraging the ability of RL to learn optimal strategies from noisy syndrome measurements, we explore the potential for achieving improved logical error rates and scalability compared to traditional decoding methods. Our approach examines the adaptation of reinforcement learning to exploit the structural properties of these modern QEC models. We also explore the benefits of combining different RL algorithms to address the multifaceted nature of the decoding problem, considering factors such as code degeneracy and real-world noise characteristics. With our proposed method, we are able to demonstrate that an autonomously trained agent can derive decoding schemes for the complex decoding requirement of advanced QEC architectures.

quant-pharXiv:2601.19279•Jan 28, 2026•0 citations

Adaptive Quantum Error Correction for Surface Codes Using Deep Reinforcement Learning

Oluwasesan Adeleke, Ajmery Sultana

Quantum error correction (QEC) is indispensable for the realization of fault-tolerant quantum computers. Surface codes are a leading QEC architecture due to their high error thresholds and planar design. The core challenge lies in the decoding process: inferring latent physical errors from syndrome measurements and applying appropriate corrections. Although there are classical decoders, their performance can degrade with complex noise or be computationally demanding. This paper investigates an advanced Deep Reinforcement Learning (DRL) approach, employing a Dueling Double Deep Q-Network (DQN), to decode X-type Pauli errors on a$\mathbf{d}=\mathbf{3}$rotated surface code. The DRL agent leverages syndrome history to discern temporal error correlations and utilizes a multi-round correction strategy, enabling iterative refinement of corrective actions within an environment that simulates both physical qubit errors and syndrome measurement noise. After 30,000 training episodes with a 5 % physical error rate and 1 % measurement noise, the agent achieved a 0.00 % logical error rate in the extended test, an average episode reward of 0.38, and successfully cleared syndromes in 63 % of the test cases, demonstrating robust learning of a safe decoding policy. The persistent challenge of high training loss, despite good policy performance, is discussed, offering insights into value function approximation in complex QEC environments.

IEEE International Conference on Quantum Photonics, Artificial Intelligence & Networking (QPAIN)•Apr 16, 2026•0 citations

DQN-Based Decoder for depolarizing noise on Rotated Surface Code

Ziyao Zhao

In quantum fault-tolerant computing, fast decoding of encoded bits is related to the running speed of quantum computers. In this paper, we propose a decoding agent based on reinforcement learning for correcting rotated surface codes in quantum error correction. The intelligent agent is trained using Deep Q-Network (DQN) and uses Convolutional Neural Network (CNN) to extract features of error syndrome and calculate the corresponding Q values for X, Y, and Z error correction operations. In the environment of depolarizing noise, the trained model achieves a fault-tolerant threshold that is superior to the Minimum Weight Perfect Matching (MWPM) and CNN decoder, providing a feasible framework for surface code error correction in practical applications.

International Conference on Next Generation Data-driven Networks (NGDN)•Jun 6, 2025•0 citations

Compound Error Modeling and Synthesis in Deep Reinforcement Learning for Quantum Error Correction

Liu Ming, Tanaka Hiroshi, Wang Jian

The practical realization of fault-tolerant quantum computing depends critically on the efficiency and adaptability of quantum error correction (QEC) decoders. Recent advancements have highlighted deep reinforcement learning (DRL) as a promising approach for decoding topological codes. However, training these agents typically assumes simplified, uniform noise models, which fail to capture the complex, compounding errors characteristic of near-term quantum hardware. In this paper, we rigorously formulate the synthesis of compound error models—encompassing thermal relaxation, spatial cross-talk, and non-Markovian memory effects—within the framework of deep reinforcement learning for surface code decoding. We propose a methodological framework for simulating these mismatch environments during training to foster robust policy generalization. Extensive simulations demonstrate that DRL agents trained on our synthesized compound error models achieve up to a $24\%$ reduction in logical error rates under realistic noise conditions compared to agents trained on standard depolarizing noise. This work provides a rigorous foundation for synthesizing accurate quantum noise environments for reinforcement learning applications.

•Jul 21, 2026•0 citations

Compound Error Modeling and Synthesis in Deep Reinforcement Learning for Quantum Error Correction

Hiroshi Tanaka, Ming Liu, Jian Wang

The practical realization of fault-tolerant quantum computing depends critically on the efficiency and adaptability of quantum error correction (QEC) decoders. Recent advancements have highlighted deep reinforcement learning (DRL) as a promising approach for decoding topological codes. However, training these agents typically assumes simplified, uniform noise models, which fail to capture the complex, compounding errors characteristic of near-term quantum hardware. In this paper, we rigorously formulate the synthesis of compound error models—encompassing thermal relaxation, spatial cross-talk, and non-Markovian memory effects—within the framework of deep reinforcement learning for surface code decoding. We propose a methodological framework for simulating these mismatch environments during training to foster robust policy generalization. Extensive simulations demonstrate that DRL agents trained on our synthesized compound error models achieve up to a $24\%$ reduction in logical error rates under realistic noise conditions compared to agents trained on standard depolarizing noise. This work provides a rigorous foundation for synthesizing accurate quantum noise environments for reinforcement learning applications.

•Jul 21, 2026•0 citations

Fooling the Decoder: An Adversarial Attack on Quantum Error Correction

Jerome Lenssen, Alexandru Paler

Neural network decoders are essential for fault-tolerant quantum computation, yet their internal mechanisms remain poorly understood. Leading machine learning decoders use recurrent and transformer models (e.g., AlphaQubit), and reinforcement learning (RL) is increasingly used to train such models. In this work we ask whether an RL surface code decoder, DeepQ, can be made to fail systematically. Our white-box search repeatedly draws candidate syndrome volumes from the unmodified noise model and retains the one the decoder's learned $Q$-function is most pessimistic about. At code distance 5 under phenomenological depolarising noise, it cuts the average logical qubit lifetime from roughly $10^5$ cycles to 60. We quantify what this costs: the retained error histories are individually plausible, but the selected trajectory is roughly a $10^{-18}$ tail event under undisturbed noise, so the effect is bought entirely by the ability to select among physically realisable error configurations. Controls confirm this is not an artefact of a weak target: the decoder resists noise fluctuations, tolerates an MWPM referee, and is fault tolerant at checkable distances. The result is a robustness upper bound for one decoder under one noise model, but to our knowledge the first adversarial attack of a machine learning QEC decoder.

arXiv:2504.19651•Apr 28, 2025•0 citations
Abstract Topological error correcting codes, and particularly the surface code, currently provide the most feasible road-map towards large-scale fault-tolerant quantum computation. As such, obtaining fast and flexible decoding algorithms for these codes, within the experimentally realistic and challenging context of faulty syndrome measurements, without requiring any final read-out of the physical qubits, is of critical importance. In this work, we show that the problem of decoding such codes can be naturally reformulated as a process of repeated interactions between a decoding agent and a code environment, to which the machinery of reinforcement learning can be applied to obtain decoding agents. While in principle this framework can be instantiated with environments modelling circuit level noise, we take a first step towards this goal by using deepQ learning to obtain decoding agents for a variety of simplified phenomenological noise models, which yield faulty syndrome measurements without including the propagation of errors which arise in full circuit level noise models.
The advent of promising quantum error correction (QEC) codes with efficient resource utilization and high-performance fault-tolerant quantum memories signifies a critical step towards realizing practical quantum computation. While surface codes have been a dominant approach, their limitations have spurred the development of more advanced QEC architectures. These advanced codes often present increased complexity, demanding innovative decoding methodologies. This work investigates the application of reinforcement learning (RL) techniques, including hybrid and multi-agent approaches, to enhance the decoding of various advanced QEC architectures. By leveraging the ability of RL to learn optimal strategies from noisy syndrome measurements, we explore the potential for achieving improved logical error rates and scalability compared to traditional decoding methods. Our approach examines the adaptation of reinforcement learning to exploit the structural properties of these modern QEC models. We also explore the benefits of combining different RL algorithms to address the multifaceted nature of the decoding problem, considering factors such as code degeneracy and real-world noise characteristics. With our proposed method, we are able to demonstrate that an autonomously trained agent can derive decoding schemes for the complex decoding requirement of advanced QEC architectures.
Quantum error correction (QEC) is indispensable for the realization of fault-tolerant quantum computers. Surface codes are a leading QEC architecture due to their high error thresholds and planar design. The core challenge lies in the decoding process: inferring latent physical errors from syndrome measurements and applying appropriate corrections. Although there are classical decoders, their performance can degrade with complex noise or be computationally demanding. This paper investigates an advanced Deep Reinforcement Learning (DRL) approach, employing a Dueling Double Deep Q-Network (DQN), to decode X-type Pauli errors on ad=3\mathbf{d}=\mathbf{3}d=3rotated surface code. The DRL agent leverages syndrome history to discern temporal error correlations and utilizes a multi-round correction strategy, enabling iterative refinement of corrective actions within an environment that simulates both physical qubit errors and syndrome measurement noise. After 30,000 training episodes with a 5 % physical error rate and 1 % measurement noise, the agent achieved a 0.00 % logical error rate in the extended test, an average episode reward of 0.38, and successfully cleared syndromes in 63 % of the test cases, demonstrating robust learning of a safe decoding policy. The persistent challenge of high training loss, despite good policy performance, is discussed, offering insights into value function approximation in complex QEC environments.
In quantum fault-tolerant computing, fast decoding of encoded bits is related to the running speed of quantum computers. In this paper, we propose a decoding agent based on reinforcement learning for correcting rotated surface codes in quantum error correction. The intelligent agent is trained using Deep Q-Network (DQN) and uses Convolutional Neural Network (CNN) to extract features of error syndrome and calculate the corresponding Q values for X, Y, and Z error correction operations. In the environment of depolarizing noise, the trained model achieves a fault-tolerant threshold that is superior to the Minimum Weight Perfect Matching (MWPM) and CNN decoder, providing a feasible framework for surface code error correction in practical applications.
The practical realization of fault-tolerant quantum computing depends critically on the efficiency and adaptability of quantum error correction (QEC) decoders. Recent advancements have highlighted deep reinforcement learning (DRL) as a promising approach for decoding topological codes. However, training these agents typically assumes simplified, uniform noise models, which fail to capture the complex, compounding errors characteristic of near-term quantum hardware. In this paper, we rigorously formulate the synthesis of compound error models—encompassing thermal relaxation, spatial cross-talk, and non-Markovian memory effects—within the framework of deep reinforcement learning for surface code decoding. We propose a methodological framework for simulating these mismatch environments during training to foster robust policy generalization. Extensive simulations demonstrate that DRL agents trained on our synthesized compound error models achieve up to a 24%24\%24% reduction in logical error rates under realistic noise conditions compared to agents trained on standard depolarizing noise. This work provides a rigorous foundation for synthesizing accurate quantum noise environments for reinforcement learning applications.
The practical realization of fault-tolerant quantum computing depends critically on the efficiency and adaptability of quantum error correction (QEC) decoders. Recent advancements have highlighted deep reinforcement learning (DRL) as a promising approach for decoding topological codes. However, training these agents typically assumes simplified, uniform noise models, which fail to capture the complex, compounding errors characteristic of near-term quantum hardware. In this paper, we rigorously formulate the synthesis of compound error models—encompassing thermal relaxation, spatial cross-talk, and non-Markovian memory effects—within the framework of deep reinforcement learning for surface code decoding. We propose a methodological framework for simulating these mismatch environments during training to foster robust policy generalization. Extensive simulations demonstrate that DRL agents trained on our synthesized compound error models achieve up to a 24%24\%24% reduction in logical error rates under realistic noise conditions compared to agents trained on standard depolarizing noise. This work provides a rigorous foundation for synthesizing accurate quantum noise environments for reinforcement learning applications.
Neural network decoders are essential for fault-tolerant quantum computation, yet their internal mechanisms remain poorly understood. Leading machine learning decoders use recurrent and transformer models (e.g., AlphaQubit), and reinforcement learning (RL) is increasingly used to train such models. In this work we ask whether an RL surface code decoder, DeepQ, can be made to fail systematically. Our white-box search repeatedly draws candidate syndrome volumes from the unmodified noise model and retains the one the decoder's learned QQQ-function is most pessimistic about. At code distance 5 under phenomenological depolarising noise, it cuts the average logical qubit lifetime from roughly 10510^5105 cycles to 60. We quantify what this costs: the retained error histories are individually plausible, but the selected trajectory is roughly a 10−1810^{-18}10−18 tail event under undisturbed noise, so the effect is bought entirely by the ability to select among physically realisable error configurations. Controls confirm this is not an artefact of a weak target: the decoder resists noise fluctuations, tolerates an MWPM referee, and is fault tolerant at checkable distances. The result is a robustness upper bound for one decoder under one noise model, but to our knowledge the first adversarial attack of a machine learning QEC decoder.