Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2311.04750 · npj Quantum Information

Simultaneous discovery of quantum error correction codes and encoders with a noise-aware reinforcement learning agent

Jan Ollé, Remmy A. M. Zen, Matteo Ballerini Puviani, Florian Marquardt

Published Dec 3, 2024• 24 citations
arXivHTMLPDF

Abstract

Abstract In the ongoing race towards experimental implementations of quantum error correction (QEC), finding ways to automatically discover codes and encoding strategies tailored to the qubit hardware platform is emerging as a critical problem. Reinforcement learning (RL) has been identified as a promising approach, but so far it has been severely restricted in terms of scalability. In this work, we significantly expand the power of RL approaches to QEC code discovery. Explicitly, we train an RL agent that automatically discovers both QEC codes and their encoding circuits for a given gate set, qubit connectivity and error model, from scratch. This is enabled by a reward based on the Knill-Laflamme conditions and a vectorized Clifford simulator, showing its effectiveness with up to 25 physical qubits and distance 5 codes, while presenting a roadmap to scale this approach to 100 qubits and distance 10 codes in the near future. We also introduce the concept of a noise-aware meta-agent, which learns to produce encoding strategies simultaneously for a range of noise models, thus leveraging transfer of insights between different situations. Our approach opens the door towards hardware-adapted accelerated discovery of QEC approaches across the full spectrum of quantum hardware platforms of interest.

DOI
10.1038/s41534-024-00920-y

Related papers

Accelerating searching for quantum error correction codes with reinforcement learning

Evan E. Dobbs

Quantum Error Correction (QEC) codes are a necessary component of utility-scale quantum computers, and the discovery of new codes with desirable properties is one of the most promising avenues for reducing the estimated resources required to perform useful quantum computations. Most current methods for code discovery rely on analytical techniques and construct codes from algebraic or geometric structures, but there also exist some techniques for discovering codes via computational search. In this work we focus on one of the most promising of these methods where a reinforcement learning agent is trained to construct encoding circuits and evaluates the performance of the corresponding code. We seek to improve upon this method by reducing the computational resources required to discover codes of given parameters, primarily by utilizing a probabilistic approximation of the reward function which relies on randomly sampling error operators according to some specified distribution. We investigate the performance of our modified framework and compare the discovered codes and required computational resources to the original work, concluding that probabilistic methods such as our sampling technique are effective strategies for reducing the computational cost of the evaluation component of code searching without excessive penalties to performance.

•Dec 31, 2025•0 citations

Scaling the Automated Discovery of Quantum Circuits via Reinforcement Learning with Gadgets

Jan Ollé, Oleg M. Yevtushenko, Florian Marquardt

Reinforcement Learning (RL) has established itself as a powerful tool for designing quantum circuits, which are essential for processing quantum information. RL applications have typically focused on circuits of small to intermediate complexity, as computation times tend to increase exponentially with growing circuit complexity. This computational explosion severely limits the scalability of RL and casts significant doubt on its broader applicability. In this paper, we propose a principled approach based on the systematic discovery and introduction of composite gates -- {\it gadgets}, that enables RL scalability, thereby expanding its potential applications. As a case study, we explore the discovery of Clifford encoders for Quantum Error Correction. We demonstrate that incorporating gadgets in the form of composite Clifford gates, in addition to standard CNOT and Hadamard gates, significantly enhances the efficiency of RL agents. Specifically, the computation speed increases (by one or even two orders of magnitude), enabling RL to discover highly complex quantum codes without previous knowledge. We illustrate this advancement with examples of QEC code discovery with parameters $ [[n,1,d]] $ for $ d \leq 7 $ and $ [[n,k,6]] $ for $ k \leq 7 $. We note that the most complicated circuits of these classes were not previously found. We highlight the advantages and limitations of the gadget-based approach. Our method paves the way for scaling the RL-based automatic discovery of complicated quantum circuits for various tasks, which may include designing logical operations between logical qubits or discovering quantum algorithms.

arXiv:2503.11638•Mar 14, 2025•1 citation

Reinforcement Learning for Enhanced Advanced QEC Architecture Decoding

Yidong Zhou, Lingyi Kong, Yifeng Peng +1 more

The advent of promising quantum error correction (QEC) codes with efficient resource utilization and high-performance fault-tolerant quantum memories signifies a critical step towards realizing practical quantum computation. While surface codes have been a dominant approach, their limitations have spurred the development of more advanced QEC architectures. These advanced codes often present increased complexity, demanding innovative decoding methodologies. This work investigates the application of reinforcement learning (RL) techniques, including hybrid and multi-agent approaches, to enhance the decoding of various advanced QEC architectures. By leveraging the ability of RL to learn optimal strategies from noisy syndrome measurements, we explore the potential for achieving improved logical error rates and scalability compared to traditional decoding methods. Our approach examines the adaptation of reinforcement learning to exploit the structural properties of these modern QEC models. We also explore the benefits of combining different RL algorithms to address the multifaceted nature of the decoding problem, considering factors such as code degeneracy and real-world noise characteristics. With our proposed method, we are able to demonstrate that an autonomously trained agent can derive decoding schemes for the complex decoding requirement of advanced QEC architectures.

quant-pharXiv:2601.19279•Jan 28, 2026•0 citations

Optimizing Quantum Error Correction Codes with Reinforcement Learning

Hendrik Poulsen Nautrup, Nicolas Delfosse, Vedran Dunjko +2 more

Quantum error correction is widely thought to be the key to fault-tolerant quantum computation. However, determining the most suited encoding for unknown error channels or specific laboratory setups is highly challenging. Here, we present a reinforcement learning framework for optimizing and fault-tolerantly adapting quantum error correction codes. We consider a reinforcement learning agent tasked with modifying a family of surface code quantum memories until a desired logical error rate is reached. Using efficient simulations with about 70 data qubits with arbitrary connectivity, we demonstrate that such a reinforcement learning agent can determine near-optimal solutions, in terms of the number of data qubits, for various error models of interest. Moreover, we show that agents trained on one setting are able to successfully transfer their experience to different settings. This ability for transfer learning showcases the inherent strengths of reinforcement learning and the applicability of our approach for optimization from off-line simulations to on-line laboratory settings.

arXiv:1812.08451•Quantum•Dec 16, 2019•162 citations

Reinforcement Learning–Based Quantum Error Correction for a Five-Qubit Quantum System

A.SUBRAMANYAM, B.L.KRISHNA, S.PAVITHRA, CH.H.SAI G.RAJESH CHANDRA

Quantum error correction (QEC) is needed for building reliable and scalable quantum computers because of the susceptibility of quantum states to decoherence and operational noise. Reinforcement learning (RL) has recently emerged as an adaptive approach for quantum control and error mitiga- tion;however, existing RL-based QEC methods are largely re- stricted to small qubit systems and are often dependent on specific hardware architectures. We present a RL–enabled QEC ap- proach for a five qubit quantum system. The proposed approach relies exclusively on syndrome measurements and reward feed- back, ensuring architecture independence and portability across quantum platforms. The framework is evaluated using Qiskit and Cirq simulators under realistic noise models. Experimental results demonstrate improved logical qubit lifetimes and reduced error rates compared to static and smaller qubit correction strategies.

International Journal for Research In Science & Advanced Technologies•Apr 20, 2026•0 citations

Discovering autonomous quantum error correction via deep reinforcement learning

Yue Yin, Tailong Xiao, Xiaoyang Deng +3 more

Quantum error correction is essential for fault-tolerant quantum computing. However, standard methods relying on active measurements may introduce additional errors. Autonomous quantum error correction (AQEC) circumvents this by utilizing engineered dissipation and drives in bosonic systems, but identifying practical encoding remains challenging due to stringent Knill-Laflamme conditions. In this work, we utilize curriculum learning enabled deep reinforcement learning to discover Bosonic codes under approximate AQEC framework to resist both single-photon and double-photon losses. We present an analytical solution of solving the master equation under approximation conditions, which can significantly accelerate the training process of reinforcement learning. The agent first identifies an encoded subspace surpassing the breakeven point through rapid exploration within a constrained evolutionary time-frame, then strategically fine-tunes its policy to sustain this performance advantage over extended temporal horizons. We find that the two-phase trained agent can discover the optimal set of codewords, i.e., the Fock states $\ket{4}$ and $\ket{7}$ considering the effect of both single-photon and double-photon loss. We identify that the discovered code surpasses the breakeven threshold over a longer evolution time and achieve the state-of-art performance. We also analyze the robustness of the code against the phase damping and amplitude damping noise. Our work highlights the potential of curriculum learning enabled deep reinforcement learning in discovering the optimal quantum error correct code especially in early fault-tolerant quantum systems.

quant-pharXiv:2511.12482•Physical Review A•Nov 18, 2025•0 citations
Abstract In the ongoing race towards experimental implementations of quantum error correction (QEC), finding ways to automatically discover codes and encoding strategies tailored to the qubit hardware platform is emerging as a critical problem. Reinforcement learning (RL) has been identified as a promising approach, but so far it has been severely restricted in terms of scalability. In this work, we significantly expand the power of RL approaches to QEC code discovery. Explicitly, we train an RL agent that automatically discovers both QEC codes and their encoding circuits for a given gate set, qubit connectivity and error model, from scratch. This is enabled by a reward based on the Knill-Laflamme conditions and a vectorized Clifford simulator, showing its effectiveness with up to 25 physical qubits and distance 5 codes, while presenting a roadmap to scale this approach to 100 qubits and distance 10 codes in the near future. We also introduce the concept of a noise-aware meta-agent, which learns to produce encoding strategies simultaneously for a range of noise models, thus leveraging transfer of insights between different situations. Our approach opens the door towards hardware-adapted accelerated discovery of QEC approaches across the full spectrum of quantum hardware platforms of interest.
Quantum Error Correction (QEC) codes are a necessary component of utility-scale quantum computers, and the discovery of new codes with desirable properties is one of the most promising avenues for reducing the estimated resources required to perform useful quantum computations. Most current methods for code discovery rely on analytical techniques and construct codes from algebraic or geometric structures, but there also exist some techniques for discovering codes via computational search. In this work we focus on one of the most promising of these methods where a reinforcement learning agent is trained to construct encoding circuits and evaluates the performance of the corresponding code. We seek to improve upon this method by reducing the computational resources required to discover codes of given parameters, primarily by utilizing a probabilistic approximation of the reward function which relies on randomly sampling error operators according to some specified distribution. We investigate the performance of our modified framework and compare the discovered codes and required computational resources to the original work, concluding that probabilistic methods such as our sampling technique are effective strategies for reducing the computational cost of the evaluation component of code searching without excessive penalties to performance.
Reinforcement Learning (RL) has established itself as a powerful tool for designing quantum circuits, which are essential for processing quantum information. RL applications have typically focused on circuits of small to intermediate complexity, as computation times tend to increase exponentially with growing circuit complexity. This computational explosion severely limits the scalability of RL and casts significant doubt on its broader applicability. In this paper, we propose a principled approach based on the systematic discovery and introduction of composite gates -- {\it gadgets}, that enables RL scalability, thereby expanding its potential applications. As a case study, we explore the discovery of Clifford encoders for Quantum Error Correction. We demonstrate that incorporating gadgets in the form of composite Clifford gates, in addition to standard CNOT and Hadamard gates, significantly enhances the efficiency of RL agents. Specifically, the computation speed increases (by one or even two orders of magnitude), enabling RL to discover highly complex quantum codes without previous knowledge. We illustrate this advancement with examples of QEC code discovery with parameters $ [[n,1,d]] $ for $ d \leq 7 $ and $ [[n,k,6]] $ for $ k \leq 7 $. We note that the most complicated circuits of these classes were not previously found. We highlight the advantages and limitations of the gadget-based approach. Our method paves the way for scaling the RL-based automatic discovery of complicated quantum circuits for various tasks, which may include designing logical operations between logical qubits or discovering quantum algorithms.
The advent of promising quantum error correction (QEC) codes with efficient resource utilization and high-performance fault-tolerant quantum memories signifies a critical step towards realizing practical quantum computation. While surface codes have been a dominant approach, their limitations have spurred the development of more advanced QEC architectures. These advanced codes often present increased complexity, demanding innovative decoding methodologies. This work investigates the application of reinforcement learning (RL) techniques, including hybrid and multi-agent approaches, to enhance the decoding of various advanced QEC architectures. By leveraging the ability of RL to learn optimal strategies from noisy syndrome measurements, we explore the potential for achieving improved logical error rates and scalability compared to traditional decoding methods. Our approach examines the adaptation of reinforcement learning to exploit the structural properties of these modern QEC models. We also explore the benefits of combining different RL algorithms to address the multifaceted nature of the decoding problem, considering factors such as code degeneracy and real-world noise characteristics. With our proposed method, we are able to demonstrate that an autonomously trained agent can derive decoding schemes for the complex decoding requirement of advanced QEC architectures.
Quantum error correction is widely thought to be the key to fault-tolerant quantum computation. However, determining the most suited encoding for unknown error channels or specific laboratory setups is highly challenging. Here, we present a reinforcement learning framework for optimizing and fault-tolerantly adapting quantum error correction codes. We consider a reinforcement learning agent tasked with modifying a family of surface code quantum memories until a desired logical error rate is reached. Using efficient simulations with about 70 data qubits with arbitrary connectivity, we demonstrate that such a reinforcement learning agent can determine near-optimal solutions, in terms of the number of data qubits, for various error models of interest. Moreover, we show that agents trained on one setting are able to successfully transfer their experience to different settings. This ability for transfer learning showcases the inherent strengths of reinforcement learning and the applicability of our approach for optimization from off-line simulations to on-line laboratory settings.
Quantum error correction (QEC) is needed for building reliable and scalable quantum computers because of the susceptibility of quantum states to decoherence and operational noise. Reinforcement learning (RL) has recently emerged as an adaptive approach for quantum control and error mitiga- tion;however, existing RL-based QEC methods are largely re- stricted to small qubit systems and are often dependent on specific hardware architectures. We present a RL–enabled QEC ap- proach for a five qubit quantum system. The proposed approach relies exclusively on syndrome measurements and reward feed- back, ensuring architecture independence and portability across quantum platforms. The framework is evaluated using Qiskit and Cirq simulators under realistic noise models. Experimental results demonstrate improved logical qubit lifetimes and reduced error rates compared to static and smaller qubit correction strategies.
Quantum error correction is essential for fault-tolerant quantum computing. However, standard methods relying on active measurements may introduce additional errors. Autonomous quantum error correction (AQEC) circumvents this by utilizing engineered dissipation and drives in bosonic systems, but identifying practical encoding remains challenging due to stringent Knill-Laflamme conditions. In this work, we utilize curriculum learning enabled deep reinforcement learning to discover Bosonic codes under approximate AQEC framework to resist both single-photon and double-photon losses. We present an analytical solution of solving the master equation under approximation conditions, which can significantly accelerate the training process of reinforcement learning. The agent first identifies an encoded subspace surpassing the breakeven point through rapid exploration within a constrained evolutionary time-frame, then strategically fine-tunes its policy to sustain this performance advantage over extended temporal horizons. We find that the two-phase trained agent can discover the optimal set of codewords, i.e., the Fock states ∣4⟩\ket{4}∣4⟩ and ∣7⟩\ket{7}∣7⟩ considering the effect of both single-photon and double-photon loss. We identify that the discovered code surpasses the breakeven threshold over a longer evolution time and achieve the state-of-art performance. We also analyze the robustness of the code against the phase damping and amplitude damping noise. Our work highlights the potential of curriculum learning enabled deep reinforcement learning in discovering the optimal quantum error correct code especially in early fault-tolerant quantum systems.