Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Decoding under Backlog Constraints as a Sequential Control Problem: A Deep Reinforcement Learning Approach for Quantum Error Correction

Hiroshi Tanaka, Anwar Ali, Muhammad Imran

Published Jul 21, 2026• 0 citations
Publisher

Abstract

The practical realization of fault-tolerant quantum computing relies fundamentally on the rapid and accurate decoding of quantum error-correcting codes. However, existing analyses of decoding algorithms often treat the problem as a sequence of independent, state-less optimization tasks, typically optimizing for a scalar metric such as logical error rate or average latency. In reality, the decoding process in a continuously operating quantum computer is tightly constrained by the physical rate of syndrome extraction and the finite capacity of the classical processing buffer, creating a non-trivial backlog dynamic. In this work, we present a rigorous mathematical framework that formulates the quantum decoding problem under strict backlog constraints as a discrete-time Markov Decision Process (MDP). By augmenting the state space to include both the real-time syndrome weight and the instantaneous buffer backlog, we shift the paradigm from one-dimensional thresholding to a multi-dimensional sequential control problem. We demonstrate mathematically and empirically that when the queue dynamics are explicitly modeled, the optimal policy diverges significantly from simple syndrome-weight thresholds. Furthermore, we employ Deep Q-Networks (DQN) to discover non-linear policies that inherently balance the trade-off between logical error suppression and queue stability. Our control-theoretic analysis reveals that incorporating state-space representations of the backlog allows RL agents to robustly prevent buffer overflow while maintaining near-optimal logical fidelity. Through extensive simulation on surface codes under circuit-level noise, our DQN agent strictly outperforms naive two-dimensional grid-search baselines, achieving a zero overflow rate in critical regimes where traditional thresholding fails catastrophically.

DOI
10.6084/m9.figshare.33047096

Related papers

Decoding under Backlog Constraints as a Sequential Control Problem: A Deep Reinforcement Learning Approach for Quantum Error Correction

Ali Anwar, Tanaka Hiroshi, Imran Muhammad

The practical realization of fault-tolerant quantum computing relies fundamentally on the rapid and accurate decoding of quantum error-correcting codes. However, existing analyses of decoding algorithms often treat the problem as a sequence of independent, state-less optimization tasks, typically optimizing for a scalar metric such as logical error rate or average latency. In reality, the decoding process in a continuously operating quantum computer is tightly constrained by the physical rate of syndrome extraction and the finite capacity of the classical processing buffer, creating a non-trivial backlog dynamic. In this work, we present a rigorous mathematical framework that formulates the quantum decoding problem under strict backlog constraints as a discrete-time Markov Decision Process (MDP). By augmenting the state space to include both the real-time syndrome weight and the instantaneous buffer backlog, we shift the paradigm from one-dimensional thresholding to a multi-dimensional sequential control problem. We demonstrate mathematically and empirically that when the queue dynamics are explicitly modeled, the optimal policy diverges significantly from simple syndrome-weight thresholds. Furthermore, we employ Deep Q-Networks (DQN) to discover non-linear policies that inherently balance the trade-off between logical error suppression and queue stability. Our control-theoretic analysis reveals that incorporating state-space representations of the backlog allows RL agents to robustly prevent buffer overflow while maintaining near-optimal logical fidelity. Through extensive simulation on surface codes under circuit-level noise, our DQN agent strictly outperforms naive two-dimensional grid-search baselines, achieving a zero overflow rate in critical regimes where traditional thresholding fails catastrophically.

•Jul 21, 2026•0 citations

Adaptive Quantum Error Correction for Surface Codes Using Deep Reinforcement Learning

Oluwasesan Adeleke, Ajmery Sultana

Quantum error correction (QEC) is indispensable for the realization of fault-tolerant quantum computers. Surface codes are a leading QEC architecture due to their high error thresholds and planar design. The core challenge lies in the decoding process: inferring latent physical errors from syndrome measurements and applying appropriate corrections. Although there are classical decoders, their performance can degrade with complex noise or be computationally demanding. This paper investigates an advanced Deep Reinforcement Learning (DRL) approach, employing a Dueling Double Deep Q-Network (DQN), to decode X-type Pauli errors on a$\mathbf{d}=\mathbf{3}$rotated surface code. The DRL agent leverages syndrome history to discern temporal error correlations and utilizes a multi-round correction strategy, enabling iterative refinement of corrective actions within an environment that simulates both physical qubit errors and syndrome measurement noise. After 30,000 training episodes with a 5 % physical error rate and 1 % measurement noise, the agent achieved a 0.00 % logical error rate in the extended test, an average episode reward of 0.38, and successfully cleared syndromes in 63 % of the test cases, demonstrating robust learning of a safe decoding policy. The persistent challenge of high training loss, despite good policy performance, is discussed, offering insights into value function approximation in complex QEC environments.

IEEE International Conference on Quantum Photonics, Artificial Intelligence & Networking (QPAIN)•Apr 16, 2026•0 citations

Hardware-in-the-Loop Reinforcement Learning for Quantum Error Correction: A Systems-Engineering Approach to Latency-Aware Decoding

Hiroshi Tanaka, Kenji Yamamoto, Takeshi Saito

The scalability of fault-tolerant quantum computing is strictly bounded by the performance and latency of Quantum Error Correction (QEC) decoding systems. While Reinforcement Learning (RL) has emerged as a promising approach for deriving optimal decoding policies, current Deep Q-Network (DQN) agents are overwhelmingly trained using idealized software simulators. These simulators employ deterministic latency models that drastically fail to capture the stochastic load noise, memory contention, thermal throttling, and bus jitter inherent to real-world hardware deployment on Field-Programmable Gate Arrays (FPGAs). In this paper, we propose a comprehensive Hardware-in-the-Loop (HIL) Reinforcement Learning architecture that physically situates the FPGA in the RL environment feedback loop. By directly offloading syndrome processing jobs to the FPGA hardware during training, the DQN agent observes the exact microarchitectural execution latency as its environmental reward. We formulate a robust, systems-engineering oriented framework that captures true edge-deployment constraints. Our extensive HIL empirical results demonstrate that agents trained against true hardware telemetry develop distinct decoding strategies that actively avoid routing congestion in the FPGA fabric, yielding up to a $42\%$ reduction in long-tail latency events and improving the effective logical error rate under strict real-time decoding constraints.

•Jul 21, 2026•0 citations

Hardware-in-the-Loop Reinforcement Learning for Quantum Error Correction: A Systems-Engineering Approach to Latency-Aware Decoding

Yamamoto Kenji, Tanaka Hiroshi, Takeshi Saito

The scalability of fault-tolerant quantum computing is strictly bounded by the performance and latency of Quantum Error Correction (QEC) decoding systems. While Reinforcement Learning (RL) has emerged as a promising approach for deriving optimal decoding policies, current Deep Q-Network (DQN) agents are overwhelmingly trained using idealized software simulators. These simulators employ deterministic latency models that drastically fail to capture the stochastic load noise, memory contention, thermal throttling, and bus jitter inherent to real-world hardware deployment on Field-Programmable Gate Arrays (FPGAs). In this paper, we propose a comprehensive Hardware-in-the-Loop (HIL) Reinforcement Learning architecture that physically situates the FPGA in the RL environment feedback loop. By directly offloading syndrome processing jobs to the FPGA hardware during training, the DQN agent observes the exact microarchitectural execution latency as its environmental reward. We formulate a robust, systems-engineering oriented framework that captures true edge-deployment constraints. Our extensive HIL empirical results demonstrate that agents trained against true hardware telemetry develop distinct decoding strategies that actively avoid routing congestion in the FPGA fabric, yielding up to a $42\%$ reduction in long-tail latency events and improving the effective logical error rate under strict real-time decoding constraints.

•Jul 21, 2026•0 citations

Adaptive Control of Stochastic Error Accumulation in Fault-Tolerant Quantum Computation

Tirtha Haque

In realistic hardware for quantum computation that possesses fault-tolerance, non-stationary noise and stochastic drift lead to logical failure from the temporal accumulation of errors, not from independent events. Static decoding and fixed calibration techniques are structurally incompatible with this situation because they do not take into account temporal correlations between errors or control-induced back-action of errors. These effects motivate control policies that must track noise evolution across correction cycles, rather than respond to individual syndromes in isolation. We treat fault-tolerant quantum computation as a stochastic control problem, modelled using reduced quantum dynamics in which Pauli error processes are governed by latent noise parameters that vary temporally. From this perspective, logical failure arises through the accumulation of a hazard variable, and the corresponding control objective depends on the full history of observations. Operating under these conditions, a Chronological Deep Q-Network (Ch-DQN) maintains an internal belief state that tracks both noise evolution and accumulated hazard. During training, backward refinement of trajectories is used to sample slowly drifting modes of operation, while runtime inference remains strictly causal. A fractional meta-update stabilizes learning in the presence of non-stationary, control-coupled dynamics. Through multi-distance simulations that incorporate stochastic drift and feedback from decision-making, Ch-DQN suppresses hazard accumulation and extends logical survival time relative to static and recurrent baselines. Error correction in this regime is therefore no longer a static decoding task, but a control process whose success is determined over time by the underlying noise dynamics.

quant-pharXiv:2603.14687•Mar 17, 2026•0 citations

Transfer Learning in Reinforcement Learning for Quantum Error Correction: Evaluating Scale-Invariant Decoding of Quantum LDPC Codes

Zhang Shuhan, Fan Yang, Tanaka Hiroshi +2 more

The realization of fault-tolerant quantum computing relies fundamentally on the efficient and accurate decoding of quantum error-correcting codes. Recently, Deep Reinforcement Learning (DRL) has emerged as a promising approach for decoding complex codes, such as quantum Low-Density Parity-Check (qLDPC) codes, due to its ability to adapt to correlated noise and system specific irregularities. However, the computational cost of training these intelligent agents scales prohibitively with the code distance and the number of physical qubits. In this work, we present an analytical and theoretical framework for transfer learning in DRL-based quantum decoders. We explore the scale-invariant properties of qLDPC codes, specifically focusing on how the structural symmetries in hypergraph product and lifted product codes can be exploited by scalable neural architectures. By formulating the decoding process as a Markov Decision Process (MDP) mapped onto a translationally and structurally invariant graph, we demonstrate that a Deep Q-Network (DQN) trained on a small lattice (e.g., an $[[625, 25]]$ qLDPC code) can be zero-shot transferred or rapidly fine-tuned for a significantly larger lattice (e.g., an $[[900, 50]]$ qLDPC code). We provide rigorous theoretical bounds on the performance degradation under dimensionality mismatch and present comprehensive empirical results. Our experiments show that fine-tuning a pre-trained agent requires orders of magnitude fewer samples than training from scratch, while retaining near-optimal logical error suppression. These findings provide a concrete pathway toward scalable, low-latency machine learning architectures for large-scale fault-tolerant quantum processors.

•Jul 21, 2026•0 citations
The practical realization of fault-tolerant quantum computing relies fundamentally on the rapid and accurate decoding of quantum error-correcting codes. However, existing analyses of decoding algorithms often treat the problem as a sequence of independent, state-less optimization tasks, typically optimizing for a scalar metric such as logical error rate or average latency. In reality, the decoding process in a continuously operating quantum computer is tightly constrained by the physical rate of syndrome extraction and the finite capacity of the classical processing buffer, creating a non-trivial backlog dynamic. In this work, we present a rigorous mathematical framework that formulates the quantum decoding problem under strict backlog constraints as a discrete-time Markov Decision Process (MDP). By augmenting the state space to include both the real-time syndrome weight and the instantaneous buffer backlog, we shift the paradigm from one-dimensional thresholding to a multi-dimensional sequential control problem. We demonstrate mathematically and empirically that when the queue dynamics are explicitly modeled, the optimal policy diverges significantly from simple syndrome-weight thresholds. Furthermore, we employ Deep Q-Networks (DQN) to discover non-linear policies that inherently balance the trade-off between logical error suppression and queue stability. Our control-theoretic analysis reveals that incorporating state-space representations of the backlog allows RL agents to robustly prevent buffer overflow while maintaining near-optimal logical fidelity. Through extensive simulation on surface codes under circuit-level noise, our DQN agent strictly outperforms naive two-dimensional grid-search baselines, achieving a zero overflow rate in critical regimes where traditional thresholding fails catastrophically.
The practical realization of fault-tolerant quantum computing relies fundamentally on the rapid and accurate decoding of quantum error-correcting codes. However, existing analyses of decoding algorithms often treat the problem as a sequence of independent, state-less optimization tasks, typically optimizing for a scalar metric such as logical error rate or average latency. In reality, the decoding process in a continuously operating quantum computer is tightly constrained by the physical rate of syndrome extraction and the finite capacity of the classical processing buffer, creating a non-trivial backlog dynamic. In this work, we present a rigorous mathematical framework that formulates the quantum decoding problem under strict backlog constraints as a discrete-time Markov Decision Process (MDP). By augmenting the state space to include both the real-time syndrome weight and the instantaneous buffer backlog, we shift the paradigm from one-dimensional thresholding to a multi-dimensional sequential control problem. We demonstrate mathematically and empirically that when the queue dynamics are explicitly modeled, the optimal policy diverges significantly from simple syndrome-weight thresholds. Furthermore, we employ Deep Q-Networks (DQN) to discover non-linear policies that inherently balance the trade-off between logical error suppression and queue stability. Our control-theoretic analysis reveals that incorporating state-space representations of the backlog allows RL agents to robustly prevent buffer overflow while maintaining near-optimal logical fidelity. Through extensive simulation on surface codes under circuit-level noise, our DQN agent strictly outperforms naive two-dimensional grid-search baselines, achieving a zero overflow rate in critical regimes where traditional thresholding fails catastrophically.
Quantum error correction (QEC) is indispensable for the realization of fault-tolerant quantum computers. Surface codes are a leading QEC architecture due to their high error thresholds and planar design. The core challenge lies in the decoding process: inferring latent physical errors from syndrome measurements and applying appropriate corrections. Although there are classical decoders, their performance can degrade with complex noise or be computationally demanding. This paper investigates an advanced Deep Reinforcement Learning (DRL) approach, employing a Dueling Double Deep Q-Network (DQN), to decode X-type Pauli errors on ad=3\mathbf{d}=\mathbf{3}d=3rotated surface code. The DRL agent leverages syndrome history to discern temporal error correlations and utilizes a multi-round correction strategy, enabling iterative refinement of corrective actions within an environment that simulates both physical qubit errors and syndrome measurement noise. After 30,000 training episodes with a 5 % physical error rate and 1 % measurement noise, the agent achieved a 0.00 % logical error rate in the extended test, an average episode reward of 0.38, and successfully cleared syndromes in 63 % of the test cases, demonstrating robust learning of a safe decoding policy. The persistent challenge of high training loss, despite good policy performance, is discussed, offering insights into value function approximation in complex QEC environments.
The scalability of fault-tolerant quantum computing is strictly bounded by the performance and latency of Quantum Error Correction (QEC) decoding systems. While Reinforcement Learning (RL) has emerged as a promising approach for deriving optimal decoding policies, current Deep Q-Network (DQN) agents are overwhelmingly trained using idealized software simulators. These simulators employ deterministic latency models that drastically fail to capture the stochastic load noise, memory contention, thermal throttling, and bus jitter inherent to real-world hardware deployment on Field-Programmable Gate Arrays (FPGAs). In this paper, we propose a comprehensive Hardware-in-the-Loop (HIL) Reinforcement Learning architecture that physically situates the FPGA in the RL environment feedback loop. By directly offloading syndrome processing jobs to the FPGA hardware during training, the DQN agent observes the exact microarchitectural execution latency as its environmental reward. We formulate a robust, systems-engineering oriented framework that captures true edge-deployment constraints. Our extensive HIL empirical results demonstrate that agents trained against true hardware telemetry develop distinct decoding strategies that actively avoid routing congestion in the FPGA fabric, yielding up to a 42%42\%42% reduction in long-tail latency events and improving the effective logical error rate under strict real-time decoding constraints.
The scalability of fault-tolerant quantum computing is strictly bounded by the performance and latency of Quantum Error Correction (QEC) decoding systems. While Reinforcement Learning (RL) has emerged as a promising approach for deriving optimal decoding policies, current Deep Q-Network (DQN) agents are overwhelmingly trained using idealized software simulators. These simulators employ deterministic latency models that drastically fail to capture the stochastic load noise, memory contention, thermal throttling, and bus jitter inherent to real-world hardware deployment on Field-Programmable Gate Arrays (FPGAs). In this paper, we propose a comprehensive Hardware-in-the-Loop (HIL) Reinforcement Learning architecture that physically situates the FPGA in the RL environment feedback loop. By directly offloading syndrome processing jobs to the FPGA hardware during training, the DQN agent observes the exact microarchitectural execution latency as its environmental reward. We formulate a robust, systems-engineering oriented framework that captures true edge-deployment constraints. Our extensive HIL empirical results demonstrate that agents trained against true hardware telemetry develop distinct decoding strategies that actively avoid routing congestion in the FPGA fabric, yielding up to a 42%42\%42% reduction in long-tail latency events and improving the effective logical error rate under strict real-time decoding constraints.
In realistic hardware for quantum computation that possesses fault-tolerance, non-stationary noise and stochastic drift lead to logical failure from the temporal accumulation of errors, not from independent events. Static decoding and fixed calibration techniques are structurally incompatible with this situation because they do not take into account temporal correlations between errors or control-induced back-action of errors. These effects motivate control policies that must track noise evolution across correction cycles, rather than respond to individual syndromes in isolation. We treat fault-tolerant quantum computation as a stochastic control problem, modelled using reduced quantum dynamics in which Pauli error processes are governed by latent noise parameters that vary temporally. From this perspective, logical failure arises through the accumulation of a hazard variable, and the corresponding control objective depends on the full history of observations. Operating under these conditions, a Chronological Deep Q-Network (Ch-DQN) maintains an internal belief state that tracks both noise evolution and accumulated hazard. During training, backward refinement of trajectories is used to sample slowly drifting modes of operation, while runtime inference remains strictly causal. A fractional meta-update stabilizes learning in the presence of non-stationary, control-coupled dynamics. Through multi-distance simulations that incorporate stochastic drift and feedback from decision-making, Ch-DQN suppresses hazard accumulation and extends logical survival time relative to static and recurrent baselines. Error correction in this regime is therefore no longer a static decoding task, but a control process whose success is determined over time by the underlying noise dynamics.
The realization of fault-tolerant quantum computing relies fundamentally on the efficient and accurate decoding of quantum error-correcting codes. Recently, Deep Reinforcement Learning (DRL) has emerged as a promising approach for decoding complex codes, such as quantum Low-Density Parity-Check (qLDPC) codes, due to its ability to adapt to correlated noise and system specific irregularities. However, the computational cost of training these intelligent agents scales prohibitively with the code distance and the number of physical qubits. In this work, we present an analytical and theoretical framework for transfer learning in DRL-based quantum decoders. We explore the scale-invariant properties of qLDPC codes, specifically focusing on how the structural symmetries in hypergraph product and lifted product codes can be exploited by scalable neural architectures. By formulating the decoding process as a Markov Decision Process (MDP) mapped onto a translationally and structurally invariant graph, we demonstrate that a Deep Q-Network (DQN) trained on a small lattice (e.g., an [[625,25]][[625, 25]][[625,25]] qLDPC code) can be zero-shot transferred or rapidly fine-tuned for a significantly larger lattice (e.g., an [[900,50]][[900, 50]][[900,50]] qLDPC code). We provide rigorous theoretical bounds on the performance degradation under dimensionality mismatch and present comprehensive empirical results. Our experiments show that fine-tuning a pre-trained agent requires orders of magnitude fewer samples than training from scratch, while retaining near-optimal logical error suppression. These findings provide a concrete pathway toward scalable, low-latency machine learning architectures for large-scale fault-tolerant quantum processors.