Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Hardware-in-the-Loop Reinforcement Learning for Quantum Error Correction: A Systems-Engineering Approach to Latency-Aware Decoding

Yamamoto Kenji, Tanaka Hiroshi, Takeshi Saito

Published Jul 21, 2026• 0 citations
Publisher

Abstract

The scalability of fault-tolerant quantum computing is strictly bounded by the performance and latency of Quantum Error Correction (QEC) decoding systems. While Reinforcement Learning (RL) has emerged as a promising approach for deriving optimal decoding policies, current Deep Q-Network (DQN) agents are overwhelmingly trained using idealized software simulators. These simulators employ deterministic latency models that drastically fail to capture the stochastic load noise, memory contention, thermal throttling, and bus jitter inherent to real-world hardware deployment on Field-Programmable Gate Arrays (FPGAs). In this paper, we propose a comprehensive Hardware-in-the-Loop (HIL) Reinforcement Learning architecture that physically situates the FPGA in the RL environment feedback loop. By directly offloading syndrome processing jobs to the FPGA hardware during training, the DQN agent observes the exact microarchitectural execution latency as its environmental reward. We formulate a robust, systems-engineering oriented framework that captures true edge-deployment constraints. Our extensive HIL empirical results demonstrate that agents trained against true hardware telemetry develop distinct decoding strategies that actively avoid routing congestion in the FPGA fabric, yielding up to a $42\%$ reduction in long-tail latency events and improving the effective logical error rate under strict real-time decoding constraints.

DOI
10.5281/zenodo.21469637

Related papers

Reinforcement Learning for Enhanced Advanced QEC Architecture Decoding

Yidong Zhou, Lingyi Kong, Yifeng Peng +1 more

The advent of promising quantum error correction (QEC) codes with efficient resource utilization and high-performance fault-tolerant quantum memories signifies a critical step towards realizing practical quantum computation. While surface codes have been a dominant approach, their limitations have spurred the development of more advanced QEC architectures. These advanced codes often present increased complexity, demanding innovative decoding methodologies. This work investigates the application of reinforcement learning (RL) techniques, including hybrid and multi-agent approaches, to enhance the decoding of various advanced QEC architectures. By leveraging the ability of RL to learn optimal strategies from noisy syndrome measurements, we explore the potential for achieving improved logical error rates and scalability compared to traditional decoding methods. Our approach examines the adaptation of reinforcement learning to exploit the structural properties of these modern QEC models. We also explore the benefits of combining different RL algorithms to address the multifaceted nature of the decoding problem, considering factors such as code degeneracy and real-world noise characteristics. With our proposed method, we are able to demonstrate that an autonomously trained agent can derive decoding schemes for the complex decoding requirement of advanced QEC architectures.

quant-pharXiv:2601.19279•Jan 28, 2026•0 citations

Hardware-in-the-Loop Reinforcement Learning for Quantum Error Correction: A Systems-Engineering Approach to Latency-Aware Decoding

Hiroshi Tanaka, Kenji Yamamoto, Takeshi Saito

The scalability of fault-tolerant quantum computing is strictly bounded by the performance and latency of Quantum Error Correction (QEC) decoding systems. While Reinforcement Learning (RL) has emerged as a promising approach for deriving optimal decoding policies, current Deep Q-Network (DQN) agents are overwhelmingly trained using idealized software simulators. These simulators employ deterministic latency models that drastically fail to capture the stochastic load noise, memory contention, thermal throttling, and bus jitter inherent to real-world hardware deployment on Field-Programmable Gate Arrays (FPGAs). In this paper, we propose a comprehensive Hardware-in-the-Loop (HIL) Reinforcement Learning architecture that physically situates the FPGA in the RL environment feedback loop. By directly offloading syndrome processing jobs to the FPGA hardware during training, the DQN agent observes the exact microarchitectural execution latency as its environmental reward. We formulate a robust, systems-engineering oriented framework that captures true edge-deployment constraints. Our extensive HIL empirical results demonstrate that agents trained against true hardware telemetry develop distinct decoding strategies that actively avoid routing congestion in the FPGA fabric, yielding up to a $42\%$ reduction in long-tail latency events and improving the effective logical error rate under strict real-time decoding constraints.

•Jul 21, 2026•0 citations

Adaptive Quantum Error Correction for Surface Codes Using Deep Reinforcement Learning

Oluwasesan Adeleke, Ajmery Sultana

Quantum error correction (QEC) is indispensable for the realization of fault-tolerant quantum computers. Surface codes are a leading QEC architecture due to their high error thresholds and planar design. The core challenge lies in the decoding process: inferring latent physical errors from syndrome measurements and applying appropriate corrections. Although there are classical decoders, their performance can degrade with complex noise or be computationally demanding. This paper investigates an advanced Deep Reinforcement Learning (DRL) approach, employing a Dueling Double Deep Q-Network (DQN), to decode X-type Pauli errors on a$\mathbf{d}=\mathbf{3}$rotated surface code. The DRL agent leverages syndrome history to discern temporal error correlations and utilizes a multi-round correction strategy, enabling iterative refinement of corrective actions within an environment that simulates both physical qubit errors and syndrome measurement noise. After 30,000 training episodes with a 5 % physical error rate and 1 % measurement noise, the agent achieved a 0.00 % logical error rate in the extended test, an average episode reward of 0.38, and successfully cleared syndromes in 63 % of the test cases, demonstrating robust learning of a safe decoding policy. The persistent challenge of high training loss, despite good policy performance, is discussed, offering insights into value function approximation in complex QEC environments.

IEEE International Conference on Quantum Photonics, Artificial Intelligence & Networking (QPAIN)•Apr 16, 2026•0 citations

Decoding under Backlog Constraints as a Sequential Control Problem: A Deep Reinforcement Learning Approach for Quantum Error Correction

Hiroshi Tanaka, Anwar Ali, Muhammad Imran

The practical realization of fault-tolerant quantum computing relies fundamentally on the rapid and accurate decoding of quantum error-correcting codes. However, existing analyses of decoding algorithms often treat the problem as a sequence of independent, state-less optimization tasks, typically optimizing for a scalar metric such as logical error rate or average latency. In reality, the decoding process in a continuously operating quantum computer is tightly constrained by the physical rate of syndrome extraction and the finite capacity of the classical processing buffer, creating a non-trivial backlog dynamic. In this work, we present a rigorous mathematical framework that formulates the quantum decoding problem under strict backlog constraints as a discrete-time Markov Decision Process (MDP). By augmenting the state space to include both the real-time syndrome weight and the instantaneous buffer backlog, we shift the paradigm from one-dimensional thresholding to a multi-dimensional sequential control problem. We demonstrate mathematically and empirically that when the queue dynamics are explicitly modeled, the optimal policy diverges significantly from simple syndrome-weight thresholds. Furthermore, we employ Deep Q-Networks (DQN) to discover non-linear policies that inherently balance the trade-off between logical error suppression and queue stability. Our control-theoretic analysis reveals that incorporating state-space representations of the backlog allows RL agents to robustly prevent buffer overflow while maintaining near-optimal logical fidelity. Through extensive simulation on surface codes under circuit-level noise, our DQN agent strictly outperforms naive two-dimensional grid-search baselines, achieving a zero overflow rate in critical regimes where traditional thresholding fails catastrophically.

•Jul 21, 2026•0 citations

Decoding under Backlog Constraints as a Sequential Control Problem: A Deep Reinforcement Learning Approach for Quantum Error Correction

Ali Anwar, Tanaka Hiroshi, Imran Muhammad

The practical realization of fault-tolerant quantum computing relies fundamentally on the rapid and accurate decoding of quantum error-correcting codes. However, existing analyses of decoding algorithms often treat the problem as a sequence of independent, state-less optimization tasks, typically optimizing for a scalar metric such as logical error rate or average latency. In reality, the decoding process in a continuously operating quantum computer is tightly constrained by the physical rate of syndrome extraction and the finite capacity of the classical processing buffer, creating a non-trivial backlog dynamic. In this work, we present a rigorous mathematical framework that formulates the quantum decoding problem under strict backlog constraints as a discrete-time Markov Decision Process (MDP). By augmenting the state space to include both the real-time syndrome weight and the instantaneous buffer backlog, we shift the paradigm from one-dimensional thresholding to a multi-dimensional sequential control problem. We demonstrate mathematically and empirically that when the queue dynamics are explicitly modeled, the optimal policy diverges significantly from simple syndrome-weight thresholds. Furthermore, we employ Deep Q-Networks (DQN) to discover non-linear policies that inherently balance the trade-off between logical error suppression and queue stability. Our control-theoretic analysis reveals that incorporating state-space representations of the backlog allows RL agents to robustly prevent buffer overflow while maintaining near-optimal logical fidelity. Through extensive simulation on surface codes under circuit-level noise, our DQN agent strictly outperforms naive two-dimensional grid-search baselines, achieving a zero overflow rate in critical regimes where traditional thresholding fails catastrophically.

•Jul 21, 2026•0 citations

Real-time Surface-Code Error Correction Using an FPGA-based Neural-Network Decoder

Xiaohan Yang, Xuandong Sun, Zhiyi Wu +9 more

Quantum error correction (QEC) is essential for achieving low error rates required for fault-tolerant quantum computation. In stabilizer-based codes such as the surface code, errors are inferred from repeated syndrome measurements and corrected by a classical decoder. To prevent error accumulation, decoding must be performed with both high throughput and low latency to keep pace with the QEC cycle and enable real-time feedback for universal logical operations. Here we report a hardware-integrated control architecture featuring an FPGA-based neural-network (NN) decoder and experimentally demonstrate real-time surface-code (distance-3) QEC on a superconducting quantum processor. The system achieves a deterministic closed-loop latency of 550 ns, including 124 ns for NN decoding, enabling feedback corrections within a 1.25 us QEC cycle. We show that real-time decoding and feedback correction achieve logical performance comparable to offline decoding while maintaining robustness against varying error conditions. We further demonstrate mid-circuit feedback correction in non-Clifford logical circuits, where Pauli-frame updating alone becomes insufficient. Our results establish a low-latency hardware architecture for embedded QEC control and provide a pathway towards scalable fault-tolerant quantum computing systems.

quant-pharXiv:2605.04892•May 7, 2026•0 citations
The scalability of fault-tolerant quantum computing is strictly bounded by the performance and latency of Quantum Error Correction (QEC) decoding systems. While Reinforcement Learning (RL) has emerged as a promising approach for deriving optimal decoding policies, current Deep Q-Network (DQN) agents are overwhelmingly trained using idealized software simulators. These simulators employ deterministic latency models that drastically fail to capture the stochastic load noise, memory contention, thermal throttling, and bus jitter inherent to real-world hardware deployment on Field-Programmable Gate Arrays (FPGAs). In this paper, we propose a comprehensive Hardware-in-the-Loop (HIL) Reinforcement Learning architecture that physically situates the FPGA in the RL environment feedback loop. By directly offloading syndrome processing jobs to the FPGA hardware during training, the DQN agent observes the exact microarchitectural execution latency as its environmental reward. We formulate a robust, systems-engineering oriented framework that captures true edge-deployment constraints. Our extensive HIL empirical results demonstrate that agents trained against true hardware telemetry develop distinct decoding strategies that actively avoid routing congestion in the FPGA fabric, yielding up to a 42%42\%42% reduction in long-tail latency events and improving the effective logical error rate under strict real-time decoding constraints.
The advent of promising quantum error correction (QEC) codes with efficient resource utilization and high-performance fault-tolerant quantum memories signifies a critical step towards realizing practical quantum computation. While surface codes have been a dominant approach, their limitations have spurred the development of more advanced QEC architectures. These advanced codes often present increased complexity, demanding innovative decoding methodologies. This work investigates the application of reinforcement learning (RL) techniques, including hybrid and multi-agent approaches, to enhance the decoding of various advanced QEC architectures. By leveraging the ability of RL to learn optimal strategies from noisy syndrome measurements, we explore the potential for achieving improved logical error rates and scalability compared to traditional decoding methods. Our approach examines the adaptation of reinforcement learning to exploit the structural properties of these modern QEC models. We also explore the benefits of combining different RL algorithms to address the multifaceted nature of the decoding problem, considering factors such as code degeneracy and real-world noise characteristics. With our proposed method, we are able to demonstrate that an autonomously trained agent can derive decoding schemes for the complex decoding requirement of advanced QEC architectures.
The scalability of fault-tolerant quantum computing is strictly bounded by the performance and latency of Quantum Error Correction (QEC) decoding systems. While Reinforcement Learning (RL) has emerged as a promising approach for deriving optimal decoding policies, current Deep Q-Network (DQN) agents are overwhelmingly trained using idealized software simulators. These simulators employ deterministic latency models that drastically fail to capture the stochastic load noise, memory contention, thermal throttling, and bus jitter inherent to real-world hardware deployment on Field-Programmable Gate Arrays (FPGAs). In this paper, we propose a comprehensive Hardware-in-the-Loop (HIL) Reinforcement Learning architecture that physically situates the FPGA in the RL environment feedback loop. By directly offloading syndrome processing jobs to the FPGA hardware during training, the DQN agent observes the exact microarchitectural execution latency as its environmental reward. We formulate a robust, systems-engineering oriented framework that captures true edge-deployment constraints. Our extensive HIL empirical results demonstrate that agents trained against true hardware telemetry develop distinct decoding strategies that actively avoid routing congestion in the FPGA fabric, yielding up to a 42%42\%42% reduction in long-tail latency events and improving the effective logical error rate under strict real-time decoding constraints.
Quantum error correction (QEC) is indispensable for the realization of fault-tolerant quantum computers. Surface codes are a leading QEC architecture due to their high error thresholds and planar design. The core challenge lies in the decoding process: inferring latent physical errors from syndrome measurements and applying appropriate corrections. Although there are classical decoders, their performance can degrade with complex noise or be computationally demanding. This paper investigates an advanced Deep Reinforcement Learning (DRL) approach, employing a Dueling Double Deep Q-Network (DQN), to decode X-type Pauli errors on ad=3\mathbf{d}=\mathbf{3}d=3rotated surface code. The DRL agent leverages syndrome history to discern temporal error correlations and utilizes a multi-round correction strategy, enabling iterative refinement of corrective actions within an environment that simulates both physical qubit errors and syndrome measurement noise. After 30,000 training episodes with a 5 % physical error rate and 1 % measurement noise, the agent achieved a 0.00 % logical error rate in the extended test, an average episode reward of 0.38, and successfully cleared syndromes in 63 % of the test cases, demonstrating robust learning of a safe decoding policy. The persistent challenge of high training loss, despite good policy performance, is discussed, offering insights into value function approximation in complex QEC environments.
The practical realization of fault-tolerant quantum computing relies fundamentally on the rapid and accurate decoding of quantum error-correcting codes. However, existing analyses of decoding algorithms often treat the problem as a sequence of independent, state-less optimization tasks, typically optimizing for a scalar metric such as logical error rate or average latency. In reality, the decoding process in a continuously operating quantum computer is tightly constrained by the physical rate of syndrome extraction and the finite capacity of the classical processing buffer, creating a non-trivial backlog dynamic. In this work, we present a rigorous mathematical framework that formulates the quantum decoding problem under strict backlog constraints as a discrete-time Markov Decision Process (MDP). By augmenting the state space to include both the real-time syndrome weight and the instantaneous buffer backlog, we shift the paradigm from one-dimensional thresholding to a multi-dimensional sequential control problem. We demonstrate mathematically and empirically that when the queue dynamics are explicitly modeled, the optimal policy diverges significantly from simple syndrome-weight thresholds. Furthermore, we employ Deep Q-Networks (DQN) to discover non-linear policies that inherently balance the trade-off between logical error suppression and queue stability. Our control-theoretic analysis reveals that incorporating state-space representations of the backlog allows RL agents to robustly prevent buffer overflow while maintaining near-optimal logical fidelity. Through extensive simulation on surface codes under circuit-level noise, our DQN agent strictly outperforms naive two-dimensional grid-search baselines, achieving a zero overflow rate in critical regimes where traditional thresholding fails catastrophically.
The practical realization of fault-tolerant quantum computing relies fundamentally on the rapid and accurate decoding of quantum error-correcting codes. However, existing analyses of decoding algorithms often treat the problem as a sequence of independent, state-less optimization tasks, typically optimizing for a scalar metric such as logical error rate or average latency. In reality, the decoding process in a continuously operating quantum computer is tightly constrained by the physical rate of syndrome extraction and the finite capacity of the classical processing buffer, creating a non-trivial backlog dynamic. In this work, we present a rigorous mathematical framework that formulates the quantum decoding problem under strict backlog constraints as a discrete-time Markov Decision Process (MDP). By augmenting the state space to include both the real-time syndrome weight and the instantaneous buffer backlog, we shift the paradigm from one-dimensional thresholding to a multi-dimensional sequential control problem. We demonstrate mathematically and empirically that when the queue dynamics are explicitly modeled, the optimal policy diverges significantly from simple syndrome-weight thresholds. Furthermore, we employ Deep Q-Networks (DQN) to discover non-linear policies that inherently balance the trade-off between logical error suppression and queue stability. Our control-theoretic analysis reveals that incorporating state-space representations of the backlog allows RL agents to robustly prevent buffer overflow while maintaining near-optimal logical fidelity. Through extensive simulation on surface codes under circuit-level noise, our DQN agent strictly outperforms naive two-dimensional grid-search baselines, achieving a zero overflow rate in critical regimes where traditional thresholding fails catastrophically.
Quantum error correction (QEC) is essential for achieving low error rates required for fault-tolerant quantum computation. In stabilizer-based codes such as the surface code, errors are inferred from repeated syndrome measurements and corrected by a classical decoder. To prevent error accumulation, decoding must be performed with both high throughput and low latency to keep pace with the QEC cycle and enable real-time feedback for universal logical operations. Here we report a hardware-integrated control architecture featuring an FPGA-based neural-network (NN) decoder and experimentally demonstrate real-time surface-code (distance-3) QEC on a superconducting quantum processor. The system achieves a deterministic closed-loop latency of 550 ns, including 124 ns for NN decoding, enabling feedback corrections within a 1.25 us QEC cycle. We show that real-time decoding and feedback correction achieve logical performance comparable to offline decoding while maintaining robustness against varying error conditions. We further demonstrate mid-circuit feedback correction in non-Clifford logical circuits, where Pauli-frame updating alone becomes insufficient. Our results establish a low-latency hardware architecture for embedded QEC control and provide a pathway towards scalable fault-tolerant quantum computing systems.