Recent neural decoders substantially improve error correction for short block codes but remain computationally intensive and lack adaptive refinement. This work introduces enhanced U-shaped transformer architectures with two key innovations: (i) double-masked attention from complementary parity-check matrices for improved structural diversity, and (ii) cross-attention message-passing that bridges neural and classical decoding principles. To enable adaptive iterative refinement, we extend these single-shot U-shaped decoders into the denoising diffusion paradigm, where channel corruption is modeled as a forward diffusion process and iteratively reversed based on continuous soft syndrome conditioning. Our key finding demonstrates that U-shaped backbones provide disproportionately larger performance gains under diffusion compared to shallow or deep alternatives, nearly matching deep architectures while utilizing half the parameters. Extensive experiments on LDPC(49,24) and Polar(64,32) codes show that the proposed diffusion-enhanced U-shaped transformers achieve state-of-the-art bit-error-rate performance while achieving superior computational efficiency.
Recent neural decoders substantially improve error correction for short block codes but remain computationally intensive and lack adaptive refinement. This work introduces enhanced U-shaped transformer architectures with two key innovations: (i) double-masked attention from complementary parity-check matrices for improved structural diversity, and (ii) cross-attention message-passing that bridges neural and classical decoding principles. To enable adaptive iterative refinement, we extend these single-shot U-shaped decoders into the denoising diffusion paradigm, where channel corruption is modeled as a forward diffusion process and iteratively reversed based on continuous soft syndrome conditioning. Our key finding demonstrates that U-shaped backbones provide disproportionately larger performance gains under diffusion compared to shallow or deep alternatives, nearly matching deep architectures while utilizing half the parameters. Extensive experiments on LDPC(49,24) and Polar(64,32) codes show that the proposed diffusion-enhanced U-shaped transformers achieve state-of-the-art bit-error-rate performance while achieving superior computational efficiency.
Error correction code (ECC) is an integral part of the physical communication layer, ensuring reliable data transfer over noisy channels. Recently, neural decoders have demonstrated their advantage over classical decoding techniques. However, recent state-of-the-art neural decoders suffer from high complexity and lack the important iterative scheme characteristic of many legacy decoders. In this work, we propose to employ denoising diffusion models for the soft decoding of linear codes at arbitrary block lengths. Our framework models the forward channel corruption as a series of diffusion steps that can be reversed iteratively. Three contributions are made: (i) a diffusion process suitable for the decoding setting is introduced, (ii) the neural diffusion decoder is conditioned on the number of parity errors, which indicates the level of corruption at a given step, (iii) a line search procedure based on the code's syndrome obtains the optimal reverse diffusion step size. The proposed approach demonstrates the power of diffusion models for ECC and is able to achieve state of the art accuracy, outperforming the other neural decoders by sizable margins, even for a single reverse diffusion step.
Error correcting codes (ECCs) are indispensable for reliable transmission in communication systems. The recent advancements in deep learning have catalyzed the exploration of ECC decoders based on neural networks. Among these, transformer-based neural decoders have achieved state-of-the-art decoding performance. In this paper, we propose a novel Cross-attention Message-Passing Transformer (CrossMPT), which shares key operational principles with conventional message-passing decoders. While conventional transformer-based decoders employ self-attention mechanism without distinguishing between the types of input vectors (i.e., magnitude and syndrome vectors), CrossMPT updates the two types of input vectors separately and iteratively using two masked cross-attention blocks. The mask matrices are determined by the code's parity-check matrix, which explicitly captures the irrelevant relationship between two input vectors. Our experimental results show that CrossMPT significantly outperforms existing neural network-based decoders for various code classes. Notably, CrossMPT achieves this decoding performance improvement, while significantly reducing the memory usage, complexity, inference time, and training time.
With the broadening applications of deep learning, neural decoders have emerged as a key research focus, specifically aimed at improving the decoding performance of conventional decoding algorithms. In particular, error correction code transformer (ECCT), which utilizes the transformer architecture, has achieved state-of-the-art performance among neural network-based decoders. We present three technical contributions to significantly enhance the performance of ECCT. First, we propose a novel transformer architecture of ECCT, termed themultiple-masks ECCT (MM ECCT). We employ multiple masked self-attention blocks with different mask matrices in a parallel manner to learn diverse relationships among the codeword bits. Second, we discover that constructing mask matrices based on systematic parity check matrices (PCMs) can make the attention mapssparse, which not only enhances the decoding performance but also reduces computational complexity. Finally, we propose using complementary mask matrices derived from cyclic permutations of the systematic PCM. These complementary mask matrices are specifically designed to enhance the decoding of cyclic codes. Our extensive simulation results show that the proposed MM ECCT architecture with carefully designed mask matrices outperforms the original ECCT by a large margin, achieving state-of-the-art decoding performance among neural decoders. The source code is available at https://github.com/iil-postech/mm-ecct.
Denoising diffusion error correction code (DDECC) is a recent state-of-the-art neural decoding framework by casting decoding as a deterministic reverse diffusion process conditioned on parity-check errors. However, its heuristic step parameterization and lack of sample diversity limit decoding performance. We propose a time-conditioned diffusion-based decoder that reformulates the decoding task as a discrete-time reverse diffusion process driven by an advanced diffusion sampler. Our method introduces two key components: (i) a Transformerbased denoiser conditioned on diffusion time steps, and (ii) an initialization that maps the received signal to a valid diffusion state. To further enhance error correction, we propose a multiple diffusion sampling (MDS) algorithm that leverages stochasticity to generate L parallel decoding trajectories for enhanced solution exploration. Experiments on (63,45)BCH and (64,42) polar codes show that our decoder can achieve superior decoding accuracy over DDECC with reduced complexity. As L increases, MDS approaches maximum-likelihood performance.
Neural channel decoders have attracted growing attention as promising routes for high-performance decoding of short block codes. A recent milestone is denoising diffusion error correction codes (DDECC), which fuses diffusion probabilistic models with a Transformer-based denoiser and outperforms earlier neural approaches. Yet its customized line-search update lacks formal convergence guarantees, and its convergence speed and decoding accuracy can still be improved. To overcome these limitations, we propose time-conditioned diffusion (TCD) decoder, a continuous-time neural decoder built upon stochastic differential equation (SDE) theory. TCD leverages high-order diffusion samplers to achieve theoretically-grounded, fast convergence. It further introduces 1) a time-conditioned Transformer-based denoiser with continuous embeddings, 2) an initialization that maps received signals to valid diffusion states, and 3) an early-stopping rule tailored for decoding. Moreover, we derive three analytically driven, low-complexity guidance schemes—likelihood-gradient, posterior-mean, and their hybrid—conditioned on the received signal to further enhance decoding accuracy. Simulations on BCH, LDPC, Polar codes show that guided TCD outperforms DDECC across nearly all settings while markedly reducing computational cost. Against other universal neural decoders, TCD achieves state-of-the-art performance, further closing the gap to maximum-likelihood decoding.
In communication and storage systems, error correction codes (ECCs) are pivotal in ensuring data reliability. As deep learning's applicability has broadened across diverse domains, there is a growing research focus on neural network-based decoders that outperform traditional decoding algorithms. Among these neural decoders, Error Correction Code Transformer (ECCT) has achieved the state-of-the-art performance, outperforming other methods by large margins. To further enhance the performance of ECCT, we propose two novel methods. First, leveraging the systematic encoding technique of ECCs, we introduce a new masking matrix for ECCT, aiming to improve the performance and reduce the computational complexity. Second, we propose a novel transformer architecture of ECCT called a double-masked ECCT. This architecture employs two different mask matrices in a parallel manner to learn more diverse features of the relationship between codeword bits in the masked self-attention blocks. Extensive simulation results show that the proposed double-masked ECCT outperforms the conventional ECCT, achieving the state-of-the-art decoding performance with significant margins.