In this paper, we propose a novel deep learning-based framework for the joint design and decoding of linear block codes, the end-to-end deep coding cross-attention message-passing Transformer (E2E DC-CrossMPT). To improve linear block code design and decoding, we redesign the conventional error correction code (ECC) decoder, CrossMPT, to fit within an end-to-end framework. The redesigned decoder separately utilizes magnitude and syndrome vectors obtained from the received signals as inputs. It further employs one-hot encoding based syndrome embedding and incorporates a parity-check matrix into the output layers. Experimental results demonstrate that, across various code lengths and code rates, E2E DC-CrossMPT consistently outperforms both traditional decoding algorithms and a conventional end-to-end deep coding model in terms of decoding performance. Moreover, the codes designed by E2E DC-CrossMPT achieve superior error-correction capability compared with both traditional linear block codes and those designed by the conventional end-to-end deep coding model, while requiring lower computational complexity.
In this paper, we propose a novel deep learning-based framework for the joint design and decoding of linear block codes, the end-to-end deep coding cross-attention message-passing Transformer (E2E DC-CrossMPT). To improve linear block code design and decoding, we redesign the conventional error correction code (ECC) decoder, CrossMPT, to fit within an end-to-end framework. The redesigned decoder separately utilizes magnitude and syndrome vectors obtained from the received signals as inputs. It further employs one-hot encoding based syndrome embedding and incorporates a parity-check matrix into the output layers. Experimental results demonstrate that, across various code lengths and code rates, E2E DC-CrossMPT consistently outperforms both traditional decoding algorithms and a conventional end-to-end deep coding model in terms of decoding performance. Moreover, the codes designed by E2E DC-CrossMPT achieve superior error-correction capability compared with both traditional linear block codes and those designed by the conventional end-to-end deep coding model, while requiring lower computational complexity.
Error correcting codes (ECCs) are indispensable for reliable transmission in communication systems. The recent advancements in deep learning have catalyzed the exploration of ECC decoders based on neural networks. Among these, transformer-based neural decoders have achieved state-of-the-art decoding performance. In this paper, we propose a novel Cross-attention Message-Passing Transformer (CrossMPT), which shares key operational principles with conventional message-passing decoders. While conventional transformer-based decoders employ self-attention mechanism without distinguishing between the types of input vectors (i.e., magnitude and syndrome vectors), CrossMPT updates the two types of input vectors separately and iteratively using two masked cross-attention blocks. The mask matrices are determined by the code's parity-check matrix, which explicitly captures the irrelevant relationship between two input vectors. Our experimental results show that CrossMPT significantly outperforms existing neural network-based decoders for various code classes. Notably, CrossMPT achieves this decoding performance improvement, while significantly reducing the memory usage, complexity, inference time, and training time.
Channel coding for 6G networks is expected to support a wide range of requirements arising from heterogeneous communication scenarios. These demands challenge traditional code-specific decoders, which lack the flexibility and scalability required for next-generation systems. To tackle this problem, we propose an AI-native foundation model for unified and code-agnostic decoding based on the transformer architecture. We first introduce a cross-attention message-passing transformer (CrossMPT). CrossMPT employs two masked cross-attention blocks that iteratively update two distinct input representations-magnitude and syndrome vectors-allowing the model to effectively learn the decoding problem. Notably, our CrossMPT has achieved state-of-the-art decoding performance among single neural decoders. Building on this, we develop foundation CrossMPT (FCrossMPT) by making the architecture invariant to code length, rate, and class, allowing a single trained model to decode a broad range of codes without retraining. To further enhance decoding performance, particularly for short blocklength codes, we propose CrossMPT ensemble decoder (CrossED), an ensemble decoder composed of multiple parallel CrossMPT blocks employing different parity-check matrices. This architecture can also serve as a foundation model, showing strong generalization across diverse code types. Overall, the proposed AI-native code-agnostic decoder offers flexibility, scalability, and high performance, presenting a promising direction to channel coding for 6G networks.
Error correction codes are a crucial part of the physical communication layer, ensuring the reliable transfer of data over noisy channels. The design of optimal linear block codes capable of being efficiently decoded is of major concern, especially for short block lengths. While neural decoders have recently demonstrated their advantage over classical decoding techniques, the neural design of the codes remains a challenge. In this work, we propose for the first time a unified encoder-decoder training of binary linear block codes. To this end, we adapt the coding setting to support efficient and differentiable training of the code for end-to-end optimization over the order two Galois field. We also propose a novel Transformer model in which the self-attention masking is performed in a differentiable fashion for the efficient backpropagation of the code gradient. Our results show that (i) the proposed decoder outperforms existing neural decoding on conventional codes, (ii) the suggested framework generates codes that outperform the {analogous} conventional codes, and (iii) the codes we developed not only excel with our decoder but also show enhanced performance with traditional decoding techniques.
Channel coding is vital for reliable sixth-generation (6G) data transmission, employing diverse error correction codes for various application scenarios. Traditional decoders require dedicated hardware for each code, leading to high hardware costs. Recently, artificial intelligence (AI)-driven approaches, such as the error correction code Transformer (ECCT) and its enhanced version, the foundation error correction code Transformer (FECCT), have been proposed to reduce the hardware cost by leveraging the Transformer to decode multiple codes. However, their excessively high computational complexity ofO(N2) due to the self-attention mechanism in the Transformer limits scalability, whereNrepresents the sequence length. To reduce computational complexity, we propose a unified Transformer-based decoder that handles multiple linear block codes within a single framework. Specifically, a standardized unit is employed to align code length and code rate across different code types, while a redesigned low-rank unified attention module, with computational complexity ofO(N), is shared across various heads in the Transformer. Additionally, a sparse mask, derived from the parity-check matrix’s sparsity, is introduced to enhance the decoder’s ability to capture inherent constraints between information and parity-check bits, improving decoding accuracy and further reducing computational complexity by 86%. Extensive experimental results demonstrate that the proposed unified Transformer-based decoder outperforms existing methods and provides a high-performance, low-complexity solution for next-generation wireless communication systems.
Error-correcting codes (ECCs) are fundamental to reliable communication over noisy channels. Transformer-based neural decoders achieve strong ECC decoding performance, but attention operations cause computational complexity to grow substantially with code length. We propose Cross-Selective Scan Message-Passing Mamba (CrossMPM), a low-complexity Mamba-based neural decoder, to address this issue. CrossMPM replaces cross-attention-based information exchange between magnitude and syndrome vectors in Cross-Attention Message-Passing Transformer (CrossMPT) with Tanner-graph projection and Cross-Selective Scan State-Space Model (Cross-Selective Scan SSM). Tanner-graph projection aggregates connected node representations according to the parity-check matrix to incorporate code structure, while Cross-Selective Scan SSM dynamically generates selection parameters from the aggregated representations to control state updates and outputs. CrossMPM thus models local parity constraints and long-range dependencies without attention operations and reduces computational complexity. Simulation results show that CrossMPM achieves competitive decoding performance with lower computational complexity than CrossMPT and Hybrid Mamba-Transformer Decoder (ECCM), and this performance is maintained at longer code lengths. These results show that CrossMPM provides a favorable trade-off between error correction performance and computational efficiency.
With the broadening applications of deep learning, neural decoders have emerged as a key research focus, specifically aimed at improving the decoding performance of conventional decoding algorithms. In particular, error correction code transformer (ECCT), which utilizes the transformer architecture, has achieved state-of-the-art performance among neural network-based decoders. We present three technical contributions to significantly enhance the performance of ECCT. First, we propose a novel transformer architecture of ECCT, termed themultiple-masks ECCT (MM ECCT). We employ multiple masked self-attention blocks with different mask matrices in a parallel manner to learn diverse relationships among the codeword bits. Second, we discover that constructing mask matrices based on systematic parity check matrices (PCMs) can make the attention mapssparse, which not only enhances the decoding performance but also reduces computational complexity. Finally, we propose using complementary mask matrices derived from cyclic permutations of the systematic PCM. These complementary mask matrices are specifically designed to enhance the decoding of cyclic codes. Our extensive simulation results show that the proposed MM ECCT architecture with carefully designed mask matrices outperforms the original ECCT by a large margin, achieving state-of-the-art decoding performance among neural decoders. The source code is available at https://github.com/iil-postech/mm-ecct.