Error-correcting codes (ECCs) are fundamental to reliable communication over noisy channels. Transformer-based neural decoders achieve strong ECC decoding performance, but attention operations cause computational complexity to grow substantially with code length. We propose Cross-Selective Scan Message-Passing Mamba (CrossMPM), a low-complexity Mamba-based neural decoder, to address this issue. CrossMPM replaces cross-attention-based information exchange between magnitude and syndrome vectors in Cross-Attention Message-Passing Transformer (CrossMPT) with Tanner-graph projection and Cross-Selective Scan State-Space Model (Cross-Selective Scan SSM). Tanner-graph projection aggregates connected node representations according to the parity-check matrix to incorporate code structure, while Cross-Selective Scan SSM dynamically generates selection parameters from the aggregated representations to control state updates and outputs. CrossMPM thus models local parity constraints and long-range dependencies without attention operations and reduces computational complexity. Simulation results show that CrossMPM achieves competitive decoding performance with lower computational complexity than CrossMPT and Hybrid Mamba-Transformer Decoder (ECCM), and this performance is maintained at longer code lengths. These results show that CrossMPM provides a favorable trade-off between error correction performance and computational efficiency.
Error-correcting codes (ECCs) are fundamental to reliable communication over noisy channels. Transformer-based neural decoders achieve strong ECC decoding performance, but attention operations cause computational complexity to grow substantially with code length. We propose Cross-Selective Scan Message-Passing Mamba (CrossMPM), a low-complexity Mamba-based neural decoder, to address this issue. CrossMPM replaces cross-attention-based information exchange between magnitude and syndrome vectors in Cross-Attention Message-Passing Transformer (CrossMPT) with Tanner-graph projection and Cross-Selective Scan State-Space Model (Cross-Selective Scan SSM). Tanner-graph projection aggregates connected node representations according to the parity-check matrix to incorporate code structure, while Cross-Selective Scan SSM dynamically generates selection parameters from the aggregated representations to control state updates and outputs. CrossMPM thus models local parity constraints and long-range dependencies without attention operations and reduces computational complexity. Simulation results show that CrossMPM achieves competitive decoding performance with lower computational complexity than CrossMPT and Hybrid Mamba-Transformer Decoder (ECCM), and this performance is maintained at longer code lengths. These results show that CrossMPM provides a favorable trade-off between error correction performance and computational efficiency.
Error correcting codes (ECCs) are indispensable for reliable transmission in communication systems. The recent advancements in deep learning have catalyzed the exploration of ECC decoders based on neural networks. Among these, transformer-based neural decoders have achieved state-of-the-art decoding performance. In this paper, we propose a novel Cross-attention Message-Passing Transformer (CrossMPT), which shares key operational principles with conventional message-passing decoders. While conventional transformer-based decoders employ self-attention mechanism without distinguishing between the types of input vectors (i.e., magnitude and syndrome vectors), CrossMPT updates the two types of input vectors separately and iteratively using two masked cross-attention blocks. The mask matrices are determined by the code's parity-check matrix, which explicitly captures the irrelevant relationship between two input vectors. Our experimental results show that CrossMPT significantly outperforms existing neural network-based decoders for various code classes. Notably, CrossMPT achieves this decoding performance improvement, while significantly reducing the memory usage, complexity, inference time, and training time.
Forward error correction is essential for reliable communication over noisy channels. Attention-based model-free neural decoders have shown strong performance for short codes, but their scalability to longer codes is limited by the quadratic memory and computational cost of attention. In this paper, we introduce the Mamba message-passing decoder (MMPD), an attention-free syndrome-based neural decoder for binary linear codes. MMPD retains the Tanner-graph structure of a message-passing decoder by performing local pairwise aggregation along variable-check edges. To enable efficient long-range information propagation, these local updates are combined with bidirectional Mamba state-space blocks. By avoiding dense attention matrices, MMPD scales more favorably for long codes in both memory and computation. Experiments on the (1056, 880) LDPC code show that MMPD achieves a 0.45 dB gain over the state-of-the-art CrossMPT decoder at a specified target bit error rate, while reducing memory consumption by a factor of 1.5. This reduction factor increases substantially for longer codes, demonstrating the applicability of MMPD to scalable neural decoding of practical long codes.
In this paper, we propose a novel deep learning-based framework for the joint design and decoding of linear block codes, the end-to-end deep coding cross-attention message-passing Transformer (E2E DC-CrossMPT). To improve linear block code design and decoding, we redesign the conventional error correction code (ECC) decoder, CrossMPT, to fit within an end-to-end framework. The redesigned decoder separately utilizes magnitude and syndrome vectors obtained from the received signals as inputs. It further employs one-hot encoding based syndrome embedding and incorporates a parity-check matrix into the output layers. Experimental results demonstrate that, across various code lengths and code rates, E2E DC-CrossMPT consistently outperforms both traditional decoding algorithms and a conventional end-to-end deep coding model in terms of decoding performance. Moreover, the codes designed by E2E DC-CrossMPT achieve superior error-correction capability compared with both traditional linear block codes and those designed by the conventional end-to-end deep coding model, while requiring lower computational complexity.
Channel coding for 6G networks is expected to support a wide range of requirements arising from heterogeneous communication scenarios. These demands challenge traditional code-specific decoders, which lack the flexibility and scalability required for next-generation systems. To tackle this problem, we propose an AI-native foundation model for unified and code-agnostic decoding based on the transformer architecture. We first introduce a cross-attention message-passing transformer (CrossMPT). CrossMPT employs two masked cross-attention blocks that iteratively update two distinct input representations-magnitude and syndrome vectors-allowing the model to effectively learn the decoding problem. Notably, our CrossMPT has achieved state-of-the-art decoding performance among single neural decoders. Building on this, we develop foundation CrossMPT (FCrossMPT) by making the architecture invariant to code length, rate, and class, allowing a single trained model to decode a broad range of codes without retraining. To further enhance decoding performance, particularly for short blocklength codes, we propose CrossMPT ensemble decoder (CrossED), an ensemble decoder composed of multiple parallel CrossMPT blocks employing different parity-check matrices. This architecture can also serve as a foundation model, showing strong generalization across diverse code types. Overall, the proposed AI-native code-agnostic decoder offers flexibility, scalability, and high performance, presenting a promising direction to channel coding for 6G networks.
This thesis investigates the optimization of Low-Density Parity-Check (LDPC) decoding through machine learning techniques, with a specific focus on 5G communication systems. Traditional LDPC decoding algorithms face a critical trade-off: Belief Propagation (BP) offers excellent error-correction performance but with high computational complexity, while Min-Sum (MS) reduces complexity at the cost of degraded performance. To address this limitation, a hybrid neural network architecture is proposed that incorporates learnable normalization and offset weights into the message-passing framework of LDPC decoders. This approach integrates multiplicative (normalization) and additive (offset) weights into the Tanner graph structure, allowing the decoder to adapt to specific channel conditions. A novel weighted binary cross-entropy loss function is developed that prioritizes entire block recovery by assigning different importance to information and parity bits during training. Experimental results demonstrate that ML-assisted decoder consistently outperforms conventional Min-Sum decoding, approaching the error-correction capability of Belief Propagation while maintaining comparable computational efficiency to Min-Sum. Comprehensive evaluations across different weight configurations (scalar and vector-based) reveal that increased parametrization generally improves decoding performance. Analysis of the impact of iteration counts shows that the optimal number of decoding iterations varies between different code configurations with studied Basegraph 1 case requiring 15 iterations for optimal performance while studied Basegraph 2 case benefits significantly from 20 iterations. The proposed ML-assisted LDPC decoder represents a significant advancement for 5G systems, offering improved error correction at lower signal strengths while balancing latency and computational requirements. This research contributes to the ongoing evolution of wireless technology by demonstrating how machine learning can effectively enhance traditional decoding algorithms, providing a practical pathway toward more efficient and adaptive error correction in modern communication systems.
Error control coding is an essential part of modern communications systems. LDPC codes have been demonstrated to offer performance near the fundamental limits of channels corrupted by random noise. Optimal maximum likelihood decoding of LDPC codes is too complex to be practically useful even at short block lengths and so a graph-based message passing decoder known as the belief propagation algorithm is used instead. In fact, on graphs without closed paths known as cycles the iterative message passing decoding is known to be optimal and may converge in a single iteration, although identifying the message update schedule which allows single-iteration convergence is not trivial. At finite block lengths graphs without cycles have poor minimum distance properties and perform poorly even under optimal decoding. LDPC codes with large block length have been demonstrated to offer performance close to that predicted for codes of infinite length, as the cycles present in the graph are quite long. In this thesis, LDPC codes of shorter length are considered as they offer advantages in terms of latency and complexity, at the cost of performance degradation from the increased number of short cycles in the graph. For these shorter LDPC codes, the problems considered are: \n \nFirst, improved construction of structured and unstructured LDPC code graphs of short length with a view to reducing the harmful effects of the cycles on error rate performance, based on knowledge of the decoding process. Structured code graphs are particularly interesting as they allow benefits in encoding and decoding complexity and speed. Secondly, the design and construction of LDPC codes for the block fading channel, a particularly challenging scenario from the point of view of error control code design. Both established and novel classes of codes for the channel are considered. Finally the decoding of LDPC codes by the belief propagation algorithm is considered, in particular the scheduling of messages passed in the iterative decoder. A knowledge-aided approach is developed based on message reliabilities and residuals to allow fast convergence and significant improvements in error rate performance.