arXiv:1701.05931 · IEEE International Symposium on Information Theory (ISIT)

Neural offset min-sum decoding

Loren Lugosch, Warren J. Gross

Published Jun 1, 2017• 172 citations

Abstract

Recently, it was shown that if multiplicative weights are assigned to the edges of a Tanner graph used in belief propagation decoding, it is possible to use deep learning techniques to find values for the weights which improve the error-correction performance of the decoder. Unfortunately, this approach requires many multiplications, which are generally expensive operations. In this paper, we suggest a more hardware-friendly approach in which offset min-sum decoding is augmented with learnable offset parameters. Our method uses no multiplications and has a parameter count less than half that of the multiplicative algorithm. This both speeds up training and provides a feasible path to hardware architectures. After describing our method, we compare the performance of the two neural decoding algorithms and show that our method achieves error-correction performance within 0.1 dB of the multiplicative approach and as much as 1 dB better than traditional belief propagation for the codes under consideration.

DOI: 10.1109/isit.2017.8006751

Related papers

Low Complexity Gated Neural Min-Sum Decoding for Block Codes

Pooja Pathak, Richa Bhatia

For short to moderate-length block codes, neural belief propagation (NBP) decoders are developed to ameliorate the performance of the belief propagation algorithm (BPA). The core concept underlying these decoders is to model belief propagation (BP) as a neural network, allowing adaptable decoding through trainable weights. The article proposes a novel Gated neural min-sum (GNMS) decoding technique with learnable parameters as a more hardware-efficient substitute for conventional NBP decoders. By using parameterised updates and gating techniques, our approach reformulates the min-sum (MS) algorithm, which is an approximation of BP, while preserving the flexibility of neural approaches and drastically lowering computing overhead. Additionally, we develop an autoencoder-assisted framework that supports flexible, learning-based optimisation for GNMS and Gated Neural Offset Min-Sum (GNOMS) decoders. The suggested decoders achieve error-correction performance at par with the most advanced NBP decoders by evaluating them on a variety of short to moderate-length block codes. Notably, when compared to traditional BP, the suggested approach lowers the bit error rate (BER) 10-fold for signal-to-noise ratio (SNR) greater than 3 dB. Furthermore, the efficacy of the proposed work is exhibited by comparing the computational complexity of the neural network-based decoders, highlighting the advantages in terms of practical implementation and hardware feasibility. The simulation results demonstrate that GNMS decoding with learnable parameters offers a convincing trade-off between complexity and performance, offering a potential path for future decoder design in contemporary communication systems.

IETE Journal of ResearchAug 4, 20250 citations

Optimizing LDPC Decoding with Machine Learning Using Learnable Weights: A Hybrid Framework Integrating Neural Networks with Iterative Message-Passing for Enhanced Error-Correction in 5G Systems

Niklas Savijoki

This thesis investigates the optimization of Low-Density Parity-Check (LDPC) decoding through machine learning techniques, with a specific focus on 5G communication systems. Traditional LDPC decoding algorithms face a critical trade-off: Belief Propagation (BP) offers excellent error-correction performance but with high computational complexity, while Min-Sum (MS) reduces complexity at the cost of degraded performance. To address this limitation, a hybrid neural network architecture is proposed that incorporates learnable normalization and offset weights into the message-passing framework of LDPC decoders. This approach integrates multiplicative (normalization) and additive (offset) weights into the Tanner graph structure, allowing the decoder to adapt to specific channel conditions. A novel weighted binary cross-entropy loss function is developed that prioritizes entire block recovery by assigning different importance to information and parity bits during training. Experimental results demonstrate that ML-assisted decoder consistently outperforms conventional Min-Sum decoding, approaching the error-correction capability of Belief Propagation while maintaining comparable computational efficiency to Min-Sum. Comprehensive evaluations across different weight configurations (scalar and vector-based) reveal that increased parametrization generally improves decoding performance. Analysis of the impact of iteration counts shows that the optimal number of decoding iterations varies between different code configurations with studied Basegraph 1 case requiring 15 iterations for optimal performance while studied Basegraph 2 case benefits significantly from 20 iterations. The proposed ML-assisted LDPC decoder represents a significant advancement for 5G systems, offering improved error correction at lower signal strengths while balancing latency and computational requirements. This research contributes to the ongoing evolution of wireless technology by demonstrating how machine learning can effectively enhance traditional decoding algorithms, providing a practical pathway toward more efficient and adaptive error correction in modern communication systems.

Jan 1, 20250 citations

Offset min-sum Optimization for General Decoding Scheduling: A Deep Learning Approach

Ahmed Attia Abotabl, Jung Hyun Bae, Kee-Bong Song

Deep learning has shown an unprecedented success in many fields such as computer vision and speech recognition, providing solutions to intractable problems. Deep learning has also provided solutions to several intractable problems in communications systems. This paper uses a deep learning approach to optimize the analytically intractable offset value in the offset-min-sum (OMS) algorithm. OMS algorithm is a very attractive low complexity algorithm that is used in the belief propagation decoding of linear codes. The contributions of this paper are: First, providing a low complexity offset optimization framework based on gradient descent and back propagation on the original Tanner graph. Our proposed algorithm has comparable complexity and similar operation as the forward belief propagation decoding algorithm and hence, can be trained much more efficiently. Second, the framework can be easily extended to any decoding scheduling such as flooding or layered scheduling. Training results show that the proposed framework can find the optimal offset value under different decoding scheduling with the same complexity of the belief propagation algorithm.

IEEE Vehicular Technology ConferenceSep 1, 20197 citations

New Min-Sum Decoders Based on Deep Learning for Polar Codes

Bin Dai, Rongke Liu, Zhiyuan Yan

In this paper, we propose two novel min-sum (MS) decoding algorithms based on deep learning for polar codes, an offset min-sum (OMS) and a scaling offset min-sum (SOMS) algorithm. The parameters of both algorithms are different from iteration to iteration, and are obtained by training over a deep neural network. Our simulation results show that the OMS algorithm has roughly the same error performance as a previously proposed multiple scaling min-sum (MSMS) algorithm, and that the SOMS algorithm performs better than all existing BP-based algorithms. Since the OMS algorithm requires only an addition as opposed to a multiplication in the MSMS algorithm, the OMS algorithm is more suitable for hardware implementation. The two proposed decoding algorithms provide a tradeoff between complexity and error performance.

SiPS ... design and implementation - IEEE Workshop on Signal Processing SystemsOct 1, 201814 citations

Deep Learning Methods for Improved Decoding of Linear Codes

Eliya Nachmani, Elad Marciano, Loren Lugosch +3 more

The problem of low complexity, close to optimal, channel decoding of linear codes with short to moderate block length is considered. It is shown that deep learning methods can be used to improve a standard belief propagation decoder, despite the large example space. Similar improvements are obtained for the min-sum algorithm. It is also shown that tying the parameters of the decoders across iterations, so as to form a recurrent neural network architecture, can be implemented with comparable results. The advantage is that significantly less parameters are required. We also introduce a recurrent neural decoder architecture based on the method of successive relaxation. Improvements over standard belief propagation are also observed on sparser Tanner graph representations of the codes. Furthermore, we demonstrate that the neural belief propagation decoder can be used to improve the performance, or alternatively reduce the computational complexity, of a close to optimal decoder of short BCH codes.

arXiv:1706.07043IEEE Journal of Selected Topics in Signal ProcessingJan 1, 2018546 citations

Learning strategies for neural min-sum decoding of LDPC codes

Hyeyeon Na, Hosung Park, Hee-Youl Kwak +1 more

The min-sum (MS) decoding for low-density parity-check codes, though less complex than the sum–product algorithm, suffers from worse error-correcting performance. For enhancement, neural MS decoders leveraging deep learning have recently been introduced, but how to train them has not been sufficiently discussed. In this paper, we propose a novel dataset construction method and also propose systematic learning strategies by finding a good combination of dataset composition, loss functions, weight sharing, weight assignment , and weight update method. Simulations demonstrate that the proposed method achieves better error-correcting performance than other works, especially in the error floor region, within a limited number of iterations.

ICT ExpressJan 6, 20255 citations