In the past few years, deep learning has been widely used in various fields due to its outstanding progress. One of the latest applications of deep learning is to use a neural network (NN) with trainable multiplicative weights to design decoders for error-correcting codes. High quality data are essential for deep learning to train robust NN models. In this study, two novel semi-/auto-adaptive SNR algorithms are proposed to efficiently train the neural decoders based on the Sum-Product Algorithm (SPA). For illustration, several neural SPA decoders for the Bose-Chaudhuri-Hocquenghem (BCH) code and low-density parity-check (LDPC) code have been constructed as examples. Simulation results show that, compared with the original neural decoders, the performance of these neural decoders trained by the proposed algorithms can be improved in the range of 0.2 to 0.6 dB. Moreover, the training time required for these decoders to achieve convergence can be reduced by up to 28.8% for the BCH code, and up to 35.6% for the LDPC code, without increasing decoding complexity.
In the past few years, deep learning has been widely used in various fields due to its outstanding progress. One of the latest applications of deep learning is to use a neural network (NN) with trainable multiplicative weights to design decoders for error-correcting codes. High quality data are essential for deep learning to train robust NN models. In this study, two novel semi-/auto-adaptive SNR algorithms are proposed to efficiently train the neural decoders based on the Sum-Product Algorithm (SPA). For illustration, several neural SPA decoders for the Bose-Chaudhuri-Hocquenghem (BCH) code and low-density parity-check (LDPC) code have been constructed as examples. Simulation results show that, compared with the original neural decoders, the performance of these neural decoders trained by the proposed algorithms can be improved in the range of 0.2 to 0.6 dB. Moreover, the training time required for these decoders to achieve convergence can be reduced by up to 28.8% for the BCH code, and up to 35.6% for the LDPC code, without increasing decoding complexity.
LDPC code is a widely-used high-performance error correction code. In recent years, LDPC decoding based on deep learning and neural networks becomes a research hotspot. Based on the (512,256) LDPC code of the CCSDS standard, this paper firstly studies the traditional decoding algorithms of SP, MS, NMS, and OMS, laying a foundation for the construction of neural networks. Then a data-driven (DD) decoding method is studied which adopts the information with its encoded, modulated and noise-added LDPC code as the training data within a Multi-layer Perceptron (MLP) neural network. In order to solve the problem of high bit error rate (BER) in data-driven method, the Neural Normalized Min-sum (NNMS) decoding in which the NMS algorithm is mapped to the neural network structure is proposed, achieving more excellent BER performance than that of NMS. The BER declines by 85.19% when channel SNR equals to 3.5 dB. Finally, improved training methods to enhance the SNR generalization ability of the NNMS network is studied.
In this letter, a neural self-corrected min-sum (NSCMS) decoder is proposed for 5G new radio (NR) low-density parity-check (LDPC) codes. Training in real-time on a rapidly changing channel requires low latency and minimal resource utilization. Unfortunately, typical neural decoders for LDPC codes use large and deep neural networks proportional to the code length, leading to latency and resource problems. In the network of NSCMS similar to network model pruning, certain nodes meeting the self correction condition are deleted and excluded from learning. This reduces the computational complexity of learning, compared to conventional networks. Thus, the NSCMS decoder has high practicality in real-time training using machine learning. Furthermore, self-correction allows for more reliable message-based learning and significantly improves performance. Simulation results demonstrate that NSCMS decoders exhibit lower error rates than previously proposed min-sum decoders.
A deep learning algorithm for improving the performance of the Sum-ProductAlgorithm (SPA) based decoders is investigated. The proposed Neural NetworkDecoders (NND) [22] generalizes the SPA by assigning weights to the edges ofthe Tanner graph. We elucidate the peculiar design, training, and working of theNND. We analyze the edge weight’s distribution of the trained NND and providea deeper insight into its working. The training process of NND learns the edgeweights in such a way that the effects of artifacts in the Tanner graph (such ascycles or trapping sets) are mitigated, leading to a significant improvement inperformance over the SPA.We conduct an extensive analysis of the training hyper-parameters affectingthe performance of the NND, and present hypotheses for determining theirappropriate choices for different families and sizes of codes. Experimental resultsare used to verify the hypotheses and rationale presented. Furthermore,we propose a new loss-function that improves performance over the standardcross-entropy loss. We also investigate the limitations of the NND in termsof complexity and performance. Although the SPA based design of the NNDenables faster training and reduced complexity, the design constraints restrictthe neural network to reach its maximum potential. Our experiments show thatthe NND is unable to reach Maximum Likelihood (ML) performance thresholdfor any plausible set of hyper-parameters. However for short length (n 128)High Density Parity Check (HDPC) codes such as Polar or BCH codes, theperformance improvement over the SPA is significant.
The min-sum (MS) decoding for low-density parity-check codes, though less complex than the sum–product algorithm, suffers from worse error-correcting performance. For enhancement, neural MS decoders leveraging deep learning have recently been introduced, but how to train them has not been sufficiently discussed. In this paper, we propose a novel dataset construction method and also propose systematic learning strategies by finding a good combination of dataset composition, loss functions, weight sharing, weight assignment , and weight update method. Simulations demonstrate that the proposed method achieves better error-correcting performance than other works, especially in the error floor region, within a limited number of iterations.
For recent years, deep learning has been studied and utilized among decoders in wireless and optical communication. For low density parity check (LDPC) codes, most works use neural network (NN) to implement the belief propagation (BP) decoder and unfold its iterations into layers activated by different functions. However, the whole decoder composed by neural network performs lots of non-linear operations, which are time consuming and unfriendly to hardware implementation. In this work, a neural network-aided normalized min-sum (NN-aided NMS) decoder for LDPC code is proposed, in which the normalization factors are inferred from input log likelihood ratio (LLR) by apre-trained NN, while maintaining check and variable nodes processing the same as in min-sum (MS) decoder. In order to test the proposed decoder with irregular LDPC code, a rate-7/8 one was constructed using progressive edge growth (PEG) algorithm, showing better error correction capability than the standard of consultative committee for space data systems (CCSDS) LDPC (8176,7154) code by 0.08 dB with the same rate. Simulation results show that, NN-aided NMS decoder outperforms MS decoder by 0.3 dB, and its performance approaches BP decoder with a gap of 0.02 dB, which is 50% closer than the best NMS decoder using 0.7 as the normalization factor. The proposed decoder has good performance for both regular and irregular LDPC codes, and needs much less computation complexity than BP decoder. It introduces more linear computation than NMS decoder but achieves better bit error rate (BER) performance, thus becoming a good complement between BP and NMS decoder when trading off among complexity, error correction performance and hardware implementation for wireless and satellite communication.
We investigate the application of machine learning techniques (in particular, deep neural networks, DNN) to improve the decoding algorithms of short quasi-cyclic low-density paritycheck (LDPC) codes adopted to the 5G standard. We note that straightforward application of general-purpose DNNs is not possible due to the curse of dimensionality problem – the training set size grows exponentially with the number of information bits. In our opinion, the only way to deal with this problem is to combine deep learning methods with existing decoding algorithms. We start with a Tanner-based neural network decoder with Min-Sum activation functions proposed by Nachmani et al. and extend it as follows. First, the quasi-cyclic nature of 5G LDPC codes allows us to use a single weight per circular matrix (circulant). We refer to this as weight sharing. This idea significantly reduces the training time, preserving the error-correcting performance. Second, we add residual connections to our NN architecture. Residual connections improve the performance and reduce the training time. We also present the results for the rate and length adaptation techniques. Rate adaptation allows multiple DNNs corresponding to different coding rates to run with a single set of trained weights. Length adaptation allows optimally reusing weights for multiple lifting size indices.