A deep learning algorithm for improving the performance of the Sum-ProductAlgorithm (SPA) based decoders is investigated. The proposed Neural NetworkDecoders (NND) [22] generalizes the SPA by assigning weights to the edges ofthe Tanner graph. We elucidate the peculiar design, training, and working of theNND. We analyze the edge weight’s distribution of the trained NND and providea deeper insight into its working. The training process of NND learns the edgeweights in such a way that the effects of artifacts in the Tanner graph (such ascycles or trapping sets) are mitigated, leading to a significant improvement inperformance over the SPA.We conduct an extensive analysis of the training hyper-parameters affectingthe performance of the NND, and present hypotheses for determining theirappropriate choices for different families and sizes of codes. Experimental resultsare used to verify the hypotheses and rationale presented. Furthermore,we propose a new loss-function that improves performance over the standardcross-entropy loss. We also investigate the limitations of the NND in termsof complexity and performance. Although the SPA based design of the NNDenables faster training and reduced complexity, the design constraints restrictthe neural network to reach its maximum potential. Our experiments show thatthe NND is unable to reach Maximum Likelihood (ML) performance thresholdfor any plausible set of hyper-parameters. However for short length (n 128)High Density Parity Check (HDPC) codes such as Polar or BCH codes, theperformance improvement over the SPA is significant.
A deep learning algorithm for improving the performance of the Sum-ProductAlgorithm (SPA) based decoders is investigated. The proposed Neural NetworkDecoders (NND) [22] generalizes the SPA by assigning weights to the edges ofthe Tanner graph. We elucidate the peculiar design, training, and working of theNND. We analyze the edge weight’s distribution of the trained NND and providea deeper insight into its working. The training process of NND learns the edgeweights in such a way that the effects of artifacts in the Tanner graph (such ascycles or trapping sets) are mitigated, leading to a significant improvement inperformance over the SPA.We conduct an extensive analysis of the training hyper-parameters affectingthe performance of the NND, and present hypotheses for determining theirappropriate choices for different families and sizes of codes. Experimental resultsare used to verify the hypotheses and rationale presented. Furthermore,we propose a new loss-function that improves performance over the standardcross-entropy loss. We also investigate the limitations of the NND in termsof complexity and performance. Although the SPA based design of the NNDenables faster training and reduced complexity, the design constraints restrictthe neural network to reach its maximum potential. Our experiments show thatthe NND is unable to reach Maximum Likelihood (ML) performance thresholdfor any plausible set of hyper-parameters. However for short length (n 128)High Density Parity Check (HDPC) codes such as Polar or BCH codes, theperformance improvement over the SPA is significant.
Neural network decoding algorithms are recently introduced by Nachmani et al. to decode high-density parity-check (HDPC) codes. In contrast with iterative decoding algorithms such as sum-product or min-sum algorithms in which the weight of each edge is set to 1, in the neural network decoding algorithms, the weight of every edge depends on its impact in the transmitted codeword. In this paper, we provide a novel feed-forward neural network lattice decoding algorithm suitable to decode lattices constructed based on Construction A, whose underlying codes have HDPC matrices. We first establish the concept of feed-forward neural network for HDPC codes and improve their decoding algorithms compared to Nachmani et al. We then apply our proposed decoder for a Construction A lattice with HDPC underlying code, for which the well-known iterative decoding algorithms show poor performances. The main advantage of our proposed algorithm is that instead of assigning and training weights for all edges, which turns out to be time-consuming especially for high-density parity-check matrices, we concentrate on edges which are present in most of 4-cycles and removing them gives a girth -6 Tanner graph. This approach, by slight modifications using updated LLRs instead of initial ones, simultaneously accelerates the training process and improves the error performance of our proposed decoding algorithm.
In the past few years, deep learning has been widely used in various fields due to its outstanding progress. One of the latest applications of deep learning is to use a neural network (NN) with trainable multiplicative weights to design decoders for error-correcting codes. High quality data are essential for deep learning to train robust NN models. In this study, two novel semi-/auto-adaptive SNR algorithms are proposed to efficiently train the neural decoders based on the Sum-Product Algorithm (SPA). For illustration, several neural SPA decoders for the Bose-Chaudhuri-Hocquenghem (BCH) code and low-density parity-check (LDPC) code have been constructed as examples. Simulation results show that, compared with the original neural decoders, the performance of these neural decoders trained by the proposed algorithms can be improved in the range of 0.2 to 0.6 dB. Moreover, the training time required for these decoders to achieve convergence can be reduced by up to 28.8% for the BCH code, and up to 35.6% for the LDPC code, without increasing decoding complexity.
Neural network decoding algorithms are recently introduced by Nachmani et al. to decode high-density parity-check (HDPC) codes. In contrast with iterative decoding algorithms such as sum-product or min-sum algorithms in which the weight of each edge is set to 1, in the neural network decoding algorithms, the weight of every edge depends on its impact in the transmitted codeword. In this paper, we provide a novel \emph{feed-forward neural network lattice decoding algorithm} suitable to decode lattices constructed based on Construction A, whose underlying codes have HDPC matrices. We first establish the concept of feed-forward neural network for HDPC codes and improve their decoding algorithms compared to Nachmani et al. We then apply our proposed decoder for a Construction A lattice with HDPC underlying code, for which the well-known iterative decoding algorithms show poor performances. The main advantage of our proposed algorithm is that instead of assigning and training weights for all edges, which turns out to be time-consuming especially for high-density parity-check matrices, we concentrate on edges which are present in most of 4-cycles and removing them gives a girth-6 Tanner graph. This approach, by slight modifications using updated LLRs instead of initial ones, simultaneously accelerates the training process and improves the error performance of our proposed decoding algorithm.
In this paper, we present a sparse neural network decoder (SNND) of polar codes based on belief propagation (BP) and deep learning. At first, the conventional factor graph of polar BP decoding is converted to the bipartite Tanner graph similar to low-density parity-check (LDPC) codes. Then the Tanner graph is unfolded and translated into the graphical representation of deep neural network (DNN). The complex sum-product algorithm (SPA) is modified to min-sum (MS) approximation with low complexity. We dramatically reduce the number of weight by using single weight to parameterize the networks. Optimized by the training techniques of deep learning, proposed SNND achieves comparative decoding performance of SPA and obtains about 0.5 dB gain over MS decoding on (128,64) and (256,128) codes. Moreover, 60% complexity reduction is achieved and the decoding latency is significantly lower than the conventional polar BP.
We propose a neural min-sum (MS) decoder for non-binary LDPC codes by applying deep learning to the decoding with short to moderate code lengths. In this algorithm, the iterations are unfolded into hidden layers of the neural network, and trainable weights are added to the check node updates to mitigate the correlation in the message passing due to short girths in the Tanner graph. To avoid gradient vanishing and to converge faster, we adopt an iterative training mechanism for parameter tuning. Furthermore, we investigate the impact of girth distribution on the proposed neural MS decoder, which reveals the reasons for the expected performance improvement. Numerical results in terms of bit error rate are provided, which show that the neural MS decoder achieves significant gains for decoding non-binary LDPC codes.
The min-sum (MS) decoding for low-density parity-check codes, though less complex than the sum–product algorithm, suffers from worse error-correcting performance. For enhancement, neural MS decoders leveraging deep learning have recently been introduced, but how to train them has not been sufficiently discussed. In this paper, we propose a novel dataset construction method and also propose systematic learning strategies by finding a good combination of dataset composition, loss functions, weight sharing, weight assignment , and weight update method. Simulations demonstrate that the proposed method achieves better error-correcting performance than other works, especially in the error floor region, within a limited number of iterations.