For some linear codes, neural networks have been found to be a way to enhance traditional belief propagation (BP) decoder. When deploying such methods in practice, quantization issues also need to be considered. Considering that the messages of variable/check nodes are variable during the BP decoding process, this paper proposes a dynamic quantization method to predict the optimal bit-width for the node’s message quantization in different decoding iterations. We build a shallow neural network as a bit-width predictor and train it jointly with a neural belief propagation decoding network. During actual decoding, the bit prediction network will generate appropriate bit-width for quantizing the messages of each layer of nodes. Finally, simulation experiments validate the effectiveness of the proposed quantization method.
For some linear codes, neural networks have been found to be a way to enhance traditional belief propagation (BP) decoder. When deploying such methods in practice, quantization issues also need to be considered. Considering that the messages of variable/check nodes are variable during the BP decoding process, this paper proposes a dynamic quantization method to predict the optimal bit-width for the node’s message quantization in different decoding iterations. We build a shallow neural network as a bit-width predictor and train it jointly with a neural belief propagation decoding network. During actual decoding, the bit prediction network will generate appropriate bit-width for quantizing the messages of each layer of nodes. Finally, simulation experiments validate the effectiveness of the proposed quantization method.
Machine learning based approaches are being increasingly used for designing decoders for next generation communication systems. One widely used framework is neural belief propagation (NBP), which unfolds the belief propagation (BP) iterations into a deep neural network and the parameters are trained in a data-driven manner. NBP decoders have been shown to improve upon classical decoding algorithms. In this paper, we investigate the generalization capabilities of NBP decoders. Specifically, the generalization gap of a decoder is the difference between empirical and expected bit-error-rate(s). We present new theoretical results which bound this gap and show the dependence on the decoder complexity, in terms of code parameters (blocklength, message length, variable/check node degrees), decoding iterations, and the training dataset size. Results are presented for both regular and irregular parity-check matrices. To the best of our knowledge, this is the first set of theoretical results on generalization performance of neural network based decoders. We present experimental results to show the dependence of generalization gap on the training dataset size, and decoding iterations for different codes.
Machine learning based approaches are being increasingly used for designing decoders for next generation communication systems. One widely used framework is neural belief propagation (NBP), which unfolds the belief propagation (BP) iterations into a deep neural network and the parameters are trained in a data-driven manner. NBP decoders have been shown to improve upon classical decoding algorithms. In this paper, we investigate the generalization capabilities of NBP decoders. Specifically, the generalization gap of a decoder is the difference between empirical and expected bit-error-rate(s). We present new theoretical results which bound this gap and show the dependence on thedecoder complexity, in terms of code parameters (blocklength, message length, variable/check node degrees), decoding iterations, and the training dataset size. Results are presented for both regular and irregular parity-check matrices. To the best of our knowledge, this is the first set of theoretical results on generalization performance of neural network based decoders. We present experimental results to show the dependence of generalization gap on the training dataset size, and decoding iterations for different codes.
We consider near maximum-likelihood (ML) decoding of short linear block codes. In particular, we propose a novel decoding approach based on neural belief propagation (NBP) decoding recently introduced by Nachmani et al. in which we allow a different parity-check matrix in each iteration of the algorithm. The key idea is to consider NBP decoding over an overcomplete parity-check matrix and use the weights of NBP as a measure of the importance of the check nodes (CNs) to decoding. The unimportant CNs are then pruned. In contrast to NBP, which performs decoding on a given fixed parity-check matrix, the proposed pruning-based neural belief propagation (PB-NBP) typically results in a different parity-check matrix in each iteration. For a given complexity in terms of CN evaluations, we show that PB-NBP yields significant performance improvements with respect to NBP. We apply the proposed decoder to the decoding of a Reed-Muller code, a short low-density parity-check (LDPC) code, and a polar code. PB-NBP outperforms NBP decoding over an overcomplete parity-check matrix by 0.27-0.31 dB while reducing the number of required CN evaluations by up to 97%. For the LDPC code, PB-NBP outperforms conventional belief propagation with the same number of CN evaluations by 0.52 dB. We further extend the pruning concept to offset min-sum decoding and introduce a pruning-based neural offset min-sum (PB-NOMS) decoder, for which we jointly optimize the offsets and the quantization of the messages and offsets. We demonstrate performance 0.5 dB from ML decoding with 5-bit quantization for the Reed-Muller code.
BP decoding algorithm is one of the commonly used decoding algorithms for LDPC codes. To adapt LDPC codes to different 5G scenarios and further improve the decoding performance of short LDPC codes, a scheme combining model-driven deep learning with a traditional BP decoding algorithm is proposed. With the advantages of model-driven, this solution expands the decoding iteration process between the check node and the variable node into a neural network and proposes a parameter normalization optimization solution to solve the problem of the program with many training parameters, the edge weights of the optimized Tanner graph are re-assigned and bound. Simulation results show that the proposed scheme can improve the decoding performance of LDPC codes with short lengths.
For the Belief Propagation (BP) algorithm of low-density parity-check (LDPC) codes, existing deep learning methods have a limited performance improvement and it is difficult to train deep-level networks. In this paper, a model-driven quasi-residual network (Quasi-ResNet) BP decoding architecture is proposed for LDPC codes to further improve the performance of standard BP decoding. This method feeds the reliable messages calculated in current iteration into the next iteration based on the shortcut connection, and adjusts the weight of shortcut connection based on the error Back Propagation algorithm of neural network to determine the optimal genetic proportion of reliable messages. The decoding architecture is composed of a model-driven deep neural network (DNN) and shortcut connection. Simulation results show that the decoder can not only unfold more layers quickly compared with the DNN-based BP decoder, but also further improve the decoding performance.
Short-cycles in Tanner graphs have a direct impact on the accuracy and effectiveness of the belief propagation (BP) algorithm, as they diverge the BP algorithm by disrupting the independency of message transmission. In this paper, we present a Neural Network Empowered BP (NNE-BP) algorithm as an alternative to the conventional BP algorithm, which can approach the optimum form of BP in short-cycles. The earlier deep learning (DL) based decoder treated the entire decoding process as a classification issue, making it effective only in short block lengths and with limited generalization. In the proposed NNE-BP, we design two neural networks, namely the horizontal network and vertical network, to learn, respectively, the optimized horizontal and vertical processes for single nodes in the presence of cycles. These two networks operate in an iterative fashion, allowing them to approach a theoretical message propagation equation without assuming independent message transmission. Compared to the classification based DL decoder, the proposed network topology is no longer bundled with any specific Tanner graph, and the number of network parameters are greatly reduced. The obtained experimental results show that the proposed NNE-BP outperforms the conventional BP decoder, especially in the case of short-cycles.