In this paper, a recurrent quantized neural network with a layered structure is designed to optimize the look-up table (LUT) in the layered finite alphabet iterative decoders (FAIDs). By using straight-through estimators to soften the backward gradient propagation of some low-precision activation functions, the quantization thresholds and quantization levels in the LUTs are optimized as parameters in the network. Performance evaluations on decoding codewords encoded with quasi-cyclic low-density parity-check codes show significant advantages compared to the conventional optimized FAIDs using a generalized flooding structure network. The proposed approach achieves notable performance improvements without increasing the complexity of the decoding algorithm. We also classify the check nodes by row weights of the parity-check matrix, and train different LUTs for different categories of nodes for each iteration. This approach achieves further performance improvements in online decoding and significant complexity reduction compared to the exhaustive search method.
In this paper, a recurrent quantized neural network with a layered structure is designed to optimize the look-up table (LUT) in the layered finite alphabet iterative decoders (FAIDs). By using straight-through estimators to soften the backward gradient propagation of some low-precision activation functions, the quantization thresholds and quantization levels in the LUTs are optimized as parameters in the network. Performance evaluations on decoding codewords encoded with quasi-cyclic low-density parity-check codes show significant advantages compared to the conventional optimized FAIDs using a generalized flooding structure network. The proposed approach achieves notable performance improvements without increasing the complexity of the decoding algorithm. We also classify the check nodes by row weights of the parity-check matrix, and train different LUTs for different categories of nodes for each iteration. This approach achieves further performance improvements in online decoding and significant complexity reduction compared to the exhaustive search method.
This letter proposes a novel approach for designing non-surjective (NS) finite alphabet iterative decoders (FAIDs) for quasi-cyclic low-density parity-check (LDPC) codes, especially 5G LDPC codes. We employ recurrent quantized neural networks to optimize the look-up tables used in NS-FAIDs, the design of which is the kernel of FAIDs. During the optimization of LUTs, the quantized message alphabets and quantization thresholds are jointly designed. To cope with the untrainable problem of quantization thresholds in the existing neural-network-based linear FAIDs, we use softmax distribution to soften the implied one-hot distribution of quantization thresholds, making it trainable in the neural network. The proposed decoders offer enhanced universality compared to existing neural network-based linear FAIDs, making them directly applicable to 5G LDPC codes with support for 2-bit quantization over the additive white Gaussian noise channel. Additionally, they significantly outperform the original NS-FAID in terms of performance.
In this paper, we propose a new approach to design finite alphabet iterative decoders (FAIDs) for Low-Density Parity Check (LDPC) codes over binary symmetric channel (BSC) via recurrent quantized neural networks (RQNN). We focus on the linear FAID class and use RQNNs to optimize the message update look-up tables by jointly training their message levels and RQNN parameters. Existing neural networks for channel coding work well over Additive White Gaussian Noise Channel (AWGNC) but are inefficient over BSC due to the finite channel values of BSC fed into neural networks. We propose the bit error rate (BER) as the loss function to train the RQNNs over BSC. The low precision activations in the RQNN and quantization in the BER cause a critical issue that their gradients vanish almost everywhere, making it difficult to use classical backward propagation. We leverage straight-through estimators as surrogate gradients to tackle this issue and provide a joint training scheme. We show that the framework is flexible for various code lengths and column weights. Specifically, in high column weight case, it automatically designs low precision linear FAIDs with superior performance, lower complexity, and faster convergence than the floating-point belief propagation algorithms in waterfall region.
In this paper, we introduce a method of using quantized neural networks (QNN) to design finite alphabet message passing decoders (FAID) for Low-Density Parity Check (LDPC) codes. Specifically, we construct a neural network with low precision activations to optimize a FAID over Additive White Gaussian Noise Channel (AWGNC). The low precision activations cause a critical issue that their gradients vanish almost everywhere, making it difficult to use classical backward propagation. We introduce straight-through estimators (STE) to avoid this problem, by replacing zero derivatives of quantized activations with surrogate gradients in the chain rules. We present a systematic approach to train such networks while minimizing the bit error rate, which is a widely used and accurate metric to measure the performance of iterative decoders. Examples and simulations show that by training a QNN, a FAID with 3-bit of message and 4-bit of channel output can be obtained, which performs better than the more complex floating-point minsum decoding algorithm. This methodology is promising in the sense that it facilitates designing low-precision FAID for LDPC codes while maintaining good error performance in a flexible and efficient manner.
Low-density parity-check (LDPC) codes have been one of the most popular error correction candidates in modern communication and data storage systems for the past three decades due to their capacity-approaching error correction capability under the Belief Propagation (BP) decoding algorithms. In this dissertation, we investigate three topics related to efficient and effective decoding algorithms and the code design of LDPC codes. In the first part of this dissertation, we propose a methodology with the help of artificial intelligence to design Finite Alphabet Iterative Decoders (FAID) for LDPC codes, which are not only of great theoretical but also practical importance. This model-driven neural network (NN) framework unfolds the decoding iterations, introduces quantized neural networks (QNN) to model the FAIDs, and proposes a new cost function, namely, bit error rate (BER), to address the inefficiency in discrete-output channel scenarios. The low precision activations in the QNN and quantization in the cost functions cause a critical issue that their gradients vanish almost everywhere, making it difficult to use classical backward propagation. We design and leverage straight-through estimators as surrogate gradients to tackle this issue and provide a joint training scheme. These QNN-aided FAIDs, are capable of surpassing the floating-point BP algorithms, with much lower complexity and faster convergence. The QNN-aided approach is general, as it can be applied to both continuous and discrete-output channels. In the second part of this dissertation, we propose a new code construction of LDPC codes based on the conventional parity-check matrices of Reed-Solomon (RS) codes. We investigate the girth and cycle structure and introduce an efficient searching algorithm to reduce short cycles in their Tanner graphs and to construct LDPC codes with a girth of at least eight. We use masking to further reduce the number of short cycles and to adjust the column and row weights of the parity-check matrices of the constructed codes for performance improvement. Our designs are systematic and flexible, which balance the requirements of good waterfall performance and low error-floor while maintaining a structure that facilitates implementation. In the third part of this dissertation, we investigate binary quasi-cyclic (QC) LDPC codes and their capability to correct phased bursts of erasures. We prove that the peeling algorithm when applied to such parity-check matrices of column weight two can correct all erasures that can be corrected by a maximum likelihood (ML) decoder. We show that by modifying the parity-check matrices of column weight two and globally coupling them, the erasure correcting capability can be further enhanced. For the Additive White Gaussian Noise (AWGN) channel, we design QC LDPC codes with parity-check matrices of column weight three or more that can correct phased bursts of erasures and perform well over the AWGN channel. Our analysis of QC-LDPC codes with parity-check matrices of column weight two is rather comprehensive as we determine, for all such codes, their dimensions, minimum Hamming distances, and their capabilities to correct phased bursts.
Decoder diversity is a powerful error correction framework in which a collection of decoders collaboratively correct a set of error patterns otherwise uncorrectable by any individual decoder. In this paper, we propose a new approach to design the decoder diversity of finite alphabet iterative decoders (FAIDs) for Low-Density Parity Check (LDPC) codes over the binary symmetric channel (BSC), for the purpose of lowering the error floor while guaranteeing the waterfall performance. The proposed decoder diversity is achieved by training a recurrent quantized neural network (RQNN) to learn/design FAIDs. We demonstrated for the first time that a machine-learned decoder can surpass in performance a man-made decoder of the same complexity. As RQNNs can model a broad class of FAIDs, they are capable of learning an arbitrary FAID. To provide sufficient knowledge of the error floor to the RQNN, the training sets are constructed by sampling from the set of most problematic error patterns - trapping sets. In contrast to the existing methods that use the cross-entropy function as the loss function, we introduce a frame-error-rate (FER) based loss function to train the RQNN with the objective of correcting specific error patterns rather than reducing the bit error rate (BER). The examples and simulation results show that the RQNN-aided decoder diversity increases the error correction capability of LDPC codes and lowers the error floor.
Decoder diversity is a powerful error correction framework in which a collection of decoders collaboratively correct a set of error patterns otherwise uncorrectable by any individual decoder. In this paper, we propose a new approach to design the decoder diversity of finite alphabet iterative decoders (FAIDs) for Low-Density Parity Check (LDPC) codes over the binary symmetric channel (BSC), for the purpose of lowering the error floor while guaranteeing the waterfall performance. The proposed decoder diversity is achieved by training a recurrent quantized neural network (RQNN) to learn/design FAIDs. We demonstrated for the first time that a machine-learned decoder can surpass in performance a man-made decoder of the same complexity. As RQNNs can model a broad class of FAIDs, they are capable of learning an arbitrary FAID. To provide sufficient knowledge of the error floor to the RQNN, the training sets are constructed by sampling from the set of most problematic error patterns - trapping sets. In contrast to the existing methods that use the cross-entropy function as the loss function, we introduce a frame-error-rate (FER) based loss function to train the RQNN with the objective of correcting specific error patterns rather than reducing the bit error rate (BER). The examples and simulation results show that the RQNN-aided decoder diversity increases the error correction capability of LDPC codes and lowers the error floor.