arXiv:1702.07560

RNN Decoding of Linear Block Codes

Eliya Nachmani, Elad Marciano, David Burshtein, Yair Be'ery

Published Feb 24, 2017• 74 citations

Abstract

Designing a practical, low complexity, close to optimal, channel decoder for powerful algebraic codes with short to moderate block length is an open research problem. Recently it has been shown that a feed-forward neural network architecture can improve on standard belief propagation decoding, despite the large example space. In this paper we introduce a recurrent neural network architecture for decoding linear block codes. Our method shows comparable bit error rate results compared to the feed-forward neural network with significantly less parameters. We also demonstrate improved performance over belief propagation on sparser Tanner graph representations of the codes. Furthermore, we demonstrate that the RNN decoder can be used to improve the performance or alternatively reduce the computational complexity of the mRRD algorithm for low complexity, close to optimal, decoding of short BCH codes.

DOI: 10.48550/arxiv.1702.07560

Related papers

Deep Learning Methods for Improved Decoding of Linear Codes

Eliya Nachmani, Elad Marciano, Loren Lugosch +3 more

The problem of low complexity, close to optimal, channel decoding of linear codes with short to moderate block length is considered. It is shown that deep learning methods can be used to improve a standard belief propagation decoder, despite the large example space. Similar improvements are obtained for the min-sum algorithm. It is also shown that tying the parameters of the decoders across iterations, so as to form a recurrent neural network architecture, can be implemented with comparable results. The advantage is that significantly less parameters are required. We also introduce a recurrent neural decoder architecture based on the method of successive relaxation. Improvements over standard belief propagation are also observed on sparser Tanner graph representations of the codes. Furthermore, we demonstrate that the neural belief propagation decoder can be used to improve the performance, or alternatively reduce the computational complexity, of a close to optimal decoder of short BCH codes.

arXiv:1706.07043IEEE Journal of Selected Topics in Signal ProcessingJan 1, 2018546 citations

Improved Recurrent Neural Network based BP Decoding Algorithm for Polar Codes

Xue-lu DENG, Da-qin PENG

In recent years, the emerging Deep Learning (DL) technology has made progress in the field of decoding. Current polar code neural network decoder has faster convergence speed and better Bit Error Rate (BER) performance than Belief Propagation (BP) decoding. However, it still has the problem of high computational complexity. Therefore, in order to improve this problem, this paper adopts the idea of improving information update in the iterative process, and proposes a Recurrent Neural Network (RNN) Offset Min-Sum (OMS) BP decoding algorithm that improves Left information update (RNN-OMSBP-L) . The simulation results show that, compared with the Deep Neural Network (DNN) BP(DNN-BP) decoding algorithm, this algorithm replaces all multiplications with a cost of 6.25% addition. Compared with the current optimized RNN OMS and approximate BP(RNN-OMS-BP) decoding algorithm, the decoding algorithm in this paper uses improved information to reduce 25% of the addition operations with almost no loss in BER performance, while saving part of the storage space overhead. Under the same BER performance, it reduces the number of iterations by 37.5%.

Aug 1, 20220 citations

A Low-Complexity Residual Neural Network based BP Decoder for Polar Codes

Yu‐Ting Huang, Meixiang Zhang, Yin Dou

With the development of communication technology, it is urgent to design channel decoding algorithms to maximize the high throughput and minimize the latency. As the capacity achieving polar codes are adopted by 5-th generation wireless system (5G), it is required to design efficient the decoding algorithms for polar codes with high performance and low complexity. Deep neural network (DNN)-based algorithms which have shown superiority in other fields, can obtain better performance using multi-scale belief propagation (BP) algorithm, whereas the decoding complexity needs to be decreased. We propose a residual neural network based belief propagation (ResNet- BP) algorithm to solve the inherent drawbacks in the existing methods, by reducing the number of variables in the network and reducing the complexity in terms of multiplication. Simulation results demonstrate that the proposed method can achieve the approximating performance of recurrent neural network-based belief propagation (RNN-BP) with a lower number of iterations.

International Conference on Wireless Communications and Signal Processing (WCSP)Oct 21, 20206 citations

Deep Learning for Decoding of Linear Codes - A Syndrome-Based Approach

Amir Bennatan, Yoni Choukroun, Pavel Kisilev

We present a novel framework for applying deep neural networks (DNN) to soft decoding of linear codes at arbitrary block lengths. Unlike other approaches, our framework allows unconstrained DNN design, enabling the free application of powerful designs that were developed in other contexts. Our method is robust to overfitting that inhibits many competing methods, which follows from the exponentially large number of codewords required for their training. We achieve this by transforming the channel output before feeding it to the network, extracting only the syndrome of the hard decisions and the channel output reliabilities. We prove analytically that this approach does not involve any intrinsic performance penalty, and guarantees the generalization of performance obtained during training. Our best results are obtained using a recurrent neural network (RNN) architecture combined with simple preprocessing by permutation. We provide simulation results that demonstrate performance that sometimes approaches that of the ordered statistics decoding (OSD) algorithm.

arXiv:1802.04741IEEE International Symposium on Information Theory (ISIT)Jun 1, 2018100 citations

Hyper-Graph-Network Decoders for Block Codes

Eliya Nachmani, Lior Wolf

Neural decoders were shown to outperform classical message passing techniques for short BCH codes. In this work, we extend these results to much larger families of algebraic block codes, by performing message passing with graph neural networks. The parameters of the sub-network at each variable-node in the Tanner graph are obtained from a hypernetwork that receives the absolute values of the current message as input. To add stability, we employ a simplified version of the arctanh activation that is based on a high order Taylor approximation of this activation function. Our results show that for a large number of algebraic block codes, from diverse families of codes (BCH, LDPC, Polar), the decoding obtained with our method outperforms the vanilla belief propagation method as well as other learning techniques from the literature.

arXiv:1909.09036Sep 5, 201952 citations

Deep Learning for Channel Coding

Kou Tian

Next-generation communication systems face challenges in meeting the growing demand of ultra-reliable low-latency communications (URLLC), which require advanced physical-layer designs, especially in channel coding. To satisfy the strict reliability and latency constraints of URLLC in 5G and emerging 6G networks, it is essential to use short codes with robust error-correction capabilities. This thesis explores the application of deep learning methodologies to develop advanced channel coding frameworks for short linear block codes. First, I propose a novel decoding algorithm for short block codes based on an edge-weighted graph neural network (EW-GNN). The EW-GNN decoder operates on the Tanner graph with an iterative message-passing structure, algorithmically aligning with conventional belief propagation (BP) decoding. The decoder is characterized by scalability, with trainable parameters independent of codeword length, and outperforms BP and existing deep-learning-based methods. I present a novel auto-encoder integrating deep reinforcement learning (DRL) and graph neural networks (GNN) for end-to-end channel coding. Code generation is modeled as a Markov decision process to optimize error rates and algebraic properties. An iterative joint training of the DRL-based code designer and the EW-GNN decoder is performed to optimise the end-to-end encoding and decoding process. Simulation results show significant improvements over traditional coding schemes including LDPC and BCH codes. Finally, I design a supervised learning framework using an integrated auto-encoder architecture with a systematic code generator and matrix-based adaptive double-weighted (MA-DW) decoder. This approach achieves superior performance compared to conventional systems while reducing training complexity through joint optimization of code design and decoding parameters.

Jan 1, 20250 citations