Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:1802.04741 · IEEE International Symposium on Information Theory (ISIT)

Deep Learning for Decoding of Linear Codes - A Syndrome-Based Approach

Amir Bennatan, Yoni Choukroun, Pavel Kisilev

Published Jun 1, 2018• 100 citations
arXivHTMLPDF

Abstract

We present a novel framework for applying deep neural networks (DNN) to soft decoding of linear codes at arbitrary block lengths. Unlike other approaches, our framework allows unconstrained DNN design, enabling the free application of powerful designs that were developed in other contexts. Our method is robust to overfitting that inhibits many competing methods, which follows from the exponentially large number of codewords required for their training. We achieve this by transforming the channel output before feeding it to the network, extracting only the syndrome of the hard decisions and the channel output reliabilities. We prove analytically that this approach does not involve any intrinsic performance penalty, and guarantees the generalization of performance obtained during training. Our best results are obtained using a recurrent neural network (RNN) architecture combined with simple preprocessing by permutation. We provide simulation results that demonstrate performance that sometimes approaches that of the ordered statistics decoding (OSD) algorithm.

DOI
10.1109/isit.2018.8437530

Related papers

Soft-Output Deep Neural Network-Based Decoding

Dmitry Artemasov, Kirill Andreev, Pavel S. Rybin +1 more

Deep neural network (DNN)-based channel decoding is widely considered in the literature. The existing solutions are investigated for the case of hard output, i.e. when the decoder returns the estimated information word. At the same time, soft output decoding is of critical importance for iterative receivers and decoders. In this paper, we focus on the soft-output DNN-based decoding problem. We start with the syndrome-based approach proposed by Bennatan et al. (2018) and modify it to provide soft output in the additive white Gaussian noise (AWGN) channel. The new decoder can be considered as an approximation of the maximum a-posteriori (MAP) decoder with smaller computation complexity. We discuss various regularization functions for joint DNN-MAP training and compare the resulting distributions for [64], [45] (extended Bose-Chaudhuri-Hocquenghem) eBCH code. Finally, to demonstrate the soft-output quality we consider the turbo-product code with [64], [45] eBCH codes as row and column codes. We show that the resulting DNN-based scheme is very close to the MAP-based performance and significantly outperforms the solution based on the Chase decoder. We come to the conclusion that the new method is prospective for the challenging problem of DNN-based decoding of long codes consisting of short component codes.

arXiv:2304.08868•IEEE Globecom Workshops (GC Wkshps)•Dec 4, 2023•4 citations

RNN Decoding of Linear Block Codes

Eliya Nachmani, Elad Marciano, David Burshtein +1 more

Designing a practical, low complexity, close to optimal, channel decoder for powerful algebraic codes with short to moderate block length is an open research problem. Recently it has been shown that a feed-forward neural network architecture can improve on standard belief propagation decoding, despite the large example space. In this paper we introduce a recurrent neural network architecture for decoding linear block codes. Our method shows comparable bit error rate results compared to the feed-forward neural network with significantly less parameters. We also demonstrate improved performance over belief propagation on sparser Tanner graph representations of the codes. Furthermore, we demonstrate that the RNN decoder can be used to improve the performance or alternatively reduce the computational complexity of the mRRD algorithm for low complexity, close to optimal, decoding of short BCH codes.

arXiv:1702.07560•Feb 24, 2017•74 citations

Deep Learning for Channel Coding

Kou Tian

Next-generation communication systems face challenges in meeting the growing demand of ultra-reliable low-latency communications (URLLC), which require advanced physical-layer designs, especially in channel coding. To satisfy the strict reliability and latency constraints of URLLC in 5G and emerging 6G networks, it is essential to use short codes with robust error-correction capabilities. This thesis explores the application of deep learning methodologies to develop advanced channel coding frameworks for short linear block codes. First, I propose a novel decoding algorithm for short block codes based on an edge-weighted graph neural network (EW-GNN). The EW-GNN decoder operates on the Tanner graph with an iterative message-passing structure, algorithmically aligning with conventional belief propagation (BP) decoding. The decoder is characterized by scalability, with trainable parameters independent of codeword length, and outperforms BP and existing deep-learning-based methods. I present a novel auto-encoder integrating deep reinforcement learning (DRL) and graph neural networks (GNN) for end-to-end channel coding. Code generation is modeled as a Markov decision process to optimize error rates and algebraic properties. An iterative joint training of the DRL-based code designer and the EW-GNN decoder is performed to optimise the end-to-end encoding and decoding process. Simulation results show significant improvements over traditional coding schemes including LDPC and BCH codes. Finally, I design a supervised learning framework using an integrated auto-encoder architecture with a systematic code generator and matrix-based adaptive double-weighted (MA-DW) decoder. This approach achieves superior performance compared to conventional systems while reducing training complexity through joint optimization of code design and decoding parameters.

•Jan 1, 2025•0 citations

On a Unified Deep Neural Network Decoding Architecture

Dmitry Artemasov, Kirill Andreev, Alexey Frolov

In modern communication systems, multiple types of error-correcting codes can be utilized for different transmission scenarios. Therefore, the receiver should include the decoder compatible with multiple codes used in the transmission scheme, which results in the increase of the resources required for its implementation. In this paper, we investigate the possibility of training a single syndrome-based DNN decoder to solve the problem of unified decoding. We observe, that the syndrome-based approach allows to extend the unified decoding capabilities to the codes with considerably larger lengths in comparison to the initially described by Wang et al. (2018) method. Through numerical experiments, we show that the model trained for decoding a pair of moderate lengths codes (BCH and CRC-Aided Polar) achieves performance results comparable with classical decoding solutions, while sharing the same architecture and the set of trainable weights. We note, that the unification of a syndrome-based DNN decoder does not lead to large performance degradation, in comparison to the decoder trained on a single code. The approach described in the paper is promising in terms of reducing the hardware resources required to implement the decoder.

IEEE Vehicular Technology Conference•Oct 10, 2023•2 citations

Deep Learning Methods for Improved Decoding of Linear Codes

Eliya Nachmani, Elad Marciano, Loren Lugosch +3 more

The problem of low complexity, close to optimal, channel decoding of linear codes with short to moderate block length is considered. It is shown that deep learning methods can be used to improve a standard belief propagation decoder, despite the large example space. Similar improvements are obtained for the min-sum algorithm. It is also shown that tying the parameters of the decoders across iterations, so as to form a recurrent neural network architecture, can be implemented with comparable results. The advantage is that significantly less parameters are required. We also introduce a recurrent neural decoder architecture based on the method of successive relaxation. Improvements over standard belief propagation are also observed on sparser Tanner graph representations of the codes. Furthermore, we demonstrate that the neural belief propagation decoder can be used to improve the performance, or alternatively reduce the computational complexity, of a close to optimal decoder of short BCH codes.

arXiv:1706.07043•IEEE Journal of Selected Topics in Signal Processing•Jan 1, 2018•546 citations

On deep learning-based channel decoding

Tobias Gruber, Sebastian Cammerer, Jakob Hoydis +1 more

We revisit the idea of using deep neural networks for one-shot decoding of random and structured codes, such as polar codes. Although it is possible to achieve maximum a posteriori (MAP) bit error rate (BER) performance for both code families and for short codeword lengths, we observe that (i) structured codes are easier to learn and (ii) the neural network is able to generalize to codewords that it has never seen during training for structured, but not for random codes. These results provide some evidence that neural networks can learn a form of decoding algorithm, rather than only a simple classifier. We introduce the metric normalized validation error (NVE) in order to further investigate the potential and limitations of deep learning-based decoding with respect to performance and complexity.

Conference on Information Sciences and Systems•Mar 1, 2017•574 citations
We present a novel framework for applying deep neural networks (DNN) to soft decoding of linear codes at arbitrary block lengths. Unlike other approaches, our framework allows unconstrained DNN design, enabling the free application of powerful designs that were developed in other contexts. Our method is robust to overfitting that inhibits many competing methods, which follows from the exponentially large number of codewords required for their training. We achieve this by transforming the channel output before feeding it to the network, extracting only the syndrome of the hard decisions and the channel output reliabilities. We prove analytically that this approach does not involve any intrinsic performance penalty, and guarantees the generalization of performance obtained during training. Our best results are obtained using a recurrent neural network (RNN) architecture combined with simple preprocessing by permutation. We provide simulation results that demonstrate performance that sometimes approaches that of the ordered statistics decoding (OSD) algorithm.
Deep neural network (DNN)-based channel decoding is widely considered in the literature. The existing solutions are investigated for the case of hard output, i.e. when the decoder returns the estimated information word. At the same time, soft output decoding is of critical importance for iterative receivers and decoders. In this paper, we focus on the soft-output DNN-based decoding problem. We start with the syndrome-based approach proposed by Bennatan et al. (2018) and modify it to provide soft output in the additive white Gaussian noise (AWGN) channel. The new decoder can be considered as an approximation of the maximum a-posteriori (MAP) decoder with smaller computation complexity. We discuss various regularization functions for joint DNN-MAP training and compare the resulting distributions for [64], [45] (extended Bose-Chaudhuri-Hocquenghem) eBCH code. Finally, to demonstrate the soft-output quality we consider the turbo-product code with [64], [45] eBCH codes as row and column codes. We show that the resulting DNN-based scheme is very close to the MAP-based performance and significantly outperforms the solution based on the Chase decoder. We come to the conclusion that the new method is prospective for the challenging problem of DNN-based decoding of long codes consisting of short component codes.
Designing a practical, low complexity, close to optimal, channel decoder for powerful algebraic codes with short to moderate block length is an open research problem. Recently it has been shown that a feed-forward neural network architecture can improve on standard belief propagation decoding, despite the large example space. In this paper we introduce a recurrent neural network architecture for decoding linear block codes. Our method shows comparable bit error rate results compared to the feed-forward neural network with significantly less parameters. We also demonstrate improved performance over belief propagation on sparser Tanner graph representations of the codes. Furthermore, we demonstrate that the RNN decoder can be used to improve the performance or alternatively reduce the computational complexity of the mRRD algorithm for low complexity, close to optimal, decoding of short BCH codes.
Next-generation communication systems face challenges in meeting the growing demand of ultra-reliable low-latency communications (URLLC), which require advanced physical-layer designs, especially in channel coding. To satisfy the strict reliability and latency constraints of URLLC in 5G and emerging 6G networks, it is essential to use short codes with robust error-correction capabilities. This thesis explores the application of deep learning methodologies to develop advanced channel coding frameworks for short linear block codes. First, I propose a novel decoding algorithm for short block codes based on an edge-weighted graph neural network (EW-GNN). The EW-GNN decoder operates on the Tanner graph with an iterative message-passing structure, algorithmically aligning with conventional belief propagation (BP) decoding. The decoder is characterized by scalability, with trainable parameters independent of codeword length, and outperforms BP and existing deep-learning-based methods. I present a novel auto-encoder integrating deep reinforcement learning (DRL) and graph neural networks (GNN) for end-to-end channel coding. Code generation is modeled as a Markov decision process to optimize error rates and algebraic properties. An iterative joint training of the DRL-based code designer and the EW-GNN decoder is performed to optimise the end-to-end encoding and decoding process. Simulation results show significant improvements over traditional coding schemes including LDPC and BCH codes. Finally, I design a supervised learning framework using an integrated auto-encoder architecture with a systematic code generator and matrix-based adaptive double-weighted (MA-DW) decoder. This approach achieves superior performance compared to conventional systems while reducing training complexity through joint optimization of code design and decoding parameters.
In modern communication systems, multiple types of error-correcting codes can be utilized for different transmission scenarios. Therefore, the receiver should include the decoder compatible with multiple codes used in the transmission scheme, which results in the increase of the resources required for its implementation. In this paper, we investigate the possibility of training a single syndrome-based DNN decoder to solve the problem of unified decoding. We observe, that the syndrome-based approach allows to extend the unified decoding capabilities to the codes with considerably larger lengths in comparison to the initially described by Wang et al. (2018) method. Through numerical experiments, we show that the model trained for decoding a pair of moderate lengths codes (BCH and CRC-Aided Polar) achieves performance results comparable with classical decoding solutions, while sharing the same architecture and the set of trainable weights. We note, that the unification of a syndrome-based DNN decoder does not lead to large performance degradation, in comparison to the decoder trained on a single code. The approach described in the paper is promising in terms of reducing the hardware resources required to implement the decoder.
The problem of low complexity, close to optimal, channel decoding of linear codes with short to moderate block length is considered. It is shown that deep learning methods can be used to improve a standard belief propagation decoder, despite the large example space. Similar improvements are obtained for the min-sum algorithm. It is also shown that tying the parameters of the decoders across iterations, so as to form a recurrent neural network architecture, can be implemented with comparable results. The advantage is that significantly less parameters are required. We also introduce a recurrent neural decoder architecture based on the method of successive relaxation. Improvements over standard belief propagation are also observed on sparser Tanner graph representations of the codes. Furthermore, we demonstrate that the neural belief propagation decoder can be used to improve the performance, or alternatively reduce the computational complexity, of a close to optimal decoder of short BCH codes.
We revisit the idea of using deep neural networks for one-shot decoding of random and structured codes, such as polar codes. Although it is possible to achieve maximum a posteriori (MAP) bit error rate (BER) performance for both code families and for short codeword lengths, we observe that (i) structured codes are easier to learn and (ii) the neural network is able to generalize to codewords that it has never seen during training for structured, but not for random codes. These results provide some evidence that neural networks can learn a form of decoding algorithm, rather than only a simple classifier. We introduce the metric normalized validation error (NVE) in order to further investigate the potential and limitations of deep learning-based decoding with respect to performance and complexity.