Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Quantization of Binary-Input Discrete Memoryless Channels, with Applications to LDPC Decoding

Brian M. Kurkoski, Hideki Yagi

Published Jul 28, 2011• 13 citations
OpenAlex

Abstract

Abstract—The quantization of the output of a binary-input discrete memoryless channel to a smaller number of levels is considered. The optimal quantizer, in the sense of maximizing mutual information between the channel input and the quantizer output, may be found by an algorithm with complexity which is quadratic in the number of channel outputs. This is a concave optimization problem, and results from the field of concave optimization are invoked. The quantizer design algorithm is a realization of a dynamic program. Then, this algorithm is applied to the design of message-passing decoders for low-density parity-check codes, over arbitrary discrete memoryless channels. A general, systematic method to find message-passing decoding maps which maximize mutual information at each iteration is given. This may contrasted with existing quantized message-passing algorithms which are heuristically derived. The method finds message-passing decoding maps similar to those given by Richardson and Urbanke’s Algorithm E. Using four bits per message, noise thresholds similar to belief-propagation decoding are obtained. Index Terms—discrete memoryless channel, channel quantiza-tion, mutual information maximization, LDPC decoding I.

Related papers

Noise Thresholds for Discrete LDPC Decoding Mappings

Brian M. Kurkoski, Kazuhiko Yamaguchi, Kingo Kobayashi

For decoding low-density parity-check (LDPC) codes on discrete memoryless channels, a method to quantize messages and to find message-passing decoding functions for the variable and check nodes is developed. These are used to obtain noise thresholds by density evolution. The message-passing decoding alphabet is restricted to be discrete with a fixed maximum alphabet size. Discrete quantization is required to obtain this fixed alphabet size; a greedy algorithm which uses the mutual information between the code bit and message is presented. It is argued that using this message-passing decoding framework is more efficient for approaching channel capacity than simply quantizing the belief-propagation algorithm. This method is evaluated using regular LDPC codes on the binary symmetric channel. Using a maximum alphabet size of 16 (4 bits), noise thresholds close to those of belief propagation are obtained.

Globecom•Jan 1, 2008•51 citations

Decoding LDPC codes with mutual information-maximizing lookup tables

Francisco Javier Cuadros Romero, Brian M. Kurkoski

A recent result has shown connections between statistical learning theory and channel quantization. In this paper, we present a practical application of this result to the implementation of LDPC decoders. In particular, we describe a technique for designing the message-passing decoder mappings (or lookup tables) based on the ideas of channel quantization. This technique is not derived from sum-product algorithm or any other LDPC decoding algorithm. Instead, the proposed algorithm is based on an optimal quantizer in the sense of maximization of mutual information, which is inserted in the density evolution algorithm to generate the lookup tables. This algorithm has low complexity since it only employs 3-bit messages and lookup tables, which can be easily implemented in hardware. Two quantized versions of the min-sum decoding algorithm are used for comparison. Simulation results for a binary-input AWGN channel show 0.3 dB and 1.2 dB gains versus the two quantized min-sum algorithms. On the binary symmetric channel also a gain is seen.

IEEE International Symposium on Information Theory (ISIT)•Jun 1, 2015•35 citations

The capacity of low-density parity-check codes under message-passing decoding

Tom Richardson, Rudiger L. Urbanke

We present a general method for determining the capacity of low-density parity-check (LDPC) codes under message-passing decoding when used over any binary-input memoryless channel with discrete or continuous output alphabets. Transmitting at rates below this capacity, a randomly chosen element of the given ensemble will achieve an arbitrarily small target probability of error with a probability that approaches one exponentially fast in the length of the code. (By concatenating with an appropriate outer code one can achieve a probability of error that approaches zero exponentially fast in the length of the code with arbitrarily small loss in rate.) Conversely, transmitting at rates above this capacity the probability of error is bounded away from zero by a strictly positive constant which is independent of the length of the code and of the number of iterations performed. Our results are based on the observation that the concentration of the performance of the decoder around its average performance, as observed by Luby et al. in the case of a binary-symmetric channel and a binary message-passing algorithm, is a general phenomenon. For the particularly important case of belief-propagation decoders, we provide an effective algorithm to determine the corresponding capacity to any desired degree of accuracy. The ideas presented in this paper are broadly applicable and extensions of the general method to low-density parity-check codes over larger alphabets, turbo codes, and other concatenated coding schemes are outlined.

IEEE Transactions on Information Theory•Jan 1, 2001•3,067 citations

Density Evolution for GF(q) LDPC Codes Via Simplified Message-passing Sets

Brian M. Kurkoski, Kazuhiko Yamaguchi, Kingo Kobayashi

A message-passing decoder for GF(q) low-density parity-check codes is defined, which uses discrete messages from a subset of all possible binary vectors of length q. The proposed algorithm is a generalization to GF(q) of Richardson and Urbanke's decoding "Algorithm E" for binary codes. Density evolution requires a mapping between the probability distribution spaces for the channel, variable and check messages, and under the proposed algorithm, exact density evolution is possible. Symmetries in the message densities permit reduction in the size of the probability distribution space. Noise thresholds are obtained for LDPC codes on discrete memoryless channels, and as with Algorithm E, are remarkably close to noise thresholds under more complex belief propagation decoding.

Information Theory and Applications Workshop (ITA)•Jan 1, 2007•9 citations

Message alignment for discrete LDPC decoders with quadrature amplitude modulation

Jan Lewandowsky, Maximilian Stark, Gerhard Bauch

Recent works describe the design of discrete decoders for low-density parity-check codes by application of mutual information maximizing clustering algorithms in discrete density evolution. In the resulting discrete message passing decoders only integers are exchanged and node operations become simple lookup operations. Earlier works only describe discrete decoders for binary modulation schemes. This paper presents a new technique called message alignment which enables to design discrete decoders for higher-order modulation schemes. First, we design a channel output quantizer for quadrature amplitude modulation with the Information Bottleneck method. The quantizer attempts to preserve the relevant information on the modulation symbols. Afterwards, we illustrate that the assignment of several bits to one modulation symbol does not allow straightforward decoder design with the available discrete density evolution technique. The proposed message alignment solves this problem. The resulting discrete decoder is compared with state-of-the-art decoders using bit error rate simulations.

IEEE International Symposium on Information Theory (ISIT)•Jun 1, 2017•16 citations

Decoding LDPC Codes With Locally Maximum-Likelihood Binary Messages

Chris Winstead, Emmanuel Boutillon

A new low-complexity message passing algorithm is described for decoding low-density parity-check (LDPC) codes by exchanging binary messages. The algorithm computes the local maximum-likelihood binary message (LMLBM) at each symbol node, given the combination of local channel information and partial syndrome components from adjacent parity check nodes. When channel information is quantized, the locally ML messages are pre-computed and stored in a dynamic global lookup table. The proposed algorithm uses memoryless extrinsic messages so that density evolution thresholds can be directly computed. Thresholds are obtained for regular ensembles, predicting good performance on quantized binary-input additive white Gaussian noise (biAWGN) channels.

IEEE Communications Letters•Oct 30, 2014•2 citations
Abstract—The quantization of the output of a binary-input discrete memoryless channel to a smaller number of levels is considered. The optimal quantizer, in the sense of maximizing mutual information between the channel input and the quantizer output, may be found by an algorithm with complexity which is quadratic in the number of channel outputs. This is a concave optimization problem, and results from the field of concave optimization are invoked. The quantizer design algorithm is a realization of a dynamic program. Then, this algorithm is applied to the design of message-passing decoders for low-density parity-check codes, over arbitrary discrete memoryless channels. A general, systematic method to find message-passing decoding maps which maximize mutual information at each iteration is given. This may contrasted with existing quantized message-passing algorithms which are heuristically derived. The method finds message-passing decoding maps similar to those given by Richardson and Urbanke’s Algorithm E. Using four bits per message, noise thresholds similar to belief-propagation decoding are obtained. Index Terms—discrete memoryless channel, channel quantiza-tion, mutual information maximization, LDPC decoding I.
For decoding low-density parity-check (LDPC) codes on discrete memoryless channels, a method to quantize messages and to find message-passing decoding functions for the variable and check nodes is developed. These are used to obtain noise thresholds by density evolution. The message-passing decoding alphabet is restricted to be discrete with a fixed maximum alphabet size. Discrete quantization is required to obtain this fixed alphabet size; a greedy algorithm which uses the mutual information between the code bit and message is presented. It is argued that using this message-passing decoding framework is more efficient for approaching channel capacity than simply quantizing the belief-propagation algorithm. This method is evaluated using regular LDPC codes on the binary symmetric channel. Using a maximum alphabet size of 16 (4 bits), noise thresholds close to those of belief propagation are obtained.
A recent result has shown connections between statistical learning theory and channel quantization. In this paper, we present a practical application of this result to the implementation of LDPC decoders. In particular, we describe a technique for designing the message-passing decoder mappings (or lookup tables) based on the ideas of channel quantization. This technique is not derived from sum-product algorithm or any other LDPC decoding algorithm. Instead, the proposed algorithm is based on an optimal quantizer in the sense of maximization of mutual information, which is inserted in the density evolution algorithm to generate the lookup tables. This algorithm has low complexity since it only employs 3-bit messages and lookup tables, which can be easily implemented in hardware. Two quantized versions of the min-sum decoding algorithm are used for comparison. Simulation results for a binary-input AWGN channel show 0.3 dB and 1.2 dB gains versus the two quantized min-sum algorithms. On the binary symmetric channel also a gain is seen.
We present a general method for determining the capacity of low-density parity-check (LDPC) codes under message-passing decoding when used over any binary-input memoryless channel with discrete or continuous output alphabets. Transmitting at rates below this capacity, a randomly chosen element of the given ensemble will achieve an arbitrarily small target probability of error with a probability that approaches one exponentially fast in the length of the code. (By concatenating with an appropriate outer code one can achieve a probability of error that approaches zero exponentially fast in the length of the code with arbitrarily small loss in rate.) Conversely, transmitting at rates above this capacity the probability of error is bounded away from zero by a strictly positive constant which is independent of the length of the code and of the number of iterations performed. Our results are based on the observation that the concentration of the performance of the decoder around its average performance, as observed by Luby et al. in the case of a binary-symmetric channel and a binary message-passing algorithm, is a general phenomenon. For the particularly important case of belief-propagation decoders, we provide an effective algorithm to determine the corresponding capacity to any desired degree of accuracy. The ideas presented in this paper are broadly applicable and extensions of the general method to low-density parity-check codes over larger alphabets, turbo codes, and other concatenated coding schemes are outlined.
A message-passing decoder for GF(q) low-density parity-check codes is defined, which uses discrete messages from a subset of all possible binary vectors of length q. The proposed algorithm is a generalization to GF(q) of Richardson and Urbanke's decoding "Algorithm E" for binary codes. Density evolution requires a mapping between the probability distribution spaces for the channel, variable and check messages, and under the proposed algorithm, exact density evolution is possible. Symmetries in the message densities permit reduction in the size of the probability distribution space. Noise thresholds are obtained for LDPC codes on discrete memoryless channels, and as with Algorithm E, are remarkably close to noise thresholds under more complex belief propagation decoding.
Recent works describe the design of discrete decoders for low-density parity-check codes by application of mutual information maximizing clustering algorithms in discrete density evolution. In the resulting discrete message passing decoders only integers are exchanged and node operations become simple lookup operations. Earlier works only describe discrete decoders for binary modulation schemes. This paper presents a new technique called message alignment which enables to design discrete decoders for higher-order modulation schemes. First, we design a channel output quantizer for quadrature amplitude modulation with the Information Bottleneck method. The quantizer attempts to preserve the relevant information on the modulation symbols. Afterwards, we illustrate that the assignment of several bits to one modulation symbol does not allow straightforward decoder design with the available discrete density evolution technique. The proposed message alignment solves this problem. The resulting discrete decoder is compared with state-of-the-art decoders using bit error rate simulations.
A new low-complexity message passing algorithm is described for decoding low-density parity-check (LDPC) codes by exchanging binary messages. The algorithm computes the local maximum-likelihood binary message (LMLBM) at each symbol node, given the combination of local channel information and partial syndrome components from adjacent parity check nodes. When channel information is quantized, the locally ML messages are pre-computed and stored in a dynamic global lookup table. The proposed algorithm uses memoryless extrinsic messages so that density evolution thresholds can be directly computed. Thresholds are obtained for regular ensembles, predicting good performance on quantized binary-input additive white Gaussian noise (biAWGN) channels.