Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

IEEE Information Theory Workshop (ITW)

Bounds on the Tradeoff Between Decoding Complexity and Rate for Sparse-Graph Codes

Pulkit Grover

Published Sep 1, 2007• 11 citations
Publisher

Abstract

Khandekar and McEliece suggested the problem of per bit decoding complexity for capacity achieving sparse-graph codes as a function of their gap from the channel capacity. We consider the problem for the case of the binary symmetric channel. We derive a lower bound on this complexity for some codes on graphs for Belief Propagation decoding. For bounded degree LDPC and LDGM codes, any concatenation of the two, and punctured bounded-degree LDPC codes, this reduces to a lower bound of O (log (1/isin-)). The proof of this result leads to an interesting necessary condition on the code structures which could achieve capacity with bounded decoding complexity over BSC: the average edge-degree must converge to infinity while the average node-degree must be bounded. That is, one of the node degree distributions must have a finite mean and an infinite variance.

DOI
10.1109/itw.2007.4313073

Related papers

Tradeoff between decoding complexity and rate for codes on graphs

Pulkit Grover

We consider transmission over a general memoryless channel, with bounded decoding complexity per bit under message passing decoding. We show that the achievable rate is bounded below capacity if there is a finite success in the decoding in a specified number of operations per bit at the decoder for some codes on graphs. These codes include LDPC and LDGM codes. Good performance with low decoding complexity suggests strong local structures in the graphs of these codes, which are detrimental to the code rate asymptotically. The proof method leads to an interesting necessary condition on the code structures which could achieve capacity with bounded decoding complexity. We also show that if a code sequence achieves a rate epsilon close to the channel capacity, the decoding complexity scales at least as O(log(1/epsilon).

arXiv:cs/0702073•Feb 13, 2007•0 citations

Decoding Complexity of Irregular LDGM-LDPC Codes Over the BISOM Channels

Manik Raina, Predrag Spasojević

An irregular LDGM-LDPC code is studied as a sub-code of an LDPC code with some randomly \emph{punctured} output-bits. It is shown that the LDGM-LDPC codes achieve rates arbitrarily close to the channel-capacity of the binary-input symmetric-output memoryless (BISOM) channel with bounded \emph{complexity}. The measure of complexity is the average-degree (per information-bit) of the check-nodes for the factor-graph of the code. A lower-bound on the average degree of the check-nodes of the irregular LDGM-LDPC codes is obtained. The bound does not depend on the decoder used at the receiver. The stability condition for decoding the irregular LDGM-LDPC codes over the binary-erasure channel (BEC) under iterative-decoding with message-passing is described.

arXiv:1003.2454•Mar 12, 2010•0 citations

Capacity-Achieving Codes with Bounded Graphical Complexity on Noisy Channels

Chun-Hao Hsu, Achilleas Anastasopoulos

We introduce a new family of concatenated codes with an outer low-density parity-check (LDPC) code and an inner low-density generator matrix (LDGM) code, and prove that these codes can achieve capacity under any memoryless binary-input output-symmetric (MBIOS) channel using maximum-likelihood (ML) decoding with bounded graphical complexity, i.e., the number of edges per information bit in their graphical representation is bounded. In particular, we also show that these codes can achieve capacity on the binary erasure channel (BEC) under belief propagation (BP) decoding with bounded decoding complexity per information bit per iteration for all erasure probabilities in (0, 1). By deriving and analyzing the average weight distribution (AWD) and the corresponding asymptotic growth rate of these codes with a rate-1 inner LDGM code, we also show that these codes achieve the Gilbert-Varshamov bound with asymptotically high probability. This result can be attributed to the presence of the inner rate-1 LDGM code, which is demonstrated to help eliminate high weight codewords in the LDPC code while maintaining a vanishingly small amount of low weight codewords.

arXiv:cs/0509062•Allerton Conference on Communication, Control, and Computing•Sep 19, 2005•14 citations

Decoding complexity of irregular LDGM-LDPC codes over the BISOM channels

Manik Raina, Predrag Spasojević

An irregular LDGM-LDPC code is studied as a sub-code of an LDPC code with some randomly punctured output-bits. It is shown that the LDGM-LDPC codes achieve rates arbitrarily close to the channel-capacity of the binary-input symmetric-output memoryless (BISOM) channel with bounded complexity. The measure of complexity is the average-degree (per information-bit) of the check-nodes for the factor-graph of the code. A lower-bound on the average degree of the check-nodes of the irregular LDGM-LDPC codes is obtained. The bound does not depend on the decoder used at the receiver. The stability condition for decoding the irregular LDGM-LDPC codes over the binary-erasure channel (BEC) under iterative-decoding with message-passing is described.

Conference on Information Sciences and Systems•Mar 1, 2010•1 citation

Information-theoretic lower bounds on the bit error probability of codes on graphs

Igal Sason, Rudiger L. Urbanke

We present lower bounds on the bit error probability of binary linear block codes, and on the gap between the channel capacity and the rate of these codes for which reliable communica- tion is achievable. The results are valid for memo- ryless binary-input output-symmetric (MBIOS) chan- nels. The lower bounds provide a quantitative mea- sure for the number of cycles of bipartite graphs which represent good error-correcting codes. The tightness of the lower bounds is especially pronounced for the binary erasure channel (BEC); to this end, we analyze a sequence of ensembles of low-density parity-check (LDPC) codes which closely approach these bounds. I. Introduction If a linear block code can be represented by a cycle-free fac- tor graph which only includes variable nodes and parity-check nodes, then maximum-likelihood soft decision decoding can be performed with low complexity (1). Theorem 5 in (1) indicates that these graphs represent codes with very poor minimum distance. The bounds in (1) only refer to the minimum dis- tance of cycle-free codes, and in this work we derive lower bounds on the bit error probability and the gap to capacity for codes on graphs which are represented by bipartite graphs (with or without cycles). We derive in this work information- theoretic lower bounds on the bit error probability of a binary linear code used over MBIOS channels. The bounds are ex- pressed in terms of the density of an arbitrary parity-check matrix which represents the binary linear code (i.e., the num- ber of ones in the matrix normalized per information bit), and they are valid for codes whose bipartite graphs are with or without cycles. We introduce a quantitative measure for the cycles in a bipartite graph which represents a binary lin- ear code (where the graph only includes variable nodes and parity-check nodes). The bounds provide an interpretation for the tradeofi which exists between the bit error proba- bility and gap to capacity of an arbitrary LDPC code (i.e., its performance limitations), and the density of an arbitrary parity-check matrix which represents the code (where the lat- ter afiects the decoding complexity per information bit and per iteration, under a message-passing iterative decoding al- gorithm). We present in this work quantitative results which indicate that in order to approach the channel capacity with vanishing bit error probability, LDPC codes should not have too sparse parity-check matrices, as otherwise their inherent gap to capacity becomes large. In fact, we show that the density of any parity-check matrix and the fundamental num- ber of cycles of the graph which represents the code should increase at least like log 1 where designates the gap to ca- pacity under iterative message-passing decoding (see (2), (3)). The tightness of the lower bounds which are derived in this work (and whose validity is for any sequence of codes with 1 1.5 2 2.5 3 3.5 4 4.5

IEEE International Symposium on Information Theory (ISIT)•Jan 1, 2003•4 citations

On Universal Properties of Capacity-Approaching LDPC Code Ensembles

Igal Sason

This paper is focused on the derivation of some universal properties of capacity-approaching low-density parity-check (LDPC) code ensembles whose transmission takes place over memoryless binary-input output-symmetric (MBIOS) channels. Properties of the degree distributions, graphical complexity, and the number of fundamental cycles in the bipartite graphs are considered via the derivation of information-theoretic bounds. These bounds are expressed in terms of the target block/bit error probability and the gap (in rate) to capacity. Most of the bounds are general for any decoding algorithm, and some others are proved under belief propagation (BP) decoding. Proving these bounds under a certain decoding algorithm, validates them automatically also under any suboptimal decoding algorithm. A proper modification of these bounds makes them universal for the set of all MBIOS channels which exhibit a given capacity. Bounds on the degree distributions and graphical complexity apply to finite-length LDPC codes and to the asymptotic case of an infinite block length. The bounds are compared with capacity-approaching LDPC code ensembles under BP decoding, and they are shown to be informative and are easy to calculate. Finally, some interesting open problems are considered.

arXiv:0709.0599•IEEE Transactions on Information Theory•Jun 16, 2009•41 citations
Khandekar and McEliece suggested the problem of per bit decoding complexity for capacity achieving sparse-graph codes as a function of their gap from the channel capacity. We consider the problem for the case of the binary symmetric channel. We derive a lower bound on this complexity for some codes on graphs for Belief Propagation decoding. For bounded degree LDPC and LDGM codes, any concatenation of the two, and punctured bounded-degree LDPC codes, this reduces to a lower bound of O (log (1/isin-)). The proof of this result leads to an interesting necessary condition on the code structures which could achieve capacity with bounded decoding complexity over BSC: the average edge-degree must converge to infinity while the average node-degree must be bounded. That is, one of the node degree distributions must have a finite mean and an infinite variance.
We consider transmission over a general memoryless channel, with bounded decoding complexity per bit under message passing decoding. We show that the achievable rate is bounded below capacity if there is a finite success in the decoding in a specified number of operations per bit at the decoder for some codes on graphs. These codes include LDPC and LDGM codes. Good performance with low decoding complexity suggests strong local structures in the graphs of these codes, which are detrimental to the code rate asymptotically. The proof method leads to an interesting necessary condition on the code structures which could achieve capacity with bounded decoding complexity. We also show that if a code sequence achieves a rate epsilon close to the channel capacity, the decoding complexity scales at least as O(log(1/epsilon).
An irregular LDGM-LDPC code is studied as a sub-code of an LDPC code with some randomly \emph{punctured} output-bits. It is shown that the LDGM-LDPC codes achieve rates arbitrarily close to the channel-capacity of the binary-input symmetric-output memoryless (BISOM) channel with bounded \emph{complexity}. The measure of complexity is the average-degree (per information-bit) of the check-nodes for the factor-graph of the code. A lower-bound on the average degree of the check-nodes of the irregular LDGM-LDPC codes is obtained. The bound does not depend on the decoder used at the receiver. The stability condition for decoding the irregular LDGM-LDPC codes over the binary-erasure channel (BEC) under iterative-decoding with message-passing is described.
We introduce a new family of concatenated codes with an outer low-density parity-check (LDPC) code and an inner low-density generator matrix (LDGM) code, and prove that these codes can achieve capacity under any memoryless binary-input output-symmetric (MBIOS) channel using maximum-likelihood (ML) decoding with bounded graphical complexity, i.e., the number of edges per information bit in their graphical representation is bounded. In particular, we also show that these codes can achieve capacity on the binary erasure channel (BEC) under belief propagation (BP) decoding with bounded decoding complexity per information bit per iteration for all erasure probabilities in (0, 1). By deriving and analyzing the average weight distribution (AWD) and the corresponding asymptotic growth rate of these codes with a rate-1 inner LDGM code, we also show that these codes achieve the Gilbert-Varshamov bound with asymptotically high probability. This result can be attributed to the presence of the inner rate-1 LDGM code, which is demonstrated to help eliminate high weight codewords in the LDPC code while maintaining a vanishingly small amount of low weight codewords.
An irregular LDGM-LDPC code is studied as a sub-code of an LDPC code with some randomly punctured output-bits. It is shown that the LDGM-LDPC codes achieve rates arbitrarily close to the channel-capacity of the binary-input symmetric-output memoryless (BISOM) channel with bounded complexity. The measure of complexity is the average-degree (per information-bit) of the check-nodes for the factor-graph of the code. A lower-bound on the average degree of the check-nodes of the irregular LDGM-LDPC codes is obtained. The bound does not depend on the decoder used at the receiver. The stability condition for decoding the irregular LDGM-LDPC codes over the binary-erasure channel (BEC) under iterative-decoding with message-passing is described.
We present lower bounds on the bit error probability of binary linear block codes, and on the gap between the channel capacity and the rate of these codes for which reliable communica- tion is achievable. The results are valid for memo- ryless binary-input output-symmetric (MBIOS) chan- nels. The lower bounds provide a quantitative mea- sure for the number of cycles of bipartite graphs which represent good error-correcting codes. The tightness of the lower bounds is especially pronounced for the binary erasure channel (BEC); to this end, we analyze a sequence of ensembles of low-density parity-check (LDPC) codes which closely approach these bounds. I. Introduction If a linear block code can be represented by a cycle-free fac- tor graph which only includes variable nodes and parity-check nodes, then maximum-likelihood soft decision decoding can be performed with low complexity (1). Theorem 5 in (1) indicates that these graphs represent codes with very poor minimum distance. The bounds in (1) only refer to the minimum dis- tance of cycle-free codes, and in this work we derive lower bounds on the bit error probability and the gap to capacity for codes on graphs which are represented by bipartite graphs (with or without cycles). We derive in this work information- theoretic lower bounds on the bit error probability of a binary linear code used over MBIOS channels. The bounds are ex- pressed in terms of the density of an arbitrary parity-check matrix which represents the binary linear code (i.e., the num- ber of ones in the matrix normalized per information bit), and they are valid for codes whose bipartite graphs are with or without cycles. We introduce a quantitative measure for the cycles in a bipartite graph which represents a binary lin- ear code (where the graph only includes variable nodes and parity-check nodes). The bounds provide an interpretation for the tradeofi which exists between the bit error proba- bility and gap to capacity of an arbitrary LDPC code (i.e., its performance limitations), and the density of an arbitrary parity-check matrix which represents the code (where the lat- ter afiects the decoding complexity per information bit and per iteration, under a message-passing iterative decoding al- gorithm). We present in this work quantitative results which indicate that in order to approach the channel capacity with vanishing bit error probability, LDPC codes should not have too sparse parity-check matrices, as otherwise their inherent gap to capacity becomes large. In fact, we show that the density of any parity-check matrix and the fundamental num- ber of cycles of the graph which represents the code should increase at least like log 1 where designates the gap to ca- pacity under iterative message-passing decoding (see (2), (3)). The tightness of the lower bounds which are derived in this work (and whose validity is for any sequence of codes with 1 1.5 2 2.5 3 3.5 4 4.5
This paper is focused on the derivation of some universal properties of capacity-approaching low-density parity-check (LDPC) code ensembles whose transmission takes place over memoryless binary-input output-symmetric (MBIOS) channels. Properties of the degree distributions, graphical complexity, and the number of fundamental cycles in the bipartite graphs are considered via the derivation of information-theoretic bounds. These bounds are expressed in terms of the target block/bit error probability and the gap (in rate) to capacity. Most of the bounds are general for any decoding algorithm, and some others are proved under belief propagation (BP) decoding. Proving these bounds under a certain decoding algorithm, validates them automatically also under any suboptimal decoding algorithm. A proper modification of these bounds makes them universal for the set of all MBIOS channels which exhibit a given capacity. Bounds on the degree distributions and graphical complexity apply to finite-length LDPC codes and to the asymptotic case of an infinite block length. The bounds are compared with capacity-approaching LDPC code ensembles under BP decoding, and they are shown to be informative and are easy to calculate. Finally, some interesting open problems are considered.