Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Applications of correlation inequalities to low density graphical codes

Nicolas Macris

Published Jan 1, 2006• 0 citations
OpenAlex

Abstract

Abstract.: This contribution is based on the contents of a talk delivered at the Next-SigmaPhi conference held in Crete in August 2005. It is adressed to an audience of physicists with diverse horizons and does not assume any background in communications theory. Capacity approaching error correcting codes for channel communication known as Low Density Parity Check (LDPC) codes have attracted considerable attention from coding theorists in the last decade. Surprisingly strong connections with the theory of diluted spin glasses have been discovered. In this work we elucidate one new connection, namely that a class of correlation inequalities valid for Gaussian spin glasses can be applied to the theoretical analysis of LDPC codes. This allows for a rigorous comparison between the so called (optimal) maximum a posteriori and the computationaly efficient belief propagation decoders. The main ideas of the proofs are explained and we refer to recent works for the more lengthy technical details

Related papers

Griffith–Kelly–Sherman Correlation Inequalities: A Useful Tool in the Theory of Error Correcting Codes

Nicolas Macris

It is shown that a correlation inequality of statistical mechanics can be applied to linear low-density parity-check codes. Thanks to this tool we prove that, under a natural assumption, the exponential growth rate of regular low-density parity-check (LDPC) codes, can be computed exactly by iterative methods, at least on the interval where it is a concave function of the relative weight of code words. Then, considering communication over a binary input additive white Gaussian noise channel with a Poisson LDPC code we prove that, under a natural assumption, part of the GEXIT curve (associated to MAP decoding) can also be computed exactly by the belief propagation algorithm. The correlation inequality yields a sharp lower bound on the GEXIT curve. We also make an extension of the interpolation techniques that have recently led to rigorous results in spin glass theory and in the SAT problem

IEEE Transactions on Information Theory•Jan 23, 2007•49 citations

Correlation inequalities: a useful tool in the theory of LDPC codes

Nicolas Macris

It is shown that a correlation inequality of statistical mechanics can be applied to low-density parity-check codes. Thanks to this tool we prove that the growth rate of regular LDPC codes, can be exactly calculated by iterative methods, at least on the interval where it is a concave function of the relative weight of code words. We also consider communication over a binary input additive white Gaussian noise channel with a Poisson LDPC code and prove that (at least part of) the GEXIT curve (associated to MAP decoding) can also be computed exactly by the belief propagation decoder. In both problems, the correlation inequality yields sharp lower bounds. We also use a non trivial extension of the interpolation techniques that have recently led to rigorous results in spin glass theory and in the SAT problem

IEEE International Symposium on Information Theory (ISIT)•Jan 1, 2005•8 citations

Sharp Bounds on Generalized EXIT Functions

Nicolas Macris

We consider communication over binary-input memoryless symmetric channels with low-density parity-check (LDPC) codes. The relationship between maximum a posteriori and belief propagation decoding is investigated using a set of correlation inequalities that first appeared in statistical mechanics of Gaussian spin glasses. We prove bounds on generalized extrinsic information transfer (EXIT) functions, that are believed to be tight, and discuss their relationship with the ones obtained by the interpolation method.

IEEE Transactions on Information Theory•Jun 28, 2007•46 citations

Decay of correlations: An application to low-density parity check codes

Shrinivas Kudekar, Nicolas Macris

Recently the decay of correlations between bits of low density generator matrix (LDGM) codes have been investigated by using high temperature expansions from statistical physics. In this work we apply these ideas to a special class of low density parity check codes (LDPC) on the binary input Gaussian white noise channel (BIAWGNC). We give a rigorous derivation of the maximum a posteriori (MAP) Generalized EXIT curve (the derivative with respect to the noise parameter of the input-output conditional entropy) for high values of the noise. Our result agrees with the formal expressions obtainable from replica calculations, and is the first result that fully justifies the replica formulas beyond the binary erasure channel (BEC). It also shows that the MAP and BP-GEXIT curves are equal in the high noise regime. The ensemble of LDPC codes considered here is constructed by adding randomly a sufficient fraction p of degree one variable nodes to a standard irregular LDPC(Lambda, P) Tanner graphs.

International Symposium on Turbo Codes and Iterative Information Processing•Sep 1, 2008•3 citations

Decay of Correlations for Sparse Graph Error Correcting Codes

Shrinivas Kudekar, Nicolas Macris

The subject of this paper is transmission over a general class of binary-input memoryless symmetric channels using error correcting codes based on sparse graphs, namely, low-density generator-matrix and low-density parity-check codes. The optimal (or ideal) decoder based on the posterior measure over the code-bits and its relationship to the suboptimal belief propagation decoder are investigated. We consider the correlation (or covariance) between two code-bits, averaged over the noise realizations, as a function of the graph distance for the optimal decoder. Our main result is that this correlation decays exponentially fast for given low-density generator-matrix codes and a high enough noise parameter and also for given low-density parity-check codes and a low enough noise parameter. This has many consequences. Appropriate performance curves—called generalized extrinsic information transfer (GEXIT) functions—of the belief propagation and optimal decoders match in high/low noise regimes. This means that in high/low noise regimes the performance curves of the optimal decoder can be computed by density evolution. Another interpretation is that the replica predictions of spin-glass theory are exact. Our methods are rather general and use cluster expansions first developed in the context of mathematical statistical mechanics.

arXiv:0903.1842•SIAM Journal on Discrete Mathematics•Jan 1, 2011•15 citations

LDPC Decoding Based On Statistical Mechanics Of Spin-Glasses: A Study

Z. JADDI, Abdessalam Aït Madi, Mohammed Benbrahim

In this paper, the LDPC (Low Density Parity Check Codes) decoding algorithm has been investigated using statistical mechanics properties. The main advantage of LDPC codes is their performance that works in near Shannon limit, and the ability to use iterative decoding algorithms. In 1989 N. Sourlas showed that error-correcting codes can be considered as Spin-Glass systems thus making it possible to model LDPC codes as an Ising model, opening the way for information theorists to solve coding problems with the power of statistical mechanics. According to N. Sourlas, the decoding problem can be solved by finding the ground state of the corresponding spin-system Hamiltonian. The main goal of this paper is to review as simple as possible the statistical properties of LDPC codes, and how it is used especially facing the decoding problem.

International Conference on Innovative Research in Applied Science, Engineering and Technology (IRASET)•Apr 1, 2020•0 citations
Abstract.: This contribution is based on the contents of a talk delivered at the Next-SigmaPhi conference held in Crete in August 2005. It is adressed to an audience of physicists with diverse horizons and does not assume any background in communications theory. Capacity approaching error correcting codes for channel communication known as Low Density Parity Check (LDPC) codes have attracted considerable attention from coding theorists in the last decade. Surprisingly strong connections with the theory of diluted spin glasses have been discovered. In this work we elucidate one new connection, namely that a class of correlation inequalities valid for Gaussian spin glasses can be applied to the theoretical analysis of LDPC codes. This allows for a rigorous comparison between the so called (optimal) maximum a posteriori and the computationaly efficient belief propagation decoders. The main ideas of the proofs are explained and we refer to recent works for the more lengthy technical details
It is shown that a correlation inequality of statistical mechanics can be applied to linear low-density parity-check codes. Thanks to this tool we prove that, under a natural assumption, the exponential growth rate of regular low-density parity-check (LDPC) codes, can be computed exactly by iterative methods, at least on the interval where it is a concave function of the relative weight of code words. Then, considering communication over a binary input additive white Gaussian noise channel with a Poisson LDPC code we prove that, under a natural assumption, part of the GEXIT curve (associated to MAP decoding) can also be computed exactly by the belief propagation algorithm. The correlation inequality yields a sharp lower bound on the GEXIT curve. We also make an extension of the interpolation techniques that have recently led to rigorous results in spin glass theory and in the SAT problem
It is shown that a correlation inequality of statistical mechanics can be applied to low-density parity-check codes. Thanks to this tool we prove that the growth rate of regular LDPC codes, can be exactly calculated by iterative methods, at least on the interval where it is a concave function of the relative weight of code words. We also consider communication over a binary input additive white Gaussian noise channel with a Poisson LDPC code and prove that (at least part of) the GEXIT curve (associated to MAP decoding) can also be computed exactly by the belief propagation decoder. In both problems, the correlation inequality yields sharp lower bounds. We also use a non trivial extension of the interpolation techniques that have recently led to rigorous results in spin glass theory and in the SAT problem
We consider communication over binary-input memoryless symmetric channels with low-density parity-check (LDPC) codes. The relationship between maximum a posteriori and belief propagation decoding is investigated using a set of correlation inequalities that first appeared in statistical mechanics of Gaussian spin glasses. We prove bounds on generalized extrinsic information transfer (EXIT) functions, that are believed to be tight, and discuss their relationship with the ones obtained by the interpolation method.
Recently the decay of correlations between bits of low density generator matrix (LDGM) codes have been investigated by using high temperature expansions from statistical physics. In this work we apply these ideas to a special class of low density parity check codes (LDPC) on the binary input Gaussian white noise channel (BIAWGNC). We give a rigorous derivation of the maximum a posteriori (MAP) Generalized EXIT curve (the derivative with respect to the noise parameter of the input-output conditional entropy) for high values of the noise. Our result agrees with the formal expressions obtainable from replica calculations, and is the first result that fully justifies the replica formulas beyond the binary erasure channel (BEC). It also shows that the MAP and BP-GEXIT curves are equal in the high noise regime. The ensemble of LDPC codes considered here is constructed by adding randomly a sufficient fraction p of degree one variable nodes to a standard irregular LDPC(Lambda, P) Tanner graphs.
The subject of this paper is transmission over a general class of binary-input memoryless symmetric channels using error correcting codes based on sparse graphs, namely, low-density generator-matrix and low-density parity-check codes. The optimal (or ideal) decoder based on the posterior measure over the code-bits and its relationship to the suboptimal belief propagation decoder are investigated. We consider the correlation (or covariance) between two code-bits, averaged over the noise realizations, as a function of the graph distance for the optimal decoder. Our main result is that this correlation decays exponentially fast for given low-density generator-matrix codes and a high enough noise parameter and also for given low-density parity-check codes and a low enough noise parameter. This has many consequences. Appropriate performance curves—called generalized extrinsic information transfer (GEXIT) functions—of the belief propagation and optimal decoders match in high/low noise regimes. This means that in high/low noise regimes the performance curves of the optimal decoder can be computed by density evolution. Another interpretation is that the replica predictions of spin-glass theory are exact. Our methods are rather general and use cluster expansions first developed in the context of mathematical statistical mechanics.
In this paper, the LDPC (Low Density Parity Check Codes) decoding algorithm has been investigated using statistical mechanics properties. The main advantage of LDPC codes is their performance that works in near Shannon limit, and the ability to use iterative decoding algorithms. In 1989 N. Sourlas showed that error-correcting codes can be considered as Spin-Glass systems thus making it possible to model LDPC codes as an Ising model, opening the way for information theorists to solve coding problems with the power of statistical mechanics. According to N. Sourlas, the decoding problem can be solved by finding the ground state of the corresponding spin-system Hamiltonian. The main goal of this paper is to review as simple as possible the statistical properties of LDPC codes, and how it is used especially facing the decoding problem.