This thesis addresses the problem of inference in factor graphs, especially the LDPC codes, almost solved by message-passing algorithms. In particular, the Belief Propagation algorithm (BP) is investigated as a particular message-passing algorithm whose suboptimality is discussed in the case where the factor graph has a loop-like topology. From the equivalence between the BP and the Bethe approximation in statistical physics that is generalized to the region-based approximation, is detailed the Generalized Belief Propagation algorithm (GBP), a message-passing algorithm between clusters of the factor graph. It is experimentally shown to surpass the BP in the cases where the clustering deals with the harmful topological structures that prevents the BP from rightly decoding any LDPC code, namely the trapping sets. We do not only confront the BP and the GBP algorithms according to their performance from the point of view of the channel coding with the error-rate, but also according to their dynamical behaviors for non-trivial error-events for which both algorithms can exhibit chaotic beahviors. By means of classical and original dynamical quantifiers, it is shown that the GBP algorithm can overcome the BP algorithm.
This thesis addresses the problem of inference in factor graphs, especially the LDPC codes, almost solved by message-passing algorithms. In particular, the Belief Propagation algorithm (BP) is investigated as a particular message-passing algorithm whose suboptimality is discussed in the case where the factor graph has a loop-like topology. From the equivalence between the BP and the Bethe approximation in statistical physics that is generalized to the region-based approximation, is detailed the Generalized Belief Propagation algorithm (GBP), a message-passing algorithm between clusters of the factor graph. It is experimentally shown to surpass the BP in the cases where the clustering deals with the harmful topological structures that prevents the BP from rightly decoding any LDPC code, namely the trapping sets. We do not only confront the BP and the GBP algorithms according to their performance from the point of view of the channel coding with the error-rate, but also according to their dynamical behaviors for non-trivial error-events for which both algorithms can exhibit chaotic beahviors. By means of classical and original dynamical quantifiers, it is shown that the GBP algorithm can overcome the BP algorithm.
Approximation basée régions pour résoudre l'inférence dans les graphes factoriels à boucles : application au décodage des codes LDPC par le Generalized Belief Propagation Dans cette thèse, nous étudions le problème de l'inférence bayésienne dans les graphes factoriels, en particulier les codes LDPC, quasiment résolus par les algorithmes de message-passing. Nous réalisons en particulier une étude approfondie du Belief Propagation (BP) dont la question de la sous-optimalité est soulevée dans le cas où le graphe factoriel présente des boucles. A partir de l'équivalence entre le BP et l'approximation de Bethe en physique statistique qui se généralise à l'approximation basée régions, nous détaillons le Generalized Belief Propagation (GBP), un algorithme de message-passing entre des clusters du graphe factoriel. Nous montrons par des expériences que le GBP surpasse le BP dans les cas où le clustering est réalisé selon les structures topologiques néfastes qui empêchent le BP de bien décoder, à savoir les trapping sets. Au-delà de l'étude des performances en termes de taux d'erreur, nous confrontons les deux algorithmes par rapport à leurs dynamiques face à des événements d'erreur non triviaux, en particulier lorsqu'ils présentent des comportements chaotiques. Par des estimateurs classiques et originaux, nous montrons que l'algorithme du GBP peut dominer l'algorithme du BP.
Numerous inference problems in statistical physics, computer vision or error-correcting coding theory consist in approximating the marginal probability distributions on Markov Random Fields (MRF). The Belief Propagation (BP) is an accurate solution that is optimal if the MRF is loop free and suboptimal otherwise. In the context of error-correcting coding theory, any Low-Density Parity-Check (LDPC) code has a graphical representation, the Tanner graph, which is a particular MRF. It is used as a media for the BP algorithm to correct the bits, damaged by a noisy channel, by estimating their probability distributions. Though loops and combination thereof in the Tanner graph prevent the BP from being optimal, especially harmful topological structures called the trapping-sets. The BP has been extended to the Generalized Belief Propagation (GBP). This message-passing algorithm runs on a non unique mapping of the Tanner graph, namely the regiongraph, such that its nodes are gatherings of the Tanner graph nodes. Then it appears the possibility to decrease the loops effect, making the GBP more accurate than the BP. In this article, we expose a novel region graph construction suited to the Tanner code, an LDPC code whose Tanner graph is entirely covered by trapping-sets. Furthermore, we investigate the dynamic behavior of the GBP compared with that of the BP to understand its evolution in terms of the Signal-to-Noise Ratio (SNR). To this end we make use of classical estimators and we introduce a new one called the hyperspheres method.
Generalized belief propagation (GBP) is known to be a well-suited technique for approximate inference problems in loopy factor graphs. It can absorb problematic subgraphs inside regions to reduce their influence on the inference. However, the choice of regions to be used in GBP remains a delicate issue. This paper proposes an approach to create specific regions when dealing with Low-Density Parity-Check (LDPC) codes. We split trapping sets, known to degrade the decoding performance, to make GBP locally optimal. Experiments show that GBP can then perform better than BP, especially in the error-floor region.
In this paper, we focus on the Generalized Belief Propagation (GBP) algorithm to solve trapping sets in Low-Density Parity-Check (LDPC) codes. Trapping sets are topological structures in Tanner graphs of LDPC codes that are not correctly decoded by Belief Propagation (BP), leading to exhibit an error-floor in the Bit-Error Rate (BER). Stemming from statistical physics of spin glasses, GBP consists in passing messages between clusters of Tanner graph nodes in another graph called the region-graph. Here, we introduce a specific clustering of nodes, based on a novel local loopfree principle, that breaks trapping sets such that the resulting region-graph is locally loopfree. We then construct a hybrid decoder made of BP and GBP that proves to be a powerful decoder as it clearly improves the BER and defeats the error-floor.
Abstract—In this paper, we focus on the Generalized Belief Propagation (GBP) algorithm to solve trapping sets in Low-Density Parity-Check (LDPC) codes. Trapping sets are topo-logical structures in Tanner graphs of LDPC codes that are not correctly decoded by Belief Propagation (BP), leading to exhibit an error-floor in the Bit-Error Rate (BER). Stemming from statistical physics of spin glasses, GBP consists in passing messages between clusters of Tanner graph nodes in another graph called the region-graph. Here, we introduce a specific clustering of nodes, based on a novel local loopfree principle, that breaks trapping sets such that the resulting region-graph is locally loopfree. We then construct a hybrid decoder made of BP and GBP that proves to be a powerful decoder as it clearly improves the BER and defeats the error-floor. Index Terms—LDPC codes, Generalized Belief Propagation, trapping sets, error-floor, local clustering
The Belief Propagation (BP) is an inference algorithm used to estimate marginal probability distributions for any Markov Random Field (MRF). In the realm of Low-Density Parity-Check (LDPC) codes that can be represented by MRF called Tanner graphs, the BP is used as a decoding algorithm to estimate the states of bits sent through a noisy channel. Known to be optimal when the Tanner graph is a tree, the BP suffers from suboptimality when the Tanner graph has a loop-like topology. Furthermore, combinations of loops, namely the trapping sets, are particularly harmful for the decoding. To circumvent this problem were proposed other algorithms, like the Generalized Belief Propagation (GBP) that comes from statistical physics. This algorithm allows to absorb topological structures inside new nodes called regions. An advantage is that the resulting graph, the region graph, is not unique then according to its construction this region graph is a media for the GBP that can provide more accurate estimates than the BP. In this paper, we propose novel constructions of the region graph for the famous Tanner code of length N = 155 by making use of the trapping sets as basis for the regions.