Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Entropy

The Homological Nature of Entropy

Pierre Baudot, Daniel Bennequin

Published May 13, 2015• 70 citations
Publisher

Abstract

We propose that entropy is a universal co-homological class in a theory associated to a family of observable quantities and a family of probability distributions. Three cases are presented: (1) classical probabilities and random variables; (2) quantum probabilities and observable operators; (3) dynamic probabilities and observation trees. This gives rise to a new kind of topology for information processes, that accounts for the main information functions: entropy, mutual-informations at all orders, and Kullback–Leibler divergence and generalizes them in several ways. The article is divided into two parts, that can be read independently. In the first part, the introduction, we provide an overview of the results, some open questions, future results and lines of research, and discuss briefly the application to complex data. In the second part we give the complete definitions and proofs of the theorems A, C and E in the introduction, which show why entropy is the first homological invariant of a structure of information in four contexts: static classical or quantum probability, dynamics of classical or quantum strategies of observation of a finite system.

DOI
10.3390/e17053253

Related papers

Topological forms of information

Pierre Baudot, Daniel Bennequin

Abstract. We propose that entropy is a universal co-homological class in a theory associated to a family of observable quantities and a family of probability distributions. Three cases are presented: 1) classical probabilities and random variables; 2) quantum probabilities and observable operators; 3) dynamic probabilities and observation trees. This gives rise to a new kind of topology for information processes. We discuss briefly its application to complex data, in particular to the structures of information flows in biological systems. This short note summarizes results obtained during the last years by the authors. The proofs are not included, but the definitions and theorems are stated with precision.

AIP conference proceedings•Jan 1, 2015•6 citations

Generalized information structures and their cohomology

Juan Pablo Vigneaux

Baudot and Bennequin introduced a cohomological construction adapted to information theory, called \emph{information cohomology}; it characterizes Shannon entropy as the only nontrivial cohomology class in degree one, for suitable coefficients. This text develops the relation between information cohomology and topos theory. To this end, we redefine information structures, as categories of random variables related by a notion of extension or refinement; classical and quantum probability spaces appear as models (or representations) of them. These generalized information structures form a category with finite products and coproducts. Information cohomology is a derived functor in the abelian category of modules over an algebra of random variables, a subcategory of the presheaf topos associated with the information structure. The cohomology is invariant under isomorphisms of generalized structures. We prove that the relatively free bar construction gives a projective object for the computation of this invariant. We provide detailed computations of $H^0$ and $H^1$ when the coefficients are functionals of classical probabilities, including the degenerate cases. In the process, we also establish the homological nature of Tsallis entropies, as the only cocycles in degree one for certain one-parameter family of modules.

Theory and applications of categories•Sep 22, 2017•3 citations

The Okinawa Lectures on Entropy

Klaas Landsman

After a historical introduction, the most important classical and quantum entropies are introduced as constructions in classical and quantum probability theory. Classical entropies are studied from large deviation theory, including theorems of Sanov, Cramér, Gärtner-Ellis, and Varadhan, and are illustrated in some applications to both Boltzmannian and Gibbsian statistical physics. Quantum entropies superficially connect to classical entropies at the formula level, but more deeply do so via the crucial role of entropy in statistical hypothesis testing. The classical (relative) Kullback-Leibler entropy, its quantum counterpart introduced by Umegaki, as well as their deformations proposed by Renyi all fit naturally in this context. Quantum entropy faces the new problem of defining and computing the relative entropy of a pair of states on a subsystem, here formalized as a von Neumann algebra. This requires modular (aka Tomita-Takesaki) theory, which provides the framework for the relative quantum entropies introduced by Araki and Uhlmann (these encompass both the Kullback-Leibler and Umegaki entropies as special cases). To (re)define and compute these entropies in terms of density operators and traces, further constructions are needed, namely Haagerup's noncommutative L^p spaces. Von Neumann algebras also provide the setting for the Connes-Stormer-Narnhofer-Thirring entropy, which is a quantum version of the Kolmogorov-Sinai entropy in dynamical systems and ergodic theory, to which we also provide an introduction. This course was originally inspired by, and should be relevant to, black hole thermodynamics, although we discuss neither this application nor the second law. The course tries to be both mathematically rigorous and interesting to theoretical physicists. Prerequisites are undergraduate probability theory, functional analysis, and quantum theory.

arXiv:2608.14523•Aug 14, 2026•0 citations

Mutual Information: A way to quantify correlations

Marcelo Tisoc, J. Beltrán

Within the framework of Information Theory, the existence of correlations between two random variables means that we can obtain information about one of them, just by measuring or observing the other random variable. In certain cases, this kind of relationship allows obtaining information about a variable even when the other is separated by a very large distance, that is, the process of obtaining information is non-local, an example (if not the only) is the quantum entanglement. These features of correlations make it interesting and important to study, classify and quantify them. The correlations are classified into classical correlations and quantum correlations, in addition they are quantified through the mutual information. Here we will present a natural way to define classical mutual information and then we will generalize it to the quantum case. Furthermore, every term in the definitions of mutual information will be interpreted using the concepts of classical and quantum entropy.

Revista Brasileira de Ensino de Física•Jan 1, 2022•6 citations

Information structures and their cohomology

Juan Pablo Vigneaux

We introduce the category of information structures, whose objects are suitable diagrams of measurable sets that encode the possible outputs of a given family of observables and their mutual relationships of refinement; they serve as mathematical models of contextuality in classical and quantum settings.Each information structure can be regarded as a ringed site with trivial topology; the structure ring is generated by the observables themselves and its multiplication corresponds to joint measurement.We extend Baudot and Bennequin's definition of information cohomology to this setting, as a derived functor in the category of modules over the structure ring, and show explicitly that the bar construction gives a projective resolution in that category, recovering in this way the cochain complexes previously considered in the literature.Finally, we study the particular case of a one-parameter family of coefficients made of functions of probability distributions.The only 1-cocycles are Shannon entropy or Tsallis α-entropy, depending on the value of the parameter.

arXiv:1709.07807•Theory and applications of categories•Jan 1, 2020•3 citations

A complete characterisation of conditional entropy

Roberto Rubboli, Erkka Haapasalo, Marco Tomamichel

Entropies are fundamental measures of uncertainty with central importance in information theory and statistics and applications across all the quantitative sciences. Under a natural set of operational axioms, the most general form of entropy is captured by the family of Rényi entropies, parameterized by a real number α. Conditional entropy extends the notion of entropy by quantifying uncertainty from the viewpoint of an observer with access to potentially correlated side information. However, despite their significance and the emergence of various useful definitions, a complete characterization of measures of conditional entropy that satisfy a natural set of operational axioms has remained elusive. In this work, we provide a complete characterization of conditional entropy, defined through a set of axioms that are essential for any operationally meaningful definition: additivity for independent random variables, invariance under relabeling, and monotonicity under bistochastic maps conditioned on the side information. We prove that the most general form of conditional entropy is captured by a family of conditional entropies that are averages of Rényi entropies of the conditioned distribution and parameterized by a probability measure on the positive reals.

•Feb 25, 2026•0 citations
We propose that entropy is a universal co-homological class in a theory associated to a family of observable quantities and a family of probability distributions. Three cases are presented: (1) classical probabilities and random variables; (2) quantum probabilities and observable operators; (3) dynamic probabilities and observation trees. This gives rise to a new kind of topology for information processes, that accounts for the main information functions: entropy, mutual-informations at all orders, and Kullback–Leibler divergence and generalizes them in several ways. The article is divided into two parts, that can be read independently. In the first part, the introduction, we provide an overview of the results, some open questions, future results and lines of research, and discuss briefly the application to complex data. In the second part we give the complete definitions and proofs of the theorems A, C and E in the introduction, which show why entropy is the first homological invariant of a structure of information in four contexts: static classical or quantum probability, dynamics of classical or quantum strategies of observation of a finite system.
Abstract. We propose that entropy is a universal co-homological class in a theory associated to a family of observable quantities and a family of probability distributions. Three cases are presented: 1) classical probabilities and random variables; 2) quantum probabilities and observable operators; 3) dynamic probabilities and observation trees. This gives rise to a new kind of topology for information processes. We discuss briefly its application to complex data, in particular to the structures of information flows in biological systems. This short note summarizes results obtained during the last years by the authors. The proofs are not included, but the definitions and theorems are stated with precision.
Baudot and Bennequin introduced a cohomological construction adapted to information theory, called \emph{information cohomology}; it characterizes Shannon entropy as the only nontrivial cohomology class in degree one, for suitable coefficients. This text develops the relation between information cohomology and topos theory. To this end, we redefine information structures, as categories of random variables related by a notion of extension or refinement; classical and quantum probability spaces appear as models (or representations) of them. These generalized information structures form a category with finite products and coproducts. Information cohomology is a derived functor in the abelian category of modules over an algebra of random variables, a subcategory of the presheaf topos associated with the information structure. The cohomology is invariant under isomorphisms of generalized structures. We prove that the relatively free bar construction gives a projective object for the computation of this invariant. We provide detailed computations of H0H^0H0 and H1H^1H1 when the coefficients are functionals of classical probabilities, including the degenerate cases. In the process, we also establish the homological nature of Tsallis entropies, as the only cocycles in degree one for certain one-parameter family of modules.
After a historical introduction, the most important classical and quantum entropies are introduced as constructions in classical and quantum probability theory. Classical entropies are studied from large deviation theory, including theorems of Sanov, Cramér, Gärtner-Ellis, and Varadhan, and are illustrated in some applications to both Boltzmannian and Gibbsian statistical physics. Quantum entropies superficially connect to classical entropies at the formula level, but more deeply do so via the crucial role of entropy in statistical hypothesis testing. The classical (relative) Kullback-Leibler entropy, its quantum counterpart introduced by Umegaki, as well as their deformations proposed by Renyi all fit naturally in this context. Quantum entropy faces the new problem of defining and computing the relative entropy of a pair of states on a subsystem, here formalized as a von Neumann algebra. This requires modular (aka Tomita-Takesaki) theory, which provides the framework for the relative quantum entropies introduced by Araki and Uhlmann (these encompass both the Kullback-Leibler and Umegaki entropies as special cases). To (re)define and compute these entropies in terms of density operators and traces, further constructions are needed, namely Haagerup's noncommutative L^p spaces. Von Neumann algebras also provide the setting for the Connes-Stormer-Narnhofer-Thirring entropy, which is a quantum version of the Kolmogorov-Sinai entropy in dynamical systems and ergodic theory, to which we also provide an introduction. This course was originally inspired by, and should be relevant to, black hole thermodynamics, although we discuss neither this application nor the second law. The course tries to be both mathematically rigorous and interesting to theoretical physicists. Prerequisites are undergraduate probability theory, functional analysis, and quantum theory.
Within the framework of Information Theory, the existence of correlations between two random variables means that we can obtain information about one of them, just by measuring or observing the other random variable. In certain cases, this kind of relationship allows obtaining information about a variable even when the other is separated by a very large distance, that is, the process of obtaining information is non-local, an example (if not the only) is the quantum entanglement. These features of correlations make it interesting and important to study, classify and quantify them. The correlations are classified into classical correlations and quantum correlations, in addition they are quantified through the mutual information. Here we will present a natural way to define classical mutual information and then we will generalize it to the quantum case. Furthermore, every term in the definitions of mutual information will be interpreted using the concepts of classical and quantum entropy.
We introduce the category of information structures, whose objects are suitable diagrams of measurable sets that encode the possible outputs of a given family of observables and their mutual relationships of refinement; they serve as mathematical models of contextuality in classical and quantum settings.Each information structure can be regarded as a ringed site with trivial topology; the structure ring is generated by the observables themselves and its multiplication corresponds to joint measurement.We extend Baudot and Bennequin's definition of information cohomology to this setting, as a derived functor in the category of modules over the structure ring, and show explicitly that the bar construction gives a projective resolution in that category, recovering in this way the cochain complexes previously considered in the literature.Finally, we study the particular case of a one-parameter family of coefficients made of functions of probability distributions.The only 1-cocycles are Shannon entropy or Tsallis α-entropy, depending on the value of the parameter.
Entropies are fundamental measures of uncertainty with central importance in information theory and statistics and applications across all the quantitative sciences. Under a natural set of operational axioms, the most general form of entropy is captured by the family of Rényi entropies, parameterized by a real number α. Conditional entropy extends the notion of entropy by quantifying uncertainty from the viewpoint of an observer with access to potentially correlated side information. However, despite their significance and the emergence of various useful definitions, a complete characterization of measures of conditional entropy that satisfy a natural set of operational axioms has remained elusive. In this work, we provide a complete characterization of conditional entropy, defined through a set of axioms that are essential for any operationally meaningful definition: additivity for independent random variables, invariance under relabeling, and monotonicity under bistochastic maps conditioned on the side information. We prove that the most general form of conditional entropy is captured by a family of conditional entropies that are averages of Rényi entropies of the conditioned distribution and parameterized by a probability measure on the positive reals.