Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2112.03582 · Entropy

On a 2-Relative Entropy

James Fullwood

Published Dec 31, 2021• 1 citation
arXivHTMLPDF

Abstract

We construct a 2-categorical extension of the relative entropy functor of Baez and Fritz, and show that our construction is functorial with respect to vertical morphisms. Moreover, we show such a ‘2-relative entropy’ satisfies natural 2-categorial analogues of convex linearity, vanishing under optimal hypotheses, and lower semicontinuity. While relative entropy is a relative measure of information between probability distributions, we view our construction as a relative measure of information between channels.

DOI
10.3390/e24010074

Related papers

A Bayesian characterization of relative entropy

John C. Baez, Tobias Fritz

We give a new characterization of relative entropy, also known as the Kullback-Leibler divergence.We use a number of interesting categories related to probability theory.In particular, we consider a category FinStat where an object is a finite set equipped with a probability distribution, while a morphism is a measure-preserving function f : X → Y together with a stochastic right inverse s : Y → X.The function f can be thought of as a measurement process, while s provides a hypothesis about the state of the measured system given the result of a measurement.Given this data we can define the entropy of the probability distribution on X relative to the 'prior' given by pushing the probability distribution on Y forwards along s.We say that s is 'optimal' if these distributions agree.We show that any convex linear, lower semicontinuous functor from FinStat to the additive monoid [0, ∞] which vanishes when s is optimal must be a scalar multiple of this relative entropy.Our proof is independent of all earlier characterizations, but inspired by the work of Petz.We thank Ryszard Kostecki and Rob Spekkens for discussions

arXiv:1402.3067•Theory and applications of categories•Jan 1, 2014•42 citations

A characterization of von Neumann entropy using functors

Kenji Nakahira

Baez, Fritz, and Leinster derived a method for characterizing Shannon entropy in classical systems. In this method, they considered a functor from a certain category to the monoid of non-negative real numbers with addition as a map from measure-preserving functions to non-negative real numbers, and derived Shannon entropy by imposing several simple conditions. We propose a method for characterizing von Neumann entropy by extending their results to quantum systems.

arXiv:2309.10353•Sep 19, 2023•0 citations

Categories of Brègman operations and epistemic (co)monads

Ryszard Paweł Kostecki

We construct a categorical framework for nonlinear postquantum inference, with embeddings of convex closed sets of suitable reflexive Banach spaces as objects and pullbacks of Brègman quasi-nonexpansive mappings (in particular, constrained maximisations of Brègman relative entropies) as morphisms. It provides a nonlinear convex analytic analogue of Chencov's programme of geometric study of categories of linear positive maps between spaces of states, a working model of Mielnik's nonlinear transmitters, and a setting for nonlinear resource theories (with monoids of Brègman quasi-nonexpansive maps as free operations, their asymptotic fixed point sets as free sets, and Brègman relative entropies as resource monotones). We construct a range of concrete examples for semi-finite JBW-algebras and any W*-algebras. Due to relative entropy's asymmetry, all constructions have left and right versions, with Legendre duality inducing categorical equivalence between their well-defined restrictions. Inner groupoids of these categories implement the notion of statistical equivalence. The hom-sets of a subcategory of morphisms given by entropic projections have the structure of partially ordered commutative monoids (so, they are resource theories in Fritz's sense). Further restriction of objects to affine sets turns Brègman relative entropy into a functor. Finally, following Lawvere's adjointness paradigm for deductive logic, but with a semantic twist representing Jaynes' and Chencov's views on statistical inference, we introduce a category-theoretic multi-(co)agent setting for inductive inference theories, implemented by families of monads and comonads. We show that the brègmanian approach provides some special cases of this setting.

arXiv:2103.07810•Mar 13, 2021•1 citation

Lower semicontinuity of relative entropy disturbance and its consequences

Максим Евгеньевич Широков

It is proved that the decrease of quantum relative entropy under the action of a quantum operation is a lower semicontinuous function of the pair of its arguments. This property implies, in particular, that the local discontinuity jumps of the quantum relative entropy do not increase under the action of quantum operations. It also implies the lower semicontinuity of the modulus of the joint convexity of quantum relative entropy (as a function of ensembles of quantum states). Various corollaries and applications of these results are considered. Bibliography: 42 titles.

arXiv:2302.04809•Sbornik Mathematics•Jan 1, 2024•0 citations

Generalized information structures and their cohomology

Juan Pablo Vigneaux

Baudot and Bennequin introduced a cohomological construction adapted to information theory, called \emph{information cohomology}; it characterizes Shannon entropy as the only nontrivial cohomology class in degree one, for suitable coefficients. This text develops the relation between information cohomology and topos theory. To this end, we redefine information structures, as categories of random variables related by a notion of extension or refinement; classical and quantum probability spaces appear as models (or representations) of them. These generalized information structures form a category with finite products and coproducts. Information cohomology is a derived functor in the abelian category of modules over an algebra of random variables, a subcategory of the presheaf topos associated with the information structure. The cohomology is invariant under isomorphisms of generalized structures. We prove that the relatively free bar construction gives a projective object for the computation of this invariant. We provide detailed computations of $H^0$ and $H^1$ when the coefficients are functionals of classical probabilities, including the degenerate cases. In the process, we also establish the homological nature of Tsallis entropies, as the only cocycles in degree one for certain one-parameter family of modules.

Theory and applications of categories•Sep 22, 2017•3 citations

Postquantum Br\`{e}gman relative entropies and nonlinear resource theories

Ryszard Paweł Kostecki

We introduce the family of postquantum Bregman relative entropies, based on nonlinear embeddings into reflexive Banach spaces (with examples given by reflexive noncommutative Orlicz spaces over semi-finite W*-algebras, nonassociative L$_p$ spaces over semi-finite JBW-algebras, and noncommutative L$_p$ spaces over arbitrary W*-algebras). This allows us to define a class of geometric categories for nonlinear postquantum inference theory (providing an extension of Chencov's approach to foundations of statistical inference), with constrained maximisations of Bregman relative entropies as morphisms and nonlinear images of closed convex sets as objects. Further generalisation to a framework for nonlinear convex operational theories is developed using a larger class of morphisms, determined by Bregman nonexpansive operations (which provide a well-behaved family of Mielnik's nonlinear transmitters). As an application, we derive a range of nonlinear postquantum resource theories determined in terms of this class of operations.

•Oct 5, 2017•2 citations
We construct a 2-categorical extension of the relative entropy functor of Baez and Fritz, and show that our construction is functorial with respect to vertical morphisms. Moreover, we show such a ‘2-relative entropy’ satisfies natural 2-categorial analogues of convex linearity, vanishing under optimal hypotheses, and lower semicontinuity. While relative entropy is a relative measure of information between probability distributions, we view our construction as a relative measure of information between channels.
We give a new characterization of relative entropy, also known as the Kullback-Leibler divergence.We use a number of interesting categories related to probability theory.In particular, we consider a category FinStat where an object is a finite set equipped with a probability distribution, while a morphism is a measure-preserving function f : X → Y together with a stochastic right inverse s : Y → X.The function f can be thought of as a measurement process, while s provides a hypothesis about the state of the measured system given the result of a measurement.Given this data we can define the entropy of the probability distribution on X relative to the 'prior' given by pushing the probability distribution on Y forwards along s.We say that s is 'optimal' if these distributions agree.We show that any convex linear, lower semicontinuous functor from FinStat to the additive monoid [0, ∞] which vanishes when s is optimal must be a scalar multiple of this relative entropy.Our proof is independent of all earlier characterizations, but inspired by the work of Petz.We thank Ryszard Kostecki and Rob Spekkens for discussions
We construct a categorical framework for nonlinear postquantum inference, with embeddings of convex closed sets of suitable reflexive Banach spaces as objects and pullbacks of Brègman quasi-nonexpansive mappings (in particular, constrained maximisations of Brègman relative entropies) as morphisms. It provides a nonlinear convex analytic analogue of Chencov's programme of geometric study of categories of linear positive maps between spaces of states, a working model of Mielnik's nonlinear transmitters, and a setting for nonlinear resource theories (with monoids of Brègman quasi-nonexpansive maps as free operations, their asymptotic fixed point sets as free sets, and Brègman relative entropies as resource monotones). We construct a range of concrete examples for semi-finite JBW-algebras and any W*-algebras. Due to relative entropy's asymmetry, all constructions have left and right versions, with Legendre duality inducing categorical equivalence between their well-defined restrictions. Inner groupoids of these categories implement the notion of statistical equivalence. The hom-sets of a subcategory of morphisms given by entropic projections have the structure of partially ordered commutative monoids (so, they are resource theories in Fritz's sense). Further restriction of objects to affine sets turns Brègman relative entropy into a functor. Finally, following Lawvere's adjointness paradigm for deductive logic, but with a semantic twist representing Jaynes' and Chencov's views on statistical inference, we introduce a category-theoretic multi-(co)agent setting for inductive inference theories, implemented by families of monads and comonads. We show that the brègmanian approach provides some special cases of this setting.
It is proved that the decrease of quantum relative entropy under the action of a quantum operation is a lower semicontinuous function of the pair of its arguments. This property implies, in particular, that the local discontinuity jumps of the quantum relative entropy do not increase under the action of quantum operations. It also implies the lower semicontinuity of the modulus of the joint convexity of quantum relative entropy (as a function of ensembles of quantum states). Various corollaries and applications of these results are considered. Bibliography: 42 titles.
Baudot and Bennequin introduced a cohomological construction adapted to information theory, called \emph{information cohomology}; it characterizes Shannon entropy as the only nontrivial cohomology class in degree one, for suitable coefficients. This text develops the relation between information cohomology and topos theory. To this end, we redefine information structures, as categories of random variables related by a notion of extension or refinement; classical and quantum probability spaces appear as models (or representations) of them. These generalized information structures form a category with finite products and coproducts. Information cohomology is a derived functor in the abelian category of modules over an algebra of random variables, a subcategory of the presheaf topos associated with the information structure. The cohomology is invariant under isomorphisms of generalized structures. We prove that the relatively free bar construction gives a projective object for the computation of this invariant. We provide detailed computations of H0H^0H0 and H1H^1H1 when the coefficients are functionals of classical probabilities, including the degenerate cases. In the process, we also establish the homological nature of Tsallis entropies, as the only cocycles in degree one for certain one-parameter family of modules.
We introduce the family of postquantum Bregman relative entropies, based on nonlinear embeddings into reflexive Banach spaces (with examples given by reflexive noncommutative Orlicz spaces over semi-finite W*-algebras, nonassociative Lp_pp​ spaces over semi-finite JBW-algebras, and noncommutative Lp_pp​ spaces over arbitrary W*-algebras). This allows us to define a class of geometric categories for nonlinear postquantum inference theory (providing an extension of Chencov's approach to foundations of statistical inference), with constrained maximisations of Bregman relative entropies as morphisms and nonlinear images of closed convex sets as objects. Further generalisation to a framework for nonlinear convex operational theories is developed using a larger class of morphisms, determined by Bregman nonexpansive operations (which provide a well-behaved family of Mielnik's nonlinear transmitters). As an application, we derive a range of nonlinear postquantum resource theories determined in terms of this class of operations.