Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

IEEE Transactions on Information Theory

On the Conditional Rényi Entropy

Serge Fehr, Stefan Berens

Published Sep 12, 2014• 136 citations
Publisher

Abstract

The Rényi entropy of general order unifies the well-known Shannon entropy with several other entropy notions, like the min-entropy or collision entropy. In contrast to the Shannon entropy, there seems to be no commonly accepted definition for the conditional Rényi entropy: several versions have been proposed and used in the literature. In this paper, we reconsider the definition for the conditional Rényi entropy of general order as proposed by Arimoto in the seventies. We show that this particular notion satisfies several natural properties. In particular, we show that it satisfies monotonicity under conditioning, meaning that conditioning can only reduce the entropy, and (a weak form of) chain rule, which implies that the decrease in entropy due to conditioning is bounded by the number of bits one conditions on. None of the other suggestions for the conditional Rényi entropy satisfies both these properties. Finally, we show a natural interpretation of the conditional Rényi entropy in terms of (unconditional) Rényi divergence, and we show consistency with a recently proposed notion of conditional Rényi entropy in the quantum setting.

DOI
10.1109/tit.2014.2357799

Related papers

Conditional Rényi Entropies

Andréia Teixeira, Armando B. Matos, Luís Filipe Antunes

There is no generally accepted definition of conditional Rényi entropy. The (unconditional) Rényi entropy depends on a parameter α, which for the case of min-entropy takes the value ∞. Even for this particular case, there are several proposals for the definition of conditional entropy. This paper describes three general definitions of conditional Rényi entropy that were found or suggested in the literature. Their properties are studied and their values, as a function of α, are compared. The particular case of min-entropy is widely used in cryptography as a security parameter; this case is studied in some detail.

IEEE Transactions on Information Theory•Apr 27, 2012•83 citations

A Tight Uniform Continuity Bound for the Arimoto-Rényi Conditional Entropy and its Extension to Classical-Quantum States

Michael G. Jabbour, Nilanjana Datta

We prove a tight uniform continuity bound for Arimoto’s version of the conditional$\alpha $-Rényi entropy for the range$\alpha \in [0, 1$). This definition of the conditional$\alpha $-Rényi entropy is the most natural one among the multiple forms which exist in the literature, since it satisfies two desirable properties of a conditional entropy, namely, the fact that conditioning reduces entropy, and that the associated reduction in uncertainty cannot exceed the information gained by conditioning. Furthermore, it has found interesting applications in various information theoretic tasks such as guessing with side information and sequential decoding. This conditional entropy reduces to the conditional Shannon entropy in the limit$\alpha \to 1$, and this in turn allows us to recover the recently obtained tight uniform continuity bound for the latter from our result. Finally, we apply our result to obtain a tight uniform continuity bound for the conditional$\alpha $-Rényi entropy of a classical-quantum state, for$\alpha $in the same range as above. This again yields the corresponding known bound for the conditional entropy of the state in the limit$\alpha \to 1$.

arXiv:2007.05049•IEEE Transactions on Information Theory•Jan 13, 2022•11 citations

On quantum Rényi entropies: A new generalization and some properties

Martin Müller-Lennert, Frédéric Dupuis, Oleg Szehr +2 more

The Rényi entropies constitute a family of information measures that generalizes the well-known Shannon entropy, inheriting many of its properties. They appear in the form of unconditional and conditional entropies, relative entropies, or mutual information, and have found many applications in information theory and beyond. Various generalizations of Rényi entropies to the quantum setting have been proposed, most prominently Petz's quasi-entropies and Renner's conditional min-, max-, and collision entropy. However, these quantum extensions are incompatible and thus unsatisfactory. We propose a new quantum generalization of the family of Rényi entropies that contains the von Neumann entropy, min-entropy, collision entropy, and the max-entropy as special cases, thus encompassing most quantum entropies in use today. We show several natural properties for this definition, including data-processing inequalities, a duality relation, and an entropic uncertainty relation.

arXiv:1306.3142•Journal of Mathematical Physics•Dec 1, 2013•724 citations

On quantum R\'enyi entropies: a new definition, some properties and several conjectures

Martin Müller-Lennert, Frédéric Dupuis, Oleg Szehr +2 more

The R\'enyi entropies constitute a family of information measures that generalizes the well-known Shannon entropy, inheriting many of its properties. They appear in the form of unconditional and conditional entropies, relative entropies or mutual information, and have found many applications in information theory and beyond. Various generalizations of R\'enyi entropies to the quantum setting have been proposed, most notably Petz's quasi-entropies and Renner's conditional min-, max- and collision entropy. Here, we argue that previous quantum extensions are incompatible and thus unsatisfactory. We propose a new quantum generalization of the family of R\'enyi entropies and the corresponding relative entropies that contains the min-entropy, collision entropy, von Neumann entropy as well as the max-entropy as special cases, thus encompassing most quantum entropies in use today. We show several natural properties for this definition, including data-processing inequalities and a limited duality relation. We conclude that the treatment of these entropies is technically challenging and requires sophisticated tools from linear algebra. We share several conjectured properties which we were unable to prove.

•Jun 13, 2013•7 citations

A complete characterisation of conditional entropies

Roberto Rubboli, Erkka Haapasalo, Marco Tomamichel

Entropies are fundamental measures of uncertainty with central importance in information theory and statistics and applications across all the quantitative sciences. Under a natural set of operational axioms, the most general form of entropy is captured by the family of Rényi entropies, parameterized by a real number $α$. Conditional entropy extends the notion of entropy by quantifying uncertainty from the viewpoint of an observer with access to potentially correlated side information. However, despite their significance and the emergence of various useful definitions, a complete characterisation of measures of conditional entropy that satisfy a natural set of operational axioms has remained elusive. In this work, we provide a complete characterisation of conditional entropy, defined through a set of axioms that are essential for any operationally meaningful definition: additivity for independent random variables, invariance under relabeling, and monotonicity under conditional mixing channels. We show that the most general conditional entropy is a generalized convex combination of conditional entropies defined as exponential averages of Rényi entropies of the conditioned distribution, parameterized by a real number and a probability measure on the positive reals.

arXiv:2601.23213•Jan 30, 2026•0 citations

A complete characterisation of conditional entropy

Roberto Rubboli, Erkka Haapasalo, Marco Tomamichel

Entropies are fundamental measures of uncertainty with central importance in information theory and statistics and applications across all the quantitative sciences. Under a natural set of operational axioms, the most general form of entropy is captured by the family of Rényi entropies, parameterized by a real number α. Conditional entropy extends the notion of entropy by quantifying uncertainty from the viewpoint of an observer with access to potentially correlated side information. However, despite their significance and the emergence of various useful definitions, a complete characterization of measures of conditional entropy that satisfy a natural set of operational axioms has remained elusive. In this work, we provide a complete characterization of conditional entropy, defined through a set of axioms that are essential for any operationally meaningful definition: additivity for independent random variables, invariance under relabeling, and monotonicity under bistochastic maps conditioned on the side information. We prove that the most general form of conditional entropy is captured by a family of conditional entropies that are averages of Rényi entropies of the conditioned distribution and parameterized by a probability measure on the positive reals.

•Feb 25, 2026•0 citations
The Rényi entropy of general order unifies the well-known Shannon entropy with several other entropy notions, like the min-entropy or collision entropy. In contrast to the Shannon entropy, there seems to be no commonly accepted definition for the conditional Rényi entropy: several versions have been proposed and used in the literature. In this paper, we reconsider the definition for the conditional Rényi entropy of general order as proposed by Arimoto in the seventies. We show that this particular notion satisfies several natural properties. In particular, we show that it satisfies monotonicity under conditioning, meaning that conditioning can only reduce the entropy, and (a weak form of) chain rule, which implies that the decrease in entropy due to conditioning is bounded by the number of bits one conditions on. None of the other suggestions for the conditional Rényi entropy satisfies both these properties. Finally, we show a natural interpretation of the conditional Rényi entropy in terms of (unconditional) Rényi divergence, and we show consistency with a recently proposed notion of conditional Rényi entropy in the quantum setting.
There is no generally accepted definition of conditional Rényi entropy. The (unconditional) Rényi entropy depends on a parameter α, which for the case of min-entropy takes the value ∞. Even for this particular case, there are several proposals for the definition of conditional entropy. This paper describes three general definitions of conditional Rényi entropy that were found or suggested in the literature. Their properties are studied and their values, as a function of α, are compared. The particular case of min-entropy is widely used in cryptography as a security parameter; this case is studied in some detail.
We prove a tight uniform continuity bound for Arimoto’s version of the conditional$\alpha −Reˊnyientropyfortherange-Rényi entropy for the range−Reˊnyientropyfortherange\alpha \in [0, 1).Thisdefinitionoftheconditional). This definition of the conditional).Thisdefinitionoftheconditional\alpha −Reˊnyientropyisthemostnaturaloneamongthemultipleformswhichexistintheliterature,sinceitsatisfiestwodesirablepropertiesofaconditionalentropy,namely,thefactthatconditioningreducesentropy,andthattheassociatedreductioninuncertaintycannotexceedtheinformationgainedbyconditioning.Furthermore,ithasfoundinterestingapplicationsinvariousinformationtheoretictaskssuchasguessingwithsideinformationandsequentialdecoding.ThisconditionalentropyreducestotheconditionalShannonentropyinthelimit-Rényi entropy is the most natural one among the multiple forms which exist in the literature, since it satisfies two desirable properties of a conditional entropy, namely, the fact that conditioning reduces entropy, and that the associated reduction in uncertainty cannot exceed the information gained by conditioning. Furthermore, it has found interesting applications in various information theoretic tasks such as guessing with side information and sequential decoding. This conditional entropy reduces to the conditional Shannon entropy in the limit−Reˊnyientropyisthemostnaturaloneamongthemultipleformswhichexistintheliterature,sinceitsatisfiestwodesirablepropertiesofaconditionalentropy,namely,thefactthatconditioningreducesentropy,andthattheassociatedreductioninuncertaintycannotexceedtheinformationgainedbyconditioning.Furthermore,ithasfoundinterestingapplicationsinvariousinformationtheoretictaskssuchasguessingwithsideinformationandsequentialdecoding.ThisconditionalentropyreducestotheconditionalShannonentropyinthelimit\alpha \to 1,andthisinturnallowsustorecovertherecentlyobtainedtightuniformcontinuityboundforthelatterfromourresult.Finally,weapplyourresulttoobtainatightuniformcontinuityboundfortheconditional, and this in turn allows us to recover the recently obtained tight uniform continuity bound for the latter from our result. Finally, we apply our result to obtain a tight uniform continuity bound for the conditional,andthisinturnallowsustorecovertherecentlyobtainedtightuniformcontinuityboundforthelatterfromourresult.Finally,weapplyourresulttoobtainatightuniformcontinuityboundfortheconditional\alpha −Reˊnyientropyofaclassical−quantumstate,for-Rényi entropy of a classical-quantum state, for−Reˊnyientropyofaclassical−quantumstate,for\alpha inthesamerangeasabove.Thisagainyieldsthecorrespondingknownboundfortheconditionalentropyofthestateinthelimitin the same range as above. This again yields the corresponding known bound for the conditional entropy of the state in the limitinthesamerangeasabove.Thisagainyieldsthecorrespondingknownboundfortheconditionalentropyofthestateinthelimit\alpha \to 1$.
The Rényi entropies constitute a family of information measures that generalizes the well-known Shannon entropy, inheriting many of its properties. They appear in the form of unconditional and conditional entropies, relative entropies, or mutual information, and have found many applications in information theory and beyond. Various generalizations of Rényi entropies to the quantum setting have been proposed, most prominently Petz's quasi-entropies and Renner's conditional min-, max-, and collision entropy. However, these quantum extensions are incompatible and thus unsatisfactory. We propose a new quantum generalization of the family of Rényi entropies that contains the von Neumann entropy, min-entropy, collision entropy, and the max-entropy as special cases, thus encompassing most quantum entropies in use today. We show several natural properties for this definition, including data-processing inequalities, a duality relation, and an entropic uncertainty relation.
The R\'enyi entropies constitute a family of information measures that generalizes the well-known Shannon entropy, inheriting many of its properties. They appear in the form of unconditional and conditional entropies, relative entropies or mutual information, and have found many applications in information theory and beyond. Various generalizations of R\'enyi entropies to the quantum setting have been proposed, most notably Petz's quasi-entropies and Renner's conditional min-, max- and collision entropy. Here, we argue that previous quantum extensions are incompatible and thus unsatisfactory. We propose a new quantum generalization of the family of R\'enyi entropies and the corresponding relative entropies that contains the min-entropy, collision entropy, von Neumann entropy as well as the max-entropy as special cases, thus encompassing most quantum entropies in use today. We show several natural properties for this definition, including data-processing inequalities and a limited duality relation. We conclude that the treatment of these entropies is technically challenging and requires sophisticated tools from linear algebra. We share several conjectured properties which we were unable to prove.
Entropies are fundamental measures of uncertainty with central importance in information theory and statistics and applications across all the quantitative sciences. Under a natural set of operational axioms, the most general form of entropy is captured by the family of Rényi entropies, parameterized by a real number ααα. Conditional entropy extends the notion of entropy by quantifying uncertainty from the viewpoint of an observer with access to potentially correlated side information. However, despite their significance and the emergence of various useful definitions, a complete characterisation of measures of conditional entropy that satisfy a natural set of operational axioms has remained elusive. In this work, we provide a complete characterisation of conditional entropy, defined through a set of axioms that are essential for any operationally meaningful definition: additivity for independent random variables, invariance under relabeling, and monotonicity under conditional mixing channels. We show that the most general conditional entropy is a generalized convex combination of conditional entropies defined as exponential averages of Rényi entropies of the conditioned distribution, parameterized by a real number and a probability measure on the positive reals.
Entropies are fundamental measures of uncertainty with central importance in information theory and statistics and applications across all the quantitative sciences. Under a natural set of operational axioms, the most general form of entropy is captured by the family of Rényi entropies, parameterized by a real number α. Conditional entropy extends the notion of entropy by quantifying uncertainty from the viewpoint of an observer with access to potentially correlated side information. However, despite their significance and the emergence of various useful definitions, a complete characterization of measures of conditional entropy that satisfy a natural set of operational axioms has remained elusive. In this work, we provide a complete characterization of conditional entropy, defined through a set of axioms that are essential for any operationally meaningful definition: additivity for independent random variables, invariance under relabeling, and monotonicity under bistochastic maps conditioned on the side information. We prove that the most general form of conditional entropy is captured by a family of conditional entropies that are averages of Rényi entropies of the conditioned distribution and parameterized by a probability measure on the positive reals.