. Conditional entropy extends the notion of entropy by quantifying uncertainty from the viewpoint of an observer with access to potentially correlated side information. However, despite their significance and the emergence of various useful definitions, a complete characterisation of measures of conditional entropy that satisfy a natural set of operational axioms has remained elusive. In this work, we provide a complete characterisation of conditional entropy, defined through a set of axioms that are essential for any operationally meaningful definition: additivity for independent random variables, invariance under relabeling, and monotonicity under conditional mixing channels. We show that the most general conditional entropy is a generalized convex combination of conditional entropies defined as exponential averages of Rényi entropies of the conditioned distribution, parameterized by a real number and a probability measure on the positive reals.

DOI: 10.48550/arxiv.2601.23213

Related papers

A complete characterisation of conditional entropy

Roberto Rubboli, Erkka Haapasalo, Marco Tomamichel

Entropies are fundamental measures of uncertainty with central importance in information theory and statistics and applications across all the quantitative sciences. Under a natural set of operational axioms, the most general form of entropy is captured by the family of Rényi entropies, parameterized by a real number α. Conditional entropy extends the notion of entropy by quantifying uncertainty from the viewpoint of an observer with access to potentially correlated side information. However, despite their significance and the emergence of various useful definitions, a complete characterization of measures of conditional entropy that satisfy a natural set of operational axioms has remained elusive. In this work, we provide a complete characterization of conditional entropy, defined through a set of axioms that are essential for any operationally meaningful definition: additivity for independent random variables, invariance under relabeling, and monotonicity under bistochastic maps conditioned on the side information. We prove that the most general form of conditional entropy is captured by a family of conditional entropies that are averages of Rényi entropies of the conditioned distribution and parameterized by a probability measure on the positive reals.Entropies are fundamental measures of uncertainty with central importance in information theory and statistics and applications across all the quantitative sciences. Under a natural set of operational axioms, the most general form of entropy is captured by the family of Rényi entropies, parameterized by a real number α. Conditional entropy extends the notion of entropy by quantifying uncertainty from the viewpoint of an observer with access to potentially correlated side information. However, despite their significance and the emergence of various useful definitions, a complete characterization of measures of conditional entropy that satisfy a natural set of operational axioms has remained elusive. In this work, we provide a complete characterization of conditional entropy, defined through a set of axioms that are essential for any operationally meaningful definition: additivity for independent random variables, invariance under relabeling, and monotonicity under bistochastic maps conditioned on the side information. We prove that the most general form of conditional entropy is captured by a family of conditional entropies that are averages of Rényi entropies of the conditioned distribution and parameterized by a probability measure on the positive reals.

Feb 25, 20260 citations

Conditional Entropy

Gilad Gour

Chapter 7 discusses quantum conditional entropy, extending the concept of conditional majorization and introducing the notion of negative quantum conditional entropy. The chapter starts with the basic definition of conditional entropy, exploring its key properties like monotonicity and additivity. It further delves into the concepts of conditional min- and max-entropies, emphasizing their roles in quantifying uncertainty in quantum states and their operational significance in quantum information theory. The text presents conditional entropy as a measure sensitive to the effects of entanglement, showing that negative conditional entropy is a distinctive feature of quantum systems, contrasting with the classical domain where entropy values are nonnegative. This negativity is particularly pronounced in the context of maximally entangled states and is connected to the fundamental differences between classical and quantum information processing. Moreover, the chapter includes theorems and exercises to solidify understanding, like the invariance of conditional entropy under local isometric channels and its reduction to entropy for product states. It concludes by underscoring the inevitability of negative conditional entropy in quantum systems, a topic of both theoretical and practical importance in the quantum domain.Chapter 7 discusses quantum conditional entropy, extending the concept of conditional majorization and introducing the notion of negative quantum conditional entropy. The chapter starts with the basic definition of conditional entropy, exploring its key properties like monotonicity and additivity. It further delves into the concepts of conditional min- and max-entropies, emphasizing their roles in quantifying uncertainty in quantum states and their operational significance in quantum information theory. The text presents conditional entropy as a measure sensitive to the effects of entanglement, showing that negative conditional entropy is a distinctive feature of quantum systems, contrasting with the classical domain where entropy values are nonnegative. This negativity is particularly pronounced in the context of maximally entangled states and is connected to the fundamental differences between classical and quantum information processing. Moreover, the chapter includes theorems and exercises to solidify understanding, like the invariance of conditional entropy under local isometric channels and its reduction to entropy for product states. It concludes by underscoring the inevitability of negative conditional entropy in quantum systems, a topic of both theoretical and practical importance in the quantum domain.

Cambridge University Press eBooksApr 10, 20250 citations

Quantum Entropy and Correlations in Quantum Information

Reinhold A. Bertlmann, Nicolai Friis

Abstract Chapter 20 provides a discussion of entropies and entropic quantities in quantum information theory, briefly considering the generalizations of the Rényi entropy to the quantum regime, but mainly focusing on the von Neumann entropy and its properties such as concavity, the Araki-Lieb inequality, and subadditivity. We introduce the quantum relative entropy, quantum joint entropy, quantum conditional entropy, and quantum mutual information, and prove key properties and results for these quantities, including Klein’s inequality, joint convexity, additivity, and monotonicity under completely positive and trace-preserving maps of the relative entropy, as well as weak monotonicity and strong subadditivity of the von Neumann entropy. We then analyse the relation of (negative) conditional entropy and entanglement. In this context we discuss the conditional amplitude operator and the mutual amplitude operator, as well as conditional Rényi entropies and their role for entanglement detection.Abstract Chapter 20 provides a discussion of entropies and entropic quantities in quantum information theory, briefly considering the generalizations of the Rényi entropy to the quantum regime, but mainly focusing on the von Neumann entropy and its properties such as concavity, the Araki-Lieb inequality, and subadditivity. We introduce the quantum relative entropy, quantum joint entropy, quantum conditional entropy, and quantum mutual information, and prove key properties and results for these quantities, including Klein’s inequality, joint convexity, additivity, and monotonicity under completely positive and trace-preserving maps of the relative entropy, as well as weak monotonicity and strong subadditivity of the von Neumann entropy. We then analyse the relation of (negative) conditional entropy and entanglement. In this context we discuss the conditional amplitude operator and the mutual amplitude operator, as well as conditional Rényi entropies and their role for entanglement detection.

Sep 7, 20230 citations

Two-Parameter Rényi Information Quantities with Applications to Privacy Amplification and Soft Covering

Li, Shi-Bing, Ke Li, Yu, Lei

There are no universally accepted definitions of Rényi conditional entropy and Rényi mutual information, although motivated by different applications, several definitions have been proposed in the literature. In this paper, we consider a family of two-parameter Rényi conditional entropy and a family of two-parameter Rényi mutual information. By performing a change of variables for the parameters, the two-parameter Rényi conditional entropy we study coincides precisely with the definition introduced by Hayashi and Tan [IEEE Trans. Inf. Theory, 2016], and it also emerges naturally as the classical specialization of the three-parameter quantum Rényi conditional entropy recently put forward by Rubboli, Goodarzi, and Tomamichel [arXiv:2410.21976 (2024)]. We establish several fundamental properties of the two-parameter Rényi conditional entropy, including monotonicity with respect to the parameters and variational expression. The associated two-parameter Rényi mutual information considered in this paper is new and it unifies three commonly used variants of Rényi mutual information. For this quantity, we prove several important properties, including the non-negativity, additivity, data processing inequality, monotonicity with respect to the parameters, variational expression, as well as convexity and concavity. Finally, we demonstrate that these two-parameter Rényi information quantities can be used to characterize the strong converse exponents in privacy amplification and soft covering problems under Rényi divergence of order $α\in (0, \infty)$ .There are no universally accepted definitions of Rényi conditional entropy and Rényi mutual information, although motivated by different applications, several definitions have been proposed in the literature. In this paper, we consider a family of two-parameter Rényi conditional entropy and a family of two-parameter Rényi mutual information. By performing a change of variables for the parameters, the two-parameter Rényi conditional entropy we study coincides precisely with the definition introduced by Hayashi and Tan [IEEE Trans. Inf. Theory, 2016], and it also emerges naturally as the classical specialization of the three-parameter quantum Rényi conditional entropy recently put forward by Rubboli, Goodarzi, and Tomamichel [arXiv:2410.21976 (2024)]. We establish several fundamental properties of the two-parameter Rényi conditional entropy, including monotonicity with respect to the parameters and variational expression. The associated two-parameter Rényi mutual information considered in this paper is new and it unifies three commonly used variants of Rényi mutual information. For this quantity, we prove several important properties, including the non-negativity, additivity, data processing inequality, monotonicity with respect to the parameters, variational expression, as well as convexity and concavity. Finally, we demonstrate that these two-parameter Rényi information quantities can be used to characterize the strong converse exponents in privacy amplification and soft covering problems under Rényi divergence of order $α\in (0, \infty)$.

arXiv:2511.02297Nov 4, 20250 citations

Conditional entropy and information of quantum processes

Siddhartha Das, Kaumudibikash Goswami, Vivek Pandey

What would be a reasonable definition of the conditional entropy of bipartite quantum processes, and what novel insight would it provide? We develop this notion using four information-theoretic axioms and define the corresponding quantitative formulas. Our definitions of the conditional entropies of channels are based on the generalized state and channel divergences, for instance, quantum relative entropy. We find that the conditional entropy of quantum channels has potential to reveal insights for quantum processes that aren't already captured by the existing entropic functions, entropy or conditional entropy, of the states and channels. The von Neumann conditional entropy $S[A|B]_{\mathcal{N}}$ of the channel $\mathcal{N}_{A'B'\to AB}$ is based on the quantum relative entropy, with system pairs $A',A$ and $B',B$ being nonconditioning and conditioning systems, respectively. We identify a connection between the underlying causal structure of a bipartite channel and its conditional entropy. In particular, we provide a necessary and sufficient condition for a bipartite quantum channel $\mathcal{N}_{A'B'\to AB}$ in terms of its von Neumann conditional entropy $S[A|B]_{\mathcal{N}}$ , to have no causal influence from $A'$ to $B$ . As a consequence, if $S[A|B]_{\mathcal{N}}< -\log|A|$ then the channel necessarily has causal influence (signaling) from $A'$ to $B$ . Our definition of the conditional entropy establishes the strong subadditivity of the entropy for quantum channels. We also study the total amount of correlations possible due to quantum processes by defining the multipartite mutual information of quantum channels.What would be a reasonable definition of the conditional entropy of bipartite quantum processes, and what novel insight would it provide? We develop this notion using four information-theoretic axioms and define the corresponding quantitative formulas. Our definitions of the conditional entropies of channels are based on the generalized state and channel divergences, for instance, quantum relative entropy. We find that the conditional entropy of quantum channels has potential to reveal insights for quantum processes that aren't already captured by the existing entropic functions, entropy or conditional entropy, of the states and channels. The von Neumann conditional entropy $S[A|B]_{\mathcal{N}}$ of the channel $\mathcal{N}_{A'B'\to AB}$ is based on the quantum relative entropy, with system pairs $A',A$ and $B',B$ being nonconditioning and conditioning systems, respectively. We identify a connection between the underlying causal structure of a bipartite channel and its conditional entropy. In particular, we provide a necessary and sufficient condition for a bipartite quantum channel $\mathcal{N}_{A'B'\to AB}$ in terms of its von Neumann conditional entropy $S[A|B]_{\mathcal{N}}$, to have no causal influence from $A'$ to $B$. As a consequence, if $S[A|B]_{\mathcal{N}}< -\log|A|$ then the channel necessarily has causal influence (signaling) from $A'$ to $B$. Our definition of the conditional entropy establishes the strong subadditivity of the entropy for quantum channels. We also study the total amount of correlations possible due to quantum processes by defining the multipartite mutual information of quantum channels.

arXiv:2410.01740Oct 2, 20240 citations

What is Entropy? A new perspective from games of chance

Sarah Brandsen, Isabelle Jianing Geng, Gilad Gour

Given entropy's central role in multiple areas of physics and science, one important task is to develop a systematic and unifying approach to defining entropy. Games of chance become a natural candidate for characterising the uncertainty of a physical system, as a system's performance in gambling games depends solely on the uncertainty of its output. In this work, we construct families of games which induce pre-orders corresponding to majorization, conditional majorization, and channel majorization. Finally, we provide operational interpretations for all pre-orders, show the relevance of these results to dynamical resource theories, and find the only asymptotically continuous classical dynamic entropy.Given entropy's central role in multiple areas of physics and science, one important task is to develop a systematic and unifying approach to defining entropy. Games of chance become a natural candidate for characterising the uncertainty of a physical system, as a system's performance in gambling games depends solely on the uncertainty of its output. In this work, we construct families of games which induce pre-orders corresponding to majorization, conditional majorization, and channel majorization. Finally, we provide operational interpretations for all pre-orders, show the relevance of these results to dynamical resource theories, and find the only asymptotically continuous classical dynamic entropy.

arXiv:2103.08681Mar 15, 20211 citation

α

α\in (0, \infty)

S[A|B]_{\mathcal{N}}

of the channel

\mathcal{N}_{A'B'\to AB}

is based on the quantum relative entropy, with system pairs

A',A

and

B',B

being nonconditioning and conditioning systems, respectively. We identify a connection between the underlying causal structure of a bipartite channel and its conditional entropy. In particular, we provide a necessary and sufficient condition for a bipartite quantum channel

\mathcal{N}_{A'B'\to AB}

in terms of its von Neumann conditional entropy

S[A|B]_{\mathcal{N}}

, to have no causal influence from

A'

B

. As a consequence, if

S[A|B]_{\mathcal{N}}< -\log|A|

then the channel necessarily has causal influence (signaling) from

A'

B

. Our definition of the conditional entropy establishes the strong subadditivity of the entropy for quantum channels. We also study the total amount of correlations possible due to quantum processes by defining the multipartite mutual information of quantum channels.