The chain rule for the Shannon and von Neumann entropy, which relates the total entropy of a system to the entropies of its parts, is of central importance to information theory. Here, we consider the chain rule for the more general smooth min- and max-entropies, used in one-shot information theory. For these entropy measures, the chain rule no longer holds as an equality. However, the standard chain rule for the von Neumann entropy is retrieved asymptotically when evaluating the smooth entropies for many identical and independently distributed states.
The chain rule for the Shannon and von Neumann entropy, which relates the total entropy of a system to the entropies of its parts, is of central importance to information theory. Here, we consider the chain rule for the more general smooth min- and max-entropies, used in one-shot information theory. For these entropy measures, the chain rule no longer holds as an equality. However, the standard chain rule for the von Neumann entropy is retrieved asymptotically when evaluating the smooth entropies for many identical and independently distributed states.
We study formal properties of smooth max-information, a generalization of von Neumann mutual information derived from the max-relative entropy. Recent work suggests that it is a useful quantity in one-shot channel coding, quantum rate distortion theory, and the physics of quantum many-body systems. Max-information can be defined in multiple ways. We demonstrate that different smoothed definitions are essentially equivalent (up to logarithmic terms in the smoothing parameters). These equivalence relations allow us to derive new chain rules for the max-information in terms of min- and max-entropies, thus extending the smooth entropy formalism to mutual information.
In classical and quantum information theory, operational quantities such as the amount of randomness that can be extracted from a given source or the amount of space needed to store given data are normally characterized by one of two entropy measures, called smooth min-entropy and smooth max-entropy, respectively. While both entropies are equal to the von Neumann entropy in certain special cases (e.g., asymptotically, for many independent repetitions of the given data), their values can differ arbitrarily in the general case. In this paper, a recently discovered duality relation between (nonsmooth) min- and max-entropies is extended to the smooth case. More precisely, it is shown that the smooth min-entropy of a systemAconditioned on a systemBequals the negative of the smooth max-entropy ofAconditioned on a purifying systemC. This result immediately implies that certain operational quantities (such as the amount of compression and the amount of randomness that can be extracted from given data) are related. We explain how such relations have applications in cryptographic security proofs.
Many of the traditional results in information theory, such as the channel coding theorem or the source coding theorem, are restricted to scenarios where the underlying resources are independent and identically distributed (i.i.d.) over a large number of uses. To overcome this limitation, two different techniques, the information spectrum method and the smooth entropy framework, have been developed independently. They are based on new entropy measures, called spectral entropy rates andsmoothentropies, respectively, that generalize Shannon entropy (in the classical case) and von Neumann entropy (in the more general quantum case). Here, we show that the two techniques are closely related. More precisely, the spectral entropy rate can be seen as the asymptotic limit of the smooth entropy. Our results apply to the quantum setting and thus include the classical setting as a special case.
This thesis consolidates, improves and extends the smooth entropy framework for non-asymptotic information theory and cryptography. We investigate the conditional min- and max-entropy for quantum states, generalizations of classical Rényi entropies. We introduce the purified distance, a novel metric for unnormalized quantum states and use it to define smooth entropies as optimizations of the min- and max-entropies over a ball of close states. We explore various properties of these entropies, including data-processing inequalities, chain rules and their classical limits. The most important property is an entropic formulation of the asymptotic equipartition property, which implies that the smooth entropies converge to the von Neumann entropy in the limit of many independent copies. The smooth entropies also satisfy duality and entropic uncertainty relations that provide limits on the power of two different observers to predict the outcome of a measurement on a quantum system. Finally, we discuss three example applications of the smooth entropy framework. We show a strong converse statement for source coding with quantum side information, characterize randomness extraction against quantum side information and prove information theoretic security of quantum key distribution using an intuitive argument based on the entropic uncertainty relation.
Two new relative entropy quantities, called the min- and max-relative entropies, are introduced and their properties are investigated. The well-known min- and max-entropies, introduced by Renner, are obtained from these. We define a new entanglement monotone, which we refer to as the max-relative entropy of entanglement, and which is an upper bound to the relative entropy of entanglement. We also generalize the min- and max-relative entropies to obtain smooth min-and max-relative entropies. These act as parent quantities for the smooth Renyi entropies (ETH Zurich, Ph.D. dissertation, 2005), and allow us to define the analogues of the mutual information, in the smooth Renyi entropy framework. Further, the spectral divergence rates of the information spectrum approach are shown to be obtained from the smooth min- and max-relative entropies in the asymptotic limit.
Thermodynamic entropy, as defined by Clausius, characterizes macroscopic observations of a system based on phenomenological quantities such as temperature and heat. In contrast, information-theoretic entropy, introduced by Shannon, is a measure of uncertainty. In this Letter, we connect these two notions of entropy, using an axiomatic framework for thermodynamics [E. H. Lieb and J. Yngvason Proc. R. Soc. 469, 20130408 (2013)]. In particular, we obtain a direct relation between the Clausius entropy and the Shannon entropy, or its generalization to quantum systems, the von Neumann entropy. More generally, we find that entropy measures relevant in nonequilibrium thermodynamics correspond to entropies used in one-shot information theory.