Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:0801.0282 · IEEE Transactions on Information Theory

Smooth Entropies and the Quantum Information Spectrum

Nilanjana Datta, Renato Renner

Published May 19, 2009• 85 citations
arXivHTMLPDF

Abstract

Many of the traditional results in information theory, such as the channel coding theorem or the source coding theorem, are restricted to scenarios where the underlying resources are independent and identically distributed (i.i.d.) over a large number of uses. To overcome this limitation, two different techniques, the information spectrum method and the smooth entropy framework, have been developed independently. They are based on new entropy measures, called spectral entropy rates andsmoothentropies, respectively, that generalize Shannon entropy (in the classical case) and von Neumann entropy (in the more general quantum case). Here, we show that the two techniques are closely related. More precisely, the spectral entropy rate can be seen as the asymptotic limit of the smooth entropy. Our results apply to the quantum setting and thus include the classical setting as a special case.

DOI
10.1109/tit.2009.2018340

Related papers

Smooth Entropies and the Quantum Information

Nilanjana Datta, Renato Renner

Many of the traditional results in information theory, such as the channel coding theorem or the source coding theorem, are restricted to scenarios where the underlying resources are in- dependent and identically distributed (i.i.d.) over a large number of uses. To overcome this limitation, two different techniques, the informationspectrummethodandthesmoothentropyframework, have been developed independently. They are based on new en- tropymeasures, calledspectralentropy ratesandsmooth entropies, respectively,that generalizeShannon entropy (in the classicalcase) and von Neumann entropy (in the more general quantum case). Here, we show that the two techniques are closely related. More precisely, the spectral entropy rate can be seen as the asymptotic limit of the smooth entropy. Our results apply to the quantum set- ting and thus include the classical setting as a special case.

•Jan 1, 2009•0 citations

A Framework for Non-Asymptotic Quantum Information Theory

Marco Tomamichel

This thesis consolidates, improves and extends the smooth entropy framework for non-asymptotic information theory and cryptography. We investigate the conditional min- and max-entropy for quantum states, generalizations of classical Rényi entropies. We introduce the purified distance, a novel metric for unnormalized quantum states and use it to define smooth entropies as optimizations of the min- and max-entropies over a ball of close states. We explore various properties of these entropies, including data-processing inequalities, chain rules and their classical limits. The most important property is an entropic formulation of the asymptotic equipartition property, which implies that the smooth entropies converge to the von Neumann entropy in the limit of many independent copies. The smooth entropies also satisfy duality and entropic uncertainty relations that provide limits on the power of two different observers to predict the outcome of a measurement on a quantum system. Finally, we discuss three example applications of the smooth entropy framework. We show a strong converse statement for source coding with quantum side information, characterize randomness extraction against quantum side information and prove information theoretic security of quantum key distribution using an intuitive argument based on the entropic uncertainty relation.

arXiv:1203.2142•Jan 1, 2012•225 citations

Information-Spectrum Approaches to Quantum Information Theory

Hiroshi Nagaoka

One of the principal themes of information theory is to establish a link between two different kinds of quantities: one is an operational quantity which is defined through the asymptotic optimality on some concrete concept such as code length, compression rate, error probability, etc., while the other is an abstract information quantity such as the entropy, divergence, mutual information, etc. The same theme is now being pursued extensively in quantum information theory, and lots of mathematical results have been reported so far. In the so-called information-spectrum methods in the classical information theory [1], the process of establishing a link between an operatinal quantity and an entropy-like information quantity is intentionally divided into two parts, making the third kind of quantities — information spectrum — intervene. In the methods, the asymptotic optimality of an operational quantity is characterized by a limiting expression on information spectra (asymptotic behavior of logarithmic likelihoods) in the most general setting, while the information-spectrum quantity is rewritten to an entropylike quantity in a specific situation mostly as a direct consequence of a limiting theorem in the probability theory such as the law of large numbers, the Shannon-McMillan-Breiman theorem, etc. Such a framework brings not only generality but also transparency of mathematical arguments. Extending the information-spectrum methods to the quantum case is an attractive subject, which is expected to provide both the asymptotic optimality of operational quantities and the limiting law governing quantum stochastic situations with transparent and comprehensive understanding. In this talk I would like to review recent attempts to develop the quantum informationspectrum methods made by Hayashi, Ogawa and myself, mainly based on [2, 3, 4], and discuss their significance in the quantum information theory.

•Jan 1, 2002•0 citations

Smooth Rényi Entropy of Ergodic Quantum Information Sources

Berry Schoenmakers, Jilles Tjoelker, Pim Tuyls +1 more

We investigate the recently introduced notion of smooth Renyi entropy for the case of ergodic information sources, thereby generalizing previous work which concentrated mainly on i.i.d. information sources. We will actually consider ergodic quantum information sources, of which ergodic classical information sources are a special case. We prove that the average smooth Renyi entropy rate will approach the entropy rate of a stationary, ergodic source, which is equal to the Shannon entropy rate for a classical source and the von Neumann entropy rate for a quantum source.

IEEE International Symposium on Information Theory (ISIT)•Jun 1, 2007•6 citations

Smooth Rényi Entropy of Ergodic Quantum Information Sources

Berry Schoenmakers, Jilles Tjoelker, Pim Tuyls +1 more

We prove that the average smooth Renyi entropy rate will approach the entropy rate of a stationary, ergodic information source, which is equal to the Shannon entropy rate for a classical information source and the von Neumann entropy rate for a quantum information source.

arXiv:0704.3504•Apr 26, 2007•6 citations

Duality Between Smooth Min- and Max-Entropies

Marco Tomamichel, Roger Colbeck, Renato Renner

In classical and quantum information theory, operational quantities such as the amount of randomness that can be extracted from a given source or the amount of space needed to store given data are normally characterized by one of two entropy measures, called smooth min-entropy and smooth max-entropy, respectively. While both entropies are equal to the von Neumann entropy in certain special cases (e.g., asymptotically, for many independent repetitions of the given data), their values can differ arbitrarily in the general case. In this paper, a recently discovered duality relation between (nonsmooth) min- and max-entropies is extended to the smooth case. More precisely, it is shown that the smooth min-entropy of a systemAconditioned on a systemBequals the negative of the smooth max-entropy ofAconditioned on a purifying systemC. This result immediately implies that certain operational quantities (such as the amount of compression and the amount of randomness that can be extracted from given data) are related. We explain how such relations have applications in cryptographic security proofs.

arXiv:0907.5238•IEEE Transactions on Information Theory•Aug 19, 2010•234 citations
Many of the traditional results in information theory, such as the channel coding theorem or the source coding theorem, are restricted to scenarios where the underlying resources are independent and identically distributed (i.i.d.) over a large number of uses. To overcome this limitation, two different techniques, the information spectrum method and the smooth entropy framework, have been developed independently. They are based on new entropy measures, called spectral entropy rates andsmoothentropies, respectively, that generalize Shannon entropy (in the classical case) and von Neumann entropy (in the more general quantum case). Here, we show that the two techniques are closely related. More precisely, the spectral entropy rate can be seen as the asymptotic limit of the smooth entropy. Our results apply to the quantum setting and thus include the classical setting as a special case.
Many of the traditional results in information theory, such as the channel coding theorem or the source coding theorem, are restricted to scenarios where the underlying resources are in- dependent and identically distributed (i.i.d.) over a large number of uses. To overcome this limitation, two different techniques, the informationspectrummethodandthesmoothentropyframework, have been developed independently. They are based on new en- tropymeasures, calledspectralentropy ratesandsmooth entropies, respectively,that generalizeShannon entropy (in the classicalcase) and von Neumann entropy (in the more general quantum case). Here, we show that the two techniques are closely related. More precisely, the spectral entropy rate can be seen as the asymptotic limit of the smooth entropy. Our results apply to the quantum set- ting and thus include the classical setting as a special case.
This thesis consolidates, improves and extends the smooth entropy framework for non-asymptotic information theory and cryptography. We investigate the conditional min- and max-entropy for quantum states, generalizations of classical Rényi entropies. We introduce the purified distance, a novel metric for unnormalized quantum states and use it to define smooth entropies as optimizations of the min- and max-entropies over a ball of close states. We explore various properties of these entropies, including data-processing inequalities, chain rules and their classical limits. The most important property is an entropic formulation of the asymptotic equipartition property, which implies that the smooth entropies converge to the von Neumann entropy in the limit of many independent copies. The smooth entropies also satisfy duality and entropic uncertainty relations that provide limits on the power of two different observers to predict the outcome of a measurement on a quantum system. Finally, we discuss three example applications of the smooth entropy framework. We show a strong converse statement for source coding with quantum side information, characterize randomness extraction against quantum side information and prove information theoretic security of quantum key distribution using an intuitive argument based on the entropic uncertainty relation.
One of the principal themes of information theory is to establish a link between two different kinds of quantities: one is an operational quantity which is defined through the asymptotic optimality on some concrete concept such as code length, compression rate, error probability, etc., while the other is an abstract information quantity such as the entropy, divergence, mutual information, etc. The same theme is now being pursued extensively in quantum information theory, and lots of mathematical results have been reported so far. In the so-called information-spectrum methods in the classical information theory [1], the process of establishing a link between an operatinal quantity and an entropy-like information quantity is intentionally divided into two parts, making the third kind of quantities — information spectrum — intervene. In the methods, the asymptotic optimality of an operational quantity is characterized by a limiting expression on information spectra (asymptotic behavior of logarithmic likelihoods) in the most general setting, while the information-spectrum quantity is rewritten to an entropylike quantity in a specific situation mostly as a direct consequence of a limiting theorem in the probability theory such as the law of large numbers, the Shannon-McMillan-Breiman theorem, etc. Such a framework brings not only generality but also transparency of mathematical arguments. Extending the information-spectrum methods to the quantum case is an attractive subject, which is expected to provide both the asymptotic optimality of operational quantities and the limiting law governing quantum stochastic situations with transparent and comprehensive understanding. In this talk I would like to review recent attempts to develop the quantum informationspectrum methods made by Hayashi, Ogawa and myself, mainly based on [2, 3, 4], and discuss their significance in the quantum information theory.
We investigate the recently introduced notion of smooth Renyi entropy for the case of ergodic information sources, thereby generalizing previous work which concentrated mainly on i.i.d. information sources. We will actually consider ergodic quantum information sources, of which ergodic classical information sources are a special case. We prove that the average smooth Renyi entropy rate will approach the entropy rate of a stationary, ergodic source, which is equal to the Shannon entropy rate for a classical source and the von Neumann entropy rate for a quantum source.
In classical and quantum information theory, operational quantities such as the amount of randomness that can be extracted from a given source or the amount of space needed to store given data are normally characterized by one of two entropy measures, called smooth min-entropy and smooth max-entropy, respectively. While both entropies are equal to the von Neumann entropy in certain special cases (e.g., asymptotically, for many independent repetitions of the given data), their values can differ arbitrarily in the general case. In this paper, a recently discovered duality relation between (nonsmooth) min- and max-entropies is extended to the smooth case. More precisely, it is shown that the smooth min-entropy of a systemAconditioned on a systemBequals the negative of the smooth max-entropy ofAconditioned on a purifying systemC. This result immediately implies that certain operational quantities (such as the amount of compression and the amount of randomness that can be extracted from given data) are related. We explain how such relations have applications in cryptographic security proofs.