Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

SmoothRenyiEntropy of ErgodicQuantum Information Sources

Jilles Tjoelker

Published Jan 1, 2007• 0 citations
OpenAlex

Abstract

We investigate therecently introduced notion of smoothRenyi entropy forthecaseofergodic information sources, thereby generalizing previous workwhichconcentrated mainly oni.i.d. information sources. We willactually consider ergodic quantuminformation sources, ofwhichergodic classical infor- mationsources area special case. We provethattheaverage smoothRenyientropy ratewill approach theentropy rateofa stationary, ergodic source, whichisequal totheShannon entropy rateforaclassical source andthevonNeumannentropy rate foraquantumsource. I.INTRODUCTION Theelegant notion ofsmooth Renyi entropy wasintroduced recently byRennerandWolfin(6)forclassical information sources, andthenatural extension toquantuminformation sources wasdefined byRenner andKonigin(5). Inthese two papers andfurther workbyRennerandWolf(7), (4), many properties ofsmooth Renyi entropyandsmooth min-entropy andsmooth max-entropy inparticular havebeenstudied in detail. A central property ofsmooth Renyi entropy proved inthese worksisthat formemoryless (i.i.d.) information sources, the average smooth Renyi entropy ratewill approach theentropy rate ofthesource, whichisequal totheShannon entropy fora classical source andthevonNeumannentropy foraquantum source. Whereas, ingeneral, theaverage (conventional) Renyi entropy rateofamemoryless source doesnotconverge tothe source's entropy rate. Inthis paper weextend thestudy ofsmooth Renyi entropy tothemoregeneral class ofstationary, ergodic sources rather thanmemoryless sources. We willprovethatforboththe classical andthequantum casethat theaverage smooth Renyi entropy ratewill approach theShannon andthevonNeumann entropy rate, respectively. We will dosobyfirst treating the classical caseandthenreducing thequantumcasetothe classical onewithout losing generality. Ingeneral, smooth Renyi entropy oforder a > 1,anda oc(min-entropy) inparticular, isofcryptographic relevance (e.g., forrandomness-extraction), andsmoothRenyientropy oforder a < 1,anda = 0(max-entropy) inparticular, are relevant todatacompression (minimum encoding length). In these contexts, theimportance ofsmooth Renyi entropy isthat its rate isbasically equal totheShannon/von Neumannentropy rate forani.i.d. source (andforergodic sources aswell, aswe showinthis paper). Thisisnotthecaseforconventional Renyi entropy. Moregenerally, asshowninthepapers byRenner et al.mentioned above, smooth Renyientropy behaves muchas Shannon/von Neumannentropy does. Inthis paperwefocus ontheunconditional case, whereas muchoftheabovementioned workbyRenner etal.treats the moregeneral conditional case. We leave theextension tothe conditional caseforfuture work.However, we doconsider twonotions ofe-closeness, onebasedontrace distance (also knownasvariational orstatistical distance) andonebased on non-normalized density matrices (orprobability distributions), wherethelatter ismoresuitable tohandle theconditional case.' Thus, webelieve that ourresults canbeextended tothe conditional caseaswell.

Related papers

Smooth Rényi Entropy of Ergodic Quantum Information Sources

Berry Schoenmakers, Jilles Tjoelker, Pim Tuyls +1 more

We investigate the recently introduced notion of smooth Renyi entropy for the case of ergodic information sources, thereby generalizing previous work which concentrated mainly on i.i.d. information sources. We will actually consider ergodic quantum information sources, of which ergodic classical information sources are a special case. We prove that the average smooth Renyi entropy rate will approach the entropy rate of a stationary, ergodic source, which is equal to the Shannon entropy rate for a classical source and the von Neumann entropy rate for a quantum source.

IEEE International Symposium on Information Theory (ISIT)•Jun 1, 2007•6 citations

Smooth Rényi Entropy of Ergodic Quantum Information Sources

Berry Schoenmakers, Jilles Tjoelker, Pim Tuyls +1 more

We prove that the average smooth Renyi entropy rate will approach the entropy rate of a stationary, ergodic information source, which is equal to the Shannon entropy rate for a classical information source and the von Neumann entropy rate for a quantum information source.

arXiv:0704.3504•Apr 26, 2007•6 citations

A Framework for Non-Asymptotic Quantum Information Theory

Marco Tomamichel

This thesis consolidates, improves and extends the smooth entropy framework for non-asymptotic information theory and cryptography. We investigate the conditional min- and max-entropy for quantum states, generalizations of classical Rényi entropies. We introduce the purified distance, a novel metric for unnormalized quantum states and use it to define smooth entropies as optimizations of the min- and max-entropies over a ball of close states. We explore various properties of these entropies, including data-processing inequalities, chain rules and their classical limits. The most important property is an entropic formulation of the asymptotic equipartition property, which implies that the smooth entropies converge to the von Neumann entropy in the limit of many independent copies. The smooth entropies also satisfy duality and entropic uncertainty relations that provide limits on the power of two different observers to predict the outcome of a measurement on a quantum system. Finally, we discuss three example applications of the smooth entropy framework. We show a strong converse statement for source coding with quantum side information, characterize randomness extraction against quantum side information and prove information theoretic security of quantum key distribution using an intuitive argument based on the entropic uncertainty relation.

arXiv:1203.2142•Jan 1, 2012•225 citations

Smooth Entropies and the Quantum Information Spectrum

Nilanjana Datta, Renato Renner

Many of the traditional results in information theory, such as the channel coding theorem or the source coding theorem, are restricted to scenarios where the underlying resources are independent and identically distributed (i.i.d.) over a large number of uses. To overcome this limitation, two different techniques, the information spectrum method and the smooth entropy framework, have been developed independently. They are based on new entropy measures, called spectral entropy rates andsmoothentropies, respectively, that generalize Shannon entropy (in the classical case) and von Neumann entropy (in the more general quantum case). Here, we show that the two techniques are closely related. More precisely, the spectral entropy rate can be seen as the asymptotic limit of the smooth entropy. Our results apply to the quantum setting and thus include the classical setting as a special case.

arXiv:0801.0282•IEEE Transactions on Information Theory•May 19, 2009•85 citations

Non-Hermitian Generalization of Rényi Entropy

Daili Li, Chao Zheng

From their conception to present times, different concepts and definitions of entropy take key roles in a variety of areas from thermodynamics to information science, and they can be applied to both classical and quantum systems. Among them is the Rényi entropy. It is able to characterize various properties of classical information with a unified concise form. We focus on the quantum counterpart, which unifies the von Neumann entropy, max- and min-entropy, collision entropy, etc. It can only be directly applied to Hermitian systems because it usually requires that the density matrices is normalized. For a non-Hermitian system, the evolved density matrix may not be normalized; i.e., the trace can be larger or less than one as the time evolution. However, it is not well-defined for the Rényi entropy with a non-normalized probability distribution relevant to the density matrix of a non-Hermitian system, especially when the trace of the non-normalized density matrix is larger than one. In this work, we investigate how to describe the Rényi entropy for non-Hermitian systems more appropriately. We obtain a concisely and generalized form of α-Rényi entropy, which we extend the unified order-α from finite positive real numbers to zero and infinity. Our generalized α-Rényi entropy can be directly calculated using both of the normalized and non-normalized density matrices so that it is able to describe non-Hermitian entropy dynamics. We illustrate the necessity of our generalization by showing the differences between ours and the conventional Rényi entropy for non-Hermitian detuning two-level systems.

Entropy•Oct 30, 2022•21 citations

Asymptotic Expansions of Smooth Rényi Entropies and Their Applications

Yuta Sakai, Vincent Y. F. Tan

This study considers the unconditional smooth Rényi entropy, the smooth conditional Rényi entropy proposed by Kuzuoka [\emph{IEEE Trans.\ Inf.\ Theory}, vol.~66, no.~3, pp.~1674--1690, 2020], and a new quantity which we term the conditional smooth Rényi entropy. In particular, we examine asymptotic expansions of these entropies when the underlying source with its side-information is stationary and memoryless. Using these smooth Rényi entropies, we establish one-shot coding theorems of several information-theoretic problems: Campbell's source coding, guessing problems, and task encoding problems, all allowing errors. In each problem, we consider two error formalisms: the average and maximum error criteria, where the averaging and maximization are taken with respect to the side-information of the source. Applying our asymptotic expansions to the derived one-shot coding theorems, we derive various asymptotic fundamental limits for these problems when their error probabilities are allowed to be non-vanishing. We show that, in non-degenerate settings, the first-order fundamental limits differ under the average and maximum error criteria. This is in contrast to a different but related setting considered by the present authors (for variable-length conditional source coding allowing errors) in which the first-order terms are identical but the second-order terms are different under these criteria.

arXiv:2003.05545•Mar 11, 2020•2 citations
We investigate therecently introduced notion of smoothRenyi entropy forthecaseofergodic information sources, thereby generalizing previous workwhichconcentrated mainly oni.i.d. information sources. We willactually consider ergodic quantuminformation sources, ofwhichergodic classical infor- mationsources area special case. We provethattheaverage smoothRenyientropy ratewill approach theentropy rateofa stationary, ergodic source, whichisequal totheShannon entropy rateforaclassical source andthevonNeumannentropy rate foraquantumsource. I.INTRODUCTION Theelegant notion ofsmooth Renyi entropy wasintroduced recently byRennerandWolfin(6)forclassical information sources, andthenatural extension toquantuminformation sources wasdefined byRenner andKonigin(5). Inthese two papers andfurther workbyRennerandWolf(7), (4), many properties ofsmooth Renyi entropyandsmooth min-entropy andsmooth max-entropy inparticular havebeenstudied in detail. A central property ofsmooth Renyi entropy proved inthese worksisthat formemoryless (i.i.d.) information sources, the average smooth Renyi entropy ratewill approach theentropy rate ofthesource, whichisequal totheShannon entropy fora classical source andthevonNeumannentropy foraquantum source. Whereas, ingeneral, theaverage (conventional) Renyi entropy rateofamemoryless source doesnotconverge tothe source's entropy rate. Inthis paper weextend thestudy ofsmooth Renyi entropy tothemoregeneral class ofstationary, ergodic sources rather thanmemoryless sources. We willprovethatforboththe classical andthequantum casethat theaverage smooth Renyi entropy ratewill approach theShannon andthevonNeumann entropy rate, respectively. We will dosobyfirst treating the classical caseandthenreducing thequantumcasetothe classical onewithout losing generality. Ingeneral, smooth Renyi entropy oforder a > 1,anda oc(min-entropy) inparticular, isofcryptographic relevance (e.g., forrandomness-extraction), andsmoothRenyientropy oforder a < 1,anda = 0(max-entropy) inparticular, are relevant todatacompression (minimum encoding length). In these contexts, theimportance ofsmooth Renyi entropy isthat its rate isbasically equal totheShannon/von Neumannentropy rate forani.i.d. source (andforergodic sources aswell, aswe showinthis paper). Thisisnotthecaseforconventional Renyi entropy. Moregenerally, asshowninthepapers byRenner et al.mentioned above, smooth Renyientropy behaves muchas Shannon/von Neumannentropy does. Inthis paperwefocus ontheunconditional case, whereas muchoftheabovementioned workbyRenner etal.treats the moregeneral conditional case. We leave theextension tothe conditional caseforfuture work.However, we doconsider twonotions ofe-closeness, onebasedontrace distance (also knownasvariational orstatistical distance) andonebased on non-normalized density matrices (orprobability distributions), wherethelatter ismoresuitable tohandle theconditional case.' Thus, webelieve that ourresults canbeextended tothe conditional caseaswell.
We investigate the recently introduced notion of smooth Renyi entropy for the case of ergodic information sources, thereby generalizing previous work which concentrated mainly on i.i.d. information sources. We will actually consider ergodic quantum information sources, of which ergodic classical information sources are a special case. We prove that the average smooth Renyi entropy rate will approach the entropy rate of a stationary, ergodic source, which is equal to the Shannon entropy rate for a classical source and the von Neumann entropy rate for a quantum source.
This thesis consolidates, improves and extends the smooth entropy framework for non-asymptotic information theory and cryptography. We investigate the conditional min- and max-entropy for quantum states, generalizations of classical Rényi entropies. We introduce the purified distance, a novel metric for unnormalized quantum states and use it to define smooth entropies as optimizations of the min- and max-entropies over a ball of close states. We explore various properties of these entropies, including data-processing inequalities, chain rules and their classical limits. The most important property is an entropic formulation of the asymptotic equipartition property, which implies that the smooth entropies converge to the von Neumann entropy in the limit of many independent copies. The smooth entropies also satisfy duality and entropic uncertainty relations that provide limits on the power of two different observers to predict the outcome of a measurement on a quantum system. Finally, we discuss three example applications of the smooth entropy framework. We show a strong converse statement for source coding with quantum side information, characterize randomness extraction against quantum side information and prove information theoretic security of quantum key distribution using an intuitive argument based on the entropic uncertainty relation.
Many of the traditional results in information theory, such as the channel coding theorem or the source coding theorem, are restricted to scenarios where the underlying resources are independent and identically distributed (i.i.d.) over a large number of uses. To overcome this limitation, two different techniques, the information spectrum method and the smooth entropy framework, have been developed independently. They are based on new entropy measures, called spectral entropy rates andsmoothentropies, respectively, that generalize Shannon entropy (in the classical case) and von Neumann entropy (in the more general quantum case). Here, we show that the two techniques are closely related. More precisely, the spectral entropy rate can be seen as the asymptotic limit of the smooth entropy. Our results apply to the quantum setting and thus include the classical setting as a special case.
From their conception to present times, different concepts and definitions of entropy take key roles in a variety of areas from thermodynamics to information science, and they can be applied to both classical and quantum systems. Among them is the Rényi entropy. It is able to characterize various properties of classical information with a unified concise form. We focus on the quantum counterpart, which unifies the von Neumann entropy, max- and min-entropy, collision entropy, etc. It can only be directly applied to Hermitian systems because it usually requires that the density matrices is normalized. For a non-Hermitian system, the evolved density matrix may not be normalized; i.e., the trace can be larger or less than one as the time evolution. However, it is not well-defined for the Rényi entropy with a non-normalized probability distribution relevant to the density matrix of a non-Hermitian system, especially when the trace of the non-normalized density matrix is larger than one. In this work, we investigate how to describe the Rényi entropy for non-Hermitian systems more appropriately. We obtain a concisely and generalized form of α-Rényi entropy, which we extend the unified order-α from finite positive real numbers to zero and infinity. Our generalized α-Rényi entropy can be directly calculated using both of the normalized and non-normalized density matrices so that it is able to describe non-Hermitian entropy dynamics. We illustrate the necessity of our generalization by showing the differences between ours and the conventional Rényi entropy for non-Hermitian detuning two-level systems.
This study considers the unconditional smooth Rényi entropy, the smooth conditional Rényi entropy proposed by Kuzuoka [\emph{IEEE Trans.\ Inf.\ Theory}, vol.~66, no.~3, pp.~1674--1690, 2020], and a new quantity which we term the conditional smooth Rényi entropy. In particular, we examine asymptotic expansions of these entropies when the underlying source with its side-information is stationary and memoryless. Using these smooth Rényi entropies, we establish one-shot coding theorems of several information-theoretic problems: Campbell's source coding, guessing problems, and task encoding problems, all allowing errors. In each problem, we consider two error formalisms: the average and maximum error criteria, where the averaging and maximization are taken with respect to the side-information of the source. Applying our asymptotic expansions to the derived one-shot coding theorems, we derive various asymptotic fundamental limits for these problems when their error probabilities are allowed to be non-vanishing. We show that, in non-degenerate settings, the first-order fundamental limits differ under the average and maximum error criteria. This is in contrast to a different but related setting considered by the present authors (for variable-length conditional source coding allowing errors) in which the first-order terms are identical but the second-order terms are different under these criteria.