Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE

Estimation of classical and quantum entropy and other information-theoretic quantities

Alexei Kaltchenko

Published Mar 26, 2008• 1 citation
Publisher

Abstract

In this work, we study methods of entropy and relative entropy estimation for classical and quantum information sources. We show that algorithms and methods for classical entropy estimation can be effectively used in constructing algorithms for quantum entropy estimation. Specifically, we look at the entropy estimation for a broad class of stationary ergodic processes and obtain important bounds on estimator's variance and mathematical expectation as well as analyze their asymptotic behavior.

DOI
10.1117/12.778046

Related papers

Estimation of quantum entropies

Alexei Kaltchenko

Motivated by the importance of entropy functions in quantum data compression, entanglement theory, and various quantum information-processing tasks, this study demonstrates how classical algorithms for entropy estimation can effectively contribute to the construction of quantum algorithms for universal quantum entropy estimation. Given two quantum i.i.d. sources with completely unknown density matrices, algorithms are developed for estimating quantum cross entropy and quantum relative entropy. These estimation techniques represent a quantum generalization of the classical algorithms by Lempel, Ziv, and Merhav.

i-manager’s Journal on Mathematics•Jan 1, 2024•0 citations

On the nature and decay of quantum relative entropy

Nicholas D. LaRacuente

Historically at the core of thermodynamics and information theory, entropy's use in quantum information extends to diverse topics including high-energy physics and operator algebras. Entropy can gauge the extent to which a quantum system departs from classicality, including by measuring entanglement and coherence, and in the form of entropic uncertainty relations between incompatible measurements. The theme of this dissertation is the quantum nature of entropy, and how exposure to a noisy environment limits and degrades non-classical features. An especially useful and general form of entropy is the quantum relative entropy, of which special cases include the von Neumann and Shannon entropies, coherent and mutual information, and a broad range of resource-theoretic measures. We use mathematical results on relative entropy to connect and unify features that distinguish quantum from classical information. We present generalizations of the strong subadditivity inequality and uncertainty-like entropy inequalities to subalgebras of operators on quantum systems for which usual independence assumptions fail. We construct new measures of non-classicality that simultaneously quantify entanglement and uncertainty, leading to a new resource theory of operations under which these forms of non-classicalty become interchangeable. Physically, our results deepen our understanding of how quantum entanglement relates to quantum uncertainty. We show how properties of entanglement limit the advantages of quantum superadditivity for information transmission through channels with high but detectable loss. Our method, based on the monogamy and faithfulness of the squashed entanglement, suggests a broader paradigm for bounding non-classical effects in lossy processes. We also propose an experiment to demonstrate superadditivity. Finally, we estimate decay rates in the form of modified logarithmic Sobolev inequalities for a variety of quantum channels, and in many cases we obtain the stronger, tensor-stable form known as a complete logarithmic Sobolev inequality. We compare these with our earlier results that bound relative entropy of the outputs of a particular class of quantum channels.

•Jun 2, 2020•0 citations

Smooth Rényi Entropy of Ergodic Quantum Information Sources

Berry Schoenmakers, Jilles Tjoelker, Pim Tuyls +1 more

We investigate the recently introduced notion of smooth Renyi entropy for the case of ergodic information sources, thereby generalizing previous work which concentrated mainly on i.i.d. information sources. We will actually consider ergodic quantum information sources, of which ergodic classical information sources are a special case. We prove that the average smooth Renyi entropy rate will approach the entropy rate of a stationary, ergodic source, which is equal to the Shannon entropy rate for a classical source and the von Neumann entropy rate for a quantum source.

IEEE International Symposium on Information Theory (ISIT)•Jun 1, 2007•6 citations

Limit Theorems for Quantum Entropies and Applications

Igor Bjelaković

We present our recent results on limit theorems for quantum entropies and their applications in quantum information theory. They include the full extensions of the fundamental asymptotic equipartition property (AEP) to ergodic quantum informations sources and the ergodic version of Stein's Lemma on quantum hypothesis testing. These limit theorems are then used to obtain a data compression theorem for ergodic quantum sources and a quantum version of Sanov's theorem on large deviations for uncorrelated quantum sources. Moreover, Stein's lemma clearifies the operational interpretation of the quantum Kullback-Leibler distance (relative entropy), and this operational understanding of this quantity can be employed to give a new intuitive proof of the monotonicity of this statistical distance measure relying on information-theoretic methods.

•Apr 3, 2006•0 citations

Smooth Rényi Entropy of Ergodic Quantum Information Sources

Berry Schoenmakers, Jilles Tjoelker, Pim Tuyls +1 more

We prove that the average smooth Renyi entropy rate will approach the entropy rate of a stationary, ergodic information source, which is equal to the Shannon entropy rate for a classical information source and the von Neumann entropy rate for a quantum information source.

arXiv:0704.3504•Apr 26, 2007•6 citations

Quantum Algorithmic Complexities and Entropy

Fabio Benatti

We review the basics of classical algorithmic complexity theory and two of its quantum extensions that have been prompted by the foreseeable existence of quantum computing devices. In particular, we will examine the relations between these extensions and the von Neumann entropy rate of generic quantum information sources of ergodic type.

Open Systems & Information Dynamics•Mar 1, 2009•30 citations
In this work, we study methods of entropy and relative entropy estimation for classical and quantum information sources. We show that algorithms and methods for classical entropy estimation can be effectively used in constructing algorithms for quantum entropy estimation. Specifically, we look at the entropy estimation for a broad class of stationary ergodic processes and obtain important bounds on estimator's variance and mathematical expectation as well as analyze their asymptotic behavior.
Motivated by the importance of entropy functions in quantum data compression, entanglement theory, and various quantum information-processing tasks, this study demonstrates how classical algorithms for entropy estimation can effectively contribute to the construction of quantum algorithms for universal quantum entropy estimation. Given two quantum i.i.d. sources with completely unknown density matrices, algorithms are developed for estimating quantum cross entropy and quantum relative entropy. These estimation techniques represent a quantum generalization of the classical algorithms by Lempel, Ziv, and Merhav.
Historically at the core of thermodynamics and information theory, entropy's use in quantum information extends to diverse topics including high-energy physics and operator algebras. Entropy can gauge the extent to which a quantum system departs from classicality, including by measuring entanglement and coherence, and in the form of entropic uncertainty relations between incompatible measurements. The theme of this dissertation is the quantum nature of entropy, and how exposure to a noisy environment limits and degrades non-classical features. An especially useful and general form of entropy is the quantum relative entropy, of which special cases include the von Neumann and Shannon entropies, coherent and mutual information, and a broad range of resource-theoretic measures. We use mathematical results on relative entropy to connect and unify features that distinguish quantum from classical information. We present generalizations of the strong subadditivity inequality and uncertainty-like entropy inequalities to subalgebras of operators on quantum systems for which usual independence assumptions fail. We construct new measures of non-classicality that simultaneously quantify entanglement and uncertainty, leading to a new resource theory of operations under which these forms of non-classicalty become interchangeable. Physically, our results deepen our understanding of how quantum entanglement relates to quantum uncertainty. We show how properties of entanglement limit the advantages of quantum superadditivity for information transmission through channels with high but detectable loss. Our method, based on the monogamy and faithfulness of the squashed entanglement, suggests a broader paradigm for bounding non-classical effects in lossy processes. We also propose an experiment to demonstrate superadditivity. Finally, we estimate decay rates in the form of modified logarithmic Sobolev inequalities for a variety of quantum channels, and in many cases we obtain the stronger, tensor-stable form known as a complete logarithmic Sobolev inequality. We compare these with our earlier results that bound relative entropy of the outputs of a particular class of quantum channels.
We investigate the recently introduced notion of smooth Renyi entropy for the case of ergodic information sources, thereby generalizing previous work which concentrated mainly on i.i.d. information sources. We will actually consider ergodic quantum information sources, of which ergodic classical information sources are a special case. We prove that the average smooth Renyi entropy rate will approach the entropy rate of a stationary, ergodic source, which is equal to the Shannon entropy rate for a classical source and the von Neumann entropy rate for a quantum source.
We present our recent results on limit theorems for quantum entropies and their applications in quantum information theory. They include the full extensions of the fundamental asymptotic equipartition property (AEP) to ergodic quantum informations sources and the ergodic version of Stein's Lemma on quantum hypothesis testing. These limit theorems are then used to obtain a data compression theorem for ergodic quantum sources and a quantum version of Sanov's theorem on large deviations for uncorrelated quantum sources. Moreover, Stein's lemma clearifies the operational interpretation of the quantum Kullback-Leibler distance (relative entropy), and this operational understanding of this quantity can be employed to give a new intuitive proof of the monotonicity of this statistical distance measure relying on information-theoretic methods.