Open Systems & Information Dynamics

Rényi Extrapolation of Shannon Entropy

Karol Życzkowski

Published Sep 1, 2003• 181 citations

Abstract

Relations between Shannon entropy and Rényi entropies of integer order are discussed. For any N-point discrete probability distribution for which the Rényi entropies of order two and three are known, we provide a lower and an upper bound for the Shannon entropy. The average of both bounds provide an explicit extrapolation for this quantity. These results imply relations between the von Neumann entropy of a mixed quantum state, its linear entropy and traces.

DOI: 10.1023/a:1025128024427

Related papers

Renyi extrapolation of Shannon entropy

Karol Życzkowski

Relations between Shannon entropy and Renyi entropies of integer order are discussed. For any N-point discrete probability distribution for which the Renyi entropies of order two and three are known, we provide an lower and an upper bound for the Shannon entropy. The average of both bounds provide an explicit extrapolation for this quantity. These results imply relations between the von Neumann entropy of a mixed quantum state, its linear entropy and traces.

arXiv:quant-ph/0305062May 12, 200313 citations

R enyi Extrapolation of Shannon Entropy

Centrum Fizyki Teoretycznej, Polska Akademia Nauk

Relations between Shannon entropy and R enyi entropies of integer order are dis- cussed. For any N-point discrete probability distribution for which the R enyi entropies of order two and three are known, we provide a lower and an upper bound for the Shannon entropy. The average of both bounds provide an explicit extrapolation for this quantity. These results imply relations between the von Neumann entropy of a mixed quantum state, its linear entropy and traces.

Jan 1, 20033 citations

The structure of Rényi entropic inequalities

Noah Linden, Milán Mosonyi, Andreas Winter

arXiv:1212.0248Proceedings of the Royal Society A Mathematical Physical and Engineering SciencesAug 21, 201376 citations

Improved Quantum Algorithms for Rényi Entropies Estimation

Zongxing Xiong, Chengshen Gao, Lvzhou Li

quant-pharXiv:2610.19936Oct 15, 20260 citations

Connecting the von Neumann and Rényi entropies for fermions

Mark Fannes, Nicholas Van Ryn

We explore the relation between the von Neumann entropy and the Rényi entropies of integer orders for shift-invariant quasi-free fermionic lattice systems. We investigate the approximation of the von Neumann entropy by a combination of integer-order Rényi entropies and give an estimate for the quality of such an approximation.

arXiv:1205.0945Journal of Physics A Mathematical and TheoreticalSep 5, 20129 citations

On the relation between the shannon entropy and the von neumann entropy

Siu‐Wai Ho, Raymond W. Yeung

This paper presents three approaches to explore the relation between the Shannon entropy and the von Neumann entropy. The first two approaches are based on the measurement of the quantum state, while the third approach is based on the preparation of the quantum state. Each of these approaches leads to an alternative definition of the von Neumann entropy.

IEEE International Symposium on Information Theory (ISIT)Dec 23, 20041 citation

Open Systems & Information Dynamics

Rényi Extrapolation of Shannon Entropy

Karol Życzkowski

Published Sep 1, 2003• 181 citations

Publisher

Abstract

DOI: 10.1023/a:1025128024427

Related papers

Renyi extrapolation of Shannon entropy

Karol Życzkowski

arXiv:quant-ph/0305062May 12, 200313 citations

R enyi Extrapolation of Shannon Entropy

Centrum Fizyki Teoretycznej, Polska Akademia Nauk

Jan 1, 20033 citations

The structure of Rényi entropic inequalities

Noah Linden, Milán Mosonyi, Andreas Winter

We investigate the universal inequalities relating the α -Rényi entropies of the marginals of a multi-partite quantum state. This is in analogy to the same question for the Shannon and von Neumann entropies ( α =1), which are known to satisfy several non-trivial inequalities such as strong subadditivity. Somewhat surprisingly, we find for 0< α <1 that the only inequality is non-negativity: in other words, any collection of non-negative numbers assigned to the non-empty subsets of n parties can be arbitrarily well approximated by the α -entropies of the 2 n −1 marginals of a quantum state. For α >1, we show analogously that there are no non-trivial homogeneous (in particular, no linear) inequalities. On the other hand, it is known that there are further, nonlinear and indeed non-homogeneous, inequalities delimiting the α -entropies of a general quantum state. Finally, we also treat the case of Rényi entropies restricted to classical states (i.e. probability distributions), which, in addition to non-negativity, are also subject to monotonicity. For α ≠0,1, we show that this is the only other homogeneous relation.We investigate the universal inequalities relating the α -Rényi entropies of the marginals of a multi-partite quantum state. This is in analogy to the same question for the Shannon and von Neumann entropies ( α =1), which are known to satisfy several non-trivial inequalities such as strong subadditivity. Somewhat surprisingly, we find for 0< α <1 that the only inequality is non-negativity: in other words, any collection of non-negative numbers assigned to the non-empty subsets of n parties can be arbitrarily well approximated by the α -entropies of the 2 n −1 marginals of a quantum state. For α >1, we show analogously that there are no non-trivial homogeneous (in particular, no linear) inequalities. On the other hand, it is known that there are further, nonlinear and indeed non-homogeneous, inequalities delimiting the α -entropies of a general quantum state. Finally, we also treat the case of Rényi entropies restricted to classical states (i.e. probability distributions), which, in addition to non-negativity, are also subject to monotonicity. For α ≠0,1, we show that this is the only other homogeneous relation.

arXiv:1212.0248Proceedings of the Royal Society A Mathematical Physical and Engineering SciencesAug 21, 201376 citations

Improved Quantum Algorithms for Rényi Entropies Estimation

Zongxing Xiong, Chengshen Gao, Lvzhou Li

Entropy estimation of discrete probability distributions is a fundamental problem in statistical inference and quantum property estimation. In this paper, we study the problem of estimating Rényi entropy $H_α(\mathbf{p})$ for any distribution $\mathbf{p} = (p_i)_{i=1}^n$ given its purified quantum query-access. For non-integer $α> 1$ , our algorithm estimates $H_α(\mathbf{p})$ to within additive error $ε$ using only $\widetilde{\mathcal{O}}\big(n^{\max\{1/2,1-1/α\}}/ε\big)$ queries, which improves the best known upper bound for dependence on both $n$ and $1/ε$ . For $0<α<1$ , our algorithm has query complexity $\widetilde{\mathcal{O}}\big(n^{1/(2α)}/ε^{\max\{1,1/(2α)\}}\big)$ , which improves the dependence on $1/ε$ of the current best upper bound without incresing its $n$ dependence. Our approach adapts the multi-level estimation framework to relative approximation of the power sums and leverage binary search to remove the requirement of prior knowledge of them. We also strengthen the precision-dependent lower bound to $Ω\big(1/ε^{\max\{1,1/(2α)\}}\big)$ for $0<α<1$ , which identifies the optimal precision dependence of our algorithm.Entropy estimation of discrete probability distributions is a fundamental problem in statistical inference and quantum property estimation. In this paper, we study the problem of estimating Rényi entropy $H_α(\mathbf{p})$ for any distribution $\mathbf{p} = (p_i)_{i=1}^n$ given its purified quantum query-access. For non-integer $α> 1$, our algorithm estimates $H_α(\mathbf{p})$ to within additive error $ε$ using only $\widetilde{\mathcal{O}}\big(n^{\max\{1/2,1-1/α\}}/ε\big)$ queries, which improves the best known upper bound for dependence on both $n$ and $1/ε$. For $0<α<1$, our algorithm has query complexity $\widetilde{\mathcal{O}}\big(n^{1/(2α)}/ε^{\max\{1,1/(2α)\}}\big)$, which improves the dependence on $1/ε$ of the current best upper bound without incresing its $n$ dependence. Our approach adapts the multi-level estimation framework to relative approximation of the power sums and leverage binary search to remove the requirement of prior knowledge of them. We also strengthen the precision-dependent lower bound to $Ω\big(1/ε^{\max\{1,1/(2α)\}}\big)$ for $0<α<1$, which identifies the optimal precision dependence of our algorithm.

quant-pharXiv:2610.19936Oct 15, 20260 citations

Connecting the von Neumann and Rényi entropies for fermions

Mark Fannes, Nicholas Van Ryn

arXiv:1205.0945Journal of Physics A Mathematical and TheoreticalSep 5, 20129 citations

On the relation between the shannon entropy and the von neumann entropy

Siu‐Wai Ho, Raymond W. Yeung

IEEE International Symposium on Information Theory (ISIT)Dec 23, 20041 citation

H_α(\mathbf{p})

for any distribution

\mathbf{p} = (p_i)_{i=1}^n

given its purified quantum query-access. For non-integer

α> 1

, our algorithm estimates

H_α(\mathbf{p})

to within additive error

ε

using only

\widetilde{\mathcal{O}}\big(n^{\max\{1/2,1-1/α\}}/ε\big)

queries, which improves the best known upper bound for dependence on both

n

and

1/ε

. For

0<α<1

, our algorithm has query complexity

\widetilde{\mathcal{O}}\big(n^{1/(2α)}/ε^{\max\{1,1/(2α)\}}\big)

, which improves the dependence on

1/ε

of the current best upper bound without incresing its

n

dependence. Our approach adapts the multi-level estimation framework to relative approximation of the power sums and leverage binary search to remove the requirement of prior knowledge of them. We also strengthen the precision-dependent lower bound to

Ω\big(1/ε^{\max\{1,1/(2α)\}}\big)

for

0<α<1

, which identifies the optimal precision dependence of our algorithm.