Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2008.05932

Kullback-Leibler divergence between quantum distributions, and its upper-bound

Vincenzo Bonnici

Published Aug 13, 2020• 3 citations
arXivHTMLPDF

Abstract

This work presents an upper-bound to value that the Kullback-Leibler (KL) divergence can reach for a class of probability distributions called quantum distributions (QD). The aim is to find a distribution $U$ which maximizes the KL divergence from a given distribution $P$ under the assumption that $P$ and $U$ have been generated by distributing a given discrete quantity, a quantum. Quantum distributions naturally represent a wide range of probability distributions that are used in practical applications. Moreover, such a class of distributions can be obtained as an approximation of any probability distribution. The retrieving of an upper-bound for the entropic divergence is here shown to be possible under the condition that the compared distributions are quantum distributions over the same quantum value, thus they become comparable. Thus, entropic divergence acquires a more powerful meaning when it is applied to comparable distributions. This aspect should be taken into account in future developments of divergences. The theoretical findings are used for proposing a notion of normalized KL divergence that is empirically shown to behave differently from already known measures.

DOI
10.48550/arxiv.2008.05932

Related papers

A maximum value for the Kullback-Leibler divergence between quantum discrete distributions

Vincenzo Bonnici

This work presents an upper-bound for the maximum value that the Kullback-Leibler (KL) divergence can reach for a newly introduced class of probability distributions, called quantum distributions (QD). In particular, the aim is to find a discrete distribution $U$ which maximize the finite KL divergence from a given $P$ under the assumption that $P$ and $U$ have been generated by distributing a fixed discretized quantity. The theoretical findings are used for proposing a notion of normalized KL divergence that is empirically shown to behave differently from already known measures.

•Aug 13, 2020•0 citations

Quantum-Inspired Fidelity-based Divergence

Yifeng Peng, Dantong Li, Xinyi Li +3 more

Kullback--Leibler (KL) divergence is a fundamental measure of the dissimilarity between two probability distributions, but it can become unstable in high-dimensional settings due to its sensitivity to mismatches in distributional support. To address robustness limitations, we propose a novel Quantum-Inspired Fidelity-based Divergence (QIF), leveraging quantum information principles yet efficiently computable on classical hardware. Compared to KL divergence, QIF demonstrates improved numerical stability under partial or near-disjoint support conditions, thereby reducing the need for extensive regularization in specific scenarios. Moreover, QIF admits well-defined theoretical bounds and continuous similarity measures. Building on this, we introduce a novel regularization method, QR-Drop, which utilizes QIF to improve generalization in machine learning models. Empirical results show that QR-Drop effectively mitigates overfitting and outperforms state-of-the-art methods.

arXiv:2501.19307•Jan 31, 2025•0 citations

On maximization of measured $f$-divergence between a given pair of quantum states

Keiji Matsumoto

This paper deals with maximization of classical $f$-divergence between the distributions of a measurement outputs of a given pair of quantum states. $f$-divergence $D_{f}$ between the probability density functions $p_{1}$ and $p_{2}$ over a discrete set is defined as $D_{f}( p_{1}||p_{2}) :=\sum_{x}p_{2}(x) f\left(p_{1}(x)/p_{2}( x) \right) $. For example, Kullback-Leibler divergence and Renyi type relative entropy are well-known examples with good operational meanings. Thus, finding the maximal value $D_{f}^{\min}$ of measured measured $f$-divergence is also an interesting question. But so far the question is solved only for very restricted example of $f$. \ The purposes of the present paper is to advance the study further, by investigating its properties, rewriting the maximization problem to more tractable form, and giving closed formulas of the quantity in some special cases.

arXiv:1412.3676•Dec 11, 2014•2 citations

Asymptotic Attainment for Quantum Relative Entropy

Masahito Hayashi

In this paper it was proved that the quantum relative entropy D(k) can be asymptotically attained by Kullback Leibler divergences of probabilities given by a certain sequence of measurements. The sequence of measurements depends on , but is independent of the choice of . In classical statistical theory the relative entropy D(pkq) is an information quantity which means the statistical eÆciency in order to distinguish a probability measure p of a measurable space from another probability measure q of the same measurable space. The states correspond to measures on measurable space. When p; q are discrete probabilities, the relative entropy (called also information divergence) introduced by Kullback and Leibler is dened by [1]: D(pkq) := X i p i log p i q i : In general, when p; q are measures on measurable space the relative entropy is dened by: D(pkq) := Z log dq dp (!)p( d!); where dq dp (!) is Radon-Nikodym derivative of q with respect to p. In this paper, we co...

•Apr 24, 1997•2 citations

Nonadditive generalization of the quantum Kullback-Leibler divergence for measuring the degree of purification

Sumiyoshi Abe

The Kullback-Leibler divergence offers an information-theoretic basis for measuring the difference between two given distributions. Its quantum analog, however, fails to play a corresponding role for comparing two density matrices, if the reference states are pure states. Here it is shown that nonadditive quantum information theory inspired by nonextensive statistical mechanics is free from such a difficulty and the associated quantity, termed the quantum q-divergence, can in fact be a good information-theoretic measure of the degree of state purification. The correspondence relation between the ordinary divergence and the q-divergence is violated for the pure reference states, in general.

arXiv:quant-ph/0301136•Physical Review A•Sep 10, 2003•63 citations

Jensen-Shannon divergence as a measure of distinguishability between mixed quantum states

A. P. Majtey, Pedro W. Lamberti, Domingo P. Prato

We discuss an alternative to relative entropy as a measure of distance between mixed quantum states. The proposed quantity is an extension to the realm of quantum theory of the Jensen-Shannon divergence (JSD) between probability distributions. The JSD has several interesting properties. It arises in information theory and, unlike the Kullback-Leibler divergence, it is symmetric, always well-defined, and bounded. We show that the quantum JSD shares with the relative entropy most of the physically relevant properties, in particular those required for a ``good'' quantum distinguishability measure. We relate it to other known quantum distances and we suggest possible applications in the field of the quantum information theory.

arXiv:quant-ph/0508138•Physical Review A•Nov 11, 2005•219 citations
This work presents an upper-bound to value that the Kullback-Leibler (KL) divergence can reach for a class of probability distributions called quantum distributions (QD). The aim is to find a distribution UUU which maximizes the KL divergence from a given distribution PPP under the assumption that PPP and UUU have been generated by distributing a given discrete quantity, a quantum. Quantum distributions naturally represent a wide range of probability distributions that are used in practical applications. Moreover, such a class of distributions can be obtained as an approximation of any probability distribution. The retrieving of an upper-bound for the entropic divergence is here shown to be possible under the condition that the compared distributions are quantum distributions over the same quantum value, thus they become comparable. Thus, entropic divergence acquires a more powerful meaning when it is applied to comparable distributions. This aspect should be taken into account in future developments of divergences. The theoretical findings are used for proposing a notion of normalized KL divergence that is empirically shown to behave differently from already known measures.
This work presents an upper-bound for the maximum value that the Kullback-Leibler (KL) divergence can reach for a newly introduced class of probability distributions, called quantum distributions (QD). In particular, the aim is to find a discrete distribution UUU which maximize the finite KL divergence from a given PPP under the assumption that PPP and UUU have been generated by distributing a fixed discretized quantity. The theoretical findings are used for proposing a notion of normalized KL divergence that is empirically shown to behave differently from already known measures.
Kullback--Leibler (KL) divergence is a fundamental measure of the dissimilarity between two probability distributions, but it can become unstable in high-dimensional settings due to its sensitivity to mismatches in distributional support. To address robustness limitations, we propose a novel Quantum-Inspired Fidelity-based Divergence (QIF), leveraging quantum information principles yet efficiently computable on classical hardware. Compared to KL divergence, QIF demonstrates improved numerical stability under partial or near-disjoint support conditions, thereby reducing the need for extensive regularization in specific scenarios. Moreover, QIF admits well-defined theoretical bounds and continuous similarity measures. Building on this, we introduce a novel regularization method, QR-Drop, which utilizes QIF to improve generalization in machine learning models. Empirical results show that QR-Drop effectively mitigates overfitting and outperforms state-of-the-art methods.
On maximization of measured fff-divergence between a given pair of quantum states
This paper deals with maximization of classical fff-divergence between the distributions of a measurement outputs of a given pair of quantum states. fff-divergence DfD_{f}Df​ between the probability density functions p1p_{1}p1​ and p2p_{2}p2​ over a discrete set is defined as $D_{f}( p_{1}||p_{2}) :=\sum_{x}p_{2}(x) f\left(p_{1}(x)/p_{2}( x) \right) $. For example, Kullback-Leibler divergence and Renyi type relative entropy are well-known examples with good operational meanings. Thus, finding the maximal value Dfmin⁡D_{f}^{\min}Dfmin​ of measured measured fff-divergence is also an interesting question. But so far the question is solved only for very restricted example of fff. \ The purposes of the present paper is to advance the study further, by investigating its properties, rewriting the maximization problem to more tractable form, and giving closed formulas of the quantity in some special cases.
In this paper it was proved that the quantum relative entropy D(k) can be asymptotically attained by Kullback Leibler divergences of probabilities given by a certain sequence of measurements. The sequence of measurements depends on , but is independent of the choice of . In classical statistical theory the relative entropy D(pkq) is an information quantity which means the statistical eÆciency in order to distinguish a probability measure p of a measurable space from another probability measure q of the same measurable space. The states correspond to measures on measurable space. When p; q are discrete probabilities, the relative entropy (called also information divergence) introduced by Kullback and Leibler is dened by [1]: D(pkq) := X i p i log p i q i : In general, when p; q are measures on measurable space the relative entropy is dened by: D(pkq) := Z log dq dp (!)p( d!); where dq dp (!) is Radon-Nikodym derivative of q with respect to p. In this paper, we co...
The Kullback-Leibler divergence offers an information-theoretic basis for measuring the difference between two given distributions. Its quantum analog, however, fails to play a corresponding role for comparing two density matrices, if the reference states are pure states. Here it is shown that nonadditive quantum information theory inspired by nonextensive statistical mechanics is free from such a difficulty and the associated quantity, termed the quantum q-divergence, can in fact be a good information-theoretic measure of the degree of state purification. The correspondence relation between the ordinary divergence and the q-divergence is violated for the pure reference states, in general.
We discuss an alternative to relative entropy as a measure of distance between mixed quantum states. The proposed quantity is an extension to the realm of quantum theory of the Jensen-Shannon divergence (JSD) between probability distributions. The JSD has several interesting properties. It arises in information theory and, unlike the Kullback-Leibler divergence, it is symmetric, always well-defined, and bounded. We show that the quantum JSD shares with the relative entropy most of the physically relevant properties, in particular those required for a ``good'' quantum distinguishability measure. We relate it to other known quantum distances and we suggest possible applications in the field of the quantum information theory.