The MAXENT (maximum entropy) principle gives a natural correspondance between (classical and quantum) statistical mechanics and information theory. Its general formulation leverages on log-concavity (or Klein's inequality) and the celebrated information inequality, which is equivalent to Gibbs' inequality. The resulting convex optimization allows one to relate entropy to the log-canonical partition function by Legendre-Fenchel duality. This can be further generalized to dual variational characterization of relative entropy or Kullback-Leibler divergence, relating Gibbs variationnal principle to Donsker-Vardhan variational representations.From an information theoretical point of view, the Gibbs principle and its associated evidence lower bound serves as the basis of modern machine learning algorithms, such as the expectation-maximization (EM) algorithm and its restriction to parameters output by a neural network. Interestingly, a large class of alternating optimization algorithms including EM and Blahut-Arimoto for computing channel capacity and source rate-distortion functions can be cast in a generic information geometric alternating projections on convex sets of measures. On the other hand, Donsker-Vardhan representations, when restricted to the output of a neural network, allows one to efficiently estimate information measures, and when applied to parametric estimators with quadratic risk, provide HCR-like and CRB-like statistical lower bounds. An alternative derivation on such lower bounds in the Bayesian setting makes use of the Weyl-Heisenberg uncertainty principle, which opens up new perspectives.In this keynote, I will select and review several of the aforementioned topics, and discuss the interactions between classical and quantum interpretations.
The MAXENT (maximum entropy) principle gives a natural correspondance between (classical and quantum) statistical mechanics and information theory. Its general formulation leverages on log-concavity (or Klein's inequality) and the celebrated information inequality, which is equivalent to Gibbs' inequality. The resulting convex optimization allows one to relate entropy to the log-canonical partition function by Legendre-Fenchel duality. This can be further generalized to dual variational characterization of relative entropy or Kullback-Leibler divergence, relating Gibbs variationnal principle to Donsker-Vardhan variational representations.From an information theoretical point of view, the Gibbs principle and its associated evidence lower bound serves as the basis of modern machine learning algorithms, such as the expectation-maximization (EM) algorithm and its restriction to parameters output by a neural network. Interestingly, a large class of alternating optimization algorithms including EM and Blahut-Arimoto for computing channel capacity and source rate-distortion functions can be cast in a generic information geometric alternating projections on convex sets of measures. On the other hand, Donsker-Vardhan representations, when restricted to the output of a neural network, allows one to efficiently estimate information measures, and when applied to parametric estimators with quadratic risk, provide HCR-like and CRB-like statistical lower bounds. An alternative derivation on such lower bounds in the Bayesian setting makes use of the Weyl-Heisenberg uncertainty principle, which opens up new perspectives.In this keynote, I will select and review several of the aforementioned topics, and discuss the interactions between classical and quantum interpretations.
A number of optimization problems in quantum information theory, such as one-shot classical capacity of a quantum channel, entanglement of formation of a state, classical capacity of a quantum observable, and accessible information of an ensemble of quantum states, can be formulated as the calculation of the convex closure of the output entropy of a certain quantum channel. This general problem can be considered as a convex programming problem for which necessary and sufficient optimality conditions can be established along with a dual problem. These optimality conditions imply some tight lower bounds for entropy, either quantum or classical, which are related to the famous logarithmic Sobolev inequality.
In thermodynamics and statistical mechanics entropy is the fundamental physical quantity to describe the evolution of a statistical ensemble. Its microscopic definition was provided by Boltzmann through the celebrated expression S = kB ln Γ, where kB is the Boltzmann constant and Γ is the number of microstates exploiting the given macroscopic properties. In this context, it is well known that in classical mechanics the entropy: i) allows the violation of the uncertainty principle [1], [2]; ii) can be considered as a special case of the so-called Boltzmann-Gibbs-Shannon entropy that enables one to apply results of information theory to physics [1], [3]. In particular, by introducing the principle of maximum entropy (MEP) it was found possible to derive rigorous hydrodynamic (HD) models based on the moments of the distribution function to all orders of a power expansion and including appropriate closure conditions [4], [5]. Accordingly, making use of the Lagrange multipliers technique, it was found possible to construct the set of evolution equations for the macro-variables of interest. Apart from some partial attempts [3], [6], this is no longer the case in quantum mechanics. Here, the main difficulties concern with: i) the definition of a proper quantum entropy that includes particle indistinguishability; ii) the formulation of a global quantum MEP (QMEP) that allows one to obtain a quantum distribution function both for thermodynamic equilibrium and nonequilibrium configurations. From one hand, in the framework of a nonlocal quantum theory, the generalization of the corresponding Lagrange multipliers is also an open problem. Fron another hand, a rigorous formulation of quantum HD (QHD) closed models is a demanding issue for many kinds of problems in quantum systems like, interacting fermionic and bosonic gases, quantum turbulence, quantum fluids, quantized vortices, nuclear physics, confined carrier transport in semiconductor heterostrucures, phonon and electron transport in nanostructures, nanowires and thin layers. The aim of this talk is to address the above drawbacks by defining a quantum entropy in terms of the reduced density matrix, thus explicitly incorporating quantum statistics into problems involving a system of identical particles. As a further step, with respect to the uncertainty principle, starting from the Wigner representation we formulate a quantum maximum entropy principle which, in the framework of information theory, allows us to obtain a non-local theory for the system under study. As a final step, we determine a closed quantum hydrodynamic model for the macroscopic variables used as constraints in the QMEP approach.
We initiate a study of the following problem: Given a continuous domain Ω along with its convex hull K, a point A∈K and a prior measure μ on Ω, find the probability density over Ω whose marginal is A and that minimizes the KL-divergence to μ. This framework gives rise to several extremal distributions that arise in mathematics, quantum mechanics, statistics, and theoretical computer science. Our technical contributions include a polynomial bound on the norm of the optimizer of the dual problem that holds in a very general setting and relies on a "balance" property of the measure μ on Ω, and exact algorithms for evaluating the dual and its gradient for several interesting settings of Ω and μ. Together, along with the ellipsoid method, these results imply polynomial-time algorithms to compute such KL-divergence minimizing distributions in several cases. Applications of our results include: 1) an optimization characterization of the Goemans-Williamson measure that is used to round a positive semidefinite matrix to a vector, 2) the computability of the entropic barrier for polytopes studied by Bubeck and Eldan, and 3) a polynomial-time algorithm to compute the barycentric quantum entropy of a density matrix that was proposed as an alternative to von Neumann entropy in the 1970s: this corresponds to the case when Ω is the set of rank one projections matrices and μ corresponds to the Haar measure on the unit sphere. Our techniques generalize to the setting of Hermitian rank k projections using the Harish-Chandra-Itzykson-Zuber formula, and are applicable even beyond, to adjoint orbits of compact Lie groups.
Jaynes' maximum entropy (MaxEnt) principle selects, from all probability distributions consistent with a set of constraints, the one that maximizes the Shannon entropy. The principle is well-defined for classical, real-valued probabilities, but extending it to complex-valued probability theories requires care. Youssef's complex probability framework, in which quantum mechanics is reformulated as a Bayesian theory with complex amplitudes and the observable probability is the squared modulus (Born rule), does not admit a natural MaxEnt extension: the L2 normalization of that framework is incompatible with the L1 normalization assumed by Shannon entropy, and the resulting complex-valued entropy functional cannot be maximized without additional structure absent from Youssef's axioms. The present paper proposes a different extension of Kolmogorov's axioms to the complex domain, replacing non-negativity with the condition that the real part of every probability be non-negative. This framework, which retains L1 normalization and recovers classical probability when the imaginary part vanishes, admits a natural MaxEnt principle: the real part of the complex Shannon entropy is a well-posed real functional, its Euler-Lagrange equations produce an explicit complex Gibbs distribution with complex Lagrange multipliers, and the classical Gibbs distribution is recovered as the special case of real multipliers. The die problem (Jaynes' Brandeis dice example) is worked out in detail for both the fair and the loaded cases, illustrating how imaginary constraints redistribute the observable probability mass in a way that has no counterpart in classical MaxEnt.
Historically at the core of thermodynamics and information theory, entropy's use in quantum information extends to diverse topics including high-energy physics and operator algebras. Entropy can gauge the extent to which a quantum system departs from classicality, including by measuring entanglement and coherence, and in the form of entropic uncertainty relations between incompatible measurements. The theme of this dissertation is the quantum nature of entropy, and how exposure to a noisy environment limits and degrades non-classical features. An especially useful and general form of entropy is the quantum relative entropy, of which special cases include the von Neumann and Shannon entropies, coherent and mutual information, and a broad range of resource-theoretic measures. We use mathematical results on relative entropy to connect and unify features that distinguish quantum from classical information. We present generalizations of the strong subadditivity inequality and uncertainty-like entropy inequalities to subalgebras of operators on quantum systems for which usual independence assumptions fail. We construct new measures of non-classicality that simultaneously quantify entanglement and uncertainty, leading to a new resource theory of operations under which these forms of non-classicalty become interchangeable. Physically, our results deepen our understanding of how quantum entanglement relates to quantum uncertainty. We show how properties of entanglement limit the advantages of quantum superadditivity for information transmission through channels with high but detectable loss. Our method, based on the monogamy and faithfulness of the squashed entanglement, suggests a broader paradigm for bounding non-classical effects in lossy processes. We also propose an experiment to demonstrate superadditivity. Finally, we estimate decay rates in the form of modified logarithmic Sobolev inequalities for a variety of quantum channels, and in many cases we obtain the stronger, tensor-stable form known as a complete logarithmic Sobolev inequality. We compare these with our earlier results that bound relative entropy of the outputs of a particular class of quantum channels.
The measured relative entropy and measured Rényi relative entropy are quantifiers of the distinguishability of two quantum states ρ and σ. They are defined as the maximum classical relative entropy or Rényi relative entropy realizable by performing a measurement on ρ and σ, and they have interpretations in terms of asymptotic quantum hypothesis testing. Crucially, they can be rewritten in terms of variational formulas involving the optimization of a concave or convex objective function over the set of positive definite operators. In this paper, we establish foundational properties of these objective functions by analyzing their matrix gradients and Hessian superoperators; namely, we prove that these objective functions are β-smooth and γ-strongly convex / concave, where β and γ depend on the max-relative entropies of ρ and σ. A practical consequence of these properties is that we can conduct Nesterov accelerated projected gradient descent / ascent, a well known classical optimization technique, to calculate the measured relative entropy and measured Rényi relative entropy to arbitrary precision. These algorithms are generally more memory efficient than our previous algorithms based on semi-definite optimization [Huang and Wilde, arXiv:2406.19060], and for well conditioned states ρ and σ, these algorithms are notably faster.