Entropy

Intrinsic Information-Theoretic Models

David Bernal‐Casas, Josep Maria Oller

Published Apr 28, 2024• 3 citations

Abstract

With this follow-up paper, we continue developing a mathematical framework based on information geometry for representing physical objects. The long-term goal is to lay down informational foundations for physics, especially quantum physics. We assume that we can now model information sources as univariate normal probability distributions N (μ, σ0), as before, but with a constant σ0 not necessarily equal to 1. Then, we also relaxed the independence condition when modeling m sources of information. Now, we model m sources with a multivariate normal probability distribution Nm(μ,Σ0) with a constant variance–covariance matrix Σ0 not necessarily diagonal, i.e., with covariance values different to 0, which leads to the concept of modes rather than sources. Invoking Schrödinger’s equation, we can still break the information into m quantum harmonic oscillators, one for each mode, and with energy levels independent of the values of σ0, altogether leading to the concept of “intrinsic”. Similarly, as in our previous work with the estimator’s variance, we found that the expectation of the quadratic Mahalanobis distance to the sample mean equals the energy levels of the quantum harmonic oscillator, being the minimum quadratic Mahalanobis distance at the minimum energy level of the oscillator and reaching the “intrinsic” Cramér–Rao lower bound at the lowest energy level. Also, we demonstrate that the global probability density function of the collective mode of a set of m quantum harmonic oscillators at the lowest energy level still equals the posterior probability distribution calculated using Bayes’ theorem from the sources of information for all data values, taking as a prior the Riemannian volume of the informative metric. While these new assumptions certainly add complexity to the mathematical framework, the results proven are invariant under transformations, leading to the concept of “intrinsic” information-theoretic models, which are essential for developing physics.

DOI: 10.3390/e26050370

Related papers

Information-Theoretic Models for Physical Observables

David Bernal‐Casas, Josep Maria Oller

This work addresses J.A. Wheeler's critical idea that all things physical are information-theoretic in origin. In this paper, we introduce a novel mathematical framework based on information geometry, using the Fisher information metric as a particular Riemannian metric, defined in the parameter space of a smooth statistical manifold of normal probability distributions. Following this approach, we study the stationary states with the time-independent Schrödinger's equation to discover that the information could be represented and distributed over a set of quantum harmonic oscillators, one for each independent source of data, whose coordinate for each oscillator is a parameter of the smooth statistical manifold to estimate. We observe that the estimator's variance equals the energy levels of the quantum harmonic oscillator, proving that the estimator's variance is definitively quantized, being the minimum variance at the minimum energy level of the oscillator. Interestingly, we demonstrate that quantum harmonic oscillators reach the Cramér-Rao lower bound on the estimator's variance at the lowest energy level. In parallel, we find that the global probability density function of the collective mode of a set of quantum harmonic oscillators at the lowest energy level equals the posterior probability distribution calculated using Bayes' theorem from the sources of information for all data values, taking as a prior the Riemannian volume of the informative metric. Interestingly, the opposite is also true, as the prior is constant. Altogether, these results suggest that we can break the sources of information into little elements: quantum harmonic oscillators, with the square modulus of the collective mode at the lowest energy representing the most likely reality, supporting A. Zeilinger's recent statement that the world is not broken into physical but informational parts.

EntropyOct 14, 20236 citations

Statistical geometry in quantum mechanics

Dorje C. Brody, Lane P. Hughston

A statistical model M is a family of probability distributions, characterised by a set of continuous parameters known as the parameter space. This possesses natural geometrical properties induced by the embedding of the family of probability distributions into the space of all square–integrable functions. More precisely, by consideration of the square–root density function we can regard M as a submanifold of the unit sphere S in a real Hilbert space H. Therefore, H embodies the ‘state space’ of the probability distributions, and the geometry of the given statistical model can be described in terms of the embedding of M in S. The geometry in question is characterised by a natural Riemannian metric (the Fisher–Rao metric), thus allowing us to formulate the principles of classical statistical inference in a natural geometric setting. In particular, we focus attention on the variance lower bounds for statistical estimation, and establish generalisations of the classical Cramér–Rao and Bhattacharyya inequalities, described in terms of the geometry of the underlying real Hilbert space. As a comprehensive illustration of the utility of the geometric framework, the statistical model M is then specialised to the case of a submanifold of the state space of a quantum mechanical system. This is pursued by introducing a compatible complex structure on the underlying real Hilbert space, which allows the operations of ordinary quantum mechanics to be reinterpreted in the language of real Hilbert space geometry. The application of generalised variance bounds in the case of quantum statistical estimation leads to a set of higher order corrections to the Heisenberg uncertainty relations for canonically conjugate observables.

arXiv:gr-qc/9701051Proceedings of the Royal Society A Mathematical Physical and Engineering SciencesSep 8, 199880 citations

Εφαρμογές της γεωμετρίας της πληροφορίας

Fotios D. Oikonomou, Φώτιος Οικονόμου

Information geometry usually refers to a class of geometries characterized from metrics and corresponding connections related to the notion of statistical information during the process of parameter estimation on classical or quantum systems. We are aware of the fact that in parameter estimation we can't achieve perfectness. There is a lower bound, which is not zero, for the variance - covariance of the estimated parameter compared to the real parameter, the Cramer - Rao bound. In classical parameter estimation, this bound is given by the reciprocal of Fisher information and is unique in the sense that the Fisher is the unique monotone metric under stochastic transformations. In quantum parameter estimation there is a class of monotone metrics and the lowest possible bound, which determines the best quality of estimation, is given by that of the Symmetric Logarithmic Derivative (SLD). The term symmetric determines the derivative type, since there are more than one types due to the non-commutativity of operators, in general, in quantum mechanics. In our first published paper in the framework of this Ph.D. thesis, we are working beyond the class on monotone metrics. We are using the phase space to define a new statistical metric. This is achieved by introducing logarithmic derivatives through the Husimi Q distribution and consequently the new metric. As it turns out the related bound of Cramer - Rao type, can be greater even of this of the Symmetric Logarithmic Derivative. In our second paper, we refer to a particular bi-parameter family of generalized entropies used for the classification of complex systems. By defining a divergence for the discrete distribution, we consequently define a metric and the corresponding Levi - Civita connection, which depend on the two parameters say c and d. So, we are having geometrical objects with statistical content like the Cramer - Rao bound and the scalar curvature, which is characteristic (and different) for every class of complex systems. In our third paper, we are treating the above parameters c and d as coordinates of a two dimensional manifold. By calculating the Fisher metric and the corresponding scalar curvature we observe particular values of c and d for which we are making the conjecture that they related with particular properties of complex systems. Finally, we are highlighting a correlation of the curvature of a particular manifold and quantum entanglement. In particular, we are using the entropy of entanglement to define a two dimensional manifold, which we embed into three dimensional real space. We find that in the region which this surface has maximum curvature we have product quantum states and only there.

Jan 1, 20190 citations

Statistical Geometry in Quantum Mechanics

Dorje C. Brody, Lane P. Hughston

Jan 1, 199867 citations

Quantum-mechanical description of physical systems

Carlos Navarrete–Benlloch

Quantum information is an emerging field which has attracted a lot of attention in the last couple of decades. It is a broad subject which extends from the most applied questions (e.g. how to build quantum computers or secure cryptographic systems) to the most theoretical problems concerning the formalism and interpretation of quantum mechanics, its complexity, and its potential to go beyond classical physics. This book is an introduction to quantum information with special emphasis on continuous-variable systems (such as light) which can be described as collections of harmonic oscillators. It covers a selection of basic concepts, focusing on their physical meaning and mathematical treatment. It starts from the very first principles of quantum mechanics, and builds up the concepts and techniques following a logical progression. This is an excellent reference for students with a full semester of standard quantum mechanics and researchers in closely related fields.

Morgan & Claypool Publishers eBooksDec 1, 20150 citations

A Bayes Rule for Density Matrices

Manfred K. Warmuth

The classical Bayes rule computes the posterior model probability from the prior probability and the data likelihood. We generalize this rule to the case when the prior is a density matrix (symmetric positive definite and trace one) and the data likelihood a covariance matrix. The classical Bayes rule is retained as the special case when the matrices are diagonal. In the classical setting, the calculation of the probability of the data is an expected likelihood, where the expectation is over the prior distribution. In the generalized setting, this is replaced by an expected variance calculation where the variance is computed along the eigenvectors of the prior density matrix and the expectation is over the eigenvalues of the density matrix (which form a probability vector). The variances along any direction is determined by the covariance matrix. Curiously enough this expected variance calculation is a quantum measurement where the co-variance matrix specifies the instrument and the prior density matrix the mixture state of the particle. We motivate both the classical and the generalized Bayes rule with a minimum relative entropy principle, where the Kullbach-Leibler version gives the classical Bayes rule and Umegaki’s quantum relative entropy the new Bayes rule for density matrices. 1

neural information processing systemsDec 5, 200518 citations