It is shown that Shannon entropy, Fisher information, and quantum-mechanical kinetic energy may all be viewed as measures of correlation. We begin with a fundamental correlation-destroying map: a joint probability density $ρ(x,y)$ is replaced by $ρ_x(x)ρ_y(y)$, the product of its marginal distributions. In the discrete case, Shannon's entropy is nondecreasing under this map. For continuous distributions, this fundamental map leads to a well-behaved, coordinate-invariant correlation measure, analogous to the discrete Shannon entropy: \[ I[ρ] = h[ρ_x] + h[ρ_y] - h[ρ]. \] In the continuous case, however, another measure appears: Fisher information. The relative Fisher information $J(ρ\|ρ_xρ_y)$ behaves similarly under the fundamental map. For a normalized real quantum wavefunction, this decrease is exactly proportional to the decrease in mean kinetic energy, \[ \langle T\rangle_ψ- \langle T\rangle_Φ= \frac{\hbar^2}{8m} J(ρ\|ρ_xρ_y), \qquad Φ= \sqrt{ρ_x ρ_y}. \] In Jaynes's language, the result supports a principle of minimum justified correlation: given physical constraints and a set of possible distributions or related amplitudes satisfying those constraints, select from that set those with the least correlation.
This is Part I of a two-part paper. Here we develop the entropy, Fisher-information, and kinetic-energy identities above, and state the principle they support. Part II addresses questions which Part I raises but leaves unanswered: multiple solutions, time dependence, the role of spin, a route to the Schrödinger equation itself, and a proposed experimental test. AI has been used.
It is shown that Shannon entropy, Fisher information, and quantum-mechanical kinetic energy may all be viewed as measures of correlation. We begin with a fundamental correlation-destroying map: a joint probability density ρ(x,y) is replaced by ρx(x)ρy(y), the product of its marginal distributions. In the discrete case, Shannon's entropy is nondecreasing under this map. For continuous distributions, this fundamental map leads to a well-behaved, coordinate-invariant correlation measure, analogous to the discrete Shannon entropy: [ I[ρ] = h[ρ_x] + h[ρ_y] - h[ρ]. ] In the continuous case, however, another measure appears: Fisher information. The relative Fisher information J(ρ∥ρxρy) behaves similarly under the fundamental map. For a normalized real quantum wavefunction, this decrease is exactly proportional to the decrease in mean kinetic energy, [ \langle T\rangle_ψ- \langle T\rangle_Φ= \frac{\hbar^2}{8m} J(ρ|ρ_xρ_y), \qquad Φ= \sqrt{ρ_x ρ_y}. ] In Jaynes's language, the result supports a principle of minimum justified correlation: given physical constraints and a set of possible distributions or related amplitudes satisfying those constraints, select from that set those with the least correlation.
This is Part I of a two-part paper. Here we develop the entropy, Fisher-information, and kinetic-energy identities above, and state the principle they support. Part II addresses questions which Part I raises but leaves unanswered: multiple solutions, time dependence, the role of spin, a route to the Schrödinger equation itself, and a proposed experimental test. AI has been used.
Consider an isolated statistical system specified by a coordinate x and its probability density p(x). A functional of p(x) called ``Fisher information'' can be used to measure the degree of disorder of the system due to the spread in p(x). Fisher information may be minimized, subject to a physical constraint, to attain a temporal equilibrium solution p(x). When the constraint is linear in the mean kinetic energy of the system, the equilibrium solution p(x) often obeys the correct differential equation for the system. In this way, the Schr\"odinger (energy) wave equation, Klein-Gordon equation, Helmholtz wave equation, diffusion equation, Boltzmann law, and Maxwell-Boltzmann law may be derived from one classical principle of disorder. The convergence rate for Fisher information is about that for alternative use of maximum entropy (in problems where both have the same equilibrium solution). This suggests that Fisher information defines an arrow of time. The arrow points in the direction of decreasing accuracy for the determination of the mean, or ideal, value of a parameter.
The Klein–Gordon equation for spinless particles, as well as the Weyl–Pauli and Dirac equations for particles with spin, may be derived from one information-theoretic principle. Consider a gedanken experiment whereby the mean position of a particle in a central force field is to be estimated by means of one position measurement. An efficient (optimum) estimate obeys a condition of minimum Fisher information, or minimum precision, according to the second law of thermodynamics. When the Fisher information is minimized subject to a constraint on the mean-square kinetic energy for the particle, the solution is a probability law on position whose corresponding wave amplitude obeys the Klein-Gordon equation, the Weyl–Pauli equation, or the Dirac equation, depending on whether scalar (without spin) or vector (with spin) solutions are sought. The minimized Fisher information is proportional to the mean-square kinetic energy, minus the rest energy of the particle.
By considering the kinetic-energy term of the minimum principle for the Schr\{o}dinger equation as a measure of information, that minimum principle is viewed as a statistical estimation procedure, analogous to the manner in which Jaynes ({\it Phys. Rev.},{\bf 106}, 620, 1957) interpreted statistical mechanics. It is shown that the entropy formula of Boltzmann and Jaynes obey a property in common with the quantum-mechanical kinetic energy, in which both quantities are interpreted as measures of correlation. It is shown that this property is shared by the key terms in the minimum principles of relativistic quantum mechanics and General Relativity. It is shown how this principle may be extended to non-Riemannian nonEuclidean spaces, which leads to novel field equations for the torsion.
Classical and nonclassical contributions to Author’s resultant Shannon- and Fisher-type measures of the information content in general electronic state $$\varphi ( {\varvec{r}} ) =R( {\varvec{r}}) \hbox { exp}[\hbox {i}\phi ( {\varvec{r}} )]$$ , due to the state probability density $$p( {\varvec{r}} ) =R( {\varvec{r}} )^{2}$$ and its phase $$\phi ( {\varvec{r}} )$$ or current $${\varvec{j}}( {\varvec{r}} )=\left( \hbar /m \right) p( {\varvec{r}} )\nabla \phi \left( {\varvec{r}} \right) $$ distributions, respectively, are reexamined. The components of the overall entropy, $$\begin{aligned} S[\varphi ]\equiv -\int {p({\varvec{r}})[\ln p({\varvec{r}})+ 2\phi ( {\varvec{r}} )] \,d{\varvec{r}}\equiv S[p]+S[\phi ]}, \end{aligned}$$ are shown to determine the real and imaginary parts of the state complex Shannon entropy, H[φ]≡−2⟨φ∣lnφ∣φ⟩=S[p]+iS[ϕ], a natural quantum-amplitude generalization of the classical Shannon entropy. Its contributions are related to the associated terms in the state resultant Fisher information, $$\begin{aligned} I[\varphi ]\equiv & {} -4\langle {\varphi |\nabla ^{2}|\varphi } \rangle \equiv \int {p({\varvec{r}})\{[} \nabla \ln p({\varvec{r}})]^{2}+[2\nabla \phi \left( {\varvec{r}} \right) ]^{2}\}\,d{\varvec{r}}\equiv I[p]+I[\phi ] \\= & {} I[p]+\int {p({\varvec{r}})[(2m/\hbar ){\varvec{j}}({\varvec{r}})/p({\varvec{r}})]^{2}\,d{\varvec{r}}\equiv I[p]+I[{\varvec{j}}],} \\ \end{aligned}$$ and the gradient entropy: I~[φ]≡⟨φ∣[(∇lnp)2+(i2∇ϕ)2]∣φ⟩=I[p]−I[ϕ]=I~[p]+I~[ϕ].
In (1), we noted that a Gaussian wavefunction may be a solution to the time-independent Schrodinger equation (e.g. the ground state solution of the quantum oscillator) and simultaneously a solution of maximizing spatial Shannon’s entropy - density(x) ln(density(x)) subject to the constraint V(x) xx constant, where density(x) =W*(x)W(x) with W(x) being the wavefunction. In (2), which deals strictly with classical probabilities and the Maxwell-Boltzmann factor n(ei) = exp(-ei/T), we argued that this function is equivalent to: ln(n(ei)) = -ei/T. This in turn is equivalent to the maximization of ln(n(ei)!) with the constraint ei n(ei) which is part of: ln( N!/ Product over i n(ei)!) with the full constraint Sum over i ei n(ei)=Eave. Here n(ei)=N p(ei) where p(ei) is a probability and N a very large number. Thus a function of exp( something) may indicate that “something” * density(x) is a constraint with -density ln(density) being Shannon’s entropy. Quantum mechanics for a bound system yields a real wavefunction W(x) such that density(x)=W(x)W(x)=probability(x). Thus maximization of Shannon’s entropy - density(x) ln (density) subject to a constraint: density(x)B(x) should correspond to a solution for density of exp( B(x)). B(x) as part of the constraint, however, should represent a physical constraint. Thus if there is a solution to the Schrodinger equation of the form exp(.5B(x)) and B(x)density(x) represents the physical constraint in the system, classical maximization of probability solves the quantum problem in such a case. The above ideas may be applied to the quantum oscillator ground state and essentially repeat what is done in (1). We, however, try to generalize them to see if they may apply to a free quantum particle. In this case the probability used is exp(ipx) which is complex. It corresponds to P(x)=exp(ipx)exp(-ipx)=1 which is a uniform distribution which has maximum lack of information.
Given a coin, one has two complementary pieces of information and a corresponding probability of .5 and .5. Similar considerations apply to a die. As a result, one may get the impression that there exists a unique probability distribution in a probabilistic system and that this single probability scheme is linked to the constrained maximization of Shannon’s entropy or Fisher’s information. We argue however, that there may be different probability schemes in the same problem associated with different pieces of information which are relevant to the physics which occurs in the system. Thus, we argue that it is the physics which selects the probability distribution to be examined. For example, one may loosely think in terms of a detailed probability scheme versus a kind-of coarse grained one in the same system. For example, for a quantum free particle, P(x)=1 i.e there is the same probability for a particle to be at any x point. This, however, is based on velocity, i.e. the relationship between dx and dt. There is another piece of information in the system, namely mo which together with v yields momentum (either relativistic or nonrelativistic) .This need not be measured using a clock (which velocity requires), but by the impact of an impulse on a target. Thus, nonrelativistically p=m1v1=m2v2 give the same impulse, but have different velocities. Quantum mechanics shows that a free particle is governed by exp(ipx) which contains two unnormalized probability distributions cos(px) and sin(px). These are key for impulse interactions (i.e. the physics of interactions such as with a 2-slit apparatus etc), yet exp(ipx)/sqrt(L) maps into the “coarse-grained” P(x)=exp(ipx)exp(-ipx) / L = constant. P(x) is one probability scheme in the problem and exp(ipx) another (for a free particle). Each answers some physical questions, but one needs to know which probability to use (i.e. what information is relevant) to a particular situation. P(x) is relevant to motion in space not involving interactions. exp(ipx) is relevant to interactions (even with a magnetic vector potential causing no magnetic field) or with a 2-slit apparatus. Furthermore, in a quantum bound state, one has W*(x)W(x) = P(x) (W(x)=wavefunction =Sum over p a(p)exp(ipx)) and a*(p)a(p).=P(p). Each probability scheme corresponds to the analysis of a different kind of measurement, but various schemes may also be linked. Thus one may have to start with one to create the others. For example, exp(ipx) is the base for creating W(x)=Sum over p a(p)exp(ipx), which ultimately yields P(x)=W*(x)W(x). As a result, we argue that if probability exists in a system, it is not necessarily the case that there is a unique probability distribution. There may be various probability schemes linked with different information and different measurements/interactions. Thus, physics leads one to focus on a specific probability scheme. As a result, we argue that one must be careful when discussing pure statistical quantities and their extremization, because these quantities, like Shannon’s entropy and Fisher’ s information, contain a classical probability distribution, but they do not indicate which distribution to use in the case that a system has more than one. . As a result, the analysis of a system requires knowledge of the physics (interactions) occurring.We argue against thinking in terms of physics following directly from statistical considerations. The constrained extremization of one statistical quantity may yield a probability distribution related to one piece of information, but not to another. Furthermore, Shannon’s entropy and Fisher’s information pertain to classical probability 0<=P(x)<1 and Integral P(x)dx =1. The wavefunction, exp(ipx)/sqrt(L), is comprised of two probability distributions and ADD and MULTIPLIES statistically (i.e. OR and AND ) situations, but is involved in relative average calculations because it is not normalized. It is also different because it allows for positive and negative values. These are properties which do not follow from statistics using a classical P(x).