For the case of physical quantities with units, such as mass, time, length etc, a change in the value of the quantity makes sense by itself in addition to a relative change. In other words, one does not need to know the mass of an object to understand what a change of 1 kg is. Probability, on the other hand, is a strictly relative quantity and so needs to be normalized. If it is not normalized, one needs to know relative probabilities. To say that an unnormalized probability changes by a value of 10 does not mean anything. Given that equations often solve for an unnormalized probability function, we argue that the notion of the relative probability should appear in equations. Even in the case of ln(P(y)), called information in information theory, for which there is no derivative, taking a derivative, say d/dy, immediately creates a relative probability derivative. Thus the form ln(P) is in keeping with the notion of a relative derivative. Here we examine the idea of a relative derivative and its association with information ln(P), Fisher information, quantum mechanics and the Fokker-Planck equation as well as classical statistical balance in the presence of a potential.
For the case of physical quantities with units, such as mass, time, length etc, a change in the value of the quantity makes sense by itself in addition to a relative change. In other words, one does not need to know the mass of an object to understand what a change of 1 kg is. Probability, on the other hand, is a strictly relative quantity and so needs to be normalized. If it is not normalized, one needs to know relative probabilities. To say that an unnormalized probability changes by a value of 10 does not mean anything. Given that equations often solve for an unnormalized probability function, we argue that the notion of the relative probability should appear in equations. Even in the case of ln(P(y)), called information in information theory, for which there is no derivative, taking a derivative, say d/dy, immediately creates a relative probability derivative. Thus the form ln(P) is in keeping with the notion of a relative derivative. Here we examine the idea of a relative derivative and its association with information ln(P), Fisher information, quantum mechanics and the Fokker-Planck equation as well as classical statistical balance in the presence of a potential.
In information theory, ln(Probability) is called information. We argue that in various physical problems this information is linked with a conserved quantity and a variable which changes. For example, in the Maxwell-Boltzmann case the conserved quantity is 1/Temperature and the variable e+V(x) = energy. For a quantum free particle, the conserved quantity is momentum and the variable x. “e” may change subject to the conserved quantity 1/T and x may change subject to the conserved quantity p. Given that probability may be unnormalized, changes may be expressed in relative terms i.e. in terms of a derivative of ln(P) i.e. information. We argue that the derivative of ln(P) with respect to the variable which changes yields the value of the conserved quantity in special cases for which the conserved quantity is the only information governing changes in the probability distribution. Thus one has an eigenvalue equation: d/dvariable ln(P) = conserved quantity. This implies that ln(P) = variable * conserved quantity. This may lead to two different kinds of conservation equations. In the MB case, e is the variable and 1/T the conserved quantity, but ln(P(e1)) + ln(P(e2)) = 1/T (e1+e2) = ln(P(e1+e2)). An eigenvalue equation allows for this conservation. In the quantum free particle case: ln(P(k1,x)) + ln(P(k2,x)) where P=wavefunction = exp(ipx) (i.e. the square-root type probability is used), one has k1+k2=k3 with x the variable canceling out. (In the MB case, it was 1/T the conserved quantity canceling). Again, the eigenvalue equation allows for this conservation, this time of the conserved quantity. We note that the eigenvalue equation is a special case and not only probability distributions are associated with it. For example, a power law distribution’s changes in its variable do not depend only on the conserved quantity 1/T in P(e/T)=power law.
In (1) it is suggested that Shannon’s entropy - Sum over i P(i) ln(P(i)) be thought of as the average of lost information defined by ln(P(i)). To see why the function ln appears, it seems one needs to consider a product of probabilities: P=P(i)P(j). It is possible that another two probabilities k,l yield the same product probability P(k)P(l)=P. In such a case, given P one would not know which two probabilities created it. For a specific instance, however, one specifically knows P(i) and P(j) and so “information” so to speak is lost through the product. To describe information as a quantity which “adds” one may take ln i.e. ln(P)=ln(P(i)) + ln(P(j)) = ln(P(k)) + ln(P(l)) with ln(P(i)) being information. The above is established for only two probabilities, but a P(i)P(j) result suggests two independent “coin tosses” so to speak or roll of a die. As a result, there seems to be a generalization in the literature to N successive independent events with N being large or a series of arrangements of results making use of Stirling’s approximation. Stirling’s approximation depends on large N and so this approach suggests that Shannon’s entropy holds for a large number of events. We have argued above that if considered as a loss of entropy (1), one only needs a product of two events. It is possible that one does not even need two successive events such as the toss of a coin or a die. For a quantum bound state, a product of probabilities P(x)P(p/x) appears if one wishes to know the probability for a particle to be at x and have p. In general this is not allowable in quantum mechanics so P(p/x) is complex. It is possible to have different (x,p) pairs yield the same P(x)P(p/x) product so lost information would be: ln{ WW a(p)exp(ipx)/W) yielding an entropy of S=Sp+Sx where Sp and Sx are Shannon’s entropy for p and x. Thus the idea of successive events or arrangements is not needed, rather entropy deals with a loss of information through a product of probabilities as argued in (1). One may also have a loss of information through a sum of probabilities, but in such a case ln does not isolate the pieces of lost information. In physical systems, however, it seems that products of probabilities appear especially due to two body scattering or to an energy level absorbing a photon. The idea of sum then appears through a conserved quantity such as energy or momentum which dictates the form of P(i).
Given a coin, one has two complementary pieces of information and a corresponding probability of .5 and .5. Similar considerations apply to a die. As a result, one may get the impression that there exists a unique probability distribution in a probabilistic system and that this single probability scheme is linked to the constrained maximization of Shannon’s entropy or Fisher’s information. We argue however, that there may be different probability schemes in the same problem associated with different pieces of information which are relevant to the physics which occurs in the system. Thus, we argue that it is the physics which selects the probability distribution to be examined. For example, one may loosely think in terms of a detailed probability scheme versus a kind-of coarse grained one in the same system. For example, for a quantum free particle, P(x)=1 i.e there is the same probability for a particle to be at any x point. This, however, is based on velocity, i.e. the relationship between dx and dt. There is another piece of information in the system, namely mo which together with v yields momentum (either relativistic or nonrelativistic) .This need not be measured using a clock (which velocity requires), but by the impact of an impulse on a target. Thus, nonrelativistically p=m1v1=m2v2 give the same impulse, but have different velocities. Quantum mechanics shows that a free particle is governed by exp(ipx) which contains two unnormalized probability distributions cos(px) and sin(px). These are key for impulse interactions (i.e. the physics of interactions such as with a 2-slit apparatus etc), yet exp(ipx)/sqrt(L) maps into the “coarse-grained” P(x)=exp(ipx)exp(-ipx) / L = constant. P(x) is one probability scheme in the problem and exp(ipx) another (for a free particle). Each answers some physical questions, but one needs to know which probability to use (i.e. what information is relevant) to a particular situation. P(x) is relevant to motion in space not involving interactions. exp(ipx) is relevant to interactions (even with a magnetic vector potential causing no magnetic field) or with a 2-slit apparatus. Furthermore, in a quantum bound state, one has W*(x)W(x) = P(x) (W(x)=wavefunction =Sum over p a(p)exp(ipx)) and a*(p)a(p).=P(p). Each probability scheme corresponds to the analysis of a different kind of measurement, but various schemes may also be linked. Thus one may have to start with one to create the others. For example, exp(ipx) is the base for creating W(x)=Sum over p a(p)exp(ipx), which ultimately yields P(x)=W*(x)W(x). As a result, we argue that if probability exists in a system, it is not necessarily the case that there is a unique probability distribution. There may be various probability schemes linked with different information and different measurements/interactions. Thus, physics leads one to focus on a specific probability scheme. As a result, we argue that one must be careful when discussing pure statistical quantities and their extremization, because these quantities, like Shannon’s entropy and Fisher’ s information, contain a classical probability distribution, but they do not indicate which distribution to use in the case that a system has more than one. . As a result, the analysis of a system requires knowledge of the physics (interactions) occurring.We argue against thinking in terms of physics following directly from statistical considerations. The constrained extremization of one statistical quantity may yield a probability distribution related to one piece of information, but not to another. Furthermore, Shannon’s entropy and Fisher’s information pertain to classical probability 0<=P(x)<1 and Integral P(x)dx =1. The wavefunction, exp(ipx)/sqrt(L), is comprised of two probability distributions and ADD and MULTIPLIES statistically (i.e. OR and AND ) situations, but is involved in relative average calculations because it is not normalized. It is also different because it allows for positive and negative values. These are properties which do not follow from statistics using a classical P(x).
Classical mechanics in the ideal gas scenario and quantum mechanics make use of a set of energy levels which a single particle may occupy. Thus the problem becomes to determine the probability associated with ei i.e. P(ei). In this note we take the approach that relative probabilities depend on the energy gap in question. For example, consider a quantum two level system where ei values are provided by the time-independent Schrodinger equation. From the point of view of statistics, these are simply a set of ei values. Their associated physics is contained in the Schrodinger equation. For the quantum two level system one must find the relative probabilities PR(e1) and PR(e2). Next, imagine a quantum system with many levels and assume for the sake of argument there are levels e1,e2 and ei,e(i+1) such that e2-e1=e(i+1)-ei. If the mechanism is the same for jumping from e1 to e2 and from ei to e(i+1) (i.e. absorbing a photon of the same energy), one might argue that the relative probabilities PR(e2)/PR(e1) = PR(e(i+1))/PR(ei). From this one may obtain the Maxwell-Boltzman result exp(-ei/T)). Note that levels do not need to be adjacent. For example in a large level system one might consider e1 and e3 and any ei and ej such that e3-e1 = ej - ei. In some situations this may need to be modified (e.g. fermions and bosons have enhancements and there may be level related issues), but as a first step we argue that insisting on relative probabilities being dependent on ei-ej has important consequences. Furthermore, in such a case the normalization constant for these relative probabilities i.e. C(T)= Sum over i exp(-ei/T) is important. Taking ln(C(T) and then taking the derivative d/dT yields an expression proportional to the average energy. An expression for dE/dT dT, however, would lead to a second order differential equation in ln(C(T)). If one introduces a new function S such that E-TS = -Tln(C), then one may show that dE/dT = T dS/dT i.e. entropy S emerges.
Classical Newtonian mechanics separates a particle (effect) from a force (cause). A particle has properties at a given x (and time t), but if it interacts with a force, this interaction occurs over dt and dx. Given that many forces do not change in time, we argue that a notion of a force is inseparable from the idea of dx. If one gives the value of a force at x, this value acts within a dx, which in Newtonian mechanics tends to 0. We have argued before that a particle with momentum represents a force or impulse itself, but at the same time may receive impulse hits from other sources (e.g. a Newtonian force -dV/dx). If a particle represents a force(impulse) it should be inseparable from the notion of dx, even if it has a constant p (momentum) and is traveling through space. Thus the idea of a momentum defining an internal ruler gradation through a probability distribution in x which we have presented in previous notes is compatible with the idea of describing a particle as a force. If one only considers a particle as an effect, there would be no need to worry about dx until a force(x) appears. An internal dx is not defined by an external ruler and does not need to tend to 0 as in Newtonian mechanics. Rather it is statistical in nature as p is a fixed impulse, but the probability to be at different x points within dx (called a wavelength) differ. In (1) we compared a classical Maxwell-Boltzmann gas to a quantum free particle. We argued that d/d density Pressure = RT = average pressure of a particle. We suggested this is analogous to the quantum free particle situation: (-id/dx probability(p,x) ) / probability(p,x) = p where probability(p,x) is unnormalized. In other words a change in a probability yields a deterministic value p. The notion of a deterministic value p associated with a probability distribution in what would be another deterministic variable x in Newtonian mechanics (except now one has a dx distribution because the particle is a force/impulse) has consequences. If one wishes to have a particle sit at x=0, say in an oscillator system, one needs to use a wavelength which is fairly small and centered at x=0. The wavelength, however, is associated with momentum and even being small spreads out from x=0 meaning there is potential energy .5kxx as well. Thus one may use an ensemble of forces/impulses Sum over p a(p)exp(ipx) to create a “dx” about x=0, but at the same time this ensemble represents an average particle (defined at x with prms(x)) which must satisfy KE(x) + V(x) = En i.e. the classical conservation expression. In other words, one sees again that average statistical quantities may behave as deterministic ones.
In (1), a variational approach is applied to Fisher’s information (with constraints) to obtain a differential equation in P(x)=probability to be at x. It is suggested the equation may be simplified using P(x)=W(x)W(x) where W(x) is real. The result is the nonrelativistic Schrodinger equation with mass=1. As a first observation, we note that Fisher’s information, which is a purely statistical quantity, should not necessarily yield a nonrelativistic equation which is an approximation of relativistic results. In particular, we argue that Fisher’s information is used to find <pp> not necessarily 1/2m <pp>, but this is a minor point. A more important issue, we argue, is the meaning of P(x) in a bound system. A classical bound system is sometimes said to have a P(x)=C/v(x) i.e. P(x)dx is proportional to dt, the amount of time a particle spends in each equally sized dx. In the high energy limit, the envelope of a quantum bound probability P(x)=W(x)W(x) (W(x)=wavefunction) is said to roughly match C/v(x), yet this density does not follow from the extremization of Fisher’s information (together with two general constraints which yield E and V(x)=potential). This begs the question: What is P(x)? If one considers a particle moving at constant speed (i.e constant momentum) then P(x)=constant because each dx point carries the same probability. Classically a particle placed in a box with infinite potential walls would have a P(x) =constant and p and -p motion would be separated in time. Thus if P(x) is to have meaning in a box, one must conclude that the probabilities associated with p and -p are not constant, but rather functions of x. This then leads to the notion of a complex new kind of probability (which maps to P(x)=constant) for a free particle i.e. W(x) =cos(px) + i sin(px) so that the modulus W*(x)W(x)=P(x)= 1. These ideas seem to go beyond the notion of Fisher information. What then is the role of Fisher information? We argue it is related to two very specific moments of the distribution cos(px) or rather W(x) = Sum over p a(p)cos(px) with a(p)=a(-p), namely and <pp>. In classical physics there exists p and pp at each x point (but one is the square of the other), but with distributions <p does not equal to <pp> in general. Both and <pp> are linked to physical concepts which exist in Newtonian mechanics, namely impulse and work. With the presence of cos(px) one has a combination of Newtonian mechanics and a statistical distribution. We argue that the role of Fisher’s information is to allow one to transform from to <pp>. In particular, -idW/dx appears in Fisher’s information (and is linked to ), but the specific role of a variation is to transform it into <pp> which appears in a conservation of energy equation. We argue that this is why Fisher’s information may be used to derive the Schrodinger equation.