In (1) it is suggested that Shannon’s entropy - Sum over i P(i) ln(P(i)) be thought of as the average of lost information defined by ln(P(i)). To see why the function ln appears, it seems one needs to consider a product of probabilities: P=P(i)P(j). It is possible that another two probabilities k,l yield the same product probability P(k)P(l)=P. In such a case, given P one would not know which two probabilities created it. For a specific instance, however, one specifically knows P(i) and P(j) and so “information” so to speak is lost through the product. To describe information as a quantity which “adds” one may take ln i.e. ln(P)=ln(P(i)) + ln(P(j)) = ln(P(k)) + ln(P(l)) with ln(P(i)) being information. The above is established for only two probabilities, but a P(i)P(j) result suggests two independent “coin tosses” so to speak or roll of a die. As a result, there seems to be a generalization in the literature to N successive independent events with N being large or a series of arrangements of results making use of Stirling’s approximation. Stirling’s approximation depends on large N and so this approach suggests that Shannon’s entropy holds for a large number of events. We have argued above that if considered as a loss of entropy (1), one only needs a product of two events. It is possible that one does not even need two successive events such as the toss of a coin or a die. For a quantum bound state, a product of probabilities P(x)P(p/x) appears if one wishes to know the probability for a particle to be at x and have p. In general this is not allowable in quantum mechanics so P(p/x) is complex. It is possible to have different (x,p) pairs yield the same P(x)P(p/x) product so lost information would be: ln{ WW a(p)exp(ipx)/W) yielding an entropy of S=Sp+Sx where Sp and Sx are Shannon’s entropy for p and x. Thus the idea of successive events or arrangements is not needed, rather entropy deals with a loss of information through a product of probabilities as argued in (1). One may also have a loss of information through a sum of probabilities, but in such a case ln does not isolate the pieces of lost information. In physical systems, however, it seems that products of probabilities appear especially due to two body scattering or to an energy level absorbing a photon. The idea of sum then appears through a conserved quantity such as energy or momentum which dictates the form of P(i).
In (1) it is suggested that Shannon’s entropy - Sum over i P(i) ln(P(i)) be thought of as the average of lost information defined by ln(P(i)). To see why the function ln appears, it seems one needs to consider a product of probabilities: P=P(i)P(j). It is possible that another two probabilities k,l yield the same product probability P(k)P(l)=P. In such a case, given P one would not know which two probabilities created it. For a specific instance, however, one specifically knows P(i) and P(j) and so “information” so to speak is lost through the product. To describe information as a quantity which “adds” one may take ln i.e. ln(P)=ln(P(i)) + ln(P(j)) = ln(P(k)) + ln(P(l)) with ln(P(i)) being information. The above is established for only two probabilities, but a P(i)P(j) result suggests two independent “coin tosses” so to speak or roll of a die. As a result, there seems to be a generalization in the literature to N successive independent events with N being large or a series of arrangements of results making use of Stirling’s approximation. Stirling’s approximation depends on large N and so this approach suggests that Shannon’s entropy holds for a large number of events. We have argued above that if considered as a loss of entropy (1), one only needs a product of two events. It is possible that one does not even need two successive events such as the toss of a coin or a die. For a quantum bound state, a product of probabilities P(x)P(p/x) appears if one wishes to know the probability for a particle to be at x and have p. In general this is not allowable in quantum mechanics so P(p/x) is complex. It is possible to have different (x,p) pairs yield the same P(x)P(p/x) product so lost information would be: ln{ WW a(p)exp(ipx)/W) yielding an entropy of S=Sp+Sx where Sp and Sx are Shannon’s entropy for p and x. Thus the idea of successive events or arrangements is not needed, rather entropy deals with a loss of information through a product of probabilities as argued in (1). One may also have a loss of information through a sum of probabilities, but in such a case ln does not isolate the pieces of lost information. In physical systems, however, it seems that products of probabilities appear especially due to two body scattering or to an energy level absorbing a photon. The idea of sum then appears through a conserved quantity such as energy or momentum which dictates the form of P(i).
Shannon’s entropy: -k Sum over i P(i) ln(P(i)) is expressed in terms of P(i) the probability for state i. If entropy exists as a concept separated from the first law of thermodynamics with dWork=0, then it is not clear what set of states i should be taken in quantum mechanics. In classical statistical mechanics, each particle is automatically described by its position and momentum vectors for no internal motion. For example a pure energy eigenstate wavefunction Wn(x) may be written in terms of many different orthonormal mathematical basis sets e.g. Wn(x)= Sum over i bi phi(i)(x) with probabilities bi*bi. Thus one may create many types of entropy expressions using Shannon’s form without any other physical considerations. For example, W(x)=Sum over p a(p)exp(ipx) and one may create Sp which is usually done in the literature. In previous notes we have argued that Sn = Sp(n)+Sx(n). For a particle in a box with infinite potential walls, Sx(n)=Sx. The sum of the two entropies ensures that the parameter L (box size) vanishes so that a quantum adiabatic transformation is isentropic. Thus a consideration other than Shannon’s form has been used to obtain Sn=Sp(n)+Sx for a particle in a box. Furthermore one may argue Shannon’s form even for a pure state based on S=S1+S2 where 1 and 2 represent two systems. A question still remains: Why are Sx and Sp entropies used and not some other set? Furthermore for a system with several energy levels ei one might ask why -Sum over i P(ei)ln(P(ei)) is used as in (1)? In a previous note (2) we considered that Shannon’s form of entropy follows from dE= TdS (dWork=0) with dE=Sum over i ei dP(ei). Furthermore we argued that P(ei) = C(T)exp(-ei/T) is a solution, but so is C(T) exp(-ei/T) exp(Si) where Si=Sx,i + Sp,i for the pure bound state. In other words, specific states related to the physical situation are chosen and entropy constructed from dE=TdS to reflect this. In this note we argue that entropy may be less related to counting arrangements than to considering the issues involved in energy transfers within a system.
The notion of entropy originates historically in classical physics and is intertwined with thermodynamics. Shannon’s entropy: - Sum over i P(i) ln(P(i)), where P(i) is the probability for i, however, seems to apply to any probability, but in quantum mechanics appears to be particularly linked to spatial and momentum entropy (1) i.e. - Integral dx W*(x)W(x) ln[ W*(x)W(x)] and - Sum over p a*(p)a(p) ln[ a*(p)a(p) ] where W(x)= wavefunction = Sum over p a(p) exp(ipx). In this note we speculate that one may create an entropy from the real part of exp(ipx) i.e. cos(px) as this term is associated with probability within a wavelength even though the “center of mass” of the particle may be moving uniformly at v=x/t=constant and W*(x)W(x)=1 or a constant. The latter implies an entropy independent of p but there is uncertainty in a free particle due to its wavelength which is experimentally manifested. We argue there is no reason to limit entropy to x or p considerations. For a bound state, a pattern of crests and troughs of different lengths and heights may be produced and so again there is a pattern of unnormalized probability distributions with different weights. The absolute value may be taken so negative wavefunction values are not an issue. For both the free particle and the bound state two features emerge. First the shape of the distributions is linked to the curvature which in turn is associated with the kinetic energy. Thus the probability distributions which describe the quantum system are governed by kinetic energy. Secondly, a pattern of distributions, both negative and positive, ensures that exp(ipx) is perpendicular to other exp(ip1 x) functions and Wn(x) is perpendicular to other Wn1(x)’s. In other words energy identifies the statistical state which exists throughout space and one cannot confuse the identity of one state with another. Thus they are orthogonal throughout space and probability distributions (linked to entropy) seem to enforce this. The inner product of W*(x) and W(x) is like an inner product of a product probability of a state and its time reversed state. We suggest one should be able to define a Shannon’s entropy expression for both |cos(px)|/C1 from exp(ipx) over a half wavelength where C1 is a normalization factor and |W(x)|/C2 roughly over the length over which it has humps i.e. roughly the classical turning points for an oscillator for example. This entropy is not additive and does not map to classical entropy in the large n (energy level) limit. Furthermore it may not be linked to thermodynamics because it is not linked to x or p. Nevertheless, it might be used to describe uncertainty within the quantity system and equilibrium/stable scenarios which are observed in nature. For example, different Wn(x) states are statistically stable states, but one may knock out the particle and it will be in an exp(ipx) state which is also stable. Furthermore, a photon may excite Wn1(x) to Wn2(x), thus this new entropy seems to be linked to stable states allowed in nature. The “price to pay”, however, seems to be uncertainty in wavelength regions for a free particle and the crest/trough regions for a bound state. We also consider some properties of a similar entropy for bound states.
In a previous note (1) we suggested using P(x)P(p/x) where P(x)=W(x)W(x) and P(p/x) = a(p)exp(ipx)/W(x) for a bound state (where W(x) is the wavefunction) in Shannon’s entropy expression - Sum over i P(i) ln(P(i)). This yields a bound state entropy of S= .5Sp+ .5Sx. Historically (2) Sp+Sx, called entropic uncertainty, has been of interest as an alternative to the Heisenberg uncertainty relationships for p and x. In this note we argue that this sum of momentum and spatial entropies is directly linked to the idea of mutual information from information theory. In (3) it is noted that mutual information is “the amount of information that can be obtained about one random variable by observing another”. Thus this seems to be directly related to uncertainty in two p and x.
Shannon’s entropy is calculated using probabilities P(i) i.e. S= - Sum over i P(i) ln(P(i)). A probability is related to an uncertainty associated with event i, but this event may not be a physical event, but rather a measurement. In other words, measurements in a system may be made according to one distribution or another. This distribution may be converted into something which looks like a probability even though the system is completely deterministic. The entropy then calculated is based on the measurement distribution and not the physical system. As a first example, consider the completely deterministic motion of a bound classical system between two turning points. Person A may send values of x (one dimensional space) to person B, based on a certain distribution instead of sending the values based on time i.e. x(t1),x(t2) etc. One distribution which may be used is the uniform one i.e. P(x)= dx/L. Another distribution, used in the literature (1), is based on the time spent in each constant dx i.e. P(x) = Cdx/v(x) where v(x) is velocity. These two distributions lead to different Shannon’s entropies, but the underlying system is completely deterministic which begs the question: Why does one associate an entropy with it? The entropy seems to be associated with the sending of information. We try to apply these ideas to a quantum bound state. In the literature (2), one sometimes sees a Shannon’s entropy based on W(x)W(x) = P(x), where W(x) is the single particle bound wavefunction. This is a distribution, but is it based on uncertainty in the system? W(x)W(x) often consists of a set of crests of different heights and widths. We argue that the inherent uncertainty in x is due to the width of a crest. In other words each crest may be normalized to 1 and a Shannon’s entropy for each crest calculated. One must then weight the different crests. If one calculates the classical time associated with the base length of crest, this could be used as a weight which in the high energy level limit is associated with P(x)=Cdx/v(x) from (1), but it seems there is no reason to base the weight on time. The uncertainty in x is due to a superposition of exp(ipx)s which are not linked to time. Alternatively, one may use an unnormalized weight of the length of the base as uncertainty is only in x. One may then add a Shannon’s momentum entropy using a(p)a(p)=P(p) where W(x)=Sum over p a(p)exp(ipx).
For a classical ideal gas or single particle quantum system with energy levels {ei}, both average energy E=Sum over i ei P(ei/T) and Shannon’s entropy S= - Sum over i P(ei/T) ln(P(ei/T) change (for mechanical parameters kept fixed) in terms of d(P(ei/T)). In fact, in this case ln(P(ei/T)) appears as a “constant vector” in dS i.e. dS= - Sum over i ln(P(ei/T)) dP(ei/T). Both dE (mechanical parameter fixed) and dS change as dP(ei/T), thus there are two vectors {ei/T} and {ln(P(ei/T))} which represent constants over ei’s for a small range of dT where T is temperature. In this case, however, the two vectors are strongly linked as P(ei/T)=exp(-ei/T)/C(T) so ln(P) = -ei/T - ln(C). This is why the relationship E-TS = -T ln(C(T)) exists. For a quantum bound state one may write: e = Integral dp a(p)a(p) pp/2m + Integral dx V(x) W(x)W(x) where W(x)=Sum over p a(p)exp(ipx) is the wavefunction. Instead of having vectors of ei’s, one has two vectors, one over x and the other p (momenta) where a(p)a(p) = P(p) and P(x)=W(x)W(x) (bound states). Thus d e thermal = Integral dp pp/2m dPe(p) + Integral dx V(x) dPe(x) ((1)). In other words de thermal changes linearly in terms of dPe(p) and dPe(x) whereas for a classical ideal gas or many ei system it changes linearly in terms of dP(ei/T). Ei is the variable instead of x and p in that case. In other words {pp/2m} is a constant vector in p space and {V(x)} in x space. Mathematically Shannon’s entropy Sp = - Sum over p Pe(p) ln(Pe(p)) and Sx =- Sum over x Pe(x) ln(Pe(x)) also yield constant vectors in their respective spaces namely ln(Pe(p)) and ln(Pe(x)) which are called information. In particular dSp = - Sum over p ln(Pe(p)) dPe(p) then ln(Pe(p)) behaves like a constant vector within a region of small dPe(p) changes. Note: Sum over p dPe(p)=0. One may consider a set of single valued energy levels ei which are very close (imagine a particle in a box with infinite walls) as a region in which dPe(p) changes (or dPe(x) ones) are very small. In this region, ln(Pe(p)) and ln(Pe(x)), information, represents a constant vector statistically defining the small region of ei states. This we argue is the statistical significance of information ln(P). One may note that both de ((1)) adn dSp and dSx change linearly with dPe(p) and dPe(x), but the vectors {pp/2m} and {V(x)} over p and x space and ln(Pe(p)) and ln(Pe(x)) over the same space are constant vectors, but not equivalent except for the ground state of the quantum oscillator for which they become very similar because Pe(p)=exp(-app) and Pe(x)=exp(-xx/a) with a being proportional to 1/ e which plays the same role as temperature in the classical ideal gas case. For other quantum bound systems, the two sets of vectors are different, but the information vectors seem to give a kind of statistical identity to small regions containing very close ei levels for which dPe(p) and dPe(x) are small. Thus, though one may argue for the presence of ln(P) in Shannon’s entropy S=-Sum over i P(i)ln(P(i)) in many ways, it seems that ln(P(i)) is a constant vector in a certain space i for small dP(i) and gives a statistical identity to that small region.
In this note we wish to examine three ideas. First we consider the case of W(x,t)=Sum over n an exp(-iEn t) Wn(x) ((1)) (for n=0,1 as an example) and note that the von Neuman entropy is 0 for such a state. This is pointed out in (1) together with the fact that there exists a von Neumann density operator exp(-H/T) (H=Hamiltonian operator) which yields the same result as Shannon’s entropy S using P(ei) if one uses dQ/T=Sum over n en dP(en)/T = dS. It seems then that there is a contradiction if the usual von Neumann entropy is 0 yet using a density operator = exp(-H/T) yields nonzero results. A resolution is suggested in (2) which deals with information theory and we see if it applies to ((1)). Secondly, we consider the Shannon’s form P(e)ln(P(e)) in more detail. In Part I and in previous notes we argued that ln ensures that S=S1+S2 if one has two systems. We also argued that dE= Sum over n en dP(en) = TdS if dWork =0. In this note, we argue that a stronger idea may be at work, namely the idea that Eave= Sum over n en P(en/T) and TS are essentially the same function except for a function of T. This approach may be used to show that ln is the function associated with entropy. Finally, we note as in Part I that when using either the von Neumann approach or Shannon’s entropy one must know which probabilities to use. For a system with energy levels en, P(n)=P(en/T) is an immediate candidate, but there may be more probabilities in the system related to a particle jumping from one ei to another. In such a case, if one uses these probabilities and then sums over some of them leaving only P(en/T), the form of entropy changes from that of Shannon’s form. (Originally, however, Shannon’s form is used before summing.)