Information theory, which applies to compressing messages, suggests that ln(probability) is information. Shannon’s entropy is -1 times the average information i.e. -Sum over i P(i)ln(P(i)). For a classical gas in the canonical approximation, this is of the form of an average conserved quantity e.g Eave= Sum over i ei P(ei). In deterministic classical mechanics one may characterize an elastic two body collision by e1+e2 = e3 + e4. In a Maxwell-Boltzmann (MB) gas these same deterministic reactions occur, but there is also a static probability P(ei). The question seems to be whether P(ei) brings any new “physical” information into a scattering reaction. One already knows that a particle contributes energy ei. Is there something else brought in? If the answer is no, then using the product of probabilities P(e1)P(e2) and taking ln leads to ln(P(e1)) + ln(P(e2)) suggests that ln(P(e1))= A (e1-B) where A and B are constant. Thus information considerations link probability P(ei) to a deterministic quantity ei. One does not have to consider reaction balance or the maximization of entropy subject to a constraint. One simply finds the appropriate probability Pappropriate(e1) which describes an overall change to the state e1. For the MB case, P(appropriate)(ei) = P(ei) (MB) which gives a static description of the number of particles with ei. In other cases, however, there may be bias. Consider quantum scattering of fermions or bosons. If the collisions are elastic, one again has the classical deterministic e1+e2=e3+e4 expression. There is also a static P(ei) (Fermi-Dirac) for fermions and P(ei) Bose-Einstein for bosons. One is, however, interested in collisions which create and maintain the equilibrium situation. If there is bias in these collisions, then Pappropriate(ei) may differ from P(ei). The information e1 is the full contribution of particle 1 to the collision. Thus one also needs a full probability describing probability movement out of state e1. P(e1) is proportional only to the probability flow out of e1 due to scattering, but there is a “bias” factor namely 1-P(e1) which prevents another particle from entering state e1. This biases a reaction. As a result Pappropriate(e1) = P(e1)/ (1-P(e1)) for a fermion. This may be seen from a reaction balance equation:P(e1)(1-P(e3)) P(e2)(1-P(e4)) = P(e3)(1-P(e1))P(e4)(1-P(e2)) for fermions. For bosons there is an enhancement. One may obtain product probabilities by forming P(ei)/(1-P(ei)). This is the full contribution in terms of probabilities of particle i to a collision and so its ln should equal A(e-B)i which is its full information contribution to the reaction. If the two are not equal, it means there is more information (excluding parameters such as T temperature and u the chemical potential) than ei in the problem.
Information theory, which applies to compressing messages, suggests that ln(probability) is information. Shannon’s entropy is -1 times the average information i.e. -Sum over i P(i)ln(P(i)). For a classical gas in the canonical approximation, this is of the form of an average conserved quantity e.g Eave= Sum over i ei P(ei). In deterministic classical mechanics one may characterize an elastic two body collision by e1+e2 = e3 + e4. In a Maxwell-Boltzmann (MB) gas these same deterministic reactions occur, but there is also a static probability P(ei). The question seems to be whether P(ei) brings any new “physical” information into a scattering reaction. One already knows that a particle contributes energy ei. Is there something else brought in? If the answer is no, then using the product of probabilities P(e1)P(e2) and taking ln leads to ln(P(e1)) + ln(P(e2)) suggests that ln(P(e1))= A (e1-B) where A and B are constant. Thus information considerations link probability P(ei) to a deterministic quantity ei. One does not have to consider reaction balance or the maximization of entropy subject to a constraint. One simply finds the appropriate probability Pappropriate(e1) which describes an overall change to the state e1. For the MB case, P(appropriate)(ei) = P(ei) (MB) which gives a static description of the number of particles with ei. In other cases, however, there may be bias. Consider quantum scattering of fermions or bosons. If the collisions are elastic, one again has the classical deterministic e1+e2=e3+e4 expression. There is also a static P(ei) (Fermi-Dirac) for fermions and P(ei) Bose-Einstein for bosons. One is, however, interested in collisions which create and maintain the equilibrium situation. If there is bias in these collisions, then Pappropriate(ei) may differ from P(ei). The information e1 is the full contribution of particle 1 to the collision. Thus one also needs a full probability describing probability movement out of state e1. P(e1) is proportional only to the probability flow out of e1 due to scattering, but there is a “bias” factor namely 1-P(e1) which prevents another particle from entering state e1. This biases a reaction. As a result Pappropriate(e1) = P(e1)/ (1-P(e1)) for a fermion. This may be seen from a reaction balance equation:P(e1)(1-P(e3)) P(e2)(1-P(e4)) = P(e3)(1-P(e1))P(e4)(1-P(e2)) for fermions. For bosons there is an enhancement. One may obtain product probabilities by forming P(ei)/(1-P(ei)). This is the full contribution in terms of probabilities of particle i to a collision and so its ln should equal A(e-B)i which is its full information contribution to the reaction. If the two are not equal, it means there is more information (excluding parameters such as T temperature and u the chemical potential) than ei in the problem.
We have argued in previous notes that C(T) exp(-ei/T) (the Maxwell-Boltzmann probability) and exp(-iEt) exp(ipx) (the free particle quantum probability called a wavefunction) have the same informational structure in that product probabilities for a given E1=Ei+Ej for the MB case and Ei,pi and Ej, pj for the quantum case lead to uniform distributions, i.e. ones with minimal information if energy and momentum are conserved. We suggest here that the root for this may be same. We have argued in previous notes that dp/dt = Force cannot have dt be infinitesimally small. This means that there exists a scale at which dt is not 0 or even tiny. We also suggested that the force (even though obeying action-reaction in collisions) may strike at some probabilistic time within dt. We then used special relativity to find that dt =hbar/E and dp=hbar/p. A probability which demonstrates these gradations is exp(-iEt+ipx) (and also Lorentz invariant). This is akin to the notion that p(e1)p(e2)=p(e3)p(e4) and p(p1)p(p2) = p(p3)p(p4) (one dimension for simplicity). In other words, the stochasticity introduced is one of a uniform distribution for product probabilities. One could obtain the same result by starting with this assumption and then suggesting Lorentz invariance. We suggest that this quantum uniform distribution product probability also holds for the Maxwell-Boltzmann case. There, the probability is not complex because p(ei) must be a real number linked to the number of particles with ei. Nevertheless, in an elastic collision of ei and ej, we suggest that the quantum result (and even dp/dt=F) suggest that there is stochasticity in x and t and that one cannot deterministically predict the outcome. Any outcome which conserves energy and momentum has the same likelihood. We suggest that it is the quantum mechanical assumption which is identical to the ideal gas case (which uses quantum mechanical gas molecules albeit at high temperatures). The high temperatures mean that one can only use the conservation laws and the associated stochasticity. We suggest that the stochasticity of interaction inherent in quantum mechanics carries over to collisions in the classical gas. It is not simply that one is approximating the situation in a gas by throwing away information. If one were to try to be as deterministic as possible at some scale (the quantum one) stochasticity arises and there is no full determinism in a collision. The best one can do is use conservation of momentum and sometimes conservation of energy in certain situations like high temperature. If this is the case, one may ask: Why is it possible to perform deterministic classical calculations such as particles bouncing off walls or mirrors or interacting with a potential V(x) in a completely deterministic manner. We suggest that these involve constrained interactions, i.e. the wall or V(x) is associated with a huge mass compared to the particle, unlike in a collision of two identical particles. Action-reaction in the former case is associated with the large mass and so always the same approximately 0 kinetic energy and hence no p(ei)p(ej) =p(ek)p(el), but in the latter, with a mass the same size and so the stochasticity may be seen clearly.
In information theory, ln(probability(i)) equals information. The approach of maximization of Shannon’s entropy - P(ei) ln(P(ei)) subject to a constraint ei P(ei) yields: ln(P(ei)) = -ei/T where ei is a conserved piece of information associated with an interaction and -1/T, a Lagrange multiplier. In the case of tossing a die or coin, the constraint is P(i), so the Lagrange multiplier is not important, but P(ei) is the conserved quantity and is associated with interactions i.e. trial runs. In the case of a quantum free particle, one may ask: Where is the probability and where are the trial runs or interactions? A free particle, classically, has an energy and momentum as well as a time and x associated with v=constant=x/t. There do not seem to be any reactions so it is not clear what constraint exists, nor is it clear that there is any probability present as E,p and v=x/t are deterministic. In this note, we speculate on the relevant information and the presence of probability for a free particle. Such a particle does not seem to be probabilistic at first glance but, we argue that its intrinsic probability is not defined through interactions (like the classical statistical gas), but rather is inherent due to the various different ways it may have been formed. We argue that it creates other visible probabilities that are not based on maximization of entropy subject to a constraint.
In information theory, ln(Probability) is called information. We argue that in various physical problems this information is linked with a conserved quantity and a variable which changes. For example, in the Maxwell-Boltzmann case the conserved quantity is 1/Temperature and the variable e+V(x) = energy. For a quantum free particle, the conserved quantity is momentum and the variable x. “e” may change subject to the conserved quantity 1/T and x may change subject to the conserved quantity p. Given that probability may be unnormalized, changes may be expressed in relative terms i.e. in terms of a derivative of ln(P) i.e. information. We argue that the derivative of ln(P) with respect to the variable which changes yields the value of the conserved quantity in special cases for which the conserved quantity is the only information governing changes in the probability distribution. Thus one has an eigenvalue equation: d/dvariable ln(P) = conserved quantity. This implies that ln(P) = variable * conserved quantity. This may lead to two different kinds of conservation equations. In the MB case, e is the variable and 1/T the conserved quantity, but ln(P(e1)) + ln(P(e2)) = 1/T (e1+e2) = ln(P(e1+e2)). An eigenvalue equation allows for this conservation. In the quantum free particle case: ln(P(k1,x)) + ln(P(k2,x)) where P=wavefunction = exp(ipx) (i.e. the square-root type probability is used), one has k1+k2=k3 with x the variable canceling out. (In the MB case, it was 1/T the conserved quantity canceling). Again, the eigenvalue equation allows for this conservation, this time of the conserved quantity. We note that the eigenvalue equation is a special case and not only probability distributions are associated with it. For example, a power law distribution’s changes in its variable do not depend only on the conserved quantity 1/T in P(e/T)=power law.
In (1) it is suggested that Shannon’s entropy - Sum over i P(i) ln(P(i)) be thought of as the average of lost information defined by ln(P(i)). To see why the function ln appears, it seems one needs to consider a product of probabilities: P=P(i)P(j). It is possible that another two probabilities k,l yield the same product probability P(k)P(l)=P. In such a case, given P one would not know which two probabilities created it. For a specific instance, however, one specifically knows P(i) and P(j) and so “information” so to speak is lost through the product. To describe information as a quantity which “adds” one may take ln i.e. ln(P)=ln(P(i)) + ln(P(j)) = ln(P(k)) + ln(P(l)) with ln(P(i)) being information. The above is established for only two probabilities, but a P(i)P(j) result suggests two independent “coin tosses” so to speak or roll of a die. As a result, there seems to be a generalization in the literature to N successive independent events with N being large or a series of arrangements of results making use of Stirling’s approximation. Stirling’s approximation depends on large N and so this approach suggests that Shannon’s entropy holds for a large number of events. We have argued above that if considered as a loss of entropy (1), one only needs a product of two events. It is possible that one does not even need two successive events such as the toss of a coin or a die. For a quantum bound state, a product of probabilities P(x)P(p/x) appears if one wishes to know the probability for a particle to be at x and have p. In general this is not allowable in quantum mechanics so P(p/x) is complex. It is possible to have different (x,p) pairs yield the same P(x)P(p/x) product so lost information would be: ln{ WW a(p)exp(ipx)/W) yielding an entropy of S=Sp+Sx where Sp and Sx are Shannon’s entropy for p and x. Thus the idea of successive events or arrangements is not needed, rather entropy deals with a loss of information through a product of probabilities as argued in (1). One may also have a loss of information through a sum of probabilities, but in such a case ln does not isolate the pieces of lost information. In physical systems, however, it seems that products of probabilities appear especially due to two body scattering or to an energy level absorbing a photon. The idea of sum then appears through a conserved quantity such as energy or momentum which dictates the form of P(i).
If one tosses a die or coin, different outcomes occur with a probability p(i) where i is the outcome. This probability may be measured by performing trial runs. For N trial runs, where N is very large, one might expect Np(outcome i) results of an outcome i. In other words, p(outcome i) is both the probability and describes the trial run. Even in the Maxwell-Boltzmann case, one has a probability p(ei) where ei is kinetic energy (no V(x)) and reaction balance p(e1)p(e2) = p(e3)p(e4) represents trial runs. Thus the trial run is again characterized by p(ei) as ei’s contribution. It is, however, possible to have coupling of probabilities in certain physical situations even though p(outcome i) still exists. In such a case, the number of outcome i results for N large trials is not N p(outcome i). In other words, p(i) does not fully describe the trial run. There may be other physical rules in place. If elastic scattering of particles is the way trial runs occur, e1+e2 = e3 + e4 represents the salient information in the problem and even if there is nonindependence of p(outcome i) in the scattering, one should at least obtain a function Q(p(e1)) such that Q(p(e1)) = -e1/T. Q(p(ei)), however, need not be p(ei). We consider the physical rule given by Pauli stating that only one particle may fit in an energy level (no spin). Thus there is either 0 or 1 particle in the level. This should inhibit scattering, so the trial run is no longer simply described by p(ei). Rather the probability to scatter into level ej is (1-n(ej)) and so one has: n(e1)[1-n(e3)] n(e2)[1-n(e4)]= n(e3)[1-n(e1)]n(e4)[1-n(e2)] where n(ej) is the average occupancy, a number between 0 and 1. The opposite of only 0 or 1 particle in a state ei, is to have any possible number of particles in this state and an enhancement, not a hindrance given by 1+p(ej). This is only a guess, but experiment shows that bosons follow this rule. Given a definition of trial runs in terms of the inhibition and enhancement factors, one may do two things. First, since the trial rule seems to involve different numbers of particles in a state associated with a p(ei), one may consider sums of p(ei) (power number of particles) when describing the system statistically (i.e. grand canonical scheme). This approach gives rise to linear terms (a+bp(ei)) which may in turn be formulated in terms of a factorial/arrangement equation which may be maximized subject to the constraints Sum over i ei n(ei) and Sum over i n(ei). We note that this maximization is not simply in terms of all degrees of freedom in the system, but is specifically linked with the rules associated with the trial runs and the constraints. We note, however, that issues seem to arise with the factorial scheme in the boson case. Thus, overall we argue that a single particle probability and its trial run scheme are key to establishing the statistical behaviour of a system. (We also note that single particle independent probability, which is an approximation, also affects the final results.)
Given an unbiased die, the probability is ⅙ for a result. This probability value may be obtained directly by imposing the condition of a lack of bias and noting there are six possible results. Alternatively, one may maximize Shannon’s entropy subject to the constraint Sum over i p(i)=1 where i runs from 1 to 6. Maximization of Shannon’s entropy seems to remove as much bias as possible with the constraint indicating what bias remains. For the die case the constraint has no bias, but for an average energy constraint Sum over i ei p(i) there is, namely ei. In previous notes we have argued that the constraint used in the maximization of entropy is “special”. Here we argue that it represents a conserved quantity if interactions occur. Thus there may be many constraints in a system, but choosing a conserved quantity in an interaction is rarer, but seems to be linked with equilibration. Finally, we suggest that maximization of entropy subject to a conserved quantity constraint removes as much bias as possible and it is this maximized entropy-probability which is subject to the second law of thermodynamics. It is possible, however, to have probabilities and to create entropies which are not based on maximization of entropy. For example, Shannon’s entropy may be applied to the spatial density of various n-slit interactions for a particle with constant momentum p. In this case, even though there is no maximization of entropy, the emerging particle has the same p magnitude (hence same uncertainty within a wavelength), but there is the added uncertainty of its direction. As a result, it seems that entropy increases in accordance with the second law. A free particle with momentum p, however, has a two dimensional probability distribution in space exp(ipx) characterized by a wavelength hbar/p. This may be taken as related to the entropy of the free particle, but a free particle may undergo spontaneous interactions which either increase or decrease p, hence increase or decrease hbar/p and entropy. In other words, the second law does not apply to such an entropy, but this entropy is associated with an unusual constraint. This conclusion seems to hold when one considers two body scattering. Thus we suggest that there are various kinds of entropy. Some are linked with maximization of entropy subject to a conserved quantity constraint (and are associated with equilibration) and others are not. The latter do not necessarily obey the second law of thermodynamics, we argue.