An interesting result arises in information theory, namely that if one considers a microstate with large N trials, each representing a state i with probability p(i), then even though this is only one of many possible microstates (each being a permutation of each other) it carries information about all of the permutations based on the average occupancy number. In particular, the probability of a microstate may be written as: Product over i p(ei) [power Np(ei) ] for large N. This holds for any microstate and so all have the same probability. Given that all of the microstates are permutations in the large N case, this probability is equivalent to 1/ {number of permutations} where Number of permutations = N! / { Product over i n(ei) }. We point out the above example, because it seems there are at least two main strategies in statistical mechanics. One is to begin with a set of average occupancy numbers n(ei/T) whose specific form is unknown. The idea then seems to be to seek degeneracy in the system, possibly among the arrangement of the n(ei)’s or degeneracy within single levels ei and to maximize the number of degenerate arrangements in order to account for as little information as possible. This approach may be used to calculate the Maxwell-Boltzmann, Fermi-Dirac and Bose-Einstein distributions. In the case of the latter two, degeneracy in the single energy levels ei is key. One then maximizes the ln of the degenerate arrangements subject to constraints such as Sum over i ei n(ei) and Sum over i n(ei) = N. The degeneracies themselves are associated with momentum degeneracies yielding the same energy. A completely different approach, it seems, is to consider exp(-(E-uN)/T) as representing the probability of a state with energy E and N particles and to use either the grand canonical ensemble i.e. sum the various exp(-(E-uN)/T)’s. These may then be broken into single particle sums using E(N) = sum over i ei n(ei) for n(ei) variable or sum over i ei for N fixed. Even though momentum degeneracy still exists, it is not part of the n sum of exp(-(E-uN)/T) factors which gives a single energy level probability distribution with no maximization necessary. Thus we point out that exp(-(E-uN)/T) for microstates seems to incorporate information about degeneracies without them needing to be explicitly added. On the other hand, if one starts with average n(ei) (without knowing their form), one must explicitly consider degeneracies in order to maximize so that there is no unnecessary information in the system. Both approaches should yield the same result. We consider the Bose-Einstein example of a photon gas as an example.
An interesting result arises in information theory, namely that if one considers a microstate with large N trials, each representing a state i with probability p(i), then even though this is only one of many possible microstates (each being a permutation of each other) it carries information about all of the permutations based on the average occupancy number. In particular, the probability of a microstate may be written as: Product over i p(ei) [power Np(ei) ] for large N. This holds for any microstate and so all have the same probability. Given that all of the microstates are permutations in the large N case, this probability is equivalent to 1/ {number of permutations} where Number of permutations = N! / { Product over i n(ei) }. We point out the above example, because it seems there are at least two main strategies in statistical mechanics. One is to begin with a set of average occupancy numbers n(ei/T) whose specific form is unknown. The idea then seems to be to seek degeneracy in the system, possibly among the arrangement of the n(ei)’s or degeneracy within single levels ei and to maximize the number of degenerate arrangements in order to account for as little information as possible. This approach may be used to calculate the Maxwell-Boltzmann, Fermi-Dirac and Bose-Einstein distributions. In the case of the latter two, degeneracy in the single energy levels ei is key. One then maximizes the ln of the degenerate arrangements subject to constraints such as Sum over i ei n(ei) and Sum over i n(ei) = N. The degeneracies themselves are associated with momentum degeneracies yielding the same energy. A completely different approach, it seems, is to consider exp(-(E-uN)/T) as representing the probability of a state with energy E and N particles and to use either the grand canonical ensemble i.e. sum the various exp(-(E-uN)/T)’s. These may then be broken into single particle sums using E(N) = sum over i ei n(ei) for n(ei) variable or sum over i ei for N fixed. Even though momentum degeneracy still exists, it is not part of the n sum of exp(-(E-uN)/T) factors which gives a single energy level probability distribution with no maximization necessary. Thus we point out that exp(-(E-uN)/T) for microstates seems to incorporate information about degeneracies without them needing to be explicitly added. On the other hand, if one starts with average n(ei) (without knowing their form), one must explicitly consider degeneracies in order to maximize so that there is no unnecessary information in the system. Both approaches should yield the same result. We consider the Bose-Einstein example of a photon gas as an example.
The Maxwell-Boltzmann (MB), Fermi-Dirac FD and Bose-Einstein BE distributions may be obtained by maximizing the ln of energy level degeneracies subject to the constraint of Sum over i ei ni = Eave in the MB case (canonical distribution) and this constraint together with a second one Sum over i ni = Nave (grand canonical case) as shown in (1) page 182 . In particular, in the MB case, a single energy level ei with ni particles is said to have the degeneracy (gi) (power ni) / ni ! (which overcounts). This maximization process leads to the weight gi exp(-ei/T). Setting gi=1, however, leads to the same weight exp(-ei/T) so the MB factor is not a result of a single particle level degeneracy i.e. the direction in which the momentum vector points. To shed physical light on this situation, one may consider two particle scattering in the MB, FD and BE cases. In all three cases, e1+e2 = e3+e4 (with e=pp/2m), but only in the MB case does n(e1)n(e2)=n(e3)n(e4). This suggests that the single energy level ei degeneracy levels ei actually play a physical role in two body scattering for the FD and BE cases, but not in the MB case. Thus, calculating probabilities based on counting still involves the physics related to the interactions in the problem. Degeneracies may exist and may or may not need to be part of the counting of arrangements. As a specific example, we note that in (1) (gi) (power ni) / ni ! is used to count degenerate arrangements for the MB case (on page 182). This same expression is used in (1) on page 194 to describe a method by Fowler and Darwin to obtain the MB single particle probability exp(-ei/T). In that case, gi is taken to be a parameter which is ultimately set to 1 and not the degeneracy of the single particle level. This begs the question: When should single energy ei degeneracy be included in the counting of states?
Extremization of entropy, usually subject to an average energy constraint,(Maxwell-Boltzmann gas) is often associated with maximization of the number of microstates N gas particles may represent. The math procedure of extremization (variational calculus) is then associated with the ln of a factorial expression which reduces to n(ei)ln(n(ei)) using Stirling’s approximation. The central limit theorem, on the other hand, states that the distribution of sample means of a random variable approaches a Gaussian, i.e exp(-pp/(2sig sig)). Here sig sig is the standard deviation. For a random variable, the important or characteristic information is the standard deviation associated with the average of the square pp. We suggest here that an alternative way of looking at maximization of entropy (which is a stability statement we argue) other than ln of permutations is to consider the stability of the variation of the variance of pp subject to a change in the probability, with pp written in terms of P(p). Average kinetic energy in a statistical problem represents such a variance and should be invariant under changes of the probability subject to an average kinetic energy constraint, we suggest. How does this idea apply to quantum mechanics? In quantum mechanics, one has a statistical function exp(ipx) and for bound states creates W(x)=wavefunction = Sum over p a(p)exp(ipx). a(p)a(p) is formally the probability for p, but this does not mean that a(p)a(p) is a random variable because one has the constraint: Sum over p a(p)a(p) pp2/m + Integral dx V(x) W(x)W(x) = En. In the case of a harmonic oscillator, however, one knows a priori that the two summands are each equal to En/2. Furthermore there are a set of En values, starting at a ground state and increasing. All En values, except the ground state, are metastable. Thus the ground state is a special state. Given that Sum over p a(p)a(p)pp2/m = Eo/2, a priori, one may consider that all of the information of a(p)a(p) is associated with Eo. This is exactly the case for a Gaussian from the central limit theorem. Alternatively, from information theory, information = ln(probability) which yields the same result. If this is the case, then pp/2m a(p)a(p) written strictly in terms of P(p)=a(p)a(p) should be stationary upon changing P(p) subject to the E/2 constraint. This special situation, however, only holds for the ground state oscillator. What about average potential energy for the ground state oscillator? If the probability for p (momentum) is a Gaussian due to the Central Limit theorem, this means that the standard deviation is the information (and is linked with randomness). Taking a Fourier transform of a(p) cannot add more information to the picture, we argue. W(x)W(x) should also be a Gaussian, as it is. As a result, xxW(x)W(x), written in terms of P(x)=W(x)W(x) subject to the same Eo/2 constraint should represent a maximization or stationary situation, we argue.
The Maxwell-Boltzmann MB distribution is often derived by maximizing the number of possible arrangements of particles with different energies subject to the constraint of a fixed number of particles and fixed energy. Using the canonical approximation that only average energy is conserved and using Stirling’s approximation for large number factorials leads to the Maxwell-Boltzmann distribution. The ln of total number of arrangements using Stirling’s approximation is linked to a math form called Shannon’s entropy i.e. - Sum over i f(ei) ln(f(ei)). Interestingly this same MB result may be obtained through reaction balance which makes no use of the idea of arrangements or entropy or maximization. It is concerned with including all possible physical reactions (for example two body reactions between all energy particles) subject to conservation of energy. In this sense it deals directly with the dynamics of the problem while the arrangement approach does not. The dynamical approach considers all possible reactions with notions of independence or lack of information and ensures the system remains unchanged. Why are they the same? We argue that it is because the “recipe” for the probability of two body reactions namely n(e1)n(e2) is of the same form as the leading factor of n(e1)! n(e2)!. In the case of quantum mechanics one may apply the dynamical approach to obtain the time-independent Schrodinger equation. In such a case one does not need to think in terms of “maximizing” any quantity although one considers all possible reactions with a potential V(x). a(p) factors in W(x) = wavefunction = Sum over p a(p)exp(ipx) are interrelated, but retain their identity after all possible interactions occur. This we argue is the idea for the eigenvalue equation. At the same time, Shannon’s entropy, which is directly linked to the notion of arrangements, is applied to a bound state and possibly linked to thermodynamics i.e. dE = TdS -PdV which may be applied to an adiabatic expansion. Shannon’s entropy is created from existing probabilities in this case which do not arise from explicitly maximizing entropy subject to constraints as the probabilities follow directly from the Schrodinger equation. One might ask why Shannon’s entropy has any relevance if it is the dynamics which determines the value of W(x) and a(p). It seems, however, that one may knock out a particle in a state p with probability P(p)= a*(p)a(p) or find a particle at x with probability P(x)=W*(x)W(x). These are classical type probabilities linked to the quantum dynamical probabilities a(p) and W(x). It seems Shannon’s entropy is only applied to P(x) and P(p) or a product P(x)P(p) which may give it physical meaning in a quantum bound state. We examine these ideas in this note.
Shannon’s entropy: -k Sum over i P(i) ln(P(i)) is expressed in terms of P(i) the probability for state i. If entropy exists as a concept separated from the first law of thermodynamics with dWork=0, then it is not clear what set of states i should be taken in quantum mechanics. In classical statistical mechanics, each particle is automatically described by its position and momentum vectors for no internal motion. For example a pure energy eigenstate wavefunction Wn(x) may be written in terms of many different orthonormal mathematical basis sets e.g. Wn(x)= Sum over i bi phi(i)(x) with probabilities bi*bi. Thus one may create many types of entropy expressions using Shannon’s form without any other physical considerations. For example, W(x)=Sum over p a(p)exp(ipx) and one may create Sp which is usually done in the literature. In previous notes we have argued that Sn = Sp(n)+Sx(n). For a particle in a box with infinite potential walls, Sx(n)=Sx. The sum of the two entropies ensures that the parameter L (box size) vanishes so that a quantum adiabatic transformation is isentropic. Thus a consideration other than Shannon’s form has been used to obtain Sn=Sp(n)+Sx for a particle in a box. Furthermore one may argue Shannon’s form even for a pure state based on S=S1+S2 where 1 and 2 represent two systems. A question still remains: Why are Sx and Sp entropies used and not some other set? Furthermore for a system with several energy levels ei one might ask why -Sum over i P(ei)ln(P(ei)) is used as in (1)? In a previous note (2) we considered that Shannon’s form of entropy follows from dE= TdS (dWork=0) with dE=Sum over i ei dP(ei). Furthermore we argued that P(ei) = C(T)exp(-ei/T) is a solution, but so is C(T) exp(-ei/T) exp(Si) where Si=Sx,i + Sp,i for the pure bound state. In other words, specific states related to the physical situation are chosen and entropy constructed from dE=TdS to reflect this. In this note we argue that entropy may be less related to counting arrangements than to considering the issues involved in energy transfers within a system.
In (1) it is suggested that Shannon’s entropy - Sum over i P(i) ln(P(i)) be thought of as the average of lost information defined by ln(P(i)). To see why the function ln appears, it seems one needs to consider a product of probabilities: P=P(i)P(j). It is possible that another two probabilities k,l yield the same product probability P(k)P(l)=P. In such a case, given P one would not know which two probabilities created it. For a specific instance, however, one specifically knows P(i) and P(j) and so “information” so to speak is lost through the product. To describe information as a quantity which “adds” one may take ln i.e. ln(P)=ln(P(i)) + ln(P(j)) = ln(P(k)) + ln(P(l)) with ln(P(i)) being information. The above is established for only two probabilities, but a P(i)P(j) result suggests two independent “coin tosses” so to speak or roll of a die. As a result, there seems to be a generalization in the literature to N successive independent events with N being large or a series of arrangements of results making use of Stirling’s approximation. Stirling’s approximation depends on large N and so this approach suggests that Shannon’s entropy holds for a large number of events. We have argued above that if considered as a loss of entropy (1), one only needs a product of two events. It is possible that one does not even need two successive events such as the toss of a coin or a die. For a quantum bound state, a product of probabilities P(x)P(p/x) appears if one wishes to know the probability for a particle to be at x and have p. In general this is not allowable in quantum mechanics so P(p/x) is complex. It is possible to have different (x,p) pairs yield the same P(x)P(p/x) product so lost information would be: ln{ WW a(p)exp(ipx)/W) yielding an entropy of S=Sp+Sx where Sp and Sx are Shannon’s entropy for p and x. Thus the idea of successive events or arrangements is not needed, rather entropy deals with a loss of information through a product of probabilities as argued in (1). One may also have a loss of information through a sum of probabilities, but in such a case ln does not isolate the pieces of lost information. In physical systems, however, it seems that products of probabilities appear especially due to two body scattering or to an energy level absorbing a photon. The idea of sum then appears through a conserved quantity such as energy or momentum which dictates the form of P(i).
If one tosses a die or coin, different outcomes occur with a probability p(i) where i is the outcome. This probability may be measured by performing trial runs. For N trial runs, where N is very large, one might expect Np(outcome i) results of an outcome i. In other words, p(outcome i) is both the probability and describes the trial run. Even in the Maxwell-Boltzmann case, one has a probability p(ei) where ei is kinetic energy (no V(x)) and reaction balance p(e1)p(e2) = p(e3)p(e4) represents trial runs. Thus the trial run is again characterized by p(ei) as ei’s contribution. It is, however, possible to have coupling of probabilities in certain physical situations even though p(outcome i) still exists. In such a case, the number of outcome i results for N large trials is not N p(outcome i). In other words, p(i) does not fully describe the trial run. There may be other physical rules in place. If elastic scattering of particles is the way trial runs occur, e1+e2 = e3 + e4 represents the salient information in the problem and even if there is nonindependence of p(outcome i) in the scattering, one should at least obtain a function Q(p(e1)) such that Q(p(e1)) = -e1/T. Q(p(ei)), however, need not be p(ei). We consider the physical rule given by Pauli stating that only one particle may fit in an energy level (no spin). Thus there is either 0 or 1 particle in the level. This should inhibit scattering, so the trial run is no longer simply described by p(ei). Rather the probability to scatter into level ej is (1-n(ej)) and so one has: n(e1)[1-n(e3)] n(e2)[1-n(e4)]= n(e3)[1-n(e1)]n(e4)[1-n(e2)] where n(ej) is the average occupancy, a number between 0 and 1. The opposite of only 0 or 1 particle in a state ei, is to have any possible number of particles in this state and an enhancement, not a hindrance given by 1+p(ej). This is only a guess, but experiment shows that bosons follow this rule. Given a definition of trial runs in terms of the inhibition and enhancement factors, one may do two things. First, since the trial rule seems to involve different numbers of particles in a state associated with a p(ei), one may consider sums of p(ei) (power number of particles) when describing the system statistically (i.e. grand canonical scheme). This approach gives rise to linear terms (a+bp(ei)) which may in turn be formulated in terms of a factorial/arrangement equation which may be maximized subject to the constraints Sum over i ei n(ei) and Sum over i n(ei). We note that this maximization is not simply in terms of all degrees of freedom in the system, but is specifically linked with the rules associated with the trial runs and the constraints. We note, however, that issues seem to arise with the factorial scheme in the boson case. Thus, overall we argue that a single particle probability and its trial run scheme are key to establishing the statistical behaviour of a system. (We also note that single particle independent probability, which is an approximation, also affects the final results.)