The statistical definition of information is compared with Boltzmann's formula for entropy. The immediate result is that information I corresponds to a negative term in the total entropy S of a system. S=S0−I. A generalized second principle states that S must always increase. If an experiment yields an increase ΔI of the information concerning a physical system, it must be paid for by a larger increase ΔS0 in the entropy of the system and its surrounding laboratory. The efficiency ε of the experiment is defined as ε = ΔI/ΔS0≤1. Moreover, there is a lower limit k ln2 (k, Boltzmann's constant) for the ΔS0 required in an observation. Some specific examples are discussed: length or distance measurements, time measurements, observations under a microscope. In all cases it is found that higher accuracy always means lower efficiency. The information ΔI increases as the logarithm of the accuracy, while ΔS0 goes up faster than the accuracy itself. Exceptional circumstances arise when extremely small distances (of the order of nuclear dimensions) have to be measured, in which case the efficiency drops to exceedingly low values. This stupendous increase in the cost of observation is a new factor that should probably be included in the quantum theory.
The statistical definition of information is compared with Boltzmann's formula for entropy. The immediate result is that information I corresponds to a negative term in the total entropy S of a system. S=S0−I. A generalized second principle states that S must always increase. If an experiment yields an increase ΔI of the information concerning a physical system, it must be paid for by a larger increase ΔS0 in the entropy of the system and its surrounding laboratory. The efficiency ε of the experiment is defined as ε = ΔI/ΔS0≤1. Moreover, there is a lower limit k ln2 (k, Boltzmann's constant) for the ΔS0 required in an observation. Some specific examples are discussed: length or distance measurements, time measurements, observations under a microscope. In all cases it is found that higher accuracy always means lower efficiency. The information ΔI increases as the logarithm of the accuracy, while ΔS0 goes up faster than the accuracy itself. Exceptional circumstances arise when extremely small distances (of the order of nuclear dimensions) have to be measured, in which case the efficiency drops to exceedingly low values. This stupendous increase in the cost of observation is a new factor that should probably be included in the quantum theory.
We review with a tutorial scope the information theory foundations of quantum statistical physics. Only a small proportion of the variables that characterize a system at the microscopic scale can be controlled, for both practical and theoretical reasons, and a probabilistic description involving the observers is required. The criterion of maximum von Neumann entropy is then used for making reasonable inferences. It means that no spurious information is introduced besides the known data. Its outcomes can be given a direct justification based on the principle of indifference of Laplace. We introduce the concept of relevant entropy associated with some set of relevant variables; it characterizes the information that is missing at the microscopic level when only these variables are known. For equilibrium problems, the relevant variables are the conserved ones, and the Second Law is recovered as a second step of the inference process. For non-equilibrium problems, the increase of the relevant entropy expresses an irretrievable loss of information from the relevant variables towards the irrelevant ones. Two examples illustrate the flexibility of the choice of relevant variables and the multiplicity of the associated entropies: the thermodynamic entropy (satisfying the Clausius–Duhem inequality) and the Boltzmann entropy (satisfying the H-theorem). The identification of entropy with missing information is also supported by the paradox of Maxwell’s demon. Spin-echo experiments show that irreversibility itself is not an absolute concept: use of hidden information may overcome the arrow of time.
In Chap. 6 we presented in detail the relationship between observation and information, arriving at a very important result, namely, that for every observation there is a compensatory increase in entropy of other sources. We have shown that every physical observation, as well as every physical device, is ultimately limited by Heisenberg's uncertainty principle from quantum mechanics. In observing the most minute object or a small distance, there is a practical limit at which great accuracy cannot be achieved. Also, the smaller the distances or objects to be observed are, the more costly the energy (or entropy) required. In some instances, the amount of energy required may even approach infinity.
This book defines and develops a unifying principle of physics, that of 'extreme physical information'. The information in question is, perhaps surprisingly, not Shannon or Boltzmann entropy but, rather, Fisher information, a simple concept little known to physicists. Both statistical and physical properties of Fisher information are developed. This information is shown to be a physical measure of disorder, sharing with entropy the property of monotonic change with time. The information concept is applied 'phenomenally' to derive most known physics, from statistical mechanics and thermodynamics to quantum mechanics, the Einstein field equations, and quantum gravity. Many new physical relations and concepts are developed, including new definitions of disorder, time and temperature. The information principle is based upon a new theory of measurement, one which incorporates the observer into the phenomenon that he/she observes. The 'request' for data creates the law that, ultimately, gives rise to the data. The observer creates his or her local reality.
This book defines and develops a unifying principle of physics, that of 'extreme physical information'. The information in question is, perhaps surprisingly, not Shannon or Boltzmann entropy but, rather, Fisher information, a simple concept little known to physicists. Both statistical and physical properties of Fisher information are developed. This information is shown to be a physical measure of disorder, sharing with entropy the property of monotonic change with time. The information concept is applied 'phenomenally' to derive most known physics, from statistical mechanics and thermodynamics to quantum mechanics, the Einstein field equations, and quantum gravity. Many new physical relations and concepts are developed, including new definitions of disorder, time and temperature. The information principle is based upon a new theory of measurement, one which incorporates the observer into the phenomenon that he/she observes. The 'request' for data creates the law that, ultimately, gives rise to the data. The observer creates his or her local reality.
In view of the fact that physical measurements represent transmission of information, a general definition based on the measure of lack of knowledge in information theory can be introduced for different entropies each of them corresponding to one particular kind of measurement. By means of this entropy concept the problem of the consistency of microscopic reversibility and maeroscopic irreversibility is studied. Under certain conditions it can be shown that the reversible microscopic equation of motion leads to the result that all entropies corresponding to reduced quantities, i.e., to quantities which do not determine the microscopic state of the system completely, can never decrease. However, it is to be noted that this behavior is symmetric in time, in the classical as well as in the quantum mechanical case. Irreversibility is so related on the one hand to incomplete description of a system, but on the other also to the fact that theoretical statements can be verified by experiments in the future only. The validity of the results obtained within the framework of ensemble theory for a single physical system is dtscussed. As applications of the general ideas a system of free particles and the Boltzmann gas are treated. (auth)
Whether physical reality contains an actual infinity is usually discussed philosophically. We ask a narrower, quantitative question: what would it cost a finite experiment to certify it? We combine sharp results from quantum information with physical resource bounds. (1) Certification budget. Resolving a game value to ε requires N ≳ ln(1/4δ) w(1−w)/ε² rounds (Bretagnolle–Huber), so the cost of certifying a Hilbert-space dimension D is governed by the gap ε_G(D) between the optimal value and the best dimension-D value. Take the unspecified constants in the known bounds to be one. Then the best known two-player dimension witness needs at least 10⁸⁶ rounds to certify a dimension beyond the Bekenstein bound of a 1 kg, 1 m laboratory (2.6×10⁴³ qubits). Such a laboratory can perform only 2.4×10⁶⁸ operations in the age of the universe. Beyond the de Sitter entropy (4.8×10¹²² qubits), the witness needs at least 10²⁴⁵ rounds, against the 10¹²⁰ operations of the universe. Spatial parallelisation does not help, because the Margolus–Levitin bound limits the total rate summed over all subsystems that share the energy. For the I₃₃₂₂ inequality the requirement is doubly exponential. (2) Embezzlement asymptotics. The optimal fidelity of van Dam–Hayden embezzlement of one EPR pair is F_n = (H_{n/2} + σ_{n/2})/H_n. Hence 1 − F_n = (ln 2 − σ)/H_n + O(1/(n ln n)) with σ = 0.2992041. This holds for all n. It is confirmed to 10⁻⁶ for dimensions up to 2²⁶, and to 6×10⁻⁸ for odd n = 10⁷ + 1. The infinite-dimensional resource behind these witnesses is approached only logarithmically. (3) The Tsirelson loophole. MIP*=RE implies that commuting-operator correlations can exceed every finite-dimensional value by a finite gap, which finite statistics could in principle detect. However, the set of true bounds ω*(G) ≤ t is Π⁰₁ but not computably enumerable. No consistent theory therefore proves all of them, and this route is limited by provability rather than by statistics. In practice, no separating game with small question and answer sets and a computed gap is known. (4) Thermodynamic infinities. Kaufman's exact formula for the two-dimensional Ising model on a finite torus shows that a 1 cm sample rounds the critical singularity at |K − K_c|/K_c ~ 4×10⁻⁹. The logarithmic amplitude of the specific-heat peak equals that of the bulk singularity. The orbital helium lambda-point experiment, which reached |t| ≈ 10⁻⁹, stays about four orders of magnitude above the finite-size scale of its sample. These results do not show that nature is finite. They show the following: apart from one route whose cost is logical rather than statistical, finiteness bounded by holography cannot be refuted from inside a universe that respects that bound. Code, data and figures: https://github.com/Ruqing1963/certify-infinity