INTRODUCTION The purpose of this article is to bring out in as conceptually clear terms as possible what seems to be a major incompleteness in the probability theory of particles offered by classical quantum mechanics. The exact nature of this incompleteness is illustrated by consideration of some simple quantum-mechanical examples. In addition, these examples are contrasted with the fundamental assumptions of Brownian motion in classical physics on the one hand, and with a controversy of a decade ago in mathematical psychology. (The psychological example is described in detail in the appendix). Our central claim is that classical quantum mechanics is radically incomplete in its probabilistic account of the motion of particles. In the last part of the article we derive the time-dependent joint distribution of position and momentum of the linear harmonic oscillator, and show how the apparently physically paradoxical statistical independence of position and momentum has a natural explanation. The explanation is given within the framework of the non-quantum-mechanical stochastic theory we construct for such oscillators. Before entering into any technical details, we shall attempt to spell out in intuitive terms our incompleteness claim. To begin with, there are two senses of completeness that we want to distinguish. One sense of completeness is that demanded by any hidden variable theories of quantum mechanics. States should be found, or be theoretically characterizable at least, that lead to precise values for all observables; that is, no observable is to have a variance greater than zero.
INTRODUCTION The purpose of this article is to bring out in as conceptually clear terms as possible what seems to be a major incompleteness in the probability theory of particles offered by classical quantum mechanics. The exact nature of this incompleteness is illustrated by consideration of some simple quantum-mechanical examples. In addition, these examples are contrasted with the fundamental assumptions of Brownian motion in classical physics on the one hand, and with a controversy of a decade ago in mathematical psychology. (The psychological example is described in detail in the appendix). Our central claim is that classical quantum mechanics is radically incomplete in its probabilistic account of the motion of particles. In the last part of the article we derive the time-dependent joint distribution of position and momentum of the linear harmonic oscillator, and show how the apparently physically paradoxical statistical independence of position and momentum has a natural explanation. The explanation is given within the framework of the non-quantum-mechanical stochastic theory we construct for such oscillators. Before entering into any technical details, we shall attempt to spell out in intuitive terms our incompleteness claim. To begin with, there are two senses of completeness that we want to distinguish. One sense of completeness is that demanded by any hidden variable theories of quantum mechanics. States should be found, or be theoretically characterizable at least, that lead to precise values for all observables; that is, no observable is to have a variance greater than zero.
This article is the third of a trilogy of articles on the nature of probability in quantum mechanics. The first article [1] began by noting that superposed wavefunctions in quantum mechanics lead to a squared amplitude that introduces interference into the position probability density of an atomic particle, which happens nowhere else in probability theory. It went on to also note that there is an unexplained coincidence in quantum mechanics in that the interference term in the squared amplitude of superposed wavefunctions gives the squared amplitude the form of a variance of a sum of correlated random variables and went on to examine whether there could be an archetypical variable, in the Platonic sense of true form, behind quantum probability that would reconcile quantum probability with classic probability. This examination found such a variable, which can encompass both local and nonlocal quantum events. The second article [2] provided evidence that quantum probability has such a stochastic nature. This evidence was based on the number of electrons that need to be sent through a two-slit interferometer to gain a clear pattern of self-interference, which when compared with the number that would be expected to be sufficient in order for the position probability distribution of the self-interference wavefunction to take clear shape suggests that there is more variability present than that described by the formulation of quantum mechanics, which implies the presence of an underlying and as yet unrecognized physical process. This final article completes the trilogy by considering how a key aspect of experimental design would be affected by the increased variability that would be present if quantum probability is itself stochastic in the manner suggested in the previous two articles.
It is a scientific truism nowadays that quantum mechanical predictions are of statistical or rather of probabilistic nature: a quantum theory assigns a probability distribution to each measurable quantity (observable) of a physical system prepared according to a given preparation prescription (state). Such sets of probability distributions represent the most general quantitative predictions of physical theories. The aim of this study is to determine all theoretical frameworks which yield such a kind of output taking the probabilistic nature of quantitative physical predictions as initial hypothesis. Astonishingly, this project succeeds with a minimal number of additional assumptions, which arise quite naturally in the course of the analysis, and which essentially amount to regularity conditions. The result is that there exist only two classes of physical theories providing quantitative probabilistic predictions: classical theories where observables are represented by real functions over a phase space and states are represented by probability measures upon the phase space, and quantum theories where observables are represented by selfadjoint operators acting on a complex Hilbert space and states are represented by state operators. On empirical grounds, classical theories cannot provide the all-encompassing, fundamental frame for the mathematical description of physical systems. Quantum theories remain, and there is no more complete formal frame for quantitative physical theories. As an additional result, it is shown that there exist physical systems which possess a quantum mechanical but no classical description; a prominent example is the spin system of two spin-1/2 particles. The classical hidden variables model, which J. Bell relates to this system in order to prove his theorem, cannot be a true description of it because no such classical model exists, and Bell's theorem loses its foundation.
Abstract In this chapter, the main features of quantum theory are presented. The chapter begins with a historical account of the invention of quantum mechanics. The meaning of position and momentum in quantum mechanics is discussed and non-commuting operators are introduced. The Schrödinger equation is presented and solved for a free particle and for a harmonic oscillator potential in one dimension. The meaning of the wavefunction is considered and the probabilistic interpretation is presented. The mathematical machinery and language of quantum mechanics are developed, including Hermitian operators, observables and expectation values. The uncertainty principle is discussed and the uncertainty relations are presented. Scattering and tunnelling by potential wells and barriers is considered. The use of variational principles to estimate ground state energies is explained and illustrated with a simple example.
Quantum mechanics has irked physicists ever since its conception more than 100 years ago. While some of the misgivings, such as it being unintuitive, are merely aesthetic, quantum mechanics has one serious shortcoming: it lacks a physical description of the measurement process. This ``measurement problem'' indicates that quantum mechanics is at least an incomplete theory -- good as far as it goes, but missing a piece -- or, more radically, is in need of complete overhaul. Here we describe an approach which may provide this sought-for completion or replacement: Superdeterminism. A superdeterministic theory is one which violates the assumption of Statistical Independence (that distributions of hidden variables are independent of measurement settings). Intuition suggests that Statistical Independence is an essential ingredient of any theory of science (never mind physics), and for this reason Superdeterminism is typically discarded swiftly in any discussion of quantum foundations. The purpose of this paper is to explain why the existing objections to Superdeterminism are based on experience with classical physics and linear systems, but that this experience misleads us. Superdeterminism is a promising approach not only to solve the measurement problem, but also to understand the apparent nonlocality of quantum physics. Most importantly, we will discuss how it may be possible to test this hypothesis in an (almost) model independent way.
When we are presented with apparently bizarre properties of microsystems governed by quantum mechanics (as in our discussion of the double-slit experiment in the previous chapter), we naturally ask whether it might be possible to find another theory, or at least another story to go with the equations, that would make more sense to us. We might seek a theory and a view of fundamental physical processes that would accord more with our classically based, everyday common sense. This typically takes the form of the question of the possibility of the existence of a more detailed description of atomic processes than that afforded by the wave function (that is, by the Copenhagen interpretation of the previous chapter). In this chapter we discuss some of the central interpretive problems of the Copenhagen version of quantum mechanics. Then, in the next two chapters, we look at the severe, general restrictions that are placed on any such completion of quantum mechanics and at one particular extension of quantum mechanics. First, though, we begin here with the struggle that ensued over the status of causality in quantum theory. THE COMPLETENESS OF QUANTUM MECHANICS Although the Heisenberg uncertainty principle (Section 20.4) and the lack of absolute predictive power are an inherent feature of quantum mechanics, there have been attempts to preserve an in principle completely deterministic structure for physics. Even classically, in a macroscopic sample of gas there are so many molecules that it would be hopeless, as a practical matter, to predict the future locations of all these molecules from Newton's laws of motion.
The dominance of the Copenhagen interpretation is the result of a historical accident (Cushing, 1994). At the Solvay conference in 1927, Louis de Broglie put forward the deterministic pilot wave theory, described later. Never slow to criticize, Pauli jumped up and demolished de Broglie’s proposal, and de Broglie was so discouraged by this that he did not develop his interpretation any further. The Copenhagen interpretation was then developed by Bohr, Heisenberg, Pauli and many others, as described in the previous chapter. It was strongly supported by von Neumann’s proof of the impossibility of hidden variables. Years later, in 1952, Bohm published a deterministic hidden variable theory, showing that there must be something wrong with von Neumann’s proof. The error was found by John Bell in 1966, as described in the last chapter, and he also showed how Pauli’s objections to de Broglie’s original idea can be answered. If this had been done at the conference in 1927, quantum mechanics could have been interpreted deterministically from the beginning. The essential mistake in the Copenhagen interpretation is to treat it as a complete account of the behaviour of each individual system. It is frequently claimed that it enables us to calculate, at least in principle, everything that can be measured. Thus Hooft (1997, p. 11) said that ‘The laws of quantum mechanics have been formulated very accurately. We know exactly how to compute anything we would like to know.’ Also Peierls (1997, p. 25) has said that quantum mechanics ‘had become a complete and consistent scheme capable of giving a unique answer to any questions relating to actual or possible observations’. Even if this were true, it would not imply that it is the final complete theory, for it is always possible that some new phenomenon might be found that cannot be calculated quantum-mechanically. However, it is not true: there are many phenomena such as the time of decay of a radioactive nucleus, or the direction a particle is scattered by a nucleus, that cannot be calculated, even in principle. Quantum mechanics is therefore an incomplete theory. We can only calculate the statistical properties of these systems, such as the half-life of a radioactive decay or the differential scattering cross-section that gives the probabilities that a particle is scattered through various angles. All measurements of quantum systems are of this statistical character. It might be thought that the possibility of making measurements on a single electron provides an exception to this. However, even in such cases the electron is continually bathed in a fluctuating background radiation from nearby atoms. Since this is variable, that electron is a member of an ensemble of electrons subject to different fluctuations. All these examples clearly demonstrate the statistical character of quantum mechanics. ‘I am rather firmly convinced’, Einstein (in Schilpp, 1949, pp. 666, 671) remarked, ‘that the essentially statistical character of contemporary quantum theory is solely to be ascribed to fact that this theory operates with an incomplete description of physical systems’. As a result, ‘the c-function is to be understood as the description not of a single system but of an ensemble of systems’. So, ‘if the statistical quantum theory does not pretend to describe the individual system completely, it appears unavoidable to look elsewhere for a complete description of the individual systems’. Thus ‘the difficulties of theoretical interpretation disappear, if one views the quantum-mechanical description as a description of an ensemble of systems’. If this is achieved, ‘the statistical theory would, within the framework of future physics, take an approximately analogous position to statistical mechanics within the framework of classical mechanics’. ‘Thus, essentially, nothing has changed since Galileo or Newton or Faraday concerning the status of the ‘‘observer’’ or our ‘‘consciousness’’ or of our ‘‘information’’ in physics’ (Popper, 1982, p. 46). Once this is accepted, the quantum paradoxes that plagued the Copenhagen interpretation are easily resolved, as described below. In any textbook of quantum mechanics problems such as the calculation of the energy levels of the hydrogen atom are solved as if that hydrogen atom is alone in the universe. As Feynman (1972) remarked, ‘when we solve a quantum-mechanical problem, what we really do is to divide the universe into two parts – the system in which we are interested and the rest of the universe. We then usually act as if the system in which we are interested comprised the whole universe.’ In fact, however, every hydrogen atom on which measurements are made is surrounded by other atoms and exposed to their radiations. Quantum mechanics somehow takes this into account and gives the average behaviour of an ensemble of hydrogen atoms. A more detailed theory takes these influences into account. One attempt to do this is stochastic electrodynamics, described in a later section.