The highly speculative nature of the pilot-wave theory is obvious and acknowledged even by its keenest supporters. But many physicists, including those who do not believe it is a true description of what exists, may well find it useful when trying to interpret quantum theory. Hidden-variables theories are not the only theories susceptible of being interpreted as faithful descriptions of "what exists" (so-called ontological theories). In particular, several physicists investigated the possibility of building up ontological theories in which the state vector would constitute the complete specification of the physical state of the system. Within such an approach the main difficulty is of course the one linked with the reduction of the wave packet. In any ontologically interpretable theory elements must be added to standard quantum mechanics in order that objective states of individual physical systems can be defined. The models reviewed in this chapter are a sample of what has been proposed in the domain of ontologically interpretable theories.
The highly speculative nature of the pilot-wave theory is obvious and acknowledged even by its keenest supporters. But many physicists, including those who do not believe it is a true description of what exists, may well find it useful when trying to interpret quantum theory. Hidden-variables theories are not the only theories susceptible of being interpreted as faithful descriptions of "what exists" (so-called ontological theories). In particular, several physicists investigated the possibility of building up ontological theories in which the state vector would constitute the complete specification of the physical state of the system. Within such an approach the main difficulty is of course the one linked with the reduction of the wave packet. In any ontologically interpretable theory elements must be added to standard quantum mechanics in order that objective states of individual physical systems can be defined. The models reviewed in this chapter are a sample of what has been proposed in the domain of ontologically interpretable theories.
The usual interpretation of the quantum theory is self-consistent, but it involves an assumption that cannot be tested experimentally, viz., that the most complete possible specification of an individual system is in terms of a wave function that determines only probable results of actual measurement processes. The only way of investigating the truth of this assumption is by trying to find some other interpretation of the quantum theory in terms of at present "hidden" variables, which in principle determine the precise behavior of an individual system, but which are in practice averaged over in measurements of the types that can now be carried out. In this paper and in a subsequent paper, an interpretation of the quantum theory in terms of just such "hidden" variables is suggested. It is shown that as long as the mathematical theory retains its present general form, this suggested interpretation leads to precisely the same results for all physical processes as does the usual interpretation. Nevertheless, the suggested interpretation provides a broader conceptual framework than the usual interpretation, because it makes possible a precise and continuous description of all processes, even at the quantum level. This broader conceptual framework allows more general mathematical formulations of the theory than those allowed by the usual interpretation. Now, the usual mathematical formulation seems to lead to insoluble difficulties when it is extrapolated into the domain of distances of the order of ${10}^{\ensuremath{-}13}$ cm or less. It is therefore entirely possible that the interpretation suggested here may be needed for the resolution of these difficulties. In any case, the mere possibility of such an interpretation proves that it is not necessary for us to give up a precise, rational, and objective description of individual systems at a quantum level of accuracy.
The original version of the de Broglie-Bohm pilot-wave theory, also called Bohmian mechanics, attempted to treat the wave function or pilot wave as a part of the physical ontology of nature. More recent versions of the de Broglie-Bohm theory appearing in the last few decades have tried to regard the pilot wave instead as an aspect of the theory's nomology, or dynamical laws. This paper argues that neither of these views is correct, and that the de Broglie-Bohm pilot wave is best understood as a collection of latent variables in the sense of a hidden Markov model, a construct that was not available when de Broglie and Bohm originally formulated what became their pilot-wave theory. This paper also discusses several other challenges for the ontological view of the pilot wave. One such challenge is due to Foldy-Wouthuysen gauge transformations, which connect up with the Deotto-Ghirardi ambiguity in the de Broglie-Bohm theory. Another challenge arises from the freedom to carry out canonical transformations in the wave function's own notion of phase space, as defined by Strocchi and Heslot.
What does quantum physics tell us about the nature of reality, specifically the parts of reality we do not directly perceive called hidden variables? One may think it could tell us a lot because of our enhanced technological sensing abilities that delve into the realms that quantum physics covers so well. Surprisingly, it seems to surround us in a deeper mystery rather than reveal more of nature's secrets. It seems that we cannot escape from philosophical consideration when dealing with what is hidden in quantum physics. In Part I we will look at how Epistemology and Ontology bear upon Hidden Variables. In Part II we will consider Hidden Variables in the light of Contextuality, and Non-Classicality. Inevitably questions of subjectivity and objectivity arise in dealing with states of observation. How should we think about these states? Perhaps it is a question of the meaning associated with our knowledge of a state-that is, a question of ontology or epistemology. The issue of ontic and epistemic states is particularly important when considering hidden variables in quantum physics because, as one may argue, the interpretation of quantum states as either ontic or epistemic will naturally lead to different assumptions about how reality is constructed; if it is constructed or not. It also raises the question of what attributes we are able to observe simultaneously and that brings contextuality into the discussion. If it turns out that reality is constructed contextually what does that imply about ontological realism? If on the other hand reality is constructed non-contextually what does that imply about ontological realism? Many implications can arise when considering these questions from a quantum physical point of view. In this paper I shall discuss how quantum physics provides some answers to these questions by considering quantum physical states and their measurements.
The purpose of this paper is to present an overview of recent work on pilot-wave approaches to quantum field theory. In such approaches, systems are not only described by their wave function, as in standard quantum theory, but also by some additional variables. In the non-relativistic pilot-wave theory of deBroglie and Bohm those variables are particle positions. In the context of quantum field theory, there are two natural choices, namely particle positions and fields. The incorporation of those variables makes it possible to provide an objective description of nature in which rather ambiguous notions such as 'measurement' and 'observer' play no fundamental role. As such, the theory is free of the conceptual difficulties, such as the measurement problem, that plague standard quantum theory.
The dominance of the Copenhagen interpretation is the result of a historical accident (Cushing, 1994). At the Solvay conference in 1927, Louis de Broglie put forward the deterministic pilot wave theory, described later. Never slow to criticize, Pauli jumped up and demolished de Broglie’s proposal, and de Broglie was so discouraged by this that he did not develop his interpretation any further. The Copenhagen interpretation was then developed by Bohr, Heisenberg, Pauli and many others, as described in the previous chapter. It was strongly supported by von Neumann’s proof of the impossibility of hidden variables. Years later, in 1952, Bohm published a deterministic hidden variable theory, showing that there must be something wrong with von Neumann’s proof. The error was found by John Bell in 1966, as described in the last chapter, and he also showed how Pauli’s objections to de Broglie’s original idea can be answered. If this had been done at the conference in 1927, quantum mechanics could have been interpreted deterministically from the beginning. The essential mistake in the Copenhagen interpretation is to treat it as a complete account of the behaviour of each individual system. It is frequently claimed that it enables us to calculate, at least in principle, everything that can be measured. Thus Hooft (1997, p. 11) said that ‘The laws of quantum mechanics have been formulated very accurately. We know exactly how to compute anything we would like to know.’ Also Peierls (1997, p. 25) has said that quantum mechanics ‘had become a complete and consistent scheme capable of giving a unique answer to any questions relating to actual or possible observations’. Even if this were true, it would not imply that it is the final complete theory, for it is always possible that some new phenomenon might be found that cannot be calculated quantum-mechanically. However, it is not true: there are many phenomena such as the time of decay of a radioactive nucleus, or the direction a particle is scattered by a nucleus, that cannot be calculated, even in principle. Quantum mechanics is therefore an incomplete theory. We can only calculate the statistical properties of these systems, such as the half-life of a radioactive decay or the differential scattering cross-section that gives the probabilities that a particle is scattered through various angles. All measurements of quantum systems are of this statistical character. It might be thought that the possibility of making measurements on a single electron provides an exception to this. However, even in such cases the electron is continually bathed in a fluctuating background radiation from nearby atoms. Since this is variable, that electron is a member of an ensemble of electrons subject to different fluctuations. All these examples clearly demonstrate the statistical character of quantum mechanics. ‘I am rather firmly convinced’, Einstein (in Schilpp, 1949, pp. 666, 671) remarked, ‘that the essentially statistical character of contemporary quantum theory is solely to be ascribed to fact that this theory operates with an incomplete description of physical systems’. As a result, ‘the c-function is to be understood as the description not of a single system but of an ensemble of systems’. So, ‘if the statistical quantum theory does not pretend to describe the individual system completely, it appears unavoidable to look elsewhere for a complete description of the individual systems’. Thus ‘the difficulties of theoretical interpretation disappear, if one views the quantum-mechanical description as a description of an ensemble of systems’. If this is achieved, ‘the statistical theory would, within the framework of future physics, take an approximately analogous position to statistical mechanics within the framework of classical mechanics’. ‘Thus, essentially, nothing has changed since Galileo or Newton or Faraday concerning the status of the ‘‘observer’’ or our ‘‘consciousness’’ or of our ‘‘information’’ in physics’ (Popper, 1982, p. 46). Once this is accepted, the quantum paradoxes that plagued the Copenhagen interpretation are easily resolved, as described below. In any textbook of quantum mechanics problems such as the calculation of the energy levels of the hydrogen atom are solved as if that hydrogen atom is alone in the universe. As Feynman (1972) remarked, ‘when we solve a quantum-mechanical problem, what we really do is to divide the universe into two parts – the system in which we are interested and the rest of the universe. We then usually act as if the system in which we are interested comprised the whole universe.’ In fact, however, every hydrogen atom on which measurements are made is surrounded by other atoms and exposed to their radiations. Quantum mechanics somehow takes this into account and gives the average behaviour of an ensemble of hydrogen atoms. A more detailed theory takes these influences into account. One attempt to do this is stochastic electrodynamics, described in a later section.
Quantum mechanics is an immensely successful theory, occupying a unique position in the history of science. It has solved mysteries ranging from macroscopic superconductivity to the microscopic theory of elementary particles and has provided deep insights into the nature of vacuum on the one hand and the description of the nucleon on the other. Whole new fields such as quantum optics and quantum electronics owe their very existence to this body of knowledge. However, despite the stunning successes of quantum mechanics, there is no general agreement on the conceptual foundations and interpretation of the subject. The theory provides unambiguous information about the outcome of a measurement of a physical object. However, many feel that it does not provide a satisfactory answer to the nature of the “reality” we should attribute to the physical objects between the acts of measurement. The conceptual difficulty comes about because the wave function |ψ〉 is usually given by a coherent superposition of various distinguishable experimental outcomes. If we denote the collection of states that represent the possible outcomes of an experiment by |ψ j 〉, then |ψ〉 = ∑ j c j |ψ j 〉 where c j = 〈ψ j |ψ〉. The probability of the outcome |ψ j 〉 is P j =|c j | 2 . In the process of measurement, the so called collapse of the wave function takes place and a single, definite state |ψ i 〉 of the physical object is chosen. The difficulty comes about in the interpretation of the mechanism by which this definite state is chosen from amongst all the possible outcomes.