Forty years after the advent of quantum mechanics the problem of hidden variables, that is, the possibility of imbedding quantum theory into a classical theory, remains a controversial and obscure subject. Whereas to most physicists the possibility of a classical reinterpretation of quantum mechanics remains remote and perhaps irrelevant to current problems, a minority have kept the issue alive throughout this period. (See Freistadt [5] for a review of the problem and a comprehensive bibliography up to 1957.) As far as results are concerned there are on the one hand purported proofs of the non-existence of hidden variables, most notably von Neumann’s proof, and on the other, various attempts to introduce hidden variables such as de Broglie [4] and Bohm [1] and [2]. One of the difficulties in evaluating these contradictory results is that no exact mathematical criterion is given to enable one to judge the degree of success of these proposals. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Forty years after the advent of quantum mechanics the problem of hidden variables, that is, the possibility of imbedding quantum theory into a classical theory, remains a controversial and obscure subject. Whereas to most physicists the possibility of a classical reinterpretation of quantum mechanics remains remote and perhaps irrelevant to current problems, a minority have kept the issue alive throughout this period. (See Freistadt [5] for a review of the problem and a comprehensive bibliography up to 1957.) As far as results are concerned there are on the one hand purported proofs of the non-existence of hidden variables, most notably von Neumann’s proof, and on the other, various attempts to introduce hidden variables such as de Broglie [4] and Bohm [1] and [2]. One of the difficulties in evaluating these contradictory results is that no exact mathematical criterion is given to enable one to judge the degree of success of these proposals. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Introduction To know the quantum mechanical state of a system implies, in general, only statistical restrictions on the results of measurements. It seems interesting to ask if this statistical element be thought of as arising, as in classical statistical mechanics, because the states in question are averages over better defined states for which individually the results would be quite determined. These hypothetical ‘dispersion free’ states would be specified not only by the quantum mechanical state vector but also by additional ‘hidden variables’ – ‘hidden’ because if states with prescribed values of these variables could actually be prepared, quantum mechanics would be observably inadequate. Whether this question is indeed interesting has been the subject of debate. The present paper does not contribute to that debate. It is addressed to those who do find the question interesting, and more particularly to those among them who believe that ‘the question concerning the existence of such hidden variables received an early and rather decisive answer in the form of von Neumann's proof on the mathematical impossibility of such variables in quantum theory.’ An attempt will be made to clarify what von Neumann and his successors actually demonstrated. This will cover, as well as von Neumann's treatment, the recent version of the argument by Jauch and Piron, and the stronger result consequent on the work of Gleason. It will be urged that these analyses leave the real question untouched.
The question of the possible existence of hidden variables is re-examined. It is shown that hidden variables can exist only if every proposition (yes-no experiment) is compatible with every other one. It is further shown that this property is in contradiction with empirical facts. The theorem which leads to this conclusion is a strengthening of the theorem of von Neumann on the same subject. The question is raised whether there exist perhaps quantum mechanical systems which admit approximate dispersion-free states.
The problem of hidden variables, purely conceptual at the start, has been early influenced by the practical aims of “causal theories” towards a mathematical scheme broader than the quantum formalism. Thus several authors have examined if a new formalism, built in the same spirit as that of quantum mechanics, however more general, might admit a determination by means of hidden parameters, and concluded negatively. In the present paper the two aspects (conceptual and practical) are considered separately. It first contains a study of the unrestricted proofs (Von Neumann, Jauch and Piron), given for the non-existence of hidden parameters. They are focussed on the dispersion-free states bound to them. Thus a preliminary stage consists in analyzing what properties these states should have. A method is given here for displaying these; they are shown to have an artificial character, which fits neither Von Neumann's proof nor that of Jauch and Piron and other authors (Misra). Thus, these proofs do not establish the inexistence of hidden variables, but only that of dispersion free states built more or less on the pattern of the quantum states. Suggestions are then proposed on the probable features of hidden parameters. They lead us to regard as improbable the hope of building a new deterministic scheme to be substituted for the quantum formalism. Thus, the determinism which can be maintained appears as purely conceptual. The question of a “deeper level” and the relation between determination and pictorial description are discussed briefly.
This article reviews the history of J. von Neumann’s analysis of hidden variables in quantum mechanics and the subsequent analysis by others. In his book The Mathematical Foundations of Quantum Mechanics, published in 1932, von Neumann performed an analysis of the consequences of introducing hidden parameters (hidden variables) into quantum mechanics. He arrived at two principal conclusions: first, hidden variables cannot be incorporated into the existing theory of quantum mechanics without major modifications, and second, if they did exist, the theory would have already failed in situations where it has been successfully applied. This analysis has been taken as an “incorrect proof” against the existence of hidden variables, possibly due to a mistranslation of the German word prufen. von Neumann’s so-called proof isn’t even wrong as such a proof does not exist, but it is an examination of the limitations imposed by internal consistency of the Hilbert space formulation of the theory. One of the earliest attempts to eliminate uncertainty, by D. Bohm, requires a major modification of quantum mechanics (observables are not represented by Hermitian operators), which supports von Neumann’s first principal conclusion. However, testing the Bohm theory requires constructing a physically impossible initial state. As such, the theory has no experimental consequences, so W. Pauli referred to it as an “uncashable check”. As there are no observable consequences, the Bohm theory is possibly a counterexample to von Neumann’s second conclusion that hidden variables in particular would have already led to a failure of the theory.
Prologue A memorial conference for John Bell, open to the public, was held at Rutgers University around 20 years ago. I gave a short talk there on Bell's views about David Bohm's “hidden variables” formulation of quantum mechanics, a version of quantum mechanics often called the de Broglie–Bohm theory or Bohmian mechanics. This theory was in fact discovered by Louis de Broglie in 1927. In 1952, it was rediscovered and developed by Bohm, who was the first to appreciate its connection to the predictions of standard quantum mechanics. I did not publish the talk. I have decided that it would be appropriate to publish a very lightly edited version of it here, in this volume devoted to the work of Bell on the foundation of quantum mechanics. One reason for doing so is that the connection between Bell and Bohm continues to be somewhat underplayed, with the strength of his advocacy of Bohmian mechanics not properly appreciated. For example, about half of the papers in Bell's collected works on the foundations of quantum mechanics deal with Bohmian mechanics. But in his fine introduction to the revised edition of this great book [1], Alain Aspect mentions this theory in only a single sentence, and parenthetically at that. For several decades after Bell proved his nonlocality theorem, based in part on Bell's inequality, it was widely claimed that Bell had shown that hidden variables – and Bohmian mechanics in particular – were impossible, that they were incompatible with the predictions of quantum mechanics. For example, the great physicist Eugene Wigner, who, unlike most of his contemporaries, was profoundly concerned with the conceptual foundations of quantum mechanics and usually wrote on the subject with great clarity and insight, has written that This [hidden variables] is an interesting idea and even though few of us were ready to accept it, it must be admitted that the truly telling argument against it was produced as late as 1965, by J. S. Bell … This appears to give a convincing argument against the hidden variables theory.
Hidden variable theories, ‘the hidden parameter’ as von Neumann called them, entered the quantum scene soon after the formalization of quantum mechanics as we know it today. That formalization was led by front-line physicists like Born, Dirac, Heisenberg, Jordan, and Schrödinger. Attention to hidden variable theories garnered strength following the criticisms towards quantum mechanics formulated by Einstein and colleagues in 1935. That criticism, stating that quantum mechanics was ‘not complete,’ as a theory, led Bohm, in the 1950s, to formulate hidden variable theories. In turn, Bell in 1964 introduced his famed Bell theorem that provided a test for the existence of hidden variable theories. This chapter examines the history of the futile search, for experimental proof, for the existence of these theories.