From the margins to the mainstream: Foundations of quantum mechanics, 1950–1990

Abstract

Research on the foundations of quantum mechanics ended the 20th century as a blossoming field including promises of applications related to the processing of information. The 2010 Wolf Prize in physics was awarded to John Clauser, Alain Aspect, and Anton Zeilinger, “for their fundamental conceptual and experimental contributions to the foundations of quantum physics, specifically an increasingly sophisticated series of tests of Bell's inequalities or extensions thereof using entangled quantum states.”1 Bell's inequalities and their related physical effect, namely quantum entanglement, encapsulate the historical changes undergone by research on the possibility of enlarging quantum mechanics through additional (thus hidden) variables. Indeed this subject once occupied a very marginal position, a philosophical issue beyond the pale of scientific research. In the 1950s, David Bohm's reinterpretation of quantum mechanics was considered either “analogous to the ‘angels’ which people introduced in the Middle Ages to explain things,” or an issue belonging “to the philosophy of science, rather than to the domain of physical science proper.”2 As late as the early 1970s some physicists still wondered whether such research was “real physics.”3 Considering foundational issues worthy of investigation is thus a kind of renaissance in the history of physics as they were present in the inception of quantum physics. This change in values requires a historical explanation and may shed light on the workings of recent science. I have presented elsewhere an account of the diverse factors which have played a role in this, calling their key actors the quantum dissidents.4 They were dissidents in the sense that they challenged the common wisdom at their time that foundational issues were already solved and not worth pursuing. As stated by John Bell and Michael Nauenberg in 1966, “the typical physicist feels that [foundational issues] have long been answered, and that he will fully understand just how if ever he can spare 20 minutes to think about it.”5 I present here a short summary of this story. Since its inception, quantum mechanics brought pressing conceptual issues requiring investigation. These issues had to include the possibility of completing quantum theory with additional variables in order to recover either realism or determinism. This possibility was exploited early on by Einstein through the EPR Gedankenexperiment and revisited by Bohm in the early 1950s. Bohm's causal interpretation was a sign that something was wrong in von Neumann's mathematical proof against additional variables, but it was not widely understood this way at the time. After 30 years without causing much ado, the proof was revised by John Bell. He showed that there was a factual conflict between predictions of quantum mechanics and the assumption of local realism. Since the early 1970s, a list of experiments—begun by Clauser, further developed by Aspect, and currently exploited by Zeilinger, among others—led to the confirmation of quantum mechanics predictions for the correlation in polarization of a pair of widely-separated photons. Empirical confirmation of entanglement as a physical property of quantum systems was thus found. However, even today, physicists struggle to understand entanglement as well as to harness it in subjects such as quantum cryptography and information. Another tough conceptual issue came to be known as the quantum measurement problem. Until World War II, two solutions had been available. For Niels Bohr, devices, if they are to be used as measurement devices, should be described using plain language enriched by concepts from classical physics. John von Neumann, on the other hand, identified two different evolutions of the quantum states, one represented by the Schrödinger equation and the other represented by the mathematical trick of an operator, a projection operator, to describe measurements. Bohr's solution was too philosophical and von Neumann's too mathematical. It thus comes as no surprise that from the 1950s physicists began to look for a physical solution to the problem. Hugh Everett's relative states were motivated by his abhorrence of two different evolutions for quantum states. Eugene Wigner considered a role for the mind during measurement and suggested formulating a nonlinear Schrödinger equation. In the 1960s, even committed Bohrians, such as Léon Rosenfeld, looked for a physical solution identifying irreversibility as the signature of measurements, and supporting the “thermodynamic amplification” approach formulated by Daneri, Loinger, and Prosperi.6 Currently, the measurement problem remains unresolved from the conceptual point of view, but we have learned a lot of physics on the way. The problem was related to the issue of the transition between the quantum and the classical description, and both issues were at the heart of what we call decoherence, the physical process through which quantum states lose their coherence, i.e., the process by which a pure state becomes a mixed one. Decoherence emerged along different and independent paths and involved the work of Heinz Dieter Zeh and Erich Joos, Anthony Leggett and Amir Caldeira, and Wojciech H. Zurek, among others. The connection between quantum and gravity is far from being solved even today. The quest for this connection has, however, stimulated research in the foundations of quantum mechanics. Bryce DeWitt's career was devoted to this search and through it he became a vocal advocate of an alternative interpretation of quantum mechanics, that is, Everett's relative states, dubbed by DeWitt as the many worlds interpretation. Philosophical issues have also motivated physicists to reconsider the foundations of quantum physics. Paramount among these issues has been the quest for determinism and attempts to accommodate realism into the quantum mathematical formalism. In hindsight, the mere listing of these issues, from which entanglement and decoherence emerged, would be enough to explain the change in the professional status of research on foundations, but this would be an anachronistic reading of events, as these issues were not always considered meaningful in physics. As important as they are considered now, these foundational issues could have lain dormant for a long time if not for the role played by the physicists who worked on them. An example of an inactive issue in the history of physics was the nature of inertia and the relationship between gravitational and inertial masses in Newton's times. Centuries later, it was revisited by Ernst Mach and later by Einstein. Likewise, research on quantum foundations flourished because a bunch of maverick physicists raised the stakes. They supported the view that there was good physics to be done concerning foundations of quantum mechanics. I think of Bohm, Everett, Bell, Wigner, Abner Shimony, Clauser, Bernard d’Espagnat, Aspect, Zeh, Franco Selleri, Leggett, and others. They took a professional and cultural stand against the prevailing wisdom of the times among physicists which was that foundational issues had already been solved by the founding fathers of the discipline. I call them quantum dissidents using a metaphor related to political dissidents in the 20th century. Eventually, all of them have gained recognition in their respective contexts as their causes became widely acknowledged. In the case of the quantum dissidents, they won insofar as foundations became a highly-valued research topic on the agenda of physics. The quantum dissidents also faced professional biases against their research. Bell, after publishing his seminal papers exhibiting flaws in von Neumann's proof and presenting the theorem now called after him, in 1966 received a letter from Rosenfeld stating: “I need not tell you that I regard your hunting hidden parameters as a waste of your talent.”7 Familiar with such bias, Bell's first question to the young Aspect, who looked for advice for his planned experiments, was “Do you have a permanent job?” Aspect did, and this made all the difference. For some of the dissidents prejudices were highly effective. Indeed they paid a high professional price for their dissidence. Two cases illustrate this. In the early 1970s, Clauser was a promising young physicist. He did his PhD at Columbia under Patrick Thaddeus working on microwave measurements of cosmic background radiation and then experimented with Bell's theorem. Together with Shimony, Horne, and Holt, he translated the theorem to real labs. He performed the first experiment and obtained a corroboration of quantum predictions, which was a landmark in physics considering the limited techniques available at the time. He further developed Bell's theorem, replicated experiments on it, and participated in the debates during the inception of quantum optics. Despite these achievements, Clauser never got a permanent position in American academia, as many department heads in a position to hire him doubted if Bell's theorem was real physics.8 About the same time, Zeh was working on nuclear physics and wrote a paper which may now be considered a forerunner of the decoherence approach. The paper suggested that the ensemble of a system plus a measurement device cannot be considered a closed system, thus approximations are required when using the Schrödinger equation to describe it. The paper was not accepted by Il Nuovo Cimento and Die Naturwissenschaften. At that juncture, Zeh's senior at Heidelberg, the Nobel Prize winner J. H. D. Jensen decided to ask Rosenfeld's advice on the paper. Rosenfeld's opinion was devastating. “I have all the reasons in the world to assume that such a concentrate of wildest nonsense is not being distributed around the world with your blessing, and I think to be of service to you by directing your attention to this misfortune.”9 Zeh did not read this letter at the time—only recently was it unearthed—but he realized that the professional ambiance at Heidelberg had changed towards him. Jensen frankly informed him that any further activities on this subject would end his academic career. The paper was rescued by Wigner and eventually published. However, Zeh considered that his career had been destroyed and therefore he could dedicate himself to the forbidden subject, foundations of quantum physics. The quantum dissidents did not defend their research in a vacuum. If the professional ambiance was against them, devaluing their research, other factors played a favorable role. In the 1950s, the Cold War brought unforeseen benefits for the dissidents. As part of the ideological battles of the time, Soviet philosophers and physicists criticized the usual interpretation of quantum mechanics, namely complementarity, for being subjectivist. Unlike other cultural criticisms, this one resonated with similar criticisms made in the West. As cogitated by the historian of physics Max Jammer, the decline in the influence of the complementarity interpretation may have been fomented and supported by social-cultural movements and political factors such as the growing interest in Marxist ideology in the West.10 This stands in spite of the fact that there were Marxist physicists on both sides of the controversy, some pro, others counter to Bohr's complementarity principle. In the late 1950s and early 1960s, the old guard of the founding fathers of quantum physics were retreating from the lines of philosophical battle. A new generation emerged which was more inclined to criticize the usual interpretation of quantum physics. In addition, some of the old guard clashed among themselves, Rosenfeld defending Bohr's views and Wigner sharing von Neumann's approach. Moreover, Wigner entered these disputes considering himself the defender of the orthodoxy and ended up considering himself heterodox. The monocracy of the Copenhagen school, to use a term from Jammer, broke from the inside. In the late 1960s and early 1970s, the cultural and social unrest surrounding the Vietnam War encouraged physicists to set their research agenda to heterodox subjects, such as foundations. In Italy the Italian Physical Society promoted foundations as a highly valid topic for physics research opening the doors of its traditional summer school, in Varenna, to the subject “foundations of quantum mechanics.” D'Espagnat chaired the school while Wigner gave the opening lecture presenting the diversity of solutions for the measurement problem. Bell's theorem and the first experiments attracted the attention of many, including Franco Selleri, who went on to devote much of his energy to the subject. The 1970 Varenna school was the Woodstock of all quantum dissidents (Fig. 2). According to the recollections of Anders Barany, then 28 years old, “what most impressed me was that many (maybe even most) of the highly qualified scientists lecturing at the school could not cooperate to try and help the students form a coherent picture of the different problems connected with the foundations of quantum physics.”11 On the other side of the Atlantic, similar trends were operating. Physics Today, the prestigious magazine of the American physics community, editorialized the debate on the interpretation of quantum physics, opening it with an article by DeWitt in which the diversity of possible interpretations was presented and Everett's views were supported. On the West coast, physicists who followed the bourgeoning hippie counterculture became fascinated by entanglement and by the apparent possibilities of superluminal communication. Ultimately this latter hope did not pan out, but in the process good physics was produced.12 In the early 1980s, particularly after Aspect's 1982 experiments on entanglement, foundations became good physics. From the mid 1980s onwards, technical advances in the manipulation of single quantum systems and even applied research facilitated new experiments in foundations such as the delayed-choice experiment advocated by John Wheeler.13 Later, the merging of foundations and computer science sealed the fate of research on foundations as it became one of the pillars for the blossoming field of quantum information. This fascinating story shows us the many and varied factors shaping the cultural enterprise we call physics. The story cannot be ended without a ruse of history. The quantum dissidents won the day insofar as foundations became good physics. However, they had higher expectations. John Bell, the iconic figure among the physicists who work on these issues, used to say that quantum mechanics is “rotten,” Hamlet's famous line, in an oblique reference to the father of the standard interpretation of this theory, the Danish physicist Niels Bohr. The ruse is that in trying to attack the foundations of quantum mechanics, they ended up consolidating and developing quantum mechanics itself and quantum theory entered the 21st century as vindicated as ever.