WHAT IS QUANTUM MECHANICS? A remarkable feature of the 1968 conference of Nobel prize winners in physics at Lindau is that it was possible for me to ask such a question in the presence of two of the founders of quantum mechanics, Werner Heisenberg and P.A.M. Dirac, more than 30 years after the discovery, in a lecture attended by 400 students who had recently begun their study of the subject. Several answers to the question are possible. The only easy one is that quantum mechanics is a discipline that provides a wonderful set of rules for calculating physical properties of matter. For such simple systems as hydrogen and helium atoms the calculated energy levels agree with experiment to fantastic accuracy. In more complicated cases the computations are difficult and the accuracy is lower, but it is reasonable to believe, in principle at least, that the theory would be adequate if only the calculational problems could be overcome.
WHAT IS QUANTUM MECHANICS? A remarkable feature of the 1968 conference of Nobel prize winners in physics at Lindau is that it was possible for me to ask such a question in the presence of two of the founders of quantum mechanics, Werner Heisenberg and P.A.M. Dirac, more than 30 years after the discovery, in a lecture attended by 400 students who had recently begun their study of the subject. Several answers to the question are possible. The only easy one is that quantum mechanics is a discipline that provides a wonderful set of rules for calculating physical properties of matter. For such simple systems as hydrogen and helium atoms the calculated energy levels agree with experiment to fantastic accuracy. In more complicated cases the computations are difficult and the accuracy is lower, but it is reasonable to believe, in principle at least, that the theory would be adequate if only the calculational problems could be overcome.
The difficulties of the old quantum theory were largely dispelled by the development of quantum mechanics in the 1920s. First Heisenberg (1925) developed the rather abstract matrix mechanics and soon after Schro¨dinger (1926; see also Eckart, 1926) developed the physically more appealing wave mechanics. Subsequently he showed them to be mathematically equivalent. The physicists were greatly relieved; now they had once again a mathematical formalism that enabled the results of experiments to be calculated. It was possible to move forward. It is certainly a very elegant and indeed beautiful formalism, and does enable the results of many physical measurements to be calculated with great accuracy. It is used daily by atomic and nuclear physicists; there is no other way to make calculations. From the earliest days there has been intense debate about the interpretation of quantum mechanics, and it shows no signs of being settled; if anything it is becoming more intense over the years. As Penrose has remarked, no one understands quantum mechanics. Feynman (1967; see also Sykes, 1994) also admitted that ‘nobody understands quantum mechanics’, and Einstein once remarked: ‘I have thought a hundred times as much about the quantum problems as I have about general relativity theory’ (Pais, 1994, p. 57). It should therefore be evident that the most that can be done here is to try to show where the real difficulties lie, and how the failure to recognize that quantum mechanics is an essentially statistical theory has led to a series of confusing quantum paradoxes. It is just these paradoxes that have often been used to support theological speculations. It is useful to ask what ‘understanding quantum mechanics’ actually means. Certainly it means being able to calculate the results of many measurements. The same applies to Newton’s theory of gravitation, which we understand in the sense that we can use it as a basis of calculation, although we do not really understand the gravitational force itself. Very often, especially in modern physics, the mathematical formalism does not give us the sort of intuitive physical feeling for what is going on that we often obtain from classical mechanics. Perhaps we can never achieve more than this for quantum mechanics. In classical mechanics we can assume that a property that is measured objectively exists prior to the interaction of the measuring apparatus with the observed system. Quantum mechanics, however, ‘is incompatible with the proposition that measurement discovers some unknown but pre-existing property’ (Peres, 1997, p. 14). There is an important distinction between mysteries and absurdities. We may be forced to live with mysteries, but there is no reason why we should put up with absurdities such as the wave-particle duality, tunnelling through potential barriers, acausal events and the collapse of the wave function. These are the quantum paradoxes that will be discussed later on. They arise within the Copenhagen interpretation of quantum mechanics and are solved by other interpretations. These alternative interpretations enable us to make conjectures about what is going on, even though we cannot understand it in detail. This is acceptable and a stimulus to further research. It is certainly difficult enough to understand quantum mechanics, but there is no need to try to visualize logical absurdities in the effort to do so. Quantum mechanics is a mathematical formalism that makes it possible to calculate the results of some experiments. Anything more than this, such as questions about the interpretation of what is measured and whether it tells us what is actually going on in the system being studied, belongs to philosophy. Thus we are concerned here with what is essentially a philosophical debate; if it were just a matter of physics the debate would have been settled long ago. One of the difficulties is that it invokes the results of experiments and uses the language of quantum mechanics. Inevitably this language uses philosophical terms and then it is easy to give the impression that the philosophical propositions are in some way entailed by the physical results. These in turn can influence theological thinking. In order to be quite clear which arguments are justified and which are not, it is essential to separate the three levels of discourse: physical, philosophical and theological. Our task is to relate these together, and this immediately raises the traditional problems of the philosophy of science, including the relation of experiment to theory, and of theory to explanation. We begin by discussing experiments, distinguishing between those that are real and those that are imaginary.
When we are presented with apparently bizarre properties of microsystems governed by quantum mechanics (as in our discussion of the double-slit experiment in the previous chapter), we naturally ask whether it might be possible to find another theory, or at least another story to go with the equations, that would make more sense to us. We might seek a theory and a view of fundamental physical processes that would accord more with our classically based, everyday common sense. This typically takes the form of the question of the possibility of the existence of a more detailed description of atomic processes than that afforded by the wave function (that is, by the Copenhagen interpretation of the previous chapter). In this chapter we discuss some of the central interpretive problems of the Copenhagen version of quantum mechanics. Then, in the next two chapters, we look at the severe, general restrictions that are placed on any such completion of quantum mechanics and at one particular extension of quantum mechanics. First, though, we begin here with the struggle that ensued over the status of causality in quantum theory. THE COMPLETENESS OF QUANTUM MECHANICS Although the Heisenberg uncertainty principle (Section 20.4) and the lack of absolute predictive power are an inherent feature of quantum mechanics, there have been attempts to preserve an in principle completely deterministic structure for physics. Even classically, in a macroscopic sample of gas there are so many molecules that it would be hopeless, as a practical matter, to predict the future locations of all these molecules from Newton's laws of motion.
Quantum theory is a phenomenally successful cornerstone of physics. But a recent poll among quantum physicists shows that there is still little agreement on what the theory tells us about physical reality. Yet precious bits of consensus are emerging. As always, Albert Einstein was ahead of his time. Quantum theory was still in its infancy, its formalism a mere sketch, and its implications only beginning to be realized. Yet Einstein, in 1917, already spotted what would trouble him throughout his lifetime. He concluded that the nascent theory, which he had helped initiate, seemed to leave “time and direction of elementary processes,” such as the spontaneous emission of radiation from a molecule, “to chance” 1. Einstein's conclusion was visionary. It was also incredibly bold, for it suggested a radical departure from the powerful, centuries-old deterministic and causal worldview of classical physics. But Einstein also immediately expressed his discomfort with the idea of fundamental chance. In fact, his worries were just the seed of what would turn into one of the longest and most heated debates in the history of physics. What does quantum theory mean, and what does it tell us about the nature of reality? Some of the brightest minds in physics locked horns over these issues. Besides Einstein, there were Niels Bohr, Erwin Schrödinger, Werner Heisenberg, Pascual Jordan, Carl Friedrich von Weizsäcker, Wolfgang Pauli, and Richard Feynman, just to name a few. Ask anyone today working on foundational questions in quantum theory and you are likely to hear that there is still no consensus on many of these questions—all the while, of course, everybody seems to be in perfect agreement on how to apply the quantum formalism when it comes to making experimental predictions. Remarkably, there has never been any serious attempt to find out what the current opinions of the community really are and how they may be correlated. We jumped on the opportunity and polled 33 participants at a 2011 conference on “Quantum Physics and the Nature of Reality” (Fig. 1) 2. (We hasten to add that our poll is to be understood as merely a snapshot and certainly cannot claim to be representative.) As soon as we had collected the questionnaires, the respondents eagerly asked to hear the results. Evidently, we had not been the only curious party. What did we find? The fundamental questions, it seems, still elicit radically different responses. Take the quantum state—represented by the wavefunction—as an illustration. The wavefunction is the central mathematical symbol of quantum mechanics. It allows us to make incredibly accurate predictions, be it in atomic, molecular, or solid-state physics. But how are we to interpret it? Is it itself a physical property, or does it merely represent our knowledge? Our poll revealed a tie between these two positions. And not only that: a third of respondents preferred a mix of both viewpoints—perhaps a sign that a more nuanced view, or even a synthesis of views, may be on the horizon. Only 3% of those polled thought that quantum states never refer to a single system but only to ensembles of systems. Perhaps this departure from the ensemble view—a view that, we suspect, remains more popular in the physics community at large—has to do with the fact that experiments with single quantum systems have become routine in today's quantum-information laboratories. Sharp disagreement was in evidence on another deep question. Abraham Pais, Einstein's biographer, recalled that “during one walk Einstein suddenly stopped, turned to me and asked whether I really believed that the moon exists only when I look at it” 3. Putting the issue differently, are the physical properties of objects well defined before we actually measure them? Half of the respondents in our poll believed that they sometimes were, while the other half said that they never were. These two positions correspond to radically different ways of understanding the function of measurement: either as simply ascertaining—at least sometimes—of what is already there, or as an act of genuine creation. The role of measurement is also at the heart of another perennial favorite of foundational discussions, the measurement problem. The measurement problem highlights a curious fact about quantum theory: while the act of measurement assumes a fundamental role, it is not described—not without interpretive contortions, anyway—by the dynamical law of quantum theory, the Schrödinger equation. For decades, the measurement problem was seen by many as the main stumbling block for quantum theory, sparking a storm of rivaling attempts to deal with this perceived issue. Today, the waters seem to have calmed and attention shifted elsewhere, as only less than a fourth of respondents consider the measurement problem “a severe difficulty threatening quantum mechanics.” Changing attitudes are also evident when it comes to the issue of chance Einstein had brought to light in 1917. Today, the question of the randomness of quantum events no longer seems to be all that contentious. Most of our respondents accepted randomness as irreducible or a fundamental concept in nature (or both). Less than one-tenth of respondents thought that quantum randomness was merely apparent rather than fundamental, and no one believed in a hidden determinism governing quantum events. We wish we could hear Einstein's opinion today. Why have attitudes and priorities changed? It's hard to say, but we think there may be several driving forces. The rapid ascent of quantum information science has not only led to applications such as secure cryptography and the promise of lightning-fast quantum computers, but has also shed new light on foundational problems and handed us new tools for addressing them. Our poll shows that these tools are considered a welcome addition, with three-fourths seeing quantum information as a breath of fresh air for foundational investigations. There is also optimism about the future technological development of quantum information, with more than 75% of respondents expecting a working and useful quantum computer to be realized within the next fifty years. Such optimism might be rooted in the astonishing experimental progress we have witnessed over the past decades, another driving force behind changing attitudes. We are now at stage where almost perfect experimental control of single quantum systems has been achieved. The 2012 Nobel Prize in physics was awarded for just such experiments. This has put at our fingertips the ability to actually carry out the thought experiments—or approximations thereof, at any rate—that the founders of quantum theory had dreamed up. Take Schrödinger's cat. In 1935, Schrödinger wanted to illustrate what seemed to him an absurd prediction of the new theory: that the strange quantum superpositions of the atomic world can be amplified to everyday objects. Schrödinger's object of choice was a cat in a specifically prepared box. Quantum theory then suggested that the state of the cat should become transformed into a bizarre quantum superposition of being simultaneously alive and dead. But what Schrödinger had intended as a “burlesque” 4 thought experiment is now approaching experimental reality. Over the last decades, researchers have become increasingly skilled in breeding small versions of Schrödinger's cat. While they don't involve actual cats, they do already involve large biological molecules. Our respondents, it seems, were inspired by such developments. A solid two-thirds majority declared that they don't expect any fundamental limit in this enterprise of observing larger and larger objects in superpositions of macroscopically distinct states. No limit, then, for quantum theory—something that Bohr had already suggested when he, in some of his early thought experiments, considered apparatuses that included macroscopic parts behaving in quantum-mechanical ways. There are other new trends. Why do people seem to be much more at ease with relativity theory than with quantum theory? For one, both special and general relativity theory are grounded in a few simple physical principles. By contrast, quantum theory is lacking such a foundation and is instead based on a rather abstract mathematical body. Today, several researchers are out to change this situation, trying to derive quantum theory from a small set of physical or information-theoretic principles. They hope that in this way, we will understand why we have this peculiar theory to begin with. A majority in our poll regarded such “reconstructions” of quantum theory as giving useful insights, indicating once more that any new tool that promises fresh insights is embraced with an open mind. To be sure, many (albeit not all) of the questions we asked in the poll are interpretive questions. This means that unless we should find out some day that quantum theory is in fact not always valid—in which case we would have to rethink the whole game—experiments arguably can't play the role of the arbiter. Some physicists will conclude from this that we need not bother with such questions, and relegate them to philosophy. We think this is too quick. Many ideas in physics didn't come out of thin air, but came about when someone started looking at a problem with a certain philosophical disposition. In this sense, we may think of the plurality of views in quantum mechanics as productive. Every interpretation highlights some aspect of quantum theory, is particularly suited for explaining particular phenomena, and may inspire new technologies—such as the quantum computer—or even new physics. What, then, does the future hold? Half of the respondents in our poll believe that in fifty years from now, we will probably still be coming together for conferences on the fundamental questions of quantum theory. But what kinds of questions will be on the agenda? Will we still be worrying about the same problems as today? Or will these problems have been solved—or found to be less relevant—and our focus shifted to new questions once again? Who knows. But the location of the conference where we conducted the poll may tell a story. It was the place of a monastery, and a central position in the battle between Catholics and Protestants during the seventeenth century. Many of the questions that were hotly debated back then are no longer of interest to many people today, and other, new questions have since taken center stage in our society. Maybe some of the battles over the interpretation of quantum theory will meet a similar fate.
Introduction Despite its extraordinary predictive successes, quantum mechanics has, since its inception some seventy years ago, been plagued by conceptual difficulties. The basic problem, plainly put, is this: It is not at all clear what quantum mechanics is about. What, in fact, does quantum mechanics describe? It might seem, since it is widely agreed that the state of any quantum mechanical system is completely specified by its wave function, that quantum mechanics is fundamentally about the behavior of wave functions. Quite naturally, no physicist wanted this to be true more than did Erwin Schrodinger, the father of the wave function. Nonetheless, Schrodinger ultimately found this impossible to believe. His difficulty was not so much with the novelty of the wave function [2, page 156 of [3]]: "That it is an abstract, unintuitive mathematical construct is a scruple that almost always surfaces against new aids to thought and that carries no great message." Rather, it was that
The dominance of the Copenhagen interpretation is the result of a historical accident (Cushing, 1994). At the Solvay conference in 1927, Louis de Broglie put forward the deterministic pilot wave theory, described later. Never slow to criticize, Pauli jumped up and demolished de Broglie’s proposal, and de Broglie was so discouraged by this that he did not develop his interpretation any further. The Copenhagen interpretation was then developed by Bohr, Heisenberg, Pauli and many others, as described in the previous chapter. It was strongly supported by von Neumann’s proof of the impossibility of hidden variables. Years later, in 1952, Bohm published a deterministic hidden variable theory, showing that there must be something wrong with von Neumann’s proof. The error was found by John Bell in 1966, as described in the last chapter, and he also showed how Pauli’s objections to de Broglie’s original idea can be answered. If this had been done at the conference in 1927, quantum mechanics could have been interpreted deterministically from the beginning. The essential mistake in the Copenhagen interpretation is to treat it as a complete account of the behaviour of each individual system. It is frequently claimed that it enables us to calculate, at least in principle, everything that can be measured. Thus Hooft (1997, p. 11) said that ‘The laws of quantum mechanics have been formulated very accurately. We know exactly how to compute anything we would like to know.’ Also Peierls (1997, p. 25) has said that quantum mechanics ‘had become a complete and consistent scheme capable of giving a unique answer to any questions relating to actual or possible observations’. Even if this were true, it would not imply that it is the final complete theory, for it is always possible that some new phenomenon might be found that cannot be calculated quantum-mechanically. However, it is not true: there are many phenomena such as the time of decay of a radioactive nucleus, or the direction a particle is scattered by a nucleus, that cannot be calculated, even in principle. Quantum mechanics is therefore an incomplete theory. We can only calculate the statistical properties of these systems, such as the half-life of a radioactive decay or the differential scattering cross-section that gives the probabilities that a particle is scattered through various angles. All measurements of quantum systems are of this statistical character. It might be thought that the possibility of making measurements on a single electron provides an exception to this. However, even in such cases the electron is continually bathed in a fluctuating background radiation from nearby atoms. Since this is variable, that electron is a member of an ensemble of electrons subject to different fluctuations. All these examples clearly demonstrate the statistical character of quantum mechanics. ‘I am rather firmly convinced’, Einstein (in Schilpp, 1949, pp. 666, 671) remarked, ‘that the essentially statistical character of contemporary quantum theory is solely to be ascribed to fact that this theory operates with an incomplete description of physical systems’. As a result, ‘the c-function is to be understood as the description not of a single system but of an ensemble of systems’. So, ‘if the statistical quantum theory does not pretend to describe the individual system completely, it appears unavoidable to look elsewhere for a complete description of the individual systems’. Thus ‘the difficulties of theoretical interpretation disappear, if one views the quantum-mechanical description as a description of an ensemble of systems’. If this is achieved, ‘the statistical theory would, within the framework of future physics, take an approximately analogous position to statistical mechanics within the framework of classical mechanics’. ‘Thus, essentially, nothing has changed since Galileo or Newton or Faraday concerning the status of the ‘‘observer’’ or our ‘‘consciousness’’ or of our ‘‘information’’ in physics’ (Popper, 1982, p. 46). Once this is accepted, the quantum paradoxes that plagued the Copenhagen interpretation are easily resolved, as described below. In any textbook of quantum mechanics problems such as the calculation of the energy levels of the hydrogen atom are solved as if that hydrogen atom is alone in the universe. As Feynman (1972) remarked, ‘when we solve a quantum-mechanical problem, what we really do is to divide the universe into two parts – the system in which we are interested and the rest of the universe. We then usually act as if the system in which we are interested comprised the whole universe.’ In fact, however, every hydrogen atom on which measurements are made is surrounded by other atoms and exposed to their radiations. Quantum mechanics somehow takes this into account and gives the average behaviour of an ensemble of hydrogen atoms. A more detailed theory takes these influences into account. One attempt to do this is stochastic electrodynamics, described in a later section.
Unlike special relativity, quantum mechanics was not the inspired product of one mind. It developed, in fits and starts, over a period of 25 years (1900–1925), and many cooks stirred the broth. Even now, a century later, quantum mechanics raises profound conceptual questions. Every competent physicist can “do” quantum mechanics, and the predictions it makes are in spectacularly good agreement with experimental results; there can be little doubt that quantum mechanics is “right.” On the other hand, as Richard Feynman once remarked, “nobody understands quantum mechanics.” I'll tell the story in two parts. First the history (how the essential pieces of the puzzle were assembled), and then the implications (what the finished picture has to say about our world). Photons Like relativity, quantum mechanics began with the study of light. In both cases, this historical association is misleading. Relativity, really, has nothing to do with light – to be sure, it involves that magic speed limit, c , and of course the most familiar thing that travels at speed c is light. So light is a convenient vehicle for introducing the subject. But it is perfectly possible to imagine a universe in which there is no such thing as light, and yet relativity might still be valid. Quantum mechanics, too, starts with light – in this case the “quantum” of light (the photon). But, as it turns out, this is just one rather specialized example of much more general principles – in fact, a rather bad example, because photons are by their nature relativistic (they travel at the speed of light), and it would be more natural to begin with the quantum mechanics of slow-moving objects, in analogy with Newtonian mechanics.