We consider the classical dynamics of bosonic and fermionic matrix variables in complex Hilbert space, defined by a trace action, assuming cyclic invariance under the trace and the presence of a global unitary invariance. With plausible and explicitly stated assumptions, including the existence of a large hierarchy of scale between the underlying dynamics and observed physics, we show that (1) the equilibrium statistical mechanics of this matrix dynamics, in the canonical ensemble, gives rise to an emergent quantum mechanics for many degrees of freedom, including the standard canonical commutation/anticommutation relations and the usual unitary Heisenberg and Schrödinger picture time evolutions of operators and states, and (2) the fluctuation or Brownian motion corrections to this thermodyamics lead to an energy-driven stochastic modification of the Schrödinger equation, which is known to imply state vector reduction with Born rule probabilities. Thus, quantum mechanics and its probabilistic interpretation both arise as emergent phenomena in the underlying trace dynamics.
We consider the classical dynamics of bosonic and fermionic matrix variables in complex Hilbert space, defined by a trace action, assuming cyclic invariance under the trace and the presence of a global unitary invariance. With plausible and explicitly stated assumptions, including the existence of a large hierarchy of scale between the underlying dynamics and observed physics, we show that (1) the equilibrium statistical mechanics of this matrix dynamics, in the canonical ensemble, gives rise to an emergent quantum mechanics for many degrees of freedom, including the standard canonical commutation/anticommutation relations and the usual unitary Heisenberg and Schrödinger picture time evolutions of operators and states, and (2) the fluctuation or Brownian motion corrections to this thermodyamics lead to an energy-driven stochastic modification of the Schrödinger equation, which is known to imply state vector reduction with Born rule probabilities. Thus, quantum mechanics and its probabilistic interpretation both arise as emergent phenomena in the underlying trace dynamics.
In this chapter we set up a classical Lagrangian and Hamiltonian dynamics for matrix models. The fundamental idea is to set up an analog of classical dynamics in which the phase space variables are non-commutative, and the basic tool that allows one to accomplish this is cyclic invariance under a trace. Since no assumptions about commutativity of the phase space variables (such as canonical commutators/anticommutators) are made at this stage, the dynamics that we set up is not the same as standard quantum mechanics. Quantum mechanical behavior will be seen to emerge only when, in Chapters 4 and 5, we study the statistical mechanics of the classical matrix dynamics formulated here. In Section 1.1, we introduce our basic notation for bosonic and fermionic matrices, and give the cyclic identities that will be used repeatedly throughout the book. In Section 1.2, we define the derivative of a trace quantity with respect to an operator, and give the basic properties of this definition. In Section 1.3, we use the operator derivative to formulate a Lagrangian and Hamiltonian dynamics for matrix models. In Section 1.4, we introduce a generalized Poisson bracket appropriate to trace dynamics, constructed from the operator derivative defined in Section 1.2, and give its properties and some applications. Finally, in Section 1.5 we discuss the relation between the trace dynamics time evolution equations, and the usual unitary Heisenberg picture equations of motion obtained when one assumes standard canonical commutators/anticommutators. Bosonic and fermionic matrices and the cyclic trace identities We shall assume finite-dimensional matrices, although ultimately an extension to the infinite-dimensional case may be needed. The matrix elements of these matrices will be constructed from ordinary complex numbers, and from complex anticommuting Grassmann numbers.
I review the proposal made in my 2004 book [1], that quantum theory is an emergent theory arising from a deeper level of dynamics. The dynamics at this deeper level is taken to be an extension of classical dynamics to non-commuting matrix variables, with cyclic permutation inside a trace used as the basic calculational tool. With plausible assumptions, quantum theory is shown to emerge as the statistical thermodynamics of this underlying theory, with the canonical commutation-anticommutation relations derived from a generalized equipartition theorem. Brownian motion corrections to this thermodynamics are argued to lead to state vector reduction and to the probabilistic interpretation of quantum theory, making contact with phenomenological proposals [2, 3] for stochastic modifications to Schrödinger dynamics.
Quantum mechanics is our most successful physical theory. However, it raises conceptual issues that have perplexed physicists and philosophers of science for decades. This 2004 book develops an approach, based on the proposal that quantum theory is not a complete, final theory, but is in fact an emergent phenomenon arising from a deeper level of dynamics. The dynamics at this deeper level are taken to be an extension of classical dynamics to non-commuting matrix variables, with cyclic permutation inside a trace used as the basic calculational tool. With plausible assumptions, quantum theory is shown to emerge as the statistical thermodynamics of this underlying theory, with the canonical commutation/anticommutation relations derived from a generalized equipartition theorem. Brownian motion corrections to this thermodynamics are argued to lead to state vector reduction and to the probabilistic interpretation of quantum theory, making contact with phenomenological proposals for stochastic modifications to Schrödinger dynamics.
We develop a state-space geometric framework for measurement, classicality, and quantum paradoxes, based on one dynamical conjecture. Classical configuration space and classical phase space for a mechanical system arise as distinguished submanifolds of projective quantum state space. On these submanifolds, the Fubini--Study geometry induces Euclidean classical geometry, and the tangent component of Schrödinger evolution reproduces Newtonian dynamics. Within this framework, interactions with measuring devices and environments are described by random-matrix dynamics on projective state space, generated by matrices drawn from the Gaussian Unitary Ensemble. We show that this random-matrix dynamics yields isotropic diffusion, giving Born-rule transition probabilities in microscopic measurements and stabilizing classical behavior in macroscopic systems. We further argue that the random-matrix conjecture is not an independent ad hoc assumption: under natural translation-invariance assumptions on the distribution of state-space steps originating on the classical submanifold, the unitary lift of homogeneous and isotropic Brownian motion on that submanifold is uniquely given by the Gaussian Unitary Ensemble, up to scale and an irrelevant scalar part. The resulting framework provides a unitary account of measurement and the quantum-to-classical transition and, if accepted, offers a dynamical resolution of standard quantum paradoxes.
We address the issue of when generalized quantum dyanmics, based on a total trace Hamiltonian {\\bf H}, reduces to Heisenberg picture complex quantum mechanics. We begin by showing that when {\\bf H}={\\rm{\\bf Tr}} H_{OP}, with H_{OP} a Weyl ordered operator Hamiltonian, then the generalized quantum dynamics operator equations of motion agree with those obtained from H_{OP} in the Heisenberg picture by using canonical commutation relations. Hence generalized quantum dynamics conicides with complex quantum mechanics in this case, provided the initial values of the operators obey the canonical algebra. The remainder of the paper is devoted to a study of when this initial value condition is satisfied. We first show that for any total trace Hamiltonian which involves no noncommutative constants, there is a conserved operator \\tilde C which is closely related to the canonical algebra. We demonstrate that for a system in equilbrium, the equilbrium ensemble of initial values is governed by the distribution exp(-|\\tilde C|/\\kappa). An argument analogous to that for equipartition of energy, together with the assumption that ensemble averages of products can be replaced by products of ensemble averages, then implies that the ensemble averaged operator initial values obey the standard canonical commutation relations of complex quantum mechanics, with the imaginary unit I identified as -\\tilde C / |\\tilde C|, and with the constant \\kappa playing the role of the reduced Planck constant \\hbar. Thus there is a well--defined sense in which complex quantum mechanics and quantum field theory emerge as a statistical approximation to generalized quantum dynamics.
We develop a dynamical framework for quantum measurement based on stochastic but unitary evolution in projective state space. Random Hamiltonians drawn from the Gaussian Unitary Ensemble generate stochastic unitary dynamics of the quantum state, while equivalence classes reflecting finite detector resolution define classical observables as well as classical configuration-space and phase-space submanifolds. When the evolution is constrained to the phase-space submanifold, free Schrödinger dynamics reduces to Newtonian motion, while stochastic motion constrained to the classical configuration-space submanifold yields ordinary Brownian motion in classical space. Transition probabilities under the stochastic dynamics satisfy the Born rule, whereas the constrained classical evolution produces the normal probability distributions characteristic of classical measurements. We show that, in this setting, measurement, state reduction, and the quantum-classical transition emerge from unitary dynamics alone, without invoking nonunitary collapse or additional postulates. Entanglement and EPR correlations arise geometrically from the evolution of joint states in composite state space, preserving locality in spacetime. The framework provides a unified dynamical account of measurement and classicality compatible with the structure of quantum mechanics.