Understanding the physical origin of the piecewise-linear computation in deep rectified linear unit (ReLU) networks remains a fundamental challenge. Here we establish an exact statistical-mechanical realization of arbitrary ReLU networks. Starting from a microscopic configuration space and its state multiplicities, we construct a partition function without prescribing a neural-network activation function. The resulting system admits equivalent descriptions in terms of cascaded quantum operations with post selection, or fermionic transport model. We rigorously prove that in $β\to+\infty$ limit, the thermodynamic observables of this system exactly reproduce the hidden states, outputs, and loss function of an arbitrary deep ReLU network. Crucially, we demonstrate that the piecewise-linear behavior of the ReLU network emerges from a first-order phase transition in the microscopic system, where the discontinuities precisely coincide with the boundaries of linear regions of ReLU network. This work establishes an exact physical foundation for deep neural computation and provides a novel statistical-mechanical perspective on neural architecture design.Understanding the physical origin of the piecewise-linear computation in deep rectified linear unit (ReLU) networks remains a fundamental challenge. Here we establish an exact statistical-mechanical realization of arbitrary ReLU networks. Starting from a microscopic configuration space and its state multiplicities, we construct a partition function without prescribing a neural-network activation function. The resulting system admits equivalent descriptions in terms of cascaded quantum operations with post selection, or fermionic transport model. We rigorously prove that in $β\to+\infty$ limit, the thermodynamic observables of this system exactly reproduce the hidden states, outputs, and loss function of an arbitrary deep ReLU network. Crucially, we demonstrate that the piecewise-linear behavior of the ReLU network emerges from a first-order phase transition in the microscopic system, where the discontinuities precisely coincide with the boundaries of linear regions of ReLU network. This work establishes an exact physical foundation for deep neural computation and provides a novel statistical-mechanical perspective on neural architecture design.

quant-pharXiv:2609.22965Sep 22, 20260 citations

On the independence of the thermodynamic limit on the boundary conditions in quantum statistical mechanics

N. Angelescu, G. Nenciu

Communications in Mathematical PhysicsMar 1, 197325 citations

Proof of no Johnson noise at zero temperature

László B. Kish, Gunnar A. Niklasson, Claes‐Göran Granqvist

The Callen-Welton formula (fluctuation-dissipation theorem) of voltage and current noise of a resistance are the sum of Nyquist's classical Johnson noise equations and a (quantum) zero-point term with power density spectrum proportional to frequency and independent of temperature. At zero temperature, the classical Nyquist term vanishes however the zero-point term produces non-zero noise voltage and current. We show that the claim of zero-point noise directly contradicts to the Fermi-Dirac distribution, which defines the thermodynamics of electrons according to quantum-statistical physics. As a consequence, the Johnson noise must be zero at zero temperature, which is in accordance with Nyquist's original formula. Further investigation shows that the Callen-Welton derivation has conceptual errors such as neglecting phonon scattering, disregarding the Pauli principle during calculating the transition probabilities and using bosonic (linear oscillator) energies leading to the zero-point noise artifact. Following Kleen's proposal, the possible origin of the heterodyne (Koch - van Harlingen - Clark) experimental results are also discussed in terms of Heffner theory of quantum noise of frequency/phase-sensitive linear amplifiers. Experiments that failed to see the zero-point noise term are also mentioned.The Callen-Welton formula (fluctuation-dissipation theorem) of voltage and current noise of a resistance are the sum of Nyquist's classical Johnson noise equations and a (quantum) zero-point term with power density spectrum proportional to frequency and independent of temperature. At zero temperature, the classical Nyquist term vanishes however the zero-point term produces non-zero noise voltage and current. We show that the claim of zero-point noise directly contradicts to the Fermi-Dirac distribution, which defines the thermodynamics of electrons according to quantum-statistical physics. As a consequence, the Johnson noise must be zero at zero temperature, which is in accordance with Nyquist's original formula. Further investigation shows that the Callen-Welton derivation has conceptual errors such as neglecting phonon scattering, disregarding the Pauli principle during calculating the transition probabilities and using bosonic (linear oscillator) energies leading to the zero-point noise artifact. Following Kleen's proposal, the possible origin of the heterodyne (Koch - van Harlingen - Clark) experimental results are also discussed in terms of Heffner theory of quantum noise of frequency/phase-sensitive linear amplifiers. Experiments that failed to see the zero-point noise term are also mentioned.

Jun 1, 20150 citations

β\to+\infty

limit, the thermodynamic observables of this system exactly reproduce the hidden states, outputs, and loss function of an arbitrary deep ReLU network. Crucially, we demonstrate that the piecewise-linear behavior of the ReLU network emerges from a first-order phase transition in the microscopic system, where the discontinuities precisely coincide with the boundaries of linear regions of ReLU network. This work establishes an exact physical foundation for deep neural computation and provides a novel statistical-mechanical perspective on neural architecture design.