Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2609.22965

Exact Realization of Deep ReLU Computation in a Microscopic Statistical-Mechanical System

Junxu Li

quant-phmath-phSubmitted Sep 19, 2026• Announced Sep 22, 2026• 0 citations
arXivHTMLPDF

Abstract

Understanding the physical origin of the piecewise-linear computation in deep rectified linear unit (ReLU) networks remains a fundamental challenge. Here we establish an exact statistical-mechanical realization of arbitrary ReLU networks. Starting from a microscopic configuration space and its state multiplicities, we construct a partition function without prescribing a neural-network activation function. The resulting system admits equivalent descriptions in terms of cascaded quantum operations with post selection, or fermionic transport model. We rigorously prove that in $β\to+\infty$ limit, the thermodynamic observables of this system exactly reproduce the hidden states, outputs, and loss function of an arbitrary deep ReLU network. Crucially, we demonstrate that the piecewise-linear behavior of the ReLU network emerges from a first-order phase transition in the microscopic system, where the discontinuities precisely coincide with the boundaries of linear regions of ReLU network. This work establishes an exact physical foundation for deep neural computation and provides a novel statistical-mechanical perspective on neural architecture design.

Comments
7 pages, 3 figures

Related papers

Deep Learning and Quantum Physics : A Fundamental Bridge

Yoav Levine, David Yakira, Nadav Cohen +1 more

Deep convolutional networks have witnessed unprecedented success in various machine learning applications. Formal understanding on what makes these networks so successful is gradually unfolding, but for the most part there are still significant mysteries to unravel. The inductive bias, which reflects prior knowledge embedded in the network architecture, is one of them. In this work, we establish a fundamental connection between the fields of quantum physics and deep learning. We use this connection for asserting novel theoretical observations regarding the role that the number of channels in each layer of the convolutional network fulfills in the overall inductive bias. Specifically, we show an equivalence between the function realized by a deep convolutional arithmetic circuit (ConvAC) and a quantum many-body wave function, which relies on their common underlying tensorial structure. This facilitates the use of quantum entanglement measures as well-defined quantifiers of a deep network's expressive ability to model intricate correlation structures of its inputs. Most importantly, the construction of a deep ConvAC in terms of a Tensor Network is made available. This description enables us to carry a graph-theoretic analysis of a convolutional network, with which we demonstrate a direct control over the inductive bias of the deep network via its channel numbers, that are related to the min-cut in the underlying graph. This result is relevant to any practitioner designing a network for a specific task. We theoretically analyze ConvACs, and empirically validate our findings on more common ConvNets which involve ReLU activations and max pooling. Beyond the results described above, the description of a deep convolutional network in well-defined graph-theoretic tools and the formal connection to quantum entanglement, are two interdisciplinary bridges that are brought forth by this work.

arXiv:1704.01552v1?source=post_page---------------------------•Apr 5, 2017•1 citation

Deep Learning and Quantum Entanglement: Fundamental Connections with Implications to Network Design

Yoav Levine, David Yakira, Nadav Cohen +1 more

Deep convolutional networks have witnessed unprecedented success in various machine learning applications. Formal understanding on what makes these networks so successful is gradually unfolding, but for the most part there are still significant mysteries to unravel. The inductive bias, which reflects prior knowledge embedded in the network architecture, is one of them. In this work, we establish a fundamental connection between the fields of quantum physics and deep learning. We use this connection for asserting novel theoretical observations regarding the role that the number of channels in each layer of the convolutional network fulfills in the overall inductive bias. Specifically, we show an equivalence between the function realized by a deep convolutional arithmetic circuit (ConvAC) and a quantum many-body wave function, which relies on their common underlying tensorial structure. This facilitates the use of quantum entanglement measures as well-defined quantifiers of a deep network's expressive ability to model intricate correlation structures of its inputs. Most importantly, the construction of a deep ConvAC in terms of a Tensor Network is made available. This description enables us to carry a graph-theoretic analysis of a convolutional network, with which we demonstrate a direct control over the inductive bias of the deep network via its channel numbers, that are related to the min-cut in the underlying graph. This result is relevant to any practitioner designing a network for a specific task. We theoretically analyze ConvACs, and empirically validate our findings on more common ConvNets which involve ReLU activations and max pooling. Beyond the results described above, the description of a deep convolutional network in well-defined graph-theoretic tools and the formal connection to quantum entanglement, are two interdisciplinary bridges that are brought forth by this work.

arXiv:1704.01552•Apr 5, 2017•61 citations

Quantum-Inspired Neural Network Activation Functions

Jincheng Zhang

This paper explores a novel approach to neural network activation functions by drawing inspiration from quantum mechanical principles. Traditional activation functions, such as sigmoid and ReLU, often exhibit limitations in terms of learning efficiency and representational capacity. We hypothesize that classical activation functions can be effectively approximated by functions derived from quantum mechanics, specifically utilizing concepts like the quantum harmonic oscillator. The proposed activation functions aim to leverage the unique properties of quantum systems—superposition, entanglement, and tunneling—to potentially enhance neural network performance. We introduce a set of new activation functions and provide a theoretical framework for their design. The core claim of this work is that classical activation functions can be accurately approximated by quantum mechanical functions. The primary mechanism involves translating quantum mechanical models into differentiable functions suitable for use within a neural network architecture. We discuss the mathematical foundations, potential benefits, and challenges associated with this approach, focusing on the approximation of the quantum harmonic oscillator and its application to neural network activation. The research contributes to a fundamental shift in activation function design, moving beyond purely classical models towards a quantum-inspired paradigm.

•Sep 3, 2026•0 citations

Neural Network Statistical Mechanics

Lingxiao Wang, Yin Jiang, Kai Zhou

We propose a general framework to extract microscopic interactions from raw configurations with deep neural networks. The approach replaces the modeling Hamiltonian by the neural networks, in which the interaction is encoded. It can be trained with data collected from Ab initio computations or experiments. The well-trained neural networks give an accurate estimation of the possibility distribution of the configurations at fixed external parameters. It can be spontaneously extrapolated to detect the phase structures since classical statistical mechanics as prior knowledge here. We apply the approach to a 2D spin system, training at a fixed temperature, and reproducing the phase structure. Scaling the configuration on lattice exhibits the interaction changes with the degree of freedom, which can be naturally applied to the experimental measurements. Our approach bridges the gap between the real configurations and the microscopic dynamics with an autoregressive neural network.

arXiv:2007.01037•Jul 2, 2020•2 citations

Deep Neural Network Emulation of the Quantum-Classical Transition via Learned Wigner Function Dynamics

Kamran Majid

Abstract The emergence of classical behavior from quantum mechanics as Planck’s constant ℏ approaches zero remains a fundamental challenge in physics [1-3]. This paper introduces a novel approach employing deep neural networks to directly learn the dynamical mapping from initial quantum state parameters (for Gaussian wave packets of the one-dimensional harmonic oscillator) and ℏ to the parameters of the time-evolved Wigner function in phase space [4-6]. A comprehensive dataset of analytically derived time-evolved Wigner functions was generated, and a deep feedforward neural network with an enhanced architecture was successfully trained for this prediction task, achieving a final training loss of ∼ 0.0390. The network demonstrates a significant and previously unrealized ability to accurately capture the underlying mapping of the Wigner function dynamics. This allows for a direct emulation of the quantum-classical transition by predicting the evolution of phase-space distributions as ℏ is systematically varied. The implications of these findings for providing a new computational lens on the emergence of classicality are discussed, highlighting the potential of this direct phase-space learning approach for studying fundamental aspects of quantum mechanics. This work presents a significant advancement beyond previous efforts that focused on learning observable mappings [7], offering a direct route via the phase-space representation.

arXiv:2504.16334•Apr 25, 2025•1 citation

Fermi-Dirac machines as quantizations of neurons

Alexander He, Nana Liu, Mark M. Wilde

Fermi-Dirac machines were proposed recently as an approach to solving semidefinite optimization problems on quantum computers. Here, we reinterpret them as canonical quantizations of classical neurons. By viewing a classical neuron as an activation function applied to a parameterized classical Hamiltonian, we quantize this model by replacing classical variables with operators whose eigenvalues encode their possible values. This follows the standard approach to canonical quantization in quantum mechanics. Crucially, when the Hamiltonian consists of commuting operators, our construction reduces exactly to a classical neuron. More generally, our approach yields an activation observable, defined as an activation function applied to a parameterized quantum Hamiltonian. The output of this quantized neuron is a random variable with expectation value equal to that of the activation observable with respect to an input state. We develop efficient hybrid quantum-classical algorithms for evaluating outputs and gradients of our quantized neurons, enabling evaluation and training. These algorithms rely on basic primitives that include random sampling, Hamiltonian simulation, and the Hadamard test. We also quantize a whole host of other activation functions, including the smooth rectified linear unit (ReLU), sigmoid linear unit, Gaussian-smoothed ReLU, and Gaussian error linear unit (GeLU), which are known to be useful for deep learning applications. Numerical experiments indicate that neurons based on quantum Hamiltonians can learn functions that classical neurons cannot. We further define a computational decision problem based on Fermi-Dirac neurons and prove that it is BQP-complete, providing complexity-theoretic evidence against efficient classical simulation. Finally, we generalize our approach to continuous quantum variables and sketch two different ways of composing these neurons into networks.

quant-pharXiv:2605.24386•May 26, 2026•0 citations
Understanding the physical origin of the piecewise-linear computation in deep rectified linear unit (ReLU) networks remains a fundamental challenge. Here we establish an exact statistical-mechanical realization of arbitrary ReLU networks. Starting from a microscopic configuration space and its state multiplicities, we construct a partition function without prescribing a neural-network activation function. The resulting system admits equivalent descriptions in terms of cascaded quantum operations with post selection, or fermionic transport model. We rigorously prove that in β→+∞β\to+\inftyβ→+∞ limit, the thermodynamic observables of this system exactly reproduce the hidden states, outputs, and loss function of an arbitrary deep ReLU network. Crucially, we demonstrate that the piecewise-linear behavior of the ReLU network emerges from a first-order phase transition in the microscopic system, where the discontinuities precisely coincide with the boundaries of linear regions of ReLU network. This work establishes an exact physical foundation for deep neural computation and provides a novel statistical-mechanical perspective on neural architecture design.
Deep convolutional networks have witnessed unprecedented success in various machine learning applications. Formal understanding on what makes these networks so successful is gradually unfolding, but for the most part there are still significant mysteries to unravel. The inductive bias, which reflects prior knowledge embedded in the network architecture, is one of them. In this work, we establish a fundamental connection between the fields of quantum physics and deep learning. We use this connection for asserting novel theoretical observations regarding the role that the number of channels in each layer of the convolutional network fulfills in the overall inductive bias. Specifically, we show an equivalence between the function realized by a deep convolutional arithmetic circuit (ConvAC) and a quantum many-body wave function, which relies on their common underlying tensorial structure. This facilitates the use of quantum entanglement measures as well-defined quantifiers of a deep network's expressive ability to model intricate correlation structures of its inputs. Most importantly, the construction of a deep ConvAC in terms of a Tensor Network is made available. This description enables us to carry a graph-theoretic analysis of a convolutional network, with which we demonstrate a direct control over the inductive bias of the deep network via its channel numbers, that are related to the min-cut in the underlying graph. This result is relevant to any practitioner designing a network for a specific task. We theoretically analyze ConvACs, and empirically validate our findings on more common ConvNets which involve ReLU activations and max pooling. Beyond the results described above, the description of a deep convolutional network in well-defined graph-theoretic tools and the formal connection to quantum entanglement, are two interdisciplinary bridges that are brought forth by this work.
Deep convolutional networks have witnessed unprecedented success in various machine learning applications. Formal understanding on what makes these networks so successful is gradually unfolding, but for the most part there are still significant mysteries to unravel. The inductive bias, which reflects prior knowledge embedded in the network architecture, is one of them. In this work, we establish a fundamental connection between the fields of quantum physics and deep learning. We use this connection for asserting novel theoretical observations regarding the role that the number of channels in each layer of the convolutional network fulfills in the overall inductive bias. Specifically, we show an equivalence between the function realized by a deep convolutional arithmetic circuit (ConvAC) and a quantum many-body wave function, which relies on their common underlying tensorial structure. This facilitates the use of quantum entanglement measures as well-defined quantifiers of a deep network's expressive ability to model intricate correlation structures of its inputs. Most importantly, the construction of a deep ConvAC in terms of a Tensor Network is made available. This description enables us to carry a graph-theoretic analysis of a convolutional network, with which we demonstrate a direct control over the inductive bias of the deep network via its channel numbers, that are related to the min-cut in the underlying graph. This result is relevant to any practitioner designing a network for a specific task. We theoretically analyze ConvACs, and empirically validate our findings on more common ConvNets which involve ReLU activations and max pooling. Beyond the results described above, the description of a deep convolutional network in well-defined graph-theoretic tools and the formal connection to quantum entanglement, are two interdisciplinary bridges that are brought forth by this work.
This paper explores a novel approach to neural network activation functions by drawing inspiration from quantum mechanical principles. Traditional activation functions, such as sigmoid and ReLU, often exhibit limitations in terms of learning efficiency and representational capacity. We hypothesize that classical activation functions can be effectively approximated by functions derived from quantum mechanics, specifically utilizing concepts like the quantum harmonic oscillator. The proposed activation functions aim to leverage the unique properties of quantum systems—superposition, entanglement, and tunneling—to potentially enhance neural network performance. We introduce a set of new activation functions and provide a theoretical framework for their design. The core claim of this work is that classical activation functions can be accurately approximated by quantum mechanical functions. The primary mechanism involves translating quantum mechanical models into differentiable functions suitable for use within a neural network architecture. We discuss the mathematical foundations, potential benefits, and challenges associated with this approach, focusing on the approximation of the quantum harmonic oscillator and its application to neural network activation. The research contributes to a fundamental shift in activation function design, moving beyond purely classical models towards a quantum-inspired paradigm.
We propose a general framework to extract microscopic interactions from raw configurations with deep neural networks. The approach replaces the modeling Hamiltonian by the neural networks, in which the interaction is encoded. It can be trained with data collected from Ab initio computations or experiments. The well-trained neural networks give an accurate estimation of the possibility distribution of the configurations at fixed external parameters. It can be spontaneously extrapolated to detect the phase structures since classical statistical mechanics as prior knowledge here. We apply the approach to a 2D spin system, training at a fixed temperature, and reproducing the phase structure. Scaling the configuration on lattice exhibits the interaction changes with the degree of freedom, which can be naturally applied to the experimental measurements. Our approach bridges the gap between the real configurations and the microscopic dynamics with an autoregressive neural network.
Abstract The emergence of classical behavior from quantum mechanics as Planck’s constant ℏ approaches zero remains a fundamental challenge in physics [1-3]. This paper introduces a novel approach employing deep neural networks to directly learn the dynamical mapping from initial quantum state parameters (for Gaussian wave packets of the one-dimensional harmonic oscillator) and ℏ to the parameters of the time-evolved Wigner function in phase space [4-6]. A comprehensive dataset of analytically derived time-evolved Wigner functions was generated, and a deep feedforward neural network with an enhanced architecture was successfully trained for this prediction task, achieving a final training loss of ∼ 0.0390. The network demonstrates a significant and previously unrealized ability to accurately capture the underlying mapping of the Wigner function dynamics. This allows for a direct emulation of the quantum-classical transition by predicting the evolution of phase-space distributions as ℏ is systematically varied. The implications of these findings for providing a new computational lens on the emergence of classicality are discussed, highlighting the potential of this direct phase-space learning approach for studying fundamental aspects of quantum mechanics. This work presents a significant advancement beyond previous efforts that focused on learning observable mappings [7], offering a direct route via the phase-space representation.
Fermi-Dirac machines were proposed recently as an approach to solving semidefinite optimization problems on quantum computers. Here, we reinterpret them as canonical quantizations of classical neurons. By viewing a classical neuron as an activation function applied to a parameterized classical Hamiltonian, we quantize this model by replacing classical variables with operators whose eigenvalues encode their possible values. This follows the standard approach to canonical quantization in quantum mechanics. Crucially, when the Hamiltonian consists of commuting operators, our construction reduces exactly to a classical neuron. More generally, our approach yields an activation observable, defined as an activation function applied to a parameterized quantum Hamiltonian. The output of this quantized neuron is a random variable with expectation value equal to that of the activation observable with respect to an input state. We develop efficient hybrid quantum-classical algorithms for evaluating outputs and gradients of our quantized neurons, enabling evaluation and training. These algorithms rely on basic primitives that include random sampling, Hamiltonian simulation, and the Hadamard test. We also quantize a whole host of other activation functions, including the smooth rectified linear unit (ReLU), sigmoid linear unit, Gaussian-smoothed ReLU, and Gaussian error linear unit (GeLU), which are known to be useful for deep learning applications. Numerical experiments indicate that neurons based on quantum Hamiltonians can learn functions that classical neurons cannot. We further define a computational decision problem based on Fermi-Dirac neurons and prove that it is BQP-complete, providing complexity-theoretic evidence against efficient classical simulation. Finally, we generalize our approach to continuous quantum variables and sketch two different ways of composing these neurons into networks.