Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2502.05383

Is attention all you need to solve the correlated electron problem?

Max Geier, Khachatur G. Nazaryan, Timothy Zaklama, Liang Fu

Published Feb 7, 2025• 3 citations
arXivHTMLPDF

Abstract

The attention mechanism has transformed artificial intelligence research by its ability to learn relations between objects. In this work, we explore how a many-body wavefunction ansatz constructed from a large-parameter self-attention neural network can be used to solve the interacting electron problem in solids. By a systematic neural-network variational Monte Carlo study on a moiré quantum material, we demonstrate that the self-attention ansatz provides an accurate and efficient solution without human bias. Moreover, our numerical study finds that the required number of variational parameters scales roughly as $N^2$ with the number of electrons, which opens a path towards efficient large-scale simulations.

DOI
10.48550/arxiv.2502.05383

Related papers

Transferable neural wavefunctions for solids

Leon Gerard, Michael Scherbela, Halvard Sutterud +2 more

Abstract Deep-learning-based variational Monte Carlo has emerged as a highly accurate method for solving the many-electron Schrödinger equation. Despite favorable scaling with the number of electrons, $${\mathcal{O}}({{n}_{{\rm{el}}}}^{4})$$ O ( n el 4 ) , the practical value of deep-learning-based variational Monte Carlo is limited by the high cost of optimizing the neural network weights for every system studied. Recent research has proposed optimizing a single neural network across multiple systems, reducing the cost per system. Here we extend this approach to solids, which require numerous calculations across different geometries, boundary conditions and supercell sizes. We demonstrate that optimization of a single ansatz across these variations significantly reduces optimization steps. Furthermore, we successfully transfer a network trained on 2 × 2 × 2 supercells of LiH, to 3 × 3 × 3 supercells, reducing the number of optimization steps required to simulate the large system by a factor of 50 compared with previous work.

arXiv:2405.07599•Nature Computational Science•Oct 22, 2025•4 citations

Simulating moiré quantum matter with neural network

Di Luo, David D. Dai, Liang Fu

Moiré materials provide an ideal platform for exploring quantum phases of matter. However, solving the many-electron problem in moiré systems is challenging due to strong correlation effects. We introduce a powerful variational representation of quantum states, many-body neural Bloch wavefunction, to solve many-electron problems in moiré materials accurately and efficiently. Applying our method to the semiconductor heterobilayer WSe2/WS2 , we obtain a generalized Wigner crystal at filling factor n = 1/3, a Mott insulator n = 1, and a correlated insulator with local magnetic moments and antiferromagnetic spin correlation at n = 2. Our neural network approach improves the simulation accuracy of strongly interacting moiré materials and paves the way for discovery of new quantum phases with variational learning principle in a unified framework.

arXiv:2406.17645•Jun 25, 2024•2 citations

A Self-Attention Ansatz for Ab-initio Quantum Chemistry

Ingrid von Glehn, James S. Spencer, David Pfau

We present a novel neural network architecture using self-attention, the Wavefunction Transformer (Psiformer), which can be used as an approximation (or Ansatz) for solving the many-electron Schrödinger equation, the fundamental equation for quantum chemistry and material science. This equation can be solved from first principles, requiring no external training data. In recent years, deep neural networks like the FermiNet and PauliNet have been used to significantly improve the accuracy of these first-principle calculations, but they lack an attention-like mechanism for gating interactions between electrons. Here we show that the Psiformer can be used as a drop-in replacement for these other neural networks, often dramatically improving the accuracy of the calculations. On larger molecules especially, the ground state energy can be improved by dozens of kcal/mol, a qualitative leap over previous methods. This demonstrates that self-attention networks can learn complex quantum mechanical correlations between electrons, and are a promising route to reaching unprecedented accuracy in chemical calculations on larger systems.

arXiv:2211.13672•Nov 24, 2022•27 citations

Variational Quantum Monte Carlo Solution of the Many-Electron Schrödinger Equation Based on Deep Neural Networks

Huiping Su, Hongbo Gao, Xinmiao Wang +2 more

The solution of Schrödinger equation generates the quantisation properties of the system, which can completely describe the quantum behaviour of microscopic particles in a physical system. However, solving the Schrödinger equation is a non-deterministic polynomial time (NP)-hard problem. Numerical methods for solving the Schrödinger equation require careful selection of basis configurations, and the complete basis set has high accuracy but its complexity increases exponentially in the number of particles. Therefore, it is crucial to find an effective numerical method. To solve this problem, this paper present a deep learning architecture, VMCNet, using the powerful computational efficiency of neural networks to improve the speed of numerical computation. Moreover, this paper proposes a suitable wavefunction ansatz, witch determines the accuracy of neural networks, that achieves more accurate energy solutions of the many-electron Schrödinger equation compared to the conventional single-determinant variational molecular Monte Carlo. We introduce non-local and local blocks to represent quantum mechanical interactions, which contain more physical information about the particle system. The experimental results show that VMCNet can cover more than 93% of the correlation energy of the main group elements (Be–Ne) for the monoatomic system and more than 90% of the correlation energy for the diatomic system.

CAAI Artificial Intelligence Research•Feb 1, 2024•1 citation

First-principles Deep Learning for Quantum Many-body Physics

Khachatur G. Nazaryan

This thesis investigates emergent phenomena in strongly correlated quantum matter, with a central focus on developing first-principles deep learning methods for quantum many-body systems. A common theme throughout this work is the search for microscopic mechanisms that generate unexpected collective behavior, together with computational frameworks capable of describing such behavior in regimes where conventional approaches become inadequate. The first part of the thesis studies a doped Mott insulator in a magnetic field and uncovers a new superconducting state, which we call \emph{magnonic superconductivity}. In this state, magnons mediate pairing, and the superconducting order is carried by composite objects binding electron pairs to spin excitations. This mechanism gives rise to a partially spin-polarized superconducting phase beyond the conventional paradigm of electron pairing and further suggests the possibility of higher-charge bound states, including charge-$6e$ superconductivity. The main thrust of the thesis is the use of deep neural networks as variational, first-principles representations of many-body wave functions. In contrast to the usual data-driven role of machine learning, the neural-network ans\"atze developed here do not learn from labeled datasets; instead, they are optimized directly from the microscopic Hamiltonian through variational Monte Carlo. Within this framework, we first develop self-attention-based neural-network wave functions for correlated electrons in solids and show that they accurately describe moir\'e systems with minimal human bias, capture interaction-driven phase transitions, and exhibit favorable scaling of representational complexity with system size. We then introduce a general physics-informed initialization strategy that reconstructs complex many-body wave functions from probability density and probability current. This approach enables highly accurate pretraining of neural wave functions, achieves near-exact overlaps for topological and paired quantum states, and allows efficient simulations of fractional quantum Hall systems with realistic interactions at unprecedented sizes. Finally, we introduce QERNEL, an efficient foundational neural variational ansatz designed to learn a family of many-body ground states within a single conditional model. By combining efficient context conditioning with modern permutation-equivariant building blocks, QERNEL achieves substantially improved expressivity at low computational cost. Applied to interacting electrons in semiconductor moir\'e heterobilayers, it captures distinct many-body phases and resolves the sharp phase transition between them in systems as large as 150 electrons. These results show that context-aware neural wave functions can move beyond Hamiltonian-by-Hamiltonian solution toward genuinely foundational models of quantum matter. Taken together, the results of this thesis show that first-principles deep learning can serve not merely as a numerical tool, but as a general framework for quantum many-body physics: one that bridges microscopic Hamiltonians, expressive wave-function ansätze, and scalable computation, while opening new paths to the discovery and characterization of strongly correlated quantum matter.

•May 1, 2026•0 citations

Neural network-based nodal structures optimization for interacting fermionic systems

Freitas, William, B De Abreu, S. A. Vitiello

Simulating strongly correlated fermionic systems remains a fundamental challenge in quantum physics, largely due to the sign problem in quantum Monte Carlo (QMC) methods. We present a neural network-based variational Monte Carlo (NN-VMC) approach, leveraging a flexible neural network ansatz to represent the many-body wavefunction. Focusing on quantum dots with up to 30 electrons, we demonstrate that NN-VMC significantly reduces variational bias and achieves ground-state energies surpassing those of fixed-node diffusion Monte Carlo (DMC). A key feature is that the neural network adaptively learns and optimizes nodal structures during energy minimization. We provide qualitative insights into the nodal structure of fermionic wavefunctions by comparing the nodal structures generated by NN-VMC with those obtained from traditional trial functions. Additionally, we reveal spin-resolved radial distributions and electron density profiles, highlighting the versatility and accuracy of NN-VMC. This work underscores the potential of machine learning to advance quantum simulations and deepen our understanding of strongly correlated systems.

arXiv:2411.02257•Nov 4, 2024•0 citations
The attention mechanism has transformed artificial intelligence research by its ability to learn relations between objects. In this work, we explore how a many-body wavefunction ansatz constructed from a large-parameter self-attention neural network can be used to solve the interacting electron problem in solids. By a systematic neural-network variational Monte Carlo study on a moiré quantum material, we demonstrate that the self-attention ansatz provides an accurate and efficient solution without human bias. Moreover, our numerical study finds that the required number of variational parameters scales roughly as N2N^2N2 with the number of electrons, which opens a path towards efficient large-scale simulations.
Abstract Deep-learning-based variational Monte Carlo has emerged as a highly accurate method for solving the many-electron Schrödinger equation. Despite favorable scaling with the number of electrons, O(nel4){\mathcal{O}}({{n}_{{\rm{el}}}}^{4})O(nel​4) O ( n el 4 ) , the practical value of deep-learning-based variational Monte Carlo is limited by the high cost of optimizing the neural network weights for every system studied. Recent research has proposed optimizing a single neural network across multiple systems, reducing the cost per system. Here we extend this approach to solids, which require numerous calculations across different geometries, boundary conditions and supercell sizes. We demonstrate that optimization of a single ansatz across these variations significantly reduces optimization steps. Furthermore, we successfully transfer a network trained on 2 × 2 × 2 supercells of LiH, to 3 × 3 × 3 supercells, reducing the number of optimization steps required to simulate the large system by a factor of 50 compared with previous work.
Moiré materials provide an ideal platform for exploring quantum phases of matter. However, solving the many-electron problem in moiré systems is challenging due to strong correlation effects. We introduce a powerful variational representation of quantum states, many-body neural Bloch wavefunction, to solve many-electron problems in moiré materials accurately and efficiently. Applying our method to the semiconductor heterobilayer WSe2/WS2 , we obtain a generalized Wigner crystal at filling factor n = 1/3, a Mott insulator n = 1, and a correlated insulator with local magnetic moments and antiferromagnetic spin correlation at n = 2. Our neural network approach improves the simulation accuracy of strongly interacting moiré materials and paves the way for discovery of new quantum phases with variational learning principle in a unified framework.
We present a novel neural network architecture using self-attention, the Wavefunction Transformer (Psiformer), which can be used as an approximation (or Ansatz) for solving the many-electron Schrödinger equation, the fundamental equation for quantum chemistry and material science. This equation can be solved from first principles, requiring no external training data. In recent years, deep neural networks like the FermiNet and PauliNet have been used to significantly improve the accuracy of these first-principle calculations, but they lack an attention-like mechanism for gating interactions between electrons. Here we show that the Psiformer can be used as a drop-in replacement for these other neural networks, often dramatically improving the accuracy of the calculations. On larger molecules especially, the ground state energy can be improved by dozens of kcal/mol, a qualitative leap over previous methods. This demonstrates that self-attention networks can learn complex quantum mechanical correlations between electrons, and are a promising route to reaching unprecedented accuracy in chemical calculations on larger systems.
The solution of Schrödinger equation generates the quantisation properties of the system, which can completely describe the quantum behaviour of microscopic particles in a physical system. However, solving the Schrödinger equation is a non-deterministic polynomial time (NP)-hard problem. Numerical methods for solving the Schrödinger equation require careful selection of basis configurations, and the complete basis set has high accuracy but its complexity increases exponentially in the number of particles. Therefore, it is crucial to find an effective numerical method. To solve this problem, this paper present a deep learning architecture, VMCNet, using the powerful computational efficiency of neural networks to improve the speed of numerical computation. Moreover, this paper proposes a suitable wavefunction ansatz, witch determines the accuracy of neural networks, that achieves more accurate energy solutions of the many-electron Schrödinger equation compared to the conventional single-determinant variational molecular Monte Carlo. We introduce non-local and local blocks to represent quantum mechanical interactions, which contain more physical information about the particle system. The experimental results show that VMCNet can cover more than 93% of the correlation energy of the main group elements (Be–Ne) for the monoatomic system and more than 90% of the correlation energy for the diatomic system.
This thesis investigates emergent phenomena in strongly correlated quantum matter, with a central focus on developing first-principles deep learning methods for quantum many-body systems. A common theme throughout this work is the search for microscopic mechanisms that generate unexpected collective behavior, together with computational frameworks capable of describing such behavior in regimes where conventional approaches become inadequate. The first part of the thesis studies a doped Mott insulator in a magnetic field and uncovers a new superconducting state, which we call \emph{magnonic superconductivity}. In this state, magnons mediate pairing, and the superconducting order is carried by composite objects binding electron pairs to spin excitations. This mechanism gives rise to a partially spin-polarized superconducting phase beyond the conventional paradigm of electron pairing and further suggests the possibility of higher-charge bound states, including charge-6e6e6e superconductivity. The main thrust of the thesis is the use of deep neural networks as variational, first-principles representations of many-body wave functions. In contrast to the usual data-driven role of machine learning, the neural-network ans\"atze developed here do not learn from labeled datasets; instead, they are optimized directly from the microscopic Hamiltonian through variational Monte Carlo. Within this framework, we first develop self-attention-based neural-network wave functions for correlated electrons in solids and show that they accurately describe moir\'e systems with minimal human bias, capture interaction-driven phase transitions, and exhibit favorable scaling of representational complexity with system size. We then introduce a general physics-informed initialization strategy that reconstructs complex many-body wave functions from probability density and probability current. This approach enables highly accurate pretraining of neural wave functions, achieves near-exact overlaps for topological and paired quantum states, and allows efficient simulations of fractional quantum Hall systems with realistic interactions at unprecedented sizes. Finally, we introduce QERNEL, an efficient foundational neural variational ansatz designed to learn a family of many-body ground states within a single conditional model. By combining efficient context conditioning with modern permutation-equivariant building blocks, QERNEL achieves substantially improved expressivity at low computational cost. Applied to interacting electrons in semiconductor moir\'e heterobilayers, it captures distinct many-body phases and resolves the sharp phase transition between them in systems as large as 150 electrons. These results show that context-aware neural wave functions can move beyond Hamiltonian-by-Hamiltonian solution toward genuinely foundational models of quantum matter. Taken together, the results of this thesis show that first-principles deep learning can serve not merely as a numerical tool, but as a general framework for quantum many-body physics: one that bridges microscopic Hamiltonians, expressive wave-function ansätze, and scalable computation, while opening new paths to the discovery and characterization of strongly correlated quantum matter.
Simulating strongly correlated fermionic systems remains a fundamental challenge in quantum physics, largely due to the sign problem in quantum Monte Carlo (QMC) methods. We present a neural network-based variational Monte Carlo (NN-VMC) approach, leveraging a flexible neural network ansatz to represent the many-body wavefunction. Focusing on quantum dots with up to 30 electrons, we demonstrate that NN-VMC significantly reduces variational bias and achieves ground-state energies surpassing those of fixed-node diffusion Monte Carlo (DMC). A key feature is that the neural network adaptively learns and optimizes nodal structures during energy minimization. We provide qualitative insights into the nodal structure of fermionic wavefunctions by comparing the nodal structures generated by NN-VMC with those obtained from traditional trial functions. Additionally, we reveal spin-resolved radial distributions and electron density profiles, highlighting the versatility and accuracy of NN-VMC. This work underscores the potential of machine learning to advance quantum simulations and deepen our understanding of strongly correlated systems.