Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2607.14875

Optimization dynamics of Transformer backflow neural quantum states for the two-dimensional Hubbard model

Zong-Yu Liao, Jia-Qi Wang, Rong-Qiang He, Zhong-Yi Lu

Published Jul 16, 2026• 0 citations
arXivHTMLPDF

Abstract

Building on the multi-determinant Transformer backflow neural quantum state (NQS) ansatz and the associated multi-stage training workflow for the doped two-dimensional Hubbard model, we investigate how the optimization dynamics of the NQS depend on several key optimization and architectural hyperparameters. The workflow consists of neural-network backflow (NNB) initialization, supervised Transformer pre-training, and main energy optimization using the Moment-Adaptive ReConfiguration Heuristic (MARCH) within variational Monte Carlo. Using the doped $4\times4$ periodic Hubbard model at $U=8$ as a baseline, we examine how the update-norm threshold, Transformer width, number of determinant channels, and Monte Carlo batch size affect convergence. We find that a moderate update constraint improves the efficiency of MARCH optimization, larger Transformer width and more determinant channels improve the expressive capacity of the ansatz, and larger Monte Carlo batches reduce sampling noise in the update direction. We further test the same workflow at half filling, weaker interaction strength, open boundary conditions, and on a larger $8\times8$ doped lattice. These results identify practical optimization trends for Transformer backflow NQSs and highlight the balance between ansatz expressivity, MARCH update stability, and Monte Carlo sampling quality.

DOI
10.48550/arxiv.2607.14875

Related papers

Enhancing Neural Network Backflow

Kieran Loehr, Bryan K. Clark

Accurately describing the ground state of strongly correlated systems is essential for understanding their emergent properties. Neural Network Backflow (NNBF) is a powerful variational ansatz that enhances mean-field wave functions by introducing configuration-dependent modifications to single-particle orbitals. Although NNBF is theoretically universal in the limit of large networks, we find that practical gains saturate with increasing network size. Instead, significant improvements can be achieved by using a multi-determinant ansatz. We explore efficient ways to generate these multi-determinant expansions without increasing the number of variational parameters. In particular, we study single-step Lanczos and symmetry projection techniques, benchmarking their performance against diffusion Monte Carlo and NNBF applied to alternative mean fields. Benchmarking on a doped periodic square Hubbard model near optimal doping, we find that a Lanczos step, diffusion Monte Carlo, and projection onto a symmetry sector all give similar improvements achieving state-of-the-art energies at minimal cost. By further optimizing the projected symmetrized states directly, we gain significantly in energy. Using this technique we report the lowest variational energies for this Hamiltonian on $4\times 16$ and $4 \times 8$ lattices as well as accurate variance extrapolated energies. We also show the evolution of spin, charge, and pair correlation functions as the quality of the variational ansatz improves.

arXiv:2510.26906•Oct 30, 2025•0 citations

Autoregressive neural quantum states of Fermi Hubbard models

Eduardo Ibarra-García-Padilla, Hannah Lange, Roger G. Melko +4 more

Neural quantum states (NQSs) have emerged as a powerful ansatz for variational quantum Monte Carlo studies of strongly correlated systems. Here, we apply recurrent neural networks (RNNs) and autoregressive transformer neural networks to the Fermi-Hubbard and the (non-Hermitian) Hatano-Nelson-Hubbard models in one and two dimensions. In both cases, we observe that the convergence of the RNN ansatz is challenged when increasing the interaction strength. We present a physically motivated and easy-to-implement strategy for improving the optimization, namely, by ramping of the model parameters. Furthermore, we investigate the advantages and disadvantages of the autoregressive sampling property of both network architectures. For the Hatano-Nelson-Hubbard model, we identify convergence issues that stem from the autoregressive sampling scheme in combination with the non-Hermitian nature of the model. Our findings provide insights into the challenges of the NQS approach and make the first step towards exploring strongly correlated electrons using this ansatz.

arXiv:2411.07144•Physical Review Research•Feb 3, 2025•9 citations

Neural Network Backflow with Low-Rank Multi-Determinant Updates

Tianshu Huang, Xiaowei Ou, Vidvuds Ozolins

Simulating strongly correlated fermions remains a long-standing challenge due to the exponential complexity of the Hilbert space and the intricate sign structure of many-body wavefunctions. We introduce a variational framework centered on a neural network backflow transformation that combines deep learning with variational Monte Carlo. The proposed ansatz employs a multi-determinant expansion with low-rank shifts to capture non-local correlations and complex sign structures. Applied to the two-dimensional Hubbard model at both half-filling and $1/8$ doping, the method achieves energies within $0.45\%$ of auxiliary-field quantum Monte Carlo at half-filling and captures intertwined charge- and spin-density stripe patterns at $1/8$ doping. These results demonstrate the potential of this framework as a scalable and interpretable approach to variational simulations of strongly correlated fermionic systems.

cond-mat.str-elCross-listedarXiv:2609.26544•Sep 23, 2026•0 citations

Efficient optimization of neural network backflow for a b i n i t i o quantum chemistry

An-Jun Liu, Bryan K. Clark

The ground state of second-quantized quantum chemistry Hamiltonians is key to determining molecular properties. Neural quantum states (NQSs) offer flexible and expressive wave-function Ans\"atze for this task but face two main challenges: highly peaked ground-state wave functions hinder efficient sampling, and local energy evaluations scale quartically with system size, incurring significant computational costs. In this work, we overcome these challenges by introducing a suite of algorithmic enhancements, which includes efficient periodic compact subspace construction, truncated local energy evaluations, improved stochastic sampling, and physics-informed modifications. Applying these techniques to the neural network backflow Ansatz, we demonstrate significant gains in both accuracy and scalability. Our enhanced method surpasses traditional quantum chemistry methods like coupled cluster singles and doubles (CCSD) and CCSD and perturbative triples [CCSD(T)], outperforms other NQS approaches, and achieves competitive energies with state-of-the-art ab initio techniques such as heat-bath configuration interaction, adaptive sampling configuration interaction, full configuration interaction quantum Monte Carlo, and density matrix renormalization group. A series of ablation and comparative studies quantifies the contribution of each enhancement to the observed improvements in accuracy and efficiency. Furthermore, we investigate the representational capacity of the Ansatz, finding that its performance correlates with the inverse participation ratio, with more delocalized states being more challenging to approximate.

Physical review. B./Physical review. B•Sep 23, 2025•4 citations

Pareto Frontier of Neural Quantum States: Scalable, Affordable, and Accurate Convolutional Backflow for Strongly Correlated Lattice Fermions

Yuntian Gu, Zeyao Han, Wenrui Li +5 more

The recent development of Neural Quantum States (NQS) has established them as one of the most accurate methods for studying strongly correlated many-fermion systems, outperforming existing many-body approaches for large systems. However, NQS calculations are currently highly resource intensive. In this work, we introduce an improved Pareto frontier of efficiency and accuracy for NQS in the simulation of strongly correlated lattice fermions. This frontier is defined by two complementary backflow-related architectures: the Sparse Convolutional Ansatz for Lattice Electrons (SCALE), which achieves high computational efficiency, and the Accurate Convolutional ansatz for lattice Electrons (ACE), which delivers high variational accuracy, based on benchmark results on the Hubbard model and the t – J model on large lattice sizes. SCALE utilizes a tailored convolutional design to enable an efficient local update based on the low-rank update of determinants. This structure reduces the computational scaling from O ( N 4 ) to O ( N 3 ) in backflow methods, resulting in a practical speedup exceeding 40 × in our test cases while retaining accurate variational accuracy. As an application, we study the 1 / 8 -doped pure Hubbard model up to size 32 × 32 and find no significant energy difference between the horizontal and vertical unit-filled stripe states, in contrast to the half-filled stripe state when next-nearest-neighbor hoppings are included. ACE, conversely, employs a deep convolutional stack to maximize expressive power, achieving high accuracy on large systems. We perform extensive benchmark calculations on the Hubbard and t – J models with the two architectures. On the challenging repulsive Hubbard model, SCALE achieves variational energies competitive with leading methods at a fraction of the computational cost. Meanwhile, ACE achieves lower variational energies than previously reported results while requiring only one-sixth of the run-time for system size 16 × 4 . These NQS approaches provide , , and tools for exploring the rich physics of strongly correlated fermionic systems.

arXiv:2604.25775•PRX Intelligence•Oct 6, 2026•0 citations

Solving the Hubbard model with neural quantum states

Yuntian Gu, Wenrui Li, Heng Lin +9 more

Abstract The rapid development of neural quantum states (NQS) has established it as a promising framework for studying quantum many-body systems. In this work, by leveraging the cutting-edge transformer-based architectures and developing highly efficient optimization algorithms, we achieve the state-of-the-art results for the doped two-dimensional (2D) Hubbard model, arguably the minimum model for high-Tc superconductivity. Interestingly, we find different attention heads in the NQS ansatz can directly encode correlations at different scales, making it capable of capturing long-range correlations and entanglements in strongly correlated systems. With these advances, we establish the half-filled stripe in the ground state of 2D Hubbard model with the next nearest neighboring hopping, consistent with experimental observations in cuprates. Our work establishes NQS as a powerful tool for solving challenging many-fermions systems.

arXiv:2507.02644•Nature Communications•Jun 22, 2026•6 citations
Building on the multi-determinant Transformer backflow neural quantum state (NQS) ansatz and the associated multi-stage training workflow for the doped two-dimensional Hubbard model, we investigate how the optimization dynamics of the NQS depend on several key optimization and architectural hyperparameters. The workflow consists of neural-network backflow (NNB) initialization, supervised Transformer pre-training, and main energy optimization using the Moment-Adaptive ReConfiguration Heuristic (MARCH) within variational Monte Carlo. Using the doped 4×44\times44×4 periodic Hubbard model at U=8U=8U=8 as a baseline, we examine how the update-norm threshold, Transformer width, number of determinant channels, and Monte Carlo batch size affect convergence. We find that a moderate update constraint improves the efficiency of MARCH optimization, larger Transformer width and more determinant channels improve the expressive capacity of the ansatz, and larger Monte Carlo batches reduce sampling noise in the update direction. We further test the same workflow at half filling, weaker interaction strength, open boundary conditions, and on a larger 8×88\times88×8 doped lattice. These results identify practical optimization trends for Transformer backflow NQSs and highlight the balance between ansatz expressivity, MARCH update stability, and Monte Carlo sampling quality.
Accurately describing the ground state of strongly correlated systems is essential for understanding their emergent properties. Neural Network Backflow (NNBF) is a powerful variational ansatz that enhances mean-field wave functions by introducing configuration-dependent modifications to single-particle orbitals. Although NNBF is theoretically universal in the limit of large networks, we find that practical gains saturate with increasing network size. Instead, significant improvements can be achieved by using a multi-determinant ansatz. We explore efficient ways to generate these multi-determinant expansions without increasing the number of variational parameters. In particular, we study single-step Lanczos and symmetry projection techniques, benchmarking their performance against diffusion Monte Carlo and NNBF applied to alternative mean fields. Benchmarking on a doped periodic square Hubbard model near optimal doping, we find that a Lanczos step, diffusion Monte Carlo, and projection onto a symmetry sector all give similar improvements achieving state-of-the-art energies at minimal cost. By further optimizing the projected symmetrized states directly, we gain significantly in energy. Using this technique we report the lowest variational energies for this Hamiltonian on 4×164\times 164×16 and 4×84 \times 84×8 lattices as well as accurate variance extrapolated energies. We also show the evolution of spin, charge, and pair correlation functions as the quality of the variational ansatz improves.
Neural quantum states (NQSs) have emerged as a powerful ansatz for variational quantum Monte Carlo studies of strongly correlated systems. Here, we apply recurrent neural networks (RNNs) and autoregressive transformer neural networks to the Fermi-Hubbard and the (non-Hermitian) Hatano-Nelson-Hubbard models in one and two dimensions. In both cases, we observe that the convergence of the RNN ansatz is challenged when increasing the interaction strength. We present a physically motivated and easy-to-implement strategy for improving the optimization, namely, by ramping of the model parameters. Furthermore, we investigate the advantages and disadvantages of the autoregressive sampling property of both network architectures. For the Hatano-Nelson-Hubbard model, we identify convergence issues that stem from the autoregressive sampling scheme in combination with the non-Hermitian nature of the model. Our findings provide insights into the challenges of the NQS approach and make the first step towards exploring strongly correlated electrons using this ansatz.
Simulating strongly correlated fermions remains a long-standing challenge due to the exponential complexity of the Hilbert space and the intricate sign structure of many-body wavefunctions. We introduce a variational framework centered on a neural network backflow transformation that combines deep learning with variational Monte Carlo. The proposed ansatz employs a multi-determinant expansion with low-rank shifts to capture non-local correlations and complex sign structures. Applied to the two-dimensional Hubbard model at both half-filling and 1/81/81/8 doping, the method achieves energies within 0.45%0.45\%0.45% of auxiliary-field quantum Monte Carlo at half-filling and captures intertwined charge- and spin-density stripe patterns at 1/81/81/8 doping. These results demonstrate the potential of this framework as a scalable and interpretable approach to variational simulations of strongly correlated fermionic systems.
The ground state of second-quantized quantum chemistry Hamiltonians is key to determining molecular properties. Neural quantum states (NQSs) offer flexible and expressive wave-function Ans\"atze for this task but face two main challenges: highly peaked ground-state wave functions hinder efficient sampling, and local energy evaluations scale quartically with system size, incurring significant computational costs. In this work, we overcome these challenges by introducing a suite of algorithmic enhancements, which includes efficient periodic compact subspace construction, truncated local energy evaluations, improved stochastic sampling, and physics-informed modifications. Applying these techniques to the neural network backflow Ansatz, we demonstrate significant gains in both accuracy and scalability. Our enhanced method surpasses traditional quantum chemistry methods like coupled cluster singles and doubles (CCSD) and CCSD and perturbative triples [CCSD(T)], outperforms other NQS approaches, and achieves competitive energies with state-of-the-art ab initio techniques such as heat-bath configuration interaction, adaptive sampling configuration interaction, full configuration interaction quantum Monte Carlo, and density matrix renormalization group. A series of ablation and comparative studies quantifies the contribution of each enhancement to the observed improvements in accuracy and efficiency. Furthermore, we investigate the representational capacity of the Ansatz, finding that its performance correlates with the inverse participation ratio, with more delocalized states being more challenging to approximate.
The recent development of Neural Quantum States (NQS) has established them as one of the most accurate methods for studying strongly correlated many-fermion systems, outperforming existing many-body approaches for large systems. However, NQS calculations are currently highly resource intensive. In this work, we introduce an improved Pareto frontier of efficiency and accuracy for NQS in the simulation of strongly correlated lattice fermions. This frontier is defined by two complementary backflow-related architectures: the Sparse Convolutional Ansatz for Lattice Electrons (SCALE), which achieves high computational efficiency, and the Accurate Convolutional ansatz for lattice Electrons (ACE), which delivers high variational accuracy, based on benchmark results on the Hubbard model and the t – J model on large lattice sizes. SCALE utilizes a tailored convolutional design to enable an efficient local update based on the low-rank update of determinants. This structure reduces the computational scaling from O ( N 4 ) to O ( N 3 ) in backflow methods, resulting in a practical speedup exceeding 40 × in our test cases while retaining accurate variational accuracy. As an application, we study the 1 / 8 -doped pure Hubbard model up to size 32 × 32 and find no significant energy difference between the horizontal and vertical unit-filled stripe states, in contrast to the half-filled stripe state when next-nearest-neighbor hoppings are included. ACE, conversely, employs a deep convolutional stack to maximize expressive power, achieving high accuracy on large systems. We perform extensive benchmark calculations on the Hubbard and t – J models with the two architectures. On the challenging repulsive Hubbard model, SCALE achieves variational energies competitive with leading methods at a fraction of the computational cost. Meanwhile, ACE achieves lower variational energies than previously reported results while requiring only one-sixth of the run-time for system size 16 × 4 . These NQS approaches provide , , and tools for exploring the rich physics of strongly correlated fermionic systems.
Abstract The rapid development of neural quantum states (NQS) has established it as a promising framework for studying quantum many-body systems. In this work, by leveraging the cutting-edge transformer-based architectures and developing highly efficient optimization algorithms, we achieve the state-of-the-art results for the doped two-dimensional (2D) Hubbard model, arguably the minimum model for high-Tc superconductivity. Interestingly, we find different attention heads in the NQS ansatz can directly encode correlations at different scales, making it capable of capturing long-range correlations and entanglements in strongly correlated systems. With these advances, we establish the half-filled stripe in the ground state of 2D Hubbard model with the next nearest neighboring hopping, consistent with experimental observations in cuprates. Our work establishes NQS as a powerful tool for solving challenging many-fermions systems.