We introduce a variational wave function based on neural-network quantum states (NQS) to study lattice systems whose local Hilbert space contains both spin and fermionic degrees of freedom. Our approach is based on the use of the transformer architecture, which can naturally handle composite local Hilbert spaces through a tokenization procedure closely inspired by techniques from natural language processing. The key methodological innovation is that the neural network predicts a set of fermionic orbitals that depend on the spin configuration in a backflow-inspired manner. We apply the method to the one-dimensional ancilla layer model, consisting of a chain of mobile spin- 1 / 2 fermions coupled to a two-leg spin- 1 / 2 ladder. For open boundary conditions, we achieve excellent quantitative agreement with density matrix renormalization group results across the full range of parameters considered. We find a phase in which the chain forms an effectively decoupled Luttinger liquid (LL), and an LL * phase with a distinct Fermi wave vector in which the mobile fermions are Kondo screened by one leg of the ladder, while the other leg forms the critical Bethe spin liquid. The LL * is the analog of the FL * phase in two dimensions proposed to describe the cuprate pseudogap. We also find a Luther-Emery phase, where the LL * state becomes unstable toward the formation of a spin gap. The transformer ansatz maintains comparable accuracy for periodic boundary conditions, where tensor-network methods are computationally more demanding. Moreover, we show that the approach extends naturally to fully periodic two-dimensional clusters, a regime beyond the reach of current tensor-network methods. Together, these findings establish transformer-based NQS as an accurate and scalable variational framework for correlated lattice systems with composite local Hilbert spaces and highlight its potential for studying higher-dimensional models where boundary effects and heterogeneous local structures pose significant challenges.
We introduce a variational wave function based on neural-network quantum states (NQS) to study lattice systems whose local Hilbert space contains both spin and fermionic degrees of freedom. Our approach is based on the use of the transformer architecture, which can naturally handle composite local Hilbert spaces through a tokenization procedure closely inspired by techniques from natural language processing. The key methodological innovation is that the neural network predicts a set of fermionic orbitals that depend on the spin configuration in a backflow-inspired manner. We apply the method to the one-dimensional ancilla layer model, consisting of a chain of mobile spin- 1 / 2 fermions coupled to a two-leg spin- 1 / 2 ladder. For open boundary conditions, we achieve excellent quantitative agreement with density matrix renormalization group results across the full range of parameters considered. We find a phase in which the chain forms an effectively decoupled Luttinger liquid (LL), and an LL * phase with a distinct Fermi wave vector in which the mobile fermions are Kondo screened by one leg of the ladder, while the other leg forms the critical Bethe spin liquid. The LL * is the analog of the FL * phase in two dimensions proposed to describe the cuprate pseudogap. We also find a Luther-Emery phase, where the LL * state becomes unstable toward the formation of a spin gap. The transformer ansatz maintains comparable accuracy for periodic boundary conditions, where tensor-network methods are computationally more demanding. Moreover, we show that the approach extends naturally to fully periodic two-dimensional clusters, a regime beyond the reach of current tensor-network methods. Together, these findings establish transformer-based NQS as an accurate and scalable variational framework for correlated lattice systems with composite local Hilbert spaces and highlight its potential for studying higher-dimensional models where boundary effects and heterogeneous local structures pose significant challenges.
We introduce Neural Tensor Network States (νTNS), a variational many-body wave-function ansatz that integrates deep neural networks with tensor-network architectures. In the νTNS framework, a neural network serves as a disentangler of the wave-function, transforming the physical degrees of freedom into renormalized variables with much less entanglement. The renormalized state is then efficiently encoded by a back-flow tensor network. This construction yields a compact yet highly expressive representation of strongly correlated quantum states. Using convolutional neural networks combined with matrix product states as a concrete implementation, we obtain state-of-the-art variational energies for the spin-1/2J1-J2 Heisenberg model on the square lattice at the highly frustrated point J2/J1=0.5, for systems up to 20×20 with periodic boundary conditions. Finite-size scaling of spin, dimer, and plaquette correlations exhibits power-law decay without magnetic or valence-bond long-range order, consistent with a gapless quantum spin-liquid ground state at that point.This νTNS framework is flexible and naturally extensible to other neural and tensor-network structures, offering a general platform for investigating strongly correlated quantum many-body systems.
We design a neural network Ansatz for variationally finding the ground-state wave function of the Homogeneous Electron Gas, a fundamental model in the physics of extended systems of interacting fermions. We study the spin-polarised and paramagnetic phases with 7, 14 and 19 electrons over a broad range of densities from rs=1 to rs=100, obtaining similar or higher accuracy compared to a state-of-the-art iterative backflow baseline even in the challenging regime of very strong correlation. Our work extends previous applications of neural network Ansätze to molecular systems with methods for handling periodic boundary conditions, and makes two notable changes to improve performance: splitting the pairwise streams by spin alignment and generating backflow coordinates for the orbitals from the network. We illustrate the advantage of our high quality wave functions in computing the reduced single particle density matrix. This contribution establishes neural network models as flexible and high precision Ansätze for periodic electronic systems, an important step towards applications to crystalline solids.
The transformer architecture has become the state-of-art model for natural language processing tasks and, more recently, also for computer vision tasks, thus defining the vision transformer (ViT) architecture. The key feature is the ability to describe long-range correlations among the elements of the input sequences, through the so-called self-attention mechanism. Here, we propose an adaptation of the ViT architecture with complex parameters to define a new class of variational neural-network states for quantum many-body systems, the ViT wave function. We apply this idea to the one-dimensional J_{1}-J_{2} Heisenberg model, demonstrating that a relatively simple parametrization gets excellent results for both gapped and gapless phases. In this case, excellent accuracies are obtained by a relatively shallow architecture, with a single layer of self-attention, thus largely simplifying the original architecture. Still, the optimization of a deeper structure is possible and can be used for more challenging models, most notably highly frustrated systems in two dimensions. The success of the ViT wave function relies on mixing both local and global operations, thus enabling the study of large systems with high accuracy.
Recent advancements in neural networks have led to significant progress in addressing many-body electron correlations in small molecules and various physical models. In this work, we propose QiankunNet-Solid, which incorporates periodic boundary conditions into the neural network quantum state (NNQS) framework based on generative Transformer architecture along with a batched autoregressive sampling (BAS) method, enabling the effective ab initio calculation of real solid materials. The accuracy of this method is demonstrated in one-, two-, and three-dimensional periodic systems, with results comparable to those of full configuration interaction and coupled-cluster method, even in the strongly correlated regime. Furthermore, we compute the band structures and density of states for silicon crystal. The successful incorporation of periodic boundary conditions into the NNQS framework through QiankunNet-Solid opens up new possibilities for the accurate and efficient study of electronic structure properties in solid-state physics.
One-dimensional (1D) systems and models provide a versatile platform for emergent phenomena induced by strong electron correlation. In this work, we extend the newly developed real space neural network quantum Monte Carlo methods to study the quantum phase transition of electronic and magnetic properties. Hydrogen chains of different interatomic distances are explored systematically with both open and periodic boundary conditions, and fully correlated ground state many-body wavefunction is achieved via unsupervised training of neural networks. We demonstrate for the first time that neural networks are capable of capturing the quantum critical behavior of Tomonaga- Luttinger liquid (TLL), which is known to dominate 1D quantum systems. Moreover, we reveal the breakdown of TLL phase and the emergence of a Fermi liquid behavior, evidenced by abrupt changes in the spin structure and the momentum distribution. Such behavior is absent in commonly studied 1D lattice models and is likely due to the involvement of high-energy orbitals of hydrogen atoms. Our work highlights the powerfulness of neural networks for representing complex quantum phases.
Neural network quantum states (NQS) have emerged as a powerful and flexible framework for addressing quantum many-body problems. While successful for model Hamiltonians, their application to molecular systems remains challenging for several reasons. In this work, we introduce three innovations to overcome some of the key limitations. (1) We develop a bounded-degree graph recurrent neural network (BDG–RNN) ansatz, which hybridizes the tensor network and neural network states and is more suitable to molecular electronic structure problems. As matrix product states (MPS) can be embedded into this ansatz, good initialization is possible for complex systems. (2) We introduce neural network correlators (NNCs) to further enhance expressivity and improve accuracy, without dramatically modifying the underlying variational Monte Carlo (VMC) optimization framework. Specifically, we introduce two types of restricted Boltzmann machine (RBM)-inspired correlators, namely, cos-RBM and Ising-RBM, which unlike previous correlators, such as Jastrow and real RBM, can adjust the sign structure of the wave function. (3) We introduce a semistochastic algorithm for local energy evaluation, which significantly reduces computational cost while maintaining high accuracy. Combining these advances, we demonstrate that our approaches can achieve chemical accuracy in challenging systems, including the one-dimensional hydrogen chain H 50, the iron–sulfur cluster [Fe 2 S 2 (SCH 3 ) 4 ] 2–, and a three-dimensional 3 × 3 × 2 hydrogen cluster H 18 . These methods are implemented in an open-source package, P y NQS ( https://github.com/Quantum-Chemistry-Group-BNU/PyNQS ), to advance NQS methodologies for quantum chemistry.