Deep neural-network quantum states have emerged as a leading method for studying the ground states of quantum magnets. Successful architectures exploit translational symmetry, but the root of their effectiveness and differences between architectures remain unclear. Here, we apply the ConvNext architecture, designed to incorporate elements of transformers into convolutional networks, to quantum many-body ground states. We find that it is remarkably similar to the factored vision transformer, which has been employed successfully for several frustrated spin systems, allowing us to relate this architecture to more conventional convolutional networks. Through a series of numerical experiments, we design the ConvNext to achieve greatest performance at lowest computational cost, and then apply this network to the Shastry-Sutherland and J 1 − J 2 models, obtaining variational energies comparable to the state of the art, providing a blueprint for network design choices of translationally symmetric architectures to tackle challenging ground-state problems in frustrated magnetism.
Deep neural-network quantum states have emerged as a leading method for studying the ground states of quantum magnets. Successful architectures exploit translational symmetry, but the root of their effectiveness and differences between architectures remain unclear. Here, we apply the ConvNext architecture, designed to incorporate elements of transformers into convolutional networks, to quantum many-body ground states. We find that it is remarkably similar to the factored vision transformer, which has been employed successfully for several frustrated spin systems, allowing us to relate this architecture to more conventional convolutional networks. Through a series of numerical experiments, we design the ConvNext to achieve greatest performance at lowest computational cost, and then apply this network to the Shastry-Sutherland and J 1 − J 2 models, obtaining variational energies comparable to the state of the art, providing a blueprint for network design choices of translationally symmetric architectures to tackle challenging ground-state problems in frustrated magnetism.
The transformer architecture has become the state-of-art model for natural language processing tasks and, more recently, also for computer vision tasks, thus defining the vision transformer (ViT) architecture. The key feature is the ability to describe long-range correlations among the elements of the input sequences, through the so-called self-attention mechanism. Here, we propose an adaptation of the ViT architecture with complex parameters to define a new class of variational neural-network states for quantum many-body systems, the ViT wave function. We apply this idea to the one-dimensional J_{1}-J_{2} Heisenberg model, demonstrating that a relatively simple parametrization gets excellent results for both gapped and gapless phases. In this case, excellent accuracies are obtained by a relatively shallow architecture, with a single layer of self-attention, thus largely simplifying the original architecture. Still, the optimization of a deeper structure is possible and can be used for more challenging models, most notably highly frustrated systems in two dimensions. The success of the ViT wave function relies on mixing both local and global operations, thus enabling the study of large systems with high accuracy.
We investigate the Shastry-Sutherland model (SSM), i.e., spin-1/2 quantum Heisen- berg model on a Shastry-Sutherland lattice, using a newly emerging approach exploiting well-developed machine learning techniques. We utilize neural networks as variational quantum states in quantum Monte Carlo investigations of the ground state properties. We first focus on SSM without an external magnetic field. For small lattices accessible via exact diagonalization, we compare the precision of various architectures based on re- stricted Boltzmann machines (RBM) or group-convolutional neural networks. The most versatile and precise architecture, namely complex-valued RBM, is then applied for larger lattices. Here we investigate the frustrated regime. We show that the RBM is able to represent all three of the major and fundamentally different phases of SSM. Finally, we apply the complex-valued RBM for SSM in a finite external magnetic field. We find that it cannot capture the intriguing steps in magnetization typical for SSM correctly due to its tendency to prefer more ordered states with higher magnetization. 1
We show that neural quantum states based on very deep (4--16-layered) neural networks can outperform state-of-the-art variational approaches on highly frustrated quantum magnets, including quantum-spin-liquid candidates. We focus on group convolutional neural networks (GCNNs) that allow us to impose space-group symmetries on our ansätze. We achieve state-of-the-art ground-state energies for the J1−J2 Heisenberg models on the square and triangular lattices, in both ordered and spin-liquid phases, and discuss ways to access low-lying excited states in nontrivial symmetry sectors. We also compute spin and dimer correlation functions for the quantum paramagnetic phase on the triangular lattice, which do not indicate either conventional or valence-bond ordering.
The use of artificial neural networks to represent quantum wave functions has recently attracted interest as a way to solve complex many-body problems. The potential of these variational parametrizations has been supported by analytical and numerical evidence in controlled benchmarks. While approaching the end of the early research phase in this field, it becomes increasingly important to show how neural-network states perform for models and physical problems that constitute a clear open challenge for other many-body computational methods. In this paper, we start addressing this aspect, concentrating on a presently unsolved model describing two-dimensional frustrated magnets. Using a fully convolutional neural network model as a variational ans\"atz, we study the frustrated spin-1/$2{J}_{1}\text{\ensuremath{-}}{J}_{2}$ Heisenberg model on the square lattice. We demonstrate that the resulting predictions for both ground-state energies and properties are competitive with, and often improve upon, existing state-of-the-art methods. In a relatively small region in the parameter space, corresponding to the maximally frustrated regime, our ans\"atz exhibits comparatively good but not the best performance. The gap between the complexity of the models adopted here and those routinely adopted in deep-learning applications is, however, still substantial, such that further improvements in future generations of neural-network quantum states are likely to be expected.
Abstract Computing the ground state of interacting quantum matter is a long-standing challenge, especially for complex two-dimensional systems. Recent developments have highlighted the potential of neural quantum states to solve the quantum many-body problem by encoding the many-body wavefunction into artificial neural networks. However, this method has faced the critical limitation that existing optimization algorithms are not suitable for training modern large-scale deep network architectures. Here, we introduce a minimum-step stochastic-reconfiguration optimization algorithm, which allows us to train deep neural quantum states with up to 10 6 parameters. We demonstrate our method for paradigmatic frustrated spin-1/2 models on square and triangular lattices, for which our trained deep networks approach machine precision and yield improved variational energies compared to existing results. Equipped with our optimization algorithm, we find numerical evidence for gapless quantum-spin-liquid phases in the considered models, an open question to date. We present a method that captures the emergent complexity in quantum many-body problems through the expressive power of large-scale artificial neural networks.
My PhD research focused on the Transformer architecture, a powerful deep neural network model that has emerged as a cornerstone for solving complex problems in natural language processing, image analysis, signal processing, and beyond. In particular, we studied the learning dynamics of this architecture, and its application to the representation of many-body wavefunctions, the so-called Neural Quantum States. Initially, we investigated the representational capabilities of Transformers by characterizing the statistical structures that a simplified Transformer layer, utilizing the so-called factored attention, is capable of learning. Building on these results, we utilized factored attention in deep Transformers to develop an accurate ansatz for approximating the ground states of quantum many-body Hamiltonians within the variational Monte Carlo framework. In this specific application, factored attention is crucial for achieving accurate results, demonstrating superior performance compared to the standard attention mechanism used in most of the other applications of the Transformers, and in particular in natural language processing. Alongside the development of an efficient optimization method for large-scale neural networks, we achieved state-of-the-art results on the most popular benchmark in Neural Quantum States and addressed complex physical problems that are subjects of ongoing debate. Finally, we developed a framework to train Foundation Neural Quantum States, which are versatile neural network models that approximate quantum wave functions of multiple systems simultaneously, enabling accurate estimates of challenging quantities such as disorder averages and fidelity susceptibility. We envision numerous future directions for this approach, including its extension to quantum dynamics by explicitly modeling time-dependent variational states, as well as its application to the design of novel materials in fermionic systems.