We present an attention-based foundation model architecture for learning and predicting quantum states across Hamiltonian parameters, system sizes, and physical systems. Using only basis configurations and physical parameters as inputs, our trained neural network is able to produce highly accurate ground state wavefunctions. For example, we build the phase diagram for the 2D square-lattice t t – V V model with N N particles, from only 18 training points in parameter space (V/t,N) ( V / t , N ) . Thus, our architecture provides a basis for building a universal foundation model for quantum matter.
We present an attention-based foundation model architecture for learning and predicting quantum states across Hamiltonian parameters, system sizes, and physical systems. Using only basis configurations and physical parameters as inputs, our trained neural network is able to produce highly accurate ground state wavefunctions. For example, we build the phase diagram for the 2D square-lattice t t – V V model with N N particles, from only 18 training points in parameter space (V/t,N) ( V / t , N ) . Thus, our architecture provides a basis for building a universal foundation model for quantum matter.
Foundation models are highly versatile neural-network architectures capable of processing different data types, such as text and images, and generalizing across various tasks like classification and generation. Inspired by this success, we propose Foundation Neural-Network Quantum States (FNQS) as an integrated paradigm for studying quantum many-body systems. FNQS leverage key principles of foundation models to define variational wave functions based on a single, versatile architecture that processes multimodal inputs, including spin configurations and Hamiltonian physical couplings. Unlike specialized architectures tailored for individual Hamiltonians, FNQS can generalize to physical Hamiltonians beyond those encountered during training, offering a unified framework adaptable to various quantum systems and tasks. FNQS enable the efficient estimation of quantities that are traditionally challenging or computationally intensive to calculate using conventional methods, particularly disorder-averaged observables. Furthermore, the fidelity susceptibility can be easily obtained to uncover quantum phase transitions without prior knowledge of order parameters. These pretrained models can be efficiently fine-tuned for specific quantum systems. The architectures trained in this paper are publicly available at https://huggingface.co/nqs-models , along with examples for implementing these neural networks in NetKet.
Recent theoretical progress has established conditions under which machine learning models can efficiently predict ground-state properties of gapped local Hamiltonians when trained on quantum-generated data. Previous experimental demonstrations in this paradigm, however, have largely been limited to small systems or highly structured states, due to the difficulty of preparing many-body ground states on quantum processors. In this work, we demonstrate learning from experimental quantum data generated from approximate ground states of the two-dimensional Heisenberg XXZ model with system sizes up to 115 qubits. We construct a dataset of single-site expectation values, two-point correlations, and 12-body loop correlations across the antiferromagnetic phase. We then train neural networks on this data and show that they can accurately predict spatially resolved observables for previously unseen Hamiltonian parameters, both within the training distribution and in an out-of-distribution regime approaching the phase boundary. Our results demonstrate the practical realization of learning from quantum data for an interacting two-dimensional many-body system at scale, motivating a path toward regimes where quantum processors could provide training data beyond the reach of classical approximation methods.
We introduce Large Electron Model, a single neural network model that produces variational wavefunctions of interacting electrons over the entire Hamiltonian parameter manifold. Our model employs the Fermi Sets architecture, a universal representation of many-body fermionic wavefunctions, which is further conditioned on Hamiltonian parameter and particle number. For interacting electrons in a two-dimensional harmonic potential, a single trained model accurately predicts the ground state wavefunction while generalizing across unseen coupling strengths and particle-number sectors, producing both accurate real-space charge densities and ground state energies, even up to 50 particles. Our results establish a foundation model method for material discovery that is grounded in the variational principle, while accurately treating strong electron correlation beyond the capacity of density functional theory.
In the past decades, methods to solve the ground state given a quantum many-body Hamiltonian have been well established. In this work, we consider an inverse problem and demonstrate that convolutional neural network (CNN) can predict the physical parameters of interacting Hamiltonians, such as coupling strengths and magnetic fields, providing the quantum many-body wave-functions as the ground states. We propose QubismNet that consists of two main parts: the Qubism map that visualizes the ground states (or the purified reduced density matrices) as images, and a CNN that maps the images to the target physical parameters. QubismNet exhibits impressive powers of learning and generalization on several quantum spin models. While the training samples are restricted to the states from certain ranges of the parameters, QubismNet can accurately predict the parameters of the states beyond such training regions. For instance, our results show that QubismNet can predict the magnetic fields near the critical point by learning from the states away from the critical vicinity. Our work provides a data-driven way to infer the Hamiltonians that give the designed ground states, and therefore would benefit the existing and future generations of quantum technologies such as Hamiltonian-based quantum simulations.
My PhD research focused on the Transformer architecture, a powerful deep neural network model that has emerged as a cornerstone for solving complex problems in natural language processing, image analysis, signal processing, and beyond. In particular, we studied the learning dynamics of this architecture, and its application to the representation of many-body wavefunctions, the so-called Neural Quantum States. Initially, we investigated the representational capabilities of Transformers by characterizing the statistical structures that a simplified Transformer layer, utilizing the so-called factored attention, is capable of learning. Building on these results, we utilized factored attention in deep Transformers to develop an accurate ansatz for approximating the ground states of quantum many-body Hamiltonians within the variational Monte Carlo framework. In this specific application, factored attention is crucial for achieving accurate results, demonstrating superior performance compared to the standard attention mechanism used in most of the other applications of the Transformers, and in particular in natural language processing. Alongside the development of an efficient optimization method for large-scale neural networks, we achieved state-of-the-art results on the most popular benchmark in Neural Quantum States and addressed complex physical problems that are subjects of ongoing debate. Finally, we developed a framework to train Foundation Neural Quantum States, which are versatile neural network models that approximate quantum wave functions of multiple systems simultaneously, enabling accurate estimates of challenging quantities such as disorder averages and fidelity susceptibility. We envision numerous future directions for this approach, including its extension to quantum dynamics by explicitly modeling time-dependent variational states, as well as its application to the design of novel materials in fermionic systems.
A central promise of useful quantum advantage is the ability to compute ground states of Hamiltonian systems beyond the reach of classical simulation methods. Here we demonstrate that this problem can be effectively amortized across an arbitrary and universal set of Hamiltonians by a foundation model with ∼0.5B variational parameters, trained with contemporary techniques from large language models and deep reinforcement learning. To do this, we formulate spin-1/2 quantum ground-state learning as manifold variational optimisation over centrally odd scalar functions on SU(2)N. This replaces explicit Hilbert-space vector amplitudes with manifold functions on which the Hamiltonian acts through Lie derivatives, evaluated by custom automatic differentiation primitives. We prove that the resulting variational principle on this manifold preserves the spin-1/2 sector's ground-state upper bound using the Peter-Weyl theorem and justify the choice of such a representation with a no-go theorem for pure state foundation NQS. We then pre-train our foundation model on a dataset of hundreds of thousands of different Hamiltonian systems, varying the connection topology, system size, interaction types and strengths, bringing together a century of many-body literature. Using a novel SU(2) replica-exchange Langevin sampler and sharded natural-gradient optimisation, we train our model with our own extension of the Kronecker-Factored Approximate Curvature (KFAC) optimiser on system sizes up to 64 qubits. On a held-out generalisation dataset, we fine-tune our model on system sizes of up to 1024 qubits, and evaluate on systems up to 8100 qubits.