Obtaining accurate solutions to the Schrödinger equation is the key challenge in computational quantum chemistry. Deep-learning-based Variational Monte Carlo (DL-VMC) has recently outperformed conventional approaches in terms of accuracy, but only at large computational cost. Whereas in many domains models are trained once and subsequently applied for inference, accurate DL-VMC so far requires a full optimization for every new problem instance, consuming thousands of GPUhs even for small molecules. We instead propose a DL-VMC model which has been pre-trained using self-supervised wavefunction optimization on a large and chemically diverse set of molecules. Applying this model to new molecules without any optimization, yields wavefunctions and absolute energies that outperform established methods such as CCSD(T)-2Z. To obtain accurate relative energies, only few fine-tuning steps of this base model are required. We accomplish this with a fully end-to-end machine-learned model, consisting of an improved geometry embedding architecture and an existing SE(3)-equivariant model to represent molecular orbitals. Combining this architecture with continuous sampling of geometries, we improve zero-shot accuracy by two orders of magnitude compared to the state of the art. We extensively evaluate the accuracy, scalability and limitations of our base model on a wide variety of test systems.
Obtaining accurate solutions to the Schrödinger equation is the key challenge in computational quantum chemistry. Deep-learning-based Variational Monte Carlo (DL-VMC) has recently outperformed conventional approaches in terms of accuracy, but only at large computational cost. Whereas in many domains models are trained once and subsequently applied for inference, accurate DL-VMC so far requires a full optimization for every new problem instance, consuming thousands of GPUhs even for small molecules. We instead propose a DL-VMC model which has been pre-trained using self-supervised wavefunction optimization on a large and chemically diverse set of molecules. Applying this model to new molecules without any optimization, yields wavefunctions and absolute energies that outperform established methods such as CCSD(T)-2Z. To obtain accurate relative energies, only few fine-tuning steps of this base model are required. We accomplish this with a fully end-to-end machine-learned model, consisting of an improved geometry embedding architecture and an existing SE(3)-equivariant model to represent molecular orbitals. Combining this architecture with continuous sampling of geometries, we improve zero-shot accuracy by two orders of magnitude compared to the state of the art. We extensively evaluate the accuracy, scalability and limitations of our base model on a wide variety of test systems.
A faithful description of chemical processes requires exploring extended regions of the molecular potential energy surface (PES), which remains challenging for strongly correlated systems. Transferable deep-learning variational Monte Carlo (VMC) offers a promising route by efficiently solving the electronic Schrödinger equation jointly across molecular geometries at consistently high accuracy, yet its stochastic nature renders direct exploration of molecular configuration space nontrivial. Here, we present a framework for highly accurate ab initio exploration of PESs that combines transferable deep-learning VMC with a cost-effective estimation of energies, forces, and Hessians. By continuously sampling nuclear configurations during VMC optimization of electronic wave functions, we obtain transferable descriptions that achieve zero-shot chemical accuracy within chemically relevant distributions of molecular geometries. Throughout the subsequent characterization of molecular configuration space, the PES is evaluated only sparsely, with local approximations constructed by estimating VMC energies and forces at sampled geometries and aggregating the resulting noisy data using Gaussian process regression. Our method enables accurate and efficient exploration of complex PES landscapes, including structure relaxation, transition-state searches, and minimum-energy pathways, for both ground and excited states. This opens the door to studying bond breaking, formation, and large structural rearrangements in systems with pronounced multi-reference character.
Deep neural networks have become a highly accurate and powerful wavefunction ansatz in combination with variational Monte Carlo methods for solving the electronic Schrödinger equation. However, despite their success and favorable scaling, these methods are still computationally too costly for wide adoption. A significant obstacle is the requirement to optimize the wavefunction from scratch for each new system, thus requiring long optimization. In this work, we propose a neural network ansatz, which effectively maps uncorrelated, computationally cheap Hartree-Fock orbitals, to correlated, high-accuracy neural network orbitals. This ansatz is inherently capable of learning a single wavefunction across multiple compounds and geometries, as we demonstrate by successfully transferring a wavefunction model pre-trained on smaller fragments to larger compounds. Furthermore, we provide ample experimental evidence to support the idea that extensive pre-training of such a generalized wavefunction model across different compounds and geometries could lead to a foundation wavefunction model. Such a model could yield high-accuracy ab-initio energies using only minimal computational effort for fine-tuning and evaluation of observables.
Deep neural networks have become a highly accurate and powerful wavefunction ansatz in combination with variational Monte Carlo methods for solving the electronic Schrödinger equation. However, despite their success and favorable scaling, these methods are still computationally too costly for wide adoption. A significant obstacle is the requirement to optimize the wavefunction from scratch for each new system, thus requiring long optimization. In this work, we propose a novel neural network ansatz, which effectively maps uncorrelated, computationally cheap Hartree-Fock orbitals, to correlated, high-accuracy neural network orbitals. This ansatz is inherently capable of learning a single wavefunction across multiple compounds and geometries, as we demonstrate by successfully transferring a wavefunction model pre-trained on smaller fragments to larger compounds. Furthermore, we provide ample experimental evidence to support the idea that extensive pre-training of a such a generalized wavefunction model across different compounds and geometries could lead to a foundation wavefunction model. Such a model could yield high-accuracy ab-initio energies using only minimal computational effort for fine-tuning and evaluation of observables.
Describing the electronic structure of molecules using computational methods is of central importance in many areas of chemical research and industry. To this day, the variational quantum Monte Carlo (VMC) algorithm, which utilizes the variational principle of quantum mechanics to optimize an ansatz towards the true electronic wave function, is one of the lesser employed of such computational methods. While it is generally applicable, and provides direct access to the wave function, its accuracy has been limited by insufficiently expressive ansatzes. This barrier was recently lifted with the introduction of deep-learning-based ansatzes, providing unprecedented flexibility and corresponding accuracy, at the cost of increased computational requirements. In this thesis, our implementation of the VMC algorithm with deep-learning ansatzes is described, along with a number of developments facilitating the computation of electronic excited states, extraction of various observables, and characterization of potential energy surfaces, at a reduced computational cost. The method is implemented as open-source software, employing a common framework to encompass most ansatzes present in the literature. Utilizing the direct wave function access granted by VMC, an approach is developed to extract one-electron reduced densities, that are free from Gaussian basis set effects. Significant advances in the methodology of computing electronic excited states via the ladder-of-states approach are made, including an improved form of the overlap penalty term with an automatically determined prefactor, and a new loss term with which the total spin of the solution can be selected. The efficiency of computing excited potential energy surfaces are enhanced by optimizing a single ansatz for a range of molecular configurations. To enable such transferable training in the presence of state crossings, a dynamic, geometry-wise ordering in the ladder of states is introduced. Geometry-optimization-based tasks are identified as natural fits for such transferable ansatzes, and a transferability-native approach for extracting the necessary information from VMC data is devised. Finally, transferable training is employed to simultaneously model different chemical species, resulting in a foundation model predicting bond breaking at a cost-accuracy ratio competitive with mature multi-reference methods. The work presented here contributes significantly to the development of a practically useful deep-learning-based VMC method. The described approaches have the potential to become tools for a wide community of computational chemists, as a source of accurate data, describing strong correlation in both ground and excited states.
Abstract Deep-learning-based variational Monte Carlo has emerged as a highly accurate method for solving the many-electron Schrödinger equation. Despite favorable scaling with the number of electrons, O(nel4) O ( n el 4 ) , the practical value of deep-learning-based variational Monte Carlo is limited by the high cost of optimizing the neural network weights for every system studied. Recent research has proposed optimizing a single neural network across multiple systems, reducing the cost per system. Here we extend this approach to solids, which require numerous calculations across different geometries, boundary conditions and supercell sizes. We demonstrate that optimization of a single ansatz across these variations significantly reduces optimization steps. Furthermore, we successfully transfer a network trained on 2 × 2 × 2 supercells of LiH, to 3 × 3 × 3 supercells, reducing the number of optimization steps required to simulate the large system by a factor of 50 compared with previous work.
The central challenge in materials science and quantum chemistry is solving the electronic Schrodinger equation, complicated by the curse of dimensionality as system size grows. Neural network-based variational Monte Carlo (NN-VMC) offers a promising path forward, achieving unprecedented accuracy at far lower cost than traditional high-level methods. However, the flexibility of neural network wavefunctions introduces a bottleneck: their optimization is high-dimensional, stochastic, and non-convex. This Perspective reviews the evolution of optimization methods in NN-VMC, from stochastic reconfiguration to approximate second-order algorithms and geometric insights. We highlight key challenges currently limiting scalability and efficiency, and outline future opportunities to advance the field. With continued progress in optimization, neural network techniques, and computer architectures, NN-VMC can tackle larger and more complex quantum systems and move from trailing experiments to guiding them.