Many quantum many-body wavefunctions, such as Jastrow-Slater, tensor network, and neural quantum states, are studied with the variational Monte Carlo technique, where stochastic optimization is usually performed to obtain a faithful approximation to the ground state of a given Hamiltonian. While first-order gradient descent methods are commonly used for such optimizations, quasi-second-order optimization formulations offer the potential of faster convergence under certain theoretical conditions, but with a similar cost per sample to first-order methods. However, the relative performance of first- and second-order optimizers is influenced in practice by many factors, including the sampling requirements for a faithful optimization step, the influence of wavefunction quality, as well as the wavefunction parametrization and expressivity. Here, we analyze these performance characteristics of first- and quasi-second-order optimization methods for a variety of Hamiltonians, with the additional context of understanding the scaling of these methods (for good performance) as a function of system size. Our findings help clarify the role of first- and quasi-second-order methods in variational Monte Carlo calculations and the conditions under which they should respectively be used. In particular, we find that unlike in deterministic optimization, where closeness to the variational minimum determines the suitability of second-order methods, in stochastic optimization the main factor is the overall expressivity of the wavefunction: Quasi-second-order methods lead to an overall reduction in cost relative to first-order methods when the wavefunction is sufficiently expressive to represent the ground state, even when starting far away from the ground state. This makes quasi-second-order methods an important technique when used with wavefunctions with arbitrarily improvable accuracy.
Many quantum many-body wavefunctions, such as Jastrow-Slater, tensor network, and neural quantum states, are studied with the variational Monte Carlo technique, where stochastic optimization is usually performed to obtain a faithful approximation to the ground state of a given Hamiltonian. While first-order gradient descent methods are commonly used for such optimizations, quasi-second-order optimization formulations offer the potential of faster convergence under certain theoretical conditions, but with a similar cost per sample to first-order methods. However, the relative performance of first- and second-order optimizers is influenced in practice by many factors, including the sampling requirements for a faithful optimization step, the influence of wavefunction quality, as well as the wavefunction parametrization and expressivity. Here, we analyze these performance characteristics of first- and quasi-second-order optimization methods for a variety of Hamiltonians, with the additional context of understanding the scaling of these methods (for good performance) as a function of system size. Our findings help clarify the role of first- and quasi-second-order methods in variational Monte Carlo calculations and the conditions under which they should respectively be used. In particular, we find that unlike in deterministic optimization, where closeness to the variational minimum determines the suitability of second-order methods, in stochastic optimization the main factor is the overall expressivity of the wavefunction: Quasi-second-order methods lead to an overall reduction in cost relative to first-order methods when the wavefunction is sufficiently expressive to represent the ground state, even when starting far away from the ground state. This makes quasi-second-order methods an important technique when used with wavefunctions with arbitrarily improvable accuracy.
The central challenge in materials science and quantum chemistry is solving the electronic Schrodinger equation, complicated by the curse of dimensionality as system size grows. Neural network-based variational Monte Carlo (NN-VMC) offers a promising path forward, achieving unprecedented accuracy at far lower cost than traditional high-level methods. However, the flexibility of neural network wavefunctions introduces a bottleneck: their optimization is high-dimensional, stochastic, and non-convex. This Perspective reviews the evolution of optimization methods in NN-VMC, from stochastic reconfiguration to approximate second-order algorithms and geometric insights. We highlight key challenges currently limiting scalability and efficiency, and outline future opportunities to advance the field. With continued progress in optimization, neural network techniques, and computer architectures, NN-VMC can tackle larger and more complex quantum systems and move from trailing experiments to guiding them.
Neural network quantum states (NQS), incorporating with variational Monte Carlo (VMC) method, are shown to be a promising way to investigate quantum many-body physics. Whereas vanilla VMC methods perform one gradient update per sample, we introduce a novel objective function with proximal optimization (PO) that enables multiple updates via reusing the mismatched samples. Our VMC-PO method keeps the advantage of the previous importance sampling gradient optimization algorithm [L. Yang, {\it et al}, Phys. Rev. Research {\bf 2}, 012039(R)(2020)] that efficiently uses sampled states. PO mitigates the numerical instabilities during network updates, which is similar to stochastic reconfiguration (SR) methods, but achieves an alternative and simpler implement with lower computational complexity. We investigate the performance of our VMC-PO algorithm for ground-state searching with a 1-dimensional transverse-field Ising model and 2-dimensional Heisenberg antiferromagnet on a square lattice, and demonstrate that the reached ground-state energies are comparable to state-of-the-art results.
Since the form of PN is the same as that occurring in the statistical mechanics of the classical gas (replace f 2 ( r ij ) by exp[- V ( r ij /kT ]), we can use the same integration techniques that are used in the classical problem. W. L. McMillan, 1965 Summary Building on the random walk methods developed in the previous chapter, we show how to compute properties of many-body trial wavefunctions using a random walk. This method, called variational Monte Carlo, is the simplest stochastic quantum many-body technique. Whereas mean-field methods are usually limited to single determinants, variational Monte Carlo can treat any correlated trial function, as long as its values are computable. We discuss how to optimize such trial wavefunctions, how to compute their momentum distribution, how to use non-local pseudopotentials, how to compute excited states, and how to correct for the finite size of the simulation cell. Deterministic quantum methods have difficulties. For example, the Hartree–Fock method assumes the wavefunction is a single Slater determinant, neglecting correlation. If one expands as a sum of determinants, it is very difficult to have the results size-consistent since the number of determinants needed will grow exponentially with the system size. As we have seen, the DMFT method introduced in Ch. 16 assumes locality. In Ch. 6 we discussed general properties of many-body wavefunctions. Using Monte Carlo methods, we can directly incorporate correlation into a wavefunction, without having to make any further approximations other than the form of the correlation factors. In many cases the energy and other properties are very close to the exact results. Some of the usual restrictions on the form of the many-body wavefunction are not an issue in variational Monte Carlo. The most important generalization of the HF wavefunction is to put correlation directly into the wavefunction via the “Jastrow” factor. At next order, one can use the “backflow” wavefunction, in which correlation is also built into the determinant. The variational Monte Carlo method (VMC) was first used by McMillan [44] to calculate the ground-state properties of superfluid 4 He. One of the key problems at that time was whether the observed superfluid properties were a consequence of Bose condensation.
We present a comparison between a number of recently introduced low-memory wave function optimization methods for variational Monte Carlo in which we find that first and second derivative methods possess strongly complementary relative advantages. While we find that low-memory variants of the linear method are vastly more efficient at bringing wave functions with disparate types of nonlinear parameters to the vicinity of the energy minimum, accelerated descent approaches are then able to locate the precise minimum with less bias and lower statistical uncertainty. By constructing a simple hybrid approach that combines these methodologies, we show that all of these advantages can be had at once when simultaneously optimizing large determinant expansions, molecular orbital shapes, traditional Jastrow correlation factors, and more nonlinear many-electron Jastrow factors.
We propose an algorithm for optimizing quantum Monte Carlo wave functions. An improved steepest-descent technique is used, with step size automatically adjustable to obtain a procedure that converges superlinearly. We also propose a novel trial function, which has both correct electron–electron cusp conditions and correct electron–nucleus cusp conditions. To test the optimization procedure and the optimized trial function, the ground states for CH4 and H2O molecules were investigated using variational Monte Carlo (VMC) and fixed-node quantum Monte Carlo (FNQMC) calculations. For CH4 and H2O, the VMC recovered 73.3% and 57.9% of the correlation energy, respectively, and the FNQMC recovered 99.3% and 92.8%, respectively. The optimization procedure is three to five times faster than conventional usual steepest-descent procedures. The trial functions optimized are more accurate than prior trial functions of similar complexity.
In this work, we introduce three algorithmic improvements to reduce the cost and improve the scaling of orbital space variational Monte Carlo (VMC). First, we show that by appropriately screening the one- and two-electron integrals of the Hamiltonian one can improve the efficiency of the algorithm by several orders of magnitude. This improved efficiency comes with the added benefit that the resulting algorithm scales as the second power of the system size O(N2), down from the fourth power O(N4). Using numerical results, we demonstrate that the practical scaling obtained is in fact O(N1.5) for a chain of Hydrogen atoms, and O(N1.2) for the Hubbard model. Second, we introduce the use of the rejection-free continuous time Monte Carlo (CTMC) to sample the determinants. CTMC is usually prohibitively expensive because of the need to calculate a large number of intermediates. Here, we take advantage of the fact that these intermediates are already calculated during the evaluation of the local energy and consequently, just by storing them one can use the CTCM algorithm with virtually no overhead. Third, we show that by using the adaptive stochastic gradient descent algorithm called AMSGrad one can optimize the wavefunction energies robustly and efficiently. The combination of these three improvements allows us to calculate the ground state energy of a chain of 160 hydrogen atoms using a wavefunction containing ∼2×105 variational parameters with an accuracy of 1 mEh/particle at a cost of just 25 CPU hours, which when split over 2 nodes of 24 processors each amounts to only about half hour of wall time. This low cost coupled with embarrassing parallelizability of the VMC algorithm and great freedom in the forms of usable wavefunctions, represents a highly effective method for calculating the electronic structure of model and \emph{ab initio} systems.