Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:cond-mat/9902300 · Physical review. B, Condensed matter

Monte Carlo energy and variance-minimization techniques for optimizing many-body wave functions

Paul R. C. Kent, Richard J. Needs, Gunaretnam Rajagopal

Published May 15, 1999• 109 citations
arXivHTMLPDF

Abstract

We investigate Monte Carlo energy and variance-minimization techniques for optimizing many-body wave functions. Several variants of the basic techniques are studied, including limiting the variations in the weighting factors that arise in correlated sampling estimations of the energy and its variance. We investigate the numerical stability of the techniques and identify two reasons why variance minimization exhibits superior numerical stability to energy minimization. The characteristics of each method are studied using a noninteracting 64-electron model of crystalline silicon. While our main interest is in solid-state systems, the issues investigated are relevant to Monte Carlo studies of atoms, molecules, and solids. We identify a robust and efficient variance-minimization scheme for optimizing wave functions for large systems.

DOI
10.1103/physrevb.59.12344

Related papers

Energy and Variance Optimization of Many-Body Wave Functions

Cyrus J. Umrigar, Claudia Filippi

We present a simple, robust, and efficient method for varying the parameters in a many-body wave function to optimize the expectation value of the energy. The effectiveness of the method is demonstrated by optimizing the parameters in flexible Jastrow factors that include 3-body electron-electron-nucleus correlation terms for the NO2 and decapentaene (C10H12) molecules. The basic idea is to add terms to the straightforward expression for the Hessian of the energy that have zero expectation value, but that cancel much of the statistical fluctuations for a finite Monte Carlo sample. The method is compared to what is currently the most popular method for optimizing many-body wave functions, namely, minimization of the variance of the local energy. The most efficient wave function is obtained by optimizing a linear combination of the energy and the variance.

arXiv:cond-mat/0412634•Physical Review Letters•Apr 21, 2005•196 citations

Variational Principles in Quantum Monte Carlo: The Troubled Story of Variance Minimization

Alice Cuzzocrea, Anthony Scemama, Wim J. Briels +2 more

We investigate the use of different variational principles in quantum Monte Carlo, namely, energy and variance minimization, prompted by the interest in the robust and accurate estimation of electronic excited states. For two prototypical, challenging molecules, we readily reach the accuracy of the best available reference excitation energies using energy minimization in a state-specific or state-average fashion for states of different or equal symmetry, respectively. On the other hand, in variance minimization, where the use of suitable functionals is expected to target specific states regardless of the symmetry, we encounter severe problems for a variety of wave functions: as the variance converges, the energy drifts away from that of the selected state. This unexpected behavior is sometimes observed even when the target is the ground state and generally prevents the robust estimation of total and excitation energies. We analyze this problem using a very simple wave function and infer that the optimization finds little or no barrier to escape from a local minimum or local plateau, eventually converging to a lower-variance state instead of the target state. For the increasingly complex systems becoming in reach of quantum Monte Carlo simulations, variance minimization with current functionals appears to be an impractical route.

arXiv:2002.05951•Journal of Chemical Theory and Computation•May 17, 2020•38 citations

Optimization of quantum Monte Carlo wave functions by energy minimization

Julien Toulouse, Cyrus J. Umrigar

We study three wave function optimization methods based on energy minimization in a variational Monte Carlo framework: the Newton, linear, and perturbative methods. In the Newton method, the parameter variations are calculated from the energy gradient and Hessian, using a reduced variance statistical estimator for the latter. In the linear method, the parameter variations are found by diagonalizing a nonsymmetric estimator of the Hamiltonian matrix in the space spanned by the wave function and its derivatives with respect to the parameters, making use of a strong zero-variance principle. In the less computationally expensive perturbative method, the parameter variations are calculated by approximately solving the generalized eigenvalue equation of the linear method by a nonorthogonal perturbation theory. These general methods are illustrated here by the optimization of wave functions consisting of a Jastrow factor multiplied by an expansion in configuration state functions (CSFs) for the C2 molecule, including both valence and core electrons in the calculation. The Newton and linear methods are very efficient for the optimization of the Jastrow, CSF, and orbital parameters. The perturbative method is a good alternative for the optimization of just the CSF and orbital parameters. Although the optimization is performed at the variational Monte Carlo level, we observe for the C2 molecule studied here, and for other systems we have studied, that as more parameters in the trial wave functions are optimized, the diffusion Monte Carlo total energy improves monotonically, implying that the nodal hypersurface also improves monotonically.

arXiv:physics/0701039•The Journal of Chemical Physics•Feb 22, 2007•290 citations

Variance-minimization scheme for optimizing Jastrow factors

N. D. Drummond, Richard J. Needs

We describe a scheme for optimizing many-electron trial wave functions by minimizing the unreweighted variance of the energy using stochastic integration and correlated-sampling techniques. The scheme is restricted to parameters that are linear in the exponent of a Jastrow correlation factor, which are the most important parameters in the wave functions we use. The scheme is highly efficient and allows us to investigate the parameter space more closely than has been possible before. We search for multiple minima of the variance in the parameter space and compare the wave functions obtained using reweighted and unreweighted variance minimization.

arXiv:physics/0505072•Physical Review B•Aug 18, 2005•147 citations

Histogram filtering as a tool in variational Monte Carlo optimization

Martin. Å najdr

Optimization of wave functions in quantum Monte Carlo is a difficult task because the statistical uncertainty inherent to the technique makes the absolute determination of the global minimum difficult. To optimize these wave functions we generate a large number of possible minima using many independently generated Monte Carlo ensembles and perform a conjugate gradient optimization. Then we construct histograms of the resulting nominally optimal parameter sets and "filter" them to identify which parameter sets "go together" to generate a local minimum. We follow with correlated-sampling verification runs to find the global minimum. We illustrate this technique for variance and variational energy optimization for a variety of wave functions for small systellls. For such optimized wave functions we calculate the variational energy and variance as well as various non-differential properties. The optimizations are either on par with or superior to determinations in the literature. Furthermore, we show that this technique is sufficiently robust that for molecules one may determine the optimal geometry at tIle same time as one optimizes the variational energy.

•Jul 9, 1999•0 citations

Generalized Lanczos algorithm for variational quantum Monte Carlo

Sandro Sorella

We show that the standard Lanczos algorithm can be efficiently implemented statistically and self-consistently improved, using the stochastic reconfiguration method, which has been recently introduced to stabilize the Monte Carlo sign problem instability. With this scheme a few Lanczos steps over a given variational wave function are possible even for large size as a particular case of a more general and more accurate technique that allows to obtain lower variational energies. This method has been tested extensively for a strongly correlated model like the $t\ensuremath{-}J$ model. With the standard Lanczos technique it is possible to compute any kind of correlation functions, with no particular computational effort. By using the fact that the variance $〈{H}^{2}〉\ensuremath{-}〈H{〉}^{2}$ is zero for an exact eigenstate, we show that the approach to the exact solution with few Lanczos iterations is indeed possible even for $\ensuremath{\sim}100$ electrons for reasonably good initial wave functions. The variational stochastic reconfiguration technique presented here allows in general a many-parameter energy optimization of any computable many-body wave function, including for instance generic long-range Jastrow factors and arbitrary site-dependent orbital determinants. This scheme improves further the accuracy of the calculation, especially for long-distance correlation functions.

arXiv:cond-mat/0009149•Physical review. B, Condensed matter•Jun 19, 2001•368 citations
We investigate Monte Carlo energy and variance-minimization techniques for optimizing many-body wave functions. Several variants of the basic techniques are studied, including limiting the variations in the weighting factors that arise in correlated sampling estimations of the energy and its variance. We investigate the numerical stability of the techniques and identify two reasons why variance minimization exhibits superior numerical stability to energy minimization. The characteristics of each method are studied using a noninteracting 64-electron model of crystalline silicon. While our main interest is in solid-state systems, the issues investigated are relevant to Monte Carlo studies of atoms, molecules, and solids. We identify a robust and efficient variance-minimization scheme for optimizing wave functions for large systems.
We present a simple, robust, and efficient method for varying the parameters in a many-body wave function to optimize the expectation value of the energy. The effectiveness of the method is demonstrated by optimizing the parameters in flexible Jastrow factors that include 3-body electron-electron-nucleus correlation terms for the NO2 and decapentaene (C10H12) molecules. The basic idea is to add terms to the straightforward expression for the Hessian of the energy that have zero expectation value, but that cancel much of the statistical fluctuations for a finite Monte Carlo sample. The method is compared to what is currently the most popular method for optimizing many-body wave functions, namely, minimization of the variance of the local energy. The most efficient wave function is obtained by optimizing a linear combination of the energy and the variance.
We investigate the use of different variational principles in quantum Monte Carlo, namely, energy and variance minimization, prompted by the interest in the robust and accurate estimation of electronic excited states. For two prototypical, challenging molecules, we readily reach the accuracy of the best available reference excitation energies using energy minimization in a state-specific or state-average fashion for states of different or equal symmetry, respectively. On the other hand, in variance minimization, where the use of suitable functionals is expected to target specific states regardless of the symmetry, we encounter severe problems for a variety of wave functions: as the variance converges, the energy drifts away from that of the selected state. This unexpected behavior is sometimes observed even when the target is the ground state and generally prevents the robust estimation of total and excitation energies. We analyze this problem using a very simple wave function and infer that the optimization finds little or no barrier to escape from a local minimum or local plateau, eventually converging to a lower-variance state instead of the target state. For the increasingly complex systems becoming in reach of quantum Monte Carlo simulations, variance minimization with current functionals appears to be an impractical route.
We study three wave function optimization methods based on energy minimization in a variational Monte Carlo framework: the Newton, linear, and perturbative methods. In the Newton method, the parameter variations are calculated from the energy gradient and Hessian, using a reduced variance statistical estimator for the latter. In the linear method, the parameter variations are found by diagonalizing a nonsymmetric estimator of the Hamiltonian matrix in the space spanned by the wave function and its derivatives with respect to the parameters, making use of a strong zero-variance principle. In the less computationally expensive perturbative method, the parameter variations are calculated by approximately solving the generalized eigenvalue equation of the linear method by a nonorthogonal perturbation theory. These general methods are illustrated here by the optimization of wave functions consisting of a Jastrow factor multiplied by an expansion in configuration state functions (CSFs) for the C2 molecule, including both valence and core electrons in the calculation. The Newton and linear methods are very efficient for the optimization of the Jastrow, CSF, and orbital parameters. The perturbative method is a good alternative for the optimization of just the CSF and orbital parameters. Although the optimization is performed at the variational Monte Carlo level, we observe for the C2 molecule studied here, and for other systems we have studied, that as more parameters in the trial wave functions are optimized, the diffusion Monte Carlo total energy improves monotonically, implying that the nodal hypersurface also improves monotonically.
We describe a scheme for optimizing many-electron trial wave functions by minimizing the unreweighted variance of the energy using stochastic integration and correlated-sampling techniques. The scheme is restricted to parameters that are linear in the exponent of a Jastrow correlation factor, which are the most important parameters in the wave functions we use. The scheme is highly efficient and allows us to investigate the parameter space more closely than has been possible before. We search for multiple minima of the variance in the parameter space and compare the wave functions obtained using reweighted and unreweighted variance minimization.
Optimization of wave functions in quantum Monte Carlo is a difficult task because the statistical uncertainty inherent to the technique makes the absolute determination of the global minimum difficult. To optimize these wave functions we generate a large number of possible minima using many independently generated Monte Carlo ensembles and perform a conjugate gradient optimization. Then we construct histograms of the resulting nominally optimal parameter sets and "filter" them to identify which parameter sets "go together" to generate a local minimum. We follow with correlated-sampling verification runs to find the global minimum. We illustrate this technique for variance and variational energy optimization for a variety of wave functions for small systellls. For such optimized wave functions we calculate the variational energy and variance as well as various non-differential properties. The optimizations are either on par with or superior to determinations in the literature. Furthermore, we show that this technique is sufficiently robust that for molecules one may determine the optimal geometry at tIle same time as one optimizes the variational energy.
We show that the standard Lanczos algorithm can be efficiently implemented statistically and self-consistently improved, using the stochastic reconfiguration method, which has been recently introduced to stabilize the Monte Carlo sign problem instability. With this scheme a few Lanczos steps over a given variational wave function are possible even for large size as a particular case of a more general and more accurate technique that allows to obtain lower variational energies. This method has been tested extensively for a strongly correlated model like the $t\ensuremath{-}J$ model. With the standard Lanczos technique it is possible to compute any kind of correlation functions, with no particular computational effort. By using the fact that the variance $〈{H}^{2}〉\ensuremath{-}〈H{〉}^{2}$ is zero for an exact eigenstate, we show that the approach to the exact solution with few Lanczos iterations is indeed possible even for $\ensuremath{\sim}100$ electrons for reasonably good initial wave functions. The variational stochastic reconfiguration technique presented here allows in general a many-parameter energy optimization of any computable many-body wave function, including for instance generic long-range Jastrow factors and arbitrary site-dependent orbital determinants. This scheme improves further the accuracy of the calculation, especially for long-distance correlation functions.