Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:1912.01962 · The Journal of Chemical Physics

Unbiasing the initiator approximation in full configuration interaction quantum Monte Carlo

Khaldoon Ghanem, Alexander Y. Lozovoi, Ali Alavi

Published Dec 11, 2019• 73 citations
arXivHTMLPDF

Abstract

We identify and rectify a crucial source of bias in the initiator full configuration interaction quantum Monte Carlo algorithm. Noninitiator determinants (i.e., determinants whose population is below the initiator threshold) are subject to a systematic undersampling bias, which in large systems leads to a bias in the energy when an insufficient number of walkers are used. We show that the acceptance probability (pacc), that a noninitiator determinant has its spawns accepted, can be used to unbias the initiator bias, in a simple and accurate manner, by reducing the applied shift to the noninitiator proportionately to pacc. This modification preserves the property that in the large walker limit, when pacc → 1, the unbiasing procedure disappears, and the initiator approximation becomes exact. We demonstrate that this algorithm shows rapid convergence to the FCI limit with respect to the walker number and, furthermore, largely removes the dependence of the algorithm on the initiator threshold, enabling highly accurate results to be obtained even with large values of the threshold. This is exemplified in the case of butadiene/ANO-L-pVDZ and benzene/cc-pVDZ, correlating 22 and 30 electrons in 82 and 108 orbitals, respectively. In butadiene 5 × 107 and in benzene 108 walkers suffice to obtain an energy within a millihartree of the coupled cluster singles doubles triples and perturbative quadruples [CCSDT(Q)] result in Hilbert spaces of 1026 and 1035, respectively. Essentially converged results require ∼108 walkers for butadiene and ∼109 walkers for benzene and lie slightly lower than CCSDT(Q). Owing to large-scale parallelizability, these calculations can be executed in a matter of hours on a few hundred processors. The present method largely solves the initiator-bias problems that the initiator method suffered from when applied to medium-sized molecules.

DOI
10.1063/1.5134006

Related papers

Unbiasing the initiator approximation in full configuration interaction quantum Monte Carlo.

Khaldoon Ghanem, Alexander Y. Lozovoi, Ali Alavi

We identify and rectify a crucial source of bias in the initiator full configuration interaction quantum Monte Carlo algorithm. Noninitiator determinants (i.e., determinants whose population is below the initiator threshold) are subject to a systematic undersampling bias, which in large systems leads to a bias in the energy when an insufficient number of walkers are used. We show that the acceptance probability (pacc), that a noninitiator determinant has its spawns accepted, can be used to unbias the initiator bias, in a simple and accurate manner, by reducing the applied shift to the noninitiator proportionately to pacc. This modification preserves the property that in the large walker limit, when pacc → 1, the unbiasing procedure disappears, and the initiator approximation becomes exact. We demonstrate that this algorithm shows rapid convergence to the FCI limit with respect to the walker number and, furthermore, largely removes the dependence of the algorithm on the initiator threshold, enabling highly accurate results to be obtained even with large values of the threshold. This is exemplified in the case of butadiene/ANO-L-pVDZ and benzene/cc-pVDZ, correlating 22 and 30 electrons in 82 and 108 orbitals, respectively. In butadiene 5 × 107 and in benzene 108 walkers suffice to obtain an energy within a millihartree of the coupled cluster singles doubles triples and perturbative quadruples [CCSDT(Q)] result in Hilbert spaces of 1026 and 1035, respectively. Essentially converged results require ∼108 walkers for butadiene and ∼109 walkers for benzene and lie slightly lower than CCSDT(Q). Owing to large-scale parallelizability, these calculations can be executed in a matter of hours on a few hundred processors. The present method largely solves the initiator-bias problems that the initiator method suffered from when applied to medium-sized molecules.

•Dec 14, 2019•30 citations

Communications: Survival of the fittest: Accelerating convergence in full configuration-interaction quantum Monte Carlo

Deidre M. Cleland, George H. Booth, Ali Alavi

We provide a very simple adaptation of our recently published quantum Monte Carlo algorithm in full configuration-interaction (Slater determinant) spaces which dramatically reduces the number of walkers required to achieve convergence. A survival criterion is imposed for newly spawned walkers. We define a set of initiator determinants such that progeny of walkers spawned from such determinants onto unoccupied determinants are able to survive, while the progeny of walkers not in this set can survive only if they are spawned onto determinants which are already occupied. The set of initiators is originally defined to be all determinants constructable from a subset of orbitals, in analogy with complete-active spaces. This set is dynamically updated so that if a noninitiator determinant reaches an occupation larger than a preset limit, it becomes an initiator. The new algorithm allows sign-coherent sampling of the FCI space to be achieved with relatively few walkers. Using the N(2) molecule as an illustration, we show that rather small initiator spaces and numbers of walkers can converge with submilliHartree accuracy to the known full configuration-interaction (FCI) energy (in the cc-pVDZ basis), in both the equilibrium geometry and the multiconfigurational stretched case. We use the same method to compute the energy with cc-pVTZ and cc-pVQZ basis sets, the latter having an FCI space of over 10(15) with very modest computational resources.

The Journal of Chemical Physics•Jan 28, 2010•392 citations

The adaptive shift method in full configuration interaction quantum Monte Carlo: Development and applications

Khaldoon Ghanem, Kai Guther, Ali Alavi

In a recent paper, we proposed the adaptive shift method for correcting undersampling bias of the initiator-full configuration interaction (FCI) quantum Monte Carlo. The method allows faster convergence with the number of walkers to the FCI limit than the normal initiator method, particularly for large systems. However, in its application to some systems, mostly strongly correlated molecules, the method is prone to overshooting the FCI energy at intermediate walker numbers, with convergence to the FCI limit from below. In this paper, we present a solution to the overshooting problem in such systems, as well as further accelerating convergence to the FCI energy. This is achieved by offsetting the reference energy to a value typically below the Hartree–Fock energy but above the exact energy. This offsetting procedure does not change the exactness property of the algorithm, namely, convergence to the exact FCI solution in the large-walker limit, but at its optimal value, it greatly accelerates convergence. There is no overhead cost associated with this offsetting procedure and is therefore a pure and substantial computational gain. We illustrate the behavior of this offset adaptive shift method by applying it to the N2 molecule, the ozone molecule at three different geometries (an equilibrium open minimum, a hypothetical ring minimum, and a transition state) in three basis sets (cc-pVXZ, X = D, T, Q), and the chromium dimer in the cc-pVDZ basis set, correlating 28 electrons in 76 orbitals. We show that in most cases, the offset adaptive shift method converges much faster than both the normal initiator method and the original adaptive shift method.

arXiv:2010.05743•The Journal of Chemical Physics•Dec 11, 2020•45 citations

Toward a Full Configurational Accuracy Calculation of an Arbitrary Molecule via Fragment Embedding and a Stochastic Solver

Yi Biao Sun

We demonstrate the feasibility of using a stochastic solver, full configuration interaction quantum Monte Carlo with the initiator approximation (i-FCIQMC), to converge fragment embedding calculations, namely bootstrap embedding (BE). We first propose and test a general protocol for converging BE-i-FCIQMC calculations and then suggest how the quality of the calculation compares against that of deterministic BE-FCI using different numbers of walkers. We then demonstrate that BE-i-FCIQMC can perform as well as BE-FCI in the large walker limit and how different factors, including the size of the Hilbert space of the fragments, the number of walkers, and the nature of the chemical system, affect the achievable matching error. We finally perform BE-FCI calculations in realistic systems like benzene and cyclohexane using a double-ζ basis set. This work demonstrates the potential of performing FCI quality calculations in realistic systems using BE.

The Journal of Physical Chemistry Letters•Apr 11, 2024•8 citations

Breaking the carbon dimer: The challenges of multiple bond dissociation with full configuration interaction quantum Monte Carlo methods

George H. Booth, Deidre M. Cleland, Alex J. W. Thom +1 more

The full configuration interaction quantum Monte Carlo (FCIQMC) method, as well as its "initiator" extension (i-FCIQMC), is used to tackle the complex electronic structure of the carbon dimer across the entire dissociation reaction coordinate, as a prototypical example of a strongly correlated molecular system. Various basis sets of increasing size up to the large cc-pVQZ are used, spanning a fully accessible N-electron basis of over 10(12) Slater determinants, and the accuracy of the method is demonstrated in each basis set. Convergence to the FCI limit is achieved in the largest basis with only O[10(7)] walkers within random errorbars of a few tenths of a millihartree across the binding curve, and extensive comparisons to FCI, CCSD(T), MRCI, and CEEIS results are made where possible. A detailed exposition of the convergence properties of the FCIQMC methods is provided, considering convergence with elapsed imaginary time, number of walkers and size of the basis. Various symmetries which can be incorporated into the stochastic dynamic, beyond the standard abelian point group symmetry and spin polarisation are also described. These can have significant benefit to the computational effort of the calculations, as well as the ability to converge to various excited states. The results presented demonstrate a new benchmark accuracy in basis-set energies for systems of this size, significantly improving on previous state of the art estimates.

The Journal of Chemical Physics•Aug 22, 2011•158 citations

An explicitly correlated approach to basis set incompleteness in full configuration interaction quantum Monte Carlo

George H. Booth, Deidre M. Cleland, Ali Alavi +1 more

By performing a stochastic dynamic in a space of Slater determinants, the full configuration interaction quantum Monte Carlo (FCIQMC) method has been able to obtain energies which are essentially free from systematic error to the basis set correlation energy, within small and systematically improvable error bars. However, the weakly exponential scaling with basis size makes converging the energy with respect to basis set costly and in larger systems, impossible. To ameliorate these basis set issues, here we use perturbation theory to couple the FCIQMC wavefunction to an explicitly correlated strongly orthogonal basis of geminals, following the [2](R12) approach of Valeev et al. The required one- and two-particle density matrices are computed on-the-fly during the FCIQMC dynamic, using a sampling procedure which incurs relatively little additional computation expense. The F12 energy corrections are shown to converge rapidly as a function of sampling, both in imaginary time and number of walkers. Our pilot calculations on the binding curve for the carbon dimer, which exhibits strong correlation effects as well as substantial basis set dependence, demonstrate that the accuracy of the FCIQMC-F12 method surpasses that of all previous FCIQMC calculations, and that the F12 correction improves results equivalent to increasing the quality of the one-electron basis by two cardinal numbers.

arXiv:1208.0980•The Journal of Chemical Physics•Oct 28, 2012•60 citations
We identify and rectify a crucial source of bias in the initiator full configuration interaction quantum Monte Carlo algorithm. Noninitiator determinants (i.e., determinants whose population is below the initiator threshold) are subject to a systematic undersampling bias, which in large systems leads to a bias in the energy when an insufficient number of walkers are used. We show that the acceptance probability (pacc), that a noninitiator determinant has its spawns accepted, can be used to unbias the initiator bias, in a simple and accurate manner, by reducing the applied shift to the noninitiator proportionately to pacc. This modification preserves the property that in the large walker limit, when pacc → 1, the unbiasing procedure disappears, and the initiator approximation becomes exact. We demonstrate that this algorithm shows rapid convergence to the FCI limit with respect to the walker number and, furthermore, largely removes the dependence of the algorithm on the initiator threshold, enabling highly accurate results to be obtained even with large values of the threshold. This is exemplified in the case of butadiene/ANO-L-pVDZ and benzene/cc-pVDZ, correlating 22 and 30 electrons in 82 and 108 orbitals, respectively. In butadiene 5 × 107 and in benzene 108 walkers suffice to obtain an energy within a millihartree of the coupled cluster singles doubles triples and perturbative quadruples [CCSDT(Q)] result in Hilbert spaces of 1026 and 1035, respectively. Essentially converged results require ∼108 walkers for butadiene and ∼109 walkers for benzene and lie slightly lower than CCSDT(Q). Owing to large-scale parallelizability, these calculations can be executed in a matter of hours on a few hundred processors. The present method largely solves the initiator-bias problems that the initiator method suffered from when applied to medium-sized molecules.
We identify and rectify a crucial source of bias in the initiator full configuration interaction quantum Monte Carlo algorithm. Noninitiator determinants (i.e., determinants whose population is below the initiator threshold) are subject to a systematic undersampling bias, which in large systems leads to a bias in the energy when an insufficient number of walkers are used. We show that the acceptance probability (pacc), that a noninitiator determinant has its spawns accepted, can be used to unbias the initiator bias, in a simple and accurate manner, by reducing the applied shift to the noninitiator proportionately to pacc. This modification preserves the property that in the large walker limit, when pacc → 1, the unbiasing procedure disappears, and the initiator approximation becomes exact. We demonstrate that this algorithm shows rapid convergence to the FCI limit with respect to the walker number and, furthermore, largely removes the dependence of the algorithm on the initiator threshold, enabling highly accurate results to be obtained even with large values of the threshold. This is exemplified in the case of butadiene/ANO-L-pVDZ and benzene/cc-pVDZ, correlating 22 and 30 electrons in 82 and 108 orbitals, respectively. In butadiene 5 × 107 and in benzene 108 walkers suffice to obtain an energy within a millihartree of the coupled cluster singles doubles triples and perturbative quadruples [CCSDT(Q)] result in Hilbert spaces of 1026 and 1035, respectively. Essentially converged results require ∼108 walkers for butadiene and ∼109 walkers for benzene and lie slightly lower than CCSDT(Q). Owing to large-scale parallelizability, these calculations can be executed in a matter of hours on a few hundred processors. The present method largely solves the initiator-bias problems that the initiator method suffered from when applied to medium-sized molecules.
We provide a very simple adaptation of our recently published quantum Monte Carlo algorithm in full configuration-interaction (Slater determinant) spaces which dramatically reduces the number of walkers required to achieve convergence. A survival criterion is imposed for newly spawned walkers. We define a set of initiator determinants such that progeny of walkers spawned from such determinants onto unoccupied determinants are able to survive, while the progeny of walkers not in this set can survive only if they are spawned onto determinants which are already occupied. The set of initiators is originally defined to be all determinants constructable from a subset of orbitals, in analogy with complete-active spaces. This set is dynamically updated so that if a noninitiator determinant reaches an occupation larger than a preset limit, it becomes an initiator. The new algorithm allows sign-coherent sampling of the FCI space to be achieved with relatively few walkers. Using the N(2) molecule as an illustration, we show that rather small initiator spaces and numbers of walkers can converge with submilliHartree accuracy to the known full configuration-interaction (FCI) energy (in the cc-pVDZ basis), in both the equilibrium geometry and the multiconfigurational stretched case. We use the same method to compute the energy with cc-pVTZ and cc-pVQZ basis sets, the latter having an FCI space of over 10(15) with very modest computational resources.
In a recent paper, we proposed the adaptive shift method for correcting undersampling bias of the initiator-full configuration interaction (FCI) quantum Monte Carlo. The method allows faster convergence with the number of walkers to the FCI limit than the normal initiator method, particularly for large systems. However, in its application to some systems, mostly strongly correlated molecules, the method is prone to overshooting the FCI energy at intermediate walker numbers, with convergence to the FCI limit from below. In this paper, we present a solution to the overshooting problem in such systems, as well as further accelerating convergence to the FCI energy. This is achieved by offsetting the reference energy to a value typically below the Hartree–Fock energy but above the exact energy. This offsetting procedure does not change the exactness property of the algorithm, namely, convergence to the exact FCI solution in the large-walker limit, but at its optimal value, it greatly accelerates convergence. There is no overhead cost associated with this offsetting procedure and is therefore a pure and substantial computational gain. We illustrate the behavior of this offset adaptive shift method by applying it to the N2 molecule, the ozone molecule at three different geometries (an equilibrium open minimum, a hypothetical ring minimum, and a transition state) in three basis sets (cc-pVXZ, X = D, T, Q), and the chromium dimer in the cc-pVDZ basis set, correlating 28 electrons in 76 orbitals. We show that in most cases, the offset adaptive shift method converges much faster than both the normal initiator method and the original adaptive shift method.
We demonstrate the feasibility of using a stochastic solver, full configuration interaction quantum Monte Carlo with the initiator approximation (i-FCIQMC), to converge fragment embedding calculations, namely bootstrap embedding (BE). We first propose and test a general protocol for converging BE-i-FCIQMC calculations and then suggest how the quality of the calculation compares against that of deterministic BE-FCI using different numbers of walkers. We then demonstrate that BE-i-FCIQMC can perform as well as BE-FCI in the large walker limit and how different factors, including the size of the Hilbert space of the fragments, the number of walkers, and the nature of the chemical system, affect the achievable matching error. We finally perform BE-FCI calculations in realistic systems like benzene and cyclohexane using a double-ζ basis set. This work demonstrates the potential of performing FCI quality calculations in realistic systems using BE.
The full configuration interaction quantum Monte Carlo (FCIQMC) method, as well as its "initiator" extension (i-FCIQMC), is used to tackle the complex electronic structure of the carbon dimer across the entire dissociation reaction coordinate, as a prototypical example of a strongly correlated molecular system. Various basis sets of increasing size up to the large cc-pVQZ are used, spanning a fully accessible N-electron basis of over 10(12) Slater determinants, and the accuracy of the method is demonstrated in each basis set. Convergence to the FCI limit is achieved in the largest basis with only O[10(7)] walkers within random errorbars of a few tenths of a millihartree across the binding curve, and extensive comparisons to FCI, CCSD(T), MRCI, and CEEIS results are made where possible. A detailed exposition of the convergence properties of the FCIQMC methods is provided, considering convergence with elapsed imaginary time, number of walkers and size of the basis. Various symmetries which can be incorporated into the stochastic dynamic, beyond the standard abelian point group symmetry and spin polarisation are also described. These can have significant benefit to the computational effort of the calculations, as well as the ability to converge to various excited states. The results presented demonstrate a new benchmark accuracy in basis-set energies for systems of this size, significantly improving on previous state of the art estimates.
By performing a stochastic dynamic in a space of Slater determinants, the full configuration interaction quantum Monte Carlo (FCIQMC) method has been able to obtain energies which are essentially free from systematic error to the basis set correlation energy, within small and systematically improvable error bars. However, the weakly exponential scaling with basis size makes converging the energy with respect to basis set costly and in larger systems, impossible. To ameliorate these basis set issues, here we use perturbation theory to couple the FCIQMC wavefunction to an explicitly correlated strongly orthogonal basis of geminals, following the [2](R12) approach of Valeev et al. The required one- and two-particle density matrices are computed on-the-fly during the FCIQMC dynamic, using a sampling procedure which incurs relatively little additional computation expense. The F12 energy corrections are shown to converge rapidly as a function of sampling, both in imaginary time and number of walkers. Our pilot calculations on the binding curve for the carbon dimer, which exhibits strong correlation effects as well as substantial basis set dependence, demonstrate that the accuracy of the FCIQMC-F12 method surpasses that of all previous FCIQMC calculations, and that the F12 correction improves results equivalent to increasing the quality of the one-electron basis by two cardinal numbers.