Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:1610.06660 · Journal of Chemical Theory and Computation

Semistochastic Heat-Bath Configuration Interaction Method: Selected Configuration Interaction with Semistochastic Perturbation Theory

Sandeep Sharma, Adam A. Holmes, Guillaume Jeanmairet, Ali Alavi, Cyrus J. Umrigar

Published Mar 6, 2017• 364 citations
arXivHTMLPDF

Abstract

We extend the recently proposed heat-bath configuration interaction (HCI) method [Holmes, Tubman, Umrigar, J. Chem. Theory Comput. 2016, 12, 3674], by introducing a semistochastic algorithm for performing multireference Epstein–Nesbet perturbation theory, in order to completely eliminate the severe memory bottleneck of the original method. The proposed algorithm has several attractive features. First, there is no sign problem that plagues several quantum Monte Carlo methods. Second, instead of using Metropolis–Hastings sampling, we use the Alias method to directly sample determinants from the reference wave function, thus avoiding correlations between consecutive samples. Third, in addition to removing the memory bottleneck, semistochastic HCI (SHCI) is faster than the deterministic variant for many systems if a stochastic error of 0.1 mHa is acceptable. Fourth, within the SHCI algorithm one can trade memory for a modest increase in computer time. Fifth, the perturbative calculation is embarrassingly parallel. The SHCI algorithm extends the range of applicability of the original algorithm, allowing us to calculate the correlation energy of very large active spaces. We demonstrate this by performing calculations on several first row dimers including F 2 with an active space of (14e, 108o), Mn-Salen cluster with an active space of (28e, 22o), and Cr 2 dimer with up to a quadruple-ζ basis set with an active space of (12e, 190o). For these systems we were able to obtain better than 1 mHa accuracy with a wall time of merely 55 s, 37 s, and 56 min on 1, 1, and 4 nodes, respectively.

DOI
10.1021/acs.jctc.6b01028

Related papers

Fast semistochastic heat-bath configuration interaction

Junhao Li, Matthew Otten, Adam A. Holmes +2 more

This paper presents in detail our fast semistochastic heat-bath configuration interaction (SHCI) method for solving the many-body Schrödinger equation. We identify and eliminate computational bottlenecks in both the variational and perturbative steps of the SHCI algorithm. We also describe the parallelization and the key data structures in our implementation, such as the distributed hash table. The improved SHCI algorithm enables us to include in our variational wavefunction two orders of magnitude more determinants than has been reported previously with other selected configuration interaction methods. We use our algorithm to calculate an accurate benchmark energy for the chromium dimer with the X2C relativistic Hamiltonian in the cc-pVDZ-DK basis, correlating 28 electrons in 76 spatial orbitals. Our largest calculation uses two billion Slater determinants in the variational space and semistochastically includes perturbative contributions from at least trillions of additional determinants with better than 10−5 Ha statistical uncertainty.

arXiv:1809.04600•The Journal of Chemical Physics•Dec 6, 2018•144 citations

Almost exact energies for the Gaussian-2 set with the semistochastic heat-bath configuration interaction method

Yuan Yao, Emmanuel Giner, Junhao Li +2 more

The recently developed semistochastic heat-bath configuration interaction (SHCI) method is a systematically improvable selected configuration interaction plus perturbation theory method capable of giving essentially exact energies for larger systems than is possible with other such methods. We compute SHCI atomization energies for 55 molecules that have been used as a test set in prior studies because their atomization energies are known from experiment. Basis sets from cc-pVDZ to cc-pV5Z are used, totaling up to 500 orbitals and a Hilbert space of 1032 Slater determinants for the largest molecules. For each basis, an extrapolated energy well within chemical accuracy (1 kcal/mol or 1.6 mHa/mol) of the exact energy for that basis is computed using only a tiny fraction of the entire Hilbert space. We also use our almost exact energies to benchmark energies from the coupled cluster method with single, double, and perturbative triple excitations. The energies are extrapolated to the complete basis set limit and compared to the experimental atomization energies. The extrapolations are done both without and with a basis-set correction based on density-functional theory. The mean absolute deviations from experiment for these extrapolations are 0.46 kcal/mol and 0.51 kcal/mol, respectively. Orbital optimization methods used to obtain improved convergence of the SHCI energies are also discussed.

arXiv:2004.10059•The Journal of Chemical Physics•Sep 28, 2020•56 citations

Fast Semistochastic Heat-Bath Configuration Interaction

Junhao Li

In this thesis, I present my work on the fast semistochastic heatbath configuration interaction (Fast SHCI), which is an efficient algorithm for doing essentially exact electronic structure calculations within a finite basis. There are Hamiltonians for which the entire Hilbert space is enormous, but the important part of Hilbert space is of manageable size, say $10^{12}$. Quantum chemistry Hamiltonians, for reasonably small systems, have this property. For such Hamiltonians, selected configuration interaction plus perturbation theory (SCI+PT) methods can be useful. The most important part of the Hilbert space is treated variationally, and the resulting energy is improved by using perturbation theory. Fast SHCI is more than an order of magnitude faster than other SCI+PT algorithms, and also much faster than other essentially exact algorithms for many chemical systems. This thesis provides an in-depth description of the Fast SHCI algorithm and its implementation. I use SHCI to compute the electronic structure of several chemical systems and the homogeneous electron gas. Some of these calculations are more accurate than those achieved by other high-order quantum chemistry methods. Others treat systems larger than those that can be treated by other equally accurate methods. My implementation of SHCI uses a modular design, which not only makes the library highly extensible but also contributes several generic distributed computing building blocks to the open-source community. In this thesis, I also describe my design and implementation of these generic components. Finally, I also provide a brief discussion of the usability of general software engineering best practices for the development of medium-scale scientific software packages with lessons learned from designing, developing, and leading the development of our SHCI package. Medium-scale scientific software packages are common in scientific research where a small group of researchers works on the same code base. Due to the differences in the requirements, some best practices that are common in the industry need to be adjusted to be useful for these projects.

•Jan 1, 2019•102 citations

Efficient Heat-Bath Sampling in Fock Space

Adam A. Holmes, Hitesh J. Changlani, Cyrus J. Umrigar

We introduce an algorithm for sampling many-body quantum states in Fock space. The algorithm efficiently samples states with probability approximately proportional to an arbitrary function of the second-quantized Hamiltonian matrix element connecting the sampled state to the current state. We apply the new sampling algorithm to the recently developed semistochastic full configuration interaction quantum Monte Carlo (S-FCIQMC) method, a semistochastic implementation of the power method for projecting out the ground state energy in a basis of Slater determinants. Our new sampling method requires modest additional computational time and memory compared to uniform sampling but results in newly spawned weights that are approximately of the same magnitude, thereby greatly improving the efficiency of projection. A comparison in efficiency between our sampling algorithm and uniform sampling is performed on the all-electron nitrogen dimer at equilibrium in Dunning's cc-pVXZ basis sets with X ∈ {D, T, Q, 5}, demonstrating a large gain in efficiency that increases with basis set size. In addition, a comparison in efficiency is performed on three all-electron first-row dimers, B2, N2, and F2, in a cc-pVQZ basis, demonstrating that the gain in efficiency compared to uniform sampling also increases dramatically with the number of electrons.

arXiv:1512.03757•Journal of Chemical Theory and Computation•Mar 9, 2016•105 citations

Accurate many-body electronic structure near the basis set limit: Application to the chromium dimer

Junhao Li, Yuan Yao, Adam A. Holmes +4 more

The authors use the recently developed semistochastic heat bath configuration interaction method to calculate the potential energy curve of the very challenging Cr${}_{2}$ dimer. Despite the fact that the largest Hilbert space has dimension ${10}^{42}$, the paper obtains energies with estimated errors of a few mHa or less.

arXiv:1909.12319•Physical Review Research•Jan 15, 2020•49 citations

Full Configuration Interaction Excited-State Energies in Large Active Spaces from Subspace Iteration with Repeated Random Sparsification

Samuel M. Greene, Robert J. Webber, James E. T. Smith +2 more

We present a stable and systematically improvable quantum Monte Carlo (QMC) approach to calculating excited-state energies, which we implement using our fast randomized iteration method for the full configuration interaction problem (FCI-FRI). Unlike previous excited-state quantum Monte Carlo methods, our approach, which is based on an asymmetric variant of subspace iteration, avoids the use of dot products of random vectors and instead relies upon trial vectors to maintain orthogonality and estimate eigenvalues. By leveraging recent advances, we apply our method to calculate ground- and excited-state energies of challenging molecular systems in large active spaces, including the carbon dimer with 8 electrons in 108 orbitals (8e,108o), an oxo-Mn(salen) transition metal complex (28e,28o), ozone (18e,87o), and butadiene (22e,82o). In the majority of these test cases, our approach yields total excited-state energies that agree with those from state-of-the-art methods─including heat-bath CI, the density matrix renormalization group approach, and FCIQMC─to within sub-milliHartree accuracy. In all cases, estimated excitation energies agree to within about 0.1 eV.

arXiv:2201.12164•Journal of Chemical Theory and Computation•Nov 8, 2022•4 citations
We extend the recently proposed heat-bath configuration interaction (HCI) method [Holmes, Tubman, Umrigar, J. Chem. Theory Comput. 2016, 12, 3674], by introducing a semistochastic algorithm for performing multireference Epstein–Nesbet perturbation theory, in order to completely eliminate the severe memory bottleneck of the original method. The proposed algorithm has several attractive features. First, there is no sign problem that plagues several quantum Monte Carlo methods. Second, instead of using Metropolis–Hastings sampling, we use the Alias method to directly sample determinants from the reference wave function, thus avoiding correlations between consecutive samples. Third, in addition to removing the memory bottleneck, semistochastic HCI (SHCI) is faster than the deterministic variant for many systems if a stochastic error of 0.1 mHa is acceptable. Fourth, within the SHCI algorithm one can trade memory for a modest increase in computer time. Fifth, the perturbative calculation is embarrassingly parallel. The SHCI algorithm extends the range of applicability of the original algorithm, allowing us to calculate the correlation energy of very large active spaces. We demonstrate this by performing calculations on several first row dimers including F 2 with an active space of (14e, 108o), Mn-Salen cluster with an active space of (28e, 22o), and Cr 2 dimer with up to a quadruple-ζ basis set with an active space of (12e, 190o). For these systems we were able to obtain better than 1 mHa accuracy with a wall time of merely 55 s, 37 s, and 56 min on 1, 1, and 4 nodes, respectively.
This paper presents in detail our fast semistochastic heat-bath configuration interaction (SHCI) method for solving the many-body Schrödinger equation. We identify and eliminate computational bottlenecks in both the variational and perturbative steps of the SHCI algorithm. We also describe the parallelization and the key data structures in our implementation, such as the distributed hash table. The improved SHCI algorithm enables us to include in our variational wavefunction two orders of magnitude more determinants than has been reported previously with other selected configuration interaction methods. We use our algorithm to calculate an accurate benchmark energy for the chromium dimer with the X2C relativistic Hamiltonian in the cc-pVDZ-DK basis, correlating 28 electrons in 76 spatial orbitals. Our largest calculation uses two billion Slater determinants in the variational space and semistochastically includes perturbative contributions from at least trillions of additional determinants with better than 10−5 Ha statistical uncertainty.
The recently developed semistochastic heat-bath configuration interaction (SHCI) method is a systematically improvable selected configuration interaction plus perturbation theory method capable of giving essentially exact energies for larger systems than is possible with other such methods. We compute SHCI atomization energies for 55 molecules that have been used as a test set in prior studies because their atomization energies are known from experiment. Basis sets from cc-pVDZ to cc-pV5Z are used, totaling up to 500 orbitals and a Hilbert space of 1032 Slater determinants for the largest molecules. For each basis, an extrapolated energy well within chemical accuracy (1 kcal/mol or 1.6 mHa/mol) of the exact energy for that basis is computed using only a tiny fraction of the entire Hilbert space. We also use our almost exact energies to benchmark energies from the coupled cluster method with single, double, and perturbative triple excitations. The energies are extrapolated to the complete basis set limit and compared to the experimental atomization energies. The extrapolations are done both without and with a basis-set correction based on density-functional theory. The mean absolute deviations from experiment for these extrapolations are 0.46 kcal/mol and 0.51 kcal/mol, respectively. Orbital optimization methods used to obtain improved convergence of the SHCI energies are also discussed.
In this thesis, I present my work on the fast semistochastic heatbath configuration interaction (Fast SHCI), which is an efficient algorithm for doing essentially exact electronic structure calculations within a finite basis. There are Hamiltonians for which the entire Hilbert space is enormous, but the important part of Hilbert space is of manageable size, say 101210^{12}1012. Quantum chemistry Hamiltonians, for reasonably small systems, have this property. For such Hamiltonians, selected configuration interaction plus perturbation theory (SCI+PT) methods can be useful. The most important part of the Hilbert space is treated variationally, and the resulting energy is improved by using perturbation theory. Fast SHCI is more than an order of magnitude faster than other SCI+PT algorithms, and also much faster than other essentially exact algorithms for many chemical systems. This thesis provides an in-depth description of the Fast SHCI algorithm and its implementation. I use SHCI to compute the electronic structure of several chemical systems and the homogeneous electron gas. Some of these calculations are more accurate than those achieved by other high-order quantum chemistry methods. Others treat systems larger than those that can be treated by other equally accurate methods. My implementation of SHCI uses a modular design, which not only makes the library highly extensible but also contributes several generic distributed computing building blocks to the open-source community. In this thesis, I also describe my design and implementation of these generic components. Finally, I also provide a brief discussion of the usability of general software engineering best practices for the development of medium-scale scientific software packages with lessons learned from designing, developing, and leading the development of our SHCI package. Medium-scale scientific software packages are common in scientific research where a small group of researchers works on the same code base. Due to the differences in the requirements, some best practices that are common in the industry need to be adjusted to be useful for these projects.
We introduce an algorithm for sampling many-body quantum states in Fock space. The algorithm efficiently samples states with probability approximately proportional to an arbitrary function of the second-quantized Hamiltonian matrix element connecting the sampled state to the current state. We apply the new sampling algorithm to the recently developed semistochastic full configuration interaction quantum Monte Carlo (S-FCIQMC) method, a semistochastic implementation of the power method for projecting out the ground state energy in a basis of Slater determinants. Our new sampling method requires modest additional computational time and memory compared to uniform sampling but results in newly spawned weights that are approximately of the same magnitude, thereby greatly improving the efficiency of projection. A comparison in efficiency between our sampling algorithm and uniform sampling is performed on the all-electron nitrogen dimer at equilibrium in Dunning's cc-pVXZ basis sets with X ∈ {D, T, Q, 5}, demonstrating a large gain in efficiency that increases with basis set size. In addition, a comparison in efficiency is performed on three all-electron first-row dimers, B2, N2, and F2, in a cc-pVQZ basis, demonstrating that the gain in efficiency compared to uniform sampling also increases dramatically with the number of electrons.
The authors use the recently developed semistochastic heat bath configuration interaction method to calculate the potential energy curve of the very challenging Cr2{}_{2}2​ dimer. Despite the fact that the largest Hilbert space has dimension 1042{10}^{42}1042, the paper obtains energies with estimated errors of a few mHa or less.
We present a stable and systematically improvable quantum Monte Carlo (QMC) approach to calculating excited-state energies, which we implement using our fast randomized iteration method for the full configuration interaction problem (FCI-FRI). Unlike previous excited-state quantum Monte Carlo methods, our approach, which is based on an asymmetric variant of subspace iteration, avoids the use of dot products of random vectors and instead relies upon trial vectors to maintain orthogonality and estimate eigenvalues. By leveraging recent advances, we apply our method to calculate ground- and excited-state energies of challenging molecular systems in large active spaces, including the carbon dimer with 8 electrons in 108 orbitals (8e,108o), an oxo-Mn(salen) transition metal complex (28e,28o), ozone (18e,87o), and butadiene (22e,82o). In the majority of these test cases, our approach yields total excited-state energies that agree with those from state-of-the-art methods─including heat-bath CI, the density matrix renormalization group approach, and FCIQMC─to within sub-milliHartree accuracy. In all cases, estimated excitation energies agree to within about 0.1 eV.