Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Fast Semistochastic Heat-Bath Configuration Interaction

Junhao Li

Published Jan 1, 2019• 102 citations
Publisher

Abstract

In this thesis, I present my work on the fast semistochastic heatbath configuration interaction (Fast SHCI), which is an efficient algorithm for doing essentially exact electronic structure calculations within a finite basis. There are Hamiltonians for which the entire Hilbert space is enormous, but the important part of Hilbert space is of manageable size, say $10^{12}$. Quantum chemistry Hamiltonians, for reasonably small systems, have this property. For such Hamiltonians, selected configuration interaction plus perturbation theory (SCI+PT) methods can be useful. The most important part of the Hilbert space is treated variationally, and the resulting energy is improved by using perturbation theory. Fast SHCI is more than an order of magnitude faster than other SCI+PT algorithms, and also much faster than other essentially exact algorithms for many chemical systems. This thesis provides an in-depth description of the Fast SHCI algorithm and its implementation. I use SHCI to compute the electronic structure of several chemical systems and the homogeneous electron gas. Some of these calculations are more accurate than those achieved by other high-order quantum chemistry methods. Others treat systems larger than those that can be treated by other equally accurate methods. My implementation of SHCI uses a modular design, which not only makes the library highly extensible but also contributes several generic distributed computing building blocks to the open-source community. In this thesis, I also describe my design and implementation of these generic components. Finally, I also provide a brief discussion of the usability of general software engineering best practices for the development of medium-scale scientific software packages with lessons learned from designing, developing, and leading the development of our SHCI package. Medium-scale scientific software packages are common in scientific research where a small group of researchers works on the same code base. Due to the differences in the requirements, some best practices that are common in the industry need to be adjusted to be useful for these projects.

DOI
10.7298/xnha-nn23

Related papers

Fast semistochastic heat-bath configuration interaction

Junhao Li, Matthew Otten, Adam A. Holmes +2 more

This paper presents in detail our fast semistochastic heat-bath configuration interaction (SHCI) method for solving the many-body Schrödinger equation. We identify and eliminate computational bottlenecks in both the variational and perturbative steps of the SHCI algorithm. We also describe the parallelization and the key data structures in our implementation, such as the distributed hash table. The improved SHCI algorithm enables us to include in our variational wavefunction two orders of magnitude more determinants than has been reported previously with other selected configuration interaction methods. We use our algorithm to calculate an accurate benchmark energy for the chromium dimer with the X2C relativistic Hamiltonian in the cc-pVDZ-DK basis, correlating 28 electrons in 76 spatial orbitals. Our largest calculation uses two billion Slater determinants in the variational space and semistochastically includes perturbative contributions from at least trillions of additional determinants with better than 10−5 Ha statistical uncertainty.

arXiv:1809.04600•The Journal of Chemical Physics•Dec 6, 2018•144 citations

Almost exact energies for the Gaussian-2 set with the semistochastic heat-bath configuration interaction method

Yuan Yao, Emmanuel Giner, Junhao Li +2 more

The recently developed semistochastic heat-bath configuration interaction (SHCI) method is a systematically improvable selected configuration interaction plus perturbation theory method capable of giving essentially exact energies for larger systems than is possible with other such methods. We compute SHCI atomization energies for 55 molecules that have been used as a test set in prior studies because their atomization energies are known from experiment. Basis sets from cc-pVDZ to cc-pV5Z are used, totaling up to 500 orbitals and a Hilbert space of 1032 Slater determinants for the largest molecules. For each basis, an extrapolated energy well within chemical accuracy (1 kcal/mol or 1.6 mHa/mol) of the exact energy for that basis is computed using only a tiny fraction of the entire Hilbert space. We also use our almost exact energies to benchmark energies from the coupled cluster method with single, double, and perturbative triple excitations. The energies are extrapolated to the complete basis set limit and compared to the experimental atomization energies. The extrapolations are done both without and with a basis-set correction based on density-functional theory. The mean absolute deviations from experiment for these extrapolations are 0.46 kcal/mol and 0.51 kcal/mol, respectively. Orbital optimization methods used to obtain improved convergence of the SHCI energies are also discussed.

arXiv:2004.10059•The Journal of Chemical Physics•Sep 28, 2020•56 citations

Semistochastic Heat-Bath Configuration Interaction Method: Selected Configuration Interaction with Semistochastic Perturbation Theory

Sandeep Sharma, Adam A. Holmes, Guillaume Jeanmairet +2 more

We extend the recently proposed heat-bath configuration interaction (HCI) method [Holmes, Tubman, Umrigar, J. Chem. Theory Comput. 2016, 12, 3674], by introducing a semistochastic algorithm for performing multireference Epstein–Nesbet perturbation theory, in order to completely eliminate the severe memory bottleneck of the original method. The proposed algorithm has several attractive features. First, there is no sign problem that plagues several quantum Monte Carlo methods. Second, instead of using Metropolis–Hastings sampling, we use the Alias method to directly sample determinants from the reference wave function, thus avoiding correlations between consecutive samples. Third, in addition to removing the memory bottleneck, semistochastic HCI (SHCI) is faster than the deterministic variant for many systems if a stochastic error of 0.1 mHa is acceptable. Fourth, within the SHCI algorithm one can trade memory for a modest increase in computer time. Fifth, the perturbative calculation is embarrassingly parallel. The SHCI algorithm extends the range of applicability of the original algorithm, allowing us to calculate the correlation energy of very large active spaces. We demonstrate this by performing calculations on several first row dimers including F 2 with an active space of (14e, 108o), Mn-Salen cluster with an active space of (28e, 22o), and Cr 2 dimer with up to a quadruple-ζ basis set with an active space of (12e, 190o). For these systems we were able to obtain better than 1 mHa accuracy with a wall time of merely 55 s, 37 s, and 56 min on 1, 1, and 4 nodes, respectively.

arXiv:1610.06660•Journal of Chemical Theory and Computation•Mar 6, 2017•364 citations

One programming model to rule them all: High-performance computational chemistry and applications via OpenMP offloading mode

Melisa Alkan

This dissertation is focused around enabling large-scale quantum chemistry calculations by developing novel algorithms on new heterogeneous hardware architectures. Section 2 contains a study on assessing the importance of many-body dispersion phenomena in molecular clusters and understanding the need to incorporate the phenomena into existing quantum chemistry methods to accurately capture long-range interactions. It is known that highly accurate quantum chemistry methods are computationally expensive; therefore, there is a need for new computer algorithms that will be able to vastly improve the time to solution and enable correlated quantum chemistry calculations on large molecular systems. Sections 3 and 4 focus on the development and implementation of various algorithms that enable quantum chemistry calculations on molecules of unprecedent sizes. Specifically, Section 3 describes a novel CUDA-based algorithm that enables GPU-acceleration of the main parts of the Self-Consistent Field (SCF) cycle, resulting in the fastest SCF code to date. Section 4 introduces a novel OpenMP-based implementation of the Fock build formation, which enables GPU-accelerated calculations on all existing GPU and CPU hardware. Section 5 expands upon the OpenMP work and describes a new GPU-accelerated library named LibERI for evaluating molecular integrals – often the main computational bottleneck in many quantum chemistry methods. LibERI is intended to serve as a library for fast quantum chemistry calculations that can be interfaced with any existing quantum chemistry software packages through the MolSSI Driver Interface (MDI). MDI is a standard interface between scientific computational codes. One of LibERI’s objectives is to seamlessly integrate with a Molecular Mechanics (MM) package using MDI to enable fast QM/MM calculations. This integration would allow for efficient GPU-accelerated treatment of large QM regions via LibERI. Lastly, Section 6 concludes this dissertation with a study that applies existing quantum chemistry methods to unravel the mechanism of a Pd-catalyzed carboacylation reaction involving the activation of ester C-O bonds.

•Jan 1, 2023•0 citations

Quantum Computing for Chemistry: From Quantum Computing with Molecules to Algorithmic Design

Phillip W. K. Jensen

Determining properties of molecules and materials is one of the premier applications of quantum computing. A major question in the thesis is: how might we use long- and near-term quantum computers to solve problems of chemical interest? First, we give an overview of the standard model in electronic structure theory. Electronic structure methods have been improved significantly in the last decades; however, it is still a non-trivial task to fit experimental results for complex molecular systems. Second, we introduce quantum computation and quantum algorithms as an alternative device to solve the electronic structure problem. In the remaining part of the thesis, we describe the research projects. The thesis covers two (rather different) research directions: software and hardware projects. For the software part, we contribute with two new algorithms: Phase-Estimation Interval Target Energy Readout (PHILTER) and Recursive Variational Series Estimation (RVSE), where both are designed to be run on a quantum digital computer and extract materials and molecular properties. The PHILTER algorithm is a modification of the Quantum Phase Estimation and is designed to compute excited state energies within a target energy-interval for cases where good approximations for the states in the target energy-interval are either unknown or hard to prepare. For the RVSE algorithm, we pose the question: how might we use imperfect near-term quantum computers to solve problems of practical value? In this work, we introduce a method to compute expectation values of general functions of a Hamiltonian with respect to an input state using low depth quantum circuits suitable for noisy intermediate-scale quantum computers. For the hardware part, we propose a new chemical quantum computer by using molecular electronics as a basis for quantum computing. In this work, we pose the question: can scattering of electrons in molecules produce quantum gates? We construct one-qubit gates using one-electron scattering in molecules, and two-qubit controlled-phase gates using electron-electron scattering along metallic leads. We show initial applications of the framework by illustrating one-qubit gates using the molecular electronic structure of molecular hydrogen as a baseline model.

•Nov 1, 2022•0 citations

A configuration‐based heatbath‐CI for spin‐adapted multireference electronic structure calculations with large active spaces

Mihkel Ugandi, Michael Roemelt

This work reports on a spin-pure configuration-based implementation of the heatbath configuration interaction (HCI) algorithm for selective configuration interaction. Besides the obvious advantage of being spin-pure, the presented method combines the compactness of the configurational ansatz with the known efficiency of the HCI algorithm and a variety of algorithmic and conceptual ideas to achieve a high level of performance. In particular, through pruning of the selected configurational space after HCI selection by means of a more strict criterion, a more compact wavefunction representation is obtained. Moreover, the underlying logic of the method allows us to minimize the number of redundant matrix-matrix multiplications while making use of just-in-time compilation to achieve fast diagonalization of the Hamiltonian. The critical search for 2-electron connections within the configurational space is facilitated by a tree-based representation thereof as suggested previously by Gopal et al. Usage of a prefix-based parallelization and batching during the calculation of the PT2-correction leads to a good load balancing and significantly reduced memory requirements for these critical steps of the calculation. In this way, the need for a semistochastic approach to the PT2 correction is avoided even for large configurational spaces. Finally, several test-cases are discussed to demonstrate the strengths and weaknesses of the presented method.

Journal of Computational Chemistry•Aug 17, 2023•8 citations
In this thesis, I present my work on the fast semistochastic heatbath configuration interaction (Fast SHCI), which is an efficient algorithm for doing essentially exact electronic structure calculations within a finite basis. There are Hamiltonians for which the entire Hilbert space is enormous, but the important part of Hilbert space is of manageable size, say 101210^{12}1012. Quantum chemistry Hamiltonians, for reasonably small systems, have this property. For such Hamiltonians, selected configuration interaction plus perturbation theory (SCI+PT) methods can be useful. The most important part of the Hilbert space is treated variationally, and the resulting energy is improved by using perturbation theory. Fast SHCI is more than an order of magnitude faster than other SCI+PT algorithms, and also much faster than other essentially exact algorithms for many chemical systems. This thesis provides an in-depth description of the Fast SHCI algorithm and its implementation. I use SHCI to compute the electronic structure of several chemical systems and the homogeneous electron gas. Some of these calculations are more accurate than those achieved by other high-order quantum chemistry methods. Others treat systems larger than those that can be treated by other equally accurate methods. My implementation of SHCI uses a modular design, which not only makes the library highly extensible but also contributes several generic distributed computing building blocks to the open-source community. In this thesis, I also describe my design and implementation of these generic components. Finally, I also provide a brief discussion of the usability of general software engineering best practices for the development of medium-scale scientific software packages with lessons learned from designing, developing, and leading the development of our SHCI package. Medium-scale scientific software packages are common in scientific research where a small group of researchers works on the same code base. Due to the differences in the requirements, some best practices that are common in the industry need to be adjusted to be useful for these projects.
This paper presents in detail our fast semistochastic heat-bath configuration interaction (SHCI) method for solving the many-body Schrödinger equation. We identify and eliminate computational bottlenecks in both the variational and perturbative steps of the SHCI algorithm. We also describe the parallelization and the key data structures in our implementation, such as the distributed hash table. The improved SHCI algorithm enables us to include in our variational wavefunction two orders of magnitude more determinants than has been reported previously with other selected configuration interaction methods. We use our algorithm to calculate an accurate benchmark energy for the chromium dimer with the X2C relativistic Hamiltonian in the cc-pVDZ-DK basis, correlating 28 electrons in 76 spatial orbitals. Our largest calculation uses two billion Slater determinants in the variational space and semistochastically includes perturbative contributions from at least trillions of additional determinants with better than 10−5 Ha statistical uncertainty.
The recently developed semistochastic heat-bath configuration interaction (SHCI) method is a systematically improvable selected configuration interaction plus perturbation theory method capable of giving essentially exact energies for larger systems than is possible with other such methods. We compute SHCI atomization energies for 55 molecules that have been used as a test set in prior studies because their atomization energies are known from experiment. Basis sets from cc-pVDZ to cc-pV5Z are used, totaling up to 500 orbitals and a Hilbert space of 1032 Slater determinants for the largest molecules. For each basis, an extrapolated energy well within chemical accuracy (1 kcal/mol or 1.6 mHa/mol) of the exact energy for that basis is computed using only a tiny fraction of the entire Hilbert space. We also use our almost exact energies to benchmark energies from the coupled cluster method with single, double, and perturbative triple excitations. The energies are extrapolated to the complete basis set limit and compared to the experimental atomization energies. The extrapolations are done both without and with a basis-set correction based on density-functional theory. The mean absolute deviations from experiment for these extrapolations are 0.46 kcal/mol and 0.51 kcal/mol, respectively. Orbital optimization methods used to obtain improved convergence of the SHCI energies are also discussed.
We extend the recently proposed heat-bath configuration interaction (HCI) method [Holmes, Tubman, Umrigar, J. Chem. Theory Comput. 2016, 12, 3674], by introducing a semistochastic algorithm for performing multireference Epstein–Nesbet perturbation theory, in order to completely eliminate the severe memory bottleneck of the original method. The proposed algorithm has several attractive features. First, there is no sign problem that plagues several quantum Monte Carlo methods. Second, instead of using Metropolis–Hastings sampling, we use the Alias method to directly sample determinants from the reference wave function, thus avoiding correlations between consecutive samples. Third, in addition to removing the memory bottleneck, semistochastic HCI (SHCI) is faster than the deterministic variant for many systems if a stochastic error of 0.1 mHa is acceptable. Fourth, within the SHCI algorithm one can trade memory for a modest increase in computer time. Fifth, the perturbative calculation is embarrassingly parallel. The SHCI algorithm extends the range of applicability of the original algorithm, allowing us to calculate the correlation energy of very large active spaces. We demonstrate this by performing calculations on several first row dimers including F 2 with an active space of (14e, 108o), Mn-Salen cluster with an active space of (28e, 22o), and Cr 2 dimer with up to a quadruple-ζ basis set with an active space of (12e, 190o). For these systems we were able to obtain better than 1 mHa accuracy with a wall time of merely 55 s, 37 s, and 56 min on 1, 1, and 4 nodes, respectively.
This dissertation is focused around enabling large-scale quantum chemistry calculations by developing novel algorithms on new heterogeneous hardware architectures. Section 2 contains a study on assessing the importance of many-body dispersion phenomena in molecular clusters and understanding the need to incorporate the phenomena into existing quantum chemistry methods to accurately capture long-range interactions. It is known that highly accurate quantum chemistry methods are computationally expensive; therefore, there is a need for new computer algorithms that will be able to vastly improve the time to solution and enable correlated quantum chemistry calculations on large molecular systems. Sections 3 and 4 focus on the development and implementation of various algorithms that enable quantum chemistry calculations on molecules of unprecedent sizes. Specifically, Section 3 describes a novel CUDA-based algorithm that enables GPU-acceleration of the main parts of the Self-Consistent Field (SCF) cycle, resulting in the fastest SCF code to date. Section 4 introduces a novel OpenMP-based implementation of the Fock build formation, which enables GPU-accelerated calculations on all existing GPU and CPU hardware. Section 5 expands upon the OpenMP work and describes a new GPU-accelerated library named LibERI for evaluating molecular integrals – often the main computational bottleneck in many quantum chemistry methods. LibERI is intended to serve as a library for fast quantum chemistry calculations that can be interfaced with any existing quantum chemistry software packages through the MolSSI Driver Interface (MDI). MDI is a standard interface between scientific computational codes. One of LibERI’s objectives is to seamlessly integrate with a Molecular Mechanics (MM) package using MDI to enable fast QM/MM calculations. This integration would allow for efficient GPU-accelerated treatment of large QM regions via LibERI. Lastly, Section 6 concludes this dissertation with a study that applies existing quantum chemistry methods to unravel the mechanism of a Pd-catalyzed carboacylation reaction involving the activation of ester C-O bonds.
Determining properties of molecules and materials is one of the premier applications of quantum computing. A major question in the thesis is: how might we use long- and near-term quantum computers to solve problems of chemical interest? First, we give an overview of the standard model in electronic structure theory. Electronic structure methods have been improved significantly in the last decades; however, it is still a non-trivial task to fit experimental results for complex molecular systems. Second, we introduce quantum computation and quantum algorithms as an alternative device to solve the electronic structure problem. In the remaining part of the thesis, we describe the research projects. The thesis covers two (rather different) research directions: software and hardware projects. For the software part, we contribute with two new algorithms: Phase-Estimation Interval Target Energy Readout (PHILTER) and Recursive Variational Series Estimation (RVSE), where both are designed to be run on a quantum digital computer and extract materials and molecular properties. The PHILTER algorithm is a modification of the Quantum Phase Estimation and is designed to compute excited state energies within a target energy-interval for cases where good approximations for the states in the target energy-interval are either unknown or hard to prepare. For the RVSE algorithm, we pose the question: how might we use imperfect near-term quantum computers to solve problems of practical value? In this work, we introduce a method to compute expectation values of general functions of a Hamiltonian with respect to an input state using low depth quantum circuits suitable for noisy intermediate-scale quantum computers. For the hardware part, we propose a new chemical quantum computer by using molecular electronics as a basis for quantum computing. In this work, we pose the question: can scattering of electrons in molecules produce quantum gates? We construct one-qubit gates using one-electron scattering in molecules, and two-qubit controlled-phase gates using electron-electron scattering along metallic leads. We show initial applications of the framework by illustrating one-qubit gates using the molecular electronic structure of molecular hydrogen as a baseline model.
This work reports on a spin-pure configuration-based implementation of the heatbath configuration interaction (HCI) algorithm for selective configuration interaction. Besides the obvious advantage of being spin-pure, the presented method combines the compactness of the configurational ansatz with the known efficiency of the HCI algorithm and a variety of algorithmic and conceptual ideas to achieve a high level of performance. In particular, through pruning of the selected configurational space after HCI selection by means of a more strict criterion, a more compact wavefunction representation is obtained. Moreover, the underlying logic of the method allows us to minimize the number of redundant matrix-matrix multiplications while making use of just-in-time compilation to achieve fast diagonalization of the Hamiltonian. The critical search for 2-electron connections within the configurational space is facilitated by a tree-based representation thereof as suggested previously by Gopal et al. Usage of a prefix-based parallelization and batching during the calculation of the PT2-correction leads to a good load balancing and significantly reduced memory requirements for these critical steps of the calculation. In this way, the need for a semistochastic approach to the PT2 correction is avoided even for large configurational spaces. Finally, several test-cases are discussed to demonstrate the strengths and weaknesses of the presented method.