Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2104.02587 · Journal of Chemical Theory and Computation

Orbital Optimization in Selected Configuration Interaction Methods

Yuan Yao, Cyrus J. Umrigar

Published Jun 16, 2021• 48 citations
arXivHTMLPDF

Abstract

We study several approaches to orbital optimization in selected configuration interaction (SCI) plus perturbation theory methods and test them on the ground and excited states of three molecules using the semistochastic heat-bath configuration interaction method. We discuss the ways in which the orbital optimization problem in SCI resembles and differs from that in complete active space self-consistent field. Starting from natural orbitals, these approaches divide into three classes of optimization methods according to how they treat coupling between configuration interaction coefficients and orbital parameters, namely uncoupled, fully coupled, and quasi-fully coupled methods. We demonstrate that taking the coupling into account is crucial for fast convergence and recommend two quasi-fully coupled methods for such applications: accelerated diagonal Newton and Broyden-Fletcher-Goldfarb-Shanno.

DOI
10.1021/acs.jctc.1c00385

Related papers

Almost exact energies for the Gaussian-2 set with the semistochastic heat-bath configuration interaction method

Yuan Yao, Emmanuel Giner, Junhao Li +2 more

The recently developed semistochastic heat-bath configuration interaction (SHCI) method is a systematically improvable selected configuration interaction plus perturbation theory method capable of giving essentially exact energies for larger systems than is possible with other such methods. We compute SHCI atomization energies for 55 molecules that have been used as a test set in prior studies because their atomization energies are known from experiment. Basis sets from cc-pVDZ to cc-pV5Z are used, totaling up to 500 orbitals and a Hilbert space of 1032 Slater determinants for the largest molecules. For each basis, an extrapolated energy well within chemical accuracy (1 kcal/mol or 1.6 mHa/mol) of the exact energy for that basis is computed using only a tiny fraction of the entire Hilbert space. We also use our almost exact energies to benchmark energies from the coupled cluster method with single, double, and perturbative triple excitations. The energies are extrapolated to the complete basis set limit and compared to the experimental atomization energies. The extrapolations are done both without and with a basis-set correction based on density-functional theory. The mean absolute deviations from experiment for these extrapolations are 0.46 kcal/mol and 0.51 kcal/mol, respectively. Orbital optimization methods used to obtain improved convergence of the SHCI energies are also discussed.

arXiv:2004.10059•The Journal of Chemical Physics•Sep 28, 2020•56 citations

Selected Configuration Interaction in a Basis of Cluster State Tensor Products

Vibin Abraham, Nicholas J. Mayhall

Selected configuration interaction (SCI) methods are currently enjoying a resurgence due to several recent developments which improve either the overall computational efficiency or the compactness of the resulting SCI vector. These recent advances have made it possible to get full CI (FCI) quality results for much larger orbital active spaces compared to conventional approaches. However, due to the starting assumption that the FCI vector has only a small number of significant Slater determinants, SCI becomes intractable for systems with strong correlation. This paper introduces a method for developing SCI algorithms in a way which exploits local molecular structure to significantly reduce the number of SCI variables. The proposed method is defined by first grouping the orbitals into clusters over which we can define many-particle cluster states. We then directly perform the SCI algorithm in a basis of tensor products of cluster states instead of Slater determinants. While the approach is general for arbitrarily defined cluster states, we find significantly improved performance by defining cluster states through a Tucker decomposition of the global (and sparse) SCI vector. To demonstrate the potential of this method, called tensor product selected configuration interaction (TPSCI), we present numerical results for a diverse set of examples: (1) modified Hubbard model with different inter- and intracluster hopping terms, (2) less obviously clusterable cases of bond breaking in N 2 and F 2, and (3) ground state energies of large planar π-conjugated systems with active spaces of up to 42 electrons in 42 orbitals. These numerical results show that TPSCI can be used to significantly reduce the number of SCI variables in the variational space, thus paving a path for extending these deterministic and variational SCI approaches to a wider range of physical systems.

arXiv:2002.03107•Journal of Chemical Theory and Computation•Aug 26, 2020•65 citations

Active Space Selection Based on Natural Orbital Occupation Numbers from n-Electron Valence Perturbation Theory

Abhishek Khedkar, Michael Roemelt

Efficient and robust approximations to the full configuration interaction (full-CI) method such as the density matrix renormalization group (DMRG) and the full-CI quantum Monte Carlo (FCIQMC) algorithm allow for multiconfigurational self-consistent field (MC-SCF) calculations of molecules with many strongly correlated electrons. This opens up the possibility to treat large and complex systems that were previously untractable, but at the same time it calls for an efficient and reliable active space selection as the choice of how many electrons and orbitals enter the active space is critical for any multireference calculation. In this work we propose an Active Space Selection based on 1st order perturbation theory (ASS1ST) that follows a "bottom-up" strategy and utilizes a set of quasi-natural orbitals together with sensible thresholds for their occupation numbers. The required quasi-natural orbitals are generated by diagonalizing the virtual and internal part of the one-electron reduced density matrix that is obtained from strongly contracted n-electron valence perturbation theory (SC-NEVPT) on top of a minimal active space calculation. Self-consistent results can be obtained when the proposed selection scheme is applied iteratively. Initial applications on four chemically relevant benchmark systems indicate the capabilities of ASS1ST. Eventually, the strengths and limitations are critically discussed.

Journal of Chemical Theory and Computation•May 6, 2019•65 citations

The density matrix renormalization group self-consistent field method: Orbital optimization with the density matrix renormalization group method in the active space

Dominika Zgid, Marcel Nooijen

We present the density matrix renormalization group self-consistent field (DMRG-SCF) approach that is analogous to the complete active space self-consisted field (CASSCF) method but instead of using for the description of the active space the full configuration interaction (FCI) method, the DMRG-SCF uses the density matrix renormalization group (DMRG) method. The DMRG-SCF approach, similarly to CASSCF, properly describes the multiconfigurational character of the wave function but avoids the exponential scaling of the FCI method and replaces it with a polynomial scaling. Hence, calculations for a larger number of orbitals and electrons in the active space are possible since the DMRG method provides an efficient tool to automatically select from the full Hilbert space the many-body contracted basis states that are the most important for the description of the wave function.

The Journal of Chemical Physics•Apr 11, 2008•202 citations

Selected Coupled Cluster Guided by Selected Configuration Interaction

Minsik Cho, Troy Van Voorhis

Coupled Cluster (CC) theory is one of the most popular correlated wavefunction methods, yet the steep computational cost scaling of CC truncated at higher rank limits its application. Modern Selected Configuration Interaction (SCI) methods successfully exploit the sparsity of ground-state wavefunctions in the determinant space, motivating an analogous approach with the exponential CC ansatz. In this work, we introduce Selected Coupled Cluster (SCC), an arbitrary-order sparse CC method that explicitly solves the CC residual equations within a compact amplitude space guided by an SCI wavefunction. SCC optimizes the CC amplitudes in the sparse amplitude space. Using a pilot Just-in-Time (JIT) compiled implementation, we benchmark SCC against SCI, SCI+PT2, standard rank-truncated dense CC (CCSD, CCSDT), and DMRG across various chemical systems. We demonstrate that SCC exhibits fast, monotonic convergence with the SCI variational threshold, outperforming SCI and surpassing SCI+PT2 in the tight-threshold regime. SCC also provides accurate energies for high-dimensional topologies, where DMRG typically converges slowly with bond dimension. SCC thus presents an efficient drop-in enhancement for modern SCI workflows.

physics.chem-phCross-listedarXiv:2609.10829•Sep 11, 2026•0 citations

Stochastic Multiconfigurational Self-Consistent Field Theory

Robert E. Thomas, Qiming Sun, Ali Alavi +1 more

The multiconfigurational self-consistent field theory is considered the standard starting point for almost all multireference approaches required for strongly correlated molecular problems. The limitation of the approach is generally given by the number of strongly correlated orbitals in the molecule, since its cost will grow exponentially with this number. We present a new multiconfigurational self-consistent field approach, wherein linear determinant coefficients of a multiconfigurational wave function are optimized via the stochastic full configuration interaction quantum Monte Carlo technique at greatly reduced computational cost, with nonlinear orbital rotation parameters updated variationally based on this sampled wave function. This extends this approach to strongly correlated systems with far larger active spaces than it is possible to treat via conventional means. By comparison with this traditional approach, we demonstrate that the introduction of stochastic noise in both the determinant amplitudes and the gradient and Hessian of the orbital rotations does not preclude robust and reliable convergence of the orbital optimization. It can even improve the ability to avoid convergence to local minima in the orbital space, and therefore aid in finding variationally lower-energy solutions. We consider the effect on the convergence of the orbitals as the number of walkers and the sampling time within the active space increases, as well as the effect on the final energy and error. The scope of the new protocol is demonstrated with a study of the increasingly strongly correlated electronic structure in a series of polycyclic aromatic hydrocarbons, up to the large coronene molecule in a complete active space of 24 π electrons in 24 orbitals, requiring only modest computational resources

arXiv:1510.03635•Journal of Chemical Theory and Computation•Oct 13, 2015•103 citations
We study several approaches to orbital optimization in selected configuration interaction (SCI) plus perturbation theory methods and test them on the ground and excited states of three molecules using the semistochastic heat-bath configuration interaction method. We discuss the ways in which the orbital optimization problem in SCI resembles and differs from that in complete active space self-consistent field. Starting from natural orbitals, these approaches divide into three classes of optimization methods according to how they treat coupling between configuration interaction coefficients and orbital parameters, namely uncoupled, fully coupled, and quasi-fully coupled methods. We demonstrate that taking the coupling into account is crucial for fast convergence and recommend two quasi-fully coupled methods for such applications: accelerated diagonal Newton and Broyden-Fletcher-Goldfarb-Shanno.
The recently developed semistochastic heat-bath configuration interaction (SHCI) method is a systematically improvable selected configuration interaction plus perturbation theory method capable of giving essentially exact energies for larger systems than is possible with other such methods. We compute SHCI atomization energies for 55 molecules that have been used as a test set in prior studies because their atomization energies are known from experiment. Basis sets from cc-pVDZ to cc-pV5Z are used, totaling up to 500 orbitals and a Hilbert space of 1032 Slater determinants for the largest molecules. For each basis, an extrapolated energy well within chemical accuracy (1 kcal/mol or 1.6 mHa/mol) of the exact energy for that basis is computed using only a tiny fraction of the entire Hilbert space. We also use our almost exact energies to benchmark energies from the coupled cluster method with single, double, and perturbative triple excitations. The energies are extrapolated to the complete basis set limit and compared to the experimental atomization energies. The extrapolations are done both without and with a basis-set correction based on density-functional theory. The mean absolute deviations from experiment for these extrapolations are 0.46 kcal/mol and 0.51 kcal/mol, respectively. Orbital optimization methods used to obtain improved convergence of the SHCI energies are also discussed.
Selected configuration interaction (SCI) methods are currently enjoying a resurgence due to several recent developments which improve either the overall computational efficiency or the compactness of the resulting SCI vector. These recent advances have made it possible to get full CI (FCI) quality results for much larger orbital active spaces compared to conventional approaches. However, due to the starting assumption that the FCI vector has only a small number of significant Slater determinants, SCI becomes intractable for systems with strong correlation. This paper introduces a method for developing SCI algorithms in a way which exploits local molecular structure to significantly reduce the number of SCI variables. The proposed method is defined by first grouping the orbitals into clusters over which we can define many-particle cluster states. We then directly perform the SCI algorithm in a basis of tensor products of cluster states instead of Slater determinants. While the approach is general for arbitrarily defined cluster states, we find significantly improved performance by defining cluster states through a Tucker decomposition of the global (and sparse) SCI vector. To demonstrate the potential of this method, called tensor product selected configuration interaction (TPSCI), we present numerical results for a diverse set of examples: (1) modified Hubbard model with different inter- and intracluster hopping terms, (2) less obviously clusterable cases of bond breaking in N 2 and F 2, and (3) ground state energies of large planar π-conjugated systems with active spaces of up to 42 electrons in 42 orbitals. These numerical results show that TPSCI can be used to significantly reduce the number of SCI variables in the variational space, thus paving a path for extending these deterministic and variational SCI approaches to a wider range of physical systems.
Efficient and robust approximations to the full configuration interaction (full-CI) method such as the density matrix renormalization group (DMRG) and the full-CI quantum Monte Carlo (FCIQMC) algorithm allow for multiconfigurational self-consistent field (MC-SCF) calculations of molecules with many strongly correlated electrons. This opens up the possibility to treat large and complex systems that were previously untractable, but at the same time it calls for an efficient and reliable active space selection as the choice of how many electrons and orbitals enter the active space is critical for any multireference calculation. In this work we propose an Active Space Selection based on 1st order perturbation theory (ASS1ST) that follows a "bottom-up" strategy and utilizes a set of quasi-natural orbitals together with sensible thresholds for their occupation numbers. The required quasi-natural orbitals are generated by diagonalizing the virtual and internal part of the one-electron reduced density matrix that is obtained from strongly contracted n-electron valence perturbation theory (SC-NEVPT) on top of a minimal active space calculation. Self-consistent results can be obtained when the proposed selection scheme is applied iteratively. Initial applications on four chemically relevant benchmark systems indicate the capabilities of ASS1ST. Eventually, the strengths and limitations are critically discussed.
We present the density matrix renormalization group self-consistent field (DMRG-SCF) approach that is analogous to the complete active space self-consisted field (CASSCF) method but instead of using for the description of the active space the full configuration interaction (FCI) method, the DMRG-SCF uses the density matrix renormalization group (DMRG) method. The DMRG-SCF approach, similarly to CASSCF, properly describes the multiconfigurational character of the wave function but avoids the exponential scaling of the FCI method and replaces it with a polynomial scaling. Hence, calculations for a larger number of orbitals and electrons in the active space are possible since the DMRG method provides an efficient tool to automatically select from the full Hilbert space the many-body contracted basis states that are the most important for the description of the wave function.
Coupled Cluster (CC) theory is one of the most popular correlated wavefunction methods, yet the steep computational cost scaling of CC truncated at higher rank limits its application. Modern Selected Configuration Interaction (SCI) methods successfully exploit the sparsity of ground-state wavefunctions in the determinant space, motivating an analogous approach with the exponential CC ansatz. In this work, we introduce Selected Coupled Cluster (SCC), an arbitrary-order sparse CC method that explicitly solves the CC residual equations within a compact amplitude space guided by an SCI wavefunction. SCC optimizes the CC amplitudes in the sparse amplitude space. Using a pilot Just-in-Time (JIT) compiled implementation, we benchmark SCC against SCI, SCI+PT2, standard rank-truncated dense CC (CCSD, CCSDT), and DMRG across various chemical systems. We demonstrate that SCC exhibits fast, monotonic convergence with the SCI variational threshold, outperforming SCI and surpassing SCI+PT2 in the tight-threshold regime. SCC also provides accurate energies for high-dimensional topologies, where DMRG typically converges slowly with bond dimension. SCC thus presents an efficient drop-in enhancement for modern SCI workflows.
The multiconfigurational self-consistent field theory is considered the standard starting point for almost all multireference approaches required for strongly correlated molecular problems. The limitation of the approach is generally given by the number of strongly correlated orbitals in the molecule, since its cost will grow exponentially with this number. We present a new multiconfigurational self-consistent field approach, wherein linear determinant coefficients of a multiconfigurational wave function are optimized via the stochastic full configuration interaction quantum Monte Carlo technique at greatly reduced computational cost, with nonlinear orbital rotation parameters updated variationally based on this sampled wave function. This extends this approach to strongly correlated systems with far larger active spaces than it is possible to treat via conventional means. By comparison with this traditional approach, we demonstrate that the introduction of stochastic noise in both the determinant amplitudes and the gradient and Hessian of the orbital rotations does not preclude robust and reliable convergence of the orbital optimization. It can even improve the ability to avoid convergence to local minima in the orbital space, and therefore aid in finding variationally lower-energy solutions. We consider the effect on the convergence of the orbitals as the number of walkers and the sampling time within the active space increases, as well as the effect on the final energy and error. The scope of the new protocol is demonstrated with a study of the increasingly strongly correlated electronic structure in a series of polycyclic aromatic hydrocarbons, up to the large coronene molecule in a complete active space of 24 π electrons in 24 orbitals, requiring only modest computational resources