Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2102.00080 · Journal of Chemical Theory and Computation

The Sign Problem in Density Matrix Quantum Monte Carlo

Hayley Petras, William Z. Van Benschoten, Sai Kumar Ramadugu, James J. Shepherd

Published Sep 21, 2021• 7 citations
arXivHTMLPDF

Abstract

High Resolution Image Download MS PowerPoint Slide Density matrix quantum Monte Carlo (DMQMC) is a recently developed method for stochastically sampling the N -particle thermal density matrix to obtain exact-on-average energies for model and ab initio systems. We report a systematic numerical study of the sign problem in DMQMC based on simulations of atomic and molecular systems. In DMQMC, the density matrix is written in an outer product basis of Slater determinants. In principle, this means that DMQMC needs to sample a space that scales in the system size, N, as O[(exp(N)) 2 ]. In practice, removing the sign problem requires a total walker population that exceeds a system-dependent critical walker population ( N c ), imposing limitations on both storage and compute time. We establish that N c for DMQMC is the square of N c for FCIQMC. By contrast, the minimum N c in the interaction picture modification of DMQMC (IP-DMQMC) is only linearly related to the N c for FCIQMC. We find that this difference originates from the difference in propagation of IP-DMQMC versus canonical DMQMC: the former is asymmetric, whereas the latter is symmetric. When an asymmetric mode of propagation is used in DMQMC, there is a much greater stochastic error and is thus prohibitively expensive for DMQMC without the interaction picture adaptation. Finally, we find that the equivalence between IP-DMQMC and FCIQMC seems to extend to the initiator approximation, which is often required to study larger systems with large basis sets. This suggests that IP-DMQMC offers a way to ameliorate the cost of moving between a Slater determinant space and an outer product basis.

DOI
10.1021/acs.jctc.1c00078

Related papers

Piecewise interaction picture density matrix quantum Monte Carlo

William Z. Van Benschoten, James J. Shepherd

The density matrix quantum Monte Carlo (DMQMC) set of methods stochastically samples the exact N-body density matrix for interacting electrons at finite temperature. We introduce a simple modification to the interaction picture DMQMC (IP-DMQMC) method that overcomes the limitation of only sampling one inverse temperature point at a time, instead allowing for the sampling of a temperature range within a single calculation, thereby reducing the computational cost. At the target inverse temperature, instead of ending the simulation, we incorporate a change of picture away from the interaction picture. The resulting equations of motion have piecewise functions and use the interaction picture in the first phase of a simulation, followed by the application of the Bloch equation once the target inverse temperature is reached. We find that the performance of this method is similar to or better than the DMQMC and IP-DMQMC algorithms in a variety of molecular test systems.

arXiv:2108.06252•The Journal of Chemical Physics•Apr 22, 2022•2 citations

Using Density Matrix Quantum Monte Carlo for Calculating Exact-on-Average Energies for ab Initio Hamiltonians in a Finite Basis Set

Hayley Petras, Sai Kumar Ramadugu, Fionn D. Malone +1 more

We here apply the recently developed initiator density matrix quantum Monte Carlo (i-DMQMC) to a variety of atoms and molecules in vacuum. i-DMQMC samples the exact density matrix of a Hamiltonian at finite temperature and combines the accuracy of full configuration interaction quantum Monte Carlo (FCIQMC)—full configuration interaction (FCI) or exact energies in a finite basis set—with finite temperature. In order to explore the applicability of i-DMQMC for molecular systems, we choose to study a recently developed test set by Rubenstein and co-workers: Be, H 2 O, and H 10 at near-equilibrium and stretched geometries. We find that, for Be and H 2 O, i-DMQMC delivers energies with submillihartree accuracy when compared with finite temperature FCI. For H 2 O and both geometries of H 10, we examine the difference between FT-AFQMC and i-DMQMC, which, in turn, estimates the difference in canonical versus grand canonical energies. We close with two discussions: one of simulation settings (initiator error, the interaction picture, and different basis sets), and another of energy difference calculations in the form of specific heat capacity and ionization potential calculations.

Journal of Chemical Theory and Computation•Jan 16, 2020•14 citations

Interaction picture density matrix quantum Monte Carlo

Fionn D. Malone, Nick S. Blunt, James J. Shepherd +3 more

The recently developed density matrix quantum Monte Carlo (DMQMC) algorithm stochastically samples the N-body thermal density matrix and hence provides access to exact properties of many-particle quantum systems at arbitrary temperatures. We demonstrate that moving to the interaction picture provides substantial benefits when applying DMQMC to interacting fermions. In this first study, we focus on a system of much recent interest: the uniform electron gas in the warm dense regime. The basis set incompleteness error at finite temperature is investigated and extrapolated via a simple Monte Carlo sampling procedure. Finally, we provide benchmark calculations for a four-electron system, comparing our results to previous work where possible.

arXiv:1506.03057•The Journal of Chemical Physics•Jul 28, 2015•90 citations

Using density matrix quantum Monte Carlo for calculating exact-on-average energies for ab-initio Hamiltonians in a finite basis set

Hayley Petras, Sai Kumar Ramadugu, Fionn D. Malone +1 more

We here apply the recently developed initiator density matrix quantum Monte Carlo (i-DMQMC) to a wide range of chemical environments using atoms and molecules in vacuum. i-DMQMC samples the exact density matrix of a Hamiltonian at finite temperature and combines the accuracy of full configuration interaction quantum Monte Carlo (FCIQMC) - full configuration interaction (FCI) or exact energies in a finite basis set - with finite temperature. By way of exploring the applicability of i-DMQMC for molecular systems, we choose to study a recently developed test set by Rubenstein and coworkers: Be, H2O, and H10 at near-equilibrium and stretched geometries. We find that, for Be and H2O, i-DMQMC delivers energies which are sub-millihartree accuracy when compared with finite temperature FCI. For H2O and both geometries of H10 we examine the difference between FT-AFQMC and i-DMQMC which in turn is an estimate of the difference in canonical versus grand canonical energies. We close with a discussion of simulation parameters (initiator error and different basis sets) and by showing energy difference calculations in the form of specific heat capacity and ionization potential calculations.

arXiv:1912.00889•Dec 2, 2019•1 citation

Sign problem in full configuration interaction quantum Monte Carlo: Linear and sublinear representation regimes for the exact wave function

James J. Shepherd, Gustavo E. Scuseria, James S. Spencer

We investigate the sign problem for full configuration interaction quantum Monte Carlo (FCIQMC), a stochastic algorithm for finding the ground-state solution of the Schr\"odinger equation with substantially reduced computational cost compared with exact diagonalization. We find $k$-space Hubbard models for which the solution is yielded with storage that grows sublinearly in the size of the many-body Hilbert space, in spite of using a wave function that is simply a linear combination of states. The FCIQMC algorithm is able to find this sublinear scaling regime without bias and with only a choice of the Hamiltonian basis. By means of a demonstration we solve for the energy of a 70-site half-filled system (with a space of ${10}^{38}$ determinants) in 250 core hours, substantially quicker than the $\ensuremath{\sim}{10}^{36}$ core hours that would be required by exact diagonalization. This is the largest space that has been sampled in an unbiased fashion. The challenge for the recently developed FCIQMC method is made clear: Expand the sublinear scaling regime while retaining exact-on-average accuracy. We comment upon the relationship between this and the scaling law previously observed in the initiator adaptation (i-FCIQMC). We argue that our results change the landscape for the development of FCIQMC and related methods.

arXiv:1407.4800•Physical Review B•Oct 27, 2014•35 citations

The effect of quantization on the full configuration interaction quantum Monte Carlo sign problem

Michael Kolodrubetz, James S. Spencer, Bryan K. Clark +1 more

The sign problem in full configuration interaction quantum Monte Carlo (FCIQMC) without annihilation can be understood as an instability of the psi-particle population to the ground state of the matrix obtained by making all off-diagonal elements of the Hamiltonian negative. Such a matrix, and hence the sign problem, is basis dependent. In this paper, we discuss the properties of a physically important basis choice: first versus second quantization. For a given choice of single-particle orbitals, we identify the conditions under which the fermion sign problem in the second quantized basis of antisymmetric Slater determinants is identical to the sign problem in the first quantized basis of unsymmetrized Hartree products. We also show that, when the two differ, the fermion sign problem is always less severe in the second quantized basis. This supports the idea that FCIQMC, even in the absence of annihilation, improves the sign problem relative to first quantized methods. Finally, we point out some theoretically interesting classes of Hamiltonians where first and second quantized sign problems differ, and others where they do not.

arXiv:1209.3044•The Journal of Chemical Physics•Jan 11, 2013•30 citations
High Resolution Image Download MS PowerPoint Slide Density matrix quantum Monte Carlo (DMQMC) is a recently developed method for stochastically sampling the N -particle thermal density matrix to obtain exact-on-average energies for model and ab initio systems. We report a systematic numerical study of the sign problem in DMQMC based on simulations of atomic and molecular systems. In DMQMC, the density matrix is written in an outer product basis of Slater determinants. In principle, this means that DMQMC needs to sample a space that scales in the system size, N, as O[(exp(N)) 2 ]. In practice, removing the sign problem requires a total walker population that exceeds a system-dependent critical walker population ( N c ), imposing limitations on both storage and compute time. We establish that N c for DMQMC is the square of N c for FCIQMC. By contrast, the minimum N c in the interaction picture modification of DMQMC (IP-DMQMC) is only linearly related to the N c for FCIQMC. We find that this difference originates from the difference in propagation of IP-DMQMC versus canonical DMQMC: the former is asymmetric, whereas the latter is symmetric. When an asymmetric mode of propagation is used in DMQMC, there is a much greater stochastic error and is thus prohibitively expensive for DMQMC without the interaction picture adaptation. Finally, we find that the equivalence between IP-DMQMC and FCIQMC seems to extend to the initiator approximation, which is often required to study larger systems with large basis sets. This suggests that IP-DMQMC offers a way to ameliorate the cost of moving between a Slater determinant space and an outer product basis.
The density matrix quantum Monte Carlo (DMQMC) set of methods stochastically samples the exact N-body density matrix for interacting electrons at finite temperature. We introduce a simple modification to the interaction picture DMQMC (IP-DMQMC) method that overcomes the limitation of only sampling one inverse temperature point at a time, instead allowing for the sampling of a temperature range within a single calculation, thereby reducing the computational cost. At the target inverse temperature, instead of ending the simulation, we incorporate a change of picture away from the interaction picture. The resulting equations of motion have piecewise functions and use the interaction picture in the first phase of a simulation, followed by the application of the Bloch equation once the target inverse temperature is reached. We find that the performance of this method is similar to or better than the DMQMC and IP-DMQMC algorithms in a variety of molecular test systems.
We here apply the recently developed initiator density matrix quantum Monte Carlo (i-DMQMC) to a variety of atoms and molecules in vacuum. i-DMQMC samples the exact density matrix of a Hamiltonian at finite temperature and combines the accuracy of full configuration interaction quantum Monte Carlo (FCIQMC)—full configuration interaction (FCI) or exact energies in a finite basis set—with finite temperature. In order to explore the applicability of i-DMQMC for molecular systems, we choose to study a recently developed test set by Rubenstein and co-workers: Be, H 2 O, and H 10 at near-equilibrium and stretched geometries. We find that, for Be and H 2 O, i-DMQMC delivers energies with submillihartree accuracy when compared with finite temperature FCI. For H 2 O and both geometries of H 10, we examine the difference between FT-AFQMC and i-DMQMC, which, in turn, estimates the difference in canonical versus grand canonical energies. We close with two discussions: one of simulation settings (initiator error, the interaction picture, and different basis sets), and another of energy difference calculations in the form of specific heat capacity and ionization potential calculations.
The recently developed density matrix quantum Monte Carlo (DMQMC) algorithm stochastically samples the N-body thermal density matrix and hence provides access to exact properties of many-particle quantum systems at arbitrary temperatures. We demonstrate that moving to the interaction picture provides substantial benefits when applying DMQMC to interacting fermions. In this first study, we focus on a system of much recent interest: the uniform electron gas in the warm dense regime. The basis set incompleteness error at finite temperature is investigated and extrapolated via a simple Monte Carlo sampling procedure. Finally, we provide benchmark calculations for a four-electron system, comparing our results to previous work where possible.
We here apply the recently developed initiator density matrix quantum Monte Carlo (i-DMQMC) to a wide range of chemical environments using atoms and molecules in vacuum. i-DMQMC samples the exact density matrix of a Hamiltonian at finite temperature and combines the accuracy of full configuration interaction quantum Monte Carlo (FCIQMC) - full configuration interaction (FCI) or exact energies in a finite basis set - with finite temperature. By way of exploring the applicability of i-DMQMC for molecular systems, we choose to study a recently developed test set by Rubenstein and coworkers: Be, H2O, and H10 at near-equilibrium and stretched geometries. We find that, for Be and H2O, i-DMQMC delivers energies which are sub-millihartree accuracy when compared with finite temperature FCI. For H2O and both geometries of H10 we examine the difference between FT-AFQMC and i-DMQMC which in turn is an estimate of the difference in canonical versus grand canonical energies. We close with a discussion of simulation parameters (initiator error and different basis sets) and by showing energy difference calculations in the form of specific heat capacity and ionization potential calculations.
We investigate the sign problem for full configuration interaction quantum Monte Carlo (FCIQMC), a stochastic algorithm for finding the ground-state solution of the Schr\"odinger equation with substantially reduced computational cost compared with exact diagonalization. We find kkk-space Hubbard models for which the solution is yielded with storage that grows sublinearly in the size of the many-body Hilbert space, in spite of using a wave function that is simply a linear combination of states. The FCIQMC algorithm is able to find this sublinear scaling regime without bias and with only a choice of the Hamiltonian basis. By means of a demonstration we solve for the energy of a 70-site half-filled system (with a space of 1038{10}^{38}1038 determinants) in 250 core hours, substantially quicker than the $\ensuremath{\sim}{10}^{36}$ core hours that would be required by exact diagonalization. This is the largest space that has been sampled in an unbiased fashion. The challenge for the recently developed FCIQMC method is made clear: Expand the sublinear scaling regime while retaining exact-on-average accuracy. We comment upon the relationship between this and the scaling law previously observed in the initiator adaptation (i-FCIQMC). We argue that our results change the landscape for the development of FCIQMC and related methods.
The sign problem in full configuration interaction quantum Monte Carlo (FCIQMC) without annihilation can be understood as an instability of the psi-particle population to the ground state of the matrix obtained by making all off-diagonal elements of the Hamiltonian negative. Such a matrix, and hence the sign problem, is basis dependent. In this paper, we discuss the properties of a physically important basis choice: first versus second quantization. For a given choice of single-particle orbitals, we identify the conditions under which the fermion sign problem in the second quantized basis of antisymmetric Slater determinants is identical to the sign problem in the first quantized basis of unsymmetrized Hartree products. We also show that, when the two differ, the fermion sign problem is always less severe in the second quantized basis. This supports the idea that FCIQMC, even in the absence of annihilation, improves the sign problem relative to first quantized methods. Finally, we point out some theoretically interesting classes of Hamiltonians where first and second quantized sign problems differ, and others where they do not.