The recently proposed full configuration interaction quantum Monte Carlo method allows access to essentially exact ground-state energies of systems of interacting fermions substantially larger than previously tractable without knowledge of the nodal structure of the ground-state wave function. We investigate the nature of the sign problem in this method and how its severity depends on the system studied. We explain how cancellation of the positive and negative particles sampling the wave function ensures convergence to a stochastic representation of the many-fermion ground state and accounts for the characteristic population dynamics observed in simulations.
The recently proposed full configuration interaction quantum Monte Carlo method allows access to essentially exact ground-state energies of systems of interacting fermions substantially larger than previously tractable without knowledge of the nodal structure of the ground-state wave function. We investigate the nature of the sign problem in this method and how its severity depends on the system studied. We explain how cancellation of the positive and negative particles sampling the wave function ensures convergence to a stochastic representation of the many-fermion ground state and accounts for the characteristic population dynamics observed in simulations.
There has been recent literature discussion on the origin and severity of the `sign problem' in full configuration interaction quantum Monte Carlo (FCIQMC) and its `initiator' adaptation (i-FCIQMC), methods of interest and potential because they allow for exact (FCI) ground-state solutions to be obtained often at a much reduced computational cost. In this study we aim to use a simple order parameter, describing the `sign structure' of the stochastic wavefunction representation, to empirically characterise the fundamentally different collective behaviour of the walker population in both methods.
We investigate the sign problem for full configuration interaction quantum Monte Carlo (FCIQMC), a stochastic algorithm for finding the ground-state solution of the Schr\"odinger equation with substantially reduced computational cost compared with exact diagonalization. We find k-space Hubbard models for which the solution is yielded with storage that grows sublinearly in the size of the many-body Hilbert space, in spite of using a wave function that is simply a linear combination of states. The FCIQMC algorithm is able to find this sublinear scaling regime without bias and with only a choice of the Hamiltonian basis. By means of a demonstration we solve for the energy of a 70-site half-filled system (with a space of 1038 determinants) in 250 core hours, substantially quicker than the $\ensuremath{\sim}{10}^{36}$ core hours that would be required by exact diagonalization. This is the largest space that has been sampled in an unbiased fashion. The challenge for the recently developed FCIQMC method is made clear: Expand the sublinear scaling regime while retaining exact-on-average accuracy. We comment upon the relationship between this and the scaling law previously observed in the initiator adaptation (i-FCIQMC). We argue that our results change the landscape for the development of FCIQMC and related methods.
The sign problem in full configuration interaction quantum Monte Carlo (FCIQMC) without annihilation can be understood as an instability of the psi-particle population to the ground state of the matrix obtained by making all off-diagonal elements of the Hamiltonian negative. Such a matrix, and hence the sign problem, is basis dependent. In this paper, we discuss the properties of a physically important basis choice: first versus second quantization. For a given choice of single-particle orbitals, we identify the conditions under which the fermion sign problem in the second quantized basis of antisymmetric Slater determinants is identical to the sign problem in the first quantized basis of unsymmetrized Hartree products. We also show that, when the two differ, the fermion sign problem is always less severe in the second quantized basis. This supports the idea that FCIQMC, even in the absence of annihilation, improves the sign problem relative to first quantized methods. Finally, we point out some theoretically interesting classes of Hamiltonians where first and second quantized sign problems differ, and others where they do not.
We propose a sampling method to calculate the ground state of interacting quantum systems. This approximate method, which we call the adaptive sampling quantum Monte Carlo (ASQMC) method, utilizes information from the short projection time density matrix derived from the Monte Carlo steps. The ASQMC method circumvents the negative-sign problem by introducing a conditional probability distribution function and guiding walkers to avoid a set of canceling fields. With the ASQMC method, the negative-sign ratio is greatly reduced. The ratio becomes zero in the limit $\ensuremath{\Delta}\ensuremath{\tau}$ goes to zero without prior knowledge of the exact ground state wave function. Comparisons with numerical results obtained by using other methods are made, and we observe that the ASQMC method gives accurate results with less than 0.5 error for the energy over wide regions of physical parameter values.
In this chapter, we discuss the fixed-node and constrained-path Monte Carlo methods for computing the ground state properties of systems of interacting electrons. These methods are arguably the two most powerful ones presently available for doing such calculations, but they are approximate. By sacrificing exactness, they avoid the exponential scaling of the Monte Carlo errors with system size that typically accompanies the simulation of systems of interacting electrons. This exponential scaling is called the Fermion sign problem . After a few general comments about the sign problem, we outline both methods, noting points of similarity and difference, plus points of strength and weakness. We also discuss the constrained-phase method, an extension of the constrained-path method, which controls the phase problem that develops when the ground state wave function cannot be real. Sign problem The “sign problem” refers to the exponential increase of the Monte Carlo errors with increasing system size or decreasing temperature (e.g., Loh et al., 1990, 2005; Gubernatis and Zhang, 1994) that often accompanies a Markov chain simulation whose limiting distribution is not everywhere positive. Such a case generally arises in simulations of Fermion and frustrated quantum-spin systems. It seems so inherent to Monte Carlo simulations of Fermion systems that the phrase “the sign problem” to many seems almost synonymous with the phrase “the Fermion sign problem.” Explanations for the cause of the sign problem vary and are still debated. In this chapter, we choose to summarize two explanations that seem to connect the causes in ground state Fermion simulations in the continuum and on the lattice. The sign problem, of course, is not limited to ground state calculations or even to Fermion simulations. While the cause we discuss in Sections 11.2 and 11.3 focuses on the low-lying states of diffusion-like operators, several topological pictures have been proposed (Muramatsu et al., 1992; Samson, 1993; Gubernatis and Zhang, 1994; Samson, 1995). Some of these discussions are done in the context of the zero- and finite-temperature determinant methods (Muramatsu et al., 1992; Gubernatis and Zhang, 1994). Others are done more analytically from a Feynman path-integral point of view (Samson, 1993, 1995). Some are for particles with statistics other than Fermions. The presentation of the sign problem in this chapter is appropriate for the Monte Carlo methods discussed in this chapter.
Sign problem in quantum Monte Carlo (QMC) simulation appears to be an extremely hard yet interesting problem. In this article, we present a pedagogical overview on the origin of the sign problem in various quantum Monte Carlo simulation techniques, ranging from the world-line and stochastic series expansion Monte Carlo for boson and spin systems to the determinant and momentum-space quantum Monte Carlo for interacting fermions. We point out the basis dependency of the sign problem and summarize the progresses to cure, ease and even make use of the sign problem over the years, such as symmetry analysis of the underlying Hamiltonian, basis optimization in writting down the partition functions and many others. Moreover, we state that although traditional lore saying that in case of sign problem, the average sign in QMC simulation approaches zero exponentially fast with the space-time volume of the configurational space, there are recent breakthroughs showing this is not always the case and based on the properties of the partition function for finite size systems, one could distinguish when the average sign has the usual exponential scaling and when it is bestowed with an algebraic scaling at the low temperature limit. Fermionic QMC simulations with such algebraic sign problems have been successfully carried out for extended Hubbard-type and quantum Moiré lattice models.