Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2103.07800 · Physical review. B./Physical review. B

Stochastic differential equation approach to understanding the population control bias in full configuration interaction quantum Monte Carlo

Joachim Brand, Mingrui Yang, Elke Pahl

Published Jun 29, 2022• 10 citations
arXivHTMLPDF

Abstract

We investigate a systematic statistical bias found in full configuration quantum Monte Carlo (FCIQMC) that originates from controlling a walker population with a fluctuating shift parameter. This bias can become the dominant error when the sign problem is absent, e.g., in bosonic systems. FCIQMC is a powerful statistical method for obtaining information about the ground state of a sparse and abstract matrix. We show that, when the sign problem is absent, the shift estimator has the nice property of providing an upper bound for the exact ground-state energy and all projected energy estimators, while a variational estimator is still an upper bound to the exact energy with substantially reduced bias. A scalar model of the general FCIQMC population dynamics leads to an exactly solvable It\^o stochastic differential equation. It provides further insights into the nature of the bias and gives accurate analytical predictions for delayed cross-covariance and autocovariance functions of the shift energy estimator and the walker number. The model provides a toehold on finding a cure for the population control bias. We provide evidence for nonuniversal power-law scaling of the population control bias with walker number in the Bose-Hubbard model for various estimators of the ground-state energy based on the shift or on projected energies. For the specific case of the noninteracting Bose-Hubbard Hamiltonian we obtain a full analytical prediction for the bias of the shift energy estimator.

DOI
10.1103/physrevb.105.235144

Related papers

Improved walker population control for full configuration interaction quantum Monte Carlo

Mingrui Yang, Elke Pahl, Joachim Brand

Full configuration interaction quantum Monte Carlo (FCIQMC) is a stochastic approach for finding the ground state of a quantum many-body Hamiltonian. It is based on the dynamical evolution of a walker population in Hilbert space, which samples the ground state configuration vector over many iterations. Here, we present a modification of the original protocol for walker population control of Booth et al. [J. Chem. Phys. 131, 054106 (2009)] in order to achieve equilibration at a pre-defined average walker number and to avoid walker number overshoots. The dynamics of the walker population is described by a noisy damped harmonic oscillator and controlled by two parameters responsible for damping and forcing, respectively, for which reasonable values are suggested. We further introduce a population growth witness that can be used to detect annihilation plateaus related to overcoming the FCIQMC sign problem. Features of the new population control procedure such as precise walker number control and fast equilibration are demonstrated. The standard error of the shift estimator for the ground state energy as well as the population control bias is found to be unaffected by the population control procedure or its parameters. The improved control of the walker number, and thereby memory consumption, is a desirable feature required for automating FCIQMC calculations and requires minimal modifications to the existing code.

arXiv:2008.01927•The Journal of Chemical Physics•Nov 2, 2020•20 citations

Population control bias and importance sampling in full configuration interaction quantum Monte Carlo

Khaldoon Ghanem, Niklas Liebermann, Ali Alavi

Population control is an essential component of any projector Monte Carlo algorithm. This control mechanism usually introduces a bias in the sampled quantities that is inversely proportional to the population size. In this paper, we investigate the population control bias in the full configuration interaction quantum Monte Carlo method. We identify the precise origin of this bias and quantify it in general. We show that it has different effects on different estimators and that the shift estimator is particularly susceptible. We derive a reweighting technique, similar to the one used in diffusion Monte Carlo, for correcting this bias and apply it to the shift estimator. We also show that by using importance sampling, the bias can be reduced substantially. We demonstrate the necessity and the effectiveness of applying these techniques for sign-problem-free systems where this bias is especially notable. Specifically, we show results for large one-dimensional Hubbard models and the two-dimensional Heisenberg model, where corrected FCIQMC results are comparable to the other high-accuracy results.

arXiv:2102.11016•Physical review. B./Physical review. B•Apr 19, 2021•24 citations

Emergence of Critical Phenomena in Full Configuration Interaction Quantum Monte Carlo

James J. Shepherd, Lauretta R. Schwarz, Robert E. Thomas +3 more

There has been recent literature discussion on the origin and severity of the `sign problem' in full configuration interaction quantum Monte Carlo (FCIQMC) and its `initiator' adaptation (i-FCIQMC), methods of interest and potential because they allow for exact (FCI) ground-state solutions to be obtained often at a much reduced computational cost. In this study we aim to use a simple order parameter, describing the `sign structure' of the stochastic wavefunction representation, to empirically characterise the fundamentally different collective behaviour of the walker population in both methods.

arXiv:1209.4023•Sep 18, 2012•0 citations

Full configuration interaction quantum Monte Carlo for accurate $\textit{ab initio}$ nuclear structure calculations: algorithms and calculation details

Rongzhe Hu, Furong Xu, Baishan Hu +1 more

Full configuration interaction quantum Monte Carlo (FCIQMC) is a stochastic many-body solver that has been widely applied to electronic, molecular, and condensed-matter systems. In this work we apply FCIQMC to $\textit{ab initio}$ nuclear structure calculations using interactions derived from chiral effective field theory. We describe the algorithm in detail, including imaginary-time propagation, excitation generation, estimator choices, the initiator approximation with adaptive shift correction, and reduced-density-matrix (RDM) sampling. Benchmark calculations in small model spaces, where deterministic full configuration interaction (FCI) results are available, validate the stochastic calculation of energies, radii, and RDM-based pure estimators. For large model spaces, we analyze the residual finite-walker bias through systematic walker-number convergence and infinite-walker extrapolations. We also demonstrate that FCIQMC can be extended beyond ground-state calculations by computing the low-lying spectrum of $^6$Li.

arXiv:2607.05525•Jul 6, 2026•0 citations

Full configuration interaction quantum Monte Carlo and its application to the lattice Bose polaron at quantum criticality

Matija Čufar

This thesis provides an investigation of the application of the full configuration interaction quantum Monte Carlo (FCIQMC) algorithm to ultracold atom problems. This includes working on the algorithm, understanding the sign problem, and applying it to the study of strongly-correlated lattice polaron physics. The thesis conceptually consists of two parts. In the first, we discuss the FCIQMC algorithm, which is a modern variant of projector Monte Carlo and can be used for calculating the ground states or low-lying excited states of quantum Hamiltonians. We present our implementation of the algorithm in the Julia programming language and discuss the details of the algorithm and data structures employed. Next, we turn to a treatment of the FCIQMC sign problem, which arises when the number of walkers employed by FCIQMC is insufficient to capture the sign structure of the ground state eigenvector of the Hamiltonian. We present a new scalar model that accurately reconstructs the dynamic behaviour of FCIQMC and shows us that as the number of walkers increases, the walker population switches from a sign-incoherent regime to a sign-coherent one in way that is reminiscent of a continuous phase transition. In the second part, we focus on the application of FCIQMC to the Bose-Hubbard model (BHM) with an impurity near quantum criticality. We use FCIQMC to guide and validate a diagrammatic theory based on the quantum Gutzwiller method, which describes the polaron by treating interactions between the impurity and the elementary excitations in the BHM. This theory shows that at strong interactions, the polaron reflects the state of the host system and as such, polarons could be used as quantum sensors in experiments. Lastly, we show numerical evidence that the polaron energy --- the energy shift caused by introducing an impurity to the BHM --- is scale invariant near the phase transition and that it exhibits critical phenomena not yet explained by theory. Furthermore, we show that the polaron energy is linked to the impurity-boson density-density correlation function and that it exhibits a flattening near the phase transition. This flattening indicates that in the thermodynamic limit, the length scales associated with the polaron diverge.

•Jan 1, 2026•0 citations

Sign problem in full configuration interaction quantum Monte Carlo: Linear and sublinear representation regimes for the exact wave function

James J. Shepherd, Gustavo E. Scuseria, James S. Spencer

We investigate the sign problem for full configuration interaction quantum Monte Carlo (FCIQMC), a stochastic algorithm for finding the ground-state solution of the Schr\"odinger equation with substantially reduced computational cost compared with exact diagonalization. We find $k$-space Hubbard models for which the solution is yielded with storage that grows sublinearly in the size of the many-body Hilbert space, in spite of using a wave function that is simply a linear combination of states. The FCIQMC algorithm is able to find this sublinear scaling regime without bias and with only a choice of the Hamiltonian basis. By means of a demonstration we solve for the energy of a 70-site half-filled system (with a space of ${10}^{38}$ determinants) in 250 core hours, substantially quicker than the $\ensuremath{\sim}{10}^{36}$ core hours that would be required by exact diagonalization. This is the largest space that has been sampled in an unbiased fashion. The challenge for the recently developed FCIQMC method is made clear: Expand the sublinear scaling regime while retaining exact-on-average accuracy. We comment upon the relationship between this and the scaling law previously observed in the initiator adaptation (i-FCIQMC). We argue that our results change the landscape for the development of FCIQMC and related methods.

arXiv:1407.4800•Physical Review B•Oct 27, 2014•35 citations
We investigate a systematic statistical bias found in full configuration quantum Monte Carlo (FCIQMC) that originates from controlling a walker population with a fluctuating shift parameter. This bias can become the dominant error when the sign problem is absent, e.g., in bosonic systems. FCIQMC is a powerful statistical method for obtaining information about the ground state of a sparse and abstract matrix. We show that, when the sign problem is absent, the shift estimator has the nice property of providing an upper bound for the exact ground-state energy and all projected energy estimators, while a variational estimator is still an upper bound to the exact energy with substantially reduced bias. A scalar model of the general FCIQMC population dynamics leads to an exactly solvable It^o stochastic differential equation. It provides further insights into the nature of the bias and gives accurate analytical predictions for delayed cross-covariance and autocovariance functions of the shift energy estimator and the walker number. The model provides a toehold on finding a cure for the population control bias. We provide evidence for nonuniversal power-law scaling of the population control bias with walker number in the Bose-Hubbard model for various estimators of the ground-state energy based on the shift or on projected energies. For the specific case of the noninteracting Bose-Hubbard Hamiltonian we obtain a full analytical prediction for the bias of the shift energy estimator.
Full configuration interaction quantum Monte Carlo (FCIQMC) is a stochastic approach for finding the ground state of a quantum many-body Hamiltonian. It is based on the dynamical evolution of a walker population in Hilbert space, which samples the ground state configuration vector over many iterations. Here, we present a modification of the original protocol for walker population control of Booth et al. [J. Chem. Phys. 131, 054106 (2009)] in order to achieve equilibration at a pre-defined average walker number and to avoid walker number overshoots. The dynamics of the walker population is described by a noisy damped harmonic oscillator and controlled by two parameters responsible for damping and forcing, respectively, for which reasonable values are suggested. We further introduce a population growth witness that can be used to detect annihilation plateaus related to overcoming the FCIQMC sign problem. Features of the new population control procedure such as precise walker number control and fast equilibration are demonstrated. The standard error of the shift estimator for the ground state energy as well as the population control bias is found to be unaffected by the population control procedure or its parameters. The improved control of the walker number, and thereby memory consumption, is a desirable feature required for automating FCIQMC calculations and requires minimal modifications to the existing code.
Population control is an essential component of any projector Monte Carlo algorithm. This control mechanism usually introduces a bias in the sampled quantities that is inversely proportional to the population size. In this paper, we investigate the population control bias in the full configuration interaction quantum Monte Carlo method. We identify the precise origin of this bias and quantify it in general. We show that it has different effects on different estimators and that the shift estimator is particularly susceptible. We derive a reweighting technique, similar to the one used in diffusion Monte Carlo, for correcting this bias and apply it to the shift estimator. We also show that by using importance sampling, the bias can be reduced substantially. We demonstrate the necessity and the effectiveness of applying these techniques for sign-problem-free systems where this bias is especially notable. Specifically, we show results for large one-dimensional Hubbard models and the two-dimensional Heisenberg model, where corrected FCIQMC results are comparable to the other high-accuracy results.
There has been recent literature discussion on the origin and severity of the `sign problem' in full configuration interaction quantum Monte Carlo (FCIQMC) and its `initiator' adaptation (i-FCIQMC), methods of interest and potential because they allow for exact (FCI) ground-state solutions to be obtained often at a much reduced computational cost. In this study we aim to use a simple order parameter, describing the `sign structure' of the stochastic wavefunction representation, to empirically characterise the fundamentally different collective behaviour of the walker population in both methods.
Full configuration interaction quantum Monte Carlo for accurate ab initio\textit{ab initio}ab initio nuclear structure calculations: algorithms and calculation details
Full configuration interaction quantum Monte Carlo (FCIQMC) is a stochastic many-body solver that has been widely applied to electronic, molecular, and condensed-matter systems. In this work we apply FCIQMC to ab initio\textit{ab initio}ab initio nuclear structure calculations using interactions derived from chiral effective field theory. We describe the algorithm in detail, including imaginary-time propagation, excitation generation, estimator choices, the initiator approximation with adaptive shift correction, and reduced-density-matrix (RDM) sampling. Benchmark calculations in small model spaces, where deterministic full configuration interaction (FCI) results are available, validate the stochastic calculation of energies, radii, and RDM-based pure estimators. For large model spaces, we analyze the residual finite-walker bias through systematic walker-number convergence and infinite-walker extrapolations. We also demonstrate that FCIQMC can be extended beyond ground-state calculations by computing the low-lying spectrum of 6^66Li.
This thesis provides an investigation of the application of the full configuration interaction quantum Monte Carlo (FCIQMC) algorithm to ultracold atom problems. This includes working on the algorithm, understanding the sign problem, and applying it to the study of strongly-correlated lattice polaron physics. The thesis conceptually consists of two parts. In the first, we discuss the FCIQMC algorithm, which is a modern variant of projector Monte Carlo and can be used for calculating the ground states or low-lying excited states of quantum Hamiltonians. We present our implementation of the algorithm in the Julia programming language and discuss the details of the algorithm and data structures employed. Next, we turn to a treatment of the FCIQMC sign problem, which arises when the number of walkers employed by FCIQMC is insufficient to capture the sign structure of the ground state eigenvector of the Hamiltonian. We present a new scalar model that accurately reconstructs the dynamic behaviour of FCIQMC and shows us that as the number of walkers increases, the walker population switches from a sign-incoherent regime to a sign-coherent one in way that is reminiscent of a continuous phase transition. In the second part, we focus on the application of FCIQMC to the Bose-Hubbard model (BHM) with an impurity near quantum criticality. We use FCIQMC to guide and validate a diagrammatic theory based on the quantum Gutzwiller method, which describes the polaron by treating interactions between the impurity and the elementary excitations in the BHM. This theory shows that at strong interactions, the polaron reflects the state of the host system and as such, polarons could be used as quantum sensors in experiments. Lastly, we show numerical evidence that the polaron energy --- the energy shift caused by introducing an impurity to the BHM --- is scale invariant near the phase transition and that it exhibits critical phenomena not yet explained by theory. Furthermore, we show that the polaron energy is linked to the impurity-boson density-density correlation function and that it exhibits a flattening near the phase transition. This flattening indicates that in the thermodynamic limit, the length scales associated with the polaron diverge.
We investigate the sign problem for full configuration interaction quantum Monte Carlo (FCIQMC), a stochastic algorithm for finding the ground-state solution of the Schr\"odinger equation with substantially reduced computational cost compared with exact diagonalization. We find kkk-space Hubbard models for which the solution is yielded with storage that grows sublinearly in the size of the many-body Hilbert space, in spite of using a wave function that is simply a linear combination of states. The FCIQMC algorithm is able to find this sublinear scaling regime without bias and with only a choice of the Hamiltonian basis. By means of a demonstration we solve for the energy of a 70-site half-filled system (with a space of 1038{10}^{38}1038 determinants) in 250 core hours, substantially quicker than the $\ensuremath{\sim}{10}^{36}$ core hours that would be required by exact diagonalization. This is the largest space that has been sampled in an unbiased fashion. The challenge for the recently developed FCIQMC method is made clear: Expand the sublinear scaling regime while retaining exact-on-average accuracy. We comment upon the relationship between this and the scaling law previously observed in the initiator adaptation (i-FCIQMC). We argue that our results change the landscape for the development of FCIQMC and related methods.