Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Tagging the Zeros of the Sign Problem: Data, Model and Principle Zeros of the Average Sign in Determinant Quantum Monte Carlo

Masanori Watabe, Kurado (Claude Opus 5.5)

Published Nov 11, 2026• 4 citations
Publisher

Abstract

When the average sign of a fermionic quantum Monte Carlo simulation reads zero, the zero can mean threedifferent things: the run was too short (a data zero), the chosen Hubbard-Stratonovich decoupling created it (a modelzero), or no decoupling in reach removes it (a principle zero). We carry the tagged-zero vocabulary of an earlier noteover to the sign problem and test it in three seals on the two-dimensional Hubbard model (U = 4, 4x4 periodic lattice,plus 4- and 6-site clusters where the sign is known exactly by enumerating every auxiliary field), each run once onpoints unused during development. Model zero: with identical partition functions (relative difference below 2e-11), thespin decoupling keeps above 0.998 while the charge decoupling drops the average phase to 0.0003. Data zero:with the exact sign 0.035 known, the share of 100 chains that read zero fell from 0.94 to 0.52 at the predicted pricetag n* = 1603 sweeps and to 0.00 beyond it, as the error law says. On the 4x4 lattice a sign extrapolated from beta4-7 to beta 8 ([0.041, 0.145]) was met (0.106); at beta 10 the run read a data zero, as forecast. Principle zero: acrossa family of decouplings (discrete and Gaussian, spin and charge, and a continuous mixing parameter alpha built fromthe identity (n-1)^2 + m^2 = 1, the best member is pure spin, where the sign sits on a cusp: moving alpha either waycosts sign, more steeply on the imaginary side (slopes 7.9 and 5.0 per unit alpha at beta 6), and the decay with betaremains (0.48 to 0.18 from beta 6 to 8). Pure spin is thus empirically optimal among the tested members, and the tagreads "principle, relative to this family", not a complexity-theoretic statement. The half-filling sign, often calledprotected, is protected only by symmetric decouplings: alpha = -0.5 drops it to 0.41. Of 19 sealed forecasts 17 cameout R, one U and one F (cusp slopes not scaling linearly with beta). Checks made after AI comments on a draft showtwo verdicts to be fragile: with conservative binning errors one of four Q1 comparisons falls to 1.9 standard errors, andthe F of O4 becomes U once the shared reference point is treated as correlated; the cusp also appears in theGaussian family. The work is a check on how zeros are read, not a new algorithm; it is meant as a small, reproducibleentry point to the sign problem for readers of the tagged-zero series. (1) Text: CC-BY-4.0 | Code: GNU Affero General Public License v3.0 or later (AGPL-3.0-or-later) (2) Data and code. Seal documents with SHA-256 files, result files, development logs, the interruption log and allanalysis and figure scripts (GNU Affero General Public License v3.0 or later) are inBridge_Paper_Tagging_the_Zeros_of_the_Sign_Problem_scripts_v1.0.0.zip. No external data are used. (3) AI use. Designs, code, analyses and text were produced by Kurado (Claude Opus 5.5, Anthropic) working with M. Watabe, who chose every direction. Draft comments by ChatGPT and Gemini, relayed by M. Watabe, prompted the checksin Section 7; they are AI commentary, not peer review.

DOI
10.5281/zenodo.24136868

Related papers

Sign-problem-resilient singular-value probe in determinant quantum Monte Carlo

Wen Chen, Rubem Mondaini

The sign problem limits determinant quantum Monte Carlo studies of strongly correlated fermion systems. In the spin-channel Hubbard-Stratonovich decoupling, spin correlations are exactly related to auxiliary-field correlations. This relation implies that an antiferromagnetic transition reorganizes auxiliary-field configurations and thereby changes the statistical structure of the resulting fermion matrices. We use the adjacent gap ratio of low-lying singular values of the space-time fermion matrix to probe interaction-driven transitions in two half-filled honeycomb-lattice Hubbard models. In the sign-free honeycomb Hubbard model, the statistic tracks the established transition from a Dirac semimetal to an antiferromagnetic Mott insulator. In the complex-weight Haldane-Hubbard model, the transition-sensitive feature remains visible in the phase-quenched reference ensemble and occurs near previous estimates of the transition. Moreover, phase reweighting only weakly modifies the gap ratio over the regimes investigated, despite the rapid suppression of the average phase. These results establish singular-value statistics as a sign-problem-resilient probe of interaction-driven transitions in determinant quantum Monte Carlo.

arXiv:2608.19028•Aug 19, 2026•1 citation

Tagging the Zeros of Zero-Noise Extrapolation: Protected Ones, Protected Zeros and a Blind Spot

Masanori Watabe, Anthropic) Kurado (Claude Opus 5.5

Zero-noise extrapolation (ZNE) amplifies the noise of a quantum circuit by a factor lambda and extrapolates themeasured expectation values back to lambda = 0. The value at zero is never measured; it is made by a fit model. Wetag what ZNE does to zeros and ones whose true values are known, using an exact density-matrix simulation of asix-qubit Trotterised Ising chain with depolarising noise, amplitude damping and coherent over-rotation, amplified byunitary folding (lambda = 1, 3, 5). Two runs were sealed, with hashes posted publicly before execution. Theprotected one: the parity P = prod X commutes with every gate, so its expectation decays as a pure exponentialwhose rate follows from the noise parameters; this held to 4e-15, exponential extrapolation returns 1, and polynomialfits stop at 0.53-0.86 (Richardson). The protected zero: <Z_i> = 0 is untouched by Pauli noise and moved only byamplitude damping; Richardson restores it to within 18% of the raw shift, while an exponential fit leaves at least 79%.The blind spot: folding does not amplify coherent over-rotation, so ZNE passes it through; in 229 terms of 60 freshrandom configurations the median passed fraction was 0.98. Our failure: the first run sealed that per-termexponential extrapolation beats one fit of the total energy, and it came out F because our exponential fit was definedonly for positive values and a term was negative. Repaired, and re-sealed on fresh configurations, the claim heldwhere it is defined (94% of configurations), a data-only curvature rule chose the better fit 96% of the time, and theshot budget of ZNE was predicted within 2%. Of 11 sealed claims, 9 came out R, 1 U and 1 F. The work is asimulation; it makes no claim about any device. (1) Text: CC-BY-4.0 | Code: GNU Affero General Public License v3.0 or later (AGPL-3.0-or-later) (2) Data and code. Both seal documents with their SHA-256 files, frozen results, development and post-hoc logs, andall simulation, judging and figure scripts (AGPL-3.0-or-later) are inBridge_Paper_Tagging_the_Zeros_of_Zero_Noise_Extrapolation_scripts_v1.0.0.zip. The seal and result hashesare posted at github.com/watabe-masanori/cds-polytope, issue #2. No external data are used. (3) AI use. Designs, code, analyses and text were produced by Kurado (Claude Opus 5.5, Anthropic) working with M. Watabe, who chose every direction and posted the hashes. Draft comments by ChatGPT and Gemini, relayed by M. Watabe, prompted Appendix A, the standard-term column of Table 3 and Section 5; they are AI commentary, not peerreview.

•Nov 14, 2026•1 citation

Sign Problem in Quantum Monte Carlo Simulation

Gaopei Pan, Zi Yang Meng

Sign problem in quantum Monte Carlo (QMC) simulation appears to be an extremely hard yet interesting problem. In this article, we present a pedagogical overview on the origin of the sign problem in various quantum Monte Carlo simulation techniques, ranging from the world-line and stochastic series expansion Monte Carlo for boson and spin systems to the determinant and momentum-space quantum Monte Carlo for interacting fermions. We point out the basis dependency of the sign problem and summarize the progresses to cure, ease and even make use of the sign problem over the years, such as symmetry analysis of the underlying Hamiltonian, basis optimization in writting down the partition functions and many others. Moreover, we state that although traditional lore saying that in case of sign problem, the average sign in QMC simulation approaches zero exponentially fast with the space-time volume of the configurational space, there are recent breakthroughs showing this is not always the case and based on the properties of the partition function for finite size systems, one could distinguish when the average sign has the usual exponential scaling and when it is bestowed with an algebraic scaling at the low temperature limit. Fermionic QMC simulations with such algebraic sign problems have been successfully carried out for extended Hubbard-type and quantum Moiré lattice models.

arXiv:2204.08777•Apr 19, 2022•42 citations

Reducing the Sign Problem in Auxiliary Field Monte Carlo Simulations

Gäntgen, Christoph

The numerical sign problem is a hindrance for various areas of research that rely on stochastic sampling methods. It occurs when the sign or phase of the weight function in a path integral is not positive-definite. This leads to cancellations in the Monte Carlo sampling, which can significantly slow down convergence and render the simulation infeasible. Calculations of the Hubbard model are affected by the sign problem when the chemical potential is non-zero or when the spatial lattice is not bipartite. The overall objective of this thesis is the reduction of the sign problem in the Hubbard model with methods that may be applied to other models as well. The approach is to deform the integration contour into complex space and find a transformation that efficiently improves the distribution of the complex phase. The first publication of this cumulative thesis explores the simplest case of such a transformation, which is a constant shift of the real plane in imaginary direction. This transformation showed surprising potential and had the numerical advantage of not requiring the calculation of a Jacobian determinant, as opposed to general transformations. However, it was not obvious which offset would yield the best results. We present an algorithm that numerically approaches the optimal offset iteratively. The drawback of this method is that it relies on Markov Chain Monte Carlo sampling, which comes at significant computational cost and is affected by the sign problem itself, leading to reduced reliability. Therefore, we also present two analytic approximations that can be used directly or as a starting point for the iterative algorithm. To test the method, we calculate single particle correlation functions and the charge expectation value of carbon nanostructures, such as graphene sheets and fullerenes. The second publication introduces a refinement of the iterative optimization algorithm, increasing the stability and convergence speed. This enhancement is achieved by fitting a Gaussian curve to the average phase and its first two derivatives. We again present the physical observables, now including a tetrahedron and a larger honeycomb lattice. Additionally we extract single particle energies from the correlation functions, achieving a good agreement with the exactly solvable tetrahedron. The third publication applies a group equivariant neural network as transformation. The goal was to surpass the limitations of the constant shift while keeping the model as efficient as possible. For this purpose, we encoded the physical symmetries into the model architecture. Results show the potential to reduce the sign problem further than the constant shift. A comparison with dense neural networks shows more efficient use of parameters and training data. Tests with transfer learning lead to mixed results, indicating situational applicability. Overall these developments allow for more efficient calculations of sign problem affected systems.

•Sep 29, 2025•0 citations

Geometry dependence of the sign problem in quantum Monte Carlo simulations

Vladimir Iglovikov, Ehsan Khatami, Richard T. Scalettar

The sign problem is the fundamental limitation to quantum Monte Carlo simulations of the statistical mechanics of interacting fermions. Determinant quantum Monte Carlo (DQMC) is one of the leading methods to study lattice fermions, such as the Hubbard Hamiltonian, which describe strongly correlated phenomena including magnetism, metal-insulator transitions, and possibly exotic superconductivity. Here, we provide a comprehensive dataset on the geometry dependence of the DQMC sign problem for different densities, interaction strengths, temperatures, and spatial lattice sizes. We supplement these data with several observations concerning general trends in the data, including the dependence on spatial volume and how this can be probed by examining decoupled clusters, the scaling of the sign in the vicinity of a particle-hole symmetric point, and the correlation between the total sign and the signs for the individual spin species.

arXiv:1501.02832•Physical Review B•Jul 9, 2015•111 citations

Deconvolving the components of the sign problem

Sabyasachi Tarat, Bo Xiao, Rubem Mondaini +1 more

Auxiliary field quantum Monte Carlo simulations of interacting fermions require sampling over a Hubbard-Stratonovich field $h$ introduced to decouple the interactions. The weight for a given configuration involves the products of the determinant of matrices ${M}_{\ensuremath{\sigma}}(h)$, where $\ensuremath{\sigma}$ labels the species, and hence is typically not positive definite. Indeed, the average sign $\ensuremath{\langle}\mathcal{S}\ensuremath{\rangle}$ of the determinants goes to zero exponentially with increasing spatial size and decreasing temperature for most Hamiltonians of interest. This statement, however, does not explicitly separate two possible origins for the vanishing of $\ensuremath{\langle}\mathcal{S}\ensuremath{\rangle}$. Does $\ensuremath{\langle}\mathcal{S}\ensuremath{\rangle}\ensuremath{\rightarrow}0$ because randomly chosen field configurations have $\mathrm{det}[M(h)]<0$, or does the sign problem arise because the specific subset of configurations chosen by the weighting function have a greater preponderance of negative values? In the latter case, the process of weighting the configurations with $|\mathrm{det}[M(h)]|$ might steer the simulation to a region of configuration space of $h$ where positive and negative determinants are equally likely, even though randomly chosen $h$ would preferentially have determinants with a single dominant sign. In this paper, we address the relative importance of these two mechanisms for the vanishing of $\ensuremath{\langle}\mathcal{S}\ensuremath{\rangle}$ in quantum simulations.

arXiv:2108.00553•Physical review. B./Physical review. B•Jan 4, 2022•11 citations
When the average sign of a fermionic quantum Monte Carlo simulation reads zero, the zero can mean threedifferent things: the run was too short (a data zero), the chosen Hubbard-Stratonovich decoupling created it (a modelzero), or no decoupling in reach removes it (a principle zero). We carry the tagged-zero vocabulary of an earlier noteover to the sign problem and test it in three seals on the two-dimensional Hubbard model (U = 4, 4x4 periodic lattice,plus 4- and 6-site clusters where the sign is known exactly by enumerating every auxiliary field), each run once onpoints unused during development. Model zero: with identical partition functions (relative difference below 2e-11), thespin decoupling keeps above 0.998 while the charge decoupling drops the average phase to 0.0003. Data zero:with the exact sign 0.035 known, the share of 100 chains that read zero fell from 0.94 to 0.52 at the predicted pricetag n* = 1603 sweeps and to 0.00 beyond it, as the error law says. On the 4x4 lattice a sign extrapolated from beta4-7 to beta 8 ([0.041, 0.145]) was met (0.106); at beta 10 the run read a data zero, as forecast. Principle zero: acrossa family of decouplings (discrete and Gaussian, spin and charge, and a continuous mixing parameter alpha built fromthe identity (n-1)^2 + m^2 = 1, the best member is pure spin, where the sign sits on a cusp: moving alpha either waycosts sign, more steeply on the imaginary side (slopes 7.9 and 5.0 per unit alpha at beta 6), and the decay with betaremains (0.48 to 0.18 from beta 6 to 8). Pure spin is thus empirically optimal among the tested members, and the tagreads "principle, relative to this family", not a complexity-theoretic statement. The half-filling sign, often calledprotected, is protected only by symmetric decouplings: alpha = -0.5 drops it to 0.41. Of 19 sealed forecasts 17 cameout R, one U and one F (cusp slopes not scaling linearly with beta). Checks made after AI comments on a draft showtwo verdicts to be fragile: with conservative binning errors one of four Q1 comparisons falls to 1.9 standard errors, andthe F of O4 becomes U once the shared reference point is treated as correlated; the cusp also appears in theGaussian family. The work is a check on how zeros are read, not a new algorithm; it is meant as a small, reproducibleentry point to the sign problem for readers of the tagged-zero series. (1) Text: CC-BY-4.0 | Code: GNU Affero General Public License v3.0 or later (AGPL-3.0-or-later) (2) Data and code. Seal documents with SHA-256 files, result files, development logs, the interruption log and allanalysis and figure scripts (GNU Affero General Public License v3.0 or later) are inBridge_Paper_Tagging_the_Zeros_of_the_Sign_Problem_scripts_v1.0.0.zip. No external data are used. (3) AI use. Designs, code, analyses and text were produced by Kurado (Claude Opus 5.5, Anthropic) working with M. Watabe, who chose every direction. Draft comments by ChatGPT and Gemini, relayed by M. Watabe, prompted the checksin Section 7; they are AI commentary, not peer review.
The sign problem limits determinant quantum Monte Carlo studies of strongly correlated fermion systems. In the spin-channel Hubbard-Stratonovich decoupling, spin correlations are exactly related to auxiliary-field correlations. This relation implies that an antiferromagnetic transition reorganizes auxiliary-field configurations and thereby changes the statistical structure of the resulting fermion matrices. We use the adjacent gap ratio of low-lying singular values of the space-time fermion matrix to probe interaction-driven transitions in two half-filled honeycomb-lattice Hubbard models. In the sign-free honeycomb Hubbard model, the statistic tracks the established transition from a Dirac semimetal to an antiferromagnetic Mott insulator. In the complex-weight Haldane-Hubbard model, the transition-sensitive feature remains visible in the phase-quenched reference ensemble and occurs near previous estimates of the transition. Moreover, phase reweighting only weakly modifies the gap ratio over the regimes investigated, despite the rapid suppression of the average phase. These results establish singular-value statistics as a sign-problem-resilient probe of interaction-driven transitions in determinant quantum Monte Carlo.
Zero-noise extrapolation (ZNE) amplifies the noise of a quantum circuit by a factor lambda and extrapolates themeasured expectation values back to lambda = 0. The value at zero is never measured; it is made by a fit model. Wetag what ZNE does to zeros and ones whose true values are known, using an exact density-matrix simulation of asix-qubit Trotterised Ising chain with depolarising noise, amplitude damping and coherent over-rotation, amplified byunitary folding (lambda = 1, 3, 5). Two runs were sealed, with hashes posted publicly before execution. Theprotected one: the parity P = prod X commutes with every gate, so its expectation decays as a pure exponentialwhose rate follows from the noise parameters; this held to 4e-15, exponential extrapolation returns 1, and polynomialfits stop at 0.53-0.86 (Richardson). The protected zero: <Z_i> = 0 is untouched by Pauli noise and moved only byamplitude damping; Richardson restores it to within 18% of the raw shift, while an exponential fit leaves at least 79%.The blind spot: folding does not amplify coherent over-rotation, so ZNE passes it through; in 229 terms of 60 freshrandom configurations the median passed fraction was 0.98. Our failure: the first run sealed that per-termexponential extrapolation beats one fit of the total energy, and it came out F because our exponential fit was definedonly for positive values and a term was negative. Repaired, and re-sealed on fresh configurations, the claim heldwhere it is defined (94% of configurations), a data-only curvature rule chose the better fit 96% of the time, and theshot budget of ZNE was predicted within 2%. Of 11 sealed claims, 9 came out R, 1 U and 1 F. The work is asimulation; it makes no claim about any device. (1) Text: CC-BY-4.0 | Code: GNU Affero General Public License v3.0 or later (AGPL-3.0-or-later) (2) Data and code. Both seal documents with their SHA-256 files, frozen results, development and post-hoc logs, andall simulation, judging and figure scripts (AGPL-3.0-or-later) are inBridge_Paper_Tagging_the_Zeros_of_Zero_Noise_Extrapolation_scripts_v1.0.0.zip. The seal and result hashesare posted at github.com/watabe-masanori/cds-polytope, issue #2. No external data are used. (3) AI use. Designs, code, analyses and text were produced by Kurado (Claude Opus 5.5, Anthropic) working with M. Watabe, who chose every direction and posted the hashes. Draft comments by ChatGPT and Gemini, relayed by M. Watabe, prompted Appendix A, the standard-term column of Table 3 and Section 5; they are AI commentary, not peerreview.
Sign problem in quantum Monte Carlo (QMC) simulation appears to be an extremely hard yet interesting problem. In this article, we present a pedagogical overview on the origin of the sign problem in various quantum Monte Carlo simulation techniques, ranging from the world-line and stochastic series expansion Monte Carlo for boson and spin systems to the determinant and momentum-space quantum Monte Carlo for interacting fermions. We point out the basis dependency of the sign problem and summarize the progresses to cure, ease and even make use of the sign problem over the years, such as symmetry analysis of the underlying Hamiltonian, basis optimization in writting down the partition functions and many others. Moreover, we state that although traditional lore saying that in case of sign problem, the average sign in QMC simulation approaches zero exponentially fast with the space-time volume of the configurational space, there are recent breakthroughs showing this is not always the case and based on the properties of the partition function for finite size systems, one could distinguish when the average sign has the usual exponential scaling and when it is bestowed with an algebraic scaling at the low temperature limit. Fermionic QMC simulations with such algebraic sign problems have been successfully carried out for extended Hubbard-type and quantum Moiré lattice models.
The numerical sign problem is a hindrance for various areas of research that rely on stochastic sampling methods. It occurs when the sign or phase of the weight function in a path integral is not positive-definite. This leads to cancellations in the Monte Carlo sampling, which can significantly slow down convergence and render the simulation infeasible. Calculations of the Hubbard model are affected by the sign problem when the chemical potential is non-zero or when the spatial lattice is not bipartite. The overall objective of this thesis is the reduction of the sign problem in the Hubbard model with methods that may be applied to other models as well. The approach is to deform the integration contour into complex space and find a transformation that efficiently improves the distribution of the complex phase. The first publication of this cumulative thesis explores the simplest case of such a transformation, which is a constant shift of the real plane in imaginary direction. This transformation showed surprising potential and had the numerical advantage of not requiring the calculation of a Jacobian determinant, as opposed to general transformations. However, it was not obvious which offset would yield the best results. We present an algorithm that numerically approaches the optimal offset iteratively. The drawback of this method is that it relies on Markov Chain Monte Carlo sampling, which comes at significant computational cost and is affected by the sign problem itself, leading to reduced reliability. Therefore, we also present two analytic approximations that can be used directly or as a starting point for the iterative algorithm. To test the method, we calculate single particle correlation functions and the charge expectation value of carbon nanostructures, such as graphene sheets and fullerenes. The second publication introduces a refinement of the iterative optimization algorithm, increasing the stability and convergence speed. This enhancement is achieved by fitting a Gaussian curve to the average phase and its first two derivatives. We again present the physical observables, now including a tetrahedron and a larger honeycomb lattice. Additionally we extract single particle energies from the correlation functions, achieving a good agreement with the exactly solvable tetrahedron. The third publication applies a group equivariant neural network as transformation. The goal was to surpass the limitations of the constant shift while keeping the model as efficient as possible. For this purpose, we encoded the physical symmetries into the model architecture. Results show the potential to reduce the sign problem further than the constant shift. A comparison with dense neural networks shows more efficient use of parameters and training data. Tests with transfer learning lead to mixed results, indicating situational applicability. Overall these developments allow for more efficient calculations of sign problem affected systems.
The sign problem is the fundamental limitation to quantum Monte Carlo simulations of the statistical mechanics of interacting fermions. Determinant quantum Monte Carlo (DQMC) is one of the leading methods to study lattice fermions, such as the Hubbard Hamiltonian, which describe strongly correlated phenomena including magnetism, metal-insulator transitions, and possibly exotic superconductivity. Here, we provide a comprehensive dataset on the geometry dependence of the DQMC sign problem for different densities, interaction strengths, temperatures, and spatial lattice sizes. We supplement these data with several observations concerning general trends in the data, including the dependence on spatial volume and how this can be probed by examining decoupled clusters, the scaling of the sign in the vicinity of a particle-hole symmetric point, and the correlation between the total sign and the signs for the individual spin species.
Auxiliary field quantum Monte Carlo simulations of interacting fermions require sampling over a Hubbard-Stratonovich field hhh introduced to decouple the interactions. The weight for a given configuration involves the products of the determinant of matrices ${M}_{\ensuremath{\sigma}}(h)$, where $\ensuremath{\sigma}$ labels the species, and hence is typically not positive definite. Indeed, the average sign $\ensuremath{\langle}\mathcal{S}\ensuremath{\rangle}$ of the determinants goes to zero exponentially with increasing spatial size and decreasing temperature for most Hamiltonians of interest. This statement, however, does not explicitly separate two possible origins for the vanishing of $\ensuremath{\langle}\mathcal{S}\ensuremath{\rangle}$. Does $\ensuremath{\langle}\mathcal{S}\ensuremath{\rangle}\ensuremath{\rightarrow}0$ because randomly chosen field configurations have det[M(h)]<0\mathrm{det}[M(h)]<0det[M(h)]<0, or does the sign problem arise because the specific subset of configurations chosen by the weighting function have a greater preponderance of negative values? In the latter case, the process of weighting the configurations with ∣det[M(h)]∣|\mathrm{det}[M(h)]|∣det[M(h)]∣ might steer the simulation to a region of configuration space of hhh where positive and negative determinants are equally likely, even though randomly chosen hhh would preferentially have determinants with a single dominant sign. In this paper, we address the relative importance of these two mechanisms for the vanishing of $\ensuremath{\langle}\mathcal{S}\ensuremath{\rangle}$ in quantum simulations.