Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2610.23783

Sampling the XY model with an attention-based neural network

P. Białas, Christian Diego Morales, Tomasz Stebel, Dawid Zapolski

Published Oct 16, 2026• 0 citations
arXivHTMLPDF

Abstract

We present a transformer-based variational autoregressive network for sampling lattice systems with continuous degrees of freedom and apply it to the two-dimensional XY model near the Berezinskii-Kosterlitz-Thouless transition. The architecture is combined with physics-informed ingredients: locality-informed attention and approximate probabilities. The resulting sampler significantly outperforms existing neural samplers. We train the model on lattices up to $96\times96$, verify with a mode-dropping estimator that no parts of the configuration space are missed, and reproduce the energy, magnetization and helicity modulus obtained with the traditional Markov Chain Monte Carlo algorithm.

DOI
10.48550/arxiv.2610.23783

Related papers

Continuous-Mixture Autoregressive Networks Learning the Kosterlitz-Thouless Transition

Lingxiao Wang, Yin Jiang, Lianyi He +1 more

We develop deep autoregressive networks with multi channels to compute many-body systems with continuous spin degrees of freedom directly. As a concrete example, we demonstrate the two-dimensional XY model with the continuous-mixture networks and rediscover the Kosterlitz–Thouless (KT) phase transition on a periodic square lattice. Vortices characterizing the quasi-long range order are accurately detected by the generative model. By learning the microscopic probability distributions from the macroscopic thermal distribution, the networks are trained as an efficient physical sampler which can approximate the free energy and estimate thermodynamic observables unbiasedly with importance sampling. As a more precise evaluation, we compute the helicity modulus to determine the KT transition temperature. Although the training process becomes more time-consuming with larger lattice sizes, the training time remains unchanged around the KT transition temperature. The continuous-mixture autoregressive networks we developed thus can be potentially used to study other many-body systems with continuous degrees of freedom.

arXiv:2005.04857•Chinese Physics Letters•Nov 21, 2022•19 citations

Hierarchical autoregressive neural networks for statistical systems

P. Białas, Piotr Korcyl, Tomasz Stebel

It was recently proposed that neural networks could be used to approximate many-dimensional probability distributions that appear e.g. in lattice field theories or statistical mechanics. Subsequently they can be used as variational approximators to assess extensive properties of statistical systems, like free energy, and also as neural samplers used in Monte Carlo simulations. The practical application of this approach is unfortunately limited by its unfavourable scaling both of the numerical cost required for training, and the memory requirements with the system size. This is due to the fact that the original proposition involved a neural network of width which scaled with the total number of degrees of freedom, e.g. L 2 in case of a two dimensional L × L lattice. In this work we propose a hierarchical association of physical degrees of freedom, for instance spins, to neurons which replaces it with the scaling with the linear extent L of the system. We demonstrate our approach on the two-dimensional Ising model by simulating lattices of various sizes up to 128 × 128 spins, with time benchmarks reaching lattices of size 512 × 512 . We observe that our proposal improves the quality of neural network training, i.e. the approximated probability distribution is closer to the target that could be previously achieved. As a consequence, the variational free energy reaches a value closer to its theoretical expectation and, if applied in a Markov Chain Monte Carlo algorithm, the resulting autocorrelation time is smaller. Finally, the replacement of a single neural network by a hierarchy of smaller networks considerably reduces the memory requirements.

arXiv:2203.10989•Computer Physics Communications•Aug 18, 2022•11 citations

Continuous-mixture Autoregressive Networks for the Efficient Variational Calculation of Spin System

Lingxiao Wang, Yin Jiang, Lianyi He +1 more

We develop the deep autoregressive networks with multi-channel to compute continuous spin system directly. As a concrete example, the 2-D XY model is embedded into the continuous-mixture networks, in which the Kosterlitz-Thouless phase transition emerges on a periodic square lattice. Vortices as the quasi-Long Range Order(LRO) are accurately detected by the autoregressive neural networks. By learning the microscopic possibility distributions from macroscopic thermal distribution, the free energy is directly calculated and vortices (anti-vortices) pair emerge with temperature rise. As a more precise estimation, the helicity modulus is evaluated to anchor the transition temperature to be at $\beta_\text{c}=1.101$. Although the time-consuming of the training process inevitably increases with the lattice size, the training time remains unchanged around the transition temperature.

•May 11, 2020•0 citations

Generative neural samplers for the quantum Heisenberg chain

Johanna Vielhaben, Nils Strodthoff

Generative neural samplers offer a complementary approach to Monte Carlo methods for problems in statistical physics and quantum field theory. This paper tests the ability of generative neural samplers to estimate observables for real-world low-dimensional spin systems. It maps out how autoregressive models can sample configurations of a quantum Heisenberg chain via a classical approximation based on the Suzuki-Trotter transformation. We present results for energy, specific heat, and susceptibility for the isotropic XXX and the anisotropic XY chain are in good agreement with Monte Carlo results within the same approximation scheme.

arXiv:2012.10264•Physical review. E•Jun 7, 2021•6 citations

Deep Autoregressive Models for the Efficient Variational Simulation of Many-Body Quantum Systems

Or Sharir, Yoav Levine, Noam Wies +2 more

Artificial neural networks were recently shown to be an efficient representation of highly entangled many-body quantum states. In practical applications, neural-network states inherit numerical schemes used in variational Monte Carlo method, most notably the use of Markov-chain Monte Carlo (MCMC) sampling to estimate quantum expectations. The local stochastic sampling in MCMC caps the potential advantages of neural networks in two ways: (i) Its intrinsic computational cost sets stringent practical limits on the width and depth of the networks, and therefore limits their expressive capacity; (ii) its difficulty in generating precise and uncorrelated samples can result in estimations of observables that are very far from their true value. Inspired by the state-of-the-art generative models used in machine learning, we propose a specialized neural-network architecture that supports efficient and exact sampling, completely circumventing the need for Markov-chain sampling. We demonstrate our approach for two-dimensional interacting spin models, showcasing the ability to obtain accurate results on larger system sizes than those currently accessible to neural-network quantum states.

arXiv:1902.04057•Physical Review Letters•Jan 16, 2020•257 citations

Sampling two-dimensional spin systems with transformers

P. Białas, Piotr Korcyl, Tomasz Stebel +2 more

Autoregressive Neural Networks based on dense or convolutional layers have recently been shown to be a viable strategy for generating classical spin systems. Unlike these methods, sampling with transformers is commonly considered to be computationally inefficient. In this work, we propose a novel approach to transformer-based neural samplers in which we generate not a single spin per step but groups of spins. As an additional improvement, we construct a model of approximated probabilities, further improving the efficiency of the algorithm. Despite our approach being computationally heavier than dense networks or CNN-based approaches, we were able to sample larger systems of up to $180 \times 180$ spins in case of the Ising model. The Effective Sample Size of our sampler is $\sim 20$ times larger than that of the previous state-of-the-art neural sampler when trained for the $128 \times 128$ Ising model at critical temperature. Finally, we also test our algorithm on the 2D Edwards-Anderson model, where we train $64\times 64$ spin systems.

arXiv:2604.27738•Apr 30, 2026•0 citations
We present a transformer-based variational autoregressive network for sampling lattice systems with continuous degrees of freedom and apply it to the two-dimensional XY model near the Berezinskii-Kosterlitz-Thouless transition. The architecture is combined with physics-informed ingredients: locality-informed attention and approximate probabilities. The resulting sampler significantly outperforms existing neural samplers. We train the model on lattices up to 96×9696\times9696×96, verify with a mode-dropping estimator that no parts of the configuration space are missed, and reproduce the energy, magnetization and helicity modulus obtained with the traditional Markov Chain Monte Carlo algorithm.
We develop deep autoregressive networks with multi channels to compute many-body systems with continuous spin degrees of freedom directly. As a concrete example, we demonstrate the two-dimensional XY model with the continuous-mixture networks and rediscover the Kosterlitz–Thouless (KT) phase transition on a periodic square lattice. Vortices characterizing the quasi-long range order are accurately detected by the generative model. By learning the microscopic probability distributions from the macroscopic thermal distribution, the networks are trained as an efficient physical sampler which can approximate the free energy and estimate thermodynamic observables unbiasedly with importance sampling. As a more precise evaluation, we compute the helicity modulus to determine the KT transition temperature. Although the training process becomes more time-consuming with larger lattice sizes, the training time remains unchanged around the KT transition temperature. The continuous-mixture autoregressive networks we developed thus can be potentially used to study other many-body systems with continuous degrees of freedom.
It was recently proposed that neural networks could be used to approximate many-dimensional probability distributions that appear e.g. in lattice field theories or statistical mechanics. Subsequently they can be used as variational approximators to assess extensive properties of statistical systems, like free energy, and also as neural samplers used in Monte Carlo simulations. The practical application of this approach is unfortunately limited by its unfavourable scaling both of the numerical cost required for training, and the memory requirements with the system size. This is due to the fact that the original proposition involved a neural network of width which scaled with the total number of degrees of freedom, e.g. L 2 in case of a two dimensional L × L lattice. In this work we propose a hierarchical association of physical degrees of freedom, for instance spins, to neurons which replaces it with the scaling with the linear extent L of the system. We demonstrate our approach on the two-dimensional Ising model by simulating lattices of various sizes up to 128 × 128 spins, with time benchmarks reaching lattices of size 512 × 512 . We observe that our proposal improves the quality of neural network training, i.e. the approximated probability distribution is closer to the target that could be previously achieved. As a consequence, the variational free energy reaches a value closer to its theoretical expectation and, if applied in a Markov Chain Monte Carlo algorithm, the resulting autocorrelation time is smaller. Finally, the replacement of a single neural network by a hierarchy of smaller networks considerably reduces the memory requirements.
We develop the deep autoregressive networks with multi-channel to compute continuous spin system directly. As a concrete example, the 2-D XY model is embedded into the continuous-mixture networks, in which the Kosterlitz-Thouless phase transition emerges on a periodic square lattice. Vortices as the quasi-Long Range Order(LRO) are accurately detected by the autoregressive neural networks. By learning the microscopic possibility distributions from macroscopic thermal distribution, the free energy is directly calculated and vortices (anti-vortices) pair emerge with temperature rise. As a more precise estimation, the helicity modulus is evaluated to anchor the transition temperature to be at βc=1.101\beta_\text{c}=1.101βc​=1.101. Although the time-consuming of the training process inevitably increases with the lattice size, the training time remains unchanged around the transition temperature.
Generative neural samplers offer a complementary approach to Monte Carlo methods for problems in statistical physics and quantum field theory. This paper tests the ability of generative neural samplers to estimate observables for real-world low-dimensional spin systems. It maps out how autoregressive models can sample configurations of a quantum Heisenberg chain via a classical approximation based on the Suzuki-Trotter transformation. We present results for energy, specific heat, and susceptibility for the isotropic XXX and the anisotropic XY chain are in good agreement with Monte Carlo results within the same approximation scheme.
Artificial neural networks were recently shown to be an efficient representation of highly entangled many-body quantum states. In practical applications, neural-network states inherit numerical schemes used in variational Monte Carlo method, most notably the use of Markov-chain Monte Carlo (MCMC) sampling to estimate quantum expectations. The local stochastic sampling in MCMC caps the potential advantages of neural networks in two ways: (i) Its intrinsic computational cost sets stringent practical limits on the width and depth of the networks, and therefore limits their expressive capacity; (ii) its difficulty in generating precise and uncorrelated samples can result in estimations of observables that are very far from their true value. Inspired by the state-of-the-art generative models used in machine learning, we propose a specialized neural-network architecture that supports efficient and exact sampling, completely circumventing the need for Markov-chain sampling. We demonstrate our approach for two-dimensional interacting spin models, showcasing the ability to obtain accurate results on larger system sizes than those currently accessible to neural-network quantum states.
Autoregressive Neural Networks based on dense or convolutional layers have recently been shown to be a viable strategy for generating classical spin systems. Unlike these methods, sampling with transformers is commonly considered to be computationally inefficient. In this work, we propose a novel approach to transformer-based neural samplers in which we generate not a single spin per step but groups of spins. As an additional improvement, we construct a model of approximated probabilities, further improving the efficiency of the algorithm. Despite our approach being computationally heavier than dense networks or CNN-based approaches, we were able to sample larger systems of up to 180×180180 \times 180180×180 spins in case of the Ising model. The Effective Sample Size of our sampler is ∼20\sim 20∼20 times larger than that of the previous state-of-the-art neural sampler when trained for the 128×128128 \times 128128×128 Ising model at critical temperature. Finally, we also test our algorithm on the 2D Edwards-Anderson model, where we train 64×6464\times 6464×64 spin systems.