Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2007.09764 · Scientific Reports

A cautionary tale for machine learning generated configurations in presence of a conserved quantity

Ahmadreza Azizi, Michel Pleimling

Published Mar 18, 2021• 15 citations
arXivHTMLPDF

Abstract

We investigate the performance of machine learning algorithms trained exclusively with configurations obtained from importance sampling Monte Carlo simulations of the two-dimensional Ising model with conserved magnetization. For supervised machine learning, we use convolutional neural networks and find that the corresponding output not only allows to locate the phase transition point with high precision, it also displays a finite-size scaling characterized by an Ising critical exponent. For unsupervised learning, restricted Boltzmann machines (RBM) are trained to generate new configurations that are then used to compute various quantities. We find that RBM generates configurations with magnetizations and energies forbidden in the original physical system. The RBM generated configurations result in energy density probability distributions with incorrect weights as well as in wrong spatial correlations. We show that shortcomings are also encountered when training RBM with configurations obtained from the non-conserved Ising model.

DOI
10.1038/s41598-021-85683-8

Related papers

Machine learning determination of dynamical parameters: The Ising model case

Guido Cossu, Luigi Del Debbio, Tommaso Giani +2 more

We train a set of restricted Boltzmann machines (RBMs) on one- and two-dimensional Ising spin configurations at various values of temperature, generated using Monte Carlo simulations. We validate the training procedure by monitoring several estimators, including measurements of the log likelihood, with the corresponding partition functions estimated using annealed importance sampling. The effects of various choices of hyperparameters on training the RBM are discussed in detail, with a generic prescription provided. Finally, we present a closed-form expression for extracting the values of couplings, for every $n$-point interaction between the visible nodes of an RBM, in a binary system such as the Ising model. We aim at using this study as the foundation for further investigations of less well-known systems.

arXiv:1810.11503•Physical review. B./Physical review. B•Aug 7, 2019•41 citations

Thermodynamics and feature extraction by machine learning

Shotaro Shiba, Dimitrios Giataganas

Machine learning methods are powerful in distinguishing different phases of matter in an automated way and provide a new perspective on the study of physical phenomena.We train a restricted Boltzmann machine (RBM) on data constructed with spin configurations sampled from the Ising Hamiltonian at different values of temperature and external magnetic field using Monte Carlo methods.From the trained machine we obtain the flow of iterative reconstruction of spin state configurations to faithfully reproduce the observables of the physical system.We find that the flow of the trained RBM approaches the spin configurations of the maximal possible specific heat which resemble the near-criticality region of the Ising model.In the special case of the vanishing magnetic field the trained RBM converges to the critical point of the renormalization group (RG) flow of the lattice model.Our results suggest an explanation of how the machine identifies the physical phase transitions, by recognizing certain properties of the configuration like the maximization of the specific heat, instead of associating directly the recognition procedure with the RG flow and its fixed points.Then from the reconstructed data we deduce the critical exponent associated with the magnetization to find satisfactory agreement with the actual physical value.We assume no prior knowledge about the criticality of the system and its Hamiltonian.

arXiv:1810.08179•Physical Review Research•Sep 15, 2020•47 citations

Thermodynamics of the Ising Model Encoded in Restricted Boltzmann Machines

Jing Gu, Kai Zhang

The restricted Boltzmann machine (RBM) is a two-layer energy-based model that uses its hidden-visible connections to learn the underlying distribution of visible units, whose interactions are often complicated by high-order correlations. Previous studies on the Ising model of small system sizes have shown that RBMs are able to accurately learn the Boltzmann distribution and reconstruct thermal quantities at temperatures away from the critical point Tc. How the RBM encodes the Boltzmann distribution and captures the phase transition are, however, not well explained. In this work, we perform RBM learning of the 2d and 3d Ising model and carefully examine how the RBM extracts useful probabilistic and physical information from Ising configurations. We find several indicators derived from the weight matrix that could characterize the Ising phase transition. We verify that the hidden encoding of a visible state tends to have an equal number of positive and negative units, whose sequence is randomly assigned during training and can be inferred by analyzing the weight matrix. We also explore the physical meaning of the visible energy and loss function (pseudo-likelihood) of the RBM and show that they could be harnessed to predict the critical point or estimate physical quantities such as entropy.

arXiv:2210.06203•Entropy•Nov 22, 2022•5 citations

Super-resolving the Ising model with convolutional neural networks

Stavros Efthymiou, Matthew J. S. Beach, Roger G. Melko

Machine learning is becoming widely used in condensed matter physics. Inspired by the concept of image super-resolution, we propose a method to increase the size of lattice spin configurations using deep convolutional neural networks. Through supervised learning on Monte Carlo (MC) generated spin configurations, we train networks that invert real-space renormalization decimations. We demonstrate that super-resolution can reproduce thermodynamic observables that agree with MC calculations for the one- and two-dimensional Ising model at various temperatures. We find that it is possible to predict thermodynamic quantities for lattice sizes larger than those used in training by extrapolating the parameters of the network. We use this method to compute the critical exponents of the 2D Ising model, finding good agreement with theory.

arXiv:1810.02372•Physical review. B./Physical review. B•Feb 6, 2019•63 citations

Machine learning phases in statistical physics

Qi Chen

Conventionally, the study of phases in statistical mechan- ics is performed with the help of random sampling tools. Among the most powerful are Monte Carlo simulations consisting of a stochastic importance sampling over state space and evaluation of estimators for physical quantities. The ability of modern machine learning techniques to classify, identify, or in- terpret massive data sets provides a complementary paradigm to the above approach to analyze the exponentially large number of states in statistical physics. In this report, it is demonstrated by application on Ising-type models that deep learning has potential wide applications in solving many-body statis- tical physics problems. In application of supervised learning, we showed that the feed-forward neural network can identify phases and phase transitions in the ferromagnetic Ising model and the convolutional neural network (CNN) is extremely powerful in classifying T = 0 and T = ∞ phases in the Ising gauge model; In application of unsupervised learning, we illustrated that a deep auto-encoder constructed by stacked restricted Boltzmann machines (RBM)\nis closely related to the renormalization group (RG) method well understood in modern physics and our reconstruction of Ising spin configurations in the ferromagnetic Ising model is similar to the hand-written digits reconstruction.

•Jan 1, 2017•0 citations

Extending machine learning classification capabilities with histogram reweighting

Dimitrios Bachtis, Gert Aarts A, Biagio Lucini A

We propose the use of Monte Carlo histogram reweighting to extrapolate predictions of machine learning methods. In our approach, we treat the output from a convolutional neural network as an observable in a statistical system, enabling its extrapolation over continuous ranges in parameter space. We demonstrate our proposal using the phase transition in the two-dimensional Ising model. By interpreting the output of the neural network as an order parameter, we explore connections with known observables in the system and investigate its scaling behavior. A finite-size scaling analysis is conducted based on quantities derived from the neural network that yields accurate estimates for the critical exponents and the critical temperature. The method improves the prospects of acquiring precision measurements from machine learning in physical systems without an order parameter and those where direct sampling in regions of parameter space might not be possible.

arXiv:2004.14341•Physical review. E•Sep 9, 2020•33 citations
We investigate the performance of machine learning algorithms trained exclusively with configurations obtained from importance sampling Monte Carlo simulations of the two-dimensional Ising model with conserved magnetization. For supervised machine learning, we use convolutional neural networks and find that the corresponding output not only allows to locate the phase transition point with high precision, it also displays a finite-size scaling characterized by an Ising critical exponent. For unsupervised learning, restricted Boltzmann machines (RBM) are trained to generate new configurations that are then used to compute various quantities. We find that RBM generates configurations with magnetizations and energies forbidden in the original physical system. The RBM generated configurations result in energy density probability distributions with incorrect weights as well as in wrong spatial correlations. We show that shortcomings are also encountered when training RBM with configurations obtained from the non-conserved Ising model.
We train a set of restricted Boltzmann machines (RBMs) on one- and two-dimensional Ising spin configurations at various values of temperature, generated using Monte Carlo simulations. We validate the training procedure by monitoring several estimators, including measurements of the log likelihood, with the corresponding partition functions estimated using annealed importance sampling. The effects of various choices of hyperparameters on training the RBM are discussed in detail, with a generic prescription provided. Finally, we present a closed-form expression for extracting the values of couplings, for every nnn-point interaction between the visible nodes of an RBM, in a binary system such as the Ising model. We aim at using this study as the foundation for further investigations of less well-known systems.
Machine learning methods are powerful in distinguishing different phases of matter in an automated way and provide a new perspective on the study of physical phenomena.We train a restricted Boltzmann machine (RBM) on data constructed with spin configurations sampled from the Ising Hamiltonian at different values of temperature and external magnetic field using Monte Carlo methods.From the trained machine we obtain the flow of iterative reconstruction of spin state configurations to faithfully reproduce the observables of the physical system.We find that the flow of the trained RBM approaches the spin configurations of the maximal possible specific heat which resemble the near-criticality region of the Ising model.In the special case of the vanishing magnetic field the trained RBM converges to the critical point of the renormalization group (RG) flow of the lattice model.Our results suggest an explanation of how the machine identifies the physical phase transitions, by recognizing certain properties of the configuration like the maximization of the specific heat, instead of associating directly the recognition procedure with the RG flow and its fixed points.Then from the reconstructed data we deduce the critical exponent associated with the magnetization to find satisfactory agreement with the actual physical value.We assume no prior knowledge about the criticality of the system and its Hamiltonian.
The restricted Boltzmann machine (RBM) is a two-layer energy-based model that uses its hidden-visible connections to learn the underlying distribution of visible units, whose interactions are often complicated by high-order correlations. Previous studies on the Ising model of small system sizes have shown that RBMs are able to accurately learn the Boltzmann distribution and reconstruct thermal quantities at temperatures away from the critical point Tc. How the RBM encodes the Boltzmann distribution and captures the phase transition are, however, not well explained. In this work, we perform RBM learning of the 2d and 3d Ising model and carefully examine how the RBM extracts useful probabilistic and physical information from Ising configurations. We find several indicators derived from the weight matrix that could characterize the Ising phase transition. We verify that the hidden encoding of a visible state tends to have an equal number of positive and negative units, whose sequence is randomly assigned during training and can be inferred by analyzing the weight matrix. We also explore the physical meaning of the visible energy and loss function (pseudo-likelihood) of the RBM and show that they could be harnessed to predict the critical point or estimate physical quantities such as entropy.
Machine learning is becoming widely used in condensed matter physics. Inspired by the concept of image super-resolution, we propose a method to increase the size of lattice spin configurations using deep convolutional neural networks. Through supervised learning on Monte Carlo (MC) generated spin configurations, we train networks that invert real-space renormalization decimations. We demonstrate that super-resolution can reproduce thermodynamic observables that agree with MC calculations for the one- and two-dimensional Ising model at various temperatures. We find that it is possible to predict thermodynamic quantities for lattice sizes larger than those used in training by extrapolating the parameters of the network. We use this method to compute the critical exponents of the 2D Ising model, finding good agreement with theory.
Conventionally, the study of phases in statistical mechan- ics is performed with the help of random sampling tools. Among the most powerful are Monte Carlo simulations consisting of a stochastic importance sampling over state space and evaluation of estimators for physical quantities. The ability of modern machine learning techniques to classify, identify, or in- terpret massive data sets provides a complementary paradigm to the above approach to analyze the exponentially large number of states in statistical physics. In this report, it is demonstrated by application on Ising-type models that deep learning has potential wide applications in solving many-body statis- tical physics problems. In application of supervised learning, we showed that the feed-forward neural network can identify phases and phase transitions in the ferromagnetic Ising model and the convolutional neural network (CNN) is extremely powerful in classifying T = 0 and T = ∞ phases in the Ising gauge model; In application of unsupervised learning, we illustrated that a deep auto-encoder constructed by stacked restricted Boltzmann machines (RBM)\nis closely related to the renormalization group (RG) method well understood in modern physics and our reconstruction of Ising spin configurations in the ferromagnetic Ising model is similar to the hand-written digits reconstruction.
We propose the use of Monte Carlo histogram reweighting to extrapolate predictions of machine learning methods. In our approach, we treat the output from a convolutional neural network as an observable in a statistical system, enabling its extrapolation over continuous ranges in parameter space. We demonstrate our proposal using the phase transition in the two-dimensional Ising model. By interpreting the output of the neural network as an order parameter, we explore connections with known observables in the system and investigate its scaling behavior. A finite-size scaling analysis is conducted based on quantities derived from the neural network that yields accurate estimates for the critical exponents and the critical temperature. The method improves the prospects of acquiring precision measurements from machine learning in physical systems without an order parameter and those where direct sampling in regions of parameter space might not be possible.