Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Machine Learning techniques applied to the statistical properties of spin systems

Matteo Ferraretto

Published Oct 14, 2019• 0 citations
OpenAlex

Abstract

In recent years Machine Learning has proved to be successful in many technological applications and scientific tasks, such as image and speech recognition, natural language understanding and more. In this work we apply both supervised and unsupervised Machine Learning to three key problems in statistical physics: design of Hamiltonian from data, phase recognition and study of critical properties of a system undergoing phase transition. As an example of model design, we use a dataset with spin configurations and corresponding energies randomly sampled from a one dimensional Ising model. We then make two guesses for the correct Hamiltonian: the former involving interactions of spins with an external local field, the latter involving two body interactions among spins. The coupling constants are determined with a linear penalized regression, comparing the effects of L1 and L2 penalization terms of the cost function. We pay specific attention to the problem of overfitting and to the validation process, which is critical for accepting or rejecting the proposed model. The phase recognition problem is faced with the two dimensional Ising and XY models as examples. After showing the limits of a simple softmax regression for this task, we build suitable neural networks to overcome these limits. In particular, a feed forward network is built and the learning process is investigated for the Ising model; while a more sophisticated convolutional network is proposed for the XY model in order to detect local topological structures. The last part of the work is dedicated to the unsupervised study of phase transitions, and the determination of critical properties (order parameter, critical temperature, critical exponents). The discussed techniques are Principal Component Analysis (PCA) for dimensional reduction and K-means clustering for organizing data into subsets with specific properties. PCA is applied to the two dimensional Ising, Potts and XY models, and it is used to find suitable order parameters. The study of the proposed order parameter as a function of temperature provides evidence of phase transition, and a finite size scaling allows to extrapolate both the critical temperature in the thermodynamic limit, and the $\nu$ critical exponent for the correlation length. K-means clustering is applied to equilibrium configurations of the two dimensional Ising model, before and after the dimensional reduction, and the critical temperature is estimated. Moreover, a clustering of relaxation curves of magnetization in a Monte Carlo dynamics is used to build a phase diagram on the parameter space of temperature and magnetic field. A synergy of C++, Python and Wolfram Mathematica 12.0 is used throughout this work in order to sample input datasets and to build and control customized neural networks and learning tools. The most relevant codes are provided in the appendix.

Related papers

Discovering phases, phase transitions, and crossovers through unsupervised machine learning: A critical examination

Wenjian Hu, Rajiv R. P. Singh, Richard T. Scalettar

We apply unsupervised machine learning techniques, mainly principal component analysis (PCA), to compare and contrast the phase behavior and phase transitions in several classical spin models-the square- and triangular-lattice Ising models, the Blume-Capel model, a highly degenerate biquadratic-exchange spin-1 Ising (BSI) model, and the two-dimensional XY model-and we examine critically what machine learning is teaching us. We find that quantified principal components from PCA not only allow the exploration of different phases and symmetry-breaking, but they can distinguish phase-transition types and locate critical points. We show that the corresponding weight vectors have a clear physical interpretation, which is particularly interesting in the frustrated models such as the triangular antiferromagnet, where they can point to incipient orders. Unlike the other well-studied models, the properties of the BSI model are less well known. Using both PCA and conventional Monte Carlo analysis, we demonstrate that the BSI model shows an absence of phase transition and macroscopic ground-state degeneracy. The failure to capture the "charge" correlations (vorticity) in the BSI model (XY model) from raw spin configurations points to some of the limitations of PCA. Finally, we employ a nonlinear unsupervised machine learning procedure, the "autoencoder method," and we demonstrate that it too can be trained to capture phase transitions and critical points.

arXiv:1704.00080•Physical review. E•Jun 19, 2017•333 citations

Machine learning of phase transitions in the percolation and XY models

Wanzhou Zhang, Jiayu Liu, Tzu-Chieh Wei

In this paper, we apply machine learning methods to study phase transitions in certain statistical mechanical models on the two-dimensional lattices, whose transitions involve nonlocal or topological properties, including site and bond percolations, the XY model, and the generalized XY model. We find that using just one hidden layer in a fully connected neural network, the percolation transition can be learned and the data collapse by using the average output layer gives correct estimate of the critical exponent ν. We also study the Berezinskii-Kosterlitz-Thouless transition, which involves binding and unbinding of topological defects, vortices and antivortices, in the classical XY model. The generalized XY model contains richer phases, such as the nematic phase, the paramagnetic and the quasi-long-range ferromagnetic phases, and we also apply machine learning method to it. We obtain a consistent phase diagram from the network trained with only data along the temperature axis at two particular parameter Δ values, where Δ is the relative weight of pure XY coupling. Aside from using the spin configurations (either angles or spin components) as the input information in a convolutional neural network, we devise a feature engineering approach using the histograms of the spin orientations in order to train the network to learn the three phases in the generalized XY model and demonstrate that it indeed works. The trained network by using system size L×L can be used to the phase diagram for other sizes (L^{'}×L^{'}, where L^{'}≠L) without any further training.

arXiv:1804.02709•Physical review. E•Mar 29, 2019•114 citations

Unsupervised machine learning phase classification for the Falicov-Kimball model

Lukáš Frk, Pavel Baláž, Elguja Archemashvili +1 more

We apply various unsupervised machine learning methods for phase classification to investigate the finite-temperature phase diagram of the spinless Falicov-Kimball model in two dimensions. Using only particle occupation snapshots from Monte Carlo simulations as input, each technique, including a straightforward classification based on principal component analysis (PCA), successfully identifies the phase boundary between ordered and disordered phases, independent of the type of phase transition. Remarkably, these techniques also distinguish between the weakly localized and Anderson-localized regimes within the disordered phase, accurately identifying their crossover, which is a challenging task for standard methods. Among the machine learning approaches used, PCA based analysis outperforms more complex methods, such as neural network predictors and autoencoders. These results underscore the effectiveness of simple unsupervised techniques in examining phase transitions and electron localization in complex correlated systems.

arXiv:2411.07319•Physical review. B./Physical review. B•May 13, 2025•3 citations

Unsupervised learning the phase diagram of frustrated Ising models

Danilo Rodrigues de Assis Elias, Enzo Granato, Maurice de Koning

We assess the quality of a set of machine-learning (ML) techniques for the prediction of the phase diagrams of two frustrated 2D Ising models with competing interactions, that feature phase transitions characterized by critical manifolds rather than a single critical point. Using raw Monte Carlo spin configurations generated for random system parameters, we apply principal-component analysis (PCA) and auto-encoders to achieve dimensionality reduction, followed by clustering using the DBSCAN method and a support-vector machine classifier. The resulting estimates for the critical manifolds of both systems are in excellent agreement with available exact results, with the auto-encoders leading to quantitatively superior results. Notably, both the structure of the optimized auto-encoder as well its corresponding dimensionality-reduced and clustered space correlate with fundamental physical characteristics of the systems under consideration, which is useful when \emph{a priori} theoretical insight is unavailable.

•Dec 21, 2020•0 citations

Machine learning phases in statistical physics

Qi Chen

Conventionally, the study of phases in statistical mechan- ics is performed with the help of random sampling tools. Among the most powerful are Monte Carlo simulations consisting of a stochastic importance sampling over state space and evaluation of estimators for physical quantities. The ability of modern machine learning techniques to classify, identify, or in- terpret massive data sets provides a complementary paradigm to the above approach to analyze the exponentially large number of states in statistical physics. In this report, it is demonstrated by application on Ising-type models that deep learning has potential wide applications in solving many-body statis- tical physics problems. In application of supervised learning, we showed that the feed-forward neural network can identify phases and phase transitions in the ferromagnetic Ising model and the convolutional neural network (CNN) is extremely powerful in classifying T = 0 and T = ∞ phases in the Ising gauge model; In application of unsupervised learning, we illustrated that a deep auto-encoder constructed by stacked restricted Boltzmann machines (RBM)\nis closely related to the renormalization group (RG) method well understood in modern physics and our reconstruction of Ising spin configurations in the ferromagnetic Ising model is similar to the hand-written digits reconstruction.

•Jan 1, 2017•0 citations

Can machine learning truly decode phase transitions? A deep dive into the Ising model with competing interactions

Maninder Kaur, Ying Wai Li, Dilina Perera +1 more

We investigate phase transitions in the two-dimensional Ising model with competing nearest-neighbor (NN) and next-nearest-neighbor (NNN) interactions, where the NNN coupling generates a degenerate superantiferromagnetic (SAF) ground state, by combining machine-learning methods with Monte Carlo simulations and finite-size scaling (FSS). Going beyond earlier studies, we show that unsupervised learning can recover order-parameter and Binder cumulant analogs that obey FSS. Principal component analysis (PCA) reveals the two-component structure of the SAF order parameter, and k-means clustering is used to visualize qualitative differences between ordered and disordered configurations in the principal-component space. A fully connected neural network (FCNN), trained without prior knowledge of the order parameter or the critical temperature, learns the strength of SAF stripe ordering and uses it as the internal feature for phase classification. FSS of thermodynamic observables, the PCA-derived quantity, and the FCNN output yields consistent estimates of the critical temperature and correlation-length exponent. These results demonstrate that machine learning can decode ordering patterns, extract meaningful order-parameter analogs, and reproduce critical behavior in frustrated Ising systems.

Physical review. E•May 19, 2026•0 citations
In recent years Machine Learning has proved to be successful in many technological applications and scientific tasks, such as image and speech recognition, natural language
understanding and more. In this work we apply both supervised and unsupervised Machine Learning to three key problems in statistical physics: design of Hamiltonian from data,
phase recognition and study of critical properties of a system undergoing phase transition.
As an example of model design, we use a dataset with spin configurations and corresponding energies randomly sampled from a one dimensional Ising model. We then make two
guesses for the correct Hamiltonian: the former involving interactions of spins with an external local field, the latter involving two body interactions among spins.
The coupling constants are determined with a linear penalized regression, comparing the effects of L1 and L2 penalization terms of the cost function. We pay specific attention
to the problem of overfitting and to the validation process, which is critical for accepting or rejecting the proposed model.
The phase recognition problem is faced with the two dimensional Ising and XY models as examples. After showing the limits of a simple softmax regression for this task,
we build suitable neural networks to overcome these limits. In particular, a feed forward network is built and the learning process is investigated for the Ising model;
while a more sophisticated convolutional network is proposed for the XY model in order to detect local topological structures.
The last part of the work is dedicated to the unsupervised study of phase transitions, and the determination of critical properties (order parameter, critical
temperature, critical exponents). The discussed techniques are Principal Component Analysis (PCA) for dimensional reduction and K-means clustering for organizing data into
subsets with specific properties. PCA is applied to the two dimensional Ising, Potts and XY models, and it is used to find suitable order parameters. The study of the
proposed order parameter as a function of temperature provides evidence of phase transition, and a finite size scaling allows to extrapolate both the critical temperature
in the thermodynamic limit, and the ν\nuν critical exponent for the correlation length. K-means clustering is applied to equilibrium configurations of the two dimensional
Ising model, before and after the dimensional reduction, and the critical temperature is estimated. Moreover, a clustering of relaxation curves of magnetization in a
Monte Carlo dynamics is used to build a phase diagram on the parameter space of temperature and magnetic field.
A synergy of C++, Python and Wolfram Mathematica 12.0 is used throughout this work in order to sample input datasets and to build and control customized neural networks
and learning tools. The most relevant codes are provided in the appendix.
We apply unsupervised machine learning techniques, mainly principal component analysis (PCA), to compare and contrast the phase behavior and phase transitions in several classical spin models-the square- and triangular-lattice Ising models, the Blume-Capel model, a highly degenerate biquadratic-exchange spin-1 Ising (BSI) model, and the two-dimensional XY model-and we examine critically what machine learning is teaching us. We find that quantified principal components from PCA not only allow the exploration of different phases and symmetry-breaking, but they can distinguish phase-transition types and locate critical points. We show that the corresponding weight vectors have a clear physical interpretation, which is particularly interesting in the frustrated models such as the triangular antiferromagnet, where they can point to incipient orders. Unlike the other well-studied models, the properties of the BSI model are less well known. Using both PCA and conventional Monte Carlo analysis, we demonstrate that the BSI model shows an absence of phase transition and macroscopic ground-state degeneracy. The failure to capture the "charge" correlations (vorticity) in the BSI model (XY model) from raw spin configurations points to some of the limitations of PCA. Finally, we employ a nonlinear unsupervised machine learning procedure, the "autoencoder method," and we demonstrate that it too can be trained to capture phase transitions and critical points.
In this paper, we apply machine learning methods to study phase transitions in certain statistical mechanical models on the two-dimensional lattices, whose transitions involve nonlocal or topological properties, including site and bond percolations, the XY model, and the generalized XY model. We find that using just one hidden layer in a fully connected neural network, the percolation transition can be learned and the data collapse by using the average output layer gives correct estimate of the critical exponent ν. We also study the Berezinskii-Kosterlitz-Thouless transition, which involves binding and unbinding of topological defects, vortices and antivortices, in the classical XY model. The generalized XY model contains richer phases, such as the nematic phase, the paramagnetic and the quasi-long-range ferromagnetic phases, and we also apply machine learning method to it. We obtain a consistent phase diagram from the network trained with only data along the temperature axis at two particular parameter Δ values, where Δ is the relative weight of pure XY coupling. Aside from using the spin configurations (either angles or spin components) as the input information in a convolutional neural network, we devise a feature engineering approach using the histograms of the spin orientations in order to train the network to learn the three phases in the generalized XY model and demonstrate that it indeed works. The trained network by using system size L×L can be used to the phase diagram for other sizes (L^{'}×L^{'}, where L^{'}≠L) without any further training.
We apply various unsupervised machine learning methods for phase classification to investigate the finite-temperature phase diagram of the spinless Falicov-Kimball model in two dimensions. Using only particle occupation snapshots from Monte Carlo simulations as input, each technique, including a straightforward classification based on principal component analysis (PCA), successfully identifies the phase boundary between ordered and disordered phases, independent of the type of phase transition. Remarkably, these techniques also distinguish between the weakly localized and Anderson-localized regimes within the disordered phase, accurately identifying their crossover, which is a challenging task for standard methods. Among the machine learning approaches used, PCA based analysis outperforms more complex methods, such as neural network predictors and autoencoders. These results underscore the effectiveness of simple unsupervised techniques in examining phase transitions and electron localization in complex correlated systems.
We assess the quality of a set of machine-learning (ML) techniques for the prediction of the phase diagrams of two frustrated 2D Ising models with competing interactions, that feature phase transitions characterized by critical manifolds rather than a single critical point. Using raw Monte Carlo spin configurations generated for random system parameters, we apply principal-component analysis (PCA) and auto-encoders to achieve dimensionality reduction, followed by clustering using the DBSCAN method and a support-vector machine classifier. The resulting estimates for the critical manifolds of both systems are in excellent agreement with available exact results, with the auto-encoders leading to quantitatively superior results. Notably, both the structure of the optimized auto-encoder as well its corresponding dimensionality-reduced and clustered space correlate with fundamental physical characteristics of the systems under consideration, which is useful when \emph{a priori} theoretical insight is unavailable.
Conventionally, the study of phases in statistical mechan- ics is performed with the help of random sampling tools. Among the most powerful are Monte Carlo simulations consisting of a stochastic importance sampling over state space and evaluation of estimators for physical quantities. The ability of modern machine learning techniques to classify, identify, or in- terpret massive data sets provides a complementary paradigm to the above approach to analyze the exponentially large number of states in statistical physics. In this report, it is demonstrated by application on Ising-type models that deep learning has potential wide applications in solving many-body statis- tical physics problems. In application of supervised learning, we showed that the feed-forward neural network can identify phases and phase transitions in the ferromagnetic Ising model and the convolutional neural network (CNN) is extremely powerful in classifying T = 0 and T = ∞ phases in the Ising gauge model; In application of unsupervised learning, we illustrated that a deep auto-encoder constructed by stacked restricted Boltzmann machines (RBM)\nis closely related to the renormalization group (RG) method well understood in modern physics and our reconstruction of Ising spin configurations in the ferromagnetic Ising model is similar to the hand-written digits reconstruction.
We investigate phase transitions in the two-dimensional Ising model with competing nearest-neighbor (NN) and next-nearest-neighbor (NNN) interactions, where the NNN coupling generates a degenerate superantiferromagnetic (SAF) ground state, by combining machine-learning methods with Monte Carlo simulations and finite-size scaling (FSS). Going beyond earlier studies, we show that unsupervised learning can recover order-parameter and Binder cumulant analogs that obey FSS. Principal component analysis (PCA) reveals the two-component structure of the SAF order parameter, and k-means clustering is used to visualize qualitative differences between ordered and disordered configurations in the principal-component space. A fully connected neural network (FCNN), trained without prior knowledge of the order parameter or the critical temperature, learns the strength of SAF stripe ordering and uses it as the internal feature for phase classification. FSS of thermodynamic observables, the PCA-derived quantity, and the FCNN output yields consistent estimates of the critical temperature and correlation-length exponent. These results demonstrate that machine learning can decode ordering patterns, extract meaningful order-parameter analogs, and reproduce critical behavior in frustrated Ising systems.