The identification of universal properties from minimally processed data sets is one goal of machine learning techniques applied to statistical physics.Here, we study how the minimum number of variables needed to accurately describe the important features of a data set-the intrinsic dimension (I d )-behaves in the vicinity of phase transitions.We employ state-of-the-art nearest-neighbors-based I d estimators to compute the I d of raw Monte Carlo thermal configurations across different phase transitions: first-order, second-order, and Berezinskii-Kosterlitz-Thouless.For all the considered cases, we find that the I d uniquely characterizes the transition regime.The finite-size analysis of the I d allows us to not only identify critical points with an accuracy comparable to methods that rely on a priori identification of order parameters but also to determine the corresponding (critical) exponent ν in the case of continuous transitions.For the case of topological transitions, this analysis overcomes the reported limitations affecting other unsupervised learning methods.Our work reveals how raw data sets display unique signatures of universal behavior in the absence of any dimensional reduction scheme and suggest direct parallelism between conventional order parameters in real space and the intrinsic dimension in the data space.
The identification of universal properties from minimally processed data sets is one goal of machine learning techniques applied to statistical physics.Here, we study how the minimum number of variables needed to accurately describe the important features of a data set-the intrinsic dimension (I d )-behaves in the vicinity of phase transitions.We employ state-of-the-art nearest-neighbors-based I d estimators to compute the I d of raw Monte Carlo thermal configurations across different phase transitions: first-order, second-order, and Berezinskii-Kosterlitz-Thouless.For all the considered cases, we find that the I d uniquely characterizes the transition regime.The finite-size analysis of the I d allows us to not only identify critical points with an accuracy comparable to methods that rely on a priori identification of order parameters but also to determine the corresponding (critical) exponent ν in the case of continuous transitions.For the case of topological transitions, this analysis overcomes the reported limitations affecting other unsupervised learning methods.Our work reveals how raw data sets display unique signatures of universal behavior in the absence of any dimensional reduction scheme and suggest direct parallelism between conventional order parameters in real space and the intrinsic dimension in the data space.
Quantum many-body systems are characterized by patterns of correlations defining highly nontrivial manifolds when interpreted as data structures.Physical properties of phases and phase transitions are typically retrieved via correlation functions, that are related to observable response functions.Recent experiments have demonstrated capabilities to fully characterize quantum many-body systems via wavefunction snapshots, opening new possibilities to analyze quantum phenomena.Here, we introduce a method to data mine the correlation structure of quantum partition functions via their path integral (or equivalently, stochastic series expansion) manifold.We characterize path-integral manifolds generated via state-of-the-art quantum Monte Carlo methods utilizing the intrinsic dimension (ID) and the variance of distances between nearest-neighbor (NN) configurations: the former is related to data-set complexity, while the latter is able to diagnose connectivity features of points in configuration space.We show how these properties feature universal patterns in the vicinity of quantum criticality, that reveal how data structures simplify systematically at quantum phase transitions.This is further reflected by the fact that both ID and variance of NN distances exhibit universal scaling behavior in the vicinity of second-order and Berezinskii-Kosterlitz-Thouless critical points.Finally, we show how non-Abelian symmetries dramatically influence quantum data sets, due to the nature of (noncommuting) conserved charges in the quantum case.Complementary to neural-network representations, our approach represents a first elementary step towards a systematic characterization of path-integral manifolds before any dimensional reduction is taken, that is informative about universal behavior and complexity, and can find immediate application to both experiments and Monte Carlo simulations.
We assess the quality of a set of machine-learning (ML) techniques for the prediction of the phase diagrams of two frustrated 2D Ising models with competing interactions, that feature phase transitions characterized by critical manifolds rather than a single critical point. Using raw Monte Carlo spin configurations generated for random system parameters, we apply principal-component analysis (PCA) and auto-encoders to achieve dimensionality reduction, followed by clustering using the DBSCAN method and a support-vector machine classifier. The resulting estimates for the critical manifolds of both systems are in excellent agreement with available exact results, with the auto-encoders leading to quantitatively superior results. Notably, both the structure of the optimized auto-encoder as well its corresponding dimensionality-reduced and clustered space correlate with fundamental physical characteristics of the systems under consideration, which is useful when \emph{a priori} theoretical insight is unavailable.
We provide and critically analyze a framework to learn critical behavior in classical partition functions through the application of non-parametric methods to data sets of thermal configurations. We illustrate our approach in phase transitions in 2D and 3D Ising models. First, we extend previous studies on the intrinsic dimension of 2D partition function data sets, by exploring the effect of volume in 3D Ising data. We find that as opposed to 2D systems for which this quantity has been successfully used in unsupervised characterizations of critical phenomena, in the 3D case its estimation is far more challenging. To circumvent this limitation, we then use the principal component analysis (PCA) entropy, a “Shannon entropy” of the normalized spectrum of the covariance matrix. We find a striking qualitative similarity to the thermodynamic entropy, which the PCA entropy approaches asymptotically. The latter allows us to extract-through a conventional finite-size scaling analysis with modest lattice sizes-the critical temperature with less than 1% error for both 2D and 3D models while being computationally efficient. The PCA entropy can readily be applied to characterize correlations and critical phenomena in a huge variety of many-body problems and suggests a (direct) link between easy-to-compute quantities and entropies.
We investigate phase transitions in the two-dimensional Ising model with competing nearest-neighbor (NN) and next-nearest-neighbor (NNN) interactions, where the NNN coupling generates a degenerate superantiferromagnetic (SAF) ground state, by combining machine-learning methods with Monte Carlo simulations and finite-size scaling (FSS). Going beyond earlier studies, we show that unsupervised learning can recover order-parameter and Binder cumulant analogs that obey FSS. Principal component analysis (PCA) reveals the two-component structure of the SAF order parameter, and k-means clustering is used to visualize qualitative differences between ordered and disordered configurations in the principal-component space. A fully connected neural network (FCNN), trained without prior knowledge of the order parameter or the critical temperature, learns the strength of SAF stripe ordering and uses it as the internal feature for phase classification. FSS of thermodynamic observables, the PCA-derived quantity, and the FCNN output yields consistent estimates of the critical temperature and correlation-length exponent. These results demonstrate that machine learning can decode ordering patterns, extract meaningful order-parameter analogs, and reproduce critical behavior in frustrated Ising systems.
In recent years Machine Learning has proved to be successful in many technological applications and scientific tasks, such as image and speech recognition, natural language
understanding and more. In this work we apply both supervised and unsupervised Machine Learning to three key problems in statistical physics: design of Hamiltonian from data,
phase recognition and study of critical properties of a system undergoing phase transition.
As an example of model design, we use a dataset with spin configurations and corresponding energies randomly sampled from a one dimensional Ising model. We then make two
guesses for the correct Hamiltonian: the former involving interactions of spins with an external local field, the latter involving two body interactions among spins.
The coupling constants are determined with a linear penalized regression, comparing the effects of L1 and L2 penalization terms of the cost function. We pay specific attention
to the problem of overfitting and to the validation process, which is critical for accepting or rejecting the proposed model.
The phase recognition problem is faced with the two dimensional Ising and XY models as examples. After showing the limits of a simple softmax regression for this task,
we build suitable neural networks to overcome these limits. In particular, a feed forward network is built and the learning process is investigated for the Ising model;
while a more sophisticated convolutional network is proposed for the XY model in order to detect local topological structures.
The last part of the work is dedicated to the unsupervised study of phase transitions, and the determination of critical properties (order parameter, critical
temperature, critical exponents). The discussed techniques are Principal Component Analysis (PCA) for dimensional reduction and K-means clustering for organizing data into
subsets with specific properties. PCA is applied to the two dimensional Ising, Potts and XY models, and it is used to find suitable order parameters. The study of the
proposed order parameter as a function of temperature provides evidence of phase transition, and a finite size scaling allows to extrapolate both the critical temperature
in the thermodynamic limit, and the ν critical exponent for the correlation length. K-means clustering is applied to equilibrium configurations of the two dimensional
Ising model, before and after the dimensional reduction, and the critical temperature is estimated. Moreover, a clustering of relaxation curves of magnetization in a
Monte Carlo dynamics is used to build a phase diagram on the parameter space of temperature and magnetic field.
A synergy of C++, Python and Wolfram Mathematica 12.0 is used throughout this work in order to sample input datasets and to build and control customized neural networks
and learning tools. The most relevant codes are provided in the appendix.
This paper systematically reviews a series of works of our research group in the new paradigm of phase transition research, namely “statistical geometry of configuration space”. This method identifies phase transitions from the raw data in an unsupervised manner and extracts universal critical behaviors by analyzing the distance distribution between microscopic configurations sampled by Monte Carlo. We demonstrate how this paradigm has evolved from a numerical exploration of complex models to a universal empirical scaling law, and, finally, to an analytical theoretical framework that connects geometric quantities in configuration space with correlation functions in real space. This work reveals how macroscopic criticality is encoded in the geometry of microscopic configurations, providing a brand-new, “white-box” and analytically tractable data-driven perspective for studying complex phase transition problems with unknown order parameters.