We investigate how unsupervised machine-learning methods can characterize the active-absorbing phase transition in the contact process in one and two spatial dimensions. Our analysis focuses on principal component analysis (PCA) and variational autoencoders (VAEs), and we show that direct applications of these methods encounter a central difficulty: contact process configurations are intrinsically positive-definite. In the PCA case, standard centering procedures fail to isolate the order parameter in the leading component. To address this, we augment the data with sign-reversed configurations, creating a balanced dataset with zero mean while preserving the density information required to describe the transition. This preprocessing allows the dominant principal component to recover the order parameter and reproduce the expected finite-size scaling near criticality. For VAEs, we find a related limitation: a fixed Gaussian prior does not provide sufficient latent regularization across the full range of control parameters and system sizes for positive-definite data. We overcome this by adopting a heteroscedastic Gaussian prior that adapts to the control parameter, leading to a substantial improvement in the finite-size scaling of reconstruction-based observables near the critical threshold. Taken together, these results highlight a key lesson: the success of machine learning in phase-transition problems does not rely on algorithmic complexity alone, but on tailoring the representation to respect the physical structure of the system, including its intrinsic constraints and symmetries.
We investigate how unsupervised machine-learning methods can characterize the active-absorbing phase transition in the contact process in one and two spatial dimensions. Our analysis focuses on principal component analysis (PCA) and variational autoencoders (VAEs), and we show that direct applications of these methods encounter a central difficulty: contact process configurations are intrinsically positive-definite. In the PCA case, standard centering procedures fail to isolate the order parameter in the leading component. To address this, we augment the data with sign-reversed configurations, creating a balanced dataset with zero mean while preserving the density information required to describe the transition. This preprocessing allows the dominant principal component to recover the order parameter and reproduce the expected finite-size scaling near criticality. For VAEs, we find a related limitation: a fixed Gaussian prior does not provide sufficient latent regularization across the full range of control parameters and system sizes for positive-definite data. We overcome this by adopting a heteroscedastic Gaussian prior that adapts to the control parameter, leading to a substantial improvement in the finite-size scaling of reconstruction-based observables near the critical threshold. Taken together, these results highlight a key lesson: the success of machine learning in phase-transition problems does not rely on algorithmic complexity alone, but on tailoring the representation to respect the physical structure of the system, including its intrinsic constraints and symmetries.
Abstract We present an unsupervised machine learning framework for discovering phase transitions and order parameters in the two-dimensional Ising model without prior knowledge of the underlying physics. Using convolutional beta-variational autoencoders (β-VAE), we learn compressed representations that naturally separate ordered and disordered phases. Our physics-informed training procedures incorporate data augmentation respecting Ising symmetries, cosine annealing optimization, and progressive temperature curriculum learning. Experimental validation on Monte Carlo simulated configurations demonstrates automatic order parameter discovery with correlation ρ = 0.85 ± 0.04 to theoretical magnetization (95% CI [0.82, 0.88]) and critical temperature detection within 0.27% of the Onsager solution (p < 0.001). The method achieves 89% improvement over principal component analysis and 124% improvement over t-distributed stochastic neighbor embedding. Latent representations exhibit clear phase separation with physics consistency score S = 0.88 ± 0.03. Ablation studies reveal optimal performance at β = 1.0, balancing reconstruction fidelity with disentanglement. This framework enables automated identification of emergent phenomena in complex many-body systems, advancing machine learning applications in condensed matter physics.
We investigate phase transitions in the two-dimensional Ising model with competing nearest-neighbor (NN) and next-nearest-neighbor (NNN) interactions, where the NNN coupling generates a degenerate superantiferromagnetic (SAF) ground state, by combining machine-learning methods with Monte Carlo simulations and finite-size scaling (FSS). Going beyond earlier studies, we show that unsupervised learning can recover order-parameter and Binder cumulant analogs that obey FSS. Principal component analysis (PCA) reveals the two-component structure of the SAF order parameter, and k-means clustering is used to visualize qualitative differences between ordered and disordered configurations in the principal-component space. A fully connected neural network (FCNN), trained without prior knowledge of the order parameter or the critical temperature, learns the strength of SAF stripe ordering and uses it as the internal feature for phase classification. FSS of thermodynamic observables, the PCA-derived quantity, and the FCNN output yields consistent estimates of the critical temperature and correlation-length exponent. These results demonstrate that machine learning can decode ordering patterns, extract meaningful order-parameter analogs, and reproduce critical behavior in frustrated Ising systems.
Machine learning (ML) can process large sets of data generated from complex systems, which is ideal for classification tasks as often appeared in critical phenomena. Meanwhile ML techniques have been found effective in detecting critical points, or in a broader sense phase separation, and extracting critical exponents. But there are still many unsolved issues with the ML, one of which is the meaning of hidden variables of unsupervised learning. Some say that the hidden variables and the principal component may contain basic information regarding the order parameter of the system of interest, which sounds plausible but lacks evidence. This study aims at searching for evidence supporting the conjecture that the autoencoder's (AE) single latent variable and PCA's first principal component can only serve as signals related to particle density, which happens to be the order parameter of the non-equilibrium DP model. Indeed, in some phase transition (PT) models the order parameter is the particle density, whereas in some PT models it is not. Having conducted a certain degree of random shuffling on the DP configurations, which are then fed to the neural networks as input, we find that AE's single latent variable and PCA's first principal component can indeed represent particle density. It is found that shuffling does affect the size of maximum cluster in the system, which suggests that the second principal component of the PCA is related to the maximal cluster. This has been supported by changes in the correlation length of the transition system with variations in the shuffle ratio.
We apply unsupervised machine learning techniques, mainly principal component analysis (PCA), to compare and contrast the phase behavior and phase transitions in several classical spin models-the square- and triangular-lattice Ising models, the Blume-Capel model, a highly degenerate biquadratic-exchange spin-1 Ising (BSI) model, and the two-dimensional XY model-and we examine critically what machine learning is teaching us. We find that quantified principal components from PCA not only allow the exploration of different phases and symmetry-breaking, but they can distinguish phase-transition types and locate critical points. We show that the corresponding weight vectors have a clear physical interpretation, which is particularly interesting in the frustrated models such as the triangular antiferromagnet, where they can point to incipient orders. Unlike the other well-studied models, the properties of the BSI model are less well known. Using both PCA and conventional Monte Carlo analysis, we demonstrate that the BSI model shows an absence of phase transition and macroscopic ground-state degeneracy. The failure to capture the "charge" correlations (vorticity) in the BSI model (XY model) from raw spin configurations points to some of the limitations of PCA. Finally, we employ a nonlinear unsupervised machine learning procedure, the "autoencoder method," and we demonstrate that it too can be trained to capture phase transitions and critical points.
The identification of universal properties from minimally processed data sets is one goal of machine learning techniques applied to statistical physics.Here, we study how the minimum number of variables needed to accurately describe the important features of a data set-the intrinsic dimension (I d )-behaves in the vicinity of phase transitions.We employ state-of-the-art nearest-neighbors-based I d estimators to compute the I d of raw Monte Carlo thermal configurations across different phase transitions: first-order, second-order, and Berezinskii-Kosterlitz-Thouless.For all the considered cases, we find that the I d uniquely characterizes the transition regime.The finite-size analysis of the I d allows us to not only identify critical points with an accuracy comparable to methods that rely on a priori identification of order parameters but also to determine the corresponding (critical) exponent ν in the case of continuous transitions.For the case of topological transitions, this analysis overcomes the reported limitations affecting other unsupervised learning methods.Our work reveals how raw data sets display unique signatures of universal behavior in the absence of any dimensional reduction scheme and suggest direct parallelism between conventional order parameters in real space and the intrinsic dimension in the data space.
We apply various unsupervised machine learning methods for phase classification to investigate the finite-temperature phase diagram of the spinless Falicov-Kimball model in two dimensions. Using only particle occupation snapshots from Monte Carlo simulations as input, each technique, including a straightforward classification based on principal component analysis (PCA), successfully identifies the phase boundary between ordered and disordered phases, independent of the type of phase transition. Remarkably, these techniques also distinguish between the weakly localized and Anderson-localized regimes within the disordered phase, accurately identifying their crossover, which is a challenging task for standard methods. Among the machine learning approaches used, PCA based analysis outperforms more complex methods, such as neural network predictors and autoencoders. These results underscore the effectiveness of simple unsupervised techniques in examining phase transitions and electron localization in complex correlated systems.