Directed percolation (DP) provides a canonical framework for studying nonequilibrium phase transitions into absorbing states, yet extracting critical behavior directly from high-dimensional spatiotemporal configurations remains challenging. Here, we investigate (1 + 1)- and (2 + 1)-dimensional bond DP using a supervised Transformer encoder. Supervised learning results demonstrate that the Transformer can distinguish between the active and absorbing phases of directed percolation with high accuracy. The critical point and critical exponents estimated from the model's classification probabilities align closely with reference values from existing literature, indicating that the model not only achieves phase classification but also yields outputs containing information related to critical scaling behavior. To probe the model's internal representation, we employ attention rollout and define an attention center that quantifies the temporal centroid of the cumulative attention distribution. The attention patterns and the attention center reorganize systematically near the critical region and exhibit pronounced finite-size dependence. Finally, we examined the model's transferability between configurations generated from fully occupied initial states and those from single-seed initial states. The results indicate the presence of transferable information between the two types of data, though the transfer performance exhibits directional asymmetry. Furthermore, when using an equal amount of target-domain data, fine-tuning yielded only a marginal improvement in accuracy compared to training from scratch.
Directed percolation (DP) provides a canonical framework for studying nonequilibrium phase transitions into absorbing states, yet extracting critical behavior directly from high-dimensional spatiotemporal configurations remains challenging. Here, we investigate (1 + 1)- and (2 + 1)-dimensional bond DP using a supervised Transformer encoder. Supervised learning results demonstrate that the Transformer can distinguish between the active and absorbing phases of directed percolation with high accuracy. The critical point and critical exponents estimated from the model's classification probabilities align closely with reference values from existing literature, indicating that the model not only achieves phase classification but also yields outputs containing information related to critical scaling behavior. To probe the model's internal representation, we employ attention rollout and define an attention center that quantifies the temporal centroid of the cumulative attention distribution. The attention patterns and the attention center reorganize systematically near the critical region and exhibit pronounced finite-size dependence. Finally, we examined the model's transferability between configurations generated from fully occupied initial states and those from single-seed initial states. The results indicate the presence of transferable information between the two types of data, though the transfer performance exhibits directional asymmetry. Furthermore, when using an equal amount of target-domain data, fine-tuning yielded only a marginal improvement in accuracy compared to training from scratch.
Machine learning (ML) has been well applied to studying equilibrium phase transition models by accurately predicating critical thresholds and some critical exponents. Difficulty will be raised, however, for integrating ML into nonequilibrium phase transitions. The extra dimension in a given nonequilibrium system, namely time, can greatly slow down the procedure toward the steady state. In this paper we find that by using some simple techniques of ML, non-steady-state configurations of directed percolation (DP) suffice to capture its essential critical behaviors in both (1+1) and (2+1) dimensions. With the supervised learning method, the framework of our binary classification neural networks can identify the phase transition threshold, as well as the spatial and temporal correlation exponents. The characteristic time t_{c}, specifying the transition from active phases to absorbing ones, is also a major product of the learning. Moreover, we employ the convolutional autoencoder, an unsupervised learning technique, to extract dimensionality reduction representations and cluster configurations of (1+1) bond DP. It is quite appealing that such a method can yield a reasonable estimation of the critical point.
The latest advances of statistical physics have shown remarkable performance of machine learning in identifying phase transitions. In this paper, we apply domain adversarial neural network (DANN) based on transfer learning to studying nonequilibrium and equilibrium phase transition models, which are percolation model and directed percolation (DP) model, respectively. With the DANN, only a small fraction of input configurations (two-dimensional images) needs to be labeled, which is automatically chosen, to capture the critical point. To learn the DP model, the method is refined by an iterative procedure in determining the critical point, which is a prerequisite for the data collapse in calculating the critical exponent ν_{⊥}. We then apply the DANN to a two-dimensional site percolation with configurations filtered to include only the largest cluster which may contain the information related to the order parameter. The DANN learning of both models yields reliable results which are comparable to the ones from Monte Carlo simulations. Our study also shows that the DANN can achieve quite high accuracy at much lower cost, compared to the supervised learning.
Recently, machine learning algorithms have been used remarkably to study the equilibrium phase transitions, however there are only a few works have been done using this technique in the nonequilibrium phase transitions. In this work, we use the supervised learning with the convolutional neural network (CNN) algorithm and unsupervised learning with the density-based spatial clustering of applications with noise (DBSCAN) algorithm to study the nonequilibrium phase transition in two models. We use CNN and DBSCAN in order to determine the critical points for directed bond percolation (bond DP) model and Domany-Kinzel cellular automaton (DK) model. Both models have been proven to have a nonequilibrium phase transition belongs to the directed percolation (DP) universality class. In the case of supervised learning we train CNN using the images which are generated from Monte Carlo simulations of directed bond percolation. We use that trained CNN in studding the phase transition for the two models. In the case of unsupervised learning, we train DBSCAN using the raw data of Monte Carlo simulations. In this case, we retrain DBSCAN at each time we change the model or lattice size. Our results from both algorithms show that, even for a very small values of lattice size, machine can predict the critical points accurately for both models. Finally, we mention to that, the value of the critical point we find here for bond DP model using CNN or DBSCAN is exactly the same value that has been found using transfer learning with a domain adversarial neural network (DANN) algorithm.
Phase transitions mark qualitative reorganizations of collective behavior, yet identifying their boundaries remains challenging whenever analytic solutions are absent and conventional simulations fail. Here we introduce learnability as a universal criterion, defined as the ability of a transformer model containing attention mechanism to extract structure from microscopic states. Using self-supervised learning and Monte Carlo generated configurations of the two-dimensional Ising model, we show that ordered phases correspond to enhanced learnability, manifested in both reduced training loss and structured attention patterns, while disordered phases remain resistant to learning. Two unsupervised diagnostics, the sharp jump in training loss and the rise in attention entropy, recover the critical temperature in excellent agreement with the exact value. Our results establish learnability as a data-driven marker of phase transitions and highlight deep parallels between long-range order in condensed matter and the emergence of structure in modern language models.
We perform a principal component analysis (PCA) of two one-dimensional lattice models belonging to distinct nonequilibrium universality classes-directed bond percolation and branching and annihilating random walks with an even number of offspring. We find that the uncentered PCA of data sets storing various system's configurations can be successfully used to determine the critical properties of these nonequilibrium phase transitions. In particular, in both cases, we obtain good estimates of the critical point and the dynamical critical exponent of the models. For directed bond percolation, we are furthermore able to extract critical exponents associated with the correlation length and the order parameter. We discuss the relation of our analysis with low-rank approximations of data sets.
Abstract Machine learning (ML) of phase transitions (PTs) has gradually become an effective approach that enables us to explore the nature of various PTs more promptly in equilibrium and nonequilibrium systems. Unlike equilibrium systems, non-equilibrium systems display more complicated and diverse features because of the extra dimension of time, which is not readily tractable, both theoretically and numerically. The combination of ML and most renowned nonequilibrium model, directed percolation (DP), led to some significant findings. In this study, ML is applied to ( 1 + 1 ) -d, even offspring branching annihilating random walks (BAW), whose universality class is not DP-like. The supervised learning of ( 1 + 1 ) -d BAW via convolutional neural networks (CNN) results in a more accurate prediction of the critical point than the Monte Carlo (MC) simulation for the same system sizes. The dynamic exponent z and spatial correlation length correlation exponent ν ⊥ were also measured and found to be consistent with their respective theoretical values. Furthermore, the unsupervised learning of ( 1 + 1 ) -d BAW via an autoencoder (AE) gives rise to a transition point, which is the same as the critical point. The latent layer of AE, through a single neuron, can be regarded as the order parameter of the system being properly re-scaled. Therefore, we believe that ML has exciting application prospects in reaction-diffusion systems such as BAW and DP.