Abstract Due to their inherent capabilities of capturing non-local dependencies, Transformer neural networks have quickly been established as the paradigmatic architecture for large language models and image processing. Next to these traditional applications, machine learning (ML) methods have also been demonstrated to be versatile tools in the analysis of image-like data of quantum phases of matter, e.g. given snapshots of many-body wave functions obtained in ultracold atom experiments. While local correlation structures in image-like data of physical systems can reliably be detected, identifying phases of matter characterized by global, non-local structures with interpretable ML methods remains a challenge. Here, we introduce the correlator Transformer (CoTra), which classifies different phases of matter while at the same time yielding full interpretability in terms of physical correlation functions. The network’s underlying structure is a tailored attention mechanism, which learns efficient ways to weigh local and non-local correlations for a successful classification. We demonstrate the versatility of the CoTra by detecting local order in the Heisenberg antiferromagnet, and show that local gauge constraints in one- and two-dimensional lattice gauge theories can be identified. Furthermore, we establish that the CoTra reliably detects non-local structures in images of correlated fermions in momentum space (Cooper pairs) and that it can distinguish percolating from non-percolating images.
Abstract Due to their inherent capabilities of capturing non-local dependencies, Transformer neural networks have quickly been established as the paradigmatic architecture for large language models and image processing. Next to these traditional applications, machine learning (ML) methods have also been demonstrated to be versatile tools in the analysis of image-like data of quantum phases of matter, e.g. given snapshots of many-body wave functions obtained in ultracold atom experiments. While local correlation structures in image-like data of physical systems can reliably be detected, identifying phases of matter characterized by global, non-local structures with interpretable ML methods remains a challenge. Here, we introduce the correlator Transformer (CoTra), which classifies different phases of matter while at the same time yielding full interpretability in terms of physical correlation functions. The network’s underlying structure is a tailored attention mechanism, which learns efficient ways to weigh local and non-local correlations for a successful classification. We demonstrate the versatility of the CoTra by detecting local order in the Heisenberg antiferromagnet, and show that local gauge constraints in one- and two-dimensional lattice gauge theories can be identified. Furthermore, we establish that the CoTra reliably detects non-local structures in images of correlated fermions in momentum space (Cooper pairs) and that it can distinguish percolating from non-percolating images.
Image-like data from quantum systems promises to offer greater insight into the physics of correlated quantum matter. However, the traditional framework of condensed matter physics lacks principled approaches for analyzing such data. Machine learning models are a powerful theoretical tool for analyzing image-like data including many-body snapshots from quantum simulators. Recently, they have successfully distinguished between simulated snapshots that are indistinguishable from one and two point correlation functions. Thus far, the complexity of these models has inhibited new physical insights from such approaches. Here, we develop a set of nonlinearities for use in a neural network architecture that discovers features in the data which are directly interpretable in terms of physical observables. Applied to simulated snapshots produced by two candidate theories approximating the doped Fermi-Hubbard model, we uncover that the key distinguishing features are fourth-order spin-charge correlators. Our approach lends itself well to the construction of simple, versatile, end-to-end interpretable architectures, thus paving the way for new physical insights from machine learning studies of experimental and numerical data.
Detecting phases of matter in general relies on identifying the correct order parameter - a task that remains notoriously difficult for unknown transitions and traditionally is guided by physical intuition and educated guess. Neural networks have recently offered an alternative route by locating phase transitions in known models without any a priori physical knowledge. Yet these approaches remain black boxes and only identify phases without elucidating their properties. Moreover, they often struggle when confronted with realistic, noisy experimental data, which constitute the ultimate testbed for automated methods in physics. Here, we bridge these perspectives by introducing TetrisCNN, a convolutional architecture with parallel branches of differently shaped filters, reminiscent of Tetris blocks, that learns sparse, interpretable latent representations directly in terms of spin correlators. Applied to experimental snapshots of two-dimensional Ising and XY quantum simulators measured in multiple bases, the network not only detects phase transitions and crossovers but also expresses its latent representation and decision boundaries as symbolic formulas built from experimentally measurable spin correlators. This framework opens the way to integrating interpretable neural networks with quantum simulators to uncover and understand new phases of matter.
Microscopically understanding and classifying phases of matter is at the heart of strongly-correlated quantum physics. With quantum simulations, genuine projective measurements (snapshots) of the many-body state can be taken, which include the full information of correlations in the system. The rise of deep neural networks has made it possible to routinely solve abstract processing and classification tasks of large datasets, which can act as a guiding hand for quantum data analysis. However, though proven to be successful in differentiating between different phases of matter, conventional neural networks mostly lack interpretability on a physical footing. Here, we combine confusion learning [Nat. Phys. 13, 435 (2017)] with correlation convolutional neural networks [Nat. Commun. 12, 3905 (2021)], which yields fully interpretable phase detection in terms of correlation functions. In particular, we study thermodynamic properties of the 2D Heisenberg model, whereby the trained network is shown to pick up qualitative changes in the snapshots above and below a characteristic temperature where magnetic correlations become significantly long-range. We identify the full counting statistics of nearest neighbor spin correlations as the most important quantity for the decision process of the neural network, which go beyond averages of local observables. With access to the fluctuations of second-order correlations — which indirectly include contributions from higher order, long-range correlations — the network is able to detect changes of the specific heat and spin susceptibility, the latter being in analogy to magnetic properties of the pseudogap phase in high-temperature superconductors [Phys. Rev. Lett. 62, 957 (1989)]. By combining the confusion learning scheme with transformer neural networks, our work opens new directions in interpretable quantum image processing being sensible to long-range order.
Abstract Recent proposals have suggested the use of supervised learning with convolutional neural networks to shed light on some of the less well known phases of the Fermi-Hubbard model through the classification of snapshots from the quantum gas microscopy of ultracold atoms in optical lattices. However, there have been challenges in the interpretability of networks with more than one convolutional filter coupled to the input images. Here, we expand on previous work by considering multiple filters in the first convolutional layer and developing a process for analyzing the physical relevance of patterns obtained in the trained filters. We benchmark our approach at half-filling, where strong antiferromagnetic correlations are known to be present, and we find that upon hole doping, previously unknown patterns arise at temperatures below the tunneling amplitude. These patterns may be a signature of interesting arrangements of fermions in the lattice.
The identification and characterization of phase transitions is one of the oldest and most central problems in condensed matter physics, traditionally approached through order parameters, free-energy expansions, renormalization-group analysis, and direct numerical simulation. Over the past decade, machine learning (ML) has emerged as a complementary and, in some regimes, transformative methodology for this task. Supervised classifiers can learn to distinguish phases directly from raw configurations without an a priori order parameter; unsupervised and self-supervised techniques such as principal component analysis, autoencoders, and learning-by-confusion can locate transition points with minimal or no labeled data; and generative and variational architectures, including neural-network quantum states, now approximate ground-state wavefunctions of strongly correlated systems well beyond the reach of exact diagonalization. This paper reviews the theoretical foundations of phase transitions relevant to machine-learning applications, surveys the principal families of algorithms used to detect and classify phases, and examines their application across classical spin systems, percolation, topological and quantum phases, strongly correlated electron systems, and experimentally measured spectroscopic data. Representative case studies are discussed, including convolutional and graph-neural-network classifiers of lattice configurations, explainable-ML detection of thermodynamic transitions in photoemission spectra, and deep-learning-guided discovery of high-temperature superconductors. The review closes with a critical discussion of persistent challenges interpretability, data scarcity in correlated-electron experiments, generalization across Hamiltonians, and the reliability of learned order parameters near criticality and outlines directions for future research, including physics-informed architectures, uncertainty quantification, and closed-loop integration with autonomous experimentation.
Machine learning is capable of discriminating phases of matter, and finding associated phase transitions, directly from large data sets of raw state configurations. In the context of condensed matter physics, most progress in the field of supervised learning has come from employing neural networks as classifiers. Although very powerful, such algorithms suffer from a lack of interpretability, which is usually desired in scientific applications in order to associate learned features with physical phenomena. In this paper, we explore support vector machines (SVMs), which are a class of supervised kernel methods that provide interpretable decision functions. We find that SVMs can learn the mathematical form of physical discriminators, such as order parameters and Hamiltonian constraints, for a set of two-dimensional spin models: the ferromagnetic Ising model, a conserved-order-parameter Ising model, and the Ising gauge theory. The ability of SVMs to provide interpretable classification highlights their potential for automating feature detection in both synthetic and experimental data sets for condensed matter and other many-body systems.