Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2006.10176

Restricted Boltzmann Machine Flows and The Critical Temperature of Ising models

Rodrigo Veiga, Renato Vicente

Published Jun 17, 2020• 3 citations
arXivHTMLPDF

Abstract

We explore alternative experimental setups for the iterative sampling (flow) from Restricted Boltzmann Machines (RBM) mapped on the temperature space of square lattice Ising models by a neural network thermometer. This framework has been introduced to explore connections between RBM-based deep neural networks and the Renormalization Group (RG). It has been found that, under certain conditions, the flow of an RBM trained with Ising spin configurations approaches in the temperature space a value around the critical one: $ k_B T_c / J \approx 2.269$. In this paper we consider datasets with no information about model topology to argue that a neural network thermometer is not an accurate way to detect whether the RBM has learned scale invariance or not.

DOI
10.48550/arxiv.2006.10176

Related papers

Scale-invariant feature extraction of neural network and renormalization group flow

Satoshi Iso, Shotaro Shiba, Sumito Yokoo

Theoretical understanding of how a deep neural network (DNN) extracts features from input images is still unclear, but it is widely believed that the extraction is performed hierarchically through a process of coarse graining. It reminds us of the basic renormalization group (RG) concept in statistical physics. In order to explore possible relations between DNN and RG, we use the restricted Boltzmann machine (RBM) applied to an Ising model and construct a flow of model parameters (in particular, temperature) generated by the RBM. We show that the unsupervised RBM trained by spin configurations at various temperatures from T=0 to T=6 generates a flow along which the temperature approaches the critical value T_{c}=2.27. This behavior is the opposite of the typical RG flow of the Ising model. By analyzing various properties of the weight matrices of the trained RBM, we discuss why it flows towards T_{c} and how the RBM learns to extract features of spin configurations.

arXiv:1801.07172•Physical review. E•May 8, 2018•75 citations

Thermodynamics and feature extraction by machine learning

Shotaro Shiba, Dimitrios Giataganas

Machine learning methods are powerful in distinguishing different phases of matter in an automated way and provide a new perspective on the study of physical phenomena.We train a restricted Boltzmann machine (RBM) on data constructed with spin configurations sampled from the Ising Hamiltonian at different values of temperature and external magnetic field using Monte Carlo methods.From the trained machine we obtain the flow of iterative reconstruction of spin state configurations to faithfully reproduce the observables of the physical system.We find that the flow of the trained RBM approaches the spin configurations of the maximal possible specific heat which resemble the near-criticality region of the Ising model.In the special case of the vanishing magnetic field the trained RBM converges to the critical point of the renormalization group (RG) flow of the lattice model.Our results suggest an explanation of how the machine identifies the physical phase transitions, by recognizing certain properties of the configuration like the maximization of the specific heat, instead of associating directly the recognition procedure with the RG flow and its fixed points.Then from the reconstructed data we deduce the critical exponent associated with the magnetization to find satisfactory agreement with the actual physical value.We assume no prior knowledge about the criticality of the system and its Hamiltonian.

arXiv:1810.08179•Physical Review Research•Sep 15, 2020•47 citations

Neural network flows of low q-state Potts and clock models

Dimitrios Giataganas, Ching-Yu Huang, Feng-Li Lin

Abstract It is known that a trained restricted Boltzmann machine (RBM) on the binary Monte Carlo Ising spin configurations, generates a series of iterative reconstructed spin configurations which spontaneously flow and stabilize to the critical point of physical system. Here we construct a variety of neural network (NN) flows using the RBM and (variational) autoencoders, to study the q-state Potts and clock models on the square lattice for q = 2, 3, 4. The NN are trained on Monte Carlo spin configurations at various temperatures. We find that the trained NN flow does develop a stable point that coincides with critical point of the q-state spin models. The behavior of the NN flow is nontrivial and generative, since the training is unsupervised and without any prior knowledge about the critical point and the Hamiltonian of the underlying spin model. Moreover, we find that the convergence of the flow is independent of the types of NNs and spin models, hinting a universal behavior. Our results strengthen the potential applicability of the notion of the NN flow in studying various states of matter and offer additional evidence on the connection with the renormalization group flow.

arXiv:2102.05219•New Journal of Physics•Apr 1, 2022•17 citations

Short-sighted deep learning

Ellen De Mello Koch, Anita de Mello Koch, Nicholas Kastanos +1 more

A theory explaining how deep learning works is yet to be developed. Previous work suggests that deep learning performs a coarse graining, similar in spirit to the renormalization group (RG). This idea has been explored in the setting of a local (nearest-neighbor interactions) Ising spin lattice. We extend the discussion to the setting of a long-range spin lattice. Markov-chain Monte Carlo (MCMC) simulations determine both the critical temperature and scaling dimensions of the system. The model is used to train both a single restricted Boltzmann machine (RBM) network, as well as a stacked RBM network. Following earlier Ising model studies, the trained weights of a single-layer RBM network define a flow of lattice models. In contrast to results for nearest-neighbor Ising, the RBM flow for the long-ranged model does not converge to the correct values for the spin and energy scaling dimension. Further, correlation functions between visible and hidden nodes exhibit key differences between the stacked RBM and RG flows. The stacked RBM flow appears to move toward low temperatures, whereas the RG flow moves toward high temperature. This again differs from results obtained for nearest-neighbor Ising.

arXiv:2002.02664•Physical review. E•Jul 10, 2020•6 citations

Feature extraction of two dimensional Ising model by unsupervised neural networks

純斗 横尾, スミト ヨコオ, Sumito Yokoo

Deep Neural Network (DNN) has been very successful.Especially, unsupervised DNN can extract the features of complicated dataset without any prior knowledge.However, theoretical understanding of how it extracts the features and what it extracts as the features is not clear.We investigate these questions from the theoretical physics point of view by using the Restricted Boltzmann Machine (RBM) and the Autoencoder (AE) trained by spin configurations of the Ising model.In order to explore the features RBM and AE extract, we train them by spin configurations of the 2-dimensional Ising model and construct a flow of its parameters (in particular temperature) generated by the trained RBM and AE.This flow is motivated by the renormalization group flow of statistical model.It is thought that this flow emphasizes the "relevant" features the unsupervised networks learn, and eliminate "irrelevant" information from the dataset.We show that the RBM trained by spin configurations at various temperatures generates a flow on which the temperature approaches the critical temperature T c ≃ 2.27 if the number of visible units is less than hidden units.Unlike the RBM case, the temperature of the fixed points of AE trained sufficiently is lower than the critical temperature if AE is sufficiently trained (5000 epochs).On the other hand, if the learning epochs are 1000, the fixed point temperature monotonically increase with the number of hidden units.We also analyze the weight matrices by singular value decomposition in order to read out the information RBM and AE learn.4 Autoencoder 4.1 Setup . . . . . .

•Mar 24, 2020•0 citations

Is Deep Learning a Renormalization Group Flow?

Ellen De Mello Koch, Robert de Mello Koch, Ling Cheng

Although there has been a rapid development of practical applications, theoretical explanations of deep learning are in their infancy. Deep learning performs a sophisticated coarse graining. Since coarse graining is a key ingredient of the renormalization group (RG), RG may provide a useful theoretical framework directly relevant to deep learning. In this study we pursue this possibility. A statistical mechanics model for a magnet, the Ising model, is used to train an unsupervised restricted Boltzmann machine (RBM). The patterns generated by the trained RBM are compared to the configurations generated through an RG treatment of the Ising model. Although we are motivated by the connection between deep learning and RG flow, in this study we focus mainly on comparing a single layer of a deep network to a single step in the RG flow. We argue that correlation functions between hidden and visible neurons are capable of diagnosing RG-like coarse graining. Numerical experiments show the presence of RG-like patterns in correlators computed using the trained RBMs. The observables we consider are also able to exhibit important differences between RG and deep learning.

arXiv:1906.05212•IEEE Access•Jan 1, 2020•32 citations
We explore alternative experimental setups for the iterative sampling (flow) from Restricted Boltzmann Machines (RBM) mapped on the temperature space of square lattice Ising models by a neural network thermometer. This framework has been introduced to explore connections between RBM-based deep neural networks and the Renormalization Group (RG). It has been found that, under certain conditions, the flow of an RBM trained with Ising spin configurations approaches in the temperature space a value around the critical one: kBTc/J≈2.269 k_B T_c / J \approx 2.269kB​Tc​/J≈2.269. In this paper we consider datasets with no information about model topology to argue that a neural network thermometer is not an accurate way to detect whether the RBM has learned scale invariance or not.
Theoretical understanding of how a deep neural network (DNN) extracts features from input images is still unclear, but it is widely believed that the extraction is performed hierarchically through a process of coarse graining. It reminds us of the basic renormalization group (RG) concept in statistical physics. In order to explore possible relations between DNN and RG, we use the restricted Boltzmann machine (RBM) applied to an Ising model and construct a flow of model parameters (in particular, temperature) generated by the RBM. We show that the unsupervised RBM trained by spin configurations at various temperatures from T=0 to T=6 generates a flow along which the temperature approaches the critical value T_{c}=2.27. This behavior is the opposite of the typical RG flow of the Ising model. By analyzing various properties of the weight matrices of the trained RBM, we discuss why it flows towards T_{c} and how the RBM learns to extract features of spin configurations.
Machine learning methods are powerful in distinguishing different phases of matter in an automated way and provide a new perspective on the study of physical phenomena.We train a restricted Boltzmann machine (RBM) on data constructed with spin configurations sampled from the Ising Hamiltonian at different values of temperature and external magnetic field using Monte Carlo methods.From the trained machine we obtain the flow of iterative reconstruction of spin state configurations to faithfully reproduce the observables of the physical system.We find that the flow of the trained RBM approaches the spin configurations of the maximal possible specific heat which resemble the near-criticality region of the Ising model.In the special case of the vanishing magnetic field the trained RBM converges to the critical point of the renormalization group (RG) flow of the lattice model.Our results suggest an explanation of how the machine identifies the physical phase transitions, by recognizing certain properties of the configuration like the maximization of the specific heat, instead of associating directly the recognition procedure with the RG flow and its fixed points.Then from the reconstructed data we deduce the critical exponent associated with the magnetization to find satisfactory agreement with the actual physical value.We assume no prior knowledge about the criticality of the system and its Hamiltonian.
Abstract It is known that a trained restricted Boltzmann machine (RBM) on the binary Monte Carlo Ising spin configurations, generates a series of iterative reconstructed spin configurations which spontaneously flow and stabilize to the critical point of physical system. Here we construct a variety of neural network (NN) flows using the RBM and (variational) autoencoders, to study the q-state Potts and clock models on the square lattice for q = 2, 3, 4. The NN are trained on Monte Carlo spin configurations at various temperatures. We find that the trained NN flow does develop a stable point that coincides with critical point of the q-state spin models. The behavior of the NN flow is nontrivial and generative, since the training is unsupervised and without any prior knowledge about the critical point and the Hamiltonian of the underlying spin model. Moreover, we find that the convergence of the flow is independent of the types of NNs and spin models, hinting a universal behavior. Our results strengthen the potential applicability of the notion of the NN flow in studying various states of matter and offer additional evidence on the connection with the renormalization group flow.
A theory explaining how deep learning works is yet to be developed. Previous work suggests that deep learning performs a coarse graining, similar in spirit to the renormalization group (RG). This idea has been explored in the setting of a local (nearest-neighbor interactions) Ising spin lattice. We extend the discussion to the setting of a long-range spin lattice. Markov-chain Monte Carlo (MCMC) simulations determine both the critical temperature and scaling dimensions of the system. The model is used to train both a single restricted Boltzmann machine (RBM) network, as well as a stacked RBM network. Following earlier Ising model studies, the trained weights of a single-layer RBM network define a flow of lattice models. In contrast to results for nearest-neighbor Ising, the RBM flow for the long-ranged model does not converge to the correct values for the spin and energy scaling dimension. Further, correlation functions between visible and hidden nodes exhibit key differences between the stacked RBM and RG flows. The stacked RBM flow appears to move toward low temperatures, whereas the RG flow moves toward high temperature. This again differs from results obtained for nearest-neighbor Ising.
Deep Neural Network (DNN) has been very successful.Especially, unsupervised DNN can extract the features of complicated dataset without any prior knowledge.However, theoretical understanding of how it extracts the features and what it extracts as the features is not clear.We investigate these questions from the theoretical physics point of view by using the Restricted Boltzmann Machine (RBM) and the Autoencoder (AE) trained by spin configurations of the Ising model.In order to explore the features RBM and AE extract, we train them by spin configurations of the 2-dimensional Ising model and construct a flow of its parameters (in particular temperature) generated by the trained RBM and AE.This flow is motivated by the renormalization group flow of statistical model.It is thought that this flow emphasizes the "relevant" features the unsupervised networks learn, and eliminate "irrelevant" information from the dataset.We show that the RBM trained by spin configurations at various temperatures generates a flow on which the temperature approaches the critical temperature T c ≃ 2.27 if the number of visible units is less than hidden units.Unlike the RBM case, the temperature of the fixed points of AE trained sufficiently is lower than the critical temperature if AE is sufficiently trained (5000 epochs).On the other hand, if the learning epochs are 1000, the fixed point temperature monotonically increase with the number of hidden units.We also analyze the weight matrices by singular value decomposition in order to read out the information RBM and AE learn.4 Autoencoder 4.1 Setup . . . . . .
Although there has been a rapid development of practical applications, theoretical explanations of deep learning are in their infancy. Deep learning performs a sophisticated coarse graining. Since coarse graining is a key ingredient of the renormalization group (RG), RG may provide a useful theoretical framework directly relevant to deep learning. In this study we pursue this possibility. A statistical mechanics model for a magnet, the Ising model, is used to train an unsupervised restricted Boltzmann machine (RBM). The patterns generated by the trained RBM are compared to the configurations generated through an RG treatment of the Ising model. Although we are motivated by the connection between deep learning and RG flow, in this study we focus mainly on comparing a single layer of a deep network to a single step in the RG flow. We argue that correlation functions between hidden and visible neurons are capable of diagnosing RG-like coarse graining. Numerical experiments show the presence of RG-like patterns in correlators computed using the trained RBMs. The observables we consider are also able to exhibit important differences between RG and deep learning.