Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2010.05703 · Physical Review Research

Neural Monte Carlo renormalization group

Jui-Hui Chung, Ying-Jer Kao

Published Jun 22, 2021• 20 citations
arXivHTMLPDF

Abstract

The key idea behind the renormalization group (RG) transformation is that properties of physical systems with very different microscopic makeups can be characterized by a few universal parameters.However, finding a systematic way to construct RG transformation for particular systems remains difficult due to the many possible choices of the weight factors in the RG procedure.Here we show, by identifying the conditional distribution in the restricted Boltzmann machine and the weight factor distribution in the RG procedure, that a valid real-space RG transformation can be learned without prior knowledge of the physical system.This neural Monte Carlo RG algorithm allows for direct computation of the RG flow and critical exponents.Our results establish a solid connection between the RG transformation in physics and the deep architecture in machine learning, paving the way for further interdisciplinary research.

DOI
10.1103/physrevresearch.3.023230

Related papers

A Deep Dive into the Connections Between the Renormalization Group and Deep Learning in the Ising Model

Taylor, Kelsie

The renormalization group (RG) is an essential technique in statistical physics and quantum field theory, which considers scale-invariant properties of physical theories and how these theories' parameters change with scaling. Deep learning is a powerful computational technique that uses multi-layered neural networks to solve a myriad of complicated problems. Previous research suggests the possibility that unsupervised deep learning may be a form of RG flow, by being a layer-by-layer coarse graining of the original data. We examined this connection on a more rigorous basis for the simple example of Kadanoff block renormalization of the 2D nearest-neighbor Ising model, with our deep learning accomplished via Restricted Boltzmann Machines (RBMs). We developed extensive renormalization techniques for the 1D and 2D Ising model to provide a baseline for comparison. For the 1D Ising model, we successfully used Adam optimization on a correlation length loss function to learn the group flow, yielding results consistent with the analytical model for infinite N. For the 2D Ising model, we successfully generated Ising model samples using the Wolff algorithm, and performed the group flow using a quasi-deterministic method, validating these results by calculating the critical exponent ν. We then examined RBM learning of the Ising model layer by layer, finding a blocking structure in the learning that is qualitatively similar to RG. Lastly, we directly compared the weights of each layer from the learning to Ising spin renormalization, but found quantitative inconsistencies for the simple case of nearest-neighbor Ising models.

arXiv:2308.11075•Jun 26, 2023•0 citations

An exact mapping between the Variational Renormalization Group and Deep Learning

Pankaj Mehta, David J. Schwab

Deep learning is a broad set of techniques that uses multiple layers of representation to automatically learn relevant features directly from structured data. Recently, such techniques have yielded record-breaking results on a diverse set of difficult machine learning tasks in computer vision, speech recognition, and natural language processing. Despite the enormous success of deep learning, relatively little is understood theoretically about why these techniques are so successful at feature learning and compression. Here, we show that deep learning is intimately related to one of the most important and successful techniques in theoretical physics, the renormalization group (RG). RG is an iterative coarse-graining scheme that allows for the extraction of relevant features (i.e. operators) as a physical system is examined at different length scales. We construct an exact mapping from the variational renormalization group, first introduced by Kadanoff, and deep learning architectures based on Restricted Boltzmann Machines (RBMs). We illustrate these ideas using the nearest-neighbor Ising Model in one and two-dimensions. Our results suggests that deep learning algorithms may be employing a generalized RG-like scheme to learn relevant features from data.

arXiv:1410.3831•Oct 14, 2014•195 citations

Renormalization Group connected to Neural Networks

J.J. ter Hoeve

Even though deep learning has proved to be very powerful as the core method of machine learning, theoretical understanding behind its success is still unclear. It is been pointed out in recent years that the behaviour of deep neural networks is reminiscent of a fundamental framework in statistical physics: the renormalization group (RG). Motivated by this analogy, we develop an analytical method of directly obtaining the trained weights W resulting from a training set of 1D-Ising samples with coupling J and temperature T. The found relation W(J) drives a flow that takes 1D-Ising configurations at non-zero temperature to the critical point at T = 0. This behaviour is opposite to what a typical RG-flow dictates.

•Jan 1, 2018•0 citations

Scale-invariant feature extraction of neural network and renormalization group flow

Satoshi Iso, Shotaro Shiba, Sumito Yokoo

Theoretical understanding of how a deep neural network (DNN) extracts features from input images is still unclear, but it is widely believed that the extraction is performed hierarchically through a process of coarse graining. It reminds us of the basic renormalization group (RG) concept in statistical physics. In order to explore possible relations between DNN and RG, we use the restricted Boltzmann machine (RBM) applied to an Ising model and construct a flow of model parameters (in particular, temperature) generated by the RBM. We show that the unsupervised RBM trained by spin configurations at various temperatures from T=0 to T=6 generates a flow along which the temperature approaches the critical value T_{c}=2.27. This behavior is the opposite of the typical RG flow of the Ising model. By analyzing various properties of the weight matrices of the trained RBM, we discuss why it flows towards T_{c} and how the RBM learns to extract features of spin configurations.

arXiv:1801.07172•Physical review. E•May 8, 2018•75 citations

Deep learning and the renormalization group

Cédric Bény

Renormalization group (RG) methods, which model the way in which the effective behavior of a system depends on the scale at which it is observed, are key to modern condensed-matter theory and particle physics. We compare the ideas behind the RG on the one hand and deep machine learning on the other, where depth and scale play a similar role. In order to illustrate this connection, we review a recent numerical method based on the RG---the multiscale entanglement renormalization ansatz (MERA)---and show how it can be converted into a learning algorithm based on a generative hierarchical Bayesian network model. Under the assumption---common in physics---that the distribution to be learned is fully characterized by local correlations, this algorithm involves only explicit evaluation of probabilities, hence doing away with sampling.

arXiv:1301.3124•Jan 14, 2013•44 citations

Optimal Real-Space Renormalization-Group Transformations with Artificial Neural Networks

Jui-Hui Chung, Ying-Jer Kao

We introduce a general method for optimizing real-space renormalization-group transformations to study the critical properties of a classical system. The scheme is based on minimizing the Kullback-Leibler divergence between the distribution of the system and the normalized normalizing factor of the transformation parametrized by a restricted Boltzmann machine. We compute the thermal critical exponent of the two-dimensional Ising model using the trained optimal projector and obtain a very accurate thermal critical exponent $y_t=1.0001(11)$ after the first step of the transformation.

arXiv:1912.09005•Dec 19, 2019•0 citations
The key idea behind the renormalization group (RG) transformation is that properties of physical systems with very different microscopic makeups can be characterized by a few universal parameters.However, finding a systematic way to construct RG transformation for particular systems remains difficult due to the many possible choices of the weight factors in the RG procedure.Here we show, by identifying the conditional distribution in the restricted Boltzmann machine and the weight factor distribution in the RG procedure, that a valid real-space RG transformation can be learned without prior knowledge of the physical system.This neural Monte Carlo RG algorithm allows for direct computation of the RG flow and critical exponents.Our results establish a solid connection between the RG transformation in physics and the deep architecture in machine learning, paving the way for further interdisciplinary research.
The renormalization group (RG) is an essential technique in statistical physics and quantum field theory, which considers scale-invariant properties of physical theories and how these theories' parameters change with scaling. Deep learning is a powerful computational technique that uses multi-layered neural networks to solve a myriad of complicated problems. Previous research suggests the possibility that unsupervised deep learning may be a form of RG flow, by being a layer-by-layer coarse graining of the original data. We examined this connection on a more rigorous basis for the simple example of Kadanoff block renormalization of the 2D nearest-neighbor Ising model, with our deep learning accomplished via Restricted Boltzmann Machines (RBMs). We developed extensive renormalization techniques for the 1D and 2D Ising model to provide a baseline for comparison. For the 1D Ising model, we successfully used Adam optimization on a correlation length loss function to learn the group flow, yielding results consistent with the analytical model for infinite N. For the 2D Ising model, we successfully generated Ising model samples using the Wolff algorithm, and performed the group flow using a quasi-deterministic method, validating these results by calculating the critical exponent ν. We then examined RBM learning of the Ising model layer by layer, finding a blocking structure in the learning that is qualitatively similar to RG. Lastly, we directly compared the weights of each layer from the learning to Ising spin renormalization, but found quantitative inconsistencies for the simple case of nearest-neighbor Ising models.
Deep learning is a broad set of techniques that uses multiple layers of representation to automatically learn relevant features directly from structured data. Recently, such techniques have yielded record-breaking results on a diverse set of difficult machine learning tasks in computer vision, speech recognition, and natural language processing. Despite the enormous success of deep learning, relatively little is understood theoretically about why these techniques are so successful at feature learning and compression. Here, we show that deep learning is intimately related to one of the most important and successful techniques in theoretical physics, the renormalization group (RG). RG is an iterative coarse-graining scheme that allows for the extraction of relevant features (i.e. operators) as a physical system is examined at different length scales. We construct an exact mapping from the variational renormalization group, first introduced by Kadanoff, and deep learning architectures based on Restricted Boltzmann Machines (RBMs). We illustrate these ideas using the nearest-neighbor Ising Model in one and two-dimensions. Our results suggests that deep learning algorithms may be employing a generalized RG-like scheme to learn relevant features from data.
Even though deep learning has proved to be very powerful as the core method of machine learning, theoretical understanding behind its success is still unclear. It is been pointed out in recent years that the behaviour of deep neural networks is reminiscent of a fundamental framework in statistical physics: the renormalization group (RG). Motivated by this analogy, we develop an analytical method of directly obtaining the trained weights W resulting from a training set of 1D-Ising samples with coupling J and temperature T. The found relation W(J) drives a flow that takes 1D-Ising configurations at non-zero temperature to the critical point at T = 0. This behaviour is opposite to what a typical RG-flow dictates.
Theoretical understanding of how a deep neural network (DNN) extracts features from input images is still unclear, but it is widely believed that the extraction is performed hierarchically through a process of coarse graining. It reminds us of the basic renormalization group (RG) concept in statistical physics. In order to explore possible relations between DNN and RG, we use the restricted Boltzmann machine (RBM) applied to an Ising model and construct a flow of model parameters (in particular, temperature) generated by the RBM. We show that the unsupervised RBM trained by spin configurations at various temperatures from T=0 to T=6 generates a flow along which the temperature approaches the critical value T_{c}=2.27. This behavior is the opposite of the typical RG flow of the Ising model. By analyzing various properties of the weight matrices of the trained RBM, we discuss why it flows towards T_{c} and how the RBM learns to extract features of spin configurations.
Renormalization group (RG) methods, which model the way in which the effective behavior of a system depends on the scale at which it is observed, are key to modern condensed-matter theory and particle physics. We compare the ideas behind the RG on the one hand and deep machine learning on the other, where depth and scale play a similar role. In order to illustrate this connection, we review a recent numerical method based on the RG---the multiscale entanglement renormalization ansatz (MERA)---and show how it can be converted into a learning algorithm based on a generative hierarchical Bayesian network model. Under the assumption---common in physics---that the distribution to be learned is fully characterized by local correlations, this algorithm involves only explicit evaluation of probabilities, hence doing away with sampling.
We introduce a general method for optimizing real-space renormalization-group transformations to study the critical properties of a classical system. The scheme is based on minimizing the Kullback-Leibler divergence between the distribution of the system and the normalized normalizing factor of the transformation parametrized by a restricted Boltzmann machine. We compute the thermal critical exponent of the two-dimensional Ising model using the trained optimal projector and obtain a very accurate thermal critical exponent yt=1.0001(11)y_t=1.0001(11)yt​=1.0001(11) after the first step of the transformation.