Journal of Physics Communications

Restricted Boltzmann machines in physics: concepts, theories and applications

Kai Zhang, Sora Sakai

Published Sep 1, 2026• 0 citations

Abstract

Abstract A restricted Boltzmann machine (RBM) is a two-layer neural-network machine-learning model comprising a visible layer that represents observed data and a hidden layer that captures correlations among the visible units. Rooted in ideas from statistical mechanics, RBMs provide a probabilistic framework for modeling complex, high-dimensional distributions through an energy function and a corresponding Boltzmann distribution. This review offers an accessible introduction to RBMs from a physics-oriented perspective, focusing on their core concepts and training principles. We survey key theoretical developments that interpret RBMs as many-body systems, highlighting their equilibrium/non-equilibrium thermodynamics, learning dynamics, mean-field properties, phase diagrams, and free energy calculation. We further discuss recent applications of RBMs to a range of physics problems, including quantum-mechanical wavefunction representation, statistical-mechanical Boltzmann distribution modeling, and biological protein sequence analysis.

DOI: 10.1088/2399-6528/aea9f1

Related papers

Thermodynamics of the Ising Model Encoded in Restricted Boltzmann Machines

Jing Gu, Kai Zhang

The restricted Boltzmann machine (RBM) is a two-layer energy-based model that uses its hidden-visible connections to learn the underlying distribution of visible units, whose interactions are often complicated by high-order correlations. Previous studies on the Ising model of small system sizes have shown that RBMs are able to accurately learn the Boltzmann distribution and reconstruct thermal quantities at temperatures away from the critical point Tc. How the RBM encodes the Boltzmann distribution and captures the phase transition are, however, not well explained. In this work, we perform RBM learning of the 2d and 3d Ising model and carefully examine how the RBM extracts useful probabilistic and physical information from Ising configurations. We find several indicators derived from the weight matrix that could characterize the Ising phase transition. We verify that the hidden encoding of a visible state tends to have an equal number of positive and negative units, whose sequence is randomly assigned during training and can be inferred by analyzing the weight matrix. We also explore the physical meaning of the visible energy and loss function (pseudo-likelihood) of the RBM and show that they could be harnessed to predict the critical point or estimate physical quantities such as entropy.

arXiv:2210.06203EntropyNov 22, 20225 citations

Quantum boltzmann machine :emergence & applications

Miguel Francisco Aldana Páez

Lately, there has been an extensive state-of-the-art research in Machine Learning methods, due to their important features such as universality in approximations and dimensional reduction. In this way, the present work aims at exploiting these properties of Machine Learning on physical many-body problems, in which a restricted Boltzmann Machine (RBM) is trained with quantum Monte Carlo data to best represent the ground state of an N spin system under the transverse field Ising model Hamiltonian in one dimension. During the path coursed along the development of this project, we first review classical well-implemented methods such as the single-layer perceptron, multi-layer perceptron for the XOR problem with back-propagation algorithm, discrete Hopfield network for pattern reconstruction, continuous Hopfield network with Simulated Annealing for solving the traveling salesman problem with 10 cities and the Boltzmann machine for feature extraction. Then, we review a few aspects of what is called a Quantum Boltzmann Machine and the basics of the transverse field Ising model. Finally, we train an RBM consisting of N visible neurons, hidden density neurons alpha=M/N=2 and 1-step contrastive divergence (CD_1), by minimizing the Kullbach-Liebler divergence of the RBM and the training data set coming from the algorithmic variant of quantum Monte Carlo, known as discrete Path-Integral Monte Carlo. For this last step, we use Trotter-Suzuki decomposition with m Trotter slices and the Wolff-s cluster algorithm. After training the RBM, we calculate magnetic observables based on the RBM-s wave-function such as longitudinal and transverse magnetization, for different values of transverse field and we do an estimation of the critical field at which the transverse Ising model system suffers a quantum phase transition from ferromagnet to paramagnet

Jan 1, 20180 citations

Developing a Maximum-Entropy Restricted Boltzmann Machine with a Quantum Thermodynamics Formalism

Roshawn Marcell Valentino Terrell, Eleanor Watson, Timofey Golubev

We propose a Restricted Boltzmann Machine (RBM) neural network using a quantum thermodynamics formalism and the maximization of entropy as the cost function for the optimization problem. We verify the possibility of using an entropy maximization approach for modeling physical systems by applying it to the Inverse Ising Problem, which describes a system of interacting spins. The temperature-dependent behaviors of physical quantities, such as magnetization and Gibbs energy, were calculated from the Ising Hamiltonian whose coefficients were obtained through entropy maximization and were found to be in good agreement to literature results. These results suggest that RBM neural networks using the principle of maximum entropy can be applied to modeling physical systems that can be described by discrete states, including fundamental quantum physics such as topological systems, and biological systems such as the correlated spiking of neurons.

arXiv:2103.09482Mar 17, 20210 citations

Machine Learning Classical Spin Models

Ryan van Mastrigt

Machine learning has become increasingly popular as a computational tool in all aspects of science and the private sector, however its application as an additional computational physics tool has only recently began to gain traction. This thesis aims to study the application of supervised and unsupervised neural networks to computational classical statistical physics problems, focusing on what is learned and\nwhether it is a useful tool for these applications. Supervised neural networks are used to distinguish between the different phases of four different models, containing a second-order phase-transition (PT), infinite-order PT, and both in close proximity to one another, as well as a model without a PT but with frustration. Different iterations of the restricted Boltzmann machine (RBM), a type of unsupervised neural\nnetwork, are trained on the one- and two-dimensional Ising model. The neural network is able to differentiate between the different phases only if the PT is clearly discernible from the input configurations. It is too crude a tool to differentiate between two close subsequent PTs. It is concluded that application of neural networks to detect PTs in classical statistical physics models where an intuition for the PT exists, holds no advantage over alternative conventional computational methods. This thesis concludes that restrictions placed on the RBM, such that the RBM has translation\ninvariance, still allow the restricted RBMs to learn the magnetisation really well, while two-spin correlations are learned less well. Surprisingly, the block Gibbs sampling of the restricted RBMs is better behaved than for the unrestricted RBM, as can be explained using an analysis of the trained weight-matrix.

Jan 1, 20190 citations

Boltzmann machines as two-dimensional tensor networks

Sujie Li, Feng Ping Pan, Pengfei Zhou +1 more

Restricted Boltzmann machines (RBMs) and deep Boltzmann machines (DBMs) are important models in machine learning, and recently found numerous applications in quantum many-body physics. We show that there are fundamental connections between them and tensor networks. In particular, we demonstrate that any RBM and DBM can be exactly represented as a two-dimensional tensor network. This representation gives characterizations of the expressive power of RBMs and DBMs using entanglement structures of the tensor networks, and also provides an efficient tensor network contraction algorithm for the computing partition function of RBMs and DBMs. Using numerical experiments, we show that the proposed algorithm is more accurate than the state-of-the-art machine learning methods in estimating the partition function of RBMs and DBMs, and have potential applications in training DBMs for general machine learning tasks.

arXiv:2105.04130Physical review. B./Physical review. BAug 27, 202115 citations

Information Perspective to Probabilistic Modeling: Boltzmann Machines versus Born Machines

Song Cheng, Jing Chen, Lei Wang

We compare and contrast the statistical physics and quantum physics inspired approaches for unsupervised generative modeling of classical data. The two approaches represent probabilities of observed data using energy-based models and quantum states, respectively. Classical and quantum information patterns of the target datasets therefore provide principled guidelines for structural design and learning in these two approaches. Taking the Restricted Boltzmann Machines (RBM) as an example, we analyze the information theoretical bounds of the two approaches. We also estimate the classical mutual information of the standard MNIST datasets and the quantum Rényi entropy of corresponding Matrix Product States (MPS) representations. Both information measures are much smaller compared to their theoretical upper bound and exhibit similar patterns, which imply a common inductive bias of low information complexity. By comparing the performance of RBM with various architectures on the standard MNIST datasets, we found that the RBM with local sparse connection exhibit high learning efficiency, which supports the application of tensor network states in machine learning problems.

arXiv:1712.04144EntropyAug 7, 2018103 citations