A restricted Boltzmann machine (RBM) learns a probability distribution over its input samples and has numerous uses like dimensionality reduction, classification and generative modeling. Conventional RBMs accept vectorized data that dismiss potentially important structural information in the original tensor (multi-way) input. Matrix-variate and tensor-variate RBMs, named MvRBM and TvRBM, have been proposed but are all restrictive by model construction and have weak model expression power. This work presents the matrix product operator RBM (MPORBM) that utilizes a tensor network generalization of Mv/TvRBM, preserves input formats in both the visible and hidden layers, and results in higher expressive power. A novel training algorithm integrating contrastive divergence and an alternating optimization procedure is also developed. Numerical experiments compare the MPORBM with the traditional RBM and MvRBM for data classification and image completion and denoising tasks. The expressive power of the MPORBM as a function of the MPO-rank is also investigated.
A restricted Boltzmann machine (RBM) learns a probability distribution over its input samples and has numerous uses like dimensionality reduction, classification and generative modeling. Conventional RBMs accept vectorized data that dismiss potentially important structural information in the original tensor (multi-way) input. Matrix-variate and tensor-variate RBMs, named MvRBM and TvRBM, have been proposed but are all restrictive by model construction and have weak model expression power. This work presents the matrix product operator RBM (MPORBM) that utilizes a tensor network generalization of Mv/TvRBM, preserves input formats in both the visible and hidden layers, and results in higher expressive power. A novel training algorithm integrating contrastive divergence and an alternating optimization procedure is also developed. Numerical experiments compare the MPORBM with the traditional RBM and MvRBM for data classification and image completion and denoising tasks. The expressive power of the MPORBM as a function of the MPO-rank is also investigated.
Restricted Boltzmann machines (RBMs) and deep Boltzmann machines (DBMs) are important models in machine learning, and recently found numerous applications in quantum many-body physics. We show that there are fundamental connections between them and tensor networks. In particular, we demonstrate that any RBM and DBM can be exactly represented as a two-dimensional tensor network. This representation gives characterizations of the expressive power of RBMs and DBMs using entanglement structures of the tensor networks, and also provides an efficient tensor network contraction algorithm for the computing partition function of RBMs and DBMs. Using numerical experiments, we show that the proposed algorithm is more accurate than the state-of-the-art machine learning methods in estimating the partition function of RBMs and DBMs, and have potential applications in training DBMs for general machine learning tasks.
Modeling the joint distribution of high-dimensional data is a central task in unsupervised machine learning. In recent years, many interests have been attracted to developing learning models based on tensor networks, which have the advantages of a principle understanding of the expressive power using entanglement properties, and as a bridge connecting classical computation and quantum computation. Despite the great potential, however, existing tensor network models for unsupervised machine learning only work as a proof of principle, as their performance is much worse than the standard models such as restricted Boltzmann machines and neural networks. In this Letter, we present autoregressive matrix product states (AMPS), a tensor network model combining matrix product states from quantum many-body physics and autoregressive modeling from machine learning. Our model enjoys the exact calculation of normalized probability and unbiased sampling. We demonstrate the performance of our model using two applications, generative modeling on synthetic and real-world data, and reinforcement learning in statistical physics. Using extensive numerical experiments, we show that the proposed model significantly outperforms the existing tensor network models and the restricted Boltzmann machines, and is competitive with state-of-the-art neural network models.
Quantum computers offer the potential for efficiently sampling from complex probability distributions, attracting increasing interest in generative modeling within quantum machine learning. This surge in interest has driven the development of numerous generative quantum models, yet their trainability and scalability remain significant challenges. A notable example is a quantum restricted Boltzmann machine (QRBM), which is based on the Gibbs state of a parameterized non-commuting Hamiltonian. While QRBMs are expressive, their non-commuting Hamiltonians make gradient evaluation computationally demanding, even on fault-tolerant quantum computers. In this work, we propose a semi-quantum restricted Boltzmann machine (sqRBM), a model designed for classical data that mitigates the challenges associated with previous QRBM proposals. The sqRBM Hamiltonian is commuting in the visible subspace while remaining non-commuting in the hidden subspace. This structure allows us to derive closed-form expressions for both output probabilities and gradients. Leveraging these analytical results, we demonstrate that sqRBMs share a close relationship with classical restricted Boltzmann machines (RBM). Our theoretical analysis predicts that, to learn a given probability distribution, an RBM requires three times as many hidden units as an sqRBM, while both models have the same total number of parameters. We validate these findings through numerical simulations involving up to 100 units. Our results suggest that sqRBMs could enable practical quantum machine learning applications in the near future by significantly reducing quantum resource requirements.
Abstract A restricted Boltzmann machine (RBM) is a two-layer neural-network machine-learning model comprising a visible layer that represents observed data and a hidden layer that captures correlations among the visible units. Rooted in ideas from statistical mechanics, RBMs provide a probabilistic framework for modeling complex, high-dimensional distributions through an energy function and a corresponding Boltzmann distribution. This review offers an accessible introduction to RBMs from a physics-oriented perspective, focusing on their core concepts and training principles. We survey key theoretical developments that interpret RBMs as many-body systems, highlighting their equilibrium/non-equilibrium thermodynamics, learning dynamics, mean-field properties, phase diagrams, and free energy calculation. We further discuss recent applications of RBMs to a range of physics problems, including quantum-mechanical wavefunction representation, statistical-mechanical Boltzmann distribution modeling, and biological protein sequence analysis.
Matrix product states (MPSs), a tensor network designed for one-dimensional quantum systems, were recently proposed for generative modeling of natural data (such as images) in terms of the ``Born machine.'' However, the exponential decay of correlation in MPSs restricts its representation power heavily for modeling complex data such as natural images. In this work, we push forward the effort of applying tensor networks to machine learning by employing the tree tensor network (TTN), which exhibits balanced performance in expressibility and efficient training and sampling. We design the tree tensor network to utilize the two-dimensional prior of the natural images and develop sweeping learning and sampling algorithms which can be efficiently implemented utilizing graphical processing units. We apply our model to random binary patterns and the binary MNIST data sets of handwritten digits. We show that the TTN is superior to MPSs for generative modeling in keeping the correlation of pixels in natural images, as well as giving better log-likelihood scores in standard data sets of handwritten digits. We also compare its performance with state-of-the-art generative models such as variational autoencoders, restricted Boltzmann machines, and PixelCNN. Finally, we discuss the future development of tensor network states in machine learning problems.
Lately, there has been an extensive state-of-the-art research in Machine Learning methods, due to their important features such as universality in approximations and dimensional reduction. In this way, the present work aims at exploiting these properties of Machine Learning on physical many-body problems, in which a restricted Boltzmann Machine (RBM) is trained with quantum Monte Carlo data to best represent the ground state of an N spin system under the transverse field Ising model Hamiltonian in one dimension. During the path coursed along the development of this project, we first review classical well-implemented methods such as the single-layer perceptron, multi-layer perceptron for the XOR problem with back-propagation algorithm, discrete Hopfield network for pattern reconstruction, continuous Hopfield network with Simulated Annealing for solving the traveling salesman problem with 10 cities and the Boltzmann machine for feature extraction. Then, we review a few aspects of what is called a Quantum Boltzmann Machine and the basics of the transverse field Ising model. Finally, we train an RBM consisting of N visible neurons, hidden density neurons alpha=M/N=2 and 1-step contrastive divergence (CD_1), by minimizing the Kullbach-Liebler divergence of the RBM and the training data set coming from the algorithmic variant of quantum Monte Carlo, known as discrete Path-Integral Monte Carlo. For this last step, we use Trotter-Suzuki decomposition with m Trotter slices and the Wolff-s cluster algorithm. After training the RBM, we calculate magnetic observables based on the RBM-s wave-function such as longitudinal and transverse magnetization, for different values of transverse field and we do an estimation of the critical field at which the transverse Ising model system suffers a quantum phase transition from ferromagnet to paramagnet