One of the theoretical pillars that sustain certain machine learning models are universal approximation theorems, which prove that they can approximate all functions from a function class to arbitrary precision. Independently, classical spin models are termed universal if they can reproduce the behavior of any other spin model in their low energy sector. Universal spin models have been characterized via sufficient and necessary conditions, showing that simple models such as the 2d Ising with fields are universal. In this work, we prove that universal spin models are universal approximators of probability distributions. This enables us to leverage the characterization of the former to reveal conditions which are sufficient for universal approximation. Deriving universal approximation theorems thus amounts to verifying these conditions, yielding a unified recipe for universal approximation theorems applicable to a wide range of models. We explicitly test this recipe for restricted and deep Boltzmann machines, as well as for deep belief networks. This work illustrates that independently discovered universality statements may be intimately related, enabling the transfer of results.
One of the theoretical pillars that sustain certain machine learning models are universal approximation theorems, which prove that they can approximate all functions from a function class to arbitrary precision. Independently, classical spin models are termed universal if they can reproduce the behavior of any other spin model in their low energy sector. Universal spin models have been characterized via sufficient and necessary conditions, showing that simple models such as the 2d Ising with fields are universal. In this work, we prove that universal spin models are universal approximators of probability distributions. This enables us to leverage the characterization of the former to reveal conditions which are sufficient for universal approximation. Deriving universal approximation theorems thus amounts to verifying these conditions, yielding a unified recipe for universal approximation theorems applicable to a wide range of models. We explicitly test this recipe for restricted and deep Boltzmann machines, as well as for deep belief networks. This work illustrates that independently discovered universality statements may be intimately related, enabling the transfer of results.
Spin models are used in many studies of complex systems because they exhibit rich macroscopic behavior despite their microscopic simplicity. Here, we prove that all the physics of every classical spin model is reproduced in the low-energy sector of certain "universal models," with at most polynomial overhead. This holds for classical models with discrete or continuous degrees of freedom. We prove necessary and sufficient conditions for a spin model to be universal and show that one of the simplest and most widely studied spin models, the two-dimensional Ising model with fields, is universal. Our results may facilitate physical simulations of Hamiltonians with complex interactions.
The theory of spin models intersects with condensed matter physics, complex systems, graph theory, combinatorial optimization, computational complexity and neural networks. Many ensuing applications rely on transforming complicated spin models to simpler ones. What is the structure of such transformations? Here, we provide a framework to study and construct emulations between spin models. A spin model is a set of spin systems, and emulations are efficiently computable families of simulations with arbitrary energy cut-off, where a source spin system simulates a target system if, below the cut-off, the target Hamiltonian is encoded in the source Hamiltonian. We prove that emulations preserve important properties, as they induce reductions between computational problems such as computing ground states, approximating partition functions and approximate sampling from Boltzmann distributions. Emulations are modular (they can be added, scaled and composed), and allow for universality, i.e., under emulations, certain spin models have maximal reach. We prove that a spin model is universal if and only if it is scalable, closed and functional complete. Because the characterization is constructive, it provides a step-by-step guide to construct emulations. We prove that the 2d Ising model with fields is universal, for which we also provide two new crossing gadgets. Finally, we show that simulations can be computed by linear programs. While some ideas of this work are contained in G. De las Cuevas and T. S. Cubitt, Science 351, 1180 (2016), we provide new definitions and theorems. This framework provides a toolbox for applications involving emulations of spin models.
Encoding classical data into quantum states is considered a quantum feature map to map classical data into a quantum Hilbert space. This feature map provides opportunities to incorporate quantum advantages into machine learning algorithms to be performed on near-term intermediate-scale quantum computers. The crucial idea is using the quantum Hilbert space as a quantum-enhanced feature space in machine learning models. Although the quantum feature map has demonstrated its capability when combined with linear classification models in some specific applications, its expressive power from the theoretical perspective remains unknown. We prove that the machine learning models induced from the quantum-enhanced feature space are universal approximators of continuous functions under typical quantum feature maps. We also study the capability of quantum feature maps in the classification of disjoint regions. Our work enables an important theoretical analysis to ensure that machine learning algorithms based on quantum feature maps can handle a broad class of machine learning tasks. In light of this, one can design a quantum machine learning model with more powerful expressivity.
The main question raised in the article is whether a neural network trained on a spin lattice model in one universality class can be used to test a model in another universality class. The quantities of interest are the critical phase transition temperature and the correlation length exponent. In other words, the question of transfer learning is how "universal" the trained network is and under what conditions. For this purpose, we applied a supervised learning procedure to three two-dimensional models for which critical properties are precisely known: The Ising model, the four-state Potts model, and the Baxter-Wu model. We consider two datasets: One with spin configurations and one with binding energy configurations. We find that estimates of the critical temperature agree well with the known results for both datasets, but not with the results of cross-testing using the energy datasets of the two models: The four-state Potts model and the Ising model. Estimates of the critical length exponent are less regular, and appear to be more accurate for energy datasets. A good example is the cross-testing using the energy dataset between the Ising model and the Baxter-Wu model in both training and testing directions.
Beyond their origin in modeling many-body quantum systems, tensor networks have emerged as a promising class of models for solving machine learning problems, notably in unsupervised generative learning. While possessing many desirable features arising from their quantum-inspired nature, tensor network generative models have previously been largely restricted to binary or categorical data, limiting their utility in real-world modeling problems. We overcome this by introducing a new family of tensor network generative models for continuous data, which are capable of learning from distributions containing continuous random variables. We develop our method in the setting of matrix product states, first deriving a universal expressivity theorem proving the ability of this model family to approximate any reasonably smooth probability density function with arbitrary precision. We then benchmark the performance of this model on several synthetic and real-world datasets, finding that the model learns and generalizes well on distributions of continuous and discrete variables. We develop methods for modeling different data domains, and introduce a trainable compression layer which is found to increase model performance given limited memory or computational resources. Overall, our methods give important theoretical and empirical evidence of the efficacy of quantum-inspired methods for the rapidly growing field of generative learning.
Recent work has demonstrated the existence of universal Hamiltonians-simple spin-lattice models that can simulate any other quantum many-body system to any desired level of accuracy.Until now, proofs of universality have relied on explicit constructions, tailored to each specific family of universal Hamiltonians.In this work, we go beyond this approach and completely classify the simulation ability of quantum Hamiltonians by their complexity classes.We do this by deriving necessary and sufficient complexitytheoretic conditions characterizing universal quantum Hamiltonians.Although the result concerns the theory of analog Hamiltonian simulation-a promising application of near-term quantum technology-the proof relies on abstract complexity-theoretic concepts and the theory of quantum computation.As well as providing simplified proofs of previous Hamiltonian universality results and offering a route to new universal constructions, the results in this paper give insight into the origins of universality; for example, finally explaining the previously noted coincidences between families of universal Hamiltonian and classes of Hamiltonians appearing in complexity theory.