In this thesis, we consider quantum extension of the well-known stochastic neuralnetwork model called (classical) Boltzmann machine (CBM) from an informationgeometrical point of view. The new model is called quantum Boltzmann machine(QBM).We investigate some properties and geometrical aspects of QBM analogous tothose of CBM. Furthermore, we study the mean-field approximation for such a modelfrom the information geometrical point of view. This is in some sense motivated bythe application of mean-field approximation for probabilistic inference in the graphicalmodels in the classical probability theory. Although the problem tackled in the presentthesis is somewhat deviated from the major field of quantum information theory, wehave elucidated the relationships among several well-known fields such as statisticalphysics, differential geometry, information theory and statistics using the concepts ofthe new emerging subject of quantum information geometry.We first define QBMs which can be considered as a general class of quantumIsing spin models. The states we consider are assumed to have at most secondorderinteractions with arbitrary but deterministic coupling coefficients. We call sucha state a QBM for the reason that it can be regarded as a quantum extension ofthe equilibrium distribution of CBM. The totality of QBMs is then shown to forma quantum exponential family and thus can be considered as a smooth manifoldhaving similar geometrical structures to those of CBMs. The information geometricalstructure of the manifold of QBMs is discussed and the problem of approximating agiven quantum state (density operator) by a QBM is also treated.We also define a restricted class of QBMs called the strongly separable QBMs(SSQBMs). We consider the dynamics of SSQBMs and propose a new state renewalrule based on that of CBM. The geometrical structure of the totality of SSQBMs isshown to be equivalent to that of the totality of CBMs. Approximation process forSSQBMs is also studied. Finally, we briefly discuss the parameter estimation of aSSQBM.Next, we study the mean-field approximation for QBMs from an information geometricalpoint of view. We elaborate on the significance and usefulness of informationgeometrical concepts, in particular the e-(exponential) and m-(mixture) projections,in studying the naive mean-field approximation for QBMs and derive the naive meanfieldequation explicitly. We also discuss the higher-order corrections to the naivemean-field approximation based on the idea of Plefka expansion in statistical physics.We elucidate the geometrical essence of the corrections and provide the expansioncoefficients with expressions in terms of information geometrical quantities. Here,one may note this work as the information geometrical interpretation of [Ple06] andas the quantum extension of [Tan00].
In this thesis, we consider quantum extension of the well-known stochastic neuralnetwork model called (classical) Boltzmann machine (CBM) from an informationgeometrical point of view. The new model is called quantum Boltzmann machine(QBM).We investigate some properties and geometrical aspects of QBM analogous tothose of CBM. Furthermore, we study the mean-field approximation for such a modelfrom the information geometrical point of view. This is in some sense motivated bythe application of mean-field approximation for probabilistic inference in the graphicalmodels in the classical probability theory. Although the problem tackled in the presentthesis is somewhat deviated from the major field of quantum information theory, wehave elucidated the relationships among several well-known fields such as statisticalphysics, differential geometry, information theory and statistics using the concepts ofthe new emerging subject of quantum information geometry.We first define QBMs which can be considered as a general class of quantumIsing spin models. The states we consider are assumed to have at most secondorderinteractions with arbitrary but deterministic coupling coefficients. We call sucha state a QBM for the reason that it can be regarded as a quantum extension ofthe equilibrium distribution of CBM. The totality of QBMs is then shown to forma quantum exponential family and thus can be considered as a smooth manifoldhaving similar geometrical structures to those of CBMs. The information geometricalstructure of the manifold of QBMs is discussed and the problem of approximating agiven quantum state (density operator) by a QBM is also treated.We also define a restricted class of QBMs called the strongly separable QBMs(SSQBMs). We consider the dynamics of SSQBMs and propose a new state renewalrule based on that of CBM. The geometrical structure of the totality of SSQBMs isshown to be equivalent to that of the totality of CBMs. Approximation process forSSQBMs is also studied. Finally, we briefly discuss the parameter estimation of aSSQBM.Next, we study the mean-field approximation for QBMs from an information geometricalpoint of view. We elaborate on the significance and usefulness of informationgeometrical concepts, in particular the e-(exponential) and m-(mixture) projections,in studying the naive mean-field approximation for QBMs and derive the naive meanfieldequation explicitly. We also discuss the higher-order corrections to the naivemean-field approximation based on the idea of Plefka expansion in statistical physics.We elucidate the geometrical essence of the corrections and provide the expansioncoefficients with expressions in terms of information geometrical quantities. Here,one may note this work as the information geometrical interpretation of [Ple06] andas the quantum extension of [Tan00].
We study the mean-field approximation for a general class of quantum Ising spin states from an information geometrical point of view. The states we consider are assumed to have at most second-order interactions with arbitrary but deterministic coupling coefficients. We call such a state a quantum Boltzmann machine (QBM) for the reason that it can be regarded as a quantum extension of the equilibrium distribution of a (classical) Boltzmann machine (CBM), which is a well-known stochastic neural network model. The totality of QBMs is then shown to form a quantum exponential family and thus can be considered as a smooth manifold having similar geometrical structures to those of CBMs. We elaborate on the significance and usefulness of information geometrical concepts, in particular the e- and m-projections, in studying the naive mean-field approximation for QBMs. We also discuss the higher-order corrections to the naive mean-field approximation based on the idea of the Plefka expansion in statistical physics. We elucidate the geometrical essence of the corrections and provide the expansion coefficients with expressions in terms of information geometrical quantities. Here, one may note this work as the information geometrical interpretation of (Plefka T 2006 Phys. Rev. E 73 016129) and as the quantum extension of (Tanaka T 2000 Neural Comput. 12 1951–68).
This work is a simple extension of \cite{NNjpa}. We apply the concepts of information geometry to study the mean-field approximation for a general class of quantum statistical models namely the higher-order quantum Boltzmann machines (QBMs). The states we consider are assumed to have at most third-order interactions with deterministic coupling coefficients. Such states, taken together, can be shown to form a quantum exponential family and thus can be viewed as a smooth manifold. In our work, we explicitly obtain naive mean-field equations for the third-order classical and quantum Boltzmann machines and demonstrate how some information geometrical concepts, particularly, exponential and mixture projections used to study the naive mean-field approximation in \cite{NNjpa} can be extended to a more general case. Though our results do not differ much from those in \cite{NNjpa}, we emphasize the validity and the importance of information geometrical point of view for higher dimensional classical and quantum statistical models.
Quantum machine learning is at the crossroads of two of the most exciting current areas of research: quantum computing and classical machine learning. Although the field is still in its infancy, the body of literature is already large enough to warrant several review articles [1–3]. This short survey focuses on a selection of significant recent results on the subtopic of quantum neural networks, an area that hopes to build on the enormous impact that classical neural networks have had. In particular, we concentrate on quantum generalizations of popular neural-network concepts such as Boltzmann machines, generative adversarial networks and autoencoders, as well as applications of classical neural networks to quantum information and vice versa. Examples of Boltzmann machines. In a fully connected Boltzmann machine, connections exist between all visible and hidden neurons. In a restricted Boltzmann machine (RBM), there are no connections between neurons of the same type. A deep Boltzmann machine (DBM) consists of adding one or more additional layers of hidden neurons to an RBM, with no connections between neurons in the same layer. The energy-based nature of BMs gives a natural framework for considering quantum generalizations of their behavior. The BM energy function is equivalent to the Hamiltonian of a simple Ising model and one might hope that more general Hamiltonians allowed by quantum mechanics could explain certain data sets better than classically. Amin et al. [4] adopt this approach, and take the so-called transverse field Ising Hamiltonian as the energy function. This increased generality brings certain difficulties, however, as the transverse field term makes training the quantum Boltzmann machine (QBM) non-trivial. Although the general scaling and performance of QBMs is not yet well understood, for certain small-size examples the authors observe that their QBMs can learn data distributions better than classical BMs. Kieferova and Wiebe [5] extend the formalism of Amin et al. and consider more general Hamiltonians that, unlike the transverse Ising model, are not in general believed to be classically efficiently simulatable. Furthermore, they propose new methods for training that allow for parameters corresponding to quantum terms to be learned. Finally, their QBM enables a form of quantum-state tomography that can learn non-classical data, and also generate copies of the reconstructed state. Again, small-scale numerical evidence suggests that their proposals may provide an advantage over classical BMs, although their tendency to overfit data must be investigated further. The practical value of BMs, either classical or quantum, relies in part on the ease with which they are trained. This in turn depends on sampling efficiently from the Boltzmann distribution. Classically this can be done approximately only for the special case of restricted Boltzmann machines (RBM), i.e. where connections are between visible and hidden units, and no connections exist between neurons of the same type. Wiebe et al. [6] propose two quantum algorithms for Boltzmann sampling, based on starting from a state corresponding to the product distribution that is closest to the target Boltzmann distribution. While small-scale numerical studies are encouraging, true bounds on the running time of their algorithms rely on a problem-dependent parameter κ, the scaling of which is hard to estimate. The hope remains though that empirically their algorithms work well in practice, even for multi-layer and fully connected BMs. The expressive power of BMs for compactly representing complicated probability distributions has led to another line of investigation, namely their use for representing quantum many-body wave functions. The authors apply this technique to two prototypical spin models and achieve high accuracy in both ground-state estimation and dynamical evolution, improving on existing state-of-the-art techniques. Their results indicate that RBM states can capture complex many-body correlations in an efficient manner—evidence reinforced by the work of Deng et al. [8], who show that certain topological quantum states in one, two and three dimensions can be represented by RBM states with a number of parameters only linear in the system size. In a subsequent study, however, Gao and Duan [9] prove that although RBM states can indeed represent many highly entangled states, their expressive power is limited in that they cannot represent arbitrary states generated by polynomial-sized quantum circuits, nor those that are the ground states of k-local Hamiltonians with polynomial-sized energy gaps, unless the polynomial hierarchy from computational complexity theory collapses. On the other hand, they show that these two important classes of states can be efficiently represented by a deep Boltzmann machine of three layers. Intriguingly though, another paper by Deng et al. [10] gives an analytic construction of a family of RBM states with a number of parameters linear in the system size that exhibits maximal volume-law entanglement, and that cannot be represented efficiently by either matrix product states (MPS) or tensor-network (TN) states. This shows that the expressive power of even efficiently parametrized RBM states is not limited by entanglement in the same way that other popular compact representations are. This gives the hope that RBM states may prove a valuable tool in understanding entangled quantum systems intractable by traditional methods. Levine et al. [11] establish another connection between the many-body wave function and neural networks, this time in the form of convolutional arithmetic circuits (ConvAC), a variant of the popular convolutional neural networks in widespread use today. By proving a one-to-one equivalence between a ConvAC circuit and TN quantum states, the authors are able to use tools from quantum information theory to analyze the entanglement present in the corresponding TN, and quantify the neural network's ability to model complex correlations between its inputs. This in turn can be used to obtain bounds on the expressivity of a given ConvAC architecture and be of practical use in choosing network parameters tailored to the data. While ConvAC are not widely used in practice, the authors empirically validate that the insights gained can be applied successfully to more common convolutional neural-network designs. Generative adversarial networks (GAN) are machine-learning algorithms in which two players—a generator and a discriminator—compete against each another. The generator's goal is to create and send fake data samples to the discriminator and maximize the probability that the discriminator erroneously believes that the sample comes from some true data distribution. The discriminator attempts to discern real from fake data as best as possible. Under reasonable assumptions the game converges to a point where the discriminator is unable to discriminate with probability greater than 1/2 (i.e. no better than random guessing), and the generator in fact has learned to generate data from the true distribution. Lloyd and Weedbrook [12] consider the natural extension in which one or both of the generator and discriminator are equipped with quantum information processors. When the data to be learned are a quantum state and both parties have quantum processors, the authors show that the unique end point of the adversarial game also coincides with the generator producing a quantum state corresponding to the true data statistics, and the discriminator unable to discern true from false data better than random guessing. They further argue that a different outcome may hold when the true samples are produced by the measurement of some quantum state, and the generator is purely classical. The concept of quantum supremacy is based on the assumption that there exist probability distributions accessible to a quantum computer that no classical computer can efficiently sample from. If the true data correspond to one of these distributions, then no classical generator can efficiently generate fake data reproducing the true statistics and, in principle, the discriminator can successfully discriminate true from fake data. However, whether the discriminator can efficiently find the measurements required to do so is unclear. Quantum-computing techniques have also found applications in autoencoders—neural networks where a larger number of input and output neurons are connected via a smaller number of intermediary latent neurons. The goal is to tune the weights of the network so that a training set of classical bit strings can be approximately recovered after being passed through the autoencoder. Romero et al. [13] consider instead a training set of quantum states, and ask whether a quantum circuit with a similar bottleneck structure can compress and approximately recover the data. The circuits that they consider consist of gates chosen from some set of parametrized unitary operations, and arranged in some fixed order. The parameters are then optimized classically so that the circuit best predicts the training-set labels. The authors demonstrate the validity of this approach by showing that ground states of the Hubbard model and certain molecules can indeed be compressed and recovered by such circuits. However, in general, the performance of such quantum autoencoders will be highly dependent on the kind of programmable circuit on which they are based, and at present there does not seem to be a structured way of determining this. Quantum neural networks are still in their infancy and, while the above works are important steps towards bridging our understanding between classical neural networks and quantum computing, many questions remain unanswered. If the expressivity of Boltzmann machine states is not restricted by entanglement in the same way that MPS and tensor-network states are, what is the best way of understanding their limitations? How can physicists use the correspondence between ConvAC and TN to their advantage in finding efficient, expressive representations of many-body wave functions? Can classical data distributions be learned more efficiently by quantum GAN than classical GAN? Whatever the outcome of these and many other related questions, this promises to be an exciting and active area of research in the years to come.
Recently, the possibility of investigating single molecule, or single spin observables, as well as the necessity of a better understanding of the mechanisms underlying quantum dynamics in order to obtain nanoscale devices and nanostructered materials suitable for quantum computing tasks, have revived the interest in foundational aspects of quantum statistical mechanics. This thesis aims to give a contribution to this field by re-considering the statistical characterization of a quantum system at the light of some paradigmatic changes in our understanding of quantum theory which have taken place in the last two decades. In particular the impressive development of quantum information theory has changed the perceptions of quantum entanglement: for a long time it has been considered a somewhat paradoxical property of the matter at the atomic scale, but now it is regarded as an essential and ubiquitous phenomenon whose consequences are affecting the very macroscopic world that we experience. Still the decoherence program has brought out the importance of considering a quantum system together with its environment in order to clarify some key aspects of quantum dynamics. Thus, we start from the idea that quantum correlations are ubiquitous and somewhat uncontrollable in systems with many degrees of freedom which are typically considered in statistical mechanics. As a consequence we assume the standpoint that quantum statistical mechanics has not to be based on the underlying idea of a collection of many, independent quantum systems but rather it has to emerge at the level of a global wavefunction (pure state) which describes the system as well as its environment as a whole. In order to investigate the consequences of these assumptions we study the equilibrium distribution of an isolated quantum system. This is defined, in analogy with the ergodic foundations of classical statistical mechanics, on the basis of the time evolution of the quantum state. Then, we study the emergence of thermodynamic properties in a quantum system by studying the probability distribution of some function of interests, as the entropy and the equilibrium state of a subsystem, on Ensembles of Pure States. Such a probability distribution is derived from the geometry of the Hilbert space, and the theoretical tools suitable for its characterization are developed. On the one hand we perform a numerical sampling of the ensemble distributions by employing Monte Carlo techniques, on the other hand simpler analytical approximation of the geometrical distributions are derived by means of a maximum entropy principle. Model systems composed of an ensemble of spins are chosen to illustrate the salient features which emerge from the developed theoretical framework: the main point is that the Ensemble Distributions of “thermodynamic observables” (entropy or equilibrium state of a subsystem) are sharply peaked around a typical value. From the analysis it emerges that each of the overwhelming majority of the wavefunctions which has appreciable weight in the considered ensemble, is characterized by the same value of the “macroscopic” functions. This is a striking evidence of the “typicality” of these properties. In the essence, our impossibility to know the state of the system in detail does not matter, just for the remarkable fact that almost all quantum states behave essentially in the same way. By virtue of this typicality the study of the behaviour of the typical values of the thermodynamic function become meaningful. Notably, under certain conditions, one recovers the results of standard statistical mechanics, that is, the equilibrium average of the state of a subsystem can be cast in the Boltzmann canonical form at the temperature given by the usual thermodynamical relation . In the second part of the thesis we consider the dynamical aspects of the equilibrium state of a subsystem interacting with its environment. The fluctuations around the equilibrium average critically depends on the entanglement between the system and the environment and on the form of the interaction Hamiltonian. The connection between the dynamics of the fluctuations of an observable at the equilibrium and the relaxation toward the equilibrium from a “non typical” initial value is also investigated with the aid of simple model systems. The study presented in this thesis was partly motivated by a critical analysis of the statistical methods available for the theoretical modelling of magnetic resonance experiments. One of these, the Stochastic Liouville Equation, has been employed in a work completed during the first year of my Ph.D. program in order to interpret some feature of a two dimensional electron spin resonance experiment, [Fresch B., Frezzato D., Moro G. J., Kothe G., Freed J. H.; J. Phys. Chem. B., 110, 24238, (2006)].
Abstract Quantum Boltzmann machines (QBMs) are machine-learning models for both classical and quantum data. We give an operational definition of QBM learning in terms of the difference in expectation values between the model and target, taking into account the polynomial size of the data set. By using the relative entropy as a loss function, this problem can be solved without encountering barren plateaus. We prove that a solution can be obtained with stochastic gradient descent using at most a polynomial number of Gibbs states. We also prove that pre-training on a subset of the QBM parameters can only lower the sample complexity bounds. In particular, we give pre-training strategies based on mean-field, Gaussian Fermionic, and geometrically local Hamiltonians. We verify these models and our theoretical findings numerically on a quantum and a classical data set. Our results establish that QBMs are promising machine learning models.
Lately, there has been an extensive state-of-the-art research in Machine Learning methods, due to their important features such as universality in approximations and dimensional reduction. In this way, the present work aims at exploiting these properties of Machine Learning on physical many-body problems, in which a restricted Boltzmann Machine (RBM) is trained with quantum Monte Carlo data to best represent the ground state of an N spin system under the transverse field Ising model Hamiltonian in one dimension. During the path coursed along the development of this project, we first review classical well-implemented methods such as the single-layer perceptron, multi-layer perceptron for the XOR problem with back-propagation algorithm, discrete Hopfield network for pattern reconstruction, continuous Hopfield network with Simulated Annealing for solving the traveling salesman problem with 10 cities and the Boltzmann machine for feature extraction. Then, we review a few aspects of what is called a Quantum Boltzmann Machine and the basics of the transverse field Ising model. Finally, we train an RBM consisting of N visible neurons, hidden density neurons alpha=M/N=2 and 1-step contrastive divergence (CD_1), by minimizing the Kullbach-Liebler divergence of the RBM and the training data set coming from the algorithmic variant of quantum Monte Carlo, known as discrete Path-Integral Monte Carlo. For this last step, we use Trotter-Suzuki decomposition with m Trotter slices and the Wolff-s cluster algorithm. After training the RBM, we calculate magnetic observables based on the RBM-s wave-function such as longitudinal and transverse magnetization, for different values of transverse field and we do an estimation of the critical field at which the transverse Ising model system suffers a quantum phase transition from ferromagnet to paramagnet