Abstract Classification and data reconstruction using a restricted Boltzmann machine (RBM) is presented. RBM is an energy-based model which assigns low energy values to the configurations of interest. It is a generative model, once trained it can be used to produce samples from the target distribution. The D-Wave 2000Q is a quantum computer which has been used to exploit its quantum effect for machine learning. Bars-and-stripes (BAS) and cybersecurity (ISCX) datasets were used to train RBMs. The weights and biases of trained RBMs were used to map onto the D-Wave. Classification as well as image reconstruction were performed. Classification accuracy of both datasets indicates comparable performance using D-Wave’s adiabatic annealing and classical Gibb’s sampling.
Abstract Classification and data reconstruction using a restricted Boltzmann machine (RBM) is presented. RBM is an energy-based model which assigns low energy values to the configurations of interest. It is a generative model, once trained it can be used to produce samples from the target distribution. The D-Wave 2000Q is a quantum computer which has been used to exploit its quantum effect for machine learning. Bars-and-stripes (BAS) and cybersecurity (ISCX) datasets were used to train RBMs. The weights and biases of trained RBMs were used to map onto the D-Wave. Classification as well as image reconstruction were performed. Classification accuracy of both datasets indicates comparable performance using D-Wave’s adiabatic annealing and classical Gibb’s sampling.
Restricted Boltzmann Machine (RBM) is an energy-based, undirected graphical model. It is commonly used for unsupervised and supervised machine learning. Typically, RBM is trained using contrastive divergence (CD). However, training with CD is slow and does not estimate the exact gradient of the log-likelihood cost function. In this work, the model expectation of gradient learning for RBM has been calculated using a quantum annealer (D-Wave 2000Q), where obtaining samples is faster than Markov chain Monte Carlo (MCMC) used in CD. Training and classification results of RBM trained using quantum annealing are compared with the CD-based method. The performance of the two approaches is compared with respect to the classification accuracies, image reconstruction, and log-likelihood results. The classification accuracy results indicate comparable performances of the two methods. Image reconstruction and log-likelihood results show improved performance of the CD-based method. It is shown that the samples obtained from quantum annealer can be used to train an RBM on a 64-bit “bars and stripes” dataset with classification performance similar to an RBM trained with CD. Though training based on CD showed improved learning performance, training using a quantum annealer could be useful as it eliminates computationally expensive MCMC steps of CD.
The next generation D-Wave quantum computer (QC) having more than 2000 qubits was demonstrated to be adequate for embedding graphs of restricted Boltzmann machines (RBMs) having the connectivity between RBM units (though still limited) already sufficient for successful training by classical algorithms. As a verification of the validity and quality of the employed embedding approach, the RBM was trained by classical contrastive divergence (CD), while the D-Wave was initially used only in the recognition step for reconstruction of incomplete test images (8 × 8 bars-and-stripes), and for their classification. The label classification errors by the D-Wave compared favorably to those obtained by classical Gibbs sampling, indicating that the QC was successfully finding the most energetically favorable combinations of the unknown visible RBM units/labels for the given fixed incomplete input image. The valid RBM embedding was further applied to investigate opportunities for using QCs in the RBM training, and specifically for replacing the classical Gibbs technique in generating a representative sample from the RBM model distribution. Statistical comparison of samples obtained by the Gibbs technique versus 10000 sample states generated from the D-Wave revealed significant differences in the observed outcomes. The D-Wave samples were found insufficiently representative of the model distribution, specifically by missing many local valley found by the Gibbs sampling. On the other hand, the D-Wave possessed an ability to find local valleys of the configuration space that were consistently missed by the classical sampling algorithms, which could potentially be very interesting for sampling applications.
Understanding how the D-Wave quantum computer could be used for machine learning problems is of growing interest. Our work explores the feasibility of using the D-Wave as a sampler for a machine learning task. We describe a hybrid method that combines a classical deep neural network autoencoder with a quantum annealing Restricted Boltzmann Machine (RBM) using the D-Wave for image generation. Our method overcomes two key limitations in the 2000-qubit D-Wave processor, namely the limited number of qubits available to accommodate typical problem sizes for fully connected quantum objective functions, and samples that are binary pixel representations. As a consequence of these limitations we are able to show how we achieved nearly a 22-fold compression factor of grayscale 28 x 28 sized images to binary 6 x 6 sized images with a lossy recovery of the original 28 x 28 grayscale images. We further show how generating samples from the D-Wave after training the RBM, resulted in 28 x 28 images that were variations of the original input data distribution, as opposed to recreating the training samples. We evaluated the quality of this method by using a downstream classification method. We formulated a MNIST classification problem using a deep convolutional neural network that used samples from the quantum RBM to train the MNIST classifier and compared the results with a MNIST classifier trained with the original MNIST training data set, as well as a MNIST classifier trained using classical RBM samples. We also explored using a secondary dataset, the MNIST Fashion dataset and demonstrate the first quantum-generated fashion. Our hybrid autoencoder approach indicates advantage for RBM results relative to the use of a current RBM classical computer implementation for image-based machine learning and even more promising results for the next generation D-Wave quantum system. Our method for compression and image mappings is not constrained to RBMs, the autoencoder part of this method could be coupled with other quantum-based algorithms.
We present the application of Restricted Boltzmann Machines (RBMs) to the task of astronomical image classification using a quantum annealer built by D-Wave Systems. Morphological analysis of galaxies provides critical information for studying their formation and evolution across cosmic time scales. We compress galaxy images using principal component analysis to fit a representation on the quantum hardware. Then, we train RBMs with discriminative and generative algorithms, including contrastive divergence and hybrid generative-discriminative approaches, to classify different galaxy morphologies. The methods we compare include Quantum Annealing (QA), Markov Chain Monte Carlo (MCMC) Gibbs Sampling, and Simulated Annealing (SA) as well as machine learning algorithms like gradient boosted decision trees. We find that RBMs implemented on D-Wave hardware perform well, and that they show some classification performance advantages on small datasets, but they don't offer a broadly strategic advantage for this task. During this exploration, we analyzed the steps required for Boltzmann sampling with the D-Wave 2000Q, including a study of temperature estimation, and examined the impact of qubit noise by comparing and contrasting the original D-Wave 2000Q to the lower-noise version recently made available. While these analyses ultimately had minimal impact on the performance of the RBMs, we include them for reference.
Restricted Boltzmann Machines trained by the classical Contrastive Divergence algorithm were embedded into a next generation D-Wave quantum computer having more than 2000 qubits. The embedding was used to sample missing visible units and labels of incomplete test images. The previously introduced RBM embedding that combined qubits into individual RBM units in order to increase the connectivity between units was used. In comparison to the 1000 qubit machine investigated earlier, the same embedding approach applied to the 2000 qubit D-Wave enabled a connectivity between RBM units sufficient for RBM training on a classical computer with a satisfactory classification error. After a varied number of CD training iterations, the visible units corresponding to a particular incomplete test image were fixed in the D-Wave embedding. The resulting state returned by the D-Wave represents the most energetically favorable combination of the remaining qubits for the given fixed incomplete input image. The classification errors compared favorably to those obtained by the classical MCMC sampling. Main factors influencing the quality of the embedding were investigated. Regardless of the computational cost incentives for using the D-Wave for this operation, the reconstruction and classification errors determined in this way can be used at least as a verification of the validity and quality of the particular embedding of the given RBM architecture. This is essential for successfully employing a QC in much more computationally demanding parts of the RBM training. opportunities for using QC not only for recognition but especially for RBM training, in particular for obtaining a representative sample from the RBM model distribution, are discussed.
A D-Wave quantum annealer (QA) having a 2048 qubit lattice, with no missing qubits and couplings, allowed embedding of a complete graph of a Restricted Boltzmann Machine (RBM). A handwritten digit OptDigits data set having 8x7 pixels of visible units was used to train the RBM using a classical Contrastive Divergence. Embedding of the classically-trained RBM into the D-Wave lattice was used to demonstrate that the QA offers a high-efficiency alternative to the classical Markov Chain Monte Carlo (MCMC) for reconstructing missing labels of the test images as well as a generative model. At any training iteration, the D-Wave-based classification had classification error more than two times lower than MCMC. The main goal of this study was to investigate the quality of the sample from the RBM model distribution and its comparison to a classical MCMC sample. For the OptDigits dataset, the states in the D-Wave sample belonged to about two times more local valleys compared to the MCMC sample. All the lowest-energy (the highest joint probability) local minima in the MCMC sample were also found by the D-Wave. The D-Wave missed many of the higher-energy local valleys, while finding many "new" local valleys consistently missed by the MCMC. It was established that the "new" local valleys that the D-Wave finds are important for the model distribution in terms of the energy of the corresponding local minima, the width of the local valleys, and the height of the escape barrier.