Restricted Boltzmann machines are common machine learning models that can utilize quantum annealing devices in their training processes as quantum samplers. While this approach has shown promise as an alternative to classical sampling methods, the limitations of quantum annealing hardware, such as the number of qubits and the lack of connectivity between the qubits, still pose a barrier to wide-scale adoption. We propose the use of multiple software techniques such as dropout method, passive labeling, and parallelization techniques for addressing these hardware limitations. The study found that using these techniques along with quantum sampling showed comparable results to its classical counterparts in certain contexts, while in others the increased complexity of the sampling process hindered the performance of the trained models. This means that further research into the behavior of quantum sampling needs to be done to apply quantum annealing to training tasks of more complicated RBM models.
Restricted Boltzmann machines are common machine learning models that can utilize quantum annealing devices in their training processes as quantum samplers. While this approach has shown promise as an alternative to classical sampling methods, the limitations of quantum annealing hardware, such as the number of qubits and the lack of connectivity between the qubits, still pose a barrier to wide-scale adoption. We propose the use of multiple software techniques such as dropout method, passive labeling, and parallelization techniques for addressing these hardware limitations. The study found that using these techniques along with quantum sampling showed comparable results to its classical counterparts in certain contexts, while in others the increased complexity of the sampling process hindered the performance of the trained models. This means that further research into the behavior of quantum sampling needs to be done to apply quantum annealing to training tasks of more complicated RBM models.
Energy-based generative models, such as restricted Boltzmann machines (RBMs), require unbiased Boltzmann samples for effective training. Classical Markov chain Monte Carlo methods, however, converge slowly and yield correlated samples, making large-scale training difficult. We address this bottleneck by applying the analytic relation between annealing schedules and effective inverse temperature in diabatic quantum annealing. By implementing this prescription on a quantum annealer, we obtain temperature-controlled Boltzmann samples that enable RBM training with faster convergence and lower validation error than classical sampling. We further identify a systematic temperature misalignment intrinsic to analog quantum computers and propose an analytical rescaling method that mitigates this hardware noise, thereby enhancing the practicality of quantum annealers as Boltzmann samplers. In our method, the model's connectivity is set directly by the qubit connectivity, transforming the computational complexity inherent in classical sampling into a requirement on quantum hardware. This shift allows the approach to extend naturally from RBMs to fully connected Boltzmann machines, opening opportunities inaccessible to classical training methods.
We propose a scheme to calibrate the internal parameters of a quantum annealer to obtain well-approximated samples for training a restricted Boltzmann machine (RBM). Empirically, samples from quantum annealers obey the Boltzmann distribution, making them suitable for RBM training. Quantum annealers utilize physical phenomena to generate a large number of samples in a short time. However, hardware imperfections make it challenging to obtain accurate samples. Existing research often estimates the inverse temperature for the compensation. Our scheme efficiently utilizes samples for RBM training also to estimate internal parameters. Furthermore, we consider additional parameters and demonstrate that they improve sample quality. We evaluate our approach by comparing the obtained samples with classical Gibbs sampling, which theoretically generates accurate samples. Our results indicate that our scheme demonstrates performance on par with Gibbs sampling. In addition, the training results with our estimation scheme outperform those of the contrastive divergence algorithm, a standard training algorithm for RBM.
Quantum-assisted training of machine learning models makes use of the sampling capabilities of quantum computing devices to approximate the probability distribution of classical generative machine learning models. One example of this is the use of quantum annealers to sample from a Boltzmann distribution to assist the computation of gradients required to train Boltzmann machines (BMs). Doing so requires the interactions and weights of a BM to be mapped onto the underlying qubit connectivity built into the quantum device hardware. This opens the possibility of formulating any graph topology as a BM, instead of the more common classical approach of restricting their connectivity; i.e., Restricted Boltz-mann Machine (RBM). This work is focused on simultaneous modifications of the connectivity and qubit mapping of general BMs, which we refer to as structure-aware minor-embedding. We show how minor-embedding can be performed taking into account the nature of the BM. We also evaluate weight “pruning”, which is motivated by the intuition that, when a QAML model is pruned, the resulting qubit chains resulting from the minor-embedding methods can provide better quality samples, at the expense of a smaller model.
Restricted Boltzmann Machine (RBM) is an energy-based, undirected graphical model. It is commonly used for unsupervised and supervised machine learning. Typically, RBM is trained using contrastive divergence (CD). However, training with CD is slow and does not estimate the exact gradient of the log-likelihood cost function. In this work, the model expectation of gradient learning for RBM has been calculated using a quantum annealer (D-Wave 2000Q), where obtaining samples is faster than Markov chain Monte Carlo (MCMC) used in CD. Training and classification results of RBM trained using quantum annealing are compared with the CD-based method. The performance of the two approaches is compared with respect to the classification accuracies, image reconstruction, and log-likelihood results. The classification accuracy results indicate comparable performances of the two methods. Image reconstruction and log-likelihood results show improved performance of the CD-based method. It is shown that the samples obtained from quantum annealer can be used to train an RBM on a 64-bit “bars and stripes” dataset with classification performance similar to an RBM trained with CD. Though training based on CD showed improved learning performance, training using a quantum annealer could be useful as it eliminates computationally expensive MCMC steps of CD.
Abstract Modest statistical differences between the sampling performances of the D-Wave quantum annealer (QA) and the classical Markov Chain Monte Carlo (MCMC), when applied to Restricted Boltzmann Machines (RBMs), are explored to explain, and possibly address, the absence of significant and consistent improvements in RBM trainability when the D-Wave sampling was used in previous investigations. A novel hybrid sampling approach, combining the classical and the QA contributions, is investigated as a promising way to benefit from the modest differences between the two sampling methods. No improvements in the RBM training are achieved in this work, thereby suggesting that the differences between the QA-based and MCMC sampling, mainly found in the medium-to-low probability regions of the distribution, which are less important for the quality of the sample, are insufficient to benefit the training. Difficulties in achieving sufficiently high quality of embedding RBMs into the lattice of the newer generation of D-Wave hardware could be further complicating the task. On the other hand, the ability to generate samples of sufficient variety from lower-probability parts of the distribution has a potential to benefit other machine learning applications, such as the mitigation of catastrophic forgetting (CF) during incremental learning. The feasibility of using QA-generated patterns of desirable classes for CF mitigation by the generative replay is demonstrated in this work for the first time. While the efficiency of the CF mitigation using the D-Wave QA was comparable to that of the classical mitigation, both the speed of generating a large number of distinct desirable patterns and the potential for further improvement make this approach promising for a variety of challenging machine learning applications.
Abstract Quantum machine learning promises to revolutionize traditional machine learning by efficiently addressing hard tasks for classical computation. While claims of quantum speed-up have been announced for gate-based quantum computers and photon-based boson samplers, demonstration of an advantage by adiabatic quantum annealers (AQAs) is open. Here we quantify the computational cost and the performance of restricted Boltzmann machines (RBMs), a widely investigated machine learning model, by classical and quantum annealing. Despite the lower computational complexity of the quantum RBM being lost due to physical implementation overheads, a quantum speed-up may arise as a reduction by orders of magnitude of the computational time. By employing real-world cybersecurity datasets, we observe that the negative phase on sufficiently challenging tasks is computed up to 64 times faster by AQAs during the exploitation phase. Therefore, although a quantum speed-up highly depends on the problem’s characteristics, it emerges in existing hardware on real-world data.