We used a quantum annealing D-Wave 2X computer to obtain solutions to NP-hard sparse coding problems for inferring representation of reduced dimensional MNIST images.For comparison, we implemented two deep neural network architectures.The first (AlexNet-like) approximately matched the architecture of the sparse coding model.The second was state-of-the-art (RESNET).Classification based on the D-Wave 2X was superior to matching pursuit and AlexNet and nearly equivalent to RESNET.
We used a quantum annealing D-Wave 2X computer to obtain solutions to NP-hard sparse coding problems for inferring representation of reduced dimensional MNIST images.For comparison, we implemented two deep neural network architectures.The first (AlexNet-like) approximately matched the architecture of the sparse coding model.The second was state-of-the-art (RESNET).Classification based on the D-Wave 2X was superior to matching pursuit and AlexNet and nearly equivalent to RESNET.
We use a quantum annealing D-Wave 2X computer to obtain solutions to NP-hard sparse coding problems. To reduce the dimensionality of the sparse coding problem to fit on the quantum D-Wave 2X hardware, we passed downsampled MNIST images through a bottleneck autoencoder. To establish a benchmark for classification performance on this reduced dimensional data set, we built two deep convolutional neural networks (DCNNs). The first DCNN used an AlexNet-like architecture and the second a state-of-the-art residual network (RESNET)model, both implemented in TensorFlow. The two DCNNs yielded classification scores of 94.54 ± 0.7% and 98.8 ± 0.1%, respectively. As a control, we showed that both DCNN architectures produced near-state-of-the-art classification performance ( ~ 99%) on the original MNIST images. To obtain a set of optimized features for inferring sparse representations of the reduced dimensional MNIST dataset, we imprinted on a random set of 47 image patches followed by an off-line unsupervised learning algorithm using stochastic gradient descent to optimize for sparse coding. Our single-layer of sparse coding matched the stride and patch size of the first convolutional layer of the AlexNet-like DCNN and contained 47 fully-connected features, 47 being the maximum number of dictionary elements that could be embedded onto the D-Wave 2X hardware. When the sparse representations inferred by the D-Wave 2X were passed to a linear support vector machine, we obtained a classification score of 95.68%. We found that the classification performance supported by quantum inference was maximal at an optimal level of sparsity corresponding to a critical value of the sparsity/reconstruction error trade-off parameter that previous work has associated with a second order phase transition, an observation supported by a free energy analysis of D-Wave energy states. We mimicked a transfer learning protocol by feeding the D-Wave representations into a multilayer perceptron (MLP), yielding 98.48% classification performance. The classification performance supported by a single-layer of quantum inference was superior to that supported by a classical matching pursuit algorithm set to the same level of sparsity. Whereas the classification performance of both DCNNs declined as the number of training examples was reduced, the classification performance supported by quantum inference was insensitive to the number of training examples. We thus conclude that quantum inference supports classification of reduced dimensional MNIST images exceeding that of a size-matched AlexNet-like DCNN and nearly equivalent to a state-of-the-art RESNET DCNN.
We use a quantum annealing D-Wave 2X (1,152-qubit) computer to generate sparse representations of Canny-filtered, center-cropped 30×30 CIFAR-10 images. Each binary neuron (qubit) represents a feature kernel obtained initially by imprinting on a randomly chosen 5×5 image patch and then adapted via an off-line Hebbian learning protocol using the sparse solutions generated by the D-Wave. When using binary neurons, the energy function is non-convex (multiple local-minima) and finding a global minimum is NP-hard. Quantum annealing provides a strategy for finding sparse representations that correspond to good local minima of a non-convex cost function. To overcome the severe coupling restrictions between physical qubits on the D-Wave Chimera graph, we use embedding tools to achieve approximately all-to-all connectivity across a reduced number of logical qubits. We assess the sparse representations generated by the D-Wave using both the total energy as well as classification accuracy on a subset of the CIFAR-10 database. The D-Wave 2X outperforms two classical state-of-the-art binary solvers, GUROBI and Chimera-inspired algorithm Hamze-Freitas-Selby (HFS). Specifically, the D-Wave 2X yields lower energy sparse solutions within seconds while the largest problems take over 10 hours for both GUROBI and HFS. We obtained cross-validation classification of 31.02% for the first 4K images using 47 features on the quantum D- Wave 2X.
Just as the brain must infer 3D structure from 2D retinal images, radiologists are tasked with inferring 3D densities from 2D X-rays. Computer simulations suggest that V1 simple cells use lateral inhibition to generate sparse representations that are selective for 3D depth when presented with 2D stereo images and video. Analogously, we cast radiographic inference as a sparse coding problem employing lateral inhibition between binary neurons, resulting in a quadratic unconstrained binary optimization (QUBO)problem suitable for implementation on a quantum annealing D-Wave 2X (1152-qubit)computer. We generated synthetic radiographs by performing discrete Abel transforms on mathematically-defined objects possessing axial (cylindrical)symmetry and whose radially density profile was given by the sum of a randomly-chosen, sparse set of (nearly binary)Fourier components. We used embedding tools to map the above QUBO problem, which involved dense connections between up to 47 Fourier coefficients, onto the very sparsely connected D-Wave chimera. Using quantum inference, we were able to reconstruct reasonably accurate radial density profiles even after adding sufficiently noise to our synthetic radiographs to make inverse Abel transforms untenable. Compared to state-of-the-art classical QUBO solvers, GUROBI and the Hamze-Freitas-Selby algorithm, the quantum D-Wave 2X was orders of magnitude faster for the same final accuracy. Our results indicate a potential strategy for integrating neuromorphic and quantum computing techniques.
Understanding how the D-Wave quantum computer could be used for machine learning problems is of growing interest. Our work explores the feasibility of using the D-Wave as a sampler for a machine learning task. We describe a hybrid method that combines a classical deep neural network autoencoder with a quantum annealing Restricted Boltzmann Machine (RBM) using the D-Wave for image generation. Our method overcomes two key limitations in the 2000-qubit D-Wave processor, namely the limited number of qubits available to accommodate typical problem sizes for fully connected quantum objective functions, and samples that are binary pixel representations. As a consequence of these limitations we are able to show how we achieved nearly a 22-fold compression factor of grayscale 28 x 28 sized images to binary 6 x 6 sized images with a lossy recovery of the original 28 x 28 grayscale images. We further show how generating samples from the D-Wave after training the RBM, resulted in 28 x 28 images that were variations of the original input data distribution, as opposed to recreating the training samples. We evaluated the quality of this method by using a downstream classification method. We formulated a MNIST classification problem using a deep convolutional neural network that used samples from the quantum RBM to train the MNIST classifier and compared the results with a MNIST classifier trained with the original MNIST training data set, as well as a MNIST classifier trained using classical RBM samples. We also explored using a secondary dataset, the MNIST Fashion dataset and demonstrate the first quantum-generated fashion. Our hybrid autoencoder approach indicates advantage for RBM results relative to the use of a current RBM classical computer implementation for image-based machine learning and even more promising results for the next generation D-Wave quantum system. Our method for compression and image mappings is not constrained to RBMs, the autoencoder part of this method could be coupled with other quantum-based algorithms.
Although machine learning currently relies on conventional computer architectures, the looming end of Moore's Law necessitates exploration of novel computational platforms. Neuromorphic and quantum systems are a natural path to pursue; biological neurons are incredibly efficient, and quantum mechanics provides theoretical foundations for fast solutions to optimization problems. Here, we make the first comparison of emerging hardware (D-Wave quantum annealer and Intel Loihi spiking processor) on an identically-posed machine learning problem. We implement the bioinspired Locally Competitive Algorithm (LCA) for solving sparse coding on the different substrates. To make the comparison valid, our dataset of choice (Fashion MNIST) is dimensionally-reduced via sparse principal component analysis, under the constraint that both classification performance and a graph-based clustering metric remain unchanged. This enables the problem to be mapped identically to both devices. An analysis of several metrics, including power consumption, reconstruction, and classification accuracy are presented. When given the same specifically-constructed challenge, both substrates perform similarly. Our results suggest while neuromorphic and quantum systems are still in their infancy, they present a possible route to address certain types of classically challenging problems, such as sparse coding, in a way that leverages the unique aspects of the substrates.
Given the success of deep learning in classical machine learning, quantum algorithms for traditional neural network architectures may provide one of the most promising settings for quantum machine learning. Considering a fully-connected feedforward neural network, we show that conditions amenable to classical trainability via gradient descent coincide with those necessary for efficiently solving quantum linear systems. We propose a quantum algorithm to approximately train a wide and deep neural network up to O(1/n) error for a training set of size n by performing sparse matrix inversion in O(logn) time. To achieve an end-to-end exponential speedup over gradient descent, the data distribution must permit efficient state preparation and readout. We numerically demonstrate that the MNIST image dataset satisfies such conditions; moreover, the quantum algorithm matches the accuracy of the fully-connected network. Beyond the proven architecture, we provide empirical evidence for O(logn) training of a convolutional neural network with pooling.