Many machine learning algorithms optimize a loss function with stochastic
gradient descent and use kernel methods to extend linear learning tasks to
non-linear learning tasks. Both ideas have been discussed in the context of
quantum computing, especially for near-term quantum computing with variational
methods and the use of the Hilbert space to encode features of data. In this
work, we discuss a quantum algorithm with provable learning guarantee in the
fault-tolerant quantum computing model. In a well-defined learning model, this
quantum algorithm is able to provide a polynomial speedup for a large range of
parameters of the underlying concept class. We discuss two types of speedups,
one for evaluating the kernel matrix and one for evaluating the gradient in the
stochastic gradient descent procedure. Our work contributes to the study of
quantum learning with kernels and noise.
Many machine learning algorithms optimize a loss function with stochastic
gradient descent and use kernel methods to extend linear learning tasks to
non-linear learning tasks. Both ideas have been discussed in the context of
quantum computing, especially for near-term quantum computing with variational
methods and the use of the Hilbert space to encode features of data. In this
work, we discuss a quantum algorithm with provable learning guarantee in the
fault-tolerant quantum computing model. In a well-defined learning model, this
quantum algorithm is able to provide a polynomial speedup for a large range of
parameters of the underlying concept class. We discuss two types of speedups,
one for evaluating the kernel matrix and one for evaluating the gradient in the
stochastic gradient descent procedure. Our work contributes to the study of
quantum learning with kernels and noise.
At the interface of machine learning and quantum computing, an important question is what distributions can be learned provably with optimal sample complexities and with quantum-accelerated time complexities. In the classical case, Klivans and Goel discussed the Alphatron , an algorithm to learn distributions related to kernelized regression, which they also applied to the learning of two-layer neural networks. In this work, we provide quantum versions of the Alphatron in the fault-tolerant setting. In a well-defined learning model, this quantum algorithm is able to provide a polynomial speedup for a large range of parameters of the underlying concept class. We discuss two types of speedups, one for evaluating the kernel matrix and one for evaluating the gradient in the stochastic gradient descent procedure. We also discuss the quantum advantage in the context of learning of two-layer neural networks. Our work contributes to the study of quantum learning with kernels and from samples.
Large machine learning models are revolutionary technologies of artificial intelligence whose bottlenecks include huge computational expenses, power, and time used both in the pre-training and fine-tuning process. In this work, we show that fault-tolerant quantum computing could possibly provide provably efficient resolutions for generic (stochastic) gradient descent algorithms, scaling as [Formula: see text], where n is the size of the models and T is the number of iterations in the training, as long as the models are both sufficiently dissipative and sparse, with small learning rates. Based on earlier efficient quantum algorithms for dissipative differential equations, we find and prove that similar algorithms work for (stochastic) gradient descent, the primary algorithm for machine learning. In practice, we benchmark instances of large machine learning models from 7 million to 103 million parameters. We find that, in the context of sparse training, a quantum enhancement is possible at the early stage of learning after model pruning, motivating a sparse parameter download and re-upload scheme. Our work shows solidly that fault-tolerant quantum algorithms could potentially contribute to most state-of-the-art, large-scale machine-learning problems.
Large machine learning models are revolutionary technologies of artificial intelligence whose bottlenecks include huge computational expenses, power, and time used both in the pre-training and fine-tuning process. In this work, we show that fault-tolerant quantum computing could possibly provide provably efficient resolutions for generic (stochastic) gradient descent algorithms, scaling as O(T^2 polylog(n)), where n is the size of the models and T is the number of iterations in the training, as long as the models are both sufficiently dissipative and sparse, with small learning rates. Based on earlier efficient quantum algorithms for dissipative differential equations, we find and prove that similar algorithms work for (stochastic) gradient descent, the primary algorithm for machine learning. In practice, we benchmark instances of large machine learning models from 7 million to 103 million parameters. We find that, in the context of sparse training, a quantum enhancement is possible at the early stage of learning after model pruning, motivating a sparse parameter download and re-upload scheme. Our work shows solidly that fault-tolerant quantum algorithms could potentially contribute to most state-of-the-art, large-scale machine-learning problems.
With near-term quantum devices available and the race for fault-tolerant quantum computers in full swing, researchers became interested in the question of what happens if we replace a supervised machine learning model with a quantum circuit. While such "quantum models" are sometimes called "quantum neural networks", it has been repeatedly noted that their mathematical structure is actually much more closely related to kernel methods: they analyse data in high-dimensional Hilbert spaces to which we only have access through inner products revealed by measurements. This technical manuscript summarises and extends the idea of systematically rephrasing supervised quantum models as a kernel method. With this, a lot of near-term and fault-tolerant quantum models can be replaced by a general support vector machine whose kernel computes distances between data-encoding quantum states. Kernel-based training is then guaranteed to find better or equally good quantum models than variational circuit training. Overall, the kernel perspective of quantum machine learning tells us that the way that data is encoded into quantum states is the main ingredient that can potentially set quantum models apart from classical machine learning models.
With near-term devices available and the race for fault-tolerant computers in full swing, researchers became interested in the question of what happens if we replace a machine learning model with a circuit. While such quantum are sometimes called quantum neural networks, it has been repeatedly noted that their mathematical structure is actually much more closely related to kernel methods: they analyse data in high-dimensional Hilbert spaces to which we only have access through inner products revealed by measurements. This technical manuscript summarises, formalises and extends the link by systematically rephrasing models as a kernel method. It shows that most near-term and fault-tolerant models can be replaced by a general support vector machine whose kernel computes distances between data-encoding states. In particular, kernel-based training is guaranteed to find better or equally good models than variational circuit training. Overall, the kernel perspective of machine learning tells us that the way that data is encoded into states is the main ingredient that can potentially set models apart from classical machine learning models.
Quantum support vector machines have the potential to achieve a quantum speedup for solving certain machine learning problems. The key challenge for doing so is finding good quantum kernels for a given data set - a task called kernel alignment. In this paper we study this problem using the Pegasos algorithm, which is an algorithm that uses stochastic gradient descent to solve the support vector machine optimization problem. We extend Pegasos to the quantum case and and demonstrate its effectiveness for kernel alignment. Unlike previous work which performs kernel alignment by training a QSVM within an outer optimization loop, we show that using Pegasos it is possible to simultaneously train the support vector machine and align the kernel. Our experiments show that this approach is capable of aligning quantum feature maps with high accuracy, and outperforms existing quantum kernel alignment techniques. Specifically, we demonstrate that Pegasos is particularly effective for non-stationary data, which is an important challenge in real-world applications.