Quantum Natural Gradient for Variational Bayes

Anna Lopatnikova, Minh‐Ngoc Tran

Published Jun 10, 2021• 1 citation

Abstract

Variational Bayes (VB) is a critical method in machine learning and statistics, underpinning the recent success of Bayesian deep learning. The natural gradient is an essential component of efficient VB estimation, but it is prohibitively computationally expensive in high dimensions. We propose a hybrid quantum-classical algorithm to improve the scaling properties of natural gradient computation and make VB a truly computationally efficient method for Bayesian inference in highdimensional settings. The algorithm leverages matrix inversion from the linear systems algorithm by Harrow, Hassidim, and Lloyd [Phys. Rev. Lett. 103, 15 (2009)] (HHL). We demonstrate that the matrix to be inverted is sparse and the classical-quantum-classical handoffs are sufficiently economical to preserve computational efficiency, making the problem of natural gradient for VB an ideal application of HHL. We prove that, under standard conditions, the VB algorithm with quantum natural gradient is guaranteed to converge. Our regression-based natural gradient formulation is also highly useful for classical VB.

Related papers

Quantum Speedup of Natural Gradient for Variational Bayes

Anna Lopatnikova, Minh‐Ngoc Tran

Variational Bayes (VB) is a critical method in machine learning and statistics, underpinning the recent success of Bayesian deep learning. The natural gradient is an essential component of efficient VB estimation, but it is prohibitively computationally expensive in high dimensions. We propose a computationally efficient regression-based method for natural gradient estimation, with convergence guarantees under standard assumptions. The method enables the use of quantum matrix inversion to further speed up VB. We demonstrate that the problem setup fulfills the conditions required for quantum matrix inversion to deliver computational efficiency. The method works with a broad range of statistical models and does not require special-purpose or simplified variational distributions.

arXiv:2106.05807Jun 10, 20213 citations

Quantum Variational Bayes on Manifolds

Anna Lopatnikova, Minh‐Ngoc Tran

Variational Bayes (VB) has become a widely-used tool for Bayesian inference in statistics and machine learning. VB algorithms are generally restricted to the case where the variational parameter space is Euclidean. Recently, the VB framework has been extended to the case where the variational parameter space is a Riemannian manifold. Such a manifold-based algorithm, by utilizing the natural gradient, can exploit the geometric structures to derive an efficient VB procedure. However, natural gradient in high dimension can be prohibitively computationally expensive to compute. This work proposes using a quantum natural gradient within the manifold VB framework. We demonstrate that the quantum natural gradient does not suffer from the well-known quantum-classical readout computational bottleneck, and hence delivers a highly computational efficiency. The resulting quantum manifold VB algorithm is provably convergent.

IEEE International Conference on Acoustics Speech and Signal ProcessingMay 5, 20232 citations

Quantum extension of variational Bayes inference

Hideyuki Miyahara, Yuki Sughiyama

Variational Bayes (VB) inference is one of the most important algorithms in machine learning and is widely used in engineering and industry. However, VB is known to suffer from the problem of local optima. In this paper, we generalize VB by using quantum mechanics and propose an algorithm, which we call quantum annealing variational Bayes (QAVB) inference. We then show that QAVB drastically improves the performance of VB in a clustering problem described by a Gaussian mixture model, which is essentially important from the viewpoint of optimization. Finally, we discuss an intuitive understanding of how QAVB works well.

arXiv:1712.04709Physical Review AAug 29, 201811 citations

Quantum advantage in variational Bayes inference

Hideyuki Miyahara, Vwani Roychowdhury

Variational Bayes (VB) inference algorithm is used widely to estimate both the parameters and the unobserved hidden variables in generative statistical models. The algorithm—inspired by variational methods used in computational physics—is iterative and can get easily stuck in local minima, even when classical techniques, such as deterministic annealing (DA), are used. We study a VB inference algorithm based on a nontraditional quantum annealing approach—referred to as quantum annealing variational Bayes (QAVB) inference—and show that there is indeed a quantum advantage to QAVB over its classical counterparts. In particular, we show that such better performance is rooted in key quantum mechanics concepts: i) The ground state of the Hamiltonian of a quantum system—defined from the given data—corresponds to an optimal solution for the minimization problem of the variational free energy at very low temperatures; ii) such a ground state can be achieved by a technique paralleling the quantum annealing process; and iii) starting from this ground state, the optimal solution to the VB problem can be achieved by increasing the heat bath temperature to unity, and thereby avoiding local minima introduced by spontaneous symmetry breaking observed in classical physics based VB algorithms. We also show that the update equations of QAVB can be potentially implemented using ⌈log K ⌉ qubits and 𝒪( K ) operations per step, where K is the number of values hidden categorical variables can take. Thus, QAVB can match the time complexity of existing VB algorithms, while delivering higher performance.

arXiv:2207.03104Proceedings of the National Academy of SciencesJul 25, 20235 citations

A Hybrid Quantum Solver for Gaussian Process Regression

Kerem Bükrü, Leger, Steffen, Hickmann, M. Lautaro +3 more

Gaussian processes are widely known for their ability to provide probabilistic predictions in supervised machine learning models. Their non-parametric nature and flexibility make them particularly effective for regression tasks. However, training a Gaussian process model using standard methods requires matrix inversions with a cubic time complexity, which poses significant computational challenges for inference on larger datasets. Quantum algorithms, such as the HHL algorithm, have been proposed as solutions that overcome the need for classical matrix inversions by efficiently solving linear systems of equations using quantum computers. However, to gain a computational advantage over classical algorithms, these algorithms require fault-tolerant quantum computers with a large number of qubits, which are not yet available. The Variational Quantum Linear Solver is a hybrid quantum-classical algorithm that solves linear systems of equations by optimizing the parameters of a variational quantum circuit using a classical computer. This method is especially suitable for noisy intermediate-scale quantum computers, as it does not require many qubits. It can be used to compute the posterior distribution of a Gaussian process by reformulating the matrix inversion into a set of linear systems of equations. We empirically demonstrate that using the Variational Quantum Linear Solver to perform inference for Gaussian process regression delivers regression quality comparable to that of classical methods.

arXiv:2510.15486Oct 17, 20250 citations

Mathematical Foundations of Quantum Linear Regression: A Proof of Concept via Variational Hamiltonian Formulation

Methanon Kaeokrachang

We present a rigorous mathematical proof of concept demonstrating the theoretical feasibility of quantum linear regression through variational Hamiltonian formulation. Our framework systematically transforms classical linear regression into a quantum optimization problem by encoding regression weights as expectation values of Pauli-Z operators and reformulating the loss function as a quantum Hamiltonian. We provide explicit mathematical constructions, matrix representations, and convergence analysis to establish the theoretical foundation for quantum machine learning applications. Unlike previous quantum linear regression approaches that focus on exponential speedups through quantum linear systems solvers, our variational approach is designed for near-term quantum devices and naturally incorporates regularization through quantum parameter constraints. The mathematical framework demonstrates computational equivalence between classical and quantum approaches while revealing inherent advantages in constrained optimization scenarios. This work provides the theoretical groundwork for future empirical applications to financial time series prediction, including potential implementation on Thai SET Index forecasting using data from Yahoo Finance and quantum computing platforms. Our key contributions include: (1) complete mathematical mapping from classical to quantum regression formulation, (2) explicit Hamiltonian constructions with matrix representations, (3) theoretical convergence analysis for quantum gradient descent, and (4) identification of quantumspecific regularization properties that distinguish this approach from classical methods.

Jun 27, 20250 citations