arXiv:2304.06768 · Physical Review Research

Improving gradient methods via coordinate transformations: Applications to quantum machine learning

Pablo Bermejo, Borja Aizpurua, Román Orús

Published Apr 19, 2024• 13 citations

Abstract

In this paper, we introduce a generic strategy to accelerate and improve the overall performance of machine-learning algorithms, both in their classical and quantum versions, heavily rely on optimization algorithms based on gradients, such as gradient descent. The overall performance is dependent on the appearance of local minima and barren plateaus, which slow down calculations and lead to nonoptimal solutions. In practice, this results in dramatic computational and energy costs for artificial intelligence applications. Our method is based on coordinate transformations, like variational rotations, adding extra directions in parameter space that depend on the cost function itself, and which allowus to explore the configuration landscape more efficiently. The validity of our method is benchmarked by boosting several quantum machine-learning algorithms, getting a very significant improvement in their performance. Published by the American Physical Society 2024

DOI: 10.1103/physrevresearch.6.023069

Related papers

Random coordinate descent: A simple alternative for optimizing parameterized quantum circuits

Zhiyan Ding, Taehee Ko, Jiahao Yao +2 more

Variational quantum algorithms rely on the optimization of parameterized quantum circuits in noisy settings. The commonly used back-propagation procedure in classical machine learning is not directly applicable in this setting due to the collapse of quantum states after measurements. Thus, gradient estimations constitute a significant overhead in a gradient-based optimization of such quantum circuits. This paper introduces a random coordinate descent algorithm as a practical and easy-to-implement alternative to the full gradient descent algorithm. This algorithm only requires one partial derivative at each iteration. Motivated by the behavior of measurement noise in the practical optimization of parameterized quantum circuits, this paper presents an optimization problem setting that is amenable to analysis. Under this setting, the random coordinate descent algorithm exhibits the same level of stochastic stability as the full gradient approach, making it as resilient to noise. The complexity of the random coordinate descent method is generally no worse than that of the gradient descent and can be much better for various quantum optimization problems with anisotropic Lipschitz constants. Theoretical analysis and extensive numerical experiments validate our findings. Published by the American Physical Society 2024

arXiv:2311.00088Physical Review ResearchJul 8, 20249 citations

Quantum-Inspired Optimization of Large-Scale Machine Learning Models

Jincheng Zhang

The training of large-scale machine learning models presents significant computational challenges, primarily due to the inherent complexity of optimization algorithms like stochastic gradient descent (SGD) and its variants. These algorithms often struggle with convergence speed, getting trapped in local minima, and the ability to effectively explore the vast solution space. This paper investigates the potential of quantum-inspired optimization techniques to address these limitations. Specifically, we explore the application of quantum annealing and variational quantum eigensolver (VQE) approaches to accelerate the training and optimization process of large-scale machine learning models. Our core claim is that by mapping the optimization problem to a quantum system, we can leverage quantum computation to overcome the computational bottlenecks associated with classical optimization methods. We propose a framework where the objective function of the machine learning model is translated into a Hamiltonian for a quantum annealer or the cost function for a VQE, enabling a quantum-inspired solution. The results, while theoretical in this initial exploration, demonstrate a potential pathway for significantly reducing training times and improving the overall performance of large-scale machine learning models. Further research is needed to develop efficient hybrid quantum-classical algorithms and explore the practical limitations of this approach.

Sep 6, 20260 citations

Quantum-Inspired Optimization for Large-Scale Machine Learning

Jincheng Zhang

The rapid growth of machine learning (ML) has been accompanied by a significant challenge: the scaling of optimization algorithms to handle the immense parameter spaces of modern models, such as deep neural networks. Traditional optimization methods, including gradient descent and its variants, often encounter difficulties in efficiently exploring these spaces, leading to slow training times and potential stagnation. This paper investigates the application of quantum-inspired optimization techniques to address this scaling problem. We explore algorithms inspired by quantum annealing and other quantum phenomena, adapting their core principles to classical computation. Specifically, we propose a novel optimization framework utilizing a stochastic search strategy based on quantum superposition and measurement, designed to navigate high-dimensional parameter spaces more effectively. The efficacy of this approach is evaluated through theoretical analysis and conceptual simulations, demonstrating its potential to accelerate the training of large-scale machine learning models. We demonstrate how these methods can improve convergence speed and potentially escape local minima more effectively than conventional techniques. The core contribution lies in providing a theoretical foundation and a practical framework for applying quantum-inspired concepts to optimization within the context of machine learning.

Sep 6, 20260 citations

Provably Trainable Rotationally Equivariant Quantum Machine Learning

Maxwell T. West, Jamie Heredge, Martin E. Sevior +1 more

Exploiting the power of quantum computation to realize superior machine learning algorithms has been a major research focus of recent years, but the prospects of quantum machine learning (QML) remain dampened by considerable technical challenges. A particularly significant issue is that generic QML models suffer from so-called barren plateaus in their training landscapes—large regions where cost function gradients vanish exponentially in the number of qubits employed, rendering large models effectively untrainable. A leading strategy for combating this effect is to build problem-specific models that take into account the symmetries of their data in order to focus on a smaller, relevant subset of Hilbert space. In this work, we introduce a family of rotationally equivariant QML models built upon the quantum Fourier transform, and leverage recent insights from the Lie-algebraic study of QML models to prove that (a subset of) our models do not exhibit barren plateaus. In addition to our analytical results we numerically test our rotationally equivariant models on a dataset of simulated scanning tunneling microscope images of phosphorus impurities in silicon, where rotational symmetry naturally arises, and find that they dramatically outperform their generic counterparts in practice. Published by the American Physical Society 2024

arXiv:2311.05873PRX QuantumJul 31, 202437 citations

Quantum-Enhanced Gradient Descent for Optimization Algorithms

Jincheng Zhang

Quantum-enhanced gradient descent represents a novel approach to optimization algorithms by integrating quantum mechanics principles to accelerate convergence and enhance stability. This paper explores the implementation of a modified gradient descent method that employs quantum superposition to explore multiple solution paths concurrently, thereby reducing computational cost and improving the robustness of optimization processes. We detail the theoretical framework underpinning this approach, outlining the quantum properties leveraged and the specific algorithm architecture designed to maximize this benefit. The paper presents a detailed analysis of the performance improvements achieved through this implementation, demonstrating a significant acceleration in convergence and a demonstrable improvement in stability compared to traditional gradient descent techniques. The research focuses on the potential for harnessing quantum phenomena to overcome limitations inherent in classical optimization, offering a promising avenue for advancing machine learning and optimization workflows.

Sep 2, 20260 citations

Quantum Machine Learning: Adaptive Gradient Descent with Quantum Entanglement and Superposition

Jincheng Zhang

Quantum Machine Learning aims to leverage the unique properties of quantum mechanics to revolutionize artificial intelligence. This paper introduces a novel adaptive gradient descent algorithm incorporating quantum entanglement and superposition, promising significant acceleration in training complex models and improved accuracy. We propose a mechanism that exploits these quantum phenomena to enhance the gradient computation process, ultimately enabling faster and more effective learning. The core claim centers on the potential for quantum-enhanced gradient descent, leading to breakthroughs in machine learning applications. This work explores the theoretical underpinnings and potential practical implications of this innovative approach.

Sep 3, 20260 citations