Stochastic Gradient Descent (SGD) and its variants underpin modern machine learning by enabling efficient optimization of large-scale models. However, their local search nature limits exploration in complex landscapes. In this paper, we introduce Stochastic Quantum Hamiltonian Descent (SQHD), a quantum optimization algorithm that integrates the computational efficiency of stochastic gradient methods with the global exploration power of quantum dynamics. We propose a Lindbladian dynamics as the quantum analogue of continuous-time SGD. We further propose a discrete-time gate-based algorithm that approximates these dynamics while avoiding direct Lindbladian simulation, enabling practical implementation on near-term quantum devices. We rigorously prove the convergence of SQHD for convex and smooth objectives. Numerical experiments demonstrate that SQHD also exhibits advantages in non-convex optimization. All these results highlight its potential for quantum-enhanced machine learning.
Stochastic Gradient Descent (SGD) and its variants underpin modern machine learning by enabling efficient optimization of large-scale models. However, their local search nature limits exploration in complex landscapes. In this paper, we introduce Stochastic Quantum Hamiltonian Descent (SQHD), a quantum optimization algorithm that integrates the computational efficiency of stochastic gradient methods with the global exploration power of quantum dynamics. We propose a Lindbladian dynamics as the quantum analogue of continuous-time SGD. We further propose a discrete-time gate-based algorithm that approximates these dynamics while avoiding direct Lindbladian simulation, enabling practical implementation on near-term quantum devices. We rigorously prove the convergence of SQHD for convex and smooth objectives. Numerical experiments demonstrate that SQHD also exhibits advantages in non-convex optimization. All these results highlight its potential for quantum-enhanced machine learning.
This paper presents a novel optimization algorithm, Quantum-Inspired Stochastic Gradient Descent (QISGD), designed to accelerate convergence in training machine learning models. QISGD leverages concepts from quantum mechanics, specifically quantum tunneling and superposition, to introduce adaptive noise into gradient estimates. The algorithm dynamically adjusts the noise level based on the training progress, aiming to escape local minima more effectively and navigate the loss landscape with greater precision. Unlike traditional stochastic gradient descent (SGD) which relies solely on random perturbations, QISGD incorporates a nuanced noise strategy informed by quantum principles, offering a potentially more robust and efficient approach to optimization. The core idea is to mimic the probabilistic nature of quantum systems, allowing the algorithm to explore the search space more thoroughly. This research explores the theoretical underpinnings of QISGD and provides a framework for its implementation and evaluation. The algorithm's performance is assessed through simulations, demonstrating its potential to outperform standard SGD in specific scenarios.
Abstract: Quantum-enhanced Stochastic Gradient Descent (Q-SGD) is a novel optimization algorithm that integrates quantum computation to accelerate exploration of the solution space, offering a significant advantage over classical methods, particularly in complex optimization problems. This paper explores the potential of quantum annealing and gate operations to refine the optimization process, leading to faster convergence and improved results. We present a theoretical framework and preliminary experiments demonstrating the efficacy of this approach.
Non-smooth optimization models play a fundamental role in various disciplines, including engineering, science, management, and finance. However, classical algorithms for solving such models often struggle with convergence speed, scalability, and parameter tuning, particularly in high-dimensional and non-convex settings. In this paper, we explore how quantum mechanics can be leveraged to overcome these limitations. Specifically, we investigate the theoretical properties of the Quantum Hamiltonian Descent (QHD) algorithm for non-smooth optimization in both continuous and discrete time. First, we propose continuous-time variants of the general QHD algorithm and establish their global convergence and convergence rate for non-smooth convex and strongly convex problems through a novel Lyapunov function design. Furthermore, we prove the finite-time global convergence of continuous-time QHD for non-smooth non-convex problems under mild conditions (i.e., locally Lipschitz). In addition, we propose discrete-time QHD, a fully digitized implementation of QHD via operator splitting (i.e., product formula). We find that discrete-time QHD exhibits similar convergence properties even with large time steps. Finally, numerical experiments validate our theoretical findings and demonstrate the computational advantages of QHD over classical non-smooth non-convex optimization algorithms.
Stochastic Gradient Descent (SGD) is a cornerstone algorithm in modern machine learning, yet it often encounters challenges in navigating complex, non-convex optimization landscapes. Traditional SGD methods predominantly focus on iteratively refining the gradient estimate, which can lead to premature convergence to local minima or saddle points. This paper introduces a novel approach, Quantum-Inspired Stochastic Gradient Descent with Adaptive Entanglement (QISGD-AE), designed to mitigate these limitations. QISGD-AE draws inspiration from quantum superposition and entanglement, dynamically adjusting the "entanglement" between nodes in the network during the optimization process. This adaptive entanglement facilitates a more comprehensive exploration of the solution space, enhancing the algorithm's ability to escape local optima and converge to globally optimal or near-optimal solutions. The core mechanism involves a stochastic probabilistic adjustment of node relationships, effectively mimicking the parallel exploration inherent in quantum systems. We present a detailed formulation of the algorithm, including key equations and parameters, demonstrating its potential for improved performance in various non-convex optimization tasks. The experimental results, though focusing on conceptual demonstration, highlight the algorithm's capacity to outperform standard SGD in specific scenarios, offering a promising direction for future research in adaptive optimization.
This paper proposes a novel stochastic gradient descent (SGD) algorithm, termed Quantum-Inspired Stochastic Gradient Descent with Entangled Noise (QISGDEN), that draws inspiration from the principles of quantum entanglement to enhance the optimization process. Traditional SGD relies on adding Gaussian noise to the gradient estimates, a technique that can sometimes lead to slow convergence and limited exploration of the loss landscape. QISGDEN addresses this limitation by incorporating correlated noise, modeled after the complex correlations present in entangled quantum states. This approach introduces a more sophisticated exploration strategy, potentially leading to faster convergence rates and improved optimization performance. The algorithm utilizes a modified update rule incorporating a correlation factor that reflects the degree of entanglement, allowing for dynamic adjustment of the noise contribution based on the current state of the optimization. We explore the theoretical underpinnings of the method and present a detailed analysis of its convergence properties. Preliminary results suggest the potential of QISGDEN to outperform standard SGD in certain scenarios, particularly those with complex loss landscapes. The key contribution lies in translating quantum mechanical concepts—specifically entanglement—into a practical optimization framework.
Stochastic Gradient Descent (SGD) is a cornerstone algorithm in deep learning, widely employed for optimizing model parameters. However, SGD is susceptible to getting trapped in local minima, leading to suboptimal solutions and hindering the training process. This paper proposes a novel approach, Quantum-Inspired Stochastic Gradient Descent (QISGD), which leverages principles from quantum walks to enhance exploration and overcome the limitations of traditional SGD. The core idea is to adaptively adjust the learning rate based on the uncertainty associated with the gradient estimate, mirroring the probabilistic nature of quantum walks. Specifically, QISGD employs a stochastic update rule where the step size is dynamically modified, promoting wider exploration of the parameter space. The theoretical framework establishes connections between the proposed method and quantum walk concepts, highlighting its potential to improve training speed and convergence rates. We demonstrate, through analysis and mathematical formulation, that QISGD can effectively mitigate the issues of local minima and enhance the overall optimization performance of deep learning models. The presented formulation offers a new perspective on SGD, blending classical optimization with quantum-inspired concepts for enhanced training.