Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2608.16939

SPSA Hyperparameter Tuning for Variational Quantum Natural Language Inference

Nayan D'Souza, Christopher J. Agostino

quant-phcs.LGSubmitted Aug 11, 2026• Announced Aug 12, 2026• 0 citations
arXivHTMLPDF

Abstract

Training variational quantum models requires choosing between parameter-shift gradients, which are exact but cost $O(P)$ forward evaluations, and simultaneous perturbation stochastic approximation (SPSA), which uses only two samples but produces high-variance estimates that can degrade optimisation on small supervised tasks. Whether the cheap gradient is usable depends on the variance that results from different choices of the SPSA perturbation scale, learning rate, and gain-decay schedule. We varied those quantities across a broad grid on a 6-qubit, 60-parameter QNLI classifier and compared the best configurations to parameter-shift AdamW and BuresQNG. AdamW-style SPSA with $c_0=0.01$, $η=0.10$, $γ=0.10$ reached $55\% \pm 11\%$ test accuracy, improving over the default configuration ($49\% \pm 6\%$) but remaining 16-19 percentage points below the parameter-shift baselines because the two-sample SPSA gradient estimate has too much variance for reliable optimisation of 60 parameters in 40 epochs. Classical-gain SPSA and Bures-preconditioned SPSA performed worse, at $51\%$ and $46\%$ respectively. Bures-preconditioning a noisy two-sample SPSA gradient amplifies perturbation noise.

Comments
Accepted submission for poster to QNLP AI 2026

Related papers

An Empirical Comparison of Optimizers for Quantum Machine Learning with SPSA-Based Gradients

Marco Wiedmann, Marc Hölle, Maniraman Periyasamy +5 more

Variational quantum algorithms (VQAs) have attracted a lot of attention from the quantum computing community for the last few years. Their hybrid quantum-classical nature with relatively shallow quantum circuits makes them a promising platform for demonstrating the capabilities of noisy intermediate scale quantum (NISQ) devices. Although the classical machine learning community focuses on gradient-based parameter optimization, finding near-exact gradients for variational quantum circuits (VQCs) with the parameter-shift rule introduces a large sampling overhead. Therefore, gradient-free optimizers have gained popularity in quantum machine learning circles. Among the most promising candidates is the simultaneous perturbation stochastic approximation (SPSA) algorithm, due to its low computational cost and inherent noise resilience. We introduce a novel approach that uses the approximated gradient from SPSA in combination with state-of-the-art gradient-based classical optimizers. We demonstrate numerically that this out-performs both standard SPSA and the parameter-shift rule in terms of convergence rate and absolute error in simple regression tasks. The improvement of our novel approach over SPSA with stochastic gradient descent (SGD) is even amplified when shot- and hardware-noise are taken into account. We also demonstrate that error mitigation does not significantly affect our results.

arXiv:2305.00224•IEEE International Conference on Quantum Computing and Engineering (QCE)•Sep 17, 2023•25 citations

Guided-SPSA: Simultaneous Perturbation Stochastic Approximation Assisted by the Parameter Shift Rule

Maniraman Periyasamy, Axel Plinge, Christopher Mutschler +2 more

The study of variational quantum algorithms (VQCs) has received significant attention from the quantum computing community in recent years. These hybrid algorithms, utilizing both classical and quantum components, are well-suited for noisy intermediate-scale quantum devices. Though estimating exact gradients using the parameter-shift rule to optimize the VQCs is realizable in NISQ devices, they do not scale well for larger problem sizes. The computational complexity, in terms of the number of circuit evaluations required for gradient estimation by the parameter-shift rule, scales linearly with the number of parameters in VQCs. On the other hand, techniques that approximate the gradients of the VQCs, such as the simultaneous perturbation stochastic approximation (SPSA), do not scale with the number of parameters but struggle with instability and often attain suboptimal solutions. In this work, we introduce a novel gradient estimation approach called Guided-SPSA, which meaningfully combines the parameter-shift rule and SPSA-based gradient approximation. The Guided-SPSA results in a 15% to 25% reduction in the number of circuit evaluations required during training for a similar or better optimality of the solution found compared to the parameter-shift rule. The Guided-SPSA outperforms standard SPSA in all scenarios and outperforms the parameter-shift rule in scenarios such as suboptimal initialization of the parameters. We demonstrate numerically the performance of Guided-SPSA on different paradigms of quantum machine learning, such as regression, classification, and reinforcement learning.

arXiv:2404.15751•IEEE International Conference on Quantum Computing and Engineering (QCE)•Sep 15, 2024•10 citations

Performance comparison of optimization methods on variational quantum algorithms

Xavier Bonet-Monroig, Hao Wang, Diederick Vermetten +5 more

Variational quantum algorithms (VQAs) offer a promising path toward using near-term quantum hardware for applications in academic and industrial research. These algorithms aim to find approximate solutions to quantum problems by optimizing a parametrized quantum circuit using a classical optimization algorithm. A successful VQA requires fast and reliable classical optimization algorithms. Understanding and optimizing how off-the-shelf optimization methods perform in this context is important for the future of the field. In this work, we study the performance of four commonly used gradient-free optimization methods [sequential least-squares quadratic programming, constrained optimization by linear approximations, the covariance matrix adaptation evolutionary strategy (CMA-ES), and the simultaneous perturbation stochastic approximation (SPSA)] to find ground-state energies of a range of small chemistry and material science problems. We test a telescoping sampling scheme (where the accuracy of the cost-function estimate provided to the optimizer is increased as the optimization converges) for all methods, demonstrating mixed results across our range of optimizers and problems chosen. We further hyperparameter tune two of the four optimizers (CMA-ES and SPSA) across a large range of models and demonstrate that with appropriate hyperparameter tuning, CMA-ES is competitive with and sometimes outperforms SPSA (which is not observed in the absence of hyperparameter tuning). Finally, we investigate the ability of an optimizer to beat the ``sampling-noise floor'' given by the sampling noise of each cost-function estimate provided to the optimizer. Our results demonstrate the necessity for tailoring and hyperparameter tuning known optimization techniques for inherently noisy variational quantum algorithms and that the variational landscape that one finds in a VQA is highly problem and system dependent. This provides guidance for future implementations of these algorithms in experiments.

arXiv:2111.13454•Physical Review A•Mar 7, 2023•114 citations

A cross-domain empirical benchmark of quantum-inspired and classical optimization algorithms for machine learning

Görkem Yılmaz

Abstract Quantum-inspired (QI) optimizers are increasingly proposed as alternatives to classical gradient-based training, yet are frequently benchmarked through classical stand-ins rather than themselves. Across tabular, vision, and language tasks we evaluate three classes: direct implementations of published update rules (Spall’s SPSA; quantum-behaved particle swarm optimization, QPSO, adapted to stochastic minibatches), a documented classical analog of quantum natural gradient (an NGD-style diagonal second-moment preconditioner), and the hyperparameter-control configurations under which such methods are often reported. The gap between these classes is our central result: on the tabular benchmark the hyperparameter control trains with the classical baselines while the direct algorithms fall $$\approx $$ 10–24 points below it, and on CIFAR-10/ResNet18, where v1’s “SPSA” arm was a learning-rate-scaled Adam, Adam itself stands $$\approx $$ 59 points above genuine SPSA. The QPSO adaptation stays at chance on LSTM tasks a legacy “QPSO” proxy solves at 79–81%, at 18–38 $$\times $$ Adam’s cost there and up to 74 $$\times $$ on the small CV model. A seed-paired $$2\times 2$$ factorial on RoBERTa identifies the smaller learning rate as the dominant observed component of the earlier “QNG advantage” ( $$+0.50$$ points at the dataset level); with only three datasets, modified Hartung–Knapp uncertainty spans zero. The configured NGD arm shows no detectable advantage over a learning-rate-matched AdamW (descriptive pooled $$p{=}0.23$$ ) at 6– $$7\%$$ extra cost, and on ResNet18 its trained endpoints have larger dominant-curvature estimates than the Adam family’s. Headline comparisons use five seeds, paired tests, Holm correction, and stated detection limits. We conclude with an auditing taxonomy (direct algorithm / analog / hyperparameter control): within the arms, protocols, and scales evaluated, no advantage attributable to a QI mechanism is detected after these controls.

Quantum Machine Intelligence•Sep 11, 2026•0 citations

Adaptive directional gradients for parameterised quantum circuits

Brian Coyle, Snehal Raj, Virag Umathe +2 more

Training parameterised quantum circuits (PQCs) on quantum hardware is bottlenecked by the measurement cost of gradient estimation, which under the parameter-shift rule scales linearly in the number of trainable parameters and dominates the total shot budget of training at scale. In this work, we propose a framework of forward gradient estimators for PQCs, based on the forward mode of automatic differentiation, that yields an unbiased estimator of the gradient by averaging a freely tunable number of random directional derivatives and recovers SPSA, random coordinate descent, and the parameter-shift rule as limiting cases, with no ancilla qubits or controlled-gate overhead. We prove that stochastic quantum forward gradient descent converges under standard assumptions, with an explicit second-moment expansion that interpolates between the single-direction extreme of SPSA and the full-gradient extreme of parameter-shift. Within this framework we derive QUIVER (Quantum Iterative V-adaptive Estimator Rule), an adaptive optimiser for parameterised circuits whose update rule follows from a closed-form minimum measurement-cost allocation. We show numerically that forward gradients train Hamming-weight-preserving orthogonal quantum neural networks with up to 60 qubits and 1770 parameters on the ECG5000 and MNIST datasets orders of magnitude more efficiently than the parameter-shift rule. We also demonstrate that our proposed QUIVER optimiser can outperform iCANS and gCANS measurement-frugal optimisers on optimisation problems using the quantum approximate optimisation algorithm and quantum simulation with the variational quantum eigensolver.

quant-pharXiv:2606.09734•Jun 9, 2026•0 citations

Gradient Estimation with Constant Scaling for Hybrid Quantum Machine Learning

Thomas Hoffmann, Douglas Brown

We present a novel method for determining gradients of parameterised quantum circuits (PQCs) in hybrid quantum-classical machine learning models by applying the multivariate version of the simultaneous perturbation stochastic approximation (SPSA) algorithm. The gradients of PQC layers can be calculated with an overhead of two evaluations per circuit per forward-pass independent of the number of circuit parameters, compared to the linear scaling of the parameter shift rule. These are then used in the backpropagation algorithm by applying the chain rule. We compare our method to the parameter shift rule for different circuit widths and batch sizes, and for a range of learning rates. We find that, as the number of qubits increases, our method converges significantly faster than the parameter shift rule and to a comparable accuracy, even when considering the optimal learning rate for each method.

arXiv:2211.13981•Nov 25, 2022•2 citations
Training variational quantum models requires choosing between parameter-shift gradients, which are exact but cost O(P)O(P)O(P) forward evaluations, and simultaneous perturbation stochastic approximation (SPSA), which uses only two samples but produces high-variance estimates that can degrade optimisation on small supervised tasks. Whether the cheap gradient is usable depends on the variance that results from different choices of the SPSA perturbation scale, learning rate, and gain-decay schedule. We varied those quantities across a broad grid on a 6-qubit, 60-parameter QNLI classifier and compared the best configurations to parameter-shift AdamW and BuresQNG. AdamW-style SPSA with c0=0.01c_0=0.01c0​=0.01, η=0.10η=0.10η=0.10, γ=0.10γ=0.10γ=0.10 reached 55%±11%55\% \pm 11\%55%±11% test accuracy, improving over the default configuration (49%±6%49\% \pm 6\%49%±6%) but remaining 16-19 percentage points below the parameter-shift baselines because the two-sample SPSA gradient estimate has too much variance for reliable optimisation of 60 parameters in 40 epochs. Classical-gain SPSA and Bures-preconditioned SPSA performed worse, at 51%51\%51% and 46%46\%46% respectively. Bures-preconditioning a noisy two-sample SPSA gradient amplifies perturbation noise.
Variational quantum algorithms (VQAs) have attracted a lot of attention from the quantum computing community for the last few years. Their hybrid quantum-classical nature with relatively shallow quantum circuits makes them a promising platform for demonstrating the capabilities of noisy intermediate scale quantum (NISQ) devices. Although the classical machine learning community focuses on gradient-based parameter optimization, finding near-exact gradients for variational quantum circuits (VQCs) with the parameter-shift rule introduces a large sampling overhead. Therefore, gradient-free optimizers have gained popularity in quantum machine learning circles. Among the most promising candidates is the simultaneous perturbation stochastic approximation (SPSA) algorithm, due to its low computational cost and inherent noise resilience. We introduce a novel approach that uses the approximated gradient from SPSA in combination with state-of-the-art gradient-based classical optimizers. We demonstrate numerically that this out-performs both standard SPSA and the parameter-shift rule in terms of convergence rate and absolute error in simple regression tasks. The improvement of our novel approach over SPSA with stochastic gradient descent (SGD) is even amplified when shot- and hardware-noise are taken into account. We also demonstrate that error mitigation does not significantly affect our results.
The study of variational quantum algorithms (VQCs) has received significant attention from the quantum computing community in recent years. These hybrid algorithms, utilizing both classical and quantum components, are well-suited for noisy intermediate-scale quantum devices. Though estimating exact gradients using the parameter-shift rule to optimize the VQCs is realizable in NISQ devices, they do not scale well for larger problem sizes. The computational complexity, in terms of the number of circuit evaluations required for gradient estimation by the parameter-shift rule, scales linearly with the number of parameters in VQCs. On the other hand, techniques that approximate the gradients of the VQCs, such as the simultaneous perturbation stochastic approximation (SPSA), do not scale with the number of parameters but struggle with instability and often attain suboptimal solutions. In this work, we introduce a novel gradient estimation approach called Guided-SPSA, which meaningfully combines the parameter-shift rule and SPSA-based gradient approximation. The Guided-SPSA results in a 15% to 25% reduction in the number of circuit evaluations required during training for a similar or better optimality of the solution found compared to the parameter-shift rule. The Guided-SPSA outperforms standard SPSA in all scenarios and outperforms the parameter-shift rule in scenarios such as suboptimal initialization of the parameters. We demonstrate numerically the performance of Guided-SPSA on different paradigms of quantum machine learning, such as regression, classification, and reinforcement learning.
Variational quantum algorithms (VQAs) offer a promising path toward using near-term quantum hardware for applications in academic and industrial research. These algorithms aim to find approximate solutions to quantum problems by optimizing a parametrized quantum circuit using a classical optimization algorithm. A successful VQA requires fast and reliable classical optimization algorithms. Understanding and optimizing how off-the-shelf optimization methods perform in this context is important for the future of the field. In this work, we study the performance of four commonly used gradient-free optimization methods [sequential least-squares quadratic programming, constrained optimization by linear approximations, the covariance matrix adaptation evolutionary strategy (CMA-ES), and the simultaneous perturbation stochastic approximation (SPSA)] to find ground-state energies of a range of small chemistry and material science problems. We test a telescoping sampling scheme (where the accuracy of the cost-function estimate provided to the optimizer is increased as the optimization converges) for all methods, demonstrating mixed results across our range of optimizers and problems chosen. We further hyperparameter tune two of the four optimizers (CMA-ES and SPSA) across a large range of models and demonstrate that with appropriate hyperparameter tuning, CMA-ES is competitive with and sometimes outperforms SPSA (which is not observed in the absence of hyperparameter tuning). Finally, we investigate the ability of an optimizer to beat the ``sampling-noise floor'' given by the sampling noise of each cost-function estimate provided to the optimizer. Our results demonstrate the necessity for tailoring and hyperparameter tuning known optimization techniques for inherently noisy variational quantum algorithms and that the variational landscape that one finds in a VQA is highly problem and system dependent. This provides guidance for future implementations of these algorithms in experiments.
Abstract Quantum-inspired (QI) optimizers are increasingly proposed as alternatives to classical gradient-based training, yet are frequently benchmarked through classical stand-ins rather than themselves. Across tabular, vision, and language tasks we evaluate three classes: direct implementations of published update rules (Spall’s SPSA; quantum-behaved particle swarm optimization, QPSO, adapted to stochastic minibatches), a documented classical analog of quantum natural gradient (an NGD-style diagonal second-moment preconditioner), and the hyperparameter-control configurations under which such methods are often reported. The gap between these classes is our central result: on the tabular benchmark the hyperparameter control trains with the classical baselines while the direct algorithms fall ≈\approx ≈ 10–24 points below it, and on CIFAR-10/ResNet18, where v1’s “SPSA” arm was a learning-rate-scaled Adam, Adam itself stands ≈\approx ≈ 59 points above genuine SPSA. The QPSO adaptation stays at chance on LSTM tasks a legacy “QPSO” proxy solves at 79–81%, at 18–38 ×\times × Adam’s cost there and up to 74 ×\times × on the small CV model. A seed-paired 2×22\times 22×2 factorial on RoBERTa identifies the smaller learning rate as the dominant observed component of the earlier “QNG advantage” ( +0.50+0.50+0.50 points at the dataset level); with only three datasets, modified Hartung–Knapp uncertainty spans zero. The configured NGD arm shows no detectable advantage over a learning-rate-matched AdamW (descriptive pooled p=0.23p{=}0.23p=0.23 ) at 6– 7%7\%7% extra cost, and on ResNet18 its trained endpoints have larger dominant-curvature estimates than the Adam family’s. Headline comparisons use five seeds, paired tests, Holm correction, and stated detection limits. We conclude with an auditing taxonomy (direct algorithm / analog / hyperparameter control): within the arms, protocols, and scales evaluated, no advantage attributable to a QI mechanism is detected after these controls.
Training parameterised quantum circuits (PQCs) on quantum hardware is bottlenecked by the measurement cost of gradient estimation, which under the parameter-shift rule scales linearly in the number of trainable parameters and dominates the total shot budget of training at scale. In this work, we propose a framework of forward gradient estimators for PQCs, based on the forward mode of automatic differentiation, that yields an unbiased estimator of the gradient by averaging a freely tunable number of random directional derivatives and recovers SPSA, random coordinate descent, and the parameter-shift rule as limiting cases, with no ancilla qubits or controlled-gate overhead. We prove that stochastic quantum forward gradient descent converges under standard assumptions, with an explicit second-moment expansion that interpolates between the single-direction extreme of SPSA and the full-gradient extreme of parameter-shift. Within this framework we derive QUIVER (Quantum Iterative V-adaptive Estimator Rule), an adaptive optimiser for parameterised circuits whose update rule follows from a closed-form minimum measurement-cost allocation. We show numerically that forward gradients train Hamming-weight-preserving orthogonal quantum neural networks with up to 60 qubits and 1770 parameters on the ECG5000 and MNIST datasets orders of magnitude more efficiently than the parameter-shift rule. We also demonstrate that our proposed QUIVER optimiser can outperform iCANS and gCANS measurement-frugal optimisers on optimisation problems using the quantum approximate optimisation algorithm and quantum simulation with the variational quantum eigensolver.
We present a novel method for determining gradients of parameterised quantum circuits (PQCs) in hybrid quantum-classical machine learning models by applying the multivariate version of the simultaneous perturbation stochastic approximation (SPSA) algorithm. The gradients of PQC layers can be calculated with an overhead of two evaluations per circuit per forward-pass independent of the number of circuit parameters, compared to the linear scaling of the parameter shift rule. These are then used in the backpropagation algorithm by applying the chain rule. We compare our method to the parameter shift rule for different circuit widths and batch sizes, and for a range of learning rates. We find that, as the number of qubits increases, our method converges significantly faster than the parameter shift rule and to a comparable accuracy, even when considering the optimal learning rate for each method.