Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2609.16266

Towards Surrogate Based Dequantization of Quantum Reinforcement Learning

Pablo Rodriguez-Grasa, Sofiene Jerbi, Mikel Sanz, Ryan Sweke

quant-phcs.LGSubmitted Sep 14, 2026• Announced Sep 16, 2026• 0 citations
arXivHTMLPDF

Abstract

In recent years, the utility of parameterized quantum circuits as function approximators has been widely studied. In the context of reinforcement learning, this approach has led to variational quantum algorithms such as quantum Q-learning. While these methods show promising empirical results, and can provide provable advantages for artificial problems, it remains unclear whether they can provide a provable quantum advantage over classical approaches for problems of practical relevance. A natural way to investigate this question is through the lens of dequantization: The construction of efficient classical algorithms capable of matching the performance of quantum variational methods. Building on recent kernel-based dequantization results for supervised learning, we take steps towards extending this surrogate-based dequantization program to reinforcement learning. Specifically, we study the simplified setting of reinforcement learning with a uniform generative model in which uniformly random state-action samples are available, which models the regime of sampling from a large experience replay buffer after sufficient exploration. Within this setting, we provide finite sample guarantees for classical kernelized Fitted Q-Iteration, with classical kernels designed to match the inductive bias of particular parameterized quantum circuits. Using these results, we then provide a set of sufficient conditions, on the data-encoding strategy of a parameterized quantum circuit, the corresponding classical kernel, and the problem structure, under which kernelized Fitted Q-Iteration provides a meaningful dequantization of quantum Q-learning, in this simplified setting. Apart from providing rigorous dequantization guarantees when these conditions are met, these results also motivate the use of kernelized fitted Q-iteration as a dequantization heuristic when these sufficient conditions cannot be verified.

Related papers

Kernel-based dequantization of variational QML without Random Fourier Features

Ryan Sweke, Seongwook Shin, Elies Gil-Fuster

There is currently a huge effort to understand the potential and limitations of variational quantum machine learning (QML) based on the optimization of parameterized quantum circuits. Recent proposals toward dequantizing variational QML models for regression problems include approaches based on kernel methods with carefully chosen kernel functions, approximated via Random Fourier Features (RFF). In this short note we show that for a wide range of instances, this approach can be simplified. More specifically, we show that the kernels whose evaluation is approximated by RFF in this dequantization algorithm can in fact often be evaluated exactly and efficiently classically. As such, our results enhance the toolkit for kernel-based dequantization of variational QML.

arXiv:2503.23931•Mar 31, 2025•1 citation

On the relation between trainability and dequantization of variational quantum learning models

Elies Gil-Fuster, Casper Gyurik, Adrián Pérez-Salinas +1 more

The quest for successful variational quantum machine learning (QML) relies on the design of suitable parametrized quantum circuits (PQCs), as analogues to neural networks in classical machine learning. Successful QML models must fulfill the properties of trainability and non-dequantization, among others. Recent works have highlighted an intricate interplay between trainability and dequantization of such models, which is still unresolved. In this work we contribute to this debate from the perspective of machine learning, proving a number of results identifying, among others when trainability and non-dequantization are not mutually exclusive. We begin by providing a number of new somewhat broader definitions of the relevant concepts, compared to what is found in other literature, which are operationally motivated, and consistent with prior art. With these precise definitions given and motivated, we then study the relation between trainability and dequantization of variational QML. Next, we also discuss the degrees of "variationalness" of QML models, where we distinguish between models like the hardware efficient ansatz and quantum kernel methods. Finally, we introduce recipes for building PQC-based QML models which are both trainable and nondequantizable, and corresponding to different degrees of variationalness. We do not address the practical utility for such models. Our work however does point toward a way forward for finding more general constructions, for which finding applications may become feasible.

arXiv:2406.07072•Jun 11, 2024•2 citations

Non-variational supervised quantum kernel methods: a review

John Tanner, Chon-Fai Kam, Jingbo Wang

Quantum kernel methods (QKMs) have emerged as a prominent framework for supervised quantum machine learning. Unlike variational quantum algorithms, which rely on gradient-based optimisation and may suffer from issues such as barren plateaus, non-variational QKMs employ fixed quantum feature maps, with model selection performed classically via convex optimisation and cross-validation. This separation of quantum feature embedding from classical training ensures stable optimisation while leveraging quantum circuits to encode data in high-dimensional Hilbert spaces. In this review, we provide a thorough analysis of non-variational supervised QKMs, covering their foundations in classical kernel theory, constructions of fidelity and projected quantum kernels, and methods for their estimation in practice. We examine frameworks for assessing quantum advantage, including generalisation bounds and necessary conditions for separation from classical models, and analyse key challenges such as exponential concentration, dequantisation via tensor-network methods, and the spectral properties of kernel integral operators. We further discuss structured problem classes that may enable advantage, and synthesise insights from comparative and hardware studies. Overall, this review aims to clarify the regimes in which QKMs may offer genuine advantages, and to delineate the conceptual, methodological, and technical obstacles that must be overcome for practical quantum-enhanced learning.

quant-pharXiv:2604.07896•Apr 10, 2026•0 citations

When quantum and classical models disagree: learning beyond minimum norm least square

Slimane Thabet, Léo Monbroussou, Eliott Z. Mamon +1 more

Abstract Quantum Machine Learning algorithms based on Variational Quantum Circuits (VQCs) are important candidates for useful application of quantum computing. It is known that a VQC is a linear model in a feature space determined by its architecture. Such models can be compared to classical ones using various sets of tools, and surrogate models designed to classically approximate their results were proposed. At the same time, quantum advantages for learning tasks have been proven in the case of discrete data distributions and cryptography primitives. In this work, we propose a framework to avoid Random Feature approximation techniques. Using previous results, we establish conditions on the weight vectors of the quantum models that are necessary to avoid these dequantization methods. We show that this theory is compatible with previously proven quantum advantages on discrete inputs, and provides examples of advantages for continuous inputs. This separation is connected to large weight vector norm, and we suggest that this can only happen with a high dimensional feature map. Our results demonstrate that it is possible to design quantum models that cannot be classically approximated with good generalization. In addition, we provide a method to verify that the necessary condition is respected for a quantum model. Finally, we discuss how concentration issues must be considered to design such instances. We expect that our work will be a starting point to design near-term quantum models that avoid dequantization methods by ensuring non-classical convergence properties, and to identify existing quantum models that can be classically approximated.

arXiv:2411.04940•npj Quantum Information•Mar 27, 2026•1 citation

Training Hybrid Quantum Policies with Classical Surrogates for Actor-Critic Reinforcement Learning

Jeremy Kulcsar, Giulio Giaconi, Jie Luo +2 more

Reading out a parametrized quantum circuit through finite-shot measurements introduces a discrete stochastic component into hybrid quantum-classical neural networks, making gradient-based training inefficient. In this work, we introduce a surrogate-assisted actor-critic method for training hybrid quantum policies that avoids differentiating the quantum circuit during backpropagation. A classical neural surrogate trained on the circuit's input-output pairs from a replay buffer provides a differentiable substitute for actor updates and critic-target computation, while forward interaction and evaluation retain the quantum circuit's measurements. Our experiments on standard MuJoCo continuous-control tasks demonstrate promising policy learning. Notably, on Humanoid, more runs reach high returns than with learned classical baselines. Moreover, the best average return in our ablation comes from a smaller circuit / surrogate, suggesting that strong performance does not depend on a large network. We also derive conditional upstream gradient-error bounds that separate surrogate fitting from the mean-feature relaxation. Together, the results support surrogate-assisted training as an effective and efficient approach to learning hybrid quantum policies.

arXiv:2503.09119•Mar 12, 2025•0 citations

Variational Quantum Circuits for Deep Reinforcement Learning

Samuel Yen-Chi Chen, Chao-Han Huck Yang, Jun Qi +3 more

The state-of-the-art machine learning approaches are based on classical von Neumann computing architectures and have been widely used in many industrial and academic domains. With the recent development of quantum computing, researchers and tech-giants have attempted new quantum circuits for machine learning tasks. However, the existing quantum computing platforms are hard to simulate classical deep learning models or problems because of the intractability of deep quantum circuits. Thus, it is necessary to design feasible quantum algorithms for quantum machine learning for noisy intermediate scale quantum (NISQ) devices. This work explores variational quantum circuits for deep reinforcement learning. Specifically, we reshape classical deep reinforcement learning algorithms like experience replay and target network into a representation of variational quantum circuits. Moreover, we use a quantum information encoding scheme to reduce the number of model parameters compared to classical neural networks. To the best of our knowledge, this work is the first proof-of-principle demonstration of variational quantum circuits to approximate the deep Q-value function for decision-making and policy-selection reinforcement learning with experience replay and target network. Besides, our variational quantum circuits can be deployed in many near-term NISQ machines.

arXiv:1907.00397•IEEE Access•Jan 1, 2020•453 citations
In recent years, the utility of parameterized quantum circuits as function approximators has been widely studied. In the context of reinforcement learning, this approach has led to variational quantum algorithms such as quantum Q-learning. While these methods show promising empirical results, and can provide provable advantages for artificial problems, it remains unclear whether they can provide a provable quantum advantage over classical approaches for problems of practical relevance. A natural way to investigate this question is through the lens of dequantization: The construction of efficient classical algorithms capable of matching the performance of quantum variational methods. Building on recent kernel-based dequantization results for supervised learning, we take steps towards extending this surrogate-based dequantization program to reinforcement learning. Specifically, we study the simplified setting of reinforcement learning with a uniform generative model in which uniformly random state-action samples are available, which models the regime of sampling from a large experience replay buffer after sufficient exploration. Within this setting, we provide finite sample guarantees for classical kernelized Fitted Q-Iteration, with classical kernels designed to match the inductive bias of particular parameterized quantum circuits. Using these results, we then provide a set of sufficient conditions, on the data-encoding strategy of a parameterized quantum circuit, the corresponding classical kernel, and the problem structure, under which kernelized Fitted Q-Iteration provides a meaningful dequantization of quantum Q-learning, in this simplified setting. Apart from providing rigorous dequantization guarantees when these conditions are met, these results also motivate the use of kernelized fitted Q-iteration as a dequantization heuristic when these sufficient conditions cannot be verified.
There is currently a huge effort to understand the potential and limitations of variational quantum machine learning (QML) based on the optimization of parameterized quantum circuits. Recent proposals toward dequantizing variational QML models for regression problems include approaches based on kernel methods with carefully chosen kernel functions, approximated via Random Fourier Features (RFF). In this short note we show that for a wide range of instances, this approach can be simplified. More specifically, we show that the kernels whose evaluation is approximated by RFF in this dequantization algorithm can in fact often be evaluated exactly and efficiently classically. As such, our results enhance the toolkit for kernel-based dequantization of variational QML.
The quest for successful variational quantum machine learning (QML) relies on the design of suitable parametrized quantum circuits (PQCs), as analogues to neural networks in classical machine learning. Successful QML models must fulfill the properties of trainability and non-dequantization, among others. Recent works have highlighted an intricate interplay between trainability and dequantization of such models, which is still unresolved. In this work we contribute to this debate from the perspective of machine learning, proving a number of results identifying, among others when trainability and non-dequantization are not mutually exclusive. We begin by providing a number of new somewhat broader definitions of the relevant concepts, compared to what is found in other literature, which are operationally motivated, and consistent with prior art. With these precise definitions given and motivated, we then study the relation between trainability and dequantization of variational QML. Next, we also discuss the degrees of "variationalness" of QML models, where we distinguish between models like the hardware efficient ansatz and quantum kernel methods. Finally, we introduce recipes for building PQC-based QML models which are both trainable and nondequantizable, and corresponding to different degrees of variationalness. We do not address the practical utility for such models. Our work however does point toward a way forward for finding more general constructions, for which finding applications may become feasible.
Quantum kernel methods (QKMs) have emerged as a prominent framework for supervised quantum machine learning. Unlike variational quantum algorithms, which rely on gradient-based optimisation and may suffer from issues such as barren plateaus, non-variational QKMs employ fixed quantum feature maps, with model selection performed classically via convex optimisation and cross-validation. This separation of quantum feature embedding from classical training ensures stable optimisation while leveraging quantum circuits to encode data in high-dimensional Hilbert spaces. In this review, we provide a thorough analysis of non-variational supervised QKMs, covering their foundations in classical kernel theory, constructions of fidelity and projected quantum kernels, and methods for their estimation in practice. We examine frameworks for assessing quantum advantage, including generalisation bounds and necessary conditions for separation from classical models, and analyse key challenges such as exponential concentration, dequantisation via tensor-network methods, and the spectral properties of kernel integral operators. We further discuss structured problem classes that may enable advantage, and synthesise insights from comparative and hardware studies. Overall, this review aims to clarify the regimes in which QKMs may offer genuine advantages, and to delineate the conceptual, methodological, and technical obstacles that must be overcome for practical quantum-enhanced learning.
Abstract Quantum Machine Learning algorithms based on Variational Quantum Circuits (VQCs) are important candidates for useful application of quantum computing. It is known that a VQC is a linear model in a feature space determined by its architecture. Such models can be compared to classical ones using various sets of tools, and surrogate models designed to classically approximate their results were proposed. At the same time, quantum advantages for learning tasks have been proven in the case of discrete data distributions and cryptography primitives. In this work, we propose a framework to avoid Random Feature approximation techniques. Using previous results, we establish conditions on the weight vectors of the quantum models that are necessary to avoid these dequantization methods. We show that this theory is compatible with previously proven quantum advantages on discrete inputs, and provides examples of advantages for continuous inputs. This separation is connected to large weight vector norm, and we suggest that this can only happen with a high dimensional feature map. Our results demonstrate that it is possible to design quantum models that cannot be classically approximated with good generalization. In addition, we provide a method to verify that the necessary condition is respected for a quantum model. Finally, we discuss how concentration issues must be considered to design such instances. We expect that our work will be a starting point to design near-term quantum models that avoid dequantization methods by ensuring non-classical convergence properties, and to identify existing quantum models that can be classically approximated.
Reading out a parametrized quantum circuit through finite-shot measurements introduces a discrete stochastic component into hybrid quantum-classical neural networks, making gradient-based training inefficient. In this work, we introduce a surrogate-assisted actor-critic method for training hybrid quantum policies that avoids differentiating the quantum circuit during backpropagation. A classical neural surrogate trained on the circuit's input-output pairs from a replay buffer provides a differentiable substitute for actor updates and critic-target computation, while forward interaction and evaluation retain the quantum circuit's measurements. Our experiments on standard MuJoCo continuous-control tasks demonstrate promising policy learning. Notably, on Humanoid, more runs reach high returns than with learned classical baselines. Moreover, the best average return in our ablation comes from a smaller circuit / surrogate, suggesting that strong performance does not depend on a large network. We also derive conditional upstream gradient-error bounds that separate surrogate fitting from the mean-feature relaxation. Together, the results support surrogate-assisted training as an effective and efficient approach to learning hybrid quantum policies.
The state-of-the-art machine learning approaches are based on classical von Neumann computing architectures and have been widely used in many industrial and academic domains. With the recent development of quantum computing, researchers and tech-giants have attempted new quantum circuits for machine learning tasks. However, the existing quantum computing platforms are hard to simulate classical deep learning models or problems because of the intractability of deep quantum circuits. Thus, it is necessary to design feasible quantum algorithms for quantum machine learning for noisy intermediate scale quantum (NISQ) devices. This work explores variational quantum circuits for deep reinforcement learning. Specifically, we reshape classical deep reinforcement learning algorithms like experience replay and target network into a representation of variational quantum circuits. Moreover, we use a quantum information encoding scheme to reduce the number of model parameters compared to classical neural networks. To the best of our knowledge, this work is the first proof-of-principle demonstration of variational quantum circuits to approximate the deep Q-value function for decision-making and policy-selection reinforcement learning with experience replay and target network. Besides, our variational quantum circuits can be deployed in many near-term NISQ machines.