Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2404.10546 · IEEE International Conference on Quantum Computing and Engineering (QCE)

Warm-Start Variational Quantum Policy Iteration

Nico Meyer, Jakob Murauer, Alexander Popov, Christian Ufrecht, Axel Plinge, Christopher Mutschler, Daniel D. Scherer

Published Sep 15, 2024• 2 citations
arXivHTMLPDF

Abstract

Reinforcement learning is a powerful framework aiming to determine optimal behavior in highly complex decision-making scenarios. This objective can be achieved using policy iteration, which requires to solve a typically large linear system of equations. We propose the variational quantum policy iteration (VarQPI) algorithm, realizing this step with a NISQ-compatible quantum-enhanced subroutine. Its scalability is supported by an analysis of the structure of generic reinforcement learning environments' laying the foundation for potential quantum advantage with utility-scale quantum computers. Furthermore, we introduce the warm-start initialization variant (WS-VarQPI) that significantly reduces resource overhead. The algorithm solves a large FrozenLake environment with an underlying 256×256-dimensional linear system, indicating its practical robustness.

DOI
10.1109/qce60285.2024.00172

Related papers

Q-Policy: Quantum-Enhanced Policy Evaluation for Scalable Reinforcement Learning

Cherukuri, Kalyan, Lala, Aarav, Yash Yardi

We propose Q-Policy, a hybrid quantum-classical reinforcement learning (RL) framework that mathematically accelerates policy evaluation and optimization by exploiting quantum computing primitives. Q-Policy encodes value functions in quantum superposition, enabling simultaneous evaluation of multiple state-action pairs via amplitude encoding and quantum parallelism. We introduce a quantum-enhanced policy iteration algorithm with provable polynomial reductions in sample complexity for the evaluation step, under standard assumptions. To demonstrate the technical feasibility and theoretical soundness of our approach, we validate Q-Policy on classical emulations of small discrete control tasks. Due to current hardware and simulation limitations, our experiments focus on showcasing proof-of-concept behavior rather than large-scale empirical evaluation. Our results support the potential of Q-Policy as a theoretical foundation for scalable RL on future quantum devices, addressing RL scalability challenges beyond classical approaches.

arXiv:2505.11862•May 17, 2025•0 citations

Adaptive V16: Cost-Aware Reinforcement Learning Control for Robust NISQ Variational Optimization

Kondamudi Sivaparvathi, Cheppali Prabhavathi

Variational quantum algorithms (VQAs) are promising for near-term noisy intermediate-scale quantum (NISQ) devices, but their performance is often limited by device noise, finite-shot uncertainty, and nonstationary behavior that destabilize optimization. This paper proposes a cost-aware closed-loop hybrid framework (Adaptive V16) that augments a standard VQA loop with an online decision policy to adapt measurement effort and mitigation actions during execution. At each iteration, the controller uses runtime signals such as objective progress and uncertainty to select an action comprising the shot budget, mitigation mode, and step-scaling factor, explicitly trading solution improvement against resource overhead under a fixed budget. Experimental evaluation on a representative molecular Hamiltonian optimization task shows that Adaptive V16 achieves a lower final energy than a static VQE baseline and reduces the absolute error from 0.1391 to 0.0657. The controller’s decision module also demonstrates strong predictive performance with 0.9700 accuracy, 0.9595 F1-score, and 0.9951 ROC–AUC, indicating reliable adaptive behavior across thresholds. These results suggest that policy-driven adaptation can improve robustness and budget efficiency for hybrid quantum–classical optimization in realistic NISQ settings.

International Conference on Knowledge Engineering and Communication Systems (ICKECS)•Apr 24, 2026•0 citations

Classical State Preparation for Variational Quantum Algorithms via Reinforcement Learning

Gino Kwun, Dhanvi Bharadwaj, Gokul Subramanian Ravi

Variational Quantum Algorithms (VQAs) potentially offer a pathway to practical quantum advantage, but their optimization is heavily hindered by barren plateaus and numerous local minima. While classically simulable Clifford circuits can warm-start VQAs to accelerate convergence, existing heuristic-based initialization methods struggle to scale within vast combinatorial search spaces. We address this bottleneck with CRiSP (a Clifford Reinforcement Learning agent for State Preparation), which casts Clifford prefix selection as a sequential decision-making problem. CRiSP utilizes Neural-Guided Monte Carlo Tree Search, driven by a Transformer-based policy trained via self-play, to insert learned Clifford gates before fixed parameterized rotations. This enables the construction of high-quality initial states entirely through polynomial-time classical stabilizer simulation. By integrating a curriculum learning strategy that progressively expands the search horizon, the agent efficiently scales to deep circuits. Evaluated on QAOA benchmarks of up to $22$ qubits and $1{,}370$ parameters, CRiSP outperforms state-of-the-art Clifford initialization methods by a mean of $3.19\times$ (max $45.02\times$) in average energy accuracy and $2.41\times$ (max $16.01\times$) in best-achieved energy accuracy. Assessments on VQE tasks further demonstrate the framework's robustness and generalizability.

quant-pharXiv:2605.23138•May 25, 2026•0 citations

LAWS: Look Around and Warm-Start Natural Gradient Descent for Quantum Neural Networks

Zeyi Tao, Jindi Wu, Qi Xia +1 more

Variational quantum algorithms (VQAs) have recently received much attention due to their promising performance in Noisy Intermediate-Scale Quantum computers (NISQ). However, VQAs run on parameterized quantum circuits (PQC) with randomly initialized parameters are characterized by barren plateaus (BP) where the gradient vanishes exponentially in the number of qubits. In this paper, we proposed a Look Around Warm-Start (LAWS) quantum natural gradient (QNG) algorithm to mitigate the widespread existing BP issues. LAWS is a combinatorial optimization strategy taking advantage of model parameter initialization and fast convergence of QNG. LAWS repeatedly reinitializes parameter search space for the next iteration parameter update. The reinitialized parameter search space is carefully chosen by sampling the gradient close to the current optimal. Moreover, we present a unified framework (WS-SGD) for integrating parameter initialization techniques into the optimizer. We provide the convergence proof of the proposed framework for both convex and non-convex objective functions based on Polyak-Lojasiewicz (PL) condition. Our experiment results show that the proposed algorithm could mitigate the BP and have better generalization ability in quantum classification problems.

arXiv:2205.02666•IEEE International Conference on Quantum Software (QSW)•Jul 1, 2023•5 citations

Variational Quantum Simulation: A Case Study for Understanding Warm Starts

Ricard Puig, Marc Drudis, Supanut Thanasilp +1 more

The barren plateau phenomenon, characterized by loss gradients that vanish exponentially with system size, poses a challenge to scaling variational quantum algorithms. Here we explore the potential of warm starts, whereby one initializes closer to a solution in the hope of enjoying larger loss variances. Focusing on an iterative variational method for learning shorter-depth circuits for quantum we conduct a case study to elucidate the potential and limitations of warm starts. We start by proving that the iterative variational algorithm will exhibit substantial (at worst vanishing polynomially in system size) gradients in a small region around the initializations at each time step. Convexity guarantees for these regions are then established, suggesting trainability for polynomial-size time steps. However, our study highlights scenarios where a good minimum shifts outside the region with trainability guarantees. Our analysis leaves open the question whether such minima jumps necessitate optimization across barren plateau landscapes or whether there exist gradient flows, i.e., fertile valleys away from the plateau with substantial gradients, that allow for training. While our main focus is on this case study of variational quantum simulation, we end by discussing how our results work in other iterative settings.

arXiv:2404.10044•PRX Quantum•Jan 23, 2025•34 citations

Quantum-Inspired Algorithms for Reinforcement Learning Policy Optimization

Jincheng Zhang

This paper investigates the potential of quantum-inspired algorithms to improve reinforcement learning (RL) policy optimization. Traditional RL methods often struggle with slow convergence and exploration challenges, particularly in complex environments. We propose a novel approach that draws inspiration from quantum mechanics, specifically superposition and entanglement, to address these limitations. Our algorithms aim to exploit superposition to simultaneously evaluate multiple action possibilities and utilize entanglement to model correlations between states, thereby accelerating the learning process. We present a framework for incorporating these concepts into RL, focusing on algorithmic design rather than a direct quantum simulation. The theoretical analysis suggests that our approach could significantly enhance exploration efficiency and, consequently, policy convergence rates. This work represents a direct application of quantum-inspired ideas to RL policy optimization, offering a potentially transformative pathway for developing more efficient and robust RL agents.

•Sep 7, 2026•0 citations
Reinforcement learning is a powerful framework aiming to determine optimal behavior in highly complex decision-making scenarios. This objective can be achieved using policy iteration, which requires to solve a typically large linear system of equations. We propose the variational quantum policy iteration (VarQPI) algorithm, realizing this step with a NISQ-compatible quantum-enhanced subroutine. Its scalability is supported by an analysis of the structure of generic reinforcement learning environments' laying the foundation for potential quantum advantage with utility-scale quantum computers. Furthermore, we introduce the warm-start initialization variant (WS-VarQPI) that significantly reduces resource overhead. The algorithm solves a large FrozenLake environment with an underlying 256×256-dimensional linear system, indicating its practical robustness.
We propose Q-Policy, a hybrid quantum-classical reinforcement learning (RL) framework that mathematically accelerates policy evaluation and optimization by exploiting quantum computing primitives. Q-Policy encodes value functions in quantum superposition, enabling simultaneous evaluation of multiple state-action pairs via amplitude encoding and quantum parallelism. We introduce a quantum-enhanced policy iteration algorithm with provable polynomial reductions in sample complexity for the evaluation step, under standard assumptions. To demonstrate the technical feasibility and theoretical soundness of our approach, we validate Q-Policy on classical emulations of small discrete control tasks. Due to current hardware and simulation limitations, our experiments focus on showcasing proof-of-concept behavior rather than large-scale empirical evaluation. Our results support the potential of Q-Policy as a theoretical foundation for scalable RL on future quantum devices, addressing RL scalability challenges beyond classical approaches.
Variational quantum algorithms (VQAs) are promising for near-term noisy intermediate-scale quantum (NISQ) devices, but their performance is often limited by device noise, finite-shot uncertainty, and nonstationary behavior that destabilize optimization. This paper proposes a cost-aware closed-loop hybrid framework (Adaptive V16) that augments a standard VQA loop with an online decision policy to adapt measurement effort and mitigation actions during execution. At each iteration, the controller uses runtime signals such as objective progress and uncertainty to select an action comprising the shot budget, mitigation mode, and step-scaling factor, explicitly trading solution improvement against resource overhead under a fixed budget. Experimental evaluation on a representative molecular Hamiltonian optimization task shows that Adaptive V16 achieves a lower final energy than a static VQE baseline and reduces the absolute error from 0.1391 to 0.0657. The controller’s decision module also demonstrates strong predictive performance with 0.9700 accuracy, 0.9595 F1-score, and 0.9951 ROC–AUC, indicating reliable adaptive behavior across thresholds. These results suggest that policy-driven adaptation can improve robustness and budget efficiency for hybrid quantum–classical optimization in realistic NISQ settings.
Variational Quantum Algorithms (VQAs) potentially offer a pathway to practical quantum advantage, but their optimization is heavily hindered by barren plateaus and numerous local minima. While classically simulable Clifford circuits can warm-start VQAs to accelerate convergence, existing heuristic-based initialization methods struggle to scale within vast combinatorial search spaces. We address this bottleneck with CRiSP (a Clifford Reinforcement Learning agent for State Preparation), which casts Clifford prefix selection as a sequential decision-making problem. CRiSP utilizes Neural-Guided Monte Carlo Tree Search, driven by a Transformer-based policy trained via self-play, to insert learned Clifford gates before fixed parameterized rotations. This enables the construction of high-quality initial states entirely through polynomial-time classical stabilizer simulation. By integrating a curriculum learning strategy that progressively expands the search horizon, the agent efficiently scales to deep circuits. Evaluated on QAOA benchmarks of up to 222222 qubits and 1,3701{,}3701,370 parameters, CRiSP outperforms state-of-the-art Clifford initialization methods by a mean of 3.19×3.19\times3.19× (max 45.02×45.02\times45.02×) in average energy accuracy and 2.41×2.41\times2.41× (max 16.01×16.01\times16.01×) in best-achieved energy accuracy. Assessments on VQE tasks further demonstrate the framework's robustness and generalizability.
Variational quantum algorithms (VQAs) have recently received much attention due to their promising performance in Noisy Intermediate-Scale Quantum computers (NISQ). However, VQAs run on parameterized quantum circuits (PQC) with randomly initialized parameters are characterized by barren plateaus (BP) where the gradient vanishes exponentially in the number of qubits. In this paper, we proposed a Look Around Warm-Start (LAWS) quantum natural gradient (QNG) algorithm to mitigate the widespread existing BP issues. LAWS is a combinatorial optimization strategy taking advantage of model parameter initialization and fast convergence of QNG. LAWS repeatedly reinitializes parameter search space for the next iteration parameter update. The reinitialized parameter search space is carefully chosen by sampling the gradient close to the current optimal. Moreover, we present a unified framework (WS-SGD) for integrating parameter initialization techniques into the optimizer. We provide the convergence proof of the proposed framework for both convex and non-convex objective functions based on Polyak-Lojasiewicz (PL) condition. Our experiment results show that the proposed algorithm could mitigate the BP and have better generalization ability in quantum classification problems.
The barren plateau phenomenon, characterized by loss gradients that vanish exponentially with system size, poses a challenge to scaling variational quantum algorithms. Here we explore the potential of warm starts, whereby one initializes closer to a solution in the hope of enjoying larger loss variances. Focusing on an iterative variational method for learning shorter-depth circuits for quantum we conduct a case study to elucidate the potential and limitations of warm starts. We start by proving that the iterative variational algorithm will exhibit substantial (at worst vanishing polynomially in system size) gradients in a small region around the initializations at each time step. Convexity guarantees for these regions are then established, suggesting trainability for polynomial-size time steps. However, our study highlights scenarios where a good minimum shifts outside the region with trainability guarantees. Our analysis leaves open the question whether such minima jumps necessitate optimization across barren plateau landscapes or whether there exist gradient flows, i.e., fertile valleys away from the plateau with substantial gradients, that allow for training. While our main focus is on this case study of variational quantum simulation, we end by discussing how our results work in other iterative settings.
This paper investigates the potential of quantum-inspired algorithms to improve reinforcement learning (RL) policy optimization. Traditional RL methods often struggle with slow convergence and exploration challenges, particularly in complex environments. We propose a novel approach that draws inspiration from quantum mechanics, specifically superposition and entanglement, to address these limitations. Our algorithms aim to exploit superposition to simultaneously evaluate multiple action possibilities and utilize entanglement to model correlations between states, thereby accelerating the learning process. We present a framework for incorporating these concepts into RL, focusing on algorithmic design rather than a direct quantum simulation. The theoretical analysis suggests that our approach could significantly enhance exploration efficiency and, consequently, policy convergence rates. This work represents a direct application of quantum-inspired ideas to RL policy optimization, offering a potentially transformative pathway for developing more efficient and robust RL agents.