Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2511.17112

Dissecting Quantum Reinforcement Learning: A Systematic Evaluation of Key Components

Javier Lazaro, Juan-Ignacio Vazquez, Pablo Garcia-Bringas

quant-phcs.LGSubmitted Nov 21, 2025• Announced Nov 24, 2025• 0 citations
arXivHTMLPDF

Abstract

Parameterised quantum circuit (PQC) based Quantum Reinforcement Learning (QRL) has emerged as a promising paradigm at the intersection of quantum computing and reinforcement learning (RL). By design, PQCs create hybrid quantum-classical models, but their practical applicability remains uncertain due to training instabilities, barren plateaus (BPs), and the difficulty of isolating the contribution of individual pipeline components. In this work, we dissect PQC based QRL architectures through a systematic experimental evaluation of three aspects recurrently identified as critical: (i) data embedding strategies, with Data Reuploading (DR) as an advanced approach; (ii) ansatz design, particularly the role of entanglement; and (iii) post-processing blocks after quantum measurement, with a focus on the underexplored Output Reuse (OR) technique. Using a unified PPO-CartPole framework, we perform controlled comparisons between hybrid and classical agents under identical conditions. Our results show that OR, though purely classical, exhibits distinct behaviour in hybrid pipelines, that DR improves trainability and stability, and that stronger entanglement can degrade optimisation, offsetting classical gains. Together, these findings provide controlled empirical evidence of the interplay between quantum and classical contributions, and establish a reproducible framework for systematic benchmarking and component-wise analysis in QRL.

Related papers

Parametrized Quantum Circuits for Reinforcement Learning

Kelvin Tafadzwa Mpofu, Patience T. Mthunzi-Kufa

Research on quantum computing is still in its infancy, but it has a lot of potential uses. One topic with potential is machine learning, namely in the field of reinforcement learning. This work examines the integration of parametrized quantum circuits (PQC) into reinforcement learning (RL) algorithms, assessing the potential of quantum-enhanced models to solve classical RL tasks. It closely follows the example found on the TensorFlow website. This paper reviews applications of quantum reinforcement learning (QRL). We examine PQCs in a standard RL scenario, the CartPole-v1 environment from Gym, using TensorFlow Quantum and Cirq, to evaluate the relative performance of quantum versus conventional models. In comparison to conventional deep neural network (ONN) models, PQCs show slower convergence and higher processing needs, even if they are still able to learn the task and perform competitively. After they are fully trained, the quantum models show unique difficulties during the early training stages and reach a performance stability level like classical methods. This study sheds light on the present constraints as well as possible uses of quantum computing in reinforcement learning, particularly in situations with intricate, high-dimensional settings that prove difficult for classical computers to handle effectively. As we look to the future, we suggest that investigating hybrid quantum-classical algorithms, developing quantum hardware, and using quantum RL for increasingly difficult tasks are essential first steps. The study presents findings from both a classical reinforcement learning algorithm and a quantum integrated reinforcement learning algorithm. To provide a reliable comparison between quantum reinforcement algorithms and their classical equivalents, further work remains. This work lays the groundwork for future advances in the field by investigating the viability and use of quantum algorithms in reinforcement learning, even if it is not particularly unique. The purpose of this work is to help newcomers to this emerging field of study.

International Multidisciplinary Information Technology and Engineering Conference (IMITEC)•Nov 27, 2024•5 citations

Benchmarking Quantum Reinforcement Learning

Georg Kruse, Rodrigo Coelho, Andreas Roßkopf +2 more

Quantum Reinforcement Learning (QRL) has emerged as a promising research field, leveraging the principles of quantum mechanics to enhance the performance of reinforcement learning (RL) algorithms. However, despite its growing interest, QRL still faces significant challenges. It is still uncertain if QRL can show any advantage over classical RL beyond artificial problem formulations. Additionally, it is not yet clear which streams of QRL research show the greatest potential. The lack of a unified benchmark and the need to evaluate the reliance on quantum principles of QRL approaches are pressing questions. This work aims to address these challenges by providing a comprehensive comparison of three major QRL classes: Parameterized Quantum Circuit based QRL (PQC-QRL) (with one policy gradient (QPG) and one Q-Learning (QDQN) algorithm), Free Energy based QRL (FE-QRL), and Amplitude Amplification based QRL (AA-QRL). We introduce a set of metrics to evaluate the QRL algorithms on the widely applicable benchmark of gridworld games. Our results provide a detailed analysis of the strengths and weaknesses of the QRL classes, shedding light on the role of quantum principles in QRL and paving the way for future research in this field.

arXiv:2502.04909•International Conference on Agents and Artificial Intelligence (ICAART)•Jan 1, 2025•3 citations

Quantum Reinforcement Learning: An Overview

Gyu Seon Kim, Soohyun Park, Joongheon Kim

Recent research in quantum computing has gained significant momentum, promising transformative advances across various fields, including artificial intelligence (AI). This paper focuses on quantum reinforcement learning (QRL), which utilizes quantum neural network (QNN) to address the inherent chal-lenges in conventional reinforcement learning (RL) frameworks. For a comprehensive examination of QRL, in this paper, the structure of QNN is explored in-depth, including the functional roles of key quantum circuit gates such as the unitary, rotation, controlled-X, and Hadamard gates. This paper also discusses the primary components of QNN, namely state encoding, parameterized quantum circuits (PQC), and measurement processes. Furthermore, the application of IBM qiskit for the visualization of quantum states and circuits is highlighted, providing practical insights into the deployment of quantum principles in RL. Consequently, this paper encapsulates the potential of QRL to revolutionize AI, positioning it as a pivotal area of future research and development in quantum computing.

International Conference on Information and Communication Technology Convergence (ICTC)•Oct 16, 2024•4 citations

Variational Quantum Circuit Design for Quantum Reinforcement Learning on Continuous Environments

Georg Kruse, Theodora-Augustina Drăgan, Robert Wille +1 more

Quantum Reinforcement Learning (QRL) emerged as a branch of reinforcement learning (RL) that uses quantum submodules in the architecture of the algorithm. One branch of QRL focuses on the replacement of neural networks (NN) by variational quantum circuits (VQC) as function approximators. Initial works have shown promising results on classical environments with discrete action spaces, but many of the proposed architectural design choices of the VQC lack a detailed investigation. Hence, in this work we investigate the impact of VQC design choices such as angle embedding, encoding block architecture and postprocessesing on the training capabilities of QRL agents. We show that VQC design greatly influences training performance and heuristically derive enhancements for the analyzed components. Additionally, we show how to design a QRL agent in order to solve classical environments with continuous action spaces and benchmark our agents against classical feed-forward NNs.

arXiv:2312.13798•International Conference on Agents and Artificial Intelligence (ICAART)•Jan 1, 2024•8 citations

A quantum-classical reinforcement learning model to play Atari games

Dominik Freinberger, Julian Lemmel, Radu Grosu +1 more

Recent advances in reinforcement learning have demonstrated the potential of quantum learning models based on parametrized quantum circuits as an alternative to deep learning models. On the one hand, these findings have shown the ultimate exponential speed-ups in learning that full-blown quantum models can offer in certain -- artificially constructed -- environments. On the other hand, they have demonstrated the ability of experimentally accessible PQCs to solve OpenAI Gym benchmarking tasks. However, it remains an open question whether these near-term QRL techniques can be successfully applied to more complex problems exhibiting high-dimensional observation spaces. In this work, we bridge this gap and present a hybrid model combining a PQC with classical feature encoding and post-processing layers that is capable of tackling Atari games. A classical model, subjected to architectural restrictions similar to those present in the hybrid model is constructed to serve as a reference. Our numerical investigation demonstrates that the proposed hybrid model is capable of solving the Pong environment and achieving scores comparable to the classical reference in Breakout. Furthermore, our findings shed light on important hyperparameter settings and design choices that impact the interplay of the quantum and classical components. This work contributes to the understanding of near-term quantum learning models and makes an important step towards their deployment in real-world RL scenarios.

arXiv:2412.08725•Dec 11, 2024•2 citations

SimplyQRL: A Modular Benchmarking Library for Hybrid Quantum Reinforcement Learning

Javier Lazaro, Juan Ignacio Vázquez, Pablo G. Bringas

Hybrid quantum–classical reinforcement learning (QRL) has recently emerged as a line of research exploring the benefits of quantum models in reinforcement-learning tasks, yet progress remains hindered by the lack of standard, modular tooling. SimplyQRL fills this gap with a lightweight Python library that decouples RL algorithms from agent architecture, allowing fair, reproducible comparisons across classical, quantum and hybrid designs. Built on CleanRL, Gymnasium and PennyLane, SimplyQRL ships native PPO and DQN implementations and exposes a single configuration dictionary that controls encoding strategies, parameterised quantum circuits and optional inference networks, plus advanced mechanisms such as Data Reuploading, Output Reuse and Output Scaling. Two illustrative case studies on CartPole-v1 and FrozenLake-v1 demonstrate SimplyQRL’s modular architecture, highlighting how agent components can be flexibly swapped and combined. By providing the first purpose-built, architecture-aware benchmark for QRL, SimplyQRL offers the community a concise, extensible foundation for systematic experimentation and accelerated discovery.

Lecture notes in computer science•Oct 13, 2025•0 citations
Parameterised quantum circuit (PQC) based Quantum Reinforcement Learning (QRL) has emerged as a promising paradigm at the intersection of quantum computing and reinforcement learning (RL). By design, PQCs create hybrid quantum-classical models, but their practical applicability remains uncertain due to training instabilities, barren plateaus (BPs), and the difficulty of isolating the contribution of individual pipeline components. In this work, we dissect PQC based QRL architectures through a systematic experimental evaluation of three aspects recurrently identified as critical: (i) data embedding strategies, with Data Reuploading (DR) as an advanced approach; (ii) ansatz design, particularly the role of entanglement; and (iii) post-processing blocks after quantum measurement, with a focus on the underexplored Output Reuse (OR) technique. Using a unified PPO-CartPole framework, we perform controlled comparisons between hybrid and classical agents under identical conditions. Our results show that OR, though purely classical, exhibits distinct behaviour in hybrid pipelines, that DR improves trainability and stability, and that stronger entanglement can degrade optimisation, offsetting classical gains. Together, these findings provide controlled empirical evidence of the interplay between quantum and classical contributions, and establish a reproducible framework for systematic benchmarking and component-wise analysis in QRL.
Research on quantum computing is still in its infancy, but it has a lot of potential uses. One topic with potential is machine learning, namely in the field of reinforcement learning. This work examines the integration of parametrized quantum circuits (PQC) into reinforcement learning (RL) algorithms, assessing the potential of quantum-enhanced models to solve classical RL tasks. It closely follows the example found on the TensorFlow website. This paper reviews applications of quantum reinforcement learning (QRL). We examine PQCs in a standard RL scenario, the CartPole-v1 environment from Gym, using TensorFlow Quantum and Cirq, to evaluate the relative performance of quantum versus conventional models. In comparison to conventional deep neural network (ONN) models, PQCs show slower convergence and higher processing needs, even if they are still able to learn the task and perform competitively. After they are fully trained, the quantum models show unique difficulties during the early training stages and reach a performance stability level like classical methods. This study sheds light on the present constraints as well as possible uses of quantum computing in reinforcement learning, particularly in situations with intricate, high-dimensional settings that prove difficult for classical computers to handle effectively. As we look to the future, we suggest that investigating hybrid quantum-classical algorithms, developing quantum hardware, and using quantum RL for increasingly difficult tasks are essential first steps. The study presents findings from both a classical reinforcement learning algorithm and a quantum integrated reinforcement learning algorithm. To provide a reliable comparison between quantum reinforcement algorithms and their classical equivalents, further work remains. This work lays the groundwork for future advances in the field by investigating the viability and use of quantum algorithms in reinforcement learning, even if it is not particularly unique. The purpose of this work is to help newcomers to this emerging field of study.
Quantum Reinforcement Learning (QRL) has emerged as a promising research field, leveraging the principles of quantum mechanics to enhance the performance of reinforcement learning (RL) algorithms. However, despite its growing interest, QRL still faces significant challenges. It is still uncertain if QRL can show any advantage over classical RL beyond artificial problem formulations. Additionally, it is not yet clear which streams of QRL research show the greatest potential. The lack of a unified benchmark and the need to evaluate the reliance on quantum principles of QRL approaches are pressing questions. This work aims to address these challenges by providing a comprehensive comparison of three major QRL classes: Parameterized Quantum Circuit based QRL (PQC-QRL) (with one policy gradient (QPG) and one Q-Learning (QDQN) algorithm), Free Energy based QRL (FE-QRL), and Amplitude Amplification based QRL (AA-QRL). We introduce a set of metrics to evaluate the QRL algorithms on the widely applicable benchmark of gridworld games. Our results provide a detailed analysis of the strengths and weaknesses of the QRL classes, shedding light on the role of quantum principles in QRL and paving the way for future research in this field.
Recent research in quantum computing has gained significant momentum, promising transformative advances across various fields, including artificial intelligence (AI). This paper focuses on quantum reinforcement learning (QRL), which utilizes quantum neural network (QNN) to address the inherent chal-lenges in conventional reinforcement learning (RL) frameworks. For a comprehensive examination of QRL, in this paper, the structure of QNN is explored in-depth, including the functional roles of key quantum circuit gates such as the unitary, rotation, controlled-X, and Hadamard gates. This paper also discusses the primary components of QNN, namely state encoding, parameterized quantum circuits (PQC), and measurement processes. Furthermore, the application of IBM qiskit for the visualization of quantum states and circuits is highlighted, providing practical insights into the deployment of quantum principles in RL. Consequently, this paper encapsulates the potential of QRL to revolutionize AI, positioning it as a pivotal area of future research and development in quantum computing.
Quantum Reinforcement Learning (QRL) emerged as a branch of reinforcement learning (RL) that uses quantum submodules in the architecture of the algorithm. One branch of QRL focuses on the replacement of neural networks (NN) by variational quantum circuits (VQC) as function approximators. Initial works have shown promising results on classical environments with discrete action spaces, but many of the proposed architectural design choices of the VQC lack a detailed investigation. Hence, in this work we investigate the impact of VQC design choices such as angle embedding, encoding block architecture and postprocessesing on the training capabilities of QRL agents. We show that VQC design greatly influences training performance and heuristically derive enhancements for the analyzed components. Additionally, we show how to design a QRL agent in order to solve classical environments with continuous action spaces and benchmark our agents against classical feed-forward NNs.
Recent advances in reinforcement learning have demonstrated the potential of quantum learning models based on parametrized quantum circuits as an alternative to deep learning models. On the one hand, these findings have shown the ultimate exponential speed-ups in learning that full-blown quantum models can offer in certain -- artificially constructed -- environments. On the other hand, they have demonstrated the ability of experimentally accessible PQCs to solve OpenAI Gym benchmarking tasks. However, it remains an open question whether these near-term QRL techniques can be successfully applied to more complex problems exhibiting high-dimensional observation spaces. In this work, we bridge this gap and present a hybrid model combining a PQC with classical feature encoding and post-processing layers that is capable of tackling Atari games. A classical model, subjected to architectural restrictions similar to those present in the hybrid model is constructed to serve as a reference. Our numerical investigation demonstrates that the proposed hybrid model is capable of solving the Pong environment and achieving scores comparable to the classical reference in Breakout. Furthermore, our findings shed light on important hyperparameter settings and design choices that impact the interplay of the quantum and classical components. This work contributes to the understanding of near-term quantum learning models and makes an important step towards their deployment in real-world RL scenarios.
Hybrid quantum–classical reinforcement learning (QRL) has recently emerged as a line of research exploring the benefits of quantum models in reinforcement-learning tasks, yet progress remains hindered by the lack of standard, modular tooling. SimplyQRL fills this gap with a lightweight Python library that decouples RL algorithms from agent architecture, allowing fair, reproducible comparisons across classical, quantum and hybrid designs. Built on CleanRL, Gymnasium and PennyLane, SimplyQRL ships native PPO and DQN implementations and exposes a single configuration dictionary that controls encoding strategies, parameterised quantum circuits and optional inference networks, plus advanced mechanisms such as Data Reuploading, Output Reuse and Output Scaling. Two illustrative case studies on CartPole-v1 and FrozenLake-v1 demonstrate SimplyQRL’s modular architecture, highlighting how agent components can be flexibly swapped and combined. By providing the first purpose-built, architecture-aware benchmark for QRL, SimplyQRL offers the community a concise, extensible foundation for systematic experimentation and accelerated discovery.