At the interception between quantum computing and machine learning, Quantum Reinforcement Learning (QRL) has emerged as a promising research field. Due to its novelty, a standardized and comprehensive collection for QRL algorithms has not yet been established. Researchers rely on numerous software stacks for classical Reinforcement Learning (RL) as well as on various quantum computing frameworks for the implementation of the quantum subroutines of their QRL algorithms. Inspired by the CleanRL library for classical RL algorithms, we present CleanQRL, a library that offers singlescript implementations of many QRL algorithms. Our library provides clear and easy to understand scripts that researchers can quickly adapt to their own needs. Alongside ray tune for distributed computing and streamlined hyperparameter tuning, CleanQRL uses weights&biases to log important metrics, which facilitates benchmarking against other classical and quantum implementations. The CleanQRL library enables researchers to easily transition from theoretical considerations to practical applications.
At the interception between quantum computing and machine learning, Quantum Reinforcement Learning (QRL) has emerged as a promising research field. Due to its novelty, a standardized and comprehensive collection for QRL algorithms has not yet been established. Researchers rely on numerous software stacks for classical Reinforcement Learning (RL) as well as on various quantum computing frameworks for the implementation of the quantum subroutines of their QRL algorithms. Inspired by the CleanRL library for classical RL algorithms, we present CleanQRL, a library that offers singlescript implementations of many QRL algorithms. Our library provides clear and easy to understand scripts that researchers can quickly adapt to their own needs. Alongside ray tune for distributed computing and streamlined hyperparameter tuning, CleanQRL uses weights&biases to log important metrics, which facilitates benchmarking against other classical and quantum implementations. The CleanQRL library enables researchers to easily transition from theoretical considerations to practical applications.
Hybrid quantum–classical reinforcement learning (QRL) has recently emerged as a line of research exploring the benefits of quantum models in reinforcement-learning tasks, yet progress remains hindered by the lack of standard, modular tooling. SimplyQRL fills this gap with a lightweight Python library that decouples RL algorithms from agent architecture, allowing fair, reproducible comparisons across classical, quantum and hybrid designs. Built on CleanRL, Gymnasium and PennyLane, SimplyQRL ships native PPO and DQN implementations and exposes a single configuration dictionary that controls encoding strategies, parameterised quantum circuits and optional inference networks, plus advanced mechanisms such as Data Reuploading, Output Reuse and Output Scaling. Two illustrative case studies on CartPole-v1 and FrozenLake-v1 demonstrate SimplyQRL’s modular architecture, highlighting how agent components can be flexibly swapped and combined. By providing the first purpose-built, architecture-aware benchmark for QRL, SimplyQRL offers the community a concise, extensible foundation for systematic experimentation and accelerated discovery.
Quantum Reinforcement Learning (QRL) has emerged as a promising research field, leveraging the principles of quantum mechanics to enhance the performance of reinforcement learning (RL) algorithms. However, despite its growing interest, QRL still faces significant challenges. It is still uncertain if QRL can show any advantage over classical RL beyond artificial problem formulations. Additionally, it is not yet clear which streams of QRL research show the greatest potential. The lack of a unified benchmark and the need to evaluate the reliance on quantum principles of QRL approaches are pressing questions. This work aims to address these challenges by providing a comprehensive comparison of three major QRL classes: Parameterized Quantum Circuit based QRL (PQC-QRL) (with one policy gradient (QPG) and one Q-Learning (QDQN) algorithm), Free Energy based QRL (FE-QRL), and Amplitude Amplification based QRL (AA-QRL). We introduce a set of metrics to evaluate the QRL algorithms on the widely applicable benchmark of gridworld games. Our results provide a detailed analysis of the strengths and weaknesses of the QRL classes, shedding light on the role of quantum principles in QRL and paving the way for future research in this field.
Recent advancements in quantum computing (QC) and machine learning (ML) have sparked considerable interest in the integration of these two cutting-edge fields. Among the various ML techniques, reinforcement learning (RL) stands out for its ability to address complex sequential decision-making problems. RL has already demonstrated substantial success in the classical ML community. Now, the emerging field of Quantum Reinforcement Learning (QRL) seeks to enhance RL algorithms by incorporating principles from quantum computing. This paper offers an introduction to this exciting area for the broader AI and ML community.
Exploring the convergence of quantum computing and machine learning, this paper delves into Quantum Reinforcement Learning (QRL) with a specific focus on Variational Quantum Circuits (VQC). Alongside a comprehensive examination of the field, the study presents a concrete implementation of QRL tailored to foundational gym environments. Leveraging principles from Quantum Computing, experiments reveal QRL’s potential in enhancing reinforcement learning tasks within these environments, showcasing notable space optimization for RL models. Quantitative results indicate a reduction in the number of trainable parameters by up to 90% when compared to classical approaches, significantly improving efficiency, mainly due to quantum principles of superposition and entanglement. This highlights the promising implications of integrating quantum computing techniques to address challenges and advance capabilities in fundamental reinforcement learning scenarios. Furthermore, the study discusses the adaptability of QRL algorithms to diverse problem domains, suggesting their potential for scalability and applicability beyond simple environments. Such versatility underscores the robustness and practical relevance of QRL methodologies in real-world scenarios, positioning them as valuable tools for tackling complex reinforcement learning challenges.
Benchmarking and establishing proper statistical validation metrics for reinforcement learning (RL) remain ongoing challenges, where no consensus has been established yet. The emergence of quantum computing and its potential applications in quantum reinforcement learning (QRL) further complicate benchmarking efforts. To enable valid performance comparisons and to streamline current research in this area, we propose a novel benchmarking methodology, which is based on a statistical estimator for sample complexity and a definition of statistical outperformance. Furthermore, considering QRL, our methodology casts doubt on some previous claims regarding its superiority. We conducted experiments on a novel benchmarking environment with flexible levels of complexity. While we still identify possible advantages, our findings are more nuanced overall. We discuss the potential limitations of these results and explore their implications for empirical research on quantum advantage in QRL.
Recent research in quantum computing has gained significant momentum, promising transformative advances across various fields, including artificial intelligence (AI). This paper focuses on quantum reinforcement learning (QRL), which utilizes quantum neural network (QNN) to address the inherent chal-lenges in conventional reinforcement learning (RL) frameworks. For a comprehensive examination of QRL, in this paper, the structure of QNN is explored in-depth, including the functional roles of key quantum circuit gates such as the unitary, rotation, controlled-X, and Hadamard gates. This paper also discusses the primary components of QNN, namely state encoding, parameterized quantum circuits (PQC), and measurement processes. Furthermore, the application of IBM qiskit for the visualization of quantum states and circuits is highlighted, providing practical insights into the deployment of quantum principles in RL. Consequently, this paper encapsulates the potential of QRL to revolutionize AI, positioning it as a pivotal area of future research and development in quantum computing.