Model-based reinforcement learning (MBRL) aims to reduce the number of environment interactions by learning an internal model of the environment that supports policy training by providing learned transition dynamics. The fidelity and stability of this model are of crucial importance. While state-of-the-art methods typically employ large, parameter-heavy architectures to capture complex transition dynamics, creating compact yet expressive models remains an open challenge. Recent advances in quantum computing have motivated the exploration of quantum machine learning models as parameter-efficient architectures capable of modeling complex relationships. In this thesis, we investigate the feasibility of employing hybrid quantum-classical architectures to model the recurrent transition dynamics in MBRL. We propose an evaluation framework based on World Models, in which we can seamlessly exchange different variants of recurrent cells in the state-transition model. This allows us to evaluate quantumenhanced recurrent models against a classical baseline with regard to single-step prediction accuracy, multi-step rollout stability, downstream policy performance, and generalization to unseen environment scenarios. All experiments are conducted in a real-world-inspired autonomous driving simulation environment with multimodal sensor input. We use a multi-fold evaluation process and conduct a large-scale hyperparameter optimization. Our results demonstrate that quantum-enhanced recurrent models achieve comparable performance to classical architectures while requiring substantially fewer parameters. Although no clear quantum advantage is observed, our findings demonstrate the feasibility of hybrid quantum architectures for learning latent dynamics and provide a foundation for future research on integrating quantum machine learning into MBRL.
Model-based reinforcement learning (MBRL) aims to reduce the number of environment interactions by learning an internal model of the environment that supports policy training by providing learned transition dynamics. The fidelity and stability of this model are of crucial importance. While state-of-the-art methods typically employ large, parameter-heavy architectures to capture complex transition dynamics, creating compact yet expressive models remains an open challenge. Recent advances in quantum computing have motivated the exploration of quantum machine learning models as parameter-efficient architectures capable of modeling complex relationships. In this thesis, we investigate the feasibility of employing hybrid quantum-classical architectures to model the recurrent transition dynamics in MBRL. We propose an evaluation framework based on World Models, in which we can seamlessly exchange different variants of recurrent cells in the state-transition model. This allows us to evaluate quantumenhanced recurrent models against a classical baseline with regard to single-step prediction accuracy, multi-step rollout stability, downstream policy performance, and generalization to unseen environment scenarios. All experiments are conducted in a real-world-inspired autonomous driving simulation environment with multimodal sensor input. We use a multi-fold evaluation process and conduct a large-scale hyperparameter optimization. Our results demonstrate that quantum-enhanced recurrent models achieve comparable performance to classical architectures while requiring substantially fewer parameters. Although no clear quantum advantage is observed, our findings demonstrate the feasibility of hybrid quantum architectures for learning latent dynamics and provide a foundation for future research on integrating quantum machine learning into MBRL.
Recent advances in quantum computing (QC) and machine learning (ML) have drawn significant attention to the development of quantum machine learning (QML). Reinforcement learning (RL) is one of the ML paradigms which can be used to solve complex sequential decision making problems. Classical RL has been shown to be capable to solve various challenging tasks. However, RL algorithms in the quantum world are still in their infancy. One of the challenges yet to solve is how to train quantum RL in the partially observable environments. In this paper, we approach this challenge through building QRL agents with quantum recurrent neural networks (QRNN). Specifically, we choose the quantum long short-term memory (QLSTM) to be the core of the QRL agent and train the whole model with deep Q-learning. We demonstrate the results via numerical simulations that the QLSTM-DRQN can solve standard benchmark such as Cart-Pole with more stable and higher average scores than classical DRQN with similar architecture and number of model parameters.
Recent advances in quantum computing (QC) and machine learning (ML) have drawn significant attention to the development of quantum machine learning (QML). Reinforcement learning (RL) is one of the ML paradigms which can be used to solve complex sequential decision making problems. Classical RL has been shown to be capable to solve various challenging tasks. However, RL algorithms in the quantum world are still in their infancy. One of the challenges yet to solve is how to train quantum RL in the partially observable environments. In this paper, we approach this challenge through building QRL agents with quantum recurrent neural networks (QRNN). Specifically, we choose the quantum long short-term memory (QLSTM) to be the core of the QRL agent and train the whole model with deep Q-learning. We demonstrate the results via numerical simulations that the QLSTM-DRQN can solve standard benchmark such as Cart-Pole with more stable and higher average scores than classical DRQN with similar architecture and number of model parameters.
Quantum computing has emerged as a promising computational paradigm for machine learning (ML), with the potential to offer computational advantages over classical approaches. At this stage, the evidence supporting the performance and advantages of quantum machine learning (QML) models relative to classical models is insufficient. To address this gap, this paper presents an empirical study on the performance of QML models and their classical counterparts. We compare seven model pairs spanning supervised learning and reinforcement learning. Our results indicate that the evaluated quantum machine learning models do not yet surpass the classical baselines in overall prediction performance, policy stability, or training time. Nevertheless, QML remains a promising approach for filtering noise and controlling false positives. Our research findings summarize the challenges facing quantum machine learning across hardware environments, training efficiency, and convergence stability, providing a foundation for research into the robustness and parameter optimization of QML. This work is publicly available at https://github.com/Z-537-437/QML.
The rapid development and integration of wearable technologies or Internet-of-Things (IoT) into healthcare technologies with computational methods holds transformative potential for future healthcare systems, especially for personalized health monitoring and early disease detection. In parallel, recent advancements in quantum computing have o↵ered significant interest in quantum machine learning (QML), an intersection of quantum computing and machine learning, which provides a novel hybrid computational paradigm capable of addressing complex healthcare challenges that remain unsolvable with classical machine learning techniques. This thesis explores the application of quantum-enhanced recurrent neural network architectures, specifically Quantum Long Short-Term Memory (QLSTM) and Quantum-based Gated Recurrent Unit (QGRU), for predictive modeling and activity classification in wearable IoT systems, focusing particularly on their applicability and efficacy to elderly populations. One of the key challenges in modern healthcare intelligence systems is the simple but accurate estimation mechanism of Physical Activity Energy Expenditure (PAEE), which is an essential factor for assessing physical health status, particularly for healthy aging. In order to address this challenge, the first work proposes a model called an enhanced QLSTM with linear layer (eQLSTML) that integrates Variational Quantum Circuits (VQCs) into classical LSTM architectures. This integration notably explores the advantages of quantum computing, including properties such as entanglement and superposition, to enhance the model’s ability to capture complex temporal dependencies and subtle variations in human activity patterns. The modelling experiments were evaluated using the publicly available GOTOV Human Physical Activity dataset, which includes accelerometer data collected from older adults engaged in various daily activities. In addition to modelling architecture analysis, this thesis provides comprehensive analyses concerning the scalability, feasibility, and computational complexity of the proposed quantum-enhanced models compared to the classical backbone. The overall results demonstrate that the proposed quantum-enhanced approach significantly outperforms traditional classical machine learning algorithms, which demonstrates the potential ability for accurate predictions and robustness to noisy data like accelerometer data. In addition, accurate classification of activities of daily living (ADLs) is another critical aspect for early disease detection and e↵ective healthcare intervention. Within the context of this thesis, the second work introduces a novel hybrid model called QGRU-Multiclass Classifier (QGRU-MC). The data preprocessing involves creating a statistical feature extraction and applying oversampling techniques – SMOTE-ENN (Synthetic Minority Oversampling Technique with Edited Nearest Neighbors) for the publicly available dataset called “Dataset for ADL recognition with Wrist-worn Accelerometer”. The preliminary findings suggest that our model has good potential for healthcare applications, particularly in advancing future intelligent systems focused on daily activity monitoring. Specifically, QGRU-MC achieved significant improvements in classification accuracy, sensitivity, and specificity across multiple categories of daily activities, highlighting its potential for practical deployment in intelligent wearable systems aimed at monitoring and supporting healthy aging. Furthermore, this work also discusses the future research directions and practical considerations for transforming from the simulators to modern real quantum computers in the noisy intermediate-scale quantum (NISQ) era for a more comprehensive view on exploiting potential impacts on public health policies and personalized healthcare systems. Overall, simulator-based results indicate promising potential for integrating QML methods into wearable healthcare applications such as predictive analytics, personalized treatments, and health disease intervention to improve healthcare outcomes. Specifically, on PAEE regression, the proposed eQLSTML improved R² by 13.2% and reduced RMSE and MAE by 14.8% and 19.1%, respectively, relative to the classical LSTM; on ADL, QGRU-MC demonstrates good macro-averaged metrics across all eight classes (see Chs. 3–4). This research contributes significantly to the growing body of literature supporting quantum computing technologies in healthcare applications, making QML an incredible tool for addressing complex healthrelated computational challenges in the era of big and complex data in medical fields and smart healthcare technologies, especially in e↵ectively solving data from multiple sources, wearable accelerometry, and multi-class ADL recognition with complicated tasks.
Research on quantum computing is still in its infancy, but it has a lot of potential uses. One topic with potential is machine learning, namely in the field of reinforcement learning. This work examines the integration of parametrized quantum circuits (PQC) into reinforcement learning (RL) algorithms, assessing the potential of quantum-enhanced models to solve classical RL tasks. It closely follows the example found on the TensorFlow website. This paper reviews applications of quantum reinforcement learning (QRL). We examine PQCs in a standard RL scenario, the CartPole-v1 environment from Gym, using TensorFlow Quantum and Cirq, to evaluate the relative performance of quantum versus conventional models. In comparison to conventional deep neural network (ONN) models, PQCs show slower convergence and higher processing needs, even if they are still able to learn the task and perform competitively. After they are fully trained, the quantum models show unique difficulties during the early training stages and reach a performance stability level like classical methods. This study sheds light on the present constraints as well as possible uses of quantum computing in reinforcement learning, particularly in situations with intricate, high-dimensional settings that prove difficult for classical computers to handle effectively. As we look to the future, we suggest that investigating hybrid quantum-classical algorithms, developing quantum hardware, and using quantum RL for increasingly difficult tasks are essential first steps. The study presents findings from both a classical reinforcement learning algorithm and a quantum integrated reinforcement learning algorithm. To provide a reliable comparison between quantum reinforcement algorithms and their classical equivalents, further work remains. This work lays the groundwork for future advances in the field by investigating the viability and use of quantum algorithms in reinforcement learning, even if it is not particularly unique. The purpose of this work is to help newcomers to this emerging field of study.
In recent years, advancements in brain science and neuroscience have significantly influenced the field of computer science, particularly in the domain of reinforcement learning (RL). Drawing insights from neurobiology and neuropsychology, researchers have leveraged these findings to develop novel mechanisms for understanding intelligent decision-making processes in the brain. Concurrently, the emergence of quantum computing has opened new frontiers in artificial intelligence, leading to the development of quantum machine learning (QML). This study introduces a novel model that integrates Quantum Spiking Neural Networks (QSNN) and Quantum Long Short-Term Memory (QLSTM) architectures, inspired by the complex workings of the human brain. Specifically designed for reinforcement learning tasks in energy-efficient environments, our approach progresses through two distinct stages mirroring sensory and memory systems. In the initial stage, analogous to the brain’s hypothalamus, low-level information is extracted to emulate sensory data processing patterns. Subsequently, resembling the hippocampus, this information is processed at a higher level, capturing and memorizing correlated patterns. We conduct a comparative analysis of our model against existing quantum models such as Quantum Neural Networks and their classical counterparts, elucidating its unique contributions. Through empirical results, we aim to underscore the effectiveness of our approach in optimizing energy consumption in practical scenarios.