Reinforcement learning is playing a crucial role in sustainable developments and enabling wide applications in our daily lives ranging from self-driving cars, robotics, automation, personalised product suggestions, user notifications and many more. Kernel-based reinforcement learning holds theoretical guarantees of convergence to a unique solution for decision function approximation. This has motivated the development of implementations that exploit the advantage of kernel methods whilst overcoming the ‘curse of dimensionality’. The theoretical properties of kernel-based methods also support the combination of learning agents in loosely coupled systems into a single combined model for composable learning. Learning speedup is achievable by introducing expert knowledge to the algorithm through the kernel, which reduces variance by increasing bias and thus learning speed. Despite the theoretical promise of kernel based reinforcement learning methods, and continued evolution of more efficient algorithms, practical implementation are not prevalent. Research in quantum machine learning has successfully applied quantum algorithms to achieve computational speed-ups for various machine-learning tasks, and has also identified potential quantum advantage through the identification of patterns not accessible to classical data mapping. To date, this has not been applied to kernel based reinforcement learning. Recent work in the field of quantum learning suggests an important duality between quantum machine learning and kernel machine learning methods. Therefore, this paper proposes a framework to complement kernel-based reinforcement learning with quantum generated feature maps for function approximation to improve learning speed. This has positive consequences for optimisation of reinforcement learning techniques already in use to support sustainability targets in applications such as industry innovation and infrastructure, power generation management, data centre cooling and self-driving cars. Our established framework enables the adoption of emerging quantum kernel methods for reinforcement learning applications. The achieved results show over 20 % improvement and positive impacts of quantum map selection on model performance for different problems.
Reinforcement learning is playing a crucial role in sustainable developments and enabling wide applications in our daily lives ranging from self-driving cars, robotics, automation, personalised product suggestions, user notifications and many more. Kernel-based reinforcement learning holds theoretical guarantees of convergence to a unique solution for decision function approximation. This has motivated the development of implementations that exploit the advantage of kernel methods whilst overcoming the ‘curse of dimensionality’. The theoretical properties of kernel-based methods also support the combination of learning agents in loosely coupled systems into a single combined model for composable learning. Learning speedup is achievable by introducing expert knowledge to the algorithm through the kernel, which reduces variance by increasing bias and thus learning speed. Despite the theoretical promise of kernel based reinforcement learning methods, and continued evolution of more efficient algorithms, practical implementation are not prevalent. Research in quantum machine learning has successfully applied quantum algorithms to achieve computational speed-ups for various machine-learning tasks, and has also identified potential quantum advantage through the identification of patterns not accessible to classical data mapping. To date, this has not been applied to kernel based reinforcement learning. Recent work in the field of quantum learning suggests an important duality between quantum machine learning and kernel machine learning methods. Therefore, this paper proposes a framework to complement kernel-based reinforcement learning with quantum generated feature maps for function approximation to improve learning speed. This has positive consequences for optimisation of reinforcement learning techniques already in use to support sustainability targets in applications such as industry innovation and infrastructure, power generation management, data centre cooling and self-driving cars. Our established framework enables the adoption of emerging quantum kernel methods for reinforcement learning applications. The achieved results show over 20 % improvement and positive impacts of quantum map selection on model performance for different problems.
Abstract This work introduces a novel hybrid quantum-classical machine learning framework in which a Reinforcement Learning (RL) agent autonomously constructs quantum circuits to optimize kernels for a Quantum Support Vector Classifier (QSVC). Unlike traditional approaches that rely on fixed or manually designed quantum feature maps, the proposed method employs a Deep Q-Network (DQN) agent to dynamically select quantum gates and their parameters. These configurations define a parameterized quantum circuit whose fidelity-based kernel is used to perform classification tasks. The RL agent is trained to maximize classification accuracy by interacting with an environment that evaluates circuit performance. Experiments conducted on a realistic and potentially biased educational dataset demonstrate that the RL-optimized QSVC outperforms both classical SVC and standard QSVC baselines in terms of accuracy and fairness. Furthermore, SHAP-based explainability analysis reveals a reduction in model bias, suggesting improved interpretability and robustness. This study highlights the potential of reinforcement learning for automated quantum circuit design and kernel optimization in quantum machine learning.
In the past decade, deep learning methods have seen tremendous success in various supervised and unsupervised learning tasks such as classification and generative modeling. More recently, deep neural networks have emerged in the domain of reinforcement learning as a tool to solve decision-making problems of unprecedented complexity, e.g., navigation problems or game-playing AI. Despite the successful combinations of ideas from quantum computing with machine learning methods, there have been relatively few attempts to design quantum algorithms that would enhance deep reinforcement learning. This is partly due to the fact that quantum enhancements of deep neural networks, in general, have not been as extensively investigated as other quantum machine learning methods. In contrast, projective simulation is a reinforcement learning model inspired by the stochastic evolution of physical systems that enables a quantum speed-up in decision making. In this paper, we develop a unifying framework that connects deep learning and projective simulation, opening the route to quantum improvements in deep reinforcement learning. Our approach is based on so-called generative energy-based models to design reinforcement learning methods with a computational advantage in solving complex and large-scale decision-making problems.
Quantum-enhanced machine learning is a rapidly evolving field that aims to leverage the unique properties of quantum mechanics to enhance classical machine learning. However, the practical applicability of these methods remains an open question, particularly beyond the context of specifically crafted toy problems, and given the current limitations of quantum hardware. This study focuses on quantum kernel methods in the context of classification tasks. In particular, it examines the performance of quantum kernel estimation and quantum kernel training (QKT) in connection with two quantum feature mappings, namely, ZZFeatureMap and CovariantFeatureMap. Remarkably, these feature maps have been proposed in the literature under the conjecture of possible near-term quantum advantage and have shown promising performance in ad hoc datasets. This study aims to evaluate their versatility and generalization capabilities in a more general benchmark, encompassing both artificial and established reference datasets. Classical machine learning methods, specifically support vector machines and logistic regression, are also incorporated as baseline comparisons. Experimental results indicate that quantum methods exhibit varying performance across different datasets. Despite outperforming classical methods in ad hoc datasets, mixed results are obtained for the general case among standard classical benchmarks. The experimental data call into question a general added value of applying QKT optimization, for which the additional computational cost does not necessarily translate into improved classification performance. Instead, it is suggested that a careful choice of the quantum feature map in connection with proper hyperparameterization may prove more effective.
Recently, increased computational power and data availability, as well as algorithmic advances, have led machine learning (ML) techniques to impressive results in regression, classification, data generation and reinforcement learning tasks. Despite these successes, the proximity to the physical limits of chip fabrication alongside the increasing size of datasets is motivating a growing number of researchers to explore the possibility of harnessing the power of quantum computation to speed up classical ML algorithms. Here we review the literature in quantum ML and discuss perspectives for a mixed readership of classical ML and quantum computation experts. Particular emphasis will be placed on clarifying the limitations of quantum algorithms, how they compare with their best classical counterparts and why quantum resources are expected to provide advantages for learning problems. Learning in the presence of noise and certain computationally hard problems in ML are identified as promising directions for the field. Practical questions, such as how to upload classical data into quantum form, will also be addressed.
Quantum computers are next-generation computing devices that have the potential to enhance the performance of machine learning.An approach to achieving quantum enhancement is to utilize the Hilbert space, whose dimension scales exponentially in the number of qubits, as a feature space for machine learning.The use of such quantum-enhanced feature space could enable us to find data patterns more easily than conventional methods.Thus far, it has been theoretically shown that quantum-enhanced machine learning models can solve specific classification tasks that classical methods cannot efficiently solve.This motivates many researchers to pursue practical advantages of the methods.Toward practical applications, it is critical to establish guidelines for designs of quantum-enhanced machine learning models.This thesis aims to analyze the performance of quantum-enhanced machine learning models and give insights into design principles of the models for practical applications.More specifically, we focus on two methods, quantum kernel methods and quantum reservoir computing, and then provide guidelines for building their models in practical situations.We focus on two challenges in quantum kernel methods for real-world applications.The first is that constructing quantum feature maps is nontrivial when applied to actual machine learning problems.This thesis proposes a method for analyzing quantum feature maps, which can help screen a suitable quantum feature map among many candidates.In addition, we examine the effectiveness of a synthesis method to construct a powerful quantum kernel.Another issue is that implementation feasibility and trainability deteriorate as the number of qubits increases.We thus propose a new quantum kernel called the quantum Fisher kernel and demonstrate from analytical and numerical perspectives that our proposal can avoid the aforementioned problem when shallow alternating layered ansatzes are used.In quantum reservoir computing, there is room for investigation in designing quantum reservoir systems that are amenable to implementation and can perform well.In this thesis, we propose a quantum reservoir computing framework that positively makes use of unavoidable quantum noise in actual quantum hardware.Our experimental demonstrations on superconducting quantum devices show that quantum noise can enhance temporal information processing.Also, numerical analysis using a tool called temporal information processing capacity elucidates that dissipation noise such as amplitude damping can induce sequential data processing capabilities.
Quantum Machine Learning (QML) represents a paradigm shift in machine learning, leveraging quantum mechanical principles to accelerate computation and potentially unlock new algorithms. This paper investigates a novel reinforcement learning (RL) framework, termed "Adaptive Neural Network Reinforcement Learning" (ANNR), which employs dynamic parameter adjustment within a self-adaptive neural network to enhance learning efficiency and generalization capabilities. We demonstrate that this approach overcomes limitations of traditional RL methods, offering a promising path towards more robust and adaptable reinforcement learning agents. The core mechanism focuses on iteratively refining neural network weights through a feedback loop, enabling the system to explore complex environments effectively. The research explores the impact of parameter adjustment on both training speed and final performance across various RL tasks.