Quantum computing (QC) in the current NISQ era is still limited in size and precision. Hybrid applications mitigating those shortcomings are prevalent to gain early insight and advantages. Hybrid quantum machine learning (QML) comprises both the application of QC to improve machine learning (ML) and ML to improve QC architectures. This work considers the latter, leveraging reinforcement learning (RL) to improve quantum circuit design (QCD), which we formalize by a set of generic objectives. Furthermore, we propose qcd-gym, a concrete framework formalized as a Markov decision process, to enable learning policies capable of controlling a universal set of continuously parameterized quantum gates. Finally, we provide benchmark comparisons to assess the shortcomings and strengths of current state-of-the-art RL algorithms.
Quantum computing (QC) in the current NISQ era is still limited in size and precision. Hybrid applications mitigating those shortcomings are prevalent to gain early insight and advantages. Hybrid quantum machine learning (QML) comprises both the application of QC to improve machine learning (ML) and ML to improve QC architectures. This work considers the latter, leveraging reinforcement learning (RL) to improve quantum circuit design (QCD), which we formalize by a set of generic objectives. Furthermore, we propose qcd-gym, a concrete framework formalized as a Markov decision process, to enable learning policies capable of controlling a universal set of continuously parameterized quantum gates. Finally, we provide benchmark comparisons to assess the shortcomings and strengths of current state-of-the-art RL algorithms.
To assess the prospects of using reinforcement learning (RL) for selecting and parameterizing quantum gates to build viable circuit architectures, we introduce the quantum circuit designer (QCD). By considering quantum control a decision-making problem, we strive to profit from advanced RL exploration mechanisms to overcome the need for granular specification and hand-crafted architectures. To evaluate current state-of-the-art RL algorithms, we define generic objectives that arise from quantum architecture search and circuit optimization. Those evaluation results reveal challenges inherent to learning optimal quantum control.
The rapid advancement of quantum computing (QC) and machine learning (ML) has given rise to the burgeoning field of quantum machine learning (QML), aiming to capitalize on the strengths of quantum computing to propel ML forward. Despite its promise, crafting effective QML models necessitates profound expertise to strike a delicate balance between model intricacy and feasibility on Noisy Intermediate-Scale Quantum (NISQ) devices. While complex models offer robust representation capabilities, their extensive circuit depth may impede seamless execution on extant noisy quantum platforms. In this paper, we address this quandary of QML model design by employing deep reinforcement learning to explore proficient QML model architectures tailored for designated supervised learning tasks. Specifically, our methodology involves training an RL agent to devise policies that facilitate the discovery of QML models without predetermined ansatz. Furthermore, we integrate an adaptive mechanism to dynamically adjust the learning objectives, fostering continuous improvement in the agent's learning process. Through extensive numerical simulations, we illustrate the efficacy of our approach within the realm of classification tasks. Our proposed method successfully identifies VQC architectures capable of achieving high classification accuracy while minimizing gate depth. This pioneering approach not only advances the study of AI-driven quantum circuit design but also holds significant promise for enhancing performance in the NISQ era.
Research on quantum computing is still in its infancy, but it has a lot of potential uses. One topic with potential is machine learning, namely in the field of reinforcement learning. This work examines the integration of parametrized quantum circuits (PQC) into reinforcement learning (RL) algorithms, assessing the potential of quantum-enhanced models to solve classical RL tasks. It closely follows the example found on the TensorFlow website. This paper reviews applications of quantum reinforcement learning (QRL). We examine PQCs in a standard RL scenario, the CartPole-v1 environment from Gym, using TensorFlow Quantum and Cirq, to evaluate the relative performance of quantum versus conventional models. In comparison to conventional deep neural network (ONN) models, PQCs show slower convergence and higher processing needs, even if they are still able to learn the task and perform competitively. After they are fully trained, the quantum models show unique difficulties during the early training stages and reach a performance stability level like classical methods. This study sheds light on the present constraints as well as possible uses of quantum computing in reinforcement learning, particularly in situations with intricate, high-dimensional settings that prove difficult for classical computers to handle effectively. As we look to the future, we suggest that investigating hybrid quantum-classical algorithms, developing quantum hardware, and using quantum RL for increasingly difficult tasks are essential first steps. The study presents findings from both a classical reinforcement learning algorithm and a quantum integrated reinforcement learning algorithm. To provide a reliable comparison between quantum reinforcement algorithms and their classical equivalents, further work remains. This work lays the groundwork for future advances in the field by investigating the viability and use of quantum algorithms in reinforcement learning, even if it is not particularly unique. The purpose of this work is to help newcomers to this emerging field of study.
Quantum computing promises advantages over classical computing. The manufacturing of quantum hardware is in the infancy stage, called the Noisy Intermediate-Scale Quantum (NISQ) era. A major challenge is automated quantum circuit design that map a quantum circuit to gates in a universal gate set. In this paper, we present a generic MDP modeling and employ Q-learning and DQN algorithms for quantum circuit design. By leveraging the power of deep reinforcement learning, we aim to provide an automatic and scalable approach over traditional hand-crafted heuristic methods.
Quantum computing has promised significant improvement in solving difficult computational tasks over classical computers. Designing quantum circuits for practical use, however, is not a trivial objective and requires expert-level knowledge. To aid this endeavor, this paper proposes a machine learning-based method to construct quantum circuit architectures. Previous works have demonstrated that classical deep reinforcement learning (DRL) algorithms can successfully construct quantum circuit architectures without encoded physics knowledge. However, these DRL-based works are not generalizable to settings with changing device noises, thus requiring considerable amounts of training resources to keep the RL models up-to-date. With this in mind, we incorporated continual learning to enhance the performance of our algorithm. In this paper, we present the Probabilistic Policy Reuse with deep Q-learning (PPR-DQL) framework to tackle this circuit design challenge. By conducting numerical simulations over various noise patterns, we demonstrate that the RL agent with PPR was able to find the quantum gate sequence to generate the two-qubit Bell state faster than the agent that was trained from scratch. The proposed framework is general and can be applied to other quantum gate synthesis or control problems -- including the automatic calibration of quantum devices.
The convergence of quantum computing and artificial intelligence presents profound opportunities but faces significant hurdles, particularly in the Noisy Intermediate-Scale Quantum (NISQ) era. Quantum Machine Learning (QML) algorithms, while promising, exhibit sensitivity to noise and scalability challenges, hindering the demonstration of practical quantum advantage. Concurrently, Quantum Error Correction (QEC), essential for fault tolerance, imposes substantial resource overheads and is often developed generically, without specific adaptation to the target application's error sensitivity. This paper reviews the current state of the intersection of AI and quantum computing, examining both QML paradigms, e.g., Variational Quantum Algorithms, Quantum Kernels, and the burgeoning use of AI to enhance quantum computing itself, e.g., quantum control, QEC decoding, circuit design. A critical gap identified is the lack of frameworks that systematically co-design QML algorithms and QEC protocols adaptively. To address this, a novel framework is proposed based on Reinforcement Learning (RL). This framework employs an RL agent to dynamically adjust both the QML circuit architecture, e.g., VQC ansatz, and QEC parameters, e.g., decoding strategy, measurement frequency, based on observed application performance and estimated error characteristics. This adaptive co-design loop aims to optimize the trade-off between QML performance and QEC overhead, enhancing noise resilience and resource efficiency. The potential advantages, feasibility, and limitations of this approach are discussed, alongside with future research directions aimed at realizing robust and practical Quantum AI.