Can quantum mechanics help us build intelligent learning agents?A defining signature of intelligent behavior is the capacity to learn from experience.However, a major bottleneck for agents to learn in reallife situations is the size and complexity of the corresponding task environment.Even in a moderately realistic environment, it may simply take too long to rationally respond to a given situation.If the environment is impatient, allowing only a certain time for a response, an agent may then be unable to cope with the situation and to learn at all.Here, we show that quantum physics can help and provide a quadratic speedup for active learning as a genuine problem of artificial intelligence.This result will be particularly relevant for applications involving complex task environments.
Can quantum mechanics help us build intelligent learning agents?A defining signature of intelligent behavior is the capacity to learn from experience.However, a major bottleneck for agents to learn in reallife situations is the size and complexity of the corresponding task environment.Even in a moderately realistic environment, it may simply take too long to rationally respond to a given situation.If the environment is impatient, allowing only a certain time for a response, an agent may then be unable to cope with the situation and to learn at all.Here, we show that quantum physics can help and provide a quadratic speedup for active learning as a genuine problem of artificial intelligence.This result will be particularly relevant for applications involving complex task environments.
Combining quantum computing techniques in the form of amplitude amplification with classical reinforcement learning has led to the so-called “hybrid agent for quantum-accessible reinforcement learning”, which achieves a quadratic speedup in sample complexity for certain learning problems. So far, this hybrid agent has only been applied to stationary learning problems, that is, learning problems without any time dependency within components of the Markov decision process. In this work, we investigate the applicability of the hybrid agent to dynamic RL environments and thus to more realistic scenarios. To this end, we enhance the hybrid agent by introducing a dissipation mechanism and, with the resulting learning agent, perform an empirical comparison with a classical RL agent in an RL environment with a time-dependent reward function. Our findings suggest that the modified hybrid agent can adapt its behavior to changes in the environment quickly, leading to a higher average success probability compared to its classical counterpart.
Abstract In the last decade quantum machine learning has provided fascinating and fundamental improvements to supervised, unsupervised and reinforcement learning (RL). In RL, a so-called agent is challenged to solve a task given by some environment. The agent learns to solve the task by exploring the environment and exploiting the rewards it gets from the environment. For some classical task environments, an analogue quantum environment can be constructed which allows to find rewards quadratically faster by applying quantum algorithms. In this paper, we analytically analyze the behavior of a hybrid agent which combines this quadratic speedup in exploration with the policy update of a classical agent. This leads to a faster learning of the hybrid agent compared to the classical agent. We demonstrate that if the classical agent needs on average ⟨J⟩ rewards and ⟨T⟩cl epochs to learn how to solve the task, the hybrid agent will take ⟨ T ⟩ q ⩽ α s α o ⟨ T ⟩ c l ⟨ J ⟩ epochs on average. Here, α s and α o denote constants depending on details of the quantum search and are independent of the problem size. Additionally, we prove that if the environment allows for maximally α o k max sequential coherent interactions, e.g. due to noise effects, an improvement given by ⟨T⟩q ≈ α o ⟨T⟩cl/(4k max) is still possible.
As the field of artificial intelligence is pushed forward, the question arises of how fast autonomous machines can learn. Within artificial intelligence, an important paradigm is reinforcement learning, where agents - learning entities capable of decision making - interact with the world they are placed in, called an environment. Thanks to these interactions, agents receive feedback from the environment and thus progressively adjust their behaviour to accomplish a given goal. An important question in reinforcement learning is how fast agents can learn to fulfill their tasks. To answer this question we consider a novel reinforcement learning framework where quantum mechanics is used. In particular, we quantize the agent and the environment and grant them the possibility to also interact quantum-mechanically, that is, by using a quantum channel for their communication. We demonstrate that this feature enables a speed-up in the agent's learning process, and we further show that combining this scenario with classical communication enables the evaluation of such an improvement. This learning protocol is implemented on an integrated re-programmable photonic platform interfaced with photons at telecommunication wavelengths. Thanks to the full tunability of the device, this platform proves the best candidate for the implementation of learning protocols, where a continuous update of the learning process is required.
Quantum computers can offer dramatic improvements over classical devices for data analysis tasks such as prediction and classification. However, less is known about the advantages that quantum computers may bring in the setting of reinforcement learning, where learning is achieved via interaction with a task environment. Here, we consider a special case of reinforcement learning, where the task environment allows quantum access. In addition, we impose certain "naturalness" conditions on the task environment, which rule out the kinds of oracle problems that are studied in quantum query complexity (and for which quantum speedups are well-known). Within this framework of quantum-accessible reinforcement learning environments, we demonstrate that quantum agents can achieve exponential improvements in learning efficiency, surpassing previous results that showed only quadratic improvements. A key step in the proof is to construct task environments that encode well-known oracle problems, such as Simon's problem and Recursive Fourier Sampling, while satisfying the above "naturalness" conditions for reinforcement learning. Our results suggest that quantum agents may perform well in certain game-playing scenarios, where the game has recursive structure, and the agent can learn by playing against itself.
Quantum information technologies, on the one hand, and intelligent learning systems, on the other, are both emergent technologies that are likely to have a transformative impact on our society in the future. The respective underlying fields of basic research-quantum information versus machine learning (ML) and artificial intelligence (AI)-have their own specific questions and challenges, which have hitherto been investigated largely independently. However, in a growing body of recent work, researchers have been probing the question of the extent to which these fields can indeed learn and benefit from each other. Quantum ML explores the interaction between quantum computing and ML, investigating how results and techniques from one field can be used to solve the problems of the other. Recently we have witnessed significant breakthroughs in both directions of influence. For instance, quantum computing is finding a vital application in providing speed-ups for ML problems, critical in our 'big data' world. Conversely, ML already permeates many cutting-edge technologies and may become instrumental in advanced quantum technologies. Aside from quantum speed-up in data analysis, or classical ML optimization used in quantum experiments, quantum enhancements have also been (theoretically) demonstrated for interactive learning tasks, highlighting the potential of quantum-enhanced learning agents. Finally, works exploring the use of AI for the very design of quantum experiments and for performing parts of genuine research autonomously, have reported their first successes. Beyond the topics of mutual enhancement-exploring what ML/AI can do for quantum physics and vice versa-researchers have also broached the fundamental issue of quantum generalizations of learning and AI concepts. This deals with questions of the very meaning of learning and intelligence in a world that is fully described by quantum mechanics. In this review, we describe the main ideas, recent developments and progress in a broad spectrum of research investigating ML and AI in the quantum domain.
In recent times, there has been much interest in quantum enhancements of machine learning, specifically in the context of data mining and analysis. Reinforcement learning, an interactive form of learning, is, in turn, vital in artificial intelligence-type applications. Also in this case, quantum mechanics was shown to be useful, in certain instances. Here, we elucidate these results, and show that quantum enhancements can be achieved in a new setting: the setting of learning models which learn how to improve themselves - that is, those that metalearn. While not all learning models meta-learn, all non-trivial models have the potential of being "lifted", enhanced, to meta-learning models. Our results show that also such models can be quantum-enhanced to make even better learners. In parallel, we address one of the bottlenecks of current quantum reinforcement learning approaches: the need for so-called oracularized variants of task environments. Here we elaborate on a method which realizes these variants, with minimal changes in the setting, and with no corruption of the operative specification of the environments. This result may be important in near-term experimental demonstrations of quantum reinforcement learning.