Accurate control of quantum states is crucial for quantum computing and other quantum technologies. In the basic scenario, the task is to steer a quantum system towards a target state through a sequence of control operations. Determining the appropriate operations, however, generally requires information about the initial state of the system. Gathering this information becomes increasingly challenging when the initial state is not a priori known and the system’s size grows large. To address this problem, we develop a machine-learning algorithm that uses a small amount of measurement data to construct its internal representation of the system’s state. The algorithm compares this data-driven representation with a representation of the target state, and uses reinforcement learning to output the appropriate control operations. We illustrate the effectiveness of the algorithm showing that it achieves accurate control of unknown many-body quantum states and non-Gaussian continuous-variable states using data from a limited set of quantum measurements.
Accurate control of quantum states is crucial for quantum computing and other quantum technologies. In the basic scenario, the task is to steer a quantum system towards a target state through a sequence of control operations. Determining the appropriate operations, however, generally requires information about the initial state of the system. Gathering this information becomes increasingly challenging when the initial state is not a priori known and the system’s size grows large. To address this problem, we develop a machine-learning algorithm that uses a small amount of measurement data to construct its internal representation of the system’s state. The algorithm compares this data-driven representation with a representation of the target state, and uses reinforcement learning to output the appropriate control operations. We illustrate the effectiveness of the algorithm showing that it achieves accurate control of unknown many-body quantum states and non-Gaussian continuous-variable states using data from a limited set of quantum measurements.
There is no doubt that quantum computing has opened up new horizons and perspectives in many fields, including scientific research. With the advancement of technology and quantum computers, we can now conduct new kinds of scientific experiments: observing quantum properties of individual systems, atoms, electrons and photons as well as influencing and controlling quantum systems. Quantum mechanics describes the properties of quantum systems by their quantum states. Unlike in classical mechanics, quantum states themselves cannot be directly observed in experiments, so many-body dynamics mathematical simulations are useful. Even though many-body dynamics problems can be solved for limited cases, but when we consider real-world pieces of matter this leads to computation-intensive calculations that are beyond classic digital computers. The memory and time needed to describe the quantum state of a many-body system scales exponentially with the size of the system. This has led physics communities to focus their attention on the algorithms underlying modern machine learning with the goal of making progress in quantum matter research. As a consequence, the successful application of machine learning requires effective and informative data representation. This paper discusses some novel aspects of applying classical machine learning algorithms to quantum data derived from measurements associated with quantum systems in many-body simulation problems. In this paper, we investigate and experimentally demonstrate one of the effective methods of the quantum data representation suited for the classical machine learning applications based on quantum classical shadows.
Reinforcement Learning (RL) techniques have been increasingly applied in optimizing control systems. However, their application in quantum systems is hampered by the challenge of performing closed-loop control due to the difficulty in measuring these systems. This often leads to reliance on assumed models, introducing model bias, a problem that is exacerbated in open quantum dynamics where Markovian approximations are not valid. To address these challenges, we propose a novel approach that incorporates the non-Markovian nature of the environment into a low-dimensional effective reservoir. By initially employing a series of measurements as a 'dataset', we utilize machine learning techniques to learn the effective quantum dynamics more efficiently than traditional tomographic methods. Our methodology aims to demonstrates that by integrating reinforcement learning with model learning, it is possible to devise control policies and models that can counteract decoherence in a spin-boson system. This approach may not only mitigates the issues of model bias but also provides a more accurate representation of quantum dynamics, paving the way for more effective quantum control strategies.
Neural networks are a promising tool for characterizing intermediate-scale quantum devices from limited amounts of measurement data. A challenging problem in this area is to learn the action of an unknown quantum process on an ensemble of physically relevant input states. To tackle this problem, we introduce a neural network that emulates the unknown process by constructing an internal representation of the input ensemble and by mimicking the action of the process at the state representation level. After being trained with measurement data from a few pairs of input/output quantum states, the network becomes able to predict the measurement statistics for all inputs in the ensemble of interest. We show that our model exhibits high accuracy in applications to quantum computing, quantum photonics, and quantum many-body physics.
Characterization of quantum systems from experimental data is a central problem in quantum science and technology. But which measurements should be used to gather data in the first place? While optimal measurement choices can be worked out for small quantum systems, the optimization becomes intractable as the system size grows large. To address this problem, we introduce a deep neural network with a sequence model architecture that searches for efficient measurement choices in a data-driven, adaptive manner. The model can be applied to a variety of tasks, including the prediction of linear and nonlinear properties of quantum states, as well as state clustering and state tomography tasks. In all these tasks, we find that the measurement choices identified by our neural network consistently outperform the uniformly random choice. Intriguingly, for topological quantum systems, our model tends to recommend measurements at the system's boundaries, even when the task is to predict bulk properties. This behavior suggests that the neural network may have independently discovered a connection between boundaries and bulk, without having been provided any built-in knowledge of quantum physics.
Reinforcement learning (RL) is considered a powerful technology with the potential to revolutionize quantum control. However, the application effectiveness of traditional RL is often limited by some insurmountable experimental conditions. Thus, developing new RL algorithms that can efficiently manipulate the quantum system dynamics is a crucial task. Prior research has shown that incorporating quantum mechanical properties into RL can improve learning performance. In this article, we consider the quantum control problem where only the target state can be accurately identified and introduce a quantum-inspired RL (QiRL) method. In particular, we propose a quantum-inspired exploration strategy to replace a commonly used$\epsilon $-greedy strategy, as well as a quantum-inspired reward scheme to incentivize the learning agent. Numerical results on three quantum system control problems, i.e., one-qubit closed quantum system, two-level open quantum system, and many-qubit closed quantum system, verify the effectiveness of QiRL. Comparison results show that the proposed QiRL outperforms existing RL algorithms (deep Q-network and proximal policy optimization) in terms of stability and efficiency for solving quantum control problems.
Many challenges arising in Quantum Technology can be successfully addressed using a set of machine learning algorithms collectively known as reinforcement learning (RL), based on adaptive decision-making through interaction with the quantum device. After a concise and intuitive introduction to RL aimed at a broad physics readership, we discuss the key ideas and core concepts in reinforcement learning with a particular focus on quantum systems. We then survey recent progress in RL in all relevant areas. We discuss state preparation in few- and many-body quantum systems, the design and optimization of high-fidelity quantum gates, and the automated construction of quantum circuits, including applications to variational quantum eigensolvers and architecture search. We further highlight the interactive capabilities of RL agents, emphasizing recent progress in quantum feedback control and quantum error correction, and briefly discuss quantum reinforcement learning as well as applications to quantum metrology. The review concludes with a discussion of open challenges -- such as scalability, interpretability, and integration with experimental platforms -- and outlines promising directions for future research. Throughout, we highlight experimental implementations that exemplify the increasing role of reinforcement learning in shaping the development of quantum technologies.