We consider the problem of frequency estimation for a single bosonic field evolving under a squeezing Hamiltonian and continuously monitored via homodyne detection.In particular, we exploit reinforcement learning techniques to devise feedback control strategies achieving increased estimation precision.We show that the feedback control determined by the neural network greatly surpasses in the long-time limit the performances of both the "no-control" strategy and the standard "open-loop control" strategy, which we considered as benchmarks.We indeed observe how the devised strategy is able to optimize the nontrivial estimation problem by preparing a large fraction of trajectories corresponding to more sensitive quantum conditional states.
We consider the problem of frequency estimation for a single bosonic field evolving under a squeezing Hamiltonian and continuously monitored via homodyne detection.In particular, we exploit reinforcement learning techniques to devise feedback control strategies achieving increased estimation precision.We show that the feedback control determined by the neural network greatly surpasses in the long-time limit the performances of both the "no-control" strategy and the standard "open-loop control" strategy, which we considered as benchmarks.We indeed observe how the devised strategy is able to optimize the nontrivial estimation problem by preparing a large fraction of trajectories corresponding to more sensitive quantum conditional states.
Reinforcement learning algorithms have been shown to generate procedures for executing quantum control tasks to desired performances. Although reinforcement learning has been effective, its robustness in generating quantum control procedures under general noise condition needs to be tested. Here we consider adaptive quantum-enhanced interferometric phase estimation as a case study in quantum feedback control. The algorithm is determined to deliver a robust quantum-metrological procedure under various phase-noise models when the imprecision surpasses the standard quantum limit. The robustness against unknown environmental variations positions the reinforcement learning as a practical approach for devising quantum control procedures.
The effectiveness of measurement-based feedback control protocols is hampered by the presence of measurement noise, which affects the ability to accurately infer the underlying dynamics of a quantum system from noisy continuous measurement records to determine an accurate control strategy. To circumvent such limitations, this Letter explores a real-time stochastic state estimation approach that enables noise-free monitoring of the conditional dynamics including the full density matrix of the quantum system using noisy measurement records within a single quantum trajectory-a method we name as "conditional state tomography." This, in turn, enables the development of precise measurement-based feedback control strategies that lead to effective control of quantum systems by essentially mitigating the constraints imposed by measurement noise and has potential applications in various feedback quantum control scenarios. This approach is particularly useful for reinforcement-learning-(RL) based control, where the RL-agent can be trained with arbitrary conditional averages of observables, and/or the full density matrix as input (observation), to quickly and accurately learn control strategies.
In this thesis, we consider two simple but typical control problems and apply deep reinforcement learning to them, i.e., to cool and control a particle which is subject to continuous position measurement in a one-dimensional quadratic potential or in a quartic potential. We compare the performance of reinforcement learning control and conventional control strategies on the two problems, and show that the reinforcement learning achieves a performance comparable to the optimal control for the quadratic case, and outperforms conventional control strategies for the quartic case for which the optimal control strategy is unknown. To our knowledge, this is the first time deep reinforcement learning is applied to quantum control problems in continuous real space. Our research demonstrates that deep reinforcement learning can be used to control a stochastic quantum system in real space effectively as a measurement-feedback closed-loop controller, and our research also shows the ability of AI to discover new control strategies and properties of the quantum systems that are not well understood, and we can gain insights into these problems by learning from the AI, which opens up a new regime for scientific research.
Closed loop quantum control uses measurement to control the dynamics of a quantum system to achieve either a desired target state or target dynamics. In the case when the quantum Hamiltonian is quadratic in x and p, there are known optimal control techniques to drive the dynamics toward particular states, e.g., the ground state. However, for nonlinear Hamiltonian such control techniques often fail. We apply deep reinforcement learning (DRL), where an artificial neural agent explores and learns to control the quantum evolution of a highly nonlinear system (double well), driving the system toward the ground state with high fidelity. We consider a DRL strategy which is particularly motivated by experiment where the quantum system is continuously but weakly measured. This measurement is then fed back to the neural agent and used for training. We show that the DRL can effectively learn counterintuitive strategies to cool the system to a nearly pure "cat" state, which has a high overlap fidelity with the true ground state.
The recent advances in Machine Learning hold great promises for the field of quantum sensing and metrology. With the help of reinforcement learning, we can tame the complexity of quantum systems and solve the problem of optimal experimental design. Reinforcement learning is a powerful model-free technique that allows an agent, which is typically a neural network, to learn the best strategy to reach a certain goal in a completely a priori unknown environment. However, in general, we know something about the quantum system the agent is interacting with, at least that it follows the rules of quantum mechanics. In quantum metrology, we typically have a model for the system and only some parameters of the evolution or of the initial state are unknown. We present here a general Machine Learning technique that can optimize the precision of quantum sensors, and in doing so it exploits the knowledge we have on the system. We have developed a Python package to automate a broad class of optimizations that can be found in the tasks of quantum parameter estimation, quantum metrology and quantum hypothesis testing. What the agent is learning here is an optimal adaptive strategy, that, on the basis of the previous outcomes, decides the next measurements to perform. It works both for Bayesian estimation and for frequentist estimation. The user is required to implement the physics of the system to be studied and state which parameters in the experiment are controllable and which are unknowns. The functions of the library allow then to easily train a neural network agent for optimizing the precision of the sensor, by simulating the experiment. We have explored some applications of this technique to magnetometry on NV centers (both DC and AC), to state discrimination in quantum optics and to phase estimation. So far, we were able to certify better results than the current state-of-the-art controls for many examples. The Machine Learning technique developed here can be applied in all those scenarios where the quantum system is well characterized and relatively simple and small. In these cases, we can squeeze every last bit of information from a quantum sensor by controlling it with a neural network appropriately trained.
We investigate critical quantum metrology, that is, the estimation of parameters in many-body systems close to a quantum critical point, through the lens of Bayesian inference theory. We first derive a no-go result stating that any nonadaptive strategy will fail to exploit quantum critical enhancement (i.e., precision beyond the shot-noise limit) for a sufficiently large number of particles N whenever our prior knowledge is limited. We then consider different adaptive strategies that can overcome this no-go result and illustrate their performance in the estimation of (i) a magnetic field using a probe of 1D spin Ising chain and (ii) the coupling strength in a Bose-Hubbard square lattice. Our results show that adaptive strategies with real-time feedback control can achieve sub-shot-noise scaling even with few measurements and substantial prior uncertainty.