Quantum metrology promises unprecedented measurement precision but suffers in practice from the limited availability of resources such as the number of probes, their coherence time, or nonclassical quantum states.The adaptive Bayesian approach to parameter estimation allows an efficient use of resources because of adaptive experiment design.For its practical success, fast numerical solutions for the Bayesian update and the adaptive experiment design are crucial.Here we show that neural networks can be trained to become fast and strong experiment-design heuristics using a combination of an evolutionary strategy and reinforcement learning.Neural-network heuristics are shown to outperform established heuristics for the technologically important example of frequency estimation of a qubit that suffers from dephasing.Our method of creating neural-network heuristics is very general and complements the well-studied sequential Monte Carlo method for Bayesian updates to form a complete framework for adaptive Bayesian quantum estimation.
Quantum metrology promises unprecedented measurement precision but suffers in practice from the limited availability of resources such as the number of probes, their coherence time, or nonclassical quantum states.The adaptive Bayesian approach to parameter estimation allows an efficient use of resources because of adaptive experiment design.For its practical success, fast numerical solutions for the Bayesian update and the adaptive experiment design are crucial.Here we show that neural networks can be trained to become fast and strong experiment-design heuristics using a combination of an evolutionary strategy and reinforcement learning.Neural-network heuristics are shown to outperform established heuristics for the technologically important example of frequency estimation of a qubit that suffers from dephasing.Our method of creating neural-network heuristics is very general and complements the well-studied sequential Monte Carlo method for Bayesian updates to form a complete framework for adaptive Bayesian quantum estimation.
Estimation of physical quantities is at the core of most scientific research, and the use of quantum devices promises to enhance its performances. In real scenarios, it is fundamental to consider that resources are limited, and Bayesian adaptive estimation represents a powerful approach to efficiently allocate, during the estimation process, all the available resources. However, this framework relies on the precise knowledge of the system model, retrieved with a fine calibration, with results that are often computationally and experimentally demanding. We introduce a model-free and deep-learning-based approach to efficiently implement realistic Bayesian quantum metrology tasks accomplishing all the relevant challenges, without relying on any a priori knowledge of the system. To overcome this need, a neural network is trained directly on experimental data to learn the multiparameter Bayesian update. Then the system is set at its optimal working point through feedback provided by a reinforcement learning algorithm trained to reconstruct and enhance experiment heuristics of the investigated quantum sensor. Notably, we prove experimentally the achievement of higher estimation performances than standard methods, demonstrating the strength of the combination of these two black-box algorithms on an integrated photonic circuit. Our work represents an important step toward fully artificial intelligence-based quantum metrology.
The recent advances in Machine Learning hold great promises for the field of quantum sensing and metrology. With the help of reinforcement learning, we can tame the complexity of quantum systems and solve the problem of optimal experimental design. Reinforcement learning is a powerful model-free technique that allows an agent, which is typically a neural network, to learn the best strategy to reach a certain goal in a completely a priori unknown environment. However, in general, we know something about the quantum system the agent is interacting with, at least that it follows the rules of quantum mechanics. In quantum metrology, we typically have a model for the system and only some parameters of the evolution or of the initial state are unknown. We present here a general Machine Learning technique that can optimize the precision of quantum sensors, and in doing so it exploits the knowledge we have on the system. We have developed a Python package to automate a broad class of optimizations that can be found in the tasks of quantum parameter estimation, quantum metrology and quantum hypothesis testing. What the agent is learning here is an optimal adaptive strategy, that, on the basis of the previous outcomes, decides the next measurements to perform. It works both for Bayesian estimation and for frequentist estimation. The user is required to implement the physics of the system to be studied and state which parameters in the experiment are controllable and which are unknowns. The functions of the library allow then to easily train a neural network agent for optimizing the precision of the sensor, by simulating the experiment. We have explored some applications of this technique to magnetometry on NV centers (both DC and AC), to state discrimination in quantum optics and to phase estimation. So far, we were able to certify better results than the current state-of-the-art controls for many examples. The Machine Learning technique developed here can be applied in all those scenarios where the quantum system is well characterized and relatively simple and small. In these cases, we can squeeze every last bit of information from a quantum sensor by controlling it with a neural network appropriately trained.
Abstract Bayesian estimation is a powerful theoretical paradigm for the operation of the approach to parameter estimation. However, the Bayesian method for statistical inference generally suffers from demanding calibration requirements that have so far restricted its use to systems that can be explicitly modeled. In this theoretical study, we formulate parameter estimation as a classification task and use artificial neural networks to efficiently perform Bayesian estimation. We show that the network’s posterior distribution is centered at the true (unknown) value of the parameter within an uncertainty given by the inverse Fisher information, representing the ultimate sensitivity limit for the given apparatus. When only a limited number of calibration measurements are available, our machine-learning-based procedure outperforms standard calibration methods. Our machine-learning-based procedure is model independent, and is thus well suited to “black-box sensors”, which lack simple explicit fitting models. Thus, our work paves the way for Bayesian quantum sensors that can take advantage of complex nonclassical quantum states and/or adaptive protocols. These capabilities can significantly enhance the sensitivity of future devices.
Quantum sensors offer control flexibility during estimation by allowing manipulation by the experimenter across various parameters. For each sensing platform, pinpointing the optimal controls to enhance the sensor's precision remains a challenging task. While an analytical solution might be out of reach, machine learning offers a promising avenue for many systems of interest, especially given the capabilities of contemporary hardware. We have introduced a versatile procedure capable of optimizing a wide range of problems in quantum metrology, estimation, and hypothesis testing by combining model-aware reinforcement learning (RL) with Bayesian estimation based on particle filtering. To achieve this, we had to address the challenge of incorporating the many non-differentiable steps of the estimation in the training process, such as measurements and the resampling of the particle filter. Model-aware RL is a gradient-based method, where the derivatives of the sensor's precision are obtained through automatic differentiation (AD) in the simulation of the experiment. Our approach is suitable for optimizing both non-adaptive and adaptive strategies, using neural networks or other agents. We provide an implementation of this technique in the form of a Python library called qsensoropt, alongside several pre-made applications for relevant physical platforms, namely NV centers, photonic circuits, and optical cavities. This library will be released soon on PyPI. Leveraging our method, we've achieved results for many examples that surpass the current state-of-the-art in experimental design. In addition to Bayesian estimation, leveraging model-aware RL, it is also possible to find optimal controls for the minimization of the Cramér-Rao bound, based on Fisher information.
An important practical problem in the field of quantum metrology and sensors is to find the optimal sequences of controls for the quantum probe that realize optimal adaptive estimation. In Belliardo et al. [arXiv:2312.16985], we solved this problem in general by introducing a procedure capable of optimizing a wide range of tasks in quantum metrology and estimation by combining model-aware reinforcement learning with Bayesian inference. We take a model-based approach to the optimization where the physics describing the system is explicitly taken into account in the training through automatic differentiation. In this follow-up paper we present some applications of the framework. The first family of examples concerns the estimation of magnetic fields, hyperfine interactions, and decoherence times for electronic spins in diamond. For these examples, we perform multiple Ramsey measurements on the spin. The second family of applications concerns the estimation of phases and coherent states on photonic circuits, without squeezing elements, where the bosonic lines are measured by photon counters. This exposition showcases the broad applicability of the method, which has been implemented in the qsensoropt library released on PyPI, which can be installed with pip.
In this work, we study Bayesian quantum parameter estimation given a finite number of uses of the process encoding one or more unknown physical quantities. We analyze the performance of different kinds of quantum metrological protocols, including the conventional parallel, sequential, or general, which may involve an indefinite causal order. We also analyze a class of hybrid adaptive greedy strategies, which are based on classical feedforward between quantum protocols to optimize the next round. Within each class, the central question is to determine the optimal strategy -- namely, the choice of optimal input state, control operations, measurement, and estimators. Using the formalism of higher-order operations, we develop an algorithm that searches for the optimal solution, and we provide a numerical implementation based on semidefinite programming. Our benchmark cases, specifically those against existing analytical solutions, demonstrate how powerful and precise our method is. We further demonstrate the strength of our algorithm in several examples, from single to multiparameter estimation, and with various prior distributions. Particularly, we find examples where the adaptive greedy strategy matches the performance of general strategies with indefinite causal order, and at the same time examples that showcase a strict hierarchy between all different classes.