An important practical problem in the field of quantum metrology and sensors is to find the optimal sequences of controls for the quantum probe that realize optimal adaptive estimation. In Belliardo et al. [arXiv:2312.16985], we solved this problem in general by introducing a procedure capable of optimizing a wide range of tasks in quantum metrology and estimation by combining model-aware reinforcement learning with Bayesian inference. We take a model-based approach to the optimization where the physics describing the system is explicitly taken into account in the training through automatic differentiation. In this follow-up paper we present some applications of the framework. The first family of examples concerns the estimation of magnetic fields, hyperfine interactions, and decoherence times for electronic spins in diamond. For these examples, we perform multiple Ramsey measurements on the spin. The second family of applications concerns the estimation of phases and coherent states on photonic circuits, without squeezing elements, where the bosonic lines are measured by photon counters. This exposition showcases the broad applicability of the method, which has been implemented in the qsensoropt library released on PyPI, which can be installed with pip.
An important practical problem in the field of quantum metrology and sensors is to find the optimal sequences of controls for the quantum probe that realize optimal adaptive estimation. In Belliardo et al. [arXiv:2312.16985], we solved this problem in general by introducing a procedure capable of optimizing a wide range of tasks in quantum metrology and estimation by combining model-aware reinforcement learning with Bayesian inference. We take a model-based approach to the optimization where the physics describing the system is explicitly taken into account in the training through automatic differentiation. In this follow-up paper we present some applications of the framework. The first family of examples concerns the estimation of magnetic fields, hyperfine interactions, and decoherence times for electronic spins in diamond. For these examples, we perform multiple Ramsey measurements on the spin. The second family of applications concerns the estimation of phases and coherent states on photonic circuits, without squeezing elements, where the bosonic lines are measured by photon counters. This exposition showcases the broad applicability of the method, which has been implemented in the qsensoropt library released on PyPI, which can be installed with pip.
Quantum sensors offer control flexibility during estimation by allowing manipulation by the experimenter across various parameters. For each sensing platform, pinpointing the optimal controls to enhance the sensor's precision remains a challenging task. While an analytical solution might be out of reach, machine learning offers a promising avenue for many systems of interest, especially given the capabilities of contemporary hardware. We have introduced a versatile procedure capable of optimizing a wide range of problems in quantum metrology, estimation, and hypothesis testing by combining model-aware reinforcement learning (RL) with Bayesian estimation based on particle filtering. To achieve this, we had to address the challenge of incorporating the many non-differentiable steps of the estimation in the training process, such as measurements and the resampling of the particle filter. Model-aware RL is a gradient-based method, where the derivatives of the sensor's precision are obtained through automatic differentiation (AD) in the simulation of the experiment. Our approach is suitable for optimizing both non-adaptive and adaptive strategies, using neural networks or other agents. We provide an implementation of this technique in the form of a Python library called qsensoropt, alongside several pre-made applications for relevant physical platforms, namely NV centers, photonic circuits, and optical cavities. This library will be released soon on PyPI. Leveraging our method, we've achieved results for many examples that surpass the current state-of-the-art in experimental design. In addition to Bayesian estimation, leveraging model-aware RL, it is also possible to find optimal controls for the minimization of the Cramér-Rao bound, based on Fisher information.
The recent advances in Machine Learning hold great promises for the field of quantum sensing and metrology. With the help of reinforcement learning, we can tame the complexity of quantum systems and solve the problem of optimal experimental design. Reinforcement learning is a powerful model-free technique that allows an agent, which is typically a neural network, to learn the best strategy to reach a certain goal in a completely a priori unknown environment. However, in general, we know something about the quantum system the agent is interacting with, at least that it follows the rules of quantum mechanics. In quantum metrology, we typically have a model for the system and only some parameters of the evolution or of the initial state are unknown. We present here a general Machine Learning technique that can optimize the precision of quantum sensors, and in doing so it exploits the knowledge we have on the system. We have developed a Python package to automate a broad class of optimizations that can be found in the tasks of quantum parameter estimation, quantum metrology and quantum hypothesis testing. What the agent is learning here is an optimal adaptive strategy, that, on the basis of the previous outcomes, decides the next measurements to perform. It works both for Bayesian estimation and for frequentist estimation. The user is required to implement the physics of the system to be studied and state which parameters in the experiment are controllable and which are unknowns. The functions of the library allow then to easily train a neural network agent for optimizing the precision of the sensor, by simulating the experiment. We have explored some applications of this technique to magnetometry on NV centers (both DC and AC), to state discrimination in quantum optics and to phase estimation. So far, we were able to certify better results than the current state-of-the-art controls for many examples. The Machine Learning technique developed here can be applied in all those scenarios where the quantum system is well characterized and relatively simple and small. In these cases, we can squeeze every last bit of information from a quantum sensor by controlling it with a neural network appropriately trained.
Quantum sensors offer significant advantages over classical devices in spatial resolution and sensitivity, enabling transformative applications across materials science, healthcare, and beyond. Their practical performance, however, is often constrained by unmodelled effects, including noise, imperfect state preparation, and non-ideal control fields. In this work, we report the first experimental implementation of a graybox modelling strategy for a solid-state open quantum system. The graybox framework integrates a physics-based system model with a data-driven description of experimental imperfections, achieving higher fidelity than purely analytical (whitebox) approaches while requiring fewer training resources than fully deep-learning blackbox models. We experimentally validate the method on the task of estimating a static magnetic field using a single-spin quantum sensor, performing Bayesian inference with a graybox model trained on prior experimental data. Using roughly 10,000 training datapoints, the graybox model yields several orders of magnitude improvement in mean squared error over the corresponding physics-only model, as well as significant improvement over a blackbox model with comparable size. These results are broadly applicable to a wide range of quantum sensing platforms, not limited to single-spin systems, and are particularly valuable for real-time adaptive protocols, where model inaccuracies can otherwise lead to suboptimal control and degraded performance.
Estimation of physical quantities is at the core of most scientific research, and the use of quantum devices promises to enhance its performances. In real scenarios, it is fundamental to consider that resources are limited, and Bayesian adaptive estimation represents a powerful approach to efficiently allocate, during the estimation process, all the available resources. However, this framework relies on the precise knowledge of the system model, retrieved with a fine calibration, with results that are often computationally and experimentally demanding. We introduce a model-free and deep-learning-based approach to efficiently implement realistic Bayesian quantum metrology tasks accomplishing all the relevant challenges, without relying on any a priori knowledge of the system. To overcome this need, a neural network is trained directly on experimental data to learn the multiparameter Bayesian update. Then the system is set at its optimal working point through feedback provided by a reinforcement learning algorithm trained to reconstruct and enhance experiment heuristics of the investigated quantum sensor. Notably, we prove experimentally the achievement of higher estimation performances than standard methods, demonstrating the strength of the combination of these two black-box algorithms on an integrated photonic circuit. Our work represents an important step toward fully artificial intelligence-based quantum metrology.
The advancement and scaling of quantum technology has made the learning and identification of quantum systems and devices in highly multidimensional parameter spaces a pressing task for a variety of applications. In many cases, the integration of real-time feedback control and adaptive choice of measurement settings places strict demands on the speed of this task. Here we present a joint model selection and parameter estimation algorithm that is fast and operable on a large number of model parameters. The algorithm is based on variational Bayesian inference, which approximates the target posterior distribution by optimizing a tractable family of distributions, making it more scalable than exact inference methods relying on sampling and that generally suffer from high variance and computational cost in high-dimensional spaces. We show how a regularizing prior can be used to select between competing models, each comprising a different number of parameters, identifying the simplest model that explains the experimental data. The regularization can further separate the degrees of freedom, e.g., quantum systems in the environment or processes, which contribute to major features in the observed dynamics, with respect to others featuring small coupling, which only contribute to a background. As an application of the introduced framework, we consider the problem of the identification of multiple individual nuclear spins with a single electron spin quantum sensor, relevant for nanoscale nuclear magnetic resonance and for the implementation of multiqubit quantum networking nodes. With the number of environmental spins unknown a priori, our Bayesian approach is able to correctly identify the model, i.e., the number of spins and their couplings. We benchmark the algorithm on both simulated and experimental data, using standard figures of merit, and demonstrate that we can estimate dozens of parameters within minutes. Our methodology is compatible with the implementation of real-time adaptive choice of experimental settings in multiparameter estimation, which has the potential to greatly decrease the estimation time in a variety of metrology, tomography, and characterization tasks by prioritizing the measurements with the highest content of information.
Quantum metrology promises unprecedented measurement precision but suffers in practice from the limited availability of resources such as the number of probes, their coherence time, or nonclassical quantum states.The adaptive Bayesian approach to parameter estimation allows an efficient use of resources because of adaptive experiment design.For its practical success, fast numerical solutions for the Bayesian update and the adaptive experiment design are crucial.Here we show that neural networks can be trained to become fast and strong experiment-design heuristics using a combination of an evolutionary strategy and reinforcement learning.Neural-network heuristics are shown to outperform established heuristics for the technologically important example of frequency estimation of a qubit that suffers from dephasing.Our method of creating neural-network heuristics is very general and complements the well-studied sequential Monte Carlo method for Bayesian updates to form a complete framework for adaptive Bayesian quantum estimation.