npj Quantum Information

Parameter estimation in quantum sensing based on deep reinforcement learning

Tailong Xiao, Jianping Fan B, Guihua Zeng

Published Jan 10, 2022• 54 citations

Abstract

Abstract Parameter estimation is a pivotal task, where quantum technologies can enhance precision greatly. We investigate the time-dependent parameter estimation based on deep reinforcement learning, where the noise-free and noisy bounds of parameter estimation are derived from a geometrical perspective. We propose a physical-inspired linear time-correlated control ansatz and a general well-defined reward function integrated with the derived bounds to accelerate the network training for fast generating quantum control signals. In the light of the proposed scheme, we validate the performance of time-dependent and time-independent parameter estimation under noise-free and noisy dynamics. In particular, we evaluate the transferability of the scheme when the parameter has a shift from the true parameter. The simulation showcases the robustness and sample efficiency of the scheme and achieves the state-of-the-art performance. Our work highlights the universality and global optimality of deep reinforcement learning over conventional methods in practical parameter estimation of quantum sensing.

DOI: 10.1038/s41534-021-00513-z

Related papers

Universal Quantum Control through Deep Reinforcement Learning

Murphy Yuezhen Niu, Sergio Boixo, Vadim Smelyanskiy +1 more

Abstract Emerging reinforcement learning techniques using deep neural networks have shown great promise in control optimization. They harness non-local regularities of noisy control trajectories and facilitate transfer learning between tasks. To leverage these powerful capabilities for quantum control optimization, we propose a new control framework to simultaneously optimize the speed and fidelity of quantum computation against both leakage and stochastic control errors. For a broad family of two-qubit unitary gates that are important for quantum simulation of many-electron systems, we improve the control robustness by adding control noise into training environments for reinforcement learning agents trained with trusted-region-policy-optimization. The agent control solutions demonstrate a two-order-of-magnitude reduction in average-gate-error over baseline stochastic-gradient-descent solutions and up to a one-order-of-magnitude reduction in gate time from optimal gate synthesis counterparts. These significant improvements in both fidelity and runtime are achieved by combining new physical understandings and state-of-the-art machine learning techniques. Our results open a venue for wider applications in quantum simulation, quantum chemistry and quantum supremacy tests using near-term quantum devices.

arXiv:1803.01857npj Quantum InformationJan 6, 2019387 citations

Generalizable control for quantum parameter estimation through reinforcement learning

Han Xu, Junning Li, Liqiang Liu +3 more

Abstract Measurement and estimation of parameters are essential for science and engineering, where one of the main quests is to find systematic schemes that can achieve high precision. While conventional schemes for quantum parameter estimation focus on the optimization of the probe states and measurements, it has been recently realized that control during the evolution can significantly improve the precision. The identification of optimal controls, however, is often computationally demanding, as typically the optimal controls depend on the value of the parameter which then needs to be re-calculated after the update of the estimation in each iteration. Here we show that reinforcement learning provides an efficient way to identify the controls that can be employed to improve the precision. We also demonstrate that reinforcement learning is highly generalizable, namely the neural network trained under one particular value of the parameter can work for different values within a broad range. These desired features make reinforcement learning an efficient alternative to conventional optimal quantum control methods.

arXiv:1904.11298npj Quantum InformationOct 4, 2019107 citations

Deep reinforcement learning for quantum gate control

Zheng An, D. L. Zhou

How to implement multi-qubit gates efficiently with high precision is essential for realizing universal fault-tolerant computing. For a physical system with some external controllable parameters, it is a great challenge to control the time dependence of these parameters to achieve a target multi-qubit gate efficiently and precisely. Here we construct a dueling double deep Q-learning neural network (DDDQN) to find out the optimized time dependence of controllable parameters to implement two typical quantum gates: a single-qubit Hadamard gate and a two-qubit CNOT gate. Compared with traditional optimal control methods, this deep reinforcement learning method can realize efficient and precise gate control without requiring any gradient information during the learning process. This work attempts to pave the way to investigate more quantum control problems with deep reinforcement learning techniques.

arXiv:1902.08418Europhysics Letters (EPL)Jul 22, 2019123 citations

Deep reinforcement learning for quantum multiparameter estimation

Valeria Cimini, Mauro Valeri, Emanuele Polino +6 more

Estimation of physical quantities is at the core of most scientific research, and the use of quantum devices promises to enhance its performances. In real scenarios, it is fundamental to consider that resources are limited, and Bayesian adaptive estimation represents a powerful approach to efficiently allocate, during the estimation process, all the available resources. However, this framework relies on the precise knowledge of the system model, retrieved with a fine calibration, with results that are often computationally and experimentally demanding. We introduce a model-free and deep-learning-based approach to efficiently implement realistic Bayesian quantum metrology tasks accomplishing all the relevant challenges, without relying on any a priori knowledge of the system. To overcome this need, a neural network is trained directly on experimental data to learn the multiparameter Bayesian update. Then the system is set at its optimal working point through feedback provided by a reinforcement learning algorithm trained to reconstruct and enhance experiment heuristics of the investigated quantum sensor. Notably, we prove experimentally the achievement of higher estimation performances than standard methods, demonstrating the strength of the combination of these two black-box algorithms on an integrated photonic circuit. Our work represents an important step toward fully artificial intelligence-based quantum metrology.

arXiv:2209.00671Advanced PhotonicsFeb 6, 202351 citations

Auxiliary Task-Based Deep Reinforcement Learning for Quantum Control

Shumin Zhou, Hailan Ma, Sen Kuang +1 more

Due to its property of not requiring prior knowledge of the environment, reinforcement learning (RL) has significant potential for solving quantum control problems. In this work, we investigate the effectiveness of continuous control policies based on deep deterministic policy gradient. To achieve good control of quantum systems with high fidelity, we propose an auxiliary task-based deep RL (AT-DRL) for quantum control. In particular, we design an auxiliary task to predict the fidelity value, sharing partial parameters with the main network (from the main RL task). The auxiliary task learns synchronously with the main task, allowing one to extract intrinsic features of the environment, thus aiding the agent to achieve the desired state with high fidelity. To further enhance the control performance, we also design a guided reward function based on the fidelity of quantum states that enables gradual fidelity improvement. Numerical simulations demonstrate that the proposed AT-DRL can provide a good solution to the exploration of quantum dynamics. It not only achieves high task fidelities but also demonstrates fast learning rates. Moreover, AT-DRL has great potential in designing control pulses that achieve effective quantum state preparation.

arXiv:2302.14312IEEE Transactions on CyberneticsJan 7, 20256 citations

Optimal multi-parameter quantum metrology for frequencies of magnetic field

Zhenhua 振华 Long 龙, Shengshi Pang

Abstract Multi-parameter quantum estimation has attracted considerable attention due to its broad applications. Due to the complexity of quantum dynamics, existing research places significant emphasis on estimating parameters in time-independent Hamiltonians. Here, our work makes an effort to explore multi-parameter estimation with time-dependent Hamiltonians. In particular, we focus on the discrimination of two close frequencies of a magnetic field by using a single qubit. We optimize the quantum controls by employing both traditional optimization methods and reinforcement learning to improve the precision for estimating the frequencies of the two magnetic fields. In addition to the estimation precision, we also evaluate the robustness of the optimization schemes against the shift of the control parameters. The results demonstrate that the hybrid reinforcement learning approach achieves the highest estimation precision, and exhibits superior robustness. Moreover, a fundamental challenge in multi-parameter quantum estimation stems from the incompatibility of the optimal control strategies for different parameters. We demonstrate that the hybrid control strategies derived through numerical optimization remain effective in enhancing the precision of multi-parameter estimation in spite of the incompatibilities, thereby mitigating incompatibilities between control strategies on the estimation precision. Finally, we investigate the trade-offs in estimation precision among different parameters for different scenarios, revealing the inherent challenges in balancing the optimization of multiple parameters simultaneously and providing insights into the fundamental distinction between quantum single-parameter estimation and multi-parameter estimation.

Chinese Physics BMay 6, 20251 citation