Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:1904.11298 · npj Quantum Information

Generalizable control for quantum parameter estimation through reinforcement learning

Han Xu, Junning Li, Liqiang Liu, Yu Wang, Haidong Yuan, Xin Wang

Published Oct 4, 2019• 107 citations
arXivHTMLPDF

Abstract

Abstract Measurement and estimation of parameters are essential for science and engineering, where one of the main quests is to find systematic schemes that can achieve high precision. While conventional schemes for quantum parameter estimation focus on the optimization of the probe states and measurements, it has been recently realized that control during the evolution can significantly improve the precision. The identification of optimal controls, however, is often computationally demanding, as typically the optimal controls depend on the value of the parameter which then needs to be re-calculated after the update of the estimation in each iteration. Here we show that reinforcement learning provides an efficient way to identify the controls that can be employed to improve the precision. We also demonstrate that reinforcement learning is highly generalizable, namely the neural network trained under one particular value of the parameter can work for different values within a broad range. These desired features make reinforcement learning an efficient alternative to conventional optimal quantum control methods.

DOI
10.1038/s41534-019-0198-z

Related papers

Generalizable control for multiparameter quantum metrology

Han Xu, Lingna Wang, Haidong Yuan +1 more

Quantum control can be employed in quantum metrology to improve the precision limit for the estimation of unknown parameters. The optimal control, however, typically depends on the actual values of the parameters and thus needs to be designed adaptively with the updated estimations of those parameters. Traditional methods, such as gradient ascent pulse engineering (GRAPE), need to be rerun for each new set of parameters encountered, making the optimization costly, especially when many parameters are involved. Here we study the generalizability of optimal control, namely, optimal controls that can be systematically updated across a range of parameters with minimal cost. In cases where control channels can completely reverse the shift in the Hamiltonian due to a change in parameters, we provide an analytical method which efficiently generates optimal controls for any parameter starting from an initial optimal control found by either GRAPE or reinforcement learning. When the control channels are restricted, the analytical scheme is invalid, but reinforcement learning still retains a level of generalizability, albeit in a narrower range. In cases where the shift in the Hamiltonian is impossible to decompose to available control channels, no generalizability is found for either the reinforcement learning or the analytical scheme. We argue that the generalization of reinforcement learning is through a mechanism similar to the analytical scheme. Our results provide insights into when and how the optimal control in multiparameter quantum metrology can be generalized, thereby facilitating efficient implementation of optimal quantum estimation of multiple parameters, particularly for an ensemble of systems with ranges of parameters.

arXiv:2012.13377•Physical Review A•Apr 29, 2021•26 citations

High dimensional quantum optimal control with Reinforcement Learning

Zheng An, Qi-Kai He, Hai-Jing Song +1 more

Manipulate and control of the complex quantum system with high precision are essential for achieving universal fault tolerant quantum computing. For a physical system with restricted control resources, it is a challenge to control the dynamics of the target system efficiently and precisely under disturbances. Here we propose a high dimensional quantum control framework and show that deep reinforcement learning provides an efficient way to identify the optimal strategies with restricted control parameters of the complex quantum system. This framework can be generalized to be applied to other quantum control models. Compared with the traditional optimal control method, this deep reinforcement learning algorithm can realize efficient and precise control for high dimensional quantum systems with different types of disturbances.

•Jul 2, 2020•2 citations

Application of machine learning to experimental design in quantum mechanics

Federico Belliardo, Fabio Zoratti, Vittorio Giovannetti

The recent advances in Machine Learning hold great promises for the field of quantum sensing and metrology. With the help of reinforcement learning, we can tame the complexity of quantum systems and solve the problem of optimal experimental design. Reinforcement learning is a powerful model-free technique that allows an agent, which is typically a neural network, to learn the best strategy to reach a certain goal in a completely a priori unknown environment. However, in general, we know something about the quantum system the agent is interacting with, at least that it follows the rules of quantum mechanics. In quantum metrology, we typically have a model for the system and only some parameters of the evolution or of the initial state are unknown. We present here a general Machine Learning technique that can optimize the precision of quantum sensors, and in doing so it exploits the knowledge we have on the system. We have developed a Python package to automate a broad class of optimizations that can be found in the tasks of quantum parameter estimation, quantum metrology and quantum hypothesis testing. What the agent is learning here is an optimal adaptive strategy, that, on the basis of the previous outcomes, decides the next measurements to perform. It works both for Bayesian estimation and for frequentist estimation. The user is required to implement the physics of the system to be studied and state which parameters in the experiment are controllable and which are unknowns. The functions of the library allow then to easily train a neural network agent for optimizing the precision of the sensor, by simulating the experiment. We have explored some applications of this technique to magnetometry on NV centers (both DC and AC), to state discrimination in quantum optics and to phase estimation. So far, we were able to certify better results than the current state-of-the-art controls for many examples. The Machine Learning technique developed here can be applied in all those scenarios where the quantum system is well characterized and relatively simple and small. In these cases, we can squeeze every last bit of information from a quantum sensor by controlling it with a neural network appropriately trained.

arXiv:2403.10317•International Journal of Quantum Information•Jan 20, 2024•4 citations

Parameter estimation in quantum sensing based on deep reinforcement learning

Tailong Xiao, Jianping Fan B, Guihua Zeng

Abstract Parameter estimation is a pivotal task, where quantum technologies can enhance precision greatly. We investigate the time-dependent parameter estimation based on deep reinforcement learning, where the noise-free and noisy bounds of parameter estimation are derived from a geometrical perspective. We propose a physical-inspired linear time-correlated control ansatz and a general well-defined reward function integrated with the derived bounds to accelerate the network training for fast generating quantum control signals. In the light of the proposed scheme, we validate the performance of time-dependent and time-independent parameter estimation under noise-free and noisy dynamics. In particular, we evaluate the transferability of the scheme when the parameter has a shift from the true parameter. The simulation showcases the robustness and sample efficiency of the scheme and achieves the state-of-the-art performance. Our work highlights the universality and global optimality of deep reinforcement learning over conventional methods in practical parameter estimation of quantum sensing.

npj Quantum Information•Jan 10, 2022•54 citations

Deep reinforcement learning for quantum gate control

Zheng An, D. L. Zhou

How to implement multi-qubit gates efficiently with high precision is essential for realizing universal fault-tolerant computing. For a physical system with some external controllable parameters, it is a great challenge to control the time dependence of these parameters to achieve a target multi-qubit gate efficiently and precisely. Here we construct a dueling double deep Q-learning neural network (DDDQN) to find out the optimized time dependence of controllable parameters to implement two typical quantum gates: a single-qubit Hadamard gate and a two-qubit CNOT gate. Compared with traditional optimal control methods, this deep reinforcement learning method can realize efficient and precise gate control without requiring any gradient information during the learning process. This work attempts to pave the way to investigate more quantum control problems with deep reinforcement learning techniques.

arXiv:1902.08418•Europhysics Letters (EPL)•Jul 22, 2019•123 citations

Adaptive Quantum Control Algorithm Optimization

Jincheng Zhang

This paper investigates the optimization of quantum control algorithms using reinforcement learning (RL). The core claim is that RL can significantly enhance the control precision and efficiency of quantum devices. We propose a framework where the quantum control algorithm is treated as an intelligent agent, interacting with the quantum environment to learn optimal control strategies. This approach allows for adaptive control, addressing the challenges inherent in designing fixed control protocols for complex quantum systems. The methodology involves defining a reward function that reflects the desired quantum state evolution, using RL algorithms to train the agent, and validating the learned control strategy through simulations. We demonstrate the potential of this approach to improve control accuracy and robustness, particularly in scenarios with noisy environments or rapidly changing system parameters. The results suggest a promising direction for leveraging machine learning techniques to unlock the full potential of quantum control.

•Sep 6, 2026•0 citations
Abstract Measurement and estimation of parameters are essential for science and engineering, where one of the main quests is to find systematic schemes that can achieve high precision. While conventional schemes for quantum parameter estimation focus on the optimization of the probe states and measurements, it has been recently realized that control during the evolution can significantly improve the precision. The identification of optimal controls, however, is often computationally demanding, as typically the optimal controls depend on the value of the parameter which then needs to be re-calculated after the update of the estimation in each iteration. Here we show that reinforcement learning provides an efficient way to identify the controls that can be employed to improve the precision. We also demonstrate that reinforcement learning is highly generalizable, namely the neural network trained under one particular value of the parameter can work for different values within a broad range. These desired features make reinforcement learning an efficient alternative to conventional optimal quantum control methods.
Quantum control can be employed in quantum metrology to improve the precision limit for the estimation of unknown parameters. The optimal control, however, typically depends on the actual values of the parameters and thus needs to be designed adaptively with the updated estimations of those parameters. Traditional methods, such as gradient ascent pulse engineering (GRAPE), need to be rerun for each new set of parameters encountered, making the optimization costly, especially when many parameters are involved. Here we study the generalizability of optimal control, namely, optimal controls that can be systematically updated across a range of parameters with minimal cost. In cases where control channels can completely reverse the shift in the Hamiltonian due to a change in parameters, we provide an analytical method which efficiently generates optimal controls for any parameter starting from an initial optimal control found by either GRAPE or reinforcement learning. When the control channels are restricted, the analytical scheme is invalid, but reinforcement learning still retains a level of generalizability, albeit in a narrower range. In cases where the shift in the Hamiltonian is impossible to decompose to available control channels, no generalizability is found for either the reinforcement learning or the analytical scheme. We argue that the generalization of reinforcement learning is through a mechanism similar to the analytical scheme. Our results provide insights into when and how the optimal control in multiparameter quantum metrology can be generalized, thereby facilitating efficient implementation of optimal quantum estimation of multiple parameters, particularly for an ensemble of systems with ranges of parameters.
Manipulate and control of the complex quantum system with high precision are essential for achieving universal fault tolerant quantum computing. For a physical system with restricted control resources, it is a challenge to control the dynamics of the target system efficiently and precisely under disturbances. Here we propose a high dimensional quantum control framework and show that deep reinforcement learning provides an efficient way to identify the optimal strategies with restricted control parameters of the complex quantum system. This framework can be generalized to be applied to other quantum control models. Compared with the traditional optimal control method, this deep reinforcement learning algorithm can realize efficient and precise control for high dimensional quantum systems with different types of disturbances.
The recent advances in Machine Learning hold great promises for the field of quantum sensing and metrology. With the help of reinforcement learning, we can tame the complexity of quantum systems and solve the problem of optimal experimental design. Reinforcement learning is a powerful model-free technique that allows an agent, which is typically a neural network, to learn the best strategy to reach a certain goal in a completely a priori unknown environment. However, in general, we know something about the quantum system the agent is interacting with, at least that it follows the rules of quantum mechanics. In quantum metrology, we typically have a model for the system and only some parameters of the evolution or of the initial state are unknown. We present here a general Machine Learning technique that can optimize the precision of quantum sensors, and in doing so it exploits the knowledge we have on the system. We have developed a Python package to automate a broad class of optimizations that can be found in the tasks of quantum parameter estimation, quantum metrology and quantum hypothesis testing. What the agent is learning here is an optimal adaptive strategy, that, on the basis of the previous outcomes, decides the next measurements to perform. It works both for Bayesian estimation and for frequentist estimation. The user is required to implement the physics of the system to be studied and state which parameters in the experiment are controllable and which are unknowns. The functions of the library allow then to easily train a neural network agent for optimizing the precision of the sensor, by simulating the experiment. We have explored some applications of this technique to magnetometry on NV centers (both DC and AC), to state discrimination in quantum optics and to phase estimation. So far, we were able to certify better results than the current state-of-the-art controls for many examples. The Machine Learning technique developed here can be applied in all those scenarios where the quantum system is well characterized and relatively simple and small. In these cases, we can squeeze every last bit of information from a quantum sensor by controlling it with a neural network appropriately trained.
Abstract Parameter estimation is a pivotal task, where quantum technologies can enhance precision greatly. We investigate the time-dependent parameter estimation based on deep reinforcement learning, where the noise-free and noisy bounds of parameter estimation are derived from a geometrical perspective. We propose a physical-inspired linear time-correlated control ansatz and a general well-defined reward function integrated with the derived bounds to accelerate the network training for fast generating quantum control signals. In the light of the proposed scheme, we validate the performance of time-dependent and time-independent parameter estimation under noise-free and noisy dynamics. In particular, we evaluate the transferability of the scheme when the parameter has a shift from the true parameter. The simulation showcases the robustness and sample efficiency of the scheme and achieves the state-of-the-art performance. Our work highlights the universality and global optimality of deep reinforcement learning over conventional methods in practical parameter estimation of quantum sensing.
How to implement multi-qubit gates efficiently with high precision is essential for realizing universal fault-tolerant computing. For a physical system with some external controllable parameters, it is a great challenge to control the time dependence of these parameters to achieve a target multi-qubit gate efficiently and precisely. Here we construct a dueling double deep Q-learning neural network (DDDQN) to find out the optimized time dependence of controllable parameters to implement two typical quantum gates: a single-qubit Hadamard gate and a two-qubit CNOT gate. Compared with traditional optimal control methods, this deep reinforcement learning method can realize efficient and precise gate control without requiring any gradient information during the learning process. This work attempts to pave the way to investigate more quantum control problems with deep reinforcement learning techniques.
This paper investigates the optimization of quantum control algorithms using reinforcement learning (RL). The core claim is that RL can significantly enhance the control precision and efficiency of quantum devices. We propose a framework where the quantum control algorithm is treated as an intelligent agent, interacting with the quantum environment to learn optimal control strategies. This approach allows for adaptive control, addressing the challenges inherent in designing fixed control protocols for complex quantum systems. The methodology involves defining a reward function that reflects the desired quantum state evolution, using RL algorithms to train the agent, and validating the learned control strategy through simulations. We demonstrate the potential of this approach to improve control accuracy and robustness, particularly in scenarios with noisy environments or rapidly changing system parameters. The results suggest a promising direction for leveraging machine learning techniques to unlock the full potential of quantum control.