Reinforcement Learning for Quantum Control

Michèle Desouter-Lecomte

Ming-Zhong Ai, Yongcheng Ding, Yue Ban +7 more

Will Kaufman, B.J. Alford, Pai Peng +3 more

Haixu Yu, Xudong Xu, Hailan Ma +2 more

In recent years, some experimental studies and simulations show that reinforcement learning (RL) is an effective learning control approach for solving certain quantum control problems. In this paper, Q-learning with different exploration strategies (e.g., ε-greedy and Softmax), probabilistic Q-learning (PQL) and quantum reinforcement learning (QRL) are applied to solve the state transition problem of two-level quantum systems (e.g., spin-1/2 systems), respectively. These reinforcement learning algorithms are introduced and analyzed regarding the learning control problem of the spin-1/2 system. According to the constraints of the control fields, two typical kinds of controllers, i.e., three-switch controller and Bang-Bang controller, are designed using reinforcement learning. The learning performance of the above RL algorithms for both of the three-switch control and Bang-Bang control of two-level quantum systems are demonstrated and analyzed.

Shengyong Li, Yidian Fan, Xiangyang Li +6 more

Quantum control requires high-precision and robust control pulses to ensure optimal system performance. However, control sequences generated with a system model may suffer from model bias, leading to low fidelity. While model-free reinforcement learning (RL) methods have been developed to avoid such biases, training an RL agent from scratch can be time-consuming, often taking hours to gather enough samples for convergence. This challenge has hindered the broad application of RL techniques to larger and more complex quantum control issues, limiting their adaptability. In this work, we use Reinforcement Learning from Demonstration (RLfD) to leverage the control sequences generated with system models and further optimize them with RL to avoid model bias. By avoiding learning from scratch and starting with reasonable control pulse shapes, this approach can increase sample efficiency by reducing the number of samples, which can significantly reduce the training time. Thus, this method can effectively handle pulse shapes that are discretized into more than 1000 pieces without compromising final fidelity. We have simulated the preparation of several high-fidelity non-classical states using the RLfD method. We also find that the training process is more stable when using RLfD. In addition, this method is suitable for fast gate calibration using reinforcement learning.

Haixu Yu, Lei Ye, Peng Wang