Quantum control refers to the manipulation of dynamical quantum systems to force them to complete given tasks such as preparing a desired state and tracking a designed trajectory. We consider the state preparation problem of a two-qubit closed quantum system from an initial to a desired state. The aim is to achieve a high fidelity in a given fixed time with limited control resources. The Deep Q-learning (DQL) for solving the quantum state preparation problem is explored in this paper. We propose a novel semi-Markov DQL algorithm based on a modified action selection procedure and improved replay memory to enhance performance of standard DQL algorithm. The proposed algorithm shows high performance for discovering high-fidelity control protocols and for converging to a good policy, compared with standard DQL. The proposed modifications enhance the exploration-exploitation ability for DQL agent and the robustness for solving quantum control problem with high-fidelity at different numbers of control steps. Numerical results on a two-qubit closed system show effectiveness of the proposed algorithm.
Quantum control refers to the manipulation of dynamical quantum systems to force them to complete given tasks such as preparing a desired state and tracking a designed trajectory. We consider the state preparation problem of a two-qubit closed quantum system from an initial to a desired state. The aim is to achieve a high fidelity in a given fixed time with limited control resources. The Deep Q-learning (DQL) for solving the quantum state preparation problem is explored in this paper. We propose a novel semi-Markov DQL algorithm based on a modified action selection procedure and improved replay memory to enhance performance of standard DQL algorithm. The proposed algorithm shows high performance for discovering high-fidelity control protocols and for converging to a good policy, compared with standard DQL. The proposed modifications enhance the exploration-exploitation ability for DQL agent and the robustness for solving quantum control problem with high-fidelity at different numbers of control steps. Numerical results on a two-qubit closed system show effectiveness of the proposed algorithm.
Precise and resilient quantum gate design is important for the building of quantum devices. In this paper, we consider the optimal and robust quantum gate design problem for three classes of two-level quantum systems. The aim is to construct quantum gates in a given fixed time with limited control resources. A modified dueling deep Q-learning (MDuDQL) is employed for the optimal and robust gate design problem. To improve the performance of the classical DuDQL method, we propose a unique semi-Markov DuDQL algorithm based on a modified action selection procedure, modified replay memory, and soft update procedure. The proposed algorithm outperforms ordinary DuDQL in terms of discovering global optimal or near-global optimal control protocols and faster convergence to a better policy. Moreover, the modified DuDQL agent shows improved performance in finding robust control protocols which achieve high-fidelity quantum gate design for varying uncertainties in a certain range. The effectiveness of the proposed algorithm for the optimal and robust gate design problems has been illustrated by numerical results.
Reinforcement learning (RL) is considered a powerful technology with the potential to revolutionize quantum control. However, the application effectiveness of traditional RL is often limited by some insurmountable experimental conditions. Thus, developing new RL algorithms that can efficiently manipulate the quantum system dynamics is a crucial task. Prior research has shown that incorporating quantum mechanical properties into RL can improve learning performance. In this article, we consider the quantum control problem where only the target state can be accurately identified and introduce a quantum-inspired RL (QiRL) method. In particular, we propose a quantum-inspired exploration strategy to replace a commonly used$\epsilon $-greedy strategy, as well as a quantum-inspired reward scheme to incentivize the learning agent. Numerical results on three quantum system control problems, i.e., one-qubit closed quantum system, two-level open quantum system, and many-qubit closed quantum system, verify the effectiveness of QiRL. Comparison results show that the proposed QiRL outperforms existing RL algorithms (deep Q-network and proximal policy optimization) in terms of stability and efficiency for solving quantum control problems.
Due to the large state space of the two-qubit system, and the adoption of ladder reward function in the existing quantum state preparation methods, the convergence speed is slow and it is difficult to prepare the desired target quantum state with high fidelity under limited conditions. To solve the above problems, a difference-driven reinforcement learning (RL) algorithm for quantum state preparation of two-qubit system is proposed by improving the reward function and action selection strategy. Firstly, a model is constructed for the problem of preparing quantum states of a two-qubit system, with restrictions on the type of quantum gates and the time for quantum state evolution. In the preparation process, a weighted differential dynamic reward function is designed to assist the algorithm quickly obtain the maximum expected cumulative reward. Then, an adaptive [Formula: see text]-greedy action selection strategy is adopted to achieve a balance between exploration and utilization to a certain extent, thereby improving the fidelity of the final quantum state. The simulation results show that the proposed algorithm can prepare quantum state with high fidelity under limited conditions. Compared with other algorithms, it has different degrees of improvement in convergence speed and fidelity of the final quantum state.
This paper investigates quantum control problems using tabular Q-learning. A modified tabular Q-learning algorithm based on dynamic greedy method is proposed and the proposed algorithm succeeds for finding control sequences to drive a two-qubit system to a given target state with high fidelity. The modified algorithm also shows improved performance over the traditional Q-learning for solving quantum control problems on continuous states space. Moreover, the modified tabular Q-learning algorithm is compared with stochastic gradient descent and Krotov algorithms for solving quantum control problems. Simulation results on a two-qubit system demonstrate the effectiveness of the proposed algorithm.
Abstract The preparation of quantum states is essential in the realm of quantum information processing, and the development of efficient methodologies can significantly alleviate the strain on quantum resources. Under the framework of deep reinforcement learning (DRL), we integrate the initial and the target state information within the state preparation task together, so as to realize the control trajectory design between two arbitrary quantum states. Taking a semiconductor double quantum dots (DQDs) model as an example, our results demonstrate that the resulting control trajectories can effectively achieve arbitrary quantum state preparation (AQSP) for both single-qubit and two-qubit systems, with average fidelities of 0.9868 and 0.9556 for the test sets, respectively. For the DQDs, charge and nuclear noises will exist, thus decreases the average fidelity of AQSP. We have also incorporated noise amplitude as an input feature in the training process, leading to the development of noise-aware models. The control trajectories designed by these models can effectively suppress the influence of noise. Our research validates the effectiveness of DRL in quantum state preparation and offers solutions for multi-initial and multi-objective quantum control tasks, with potential for broader application in quantum control problems.
The progress in machine learning during the last decade has had a considerable impact on many areas of science and technology, including quantum technology. This work explores the application of reinforcement learning (RL) methods to the quantum control problem of state transfer in a single qubit. The goal is to create an RL agent that learns an optimal policy and thus discovers optimal pulses to control the qubit. The most crucial step is to mathematically formulate the problem of interest as a Markov decision process (MDP). This enables the use of RL algorithms to solve the quantum control problem. Deep learning and the use of deep neural networks provide the freedom to employ continuous action and state spaces, offering the expressivity and generalization of the process. This flexibility helps to formulate the quantum state transfer problem as an MDP in several different ways. All the developed methodologies are applied to the fundamental problem of population inversion in a qubit. In most cases, the derived optimal pulses achieve fidelity equal to or higher than 0.9999, as required by quantum computing applications. The present methods can be easily extended to quantum systems with more energy levels and may be used for the efficient control of collections of qubits and to counteract the effect of noise, which are important topics for quantum sensing applications.