Reinforcement learning has been intensively applied to tackle complex quantum control problems owing to its adaptability in dynamic environments. However, commonly used reinforcement learning algorithms are restricted to selecting actions from a discrete action space, which may result in inaccurate control. In order to surpass this constraint, we propose an improved continuous reinforcement learning algorithm that can generate control policies in a continuous action space, enabling precise control of quantum systems. Moreover, a guided reward function design method is proposed to guide the learning process toward higher fidelity. Numerical results demonstrate that the proposed continuous reinforcement learning algorithm with a guided reward function can capably prepare states on one-qubit and two-qubit systems.
Reinforcement learning has been intensively applied to tackle complex quantum control problems owing to its adaptability in dynamic environments. However, commonly used reinforcement learning algorithms are restricted to selecting actions from a discrete action space, which may result in inaccurate control. In order to surpass this constraint, we propose an improved continuous reinforcement learning algorithm that can generate control policies in a continuous action space, enabling precise control of quantum systems. Moreover, a guided reward function design method is proposed to guide the learning process toward higher fidelity. Numerical results demonstrate that the proposed continuous reinforcement learning algorithm with a guided reward function can capably prepare states on one-qubit and two-qubit systems.
Due to its property of not requiring prior knowledge of the environment, reinforcement learning (RL) has significant potential for solving quantum control problems. In this work, we investigate the effectiveness of continuous control policies based on deep deterministic policy gradient. To achieve good control of quantum systems with high fidelity, we propose an auxiliary task-based deep RL (AT-DRL) for quantum control. In particular, we design an auxiliary task to predict the fidelity value, sharing partial parameters with the main network (from the main RL task). The auxiliary task learns synchronously with the main task, allowing one to extract intrinsic features of the environment, thus aiding the agent to achieve the desired state with high fidelity. To further enhance the control performance, we also design a guided reward function based on the fidelity of quantum states that enables gradual fidelity improvement. Numerical simulations demonstrate that the proposed AT-DRL can provide a good solution to the exploration of quantum dynamics. It not only achieves high task fidelities but also demonstrates fast learning rates. Moreover, AT-DRL has great potential in designing control pulses that achieve effective quantum state preparation.
This paper investigates the application of quantum-inspired algorithms to the field of reinforcement learning (RL). Traditional RL methods often struggle with exploration challenges and can be computationally expensive, particularly in complex environments. We propose a novel RL algorithm that leverages concepts from quantum mechanics, specifically superposition and entanglement, to address these limitations. The core idea is to represent states, actions, and rewards using quantum-inspired representations, allowing the agent to explore the environment more efficiently and potentially converge to optimal policies faster. This work presents a detailed formulation of the algorithm, including the mathematical framework and key considerations for implementation. The potential benefits of this approach, including improved exploration and reduced computational complexity, are discussed. The algorithm is designed to iteratively refine its policy based on simulated quantum measurements, aiming to minimize the expected return. The exploration strategy is guided by the inherent probabilistic nature of quantum states, leading to a more diverse and potentially faster learning process. Further research is warranted to fully explore the capabilities and limitations of this approach across various RL benchmark problems.
Reinforcement learning (RL) is considered a powerful technology with the potential to revolutionize quantum control. However, the application effectiveness of traditional RL is often limited by some insurmountable experimental conditions. Thus, developing new RL algorithms that can efficiently manipulate the quantum system dynamics is a crucial task. Prior research has shown that incorporating quantum mechanical properties into RL can improve learning performance. In this article, we consider the quantum control problem where only the target state can be accurately identified and introduce a quantum-inspired RL (QiRL) method. In particular, we propose a quantum-inspired exploration strategy to replace a commonly used$\epsilon $-greedy strategy, as well as a quantum-inspired reward scheme to incentivize the learning agent. Numerical results on three quantum system control problems, i.e., one-qubit closed quantum system, two-level open quantum system, and many-qubit closed quantum system, verify the effectiveness of QiRL. Comparison results show that the proposed QiRL outperforms existing RL algorithms (deep Q-network and proximal policy optimization) in terms of stability and efficiency for solving quantum control problems.
This paper presents a novel feedback learning algorithm for adaptive quantum control. The algorithm leverages reinforcement learning principles combined with quantum control theory to dynamically adjust control strategies based on the observed state and environment of a quantum system. The core mechanism involves constructing an adaptive quantum control feedback learning algorithm that enables autonomous optimization of the quantum system. The algorithm utilizes a state representation of the quantum system, a reward function reflecting the desired outcome, and a control policy updated through a reinforcement learning process. Mathematical formulations are provided to describe the system dynamics, the control policy, and the learning process. The proposed algorithm aims to overcome the limitations of traditional fixed-parameter quantum control methods by adapting to uncertainties and complex dynamics, ultimately leading to improved control performance and robustness. The theoretical framework presented here offers a foundation for developing advanced quantum control strategies in various applications, including quantum computation, quantum sensing, and quantum metrology.
Deep reinforcement learning (DRL) has been widely used in many quantum control tasks, in which rewards are used to guide the agent in acquiring an optimal control strategy. However, designing suitable rewards is usually challenging, which seriously affects the DRL's application performance, especially for complex quantum control tasks. In this article, we propose a simple and general reward design method by constructing some events, where events are defined by fidelity intervals. In the proposed event-based DRL (EDRL), events are used to describe the evolution of a quantum system. Rewards are designed according to several defined events rather than a large number of specific evolved quantum states, thereby resulting in a reduction in dimensions to be considered. With designed event-attributed rewards, the agent can capture the main evolution characteristics of a quantum system, and then obtain an effective control strategy. The effectiveness of the proposed EDRL method is verified by three quantum systems: i.e., 1) the one-qubit closed quantum system, 2) the two-qubit closed quantum system, and 3) the two-level open quantum system. Numerical comparisons demonstrate that the proposed EDRL outperforms the traditional DRL algorithms (deep Q-network and proximal policy optimization) with some representative reward methods for solving quantum control problems.
Deep reinforcement learning (DRL) has been recognized as a powerful tool in quantum physics, where DRL's reward design is nontrivial but crucial for quantum control tasks. To address the problem of over-reliance on human empirical knowledge to design DRL's rewards, we propose a DRL with a novel reward paradigm designed by the learning process information (DRL-LPI), where the learning process information (LPI) comprises the state information and the experiences. In DRL-LPI, the state information after being classified by a fidelity threshold, and the experiences are first stored simultaneously in the respective sequences, and this process is repeated until a similar-segment ends. Then, the stored state information is converted to the real value and used to design the reward value by applying a self-amplitude function. Next, the designed reward values are integrated with the stored experiences to compose transitions for DRL's training. Through comparisons to five representative reward schemes, the proposed DRL-LPI is evaluated on two typical quantum control tasks, i.e., the spin-(1/2) quantum state control and many-coupled qubits state control, and the experimental results show the superior learning efficiency and control performance of the proposed approach. More results show that DRL-LPI exhibits the ability to learn the control strategy with few control actions compared to stochastic gradient descent (SGD) and genetic algorithm (GA).