Abstract Some problems in physics can be handled only after a suitable ansatz solution has been guessed, proving to be resilient to generalization. The coherent transport of a quantum state by adiabatic passage through an array of semiconductor quantum dots is an excellent example of such a problem, where it is necessary to introduce a so-called counterintuitive control sequence. Instead, the deep reinforcement learning (DRL) technique has proven to be able to solve very complex sequential decision-making problems, despite a lack of prior knowledge. We show that DRL discovers a control sequence that outperforms the counterintuitive control sequence. DRL can even discover novel strategies when realistic disturbances affect an ideal system, such as detuning or when dephasing or losses are added to the master equation. DRL is effective in controlling the dynamics of quantum states and, more generally, whenever an ansatz solution is unknown or insufficient to effectively treat the problem.
Abstract Some problems in physics can be handled only after a suitable ansatz solution has been guessed, proving to be resilient to generalization. The coherent transport of a quantum state by adiabatic passage through an array of semiconductor quantum dots is an excellent example of such a problem, where it is necessary to introduce a so-called counterintuitive control sequence. Instead, the deep reinforcement learning (DRL) technique has proven to be able to solve very complex sequential decision-making problems, despite a lack of prior knowledge. We show that DRL discovers a control sequence that outperforms the counterintuitive control sequence. DRL can even discover novel strategies when realistic disturbances affect an ideal system, such as detuning or when dephasing or losses are added to the master equation. DRL is effective in controlling the dynamics of quantum states and, more generally, whenever an ansatz solution is unknown or insufficient to effectively treat the problem.
Abstract Several tasks involving the determination of the time evolution of a system of solid state qubits require stochastic methods in order to identify the best sequence of gates and the time of interaction among the qubits. The major success of deep learning in several scientific disciplines has suggested its application to quantum information as well. Thanks to its capability to identify best strategy in those problems involving a competition between the short term and the long term rewards, reinforcement learning (RL) method has been successfully applied, for instance, to discover sequences of quantum gate operations minimizing the information loss. In order to extend the application of RL to the transfer of quantum information, we focus on Coherent Transport by Adiabatic Passage (CTAP) on a chain of three semiconductor quantum dots (QD). This task is usually performed by the so called counter-intuitive sequence of gate pulses. Such sequence is capable of coherently transfer an electronic population from the first to the last site of an odd chain of QDs, by leaving the central QD unpopulated. We apply a technique to find nearly optimal gate pulse sequence without explicitly give any prior knowledge of the underlying physical system to the RL agent. Using the advantage actor-critic algorithm, with a small neural net as function approximator, we trained a RL agent to choose the best action at every time step of the physical evolution to achieve the same results previously found only by ansatz solutions.
Engineering desired Hamiltonians in quantum many-body systems is essential for applications such as quantum simulation, computation, and sensing. Many widely used quantum Hamiltonian engineering sequences are designed using human intuition based on perturbation theory, which may not describe the optimal solution and is unable to accommodate complex experimental imperfections. Here we numerically search for Hamiltonian engineering sequences using deep reinforcement learning (DRL) techniques and experimentally demonstrate that they outperform celebrated decoupling sequences on a solid-state nuclear magnetic resonance quantum simulator. As an example, we aim at decoupling strongly interacting spin-1/2 systems. We train DRL agents in the presence of different experimental imperfections and verify robustness of the output sequences both in simulations and experiments. Surprisingly, many of the learned sequences exhibit a common pattern that had not been discovered before, to our knowledge, but has a meaningful analytical description. We can thus restrict the searching space based on this control pattern, allowing us to search for longer sequences, ultimately leading to sequences that are robust against dominant imperfections in our experiments. Our results not only demonstrate a general method for quantum Hamiltonian engineering, but also highlight the importance of combining black-box artificial intelligence with an understanding of the physical system in order to realize experimentally feasible applications.
Deep reinforcement learning (DRL) has been recognized as an efficient technique to design optimal strategies for different complex systems without prior knowledge of the control landscape. To achieve a fast and precise control for quantum systems, we propose a novel DRL approach by constructing a curriculum consisting of a set of intermediate tasks defined by fidelity thresholds, where the tasks among a curriculum can be statically determined before the learning process or dynamically generated during the learning process. By transferring knowledge between two successive tasks and sequencing tasks according to their difficulties, the proposed curriculum-based DRL (CDRL) method enables the agent to focus on easy tasks in the early stage, then move onto difficult tasks, and eventually approaches the final task. Numerical comparison with the traditional methods [gradient method (GD), genetic algorithm (GA), and several other DRL methods] demonstrates that CDRL exhibits improved control performance for quantum systems and also provides an efficient way to identify optimal strategies with few control pulses.
Due to its property of not requiring prior knowledge of the environment, reinforcement learning (RL) has significant potential for solving quantum control problems. In this work, we investigate the effectiveness of continuous control policies based on deep deterministic policy gradient. To achieve good control of quantum systems with high fidelity, we propose an auxiliary task-based deep RL (AT-DRL) for quantum control. In particular, we design an auxiliary task to predict the fidelity value, sharing partial parameters with the main network (from the main RL task). The auxiliary task learns synchronously with the main task, allowing one to extract intrinsic features of the environment, thus aiding the agent to achieve the desired state with high fidelity. To further enhance the control performance, we also design a guided reward function based on the fidelity of quantum states that enables gradual fidelity improvement. Numerical simulations demonstrate that the proposed AT-DRL can provide a good solution to the exploration of quantum dynamics. It not only achieves high task fidelities but also demonstrates fast learning rates. Moreover, AT-DRL has great potential in designing control pulses that achieve effective quantum state preparation.
Abstract The preparation of quantum states is essential in the realm of quantum information processing, and the development of efficient methodologies can significantly alleviate the strain on quantum resources. Under the framework of deep reinforcement learning (DRL), we integrate the initial and the target state information within the state preparation task together, so as to realize the control trajectory design between two arbitrary quantum states. Taking a semiconductor double quantum dots (DQDs) model as an example, our results demonstrate that the resulting control trajectories can effectively achieve arbitrary quantum state preparation (AQSP) for both single-qubit and two-qubit systems, with average fidelities of 0.9868 and 0.9556 for the test sets, respectively. For the DQDs, charge and nuclear noises will exist, thus decreases the average fidelity of AQSP. We have also incorporated noise amplitude as an input feature in the training process, leading to the development of noise-aware models. The control trajectories designed by these models can effectively suppress the influence of noise. Our research validates the effectiveness of DRL in quantum state preparation and offers solutions for multi-initial and multi-objective quantum control tasks, with potential for broader application in quantum control problems.
In this thesis, we consider two simple but typical control problems and apply deep reinforcement learning to them, i.e., to cool and control a particle which is subject to continuous position measurement in a one-dimensional quadratic potential or in a quartic potential. We compare the performance of reinforcement learning control and conventional control strategies on the two problems, and show that the reinforcement learning achieves a performance comparable to the optimal control for the quadratic case, and outperforms conventional control strategies for the quartic case for which the optimal control strategy is unknown. To our knowledge, this is the first time deep reinforcement learning is applied to quantum control problems in continuous real space. Our research demonstrates that deep reinforcement learning can be used to control a stochastic quantum system in real space effectively as a measurement-feedback closed-loop controller, and our research also shows the ability of AI to discover new control strategies and properties of the quantum systems that are not well understood, and we can gain insights into these problems by learning from the AI, which opens up a new regime for scientific research.