Asian Control Conference (ASCC)

Deep Reinforcement Learning for Control Design of Quantum Gates

Shouliang Hu, Chunlin Chen, Daoyi Dong

Published May 4, 2022• 4 citations

Abstract

This paper investigates quantum gate control problems using the deep reinforcement learning algorithm, i.e., a model-free machine learning method. We implement the twin delayed deep deterministic policy gradient (TD3) algorithm to search for piece-wise constant control pulses for quantum gates through the trail interaction with the quantum system. Simulation results on four typical gates, including three one-qubit gates and a two-qubit CNOT gate, demonstrate that deep reinforcement learning exhibits improved performance for quantum gate control tasks. By setting punishment for steps in the reward function, DRL can automatically find a shorter control sequence than the traditional gradient-based algorithm (e.g., GRAPE algorithm) and the evolutionary algorithm (e.g., DE algorithm) while maintaining high control precision.

DOI: 10.23919/ascc56756.2022.9828135

Related papers

Deep reinforcement learning for quantum gate control

Zheng An, D. L. Zhou

How to implement multi-qubit gates efficiently with high precision is essential for realizing universal fault-tolerant computing. For a physical system with some external controllable parameters, it is a great challenge to control the time dependence of these parameters to achieve a target multi-qubit gate efficiently and precisely. Here we construct a dueling double deep Q-learning neural network (DDDQN) to find out the optimized time dependence of controllable parameters to implement two typical quantum gates: a single-qubit Hadamard gate and a two-qubit CNOT gate. Compared with traditional optimal control methods, this deep reinforcement learning method can realize efficient and precise gate control without requiring any gradient information during the learning process. This work attempts to pave the way to investigate more quantum control problems with deep reinforcement learning techniques.

arXiv:1902.08418Europhysics Letters (EPL)Jul 22, 2019123 citations

Quantum State Generation Via Deep Reinforcement Learning

Shaojun Wu, Shan Jin, Xiaoting Wang

The quantum state generation problem is a major research goal of quantum control and quantum variational algorithms, which use iterative optimization methods to evolve the initial state to the target state. The Twin Delayed Deep Deterministic Policy (TD3) algorithm in reinforcement learning achieves high learning efficiency and better stability in continuous control tasks. Here, using the TD3, we propose a new quantum state preparation method that does not require case-by-case optimization, to find a suitable evolution path to obtain the desired state. Specifically, we input the initial state into the trained actor-network, which can output the parameters of the unitary gates step by step, thus gradually evolving the initial state to the fixed quantum state. According to the reversibility of the unitary transformation, we can obtain a sequence of unitary gates to evolve the fixed state to the desired state. To verify the effectiveness of the algorithm, we perform simulations for one-qubit, two-qubit, and four-qubit cases, and the results show that the trained actor-network can provide appropriate unitary transformations to obtain the fixed state.

Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and CyberneticsOct 1, 20231 citation

Fast Quantum Gate Design with Deep Reinforcement Learning Using Real-Time Feedback on Readout Signals

Emily Wright, Rogério de Sousa

The design of high-fidelity quantum gates is difficult because it requires the optimization of two competing effects, namely maximizing gate speed and minimizing leakage out of the qubit subspace. We propose a deep reinforcement learning algorithm that uses two agents to address the speed and leakage challenges simultaneously. The first agent constructs the qubit in-phase control pulse using a policy learned from rewards that compensate short gate times. The rewards are obtained at intermediate time steps throughout the construction of a full-length pulse, allowing the agent to explore the landscape of shorter pulses. The second agent determines an out-of-phase pulse to target leakage. Both agents are trained on real-time data from noisy hardware, thus providing model-free gate design that adapts to unpredictable hardware noise. To reduce the effect of measurement classification errors, the agents are trained directly on the readout signal from probing the qubit. We present proof-of-concept experiments by designing X and square root of X gates of various durations on IBM hardware. After just 200 training iterations, our algorithm is able to construct novel control pulses up to two times faster than the default IBM gates, while matching their performance in terms of state fidelity and leakage rate. As the length of our custom control pulses increases, they begin to outperform the default gates. Improvements to the speed and fidelity of gate operations open the way for higher circuit depth in quantum simulation, quantum chemistry and other algorithms on near-term and future quantum devices.

arXiv:2305.01169IEEE International Conference on Quantum Computing and Engineering (QCE)Sep 17, 20234 citations

Application of Deep Reinforcement Learning in Quantum Control

Yiming Cheng, Yixi Yin, Jiaying Lin +1 more

Abstract Machine learning technology based on artificial neural network has been successfully applied to solve many scientific problems. One of the most interesting areas of machine learning is reinforcement learning, which has natural applicability to optimization problems in physics. In the quantum control task, it is necessary to find a set of optimal control functions to transfer a quantum system from the initial state to the target state with the highest fidelity possible, which is essentially an optimization task. In this paper, we use Deep Deterministic Policy Gradient algorithm (DDPG) to study the quantum control tasks. We use the algorithm to control the transfer of several quantum systems from one state to another. The results show that DDPG algorithm can find a control strategy to make the fidelity of the final state and the target state of the quantum system be maximum value 1. The results show the potential of DDPG in quantum control.

Journal of Physics Conference SeriesMay 1, 20231 citation

Deep Reinforcement Learning Quantum Control on IBMQ Platforms and Qiskit Pulse

Rudy Semola, Lorenzo Del Moro, Davide Bacciu +1 more

Quantum optimal control theory (QOCT) typically addresses the control of physical qubits. Finding explicit pulse control sequences in such a framework is challenging, especially when an underlying physical model is unknown. We propose a deep reinforcement learning (DRL) method, which doesn't require any underlying gate model or qubit pre-calibration, capable of controlling a superconductive qubit via analog pulses acting in the IBM Qiskit Pulse environment. We applied the method to build a single-qubit gate with high fidelity and short duration at pulse level. In particular, the DRL agent approximated the X90 gate at the physical layer on the IBM Armonk transmon superconductive qubit simulated by the Qiskit Pulse simulator. The learned sequence has an average gate fidelity greater than 0.978 and a duration of 58 ns only, faster than the default X90 pulse IBM implementation, which has a runtime of 140 ns. Without prior knowledge and gate model knowledge, the agent learned a non-traditional shaped microwave pulse, providing an alternative strategy for controlling noisy quantum states.

IEEE International Conference on Quantum Computing and Engineering (QCE)Sep 1, 20224 citations

Deep Reinforcement Learning With Reward Design for Quantum Control

Haixu Yu, Xudong Zhao

Deep reinforcement learning (DRL) has been recognized as a powerful tool in quantum physics, where DRL's reward design is nontrivial but crucial for quantum control tasks. To address the problem of over-reliance on human empirical knowledge to design DRL's rewards, we propose a DRL with a novel reward paradigm designed by the learning process information (DRL-LPI), where the learning process information (LPI) comprises the state information and the experiences. In DRL-LPI, the state information after being classified by a fidelity threshold, and the experiences are first stored simultaneously in the respective sequences, and this process is repeated until a similar-segment ends. Then, the stored state information is converted to the real value and used to design the reward value by applying a self-amplitude function. Next, the designed reward values are integrated with the stored experiences to compose transitions for DRL's training. Through comparisons to five representative reward schemes, the proposed DRL-LPI is evaluated on two typical quantum control tasks, i.e., the spin-(1/2) quantum state control and many-coupled qubits state control, and the experimental results show the superior learning efficiency and control performance of the proposed approach. More results show that DRL-LPI exhibits the ability to learn the control strategy with few control actions compared to stochastic gradient descent (SGD) and genetic algorithm (GA).

IEEE Transactions on Artificial IntelligenceNov 28, 202227 citations