In the pursuit of realizing a quantum computer through quantum gating, generating a Schrödinger’s cat state that is robust against noise and decoherence presents a significant challenge. Recent studies have explored deep reinforcement learning methods; however, the results often fail to accurately reflect the state of the quantum system due to quantum back-action during the observation process. In this study, we propose a novel approach for quantum state generation that integrates particle filters with deep reinforcement learning. The particle filter estimates the quantum system's state based on observed results and subsequently provides feedback to the reinforcement learning agent. We compare our current findings on generating Schrödinger's cat state with previous results derived from deep reinforcement learning techniques.
In the pursuit of realizing a quantum computer through quantum gating, generating a Schrödinger’s cat state that is robust against noise and decoherence presents a significant challenge. Recent studies have explored deep reinforcement learning methods; however, the results often fail to accurately reflect the state of the quantum system due to quantum back-action during the observation process. In this study, we propose a novel approach for quantum state generation that integrates particle filters with deep reinforcement learning. The particle filter estimates the quantum system's state based on observed results and subsequently provides feedback to the reinforcement learning agent. We compare our current findings on generating Schrödinger's cat state with previous results derived from deep reinforcement learning techniques.
Quantum computers promise significant computational advantages over classical computers for specific tasks. However, one of the challenges in the noisy intermediate-scale quantum (NISQ) era is the efficient compilation of quantum states and gates, which is crucial for the implementation of quantum algorithms. Given the high dimensionality of quantum systems, this compilation task is non-trivial. Reinforcement learning (RL) is a machine learning subfield focused on developing algorithms for decision-making in dynamic environments. An agent learns by interacting with its environment, how to act to optimize its performance on a certain task. Recently, the integration of deep learning has given rise to deep reinforcement learning (DRL), where deep neural networks approximate functions like value or policy, allowing agents to tackle high-dimensional and complex tasks with greater efficiency and success. Recently, deep reinforcement learning (DRL) has been proposed as a method to automate quantum compiling. In this thesis, we explore the application of RL for quantum state and unitary matrix synthesis, focusing on the use of two reinforcement learning algorithms Deep Q-Networks (DQN) and Proximal Policy Optimization (PPO). Our results indicate that DRL shows promise in generalizing the task of quantum unitary synthesis and quantum state preparation. However, the current state-of-the-art is still limited to working in the no-volume law entangled part of the Hilbert space.
High-fidelity quantum gate design is important for various quantum technologies, such as quantum computation and quantum communication. Numerous control policies for quantum gate design have been proposed given a dynamical model of the quantum system of interest. However, a quantum system is often highly sensitive to noise, and obtaining its accurate modeling can be difficult for many practical applications. Thus, the control policy based on a quantum system model may be unpractical for quantum gate design. Also, quantum measurements collapse quantum states, which makes it challenging to obtain information through measurements during the control process. In this article, we propose a novel training framework using deep reinforcement learning for model-free quantum control. The proposed framework relies only on the measurement at the end of the control process and offers the ability to find the optimal control policy without access to quantum systems during the learning process. The effectiveness of the proposed technique is numerically demonstrated for model-free quantum gate design and quantum gate calibration using off-policy reinforcement learning algorithms.
To realize the full potential of quantum technologies, finding good strategies to control quantum information processing devices in real time becomes increasingly important. Usually these strategies require a precise understanding of the device itself, which is generally not available. Model-free reinforcement learning circumvents this need by discovering control strategies from scratch without relying on an accurate description of the quantum system. Furthermore, important tasks like state preparation, gate teleportation and error correction need feedback at time scales much shorter than the coherence time, which for superconducting circuits is in the microsecond range. Developing and training a deep reinforcement learning agent able to operate in this real-time feedback regime has been an open challenge. Here, we have implemented such an agent in the form of a latency-optimized deep neural network on a field-programmable gate array (FPGA). We demonstrate its use to efficiently initialize a superconducting qubit into a target state. To train the agent, we use model-free reinforcement learning that is based solely on measurement data. We study the agent's performance for strong and weak measurements, and for three-level readout, and compare with simple strategies based on thresholding. This demonstration motivates further research towards adoption of reinforcement learning for real-time feedback control of quantum devices and more generally any physical system requiring learnable low-latency feedback control.
Applying deep reinforcement learning (DRL) to solve quantum control problems has become a popular research direction. However, the exploration capability and reward design for the learning agent, which usually affect the DRL’s application performance, has not been sufficiently emphasized. In this article, we propose a particle-assisted DRL (PDRL) method to address the above concern by enhancing exploration capabilities and designing appropriate reward functions for efficient quantum state manipulation. In PDRL, each episode in the quantum learning process is characterized by three kinds of events, i.e., unidentifiable, identifiable, and successful events. To improve exploration, exploration particles and feedback particles are employed in the early learning phase when episodes end in identifiable and successful events, respectively. To assign rewards, three event-based reward functions are provided for the DRL’s agent, exploration particles and feedback particles, respectively. Numerical results on single-qubit, two-qubit, and many-qubit systems validate the effectiveness of PDRL. Comparative results with existing DRL methods demonstrate the superior performance of PDRL for quantum state manipulation.
Realizing the full potential of quantum technologies requires precise real-time control on time scales much shorter than the coherence time. Model-free reinforcement learning promises to discover efficient feedback strategies from scratch without relying on a description of the quantum system. However, developing and training a reinforcement learning agent able to operate in real-time using feedback has been an open challenge. Here, we have implemented such an agent for a single qubit as a sub-microsecond-latency neural network on a field-programmable gate array (FPGA). We demonstrate its use to efficiently initialize a superconducting qubit and train the agent based solely on measurements. Our work is a first step towards adoption of reinforcement learning for the control of quantum devices and more generally any physical device requiring low-latency feedback.
The extreme sensitivity of contemporary hardware to noise, manufacturing variability, and imperfect quantum logic gates are key factors that limit hardware to reliably perform quantum computation at scale. The use of reinforcement learning assisted control techniques for system dynamics optimization in quantum gate control can significantly improve hardware performance and downstream computing power. However, con-trol field noise and Hamiltonian parameter uncertainty can lead to significant deviations from the target quantum gate. As a result, quantum gates lack the generalization ability to the quantum environment, and existing reinforcement learning assisted for quantum state preparation is not suitable for quantum gate control. In this paper, we design QcontrolSR, a state representation for reinforcement learning method, enabling quantum gate control with enhanced generalization ability. It requires no knowledge of a specific Hamiltonian model of the system, or its underlying unknown process. We introduce the state representation of quantum observations and show through demonstrations that QcontrolSR-optimized quantum gate control is insensitive to driving noise of the typical strength in practice. We show thatRx(π/2)gate implemented using QcontrolSR can maintain fidelity greater than 0.95 and 0.99 on dynamical quantum stochastic system. The regions with fidelity greater than 0.99 and 0.999 are 2 and 3 times higher than the model-optimized Quantum gate control, respectively.