High-fidelity control of one- and two-qubit gates past the error correction threshold is an essential ingredient for scalable quantum computing. We present a reinforcement learning (RL) approach to find entangling protocols for semiconductor-based singlet-triplet qubits in a double quantum dot. Despite the presence of realistically modelled experimental constraints, such as various noise contributions and finite rise-time effects, we demonstrate that an RL agent can yield performative protocols, while avoiding the model-biases of traditional gradient-based methods. We optimise our RL approach for different regimes and tasks, including training from simulated process tomography reconstruction of unitary gates, and investigate the nuances of RL agent design.
High-fidelity control of one- and two-qubit gates past the error correction threshold is an essential ingredient for scalable quantum computing. We present a reinforcement learning (RL) approach to find entangling protocols for semiconductor-based singlet-triplet qubits in a double quantum dot. Despite the presence of realistically modelled experimental constraints, such as various noise contributions and finite rise-time effects, we demonstrate that an RL agent can yield performative protocols, while avoiding the model-biases of traditional gradient-based methods. We optimise our RL approach for different regimes and tasks, including training from simulated process tomography reconstruction of unitary gates, and investigate the nuances of RL agent design.
Abstract High-fidelity control of one- and two-qubit gates past the error correction threshold is an essential ingredient for scalable quantum computing. We present a reinforcement learning (RL) approach to find entangling protocols for semiconductor-based singlet-triplet qubits in a double quantum dot. Despite the presence of realistically modelled experimental constraints, such as various noise contributions and finite rise-time effects, we demonstrate that an RL agent can yield performative protocols, while avoiding the model-biases of traditional gradient-based methods. We optimise our RL approach for different regimes and tasks, including training from simulated process tomography reconstruction of unitary gates, and investigate the nuances of RL agent design.
Noisy intermediate-scale quantum computers hold the promise of tackling complex and otherwise intractable computational challenges through the massive parallelism offered by qubits. Central to realizing the potential of quantum computing are perfect entangling (PE) two-qubit gates, which serve as a critical building block for universal quantum computation. In the context of quantum optimal control, shaping electromagnetic pulses to drive quantum gates is crucial for pushing gate performance toward theoretical limits. In this work, we leverage reinforcement learning (RL) techniques to discover near-optimal pulse shapes that yield PE gates. A collection of RL agents is trained within robust simulation environments, enabling the identification of effective control strategies even under noisy conditions. Selected agents are then validated on higher-fidelity simulations, illustrating how RL-based methods can reduce calibration overhead when compared to quantum optimal control techniques. Furthermore, the RL approach is hardware agnostic with the potential for broad applicability across various quantum computing platforms.
This paper presents a novel approach to designing adaptive quantum control policies using reinforcement learning (RL). Traditional methods for quantum control often rely on hand-designed pulse shapes or iterative optimization techniques, which can be computationally expensive and may not generalize well to different systems or desired operations. We propose a framework where an RL agent learns to dynamically adjust quantum control parameters in real-time, based on the system's state and the objective of achieving a specific quantum gate with high fidelity. The core of our approach lies in formulating the quantum control problem as a Markov Decision Process (MDP) and utilizing an RL algorithm to train a policy that maps system states to control parameters. We demonstrate the feasibility and effectiveness of this method through a theoretical analysis and a simplified numerical simulation. The resulting adaptive control policies offer a significant advantage over traditional methods, particularly for complex quantum systems and demanding fidelity requirements. The key contributions are the integration of RL for adaptive control and the formalization of the problem as an MDP.
We address a wide spectrum of quantum control strategies, including various open-loop protocols and advanced adaptive methods. These methodologies apply to few-qubit scenarios and naturally scale to larger N-qubit systems. We benchmark them across fundamental quantum tasks: entanglement preservation/generation, and directed quantum transport in a disordered quantum walk. All simulations are performed in a challenging environment featuring non-Markov colored noise, imperfections, and the Markov Lindblad equation. With a complex task-dependent performance hierarchy, our deterministic protocols proved highly effective for entanglement generation/preservation, and in specific pulse configurations, they even outperformed the RL-optimization. In contrast, more advanced methods demonstrate a marked specialization. For entanglement preservation, a physics-informed hybrid Quantum Error Correction and Dynamical Decoupling (QEC-DD) protocol provides the most stable and effective solution, outperforming all other approaches. Conversely, for dynamic tasks requiring the discovery of non-trivial control sequences, such as DD, Floquet engineering, and rapid entanglement generation or coherent transport, the model-free Reinforcement Learning (RL) agents consistently find superior solutions. We further demonstrate that the control pulse envelope is a non-trivial factor that actively shapes the control landscape, which determines the difficulty for all protocols and highlights the adaptability of the RL agent. We conclude that no single strategy is universally dominant. A clear picture emerges: the future of high-fidelity quantum control lies in a synthesis of physics-informed design, as exemplified by robust hybrid methods, and the specialized, high-performance optimization power of adaptive machine learning.
Many challenges arising in Quantum Technology can be successfully addressed using a set of machine learning algorithms collectively known as reinforcement learning (RL), based on adaptive decision-making through interaction with the quantum device. After a concise and intuitive introduction to RL aimed at a broad physics readership, we discuss the key ideas and core concepts in reinforcement learning with a particular focus on quantum systems. We then survey recent progress in RL in all relevant areas. We discuss state preparation in few- and many-body quantum systems, the design and optimization of high-fidelity quantum gates, and the automated construction of quantum circuits, including applications to variational quantum eigensolvers and architecture search. We further highlight the interactive capabilities of RL agents, emphasizing recent progress in quantum feedback control and quantum error correction, and briefly discuss quantum reinforcement learning as well as applications to quantum metrology. The review concludes with a discussion of open challenges -- such as scalability, interpretability, and integration with experimental platforms -- and outlines promising directions for future research. Throughout, we highlight experimental implementations that exemplify the increasing role of reinforcement learning in shaping the development of quantum technologies.
Abstract Several tasks involving the determination of the time evolution of a system of solid state qubits require stochastic methods in order to identify the best sequence of gates and the time of interaction among the qubits. The major success of deep learning in several scientific disciplines has suggested its application to quantum information as well. Thanks to its capability to identify best strategy in those problems involving a competition between the short term and the long term rewards, reinforcement learning (RL) method has been successfully applied, for instance, to discover sequences of quantum gate operations minimizing the information loss. In order to extend the application of RL to the transfer of quantum information, we focus on Coherent Transport by Adiabatic Passage (CTAP) on a chain of three semiconductor quantum dots (QD). This task is usually performed by the so called counter-intuitive sequence of gate pulses. Such sequence is capable of coherently transfer an electronic population from the first to the last site of an odd chain of QDs, by leaving the central QD unpopulated. We apply a technique to find nearly optimal gate pulse sequence without explicitly give any prior knowledge of the underlying physical system to the RL agent. Using the advantage actor-critic algorithm, with a small neural net as function approximator, we trained a RL agent to choose the best action at every time step of the physical evolution to achieve the same results previously found only by ansatz solutions.