We present a reinforcement learning (RL) approach to the co-design of stabilizer sets of Quantum Error Correcting Codes (QECCs) and decoders. We show how to produce a generative model that produces Bivariate Bicycle (BB) codes based on the choice of decoder. Specifically, we fix a decoder architecture and use Proximal Policy Optimisation (PPO) to train an agent over BB codes to maximise decoder performance under a depolarising channel noise model.
We present a reinforcement learning (RL) approach to the co-design of stabilizer sets of Quantum Error Correcting Codes (QECCs) and decoders. We show how to produce a generative model that produces Bivariate Bicycle (BB) codes based on the choice of decoder. Specifically, we fix a decoder architecture and use Proximal Policy Optimisation (PPO) to train an agent over BB codes to maximise decoder performance under a depolarising channel noise model.
The advent of promising quantum error correction (QEC) codes with efficient resource utilization and high-performance fault-tolerant quantum memories signifies a critical step towards realizing practical quantum computation. While surface codes have been a dominant approach, their limitations have spurred the development of more advanced QEC architectures. These advanced codes often present increased complexity, demanding innovative decoding methodologies. This work investigates the application of reinforcement learning (RL) techniques, including hybrid and multi-agent approaches, to enhance the decoding of various advanced QEC architectures. By leveraging the ability of RL to learn optimal strategies from noisy syndrome measurements, we explore the potential for achieving improved logical error rates and scalability compared to traditional decoding methods. Our approach examines the adaptation of reinforcement learning to exploit the structural properties of these modern QEC models. We also explore the benefits of combining different RL algorithms to address the multifaceted nature of the decoding problem, considering factors such as code degeneracy and real-world noise characteristics. With our proposed method, we are able to demonstrate that an autonomously trained agent can derive decoding schemes for the complex decoding requirement of advanced QEC architectures.
The design and optimization of quantum codes is generally hard because of the prohibitive computational cost. The recently introduced quantum lego framework provides a powerful method for generating complex quantum error-correcting codes (QECCs) out of simple ones. We gamify this process and unlock a new avenue for code design and discovery using reinforcement learning (RL). Already for simple code measures and modest qubit numbers, we produce codes that are both optimal and novel. Moreover, we argue that this framework is both scalable, when combined with efficient tensor contraction methods, and flexible, since we can specify arbitrary properties of the code to be optimized. We train on two such properties, maximizing the code distance, and minimizing the probability of logical error under biased Pauli noise. For the first, we show that the trained agent identifies ways to increase code distance beyond naive concatenation, saturating the linear programming bound for CSS (Calderbank, Shor, Steane) codes on 13 qubits. With a learning objective to minimize the logical error probability under biased Pauli noise, we find the best-known CSS code at this task for $\ensuremath{\lesssim}20$ qubits. Compared to other (locally deformed) CSS codes, including Surface, XZZX, and two-dimensional Color codes, our [[17,1,3]] code construction actually has lower adversarial distance, yet better protects the logical information, highlighting the importance of QECC desiderata. Lastly, we comment on how this RL framework can be used in conjunction with physical quantum devices to tailor a code without explicit characterization of the noise model.
With a particular emphasis on fixing fault defects caused by bitflip and depolarizing noise, this study explores the use of RL methods within the framework of quantum error correction. This work investigates the flexibility and independence of RL-trained agents in reducing bitflip mistakes by methodically looking at error rates and qubit lifetimes. Furthermore, we study how the top-performing agents are affected by depolarizing noise and present a strategic "Decoded Surface Code" method for fixing mistakes. The results show how RL works in handling different types of errors on its own, revealing subtle dynamics in the quantum system. Quantum error correction is a rapidly developing area, and this study adds to it by shedding light on how reinforcement learning could improve quantum computing's fault tolerance. The study's findings provide a basis for future research into more robust quantum computing systems and have ramifications for the creation of autonomous error-correcting algorithms.
Quantum error correction is widely thought to be the key to fault-tolerant quantum computation. However, determining the most suited encoding for unknown error channels or specific laboratory setups is highly challenging. Here, we present a reinforcement learning framework for optimizing and fault-tolerantly adapting quantum error correction codes. We consider a reinforcement learning agent tasked with modifying a family of surface code quantum memories until a desired logical error rate is reached. Using efficient simulations with about 70 data qubits with arbitrary connectivity, we demonstrate that such a reinforcement learning agent can determine near-optimal solutions, in terms of the number of data qubits, for various error models of interest. Moreover, we show that agents trained on one setting are able to successfully transfer their experience to different settings. This ability for transfer learning showcases the inherent strengths of reinforcement learning and the applicability of our approach for optimization from off-line simulations to on-line laboratory settings.
Quantum computing (QC) promises significant advantages on certain hard computational tasks over classical computers. However, current quantum hardware, also known as noisy intermediate-scale quantum computers (NISQ), are still unable to carry out computations faithfully mainly because of the lack of quantum error correction (QEC) capability. A significant amount of theoretical studies have provided various types of QEC codes; one of the notable topological codes is the surface code, and its features, such as the requirement of only nearest-neighboring two-qubit control gates and a large error threshold, make it a leading candidate for scalable quantum computation. Recent developments of machine learning (ML)-based techniques especially the reinforcement learning (RL) methods have been applied to the decoding problem and have already made certain progress. Nevertheless, the device noise pattern may change over time, making trained decoder models ineffective. In this paper, we propose a continual reinforcement learning method to address these decoding challenges. Specifically, we implement double deep Q-learning with probabilistic policy reuse (DDQN-PPR) model to learn surface code decoding strategies for quantum environments with varying noise patterns. Through numerical simulations, we show that the proposed DDQN-PPR model can significantly reduce the computational complexity. Moreover, increasing the number of trained policies can further improve the agent's performance. Our results open a way to build more capable RL agents which can leverage previously gained knowledge to tackle QEC challenges.
The convergence of quantum computing and artificial intelligence presents profound opportunities but faces significant hurdles, particularly in the Noisy Intermediate-Scale Quantum (NISQ) era. Quantum Machine Learning (QML) algorithms, while promising, exhibit sensitivity to noise and scalability challenges, hindering the demonstration of practical quantum advantage. Concurrently, Quantum Error Correction (QEC), essential for fault tolerance, imposes substantial resource overheads and is often developed generically, without specific adaptation to the target application's error sensitivity. This paper reviews the current state of the intersection of AI and quantum computing, examining both QML paradigms, e.g., Variational Quantum Algorithms, Quantum Kernels, and the burgeoning use of AI to enhance quantum computing itself, e.g., quantum control, QEC decoding, circuit design. A critical gap identified is the lack of frameworks that systematically co-design QML algorithms and QEC protocols adaptively. To address this, a novel framework is proposed based on Reinforcement Learning (RL). This framework employs an RL agent to dynamically adjust both the QML circuit architecture, e.g., VQC ansatz, and QEC parameters, e.g., decoding strategy, measurement frequency, based on observed application performance and estimated error characteristics. This adaptive co-design loop aims to optimize the trade-off between QML performance and QEC overhead, enhancing noise resilience and resource efficiency. The potential advantages, feasibility, and limitations of this approach are discussed, alongside with future research directions aimed at realizing robust and practical Quantum AI.