arXiv:2408.01506 · Quantum Science and Technology

Quantum noise modeling through reinforcement learning

Simone Bordoni, Andrea Papaluca, Piergiorgio Buttarini, Alejandro Sopena, S. Giagu, Stefano Carrazza

Published Nov 12, 2025• 3 citations

Abstract

Abstract In the current era of quantum computing, robust and efficient tools are essential to bridge the gap between simulations and quantum hardware execution. In this work, we introduce a machine learning approach to characterize the noise impacting a quantum chip and emulate it during simulations. Our algorithm leverages reinforcement learning (RL), offering increased flexibility in reproducing various noise models compared to conventional techniques such as randomized benchmarking or heuristic noise models. The effectiveness of the RL agent has been validated through simulations and testing on real superconducting qubits. Additionally, we provide practical use-case examples for the study of renowned quantum algorithms.

DOI: 10.1088/2058-9565/ae1e98

Related papers

Dynamic Quantum Circuit Compilation via Reinforcement Learning

Jincheng Zhang

Quantum circuit compilation is a crucial step in realizing quantum algorithms on real quantum hardware. Traditionally, compilation methods are largely static, relying on pre-defined mappings and optimizations that fail to account for the inherent noise and variability present in quantum devices. This paper proposes a novel approach leveraging Reinforcement Learning (RL) to dynamically generate and optimize quantum circuits. An RL agent is trained to learn a policy for circuit design, adapting to specific hardware characteristics and real-time measurement data. The agent's goal is to minimize the overall error rate of the compiled circuit. This dynamic compilation framework offers a feedback loop between hardware performance and circuit design, resulting in adaptive and robust quantum computation. The core claim of this work is that traditional static compilation methods are insufficient, and dynamic compilation driven by RL offers a superior solution. The approach presented here represents a significant advancement towards realizing the full potential of quantum computing by mitigating the effects of hardware imperfections. The framework includes a detailed description of the RL agent architecture, the reward function design, and the exploration strategy employed. The effectiveness of the proposed method is demonstrated through simulations, highlighting its ability to outperform static compilation strategies in noisy environments.

Sep 5, 20260 citations

Robustness of quantum reinforcement learning under hardware errors

Andrea Skolik, Stefano Mangini, Thomas Bäck +2 more

Abstract Variational quantum machine learning algorithms have become the focus of recent research on how to utilize near-term quantum devices for machine learning tasks. They are considered suitable for this as the circuits that are run can be tailored to the device, and a big part of the computation is delegated to the classical optimizer. It has also been hypothesized that they may be more robust to hardware noise than conventional algorithms due to their hybrid nature. However, the effect of training quantum machine learning models under the influence of hardware-induced noise has not yet been extensively studied. In this work, we address this question for a specific type of learning, namely variational reinforcement learning, by studying its performance in the presence of various noise sources: shot noise, coherent and incoherent errors. We analytically and empirically investigate how the presence of noise during training and evaluation of variational quantum reinforcement learning algorithms affect the performance of the agents and robustness of the learned policies. Furthermore, we provide a method to reduce the number of measurements required to train Q-learning agents, using the inherent structure of the algorithm.

arXiv:2212.09431EPJ Quantum TechnologyFeb 28, 202329 citations

Adaptive Quantum Circuit Compilation with Reinforcement Learning

Jincheng Zhang

This paper presents a novel approach to quantum circuit compilation leveraging reinforcement learning (RL). The core idea is to train an RL agent to dynamically optimize quantum circuit structures, aiming to minimize gate count and enhance overall circuit performance. Traditional quantum circuit compilation methods often rely on static heuristics and pre-defined optimization strategies, which may not be optimal for all quantum algorithms or hardware platforms. Our approach addresses this limitation by allowing the RL agent to learn a policy that adapts to the specific characteristics of the algorithm and the target quantum hardware. The agent learns by interacting with a simulated quantum circuit environment, receiving rewards based on circuit fidelity, depth, and gate count. The results demonstrate the potential of RL to significantly improve the efficiency and robustness of quantum computations. The key components of the system are: a high-level quantum algorithm representation, an RL agent, a quantum circuit simulator, and a reward function. The agent's policy is represented as a neural network, enabling it to generalize across different algorithms and hardware. We demonstrate the effectiveness of this approach through simulations, showcasing its ability to outperform conventional compilation methods in terms of reducing circuit depth and gate count while maintaining high fidelity.

Sep 5, 20260 citations

Control of Qubit Dynamics Using Reinforcement Learning

Dimitris Koutromanos, Dionisis Stefanatos, Emmanuel Paspalakis

The progress in machine learning during the last decade has had a considerable impact on many areas of science and technology, including quantum technology. This work explores the application of reinforcement learning (RL) methods to the quantum control problem of state transfer in a single qubit. The goal is to create an RL agent that learns an optimal policy and thus discovers optimal pulses to control the qubit. The most crucial step is to mathematically formulate the problem of interest as a Markov decision process (MDP). This enables the use of RL algorithms to solve the quantum control problem. Deep learning and the use of deep neural networks provide the freedom to employ continuous action and state spaces, offering the expressivity and generalization of the process. This flexibility helps to formulate the quantum state transfer problem as an MDP in several different ways. All the developed methodologies are applied to the fundamental problem of population inversion in a qubit. In most cases, the derived optimal pulses achieve fidelity equal to or higher than 0.9999, as required by quantum computing applications. The present methods can be easily extended to quantum systems with more energy levels and may be used for the efficient control of collections of qubits and to counteract the effect of noise, which are important topics for quantum sensing applications.

InformationMay 11, 202411 citations

Quafu-RL: The Cloud Quantum Computers based Quantum Reinforcement Learning

BAQIS Quafu Group

With the rapid advent of quantum computing, hybrid quantum-classical machine learning has shown promising computational advantages in many key fields. Quantum reinforcement learning, as one of the most challenging tasks, has recently demonstrated its ability to solve standard benchmark environments with formally provable theoretical advantages over classical counterparts. However, despite the progress of quantum processors and the emergence of quantum computing clouds in the noisy intermediate-scale quantum (NISQ) era, algorithms based on parameterized quantum circuits (PQCs) are rarely conducted on NISQ devices. In this work, we take the first step towards executing benchmark quantum reinforcement problems on various real devices equipped with at most 136 qubits on BAQIS Quafu quantum computing cloud. The experimental results demonstrate that the Reinforcement Learning (RL) agents are capable of achieving goals that are slightly relaxed both during the training and inference stages. Moreover, we meticulously design hardware-efficient PQC architectures in the quantum model using a multi-objective evolutionary algorithm and develop a learning algorithm that is adaptable to Quafu. We hope that the Quafu-RL be a guiding example to show how to realize machine learning task by taking advantage of quantum computers on the quantum cloud platform.

arXiv:2305.17966May 29, 20232 citations

Quantum Error Mitigation with Adaptive Noise Shaping

Jincheng Zhang

Quantum error mitigation (QEM) is a crucial step in achieving fault-tolerant quantum computation. Current methods often rely on fixed noise shaping, which struggles to adapt to the dynamic nature of quantum systems. This paper proposes a novel QEM technique leveraging reinforcement learning (RL) to dynamically adjust noise shaping parameters based on observed error rates. The core mechanism involves an RL agent that learns optimal noise shaping strategies to minimize errors while maintaining accuracy, offering a continuous adaptation to error conditions. This approach significantly improves the robustness and efficiency of QEM.

Aug 31, 20260 citations