Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2401.17190 · Physical Review A

Improving robustness of quantum feedback control with reinforcement learning

Manuel Guatto, Gian Antonio Susto, Francesco Ticozzi

Published Jul 3, 2024• 5 citations
arXivHTMLPDF

Abstract

Obtaining reliable state preparation protocols is a key step toward practical implementation of many quantum technologies, and one of the main tasks in quantum control. In this work, different reinforcement learning approaches are used to derive a feedback law for state preparation of a desired state in a target system. In particular, we focus on the robustness of the obtained strategies with respect to different types and amount of noise. Comparing the results indicates that the learned controls are more robust to unmodeled perturbations with respect to simple feedback strategy based on optimized population transfer, and that training on a simulated nominal model retains the same advantages displayed by controllers trained on real data. The possibility of effective off-line training of robust controllers promises significant advantages toward practical implementation.

DOI
10.1103/physreva.110.012605

Related papers

Reinforcement Learning for Quantum Control under Physical Constraints

Jan Ole Ernst, Chatterjee, Aniket, Tim Franzmeyer +1 more

Quantum control is concerned with the realisation of desired dynamics in quantum systems, serving as a linchpin for advancing quantum technologies and fundamental research. Analytic approaches and standard optimisation algorithms do not yield satisfactory solutions for more complex quantum systems, and especially not for real world quantum systems which are open and noisy. We devise a physics-constrained Reinforcement Learning (RL) algorithm that restricts the space of possible solutions. We incorporate priors about the desired time scales of the quantum state dynamics - as well as realistic control signal limitations - as constraints to the RL algorithm. These constraints improve solution quality and enhance computational scaleability. We evaluate our method on three broadly relevant quantum systems and incorporate real-world complications, arising from dissipation and control signal perturbations. We achieve both higher fidelities - which exceed 0.999 across all systems - and better robustness to time-dependent perturbations and experimental imperfections than previous methods. Lastly, we demonstrate that incorporating multi-step feedback can yield solutions robust even to strong perturbations. Our implementation can be found at https://github.com/jan-o-e/RL4qcWpc.

arXiv:2501.14372•Jan 24, 2025•1 citation

Stochastic optimization for learning quantum state feedback control

Ethan N. Evans, Ziyi Wang, Adam G. Frim +2 more

High fidelity state preparation represents a fundamental challenge in the application of quantum technology. While the majority of optimal control approaches use feedback to improve the controller, the controller itself often does not incorporate explicit state dependence. Here, we present a general framework for training deep feedback networks for open quantum systems with quantum nondemolition measurement that allows a variety of system and control structures that are prohibitive by many other techniques and can in effect react to unmodeled effects through nonlinear filtering. We demonstrate that this method is efficient due to inherent parallelizability, robust to open system interactions, and outperforms landmark state feedback control results in simulation.

arXiv:2111.09896•Nov 18, 2021•0 citations

Feedback-Based Steering for Quantum State Preparation

Daniel Volya, Zhixin Pan, Prabhat Mishra

State preparation is an essential component in quantum information science. A recently developed steering protocol utilizes a sequence of generalized measurements on a detector to steer a quantum system towards a desired state. However, it is designed as an open-loop technique that requires accurate modeling of the overall quantum system and can be prone to errors. To address this challenge, we propose a closed-loop control technique that introduces feedback to the steering protocol, providing robustness to noise and faster state convergence. We introduce two strategies for feedback: (1) a gradient-based active steering protocol that changes the detector-system coupling conditioned on the detector's readout and (2) tuning the fixed detector-system coupling via model-free reinforcement learning. We study the effectiveness of these strategies under various noise models, including both incoherent and decoherent noise, and discuss potential applications in quantum technologies.

IEEE International Conference on Quantum Computing and Engineering (QCE)•Sep 17, 2023•6 citations

A Generic and Robust Agent to Accomplish Quantum Control Tasks

S Zhang, Zibo Miao, Qixiang Zhang +1 more

In view of inevitable uncertainties or disturbances in quantum systems, efficient and systematic numerical methods for robust control design are of significant importance. Although reinforcement learning has recently been applied in the field of quantum control, completing quantum control tasks in practice is always challenging due to the lack of data from real-world environments. In this paper, a generic and robust agent can be constructed catering to system imperfections by applying a curriculum meta-reinforcement learning quantum control (CMQC) approach. The proposed approach incorporates an objective module that can break down a complex quantum control objective into multiple sub-objectives using an exponential decay strategy, enabling the agent to gradually learn to solve the final goal through experience transfer and knowledge consolidation. Furthermore, a meta module is employed to achieve adaptation to environmental imperfections. This allows the agent to adjust its strategy in real time, obtaining strong robustness even in the presence of uncertainties. The superiority of CMQC is demonstrated in robust control problems regarding state transfer in a two-level quantum system and realization of the$R_{x}(\pi)$gate, with reduced computational cost and improved accuracy achieved. Our results pave the way towards more reliable and secure quantum information processing with uncertainties involved.

IEEE Transactions on Emerging Topics in Computational Intelligence•Feb 12, 2026•0 citations

Learning quantum-state feedback control with backpropagation-free stochastic optimization

Ethan N. Evans, Ziyi Wang, Adam G. Frim +2 more

High-fidelity state preparation represents a fundamental challenge in the application of quantum technology. While the majority of optimal control approaches use feedback to improve the controller, the controller itself often does not incorporate explicit state dependence. Here, we present a general framework for training deep feedback networks for open quantum systems with continuous weak measurement that allows a variety of system and control structures that are prohibitive by many other techniques and can in effect react to unmodeled effects through nonlinear filtering. Our approach benefits from characteristics of both stochastic sampling and gradient-based optimization methods yet does not require differentiability as in backpropagation approaches. We demonstrate that this method is efficient due to inherent parallelizability, robust to open system interactions, and outperforms landmark state-dependent feedback control results in simulation.

Physical Review A•Nov 3, 2022•1 citation

Adaptive Quantum Control via Reinforcement Learning

Jincheng Zhang

This paper investigates the application of reinforcement learning (RL) for adaptive quantum control. Traditional quantum control methods often rely on pre-designed pulse shapes optimized for specific scenarios, lacking adaptability to variations in system parameters or environmental noise. This work proposes a novel framework where an RL agent learns to dynamically adjust quantum control pulses in real-time, maximizing the fidelity of quantum operations. The agent receives feedback from the quantum system, typically measured through state variables, and utilizes this information to optimize the control pulse sequence. This adaptive approach addresses the limitations of static control schemes and promises enhanced robustness and performance in complex quantum systems. The core claim centers on utilizing reinforcement learning to design optimal quantum control pulses. The central mechanism involves a trained controller dynamically adjusting pulses based on system feedback. The presented methodology offers a new approach to quantum control, particularly relevant in noisy environments and systems with parameter variations. The research demonstrates the potential of RL for achieving superior quantum control outcomes compared to conventional techniques. ---

•Sep 4, 2026•0 citations
Obtaining reliable state preparation protocols is a key step toward practical implementation of many quantum technologies, and one of the main tasks in quantum control. In this work, different reinforcement learning approaches are used to derive a feedback law for state preparation of a desired state in a target system. In particular, we focus on the robustness of the obtained strategies with respect to different types and amount of noise. Comparing the results indicates that the learned controls are more robust to unmodeled perturbations with respect to simple feedback strategy based on optimized population transfer, and that training on a simulated nominal model retains the same advantages displayed by controllers trained on real data. The possibility of effective off-line training of robust controllers promises significant advantages toward practical implementation.
Quantum control is concerned with the realisation of desired dynamics in quantum systems, serving as a linchpin for advancing quantum technologies and fundamental research. Analytic approaches and standard optimisation algorithms do not yield satisfactory solutions for more complex quantum systems, and especially not for real world quantum systems which are open and noisy. We devise a physics-constrained Reinforcement Learning (RL) algorithm that restricts the space of possible solutions. We incorporate priors about the desired time scales of the quantum state dynamics - as well as realistic control signal limitations - as constraints to the RL algorithm. These constraints improve solution quality and enhance computational scaleability. We evaluate our method on three broadly relevant quantum systems and incorporate real-world complications, arising from dissipation and control signal perturbations. We achieve both higher fidelities - which exceed 0.999 across all systems - and better robustness to time-dependent perturbations and experimental imperfections than previous methods. Lastly, we demonstrate that incorporating multi-step feedback can yield solutions robust even to strong perturbations. Our implementation can be found at https://github.com/jan-o-e/RL4qcWpc.
High fidelity state preparation represents a fundamental challenge in the application of quantum technology. While the majority of optimal control approaches use feedback to improve the controller, the controller itself often does not incorporate explicit state dependence. Here, we present a general framework for training deep feedback networks for open quantum systems with quantum nondemolition measurement that allows a variety of system and control structures that are prohibitive by many other techniques and can in effect react to unmodeled effects through nonlinear filtering. We demonstrate that this method is efficient due to inherent parallelizability, robust to open system interactions, and outperforms landmark state feedback control results in simulation.
State preparation is an essential component in quantum information science. A recently developed steering protocol utilizes a sequence of generalized measurements on a detector to steer a quantum system towards a desired state. However, it is designed as an open-loop technique that requires accurate modeling of the overall quantum system and can be prone to errors. To address this challenge, we propose a closed-loop control technique that introduces feedback to the steering protocol, providing robustness to noise and faster state convergence. We introduce two strategies for feedback: (1) a gradient-based active steering protocol that changes the detector-system coupling conditioned on the detector's readout and (2) tuning the fixed detector-system coupling via model-free reinforcement learning. We study the effectiveness of these strategies under various noise models, including both incoherent and decoherent noise, and discuss potential applications in quantum technologies.
In view of inevitable uncertainties or disturbances in quantum systems, efficient and systematic numerical methods for robust control design are of significant importance. Although reinforcement learning has recently been applied in the field of quantum control, completing quantum control tasks in practice is always challenging due to the lack of data from real-world environments. In this paper, a generic and robust agent can be constructed catering to system imperfections by applying a curriculum meta-reinforcement learning quantum control (CMQC) approach. The proposed approach incorporates an objective module that can break down a complex quantum control objective into multiple sub-objectives using an exponential decay strategy, enabling the agent to gradually learn to solve the final goal through experience transfer and knowledge consolidation. Furthermore, a meta module is employed to achieve adaptation to environmental imperfections. This allows the agent to adjust its strategy in real time, obtaining strong robustness even in the presence of uncertainties. The superiority of CMQC is demonstrated in robust control problems regarding state transfer in a two-level quantum system and realization of theRx(π)R_{x}(\pi)Rx​(π)gate, with reduced computational cost and improved accuracy achieved. Our results pave the way towards more reliable and secure quantum information processing with uncertainties involved.
High-fidelity state preparation represents a fundamental challenge in the application of quantum technology. While the majority of optimal control approaches use feedback to improve the controller, the controller itself often does not incorporate explicit state dependence. Here, we present a general framework for training deep feedback networks for open quantum systems with continuous weak measurement that allows a variety of system and control structures that are prohibitive by many other techniques and can in effect react to unmodeled effects through nonlinear filtering. Our approach benefits from characteristics of both stochastic sampling and gradient-based optimization methods yet does not require differentiability as in backpropagation approaches. We demonstrate that this method is efficient due to inherent parallelizability, robust to open system interactions, and outperforms landmark state-dependent feedback control results in simulation.
This paper investigates the application of reinforcement learning (RL) for adaptive quantum control. Traditional quantum control methods often rely on pre-designed pulse shapes optimized for specific scenarios, lacking adaptability to variations in system parameters or environmental noise. This work proposes a novel framework where an RL agent learns to dynamically adjust quantum control pulses in real-time, maximizing the fidelity of quantum operations. The agent receives feedback from the quantum system, typically measured through state variables, and utilizes this information to optimize the control pulse sequence. This adaptive approach addresses the limitations of static control schemes and promises enhanced robustness and performance in complex quantum systems. The core claim centers on utilizing reinforcement learning to design optimal quantum control pulses. The central mechanism involves a trained controller dynamically adjusting pulses based on system feedback. The presented methodology offers a new approach to quantum control, particularly relevant in noisy environments and systems with parameter variations. The research demonstrates the potential of RL for achieving superior quantum control outcomes compared to conventional techniques. ---