Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2305.11047 · Physical Review A

Preparation of cavity-Fock-state superpositions by reinforcement learning exploiting measurement backaction

Arthur Perret, Yves Bérubé-Lauzière

Published Feb 20, 2024• 5 citations
arXivHTMLPDF

Abstract

Preparation of bosonic and general cavity quantum states usually relies on using open-loop control to reach a desired target state. In this work, a measurement-based feedback approach is used instead, exploiting the nonlinearity of weak measurements alongside a coherent drive to prepare these states. The extension of previous work on Lyapunov-based control is shown to fail for this task. This prompts for a different approach, and reinforcement learning (RL) is resorted to here for this purpose. With such an approach, cavity-eigenstate superpositions can be prepared with fidelities over $98%$ using only the measurements backaction as the nonlinearity, while naturally incorporating detection of cavity photon jumps. Two different RL frameworks are analyzed: an off-policy approach recently introduced called truncated quantile critic (TQC) and the on-policy method commonly used in quantum control, namely proximal policy optimization. It is shown that TQC performs better at reaching higher target state fidelity preparation.

DOI
10.1103/physreva.109.022609

Related papers

Deep Reinforcement Learning for Quantum State Preparation with Weak Nonlinear Measurements

Riccardo Porotti, Antoine Essig, Benjamin Huard +1 more

Quantum control has been of increasing interest in recent years, e.g. for tasks like state initialization and stabilization. Feedback-based strategies are particularly powerful, but also hard to find, due to the exponentially increased search space. Deep reinforcement learning holds great promise in this regard. It may provide new answers to difficult questions, such as whether nonlinear measurements can compensate for linear, constrained control. Here we show that reinforcement learning can successfully discover such feedback strategies, without prior knowledge. We illustrate this for state preparation in a cavity subject to quantum-non-demolition detection of photon number, with a simple linear drive as control. Fock states can be produced and stabilized at very high fidelity. It is even possible to reach superposition states, provided the measurement rates for different Fock states can be controlled as well.

arXiv:2107.08816•Quantum•Jun 28, 2022•70 citations

Experiment-compatible measurement--feedback quantum state preparation with reinforcement learning

Xiaotian Nie, Tao Zhang, Linghui Chen

Ground-state preparation is a critical task in quantum simulation and quantum computing, as it enables the study of correlated phases and the generation of entangled resource states. While measurement--feedback control has emerged as a promising route to state preparation, existing schemes either rely on handcrafted, task-specific policies or are designed using full quantum-state information that is unavailable in real experiments and becomes impractical for large many-body systems. Here we develop an adaptive measurement--feedback protocol based on reinforcement learning under partial observability. The controller uses only the history of experimentally accessible measurement outcomes to choose both the measurement operator and the feedback action in real time. To make training compatible with experiments, we introduce a stochastic terminal reward built from one-shot measurements of randomly sampled Hamiltonian components, avoiding unphysical full-state reconstruction while remaining an unbiased estimator of the target energy. We demonstrate the method by preparing ground states of the Bose--Hubbard model and by generating GHZ states, establishing a scalable and hardware-compatible route to quantum state preparation.

quant-pharXiv:2606.13005•Chinese Physics Letters•Jun 12, 2026•0 citations

Measurement-Based Feedback Quantum Control with Deep Reinforcement Learning for a Double-Well Nonlinear Potential

Sangkha Borah, Bijita Sarma, Michael J. Kewming +2 more

Closed loop quantum control uses measurement to control the dynamics of a quantum system to achieve either a desired target state or target dynamics. In the case when the quantum Hamiltonian is quadratic in x and p, there are known optimal control techniques to drive the dynamics toward particular states, e.g., the ground state. However, for nonlinear Hamiltonian such control techniques often fail. We apply deep reinforcement learning (DRL), where an artificial neural agent explores and learns to control the quantum evolution of a highly nonlinear system (double well), driving the system toward the ground state with high fidelity. We consider a DRL strategy which is particularly motivated by experiment where the quantum system is continuously but weakly measured. This measurement is then fed back to the neural agent and used for training. We show that the DRL can effectively learn counterintuitive strategies to cool the system to a nearly pure "cat" state, which has a high overlap fidelity with the true ground state.

arXiv:2104.11856•Physical Review Letters•Nov 2, 2021•77 citations

Deep Reinforcement Learning for quantum state preparation with noisy gates in the NISQ-era

Narvaez Hernandez, Roberto

Quantum computers promise significant computational advantages over classical computers for specific tasks. However, one of the challenges in the noisy intermediate-scale quantum (NISQ) era is the efficient compilation of quantum states and gates, which is crucial for the implementation of quantum algorithms. Given the high dimensionality of quantum systems, this compilation task is non-trivial. Reinforcement learning (RL) is a machine learning subfield focused on developing algorithms for decision-making in dynamic environments. An agent learns by interacting with its environment, how to act to optimize its performance on a certain task. Recently, the integration of deep learning has given rise to deep reinforcement learning (DRL), where deep neural networks approximate functions like value or policy, allowing agents to tackle high-dimensional and complex tasks with greater efficiency and success. Recently, deep reinforcement learning (DRL) has been proposed as a method to automate quantum compiling. In this thesis, we explore the application of RL for quantum state and unitary matrix synthesis, focusing on the use of two reinforcement learning algorithms Deep Q-Networks (DQN) and Proximal Policy Optimization (PPO). Our results indicate that DRL shows promise in generalizing the task of quantum unitary synthesis and quantum state preparation. However, the current state-of-the-art is still limited to working in the no-volume law entangled part of the Hilbert space.

•Sep 9, 2024•0 citations

Machine Learning Meets Quantum State Preparation. The Phase Diagram of Quantum Control

Marin Bukov, Alexandre G. R. Day, Dries Sels +3 more

The ability to prepare a physical system in a desired quantum state is central to many areas of physics such as nuclear magnetic resonance, cold atoms, and quantum computing. Yet, preparing states quickly and with high fidelity remains a formidable challenge. In this work we implement cutting-edge Reinforcement Learning (RL) techniques and show that their performance is comparable to optimal control methods in the task of finding short, high-fidelity driving protocol from an initial to a target state in non-integrable many-body quantum systems of interacting qubits. RL methods learn about the underlying physical system solely through a single scalar reward (the fidelity of the resulting state) calculated from numerical simulations of the physical system. We further show that quantum state manipulation, viewed as an optimization problem, exhibits a spin-glass-like phase transition in the space of protocols as a function of the protocol duration. Our RL-aided approach helps identify variational protocols with nearly optimal fidelity, even in the glassy phase, where optimal state manipulation is exponentially hard. This study highlights the potential usefulness of RL for applications in out-of-equilibrium quantum physics.

•May 1, 2017•4 citations

Quantum-Inspired Reinforcement Learning for Quantum Control

Haixu Yu, Xudong Zhao, Chunlin Chen

Reinforcement learning (RL) is considered a powerful technology with the potential to revolutionize quantum control. However, the application effectiveness of traditional RL is often limited by some insurmountable experimental conditions. Thus, developing new RL algorithms that can efficiently manipulate the quantum system dynamics is a crucial task. Prior research has shown that incorporating quantum mechanical properties into RL can improve learning performance. In this article, we consider the quantum control problem where only the target state can be accurately identified and introduce a quantum-inspired RL (QiRL) method. In particular, we propose a quantum-inspired exploration strategy to replace a commonly used$\epsilon $-greedy strategy, as well as a quantum-inspired reward scheme to incentivize the learning agent. Numerical results on three quantum system control problems, i.e., one-qubit closed quantum system, two-level open quantum system, and many-qubit closed quantum system, verify the effectiveness of QiRL. Comparison results show that the proposed QiRL outperforms existing RL algorithms (deep Q-network and proximal policy optimization) in terms of stability and efficiency for solving quantum control problems.

IEEE Transactions on Control Systems Technology•Aug 14, 2024•11 citations
Preparation of bosonic and general cavity quantum states usually relies on using open-loop control to reach a desired target state. In this work, a measurement-based feedback approach is used instead, exploiting the nonlinearity of weak measurements alongside a coherent drive to prepare these states. The extension of previous work on Lyapunov-based control is shown to fail for this task. This prompts for a different approach, and reinforcement learning (RL) is resorted to here for this purpose. With such an approach, cavity-eigenstate superpositions can be prepared with fidelities over 9898%98 using only the measurements backaction as the nonlinearity, while naturally incorporating detection of cavity photon jumps. Two different RL frameworks are analyzed: an off-policy approach recently introduced called truncated quantile critic (TQC) and the on-policy method commonly used in quantum control, namely proximal policy optimization. It is shown that TQC performs better at reaching higher target state fidelity preparation.
Quantum control has been of increasing interest in recent years, e.g. for tasks like state initialization and stabilization. Feedback-based strategies are particularly powerful, but also hard to find, due to the exponentially increased search space. Deep reinforcement learning holds great promise in this regard. It may provide new answers to difficult questions, such as whether nonlinear measurements can compensate for linear, constrained control. Here we show that reinforcement learning can successfully discover such feedback strategies, without prior knowledge. We illustrate this for state preparation in a cavity subject to quantum-non-demolition detection of photon number, with a simple linear drive as control. Fock states can be produced and stabilized at very high fidelity. It is even possible to reach superposition states, provided the measurement rates for different Fock states can be controlled as well.
Ground-state preparation is a critical task in quantum simulation and quantum computing, as it enables the study of correlated phases and the generation of entangled resource states. While measurement--feedback control has emerged as a promising route to state preparation, existing schemes either rely on handcrafted, task-specific policies or are designed using full quantum-state information that is unavailable in real experiments and becomes impractical for large many-body systems. Here we develop an adaptive measurement--feedback protocol based on reinforcement learning under partial observability. The controller uses only the history of experimentally accessible measurement outcomes to choose both the measurement operator and the feedback action in real time. To make training compatible with experiments, we introduce a stochastic terminal reward built from one-shot measurements of randomly sampled Hamiltonian components, avoiding unphysical full-state reconstruction while remaining an unbiased estimator of the target energy. We demonstrate the method by preparing ground states of the Bose--Hubbard model and by generating GHZ states, establishing a scalable and hardware-compatible route to quantum state preparation.
Closed loop quantum control uses measurement to control the dynamics of a quantum system to achieve either a desired target state or target dynamics. In the case when the quantum Hamiltonian is quadratic in x and p, there are known optimal control techniques to drive the dynamics toward particular states, e.g., the ground state. However, for nonlinear Hamiltonian such control techniques often fail. We apply deep reinforcement learning (DRL), where an artificial neural agent explores and learns to control the quantum evolution of a highly nonlinear system (double well), driving the system toward the ground state with high fidelity. We consider a DRL strategy which is particularly motivated by experiment where the quantum system is continuously but weakly measured. This measurement is then fed back to the neural agent and used for training. We show that the DRL can effectively learn counterintuitive strategies to cool the system to a nearly pure "cat" state, which has a high overlap fidelity with the true ground state.
Quantum computers promise significant computational advantages over classical computers for specific tasks. However, one of the challenges in the noisy intermediate-scale quantum (NISQ) era is the efficient compilation of quantum states and gates, which is crucial for the implementation of quantum algorithms. Given the high dimensionality of quantum systems, this compilation task is non-trivial. Reinforcement learning (RL) is a machine learning subfield focused on developing algorithms for decision-making in dynamic environments. An agent learns by interacting with its environment, how to act to optimize its performance on a certain task. Recently, the integration of deep learning has given rise to deep reinforcement learning (DRL), where deep neural networks approximate functions like value or policy, allowing agents to tackle high-dimensional and complex tasks with greater efficiency and success. Recently, deep reinforcement learning (DRL) has been proposed as a method to automate quantum compiling. In this thesis, we explore the application of RL for quantum state and unitary matrix synthesis, focusing on the use of two reinforcement learning algorithms Deep Q-Networks (DQN) and Proximal Policy Optimization (PPO). Our results indicate that DRL shows promise in generalizing the task of quantum unitary synthesis and quantum state preparation. However, the current state-of-the-art is still limited to working in the no-volume law entangled part of the Hilbert space.
The ability to prepare a physical system in a desired quantum state is central to many areas of physics such as nuclear magnetic resonance, cold atoms, and quantum computing. Yet, preparing states quickly and with high fidelity remains a formidable challenge. In this work we implement cutting-edge Reinforcement Learning (RL) techniques and show that their performance is comparable to optimal control methods in the task of finding short, high-fidelity driving protocol from an initial to a target state in non-integrable many-body quantum systems of interacting qubits. RL methods learn about the underlying physical system solely through a single scalar reward (the fidelity of the resulting state) calculated from numerical simulations of the physical system. We further show that quantum state manipulation, viewed as an optimization problem, exhibits a spin-glass-like phase transition in the space of protocols as a function of the protocol duration. Our RL-aided approach helps identify variational protocols with nearly optimal fidelity, even in the glassy phase, where optimal state manipulation is exponentially hard. This study highlights the potential usefulness of RL for applications in out-of-equilibrium quantum physics.
Reinforcement learning (RL) is considered a powerful technology with the potential to revolutionize quantum control. However, the application effectiveness of traditional RL is often limited by some insurmountable experimental conditions. Thus, developing new RL algorithms that can efficiently manipulate the quantum system dynamics is a crucial task. Prior research has shown that incorporating quantum mechanical properties into RL can improve learning performance. In this article, we consider the quantum control problem where only the target state can be accurately identified and introduce a quantum-inspired RL (QiRL) method. In particular, we propose a quantum-inspired exploration strategy to replace a commonly used$\epsilon $-greedy strategy, as well as a quantum-inspired reward scheme to incentivize the learning agent. Numerical results on three quantum system control problems, i.e., one-qubit closed quantum system, two-level open quantum system, and many-qubit closed quantum system, verify the effectiveness of QiRL. Comparison results show that the proposed QiRL outperforms existing RL algorithms (deep Q-network and proximal policy optimization) in terms of stability and efficiency for solving quantum control problems.