Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Machine Learning Meets Quantum State Preparation. The Phase Diagram of Quantum Control

Marin Bukov, Alexandre G. R. Day, Dries Sels, Phillip Weinberg, Anatoli Polkovnikov, Pankaj Mehta

Published May 1, 2017• 4 citations
OpenAlex

Abstract

The ability to prepare a physical system in a desired quantum state is central to many areas of physics such as nuclear magnetic resonance, cold atoms, and quantum computing. Yet, preparing states quickly and with high fidelity remains a formidable challenge. In this work we implement cutting-edge Reinforcement Learning (RL) techniques and show that their performance is comparable to optimal control methods in the task of finding short, high-fidelity driving protocol from an initial to a target state in non-integrable many-body quantum systems of interacting qubits. RL methods learn about the underlying physical system solely through a single scalar reward (the fidelity of the resulting state) calculated from numerical simulations of the physical system. We further show that quantum state manipulation, viewed as an optimization problem, exhibits a spin-glass-like phase transition in the space of protocols as a function of the protocol duration. Our RL-aided approach helps identify variational protocols with nearly optimal fidelity, even in the glassy phase, where optimal state manipulation is exponentially hard. This study highlights the potential usefulness of RL for applications in out-of-equilibrium quantum physics.

Related papers

Open quantum system control based on reinforcement learning

Peng Wei, Na Li, Zairong Xi

The fast preparation of a quantum state with high fidelity is a key problem for a quantum system. We propose two modified algorithms based on the cutting-edge reinforcement learning methods for the flip from an initial state to a target one in a quantum spin system. The balance between exploration and exploitation and the size of a state space are key factors for reinforcement learning methods. In the first algorithm, we propose a modified $\epsilon$-greedy strategy instead of the ε-greedy strategy to balance exploration and exploitation. Furthermore, we use the fidelity of the final state as the reward and leverage piecewise-constant driving protocols in every duration. The results show that the rate of correction of this algorithm is greater than that of Q-learning with ε-greedy strategy. However, it will still cost much time resource if the rotation angle from a state to the next state is too small. Thus, we initialize the state-action value table to reduce the state space in the second modified Q-learning algorithm (MQL), which can be found that the efficiency of learning is greatly improved.

Chinese Control Conference (CCC)•Jul 1, 2019•3 citations

Glassy Phase of Optimal Quantum Control

Alexandre G. R. Day, Marin Bukov, Phillip Weinberg +2 more

We study the problem of preparing a quantum many-body system from an initial to a target state by optimizing the fidelity over the family of bang-bang protocols. We present compelling numerical evidence for a universal spin-glasslike transition controlled by the protocol time duration. The glassy critical point is marked by a proliferation of protocols with close-to-optimal fidelity and with a true optimum that appears exponentially difficult to locate. Using a machine learning (ML) inspired framework based on the manifold learning algorithm t-distributed stochastic neighbor embedding, we are able to visualize the geometry of the high-dimensional control landscape in an effective low-dimensional representation. Across the transition, the control landscape features an exponential number of clusters separated by extensive barriers, which bears a strong resemblance with replica symmetry breaking in spin glasses and random satisfiability problems. We further show that the quantum control landscape maps onto a disorder-free classical Ising model with frustrated nonlocal, multibody interactions. Our work highlights an intricate but unexpected connection between optimal quantum control and spin glass physics, and shows how tools from ML can be used to visualize and understand glassy optimization landscapes.

arXiv:1803.10856•Physical Review Letters•Jan 15, 2019•67 citations

Reinforcement Learning for Quantum Technology

Marin Bukov, Florian Marquardt

Many challenges arising in Quantum Technology can be successfully addressed using a set of machine learning algorithms collectively known as reinforcement learning (RL), based on adaptive decision-making through interaction with the quantum device. After a concise and intuitive introduction to RL aimed at a broad physics readership, we discuss the key ideas and core concepts in reinforcement learning with a particular focus on quantum systems. We then survey recent progress in RL in all relevant areas. We discuss state preparation in few- and many-body quantum systems, the design and optimization of high-fidelity quantum gates, and the automated construction of quantum circuits, including applications to variational quantum eigensolvers and architecture search. We further highlight the interactive capabilities of RL agents, emphasizing recent progress in quantum feedback control and quantum error correction, and briefly discuss quantum reinforcement learning as well as applications to quantum metrology. The review concludes with a discussion of open challenges -- such as scalability, interpretability, and integration with experimental platforms -- and outlines promising directions for future research. Throughout, we highlight experimental implementations that exemplify the increasing role of reinforcement learning in shaping the development of quantum technologies.

quant-pharXiv:2601.18953•Jan 28, 2026•0 citations

Quantum control of solid state qubits via Reinforcement Learning

Dennis Herb, Mirko Rossini, Sabine Wölk +3 more

The use of reinforcement learning (RL) in the design of quantum control strategies for pulse level control of NISQ era quantum devices is attracting growing interest. Model-free RL, not requiring preexisting knowledge of the physical system, acts as a black-box optimization tool well suited for noisy devices. In this work, we provide a simulation-based analysis of the performance of model-free RL for state preparation and qubit gate design in a challenging setup: indirect control of NV nuclear spins in a spin-bath. In particular, the spin-bath is modeled via the generalized cluster correlation expansion accounting for magnetic impurities and intra-bath interactions. We analyze the performance of the RL agents by performing simulations with minimally observable environments and noisy, sparse rewards as introduced in (Phys. Rev. X 12, 011059), and compare them to other state-of-the-art black-box optimization tools.

quant-pharXiv:2610.21987•Oct 16, 2026•0 citations

Critical States Preparation With Deep Reinforcement Learning

Jia-Wen Yu, Yi-Ming Yu, Ke-Xiong Yan +4 more

The fast and efficient preparation of quantum critical states is a challenging yet crucial task for various quantum technologies. This difficulty is most particularly for systems near a quantum phase transition, where the closure of the energy gap fundamentally limits the timescale of adiabatic processes and thus precludes rapid state preparation. We propose a framework using deep reinforcement learning (DRL) to rapidly prepare quantum critical states, with broad extendibility to light-matter interaction systems. Specifically, a DRL agent optimizes a set of time-dependent control Hamiltonians to drive the system from an initial noncritical state to a target critical state within a finite time and over experimentally accessible parameter ranges. As a concrete application, we focus on the quantum Rabi model. The DRL-optimized time-dependent control Hamiltonian yield a final state with high-fidelity ($>0.999$) to the target critical state. The protocol can be readily extended to other quantum critical systems described by light-matter interaction models, such as quantum Dicke model. This investigation provides a powerful new framework for preparing and manipulating quantum critical states.

quant-pharXiv:2603.09135•Mar 11, 2026•0 citations

Adaptive Quantum Control Algorithm Optimization

Jincheng Zhang

This paper investigates the optimization of quantum control algorithms using reinforcement learning (RL). The core claim is that RL can significantly enhance the control precision and efficiency of quantum devices. We propose a framework where the quantum control algorithm is treated as an intelligent agent, interacting with the quantum environment to learn optimal control strategies. This approach allows for adaptive control, addressing the challenges inherent in designing fixed control protocols for complex quantum systems. The methodology involves defining a reward function that reflects the desired quantum state evolution, using RL algorithms to train the agent, and validating the learned control strategy through simulations. We demonstrate the potential of this approach to improve control accuracy and robustness, particularly in scenarios with noisy environments or rapidly changing system parameters. The results suggest a promising direction for leveraging machine learning techniques to unlock the full potential of quantum control.

•Sep 6, 2026•0 citations
The ability to prepare a physical system in a desired quantum state is central to many areas of physics such as nuclear magnetic resonance, cold atoms, and quantum computing. Yet, preparing states quickly and with high fidelity remains a formidable challenge. In this work we implement cutting-edge Reinforcement Learning (RL) techniques and show that their performance is comparable to optimal control methods in the task of finding short, high-fidelity driving protocol from an initial to a target state in non-integrable many-body quantum systems of interacting qubits. RL methods learn about the underlying physical system solely through a single scalar reward (the fidelity of the resulting state) calculated from numerical simulations of the physical system. We further show that quantum state manipulation, viewed as an optimization problem, exhibits a spin-glass-like phase transition in the space of protocols as a function of the protocol duration. Our RL-aided approach helps identify variational protocols with nearly optimal fidelity, even in the glassy phase, where optimal state manipulation is exponentially hard. This study highlights the potential usefulness of RL for applications in out-of-equilibrium quantum physics.
The fast preparation of a quantum state with high fidelity is a key problem for a quantum system. We propose two modified algorithms based on the cutting-edge reinforcement learning methods for the flip from an initial state to a target one in a quantum spin system. The balance between exploration and exploitation and the size of a state space are key factors for reinforcement learning methods. In the first algorithm, we propose a modified ϵ\epsilonϵ-greedy strategy instead of the ε-greedy strategy to balance exploration and exploitation. Furthermore, we use the fidelity of the final state as the reward and leverage piecewise-constant driving protocols in every duration. The results show that the rate of correction of this algorithm is greater than that of Q-learning with ε-greedy strategy. However, it will still cost much time resource if the rotation angle from a state to the next state is too small. Thus, we initialize the state-action value table to reduce the state space in the second modified Q-learning algorithm (MQL), which can be found that the efficiency of learning is greatly improved.
We study the problem of preparing a quantum many-body system from an initial to a target state by optimizing the fidelity over the family of bang-bang protocols. We present compelling numerical evidence for a universal spin-glasslike transition controlled by the protocol time duration. The glassy critical point is marked by a proliferation of protocols with close-to-optimal fidelity and with a true optimum that appears exponentially difficult to locate. Using a machine learning (ML) inspired framework based on the manifold learning algorithm t-distributed stochastic neighbor embedding, we are able to visualize the geometry of the high-dimensional control landscape in an effective low-dimensional representation. Across the transition, the control landscape features an exponential number of clusters separated by extensive barriers, which bears a strong resemblance with replica symmetry breaking in spin glasses and random satisfiability problems. We further show that the quantum control landscape maps onto a disorder-free classical Ising model with frustrated nonlocal, multibody interactions. Our work highlights an intricate but unexpected connection between optimal quantum control and spin glass physics, and shows how tools from ML can be used to visualize and understand glassy optimization landscapes.
Many challenges arising in Quantum Technology can be successfully addressed using a set of machine learning algorithms collectively known as reinforcement learning (RL), based on adaptive decision-making through interaction with the quantum device. After a concise and intuitive introduction to RL aimed at a broad physics readership, we discuss the key ideas and core concepts in reinforcement learning with a particular focus on quantum systems. We then survey recent progress in RL in all relevant areas. We discuss state preparation in few- and many-body quantum systems, the design and optimization of high-fidelity quantum gates, and the automated construction of quantum circuits, including applications to variational quantum eigensolvers and architecture search. We further highlight the interactive capabilities of RL agents, emphasizing recent progress in quantum feedback control and quantum error correction, and briefly discuss quantum reinforcement learning as well as applications to quantum metrology. The review concludes with a discussion of open challenges -- such as scalability, interpretability, and integration with experimental platforms -- and outlines promising directions for future research. Throughout, we highlight experimental implementations that exemplify the increasing role of reinforcement learning in shaping the development of quantum technologies.
The use of reinforcement learning (RL) in the design of quantum control strategies for pulse level control of NISQ era quantum devices is attracting growing interest. Model-free RL, not requiring preexisting knowledge of the physical system, acts as a black-box optimization tool well suited for noisy devices. In this work, we provide a simulation-based analysis of the performance of model-free RL for state preparation and qubit gate design in a challenging setup: indirect control of NV nuclear spins in a spin-bath. In particular, the spin-bath is modeled via the generalized cluster correlation expansion accounting for magnetic impurities and intra-bath interactions. We analyze the performance of the RL agents by performing simulations with minimally observable environments and noisy, sparse rewards as introduced in (Phys. Rev. X 12, 011059), and compare them to other state-of-the-art black-box optimization tools.
The fast and efficient preparation of quantum critical states is a challenging yet crucial task for various quantum technologies. This difficulty is most particularly for systems near a quantum phase transition, where the closure of the energy gap fundamentally limits the timescale of adiabatic processes and thus precludes rapid state preparation. We propose a framework using deep reinforcement learning (DRL) to rapidly prepare quantum critical states, with broad extendibility to light-matter interaction systems. Specifically, a DRL agent optimizes a set of time-dependent control Hamiltonians to drive the system from an initial noncritical state to a target critical state within a finite time and over experimentally accessible parameter ranges. As a concrete application, we focus on the quantum Rabi model. The DRL-optimized time-dependent control Hamiltonian yield a final state with high-fidelity (>0.999>0.999>0.999) to the target critical state. The protocol can be readily extended to other quantum critical systems described by light-matter interaction models, such as quantum Dicke model. This investigation provides a powerful new framework for preparing and manipulating quantum critical states.
This paper investigates the optimization of quantum control algorithms using reinforcement learning (RL). The core claim is that RL can significantly enhance the control precision and efficiency of quantum devices. We propose a framework where the quantum control algorithm is treated as an intelligent agent, interacting with the quantum environment to learn optimal control strategies. This approach allows for adaptive control, addressing the challenges inherent in designing fixed control protocols for complex quantum systems. The methodology involves defining a reward function that reflects the desired quantum state evolution, using RL algorithms to train the agent, and validating the learned control strategy through simulations. We demonstrate the potential of this approach to improve control accuracy and robustness, particularly in scenarios with noisy environments or rapidly changing system parameters. The results suggest a promising direction for leveraging machine learning techniques to unlock the full potential of quantum control.