Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:1808.08910 · Physical review. B./Physical review. B

Reinforcement learning for autonomous preparation of Floquet-engineered states: Inverting the quantum Kapitza oscillator

Marin Bukov

Published Dec 17, 2018• 70 citations
arXivHTMLPDF

Abstract

I demonstrate the potential of reinforcement learning (RL) to prepare quantum states of strongly periodically driven nonlinear single-particle models. The ability of Q-learning to control systems far away from equilibrium is exhibited by steering the quantum Kapitza oscillator to the Floquet-engineered stable inverted position in the presence of a strong periodic drive within several shaking cycles. The study reveals the potential of the intraperiod (micromotion) dynamics, often neglected in Floquet engineering, to take advantage over pure stroboscopic control at moderate drive frequencies. Without any knowledge about the underlying physical system, the algorithm is capable of learning solely from tried protocols and directly from simulated noisy quantum measurement data, and is stable to noise in the initial state and sources of random failure events in the control sequence. Model-free RL can provide new insights into automating experimental setups for out-of-equilibrium systems undergoing complex dynamics, with potential applications in quantum information, quantum optics, ultracold atoms, trapped ions, and condensed matter.

DOI
10.1103/physrevb.98.224305

Related papers

Quantum control and noise protection of a Floquet $0-π$ qubit

Zhaoyou Wang, Amir H. Safavi‐Naeini

Time-periodic systems allow engineering new effective Hamiltonians from limited physical interactions. For example, the inverted position of the Kapitza pendulum emerges as a stable equilibrium with rapid drive of its pivot point. In this work, we propose the $\textit{Kapitzonium}$: a Floquet qubit that is the superconducting circuit analog of a mechanical Kapitza pendulum. Under periodic driving, the emerging qubit states are exponentially protected against bit and phase flips caused by dissipation, which is the primary source of decoherence of current qubits. However, we find that dissipation causes leakage out of the Floquet qubit subspace. We engineer a passive cooling scheme to stabilize the qubit subspace, which is crucial for high fidelity quantum control under dissipation. Furthermore, we introduce a hardware-efficient fluorescence-based method for qubit measurement and discuss the experimental implementation of the Floquet qubit. The proposed Kapitzonium is one of the simplest Floquet qubits that can be realized with current technology -- and it already has many intriguing features and capabilities. Our work provides the first steps to develop more complex Floquet quantum systems from the ground up to realize large-scale protected engineered dynamics.

arXiv:2304.05601•Apr 12, 2023•0 citations

Machine Learning Meets Quantum State Preparation. The Phase Diagram of Quantum Control

Marin Bukov, Alexandre G. R. Day, Dries Sels +3 more

The ability to prepare a physical system in a desired quantum state is central to many areas of physics such as nuclear magnetic resonance, cold atoms, and quantum computing. Yet, preparing states quickly and with high fidelity remains a formidable challenge. In this work we implement cutting-edge Reinforcement Learning (RL) techniques and show that their performance is comparable to optimal control methods in the task of finding short, high-fidelity driving protocol from an initial to a target state in non-integrable many-body quantum systems of interacting qubits. RL methods learn about the underlying physical system solely through a single scalar reward (the fidelity of the resulting state) calculated from numerical simulations of the physical system. We further show that quantum state manipulation, viewed as an optimization problem, exhibits a spin-glass-like phase transition in the space of protocols as a function of the protocol duration. Our RL-aided approach helps identify variational protocols with nearly optimal fidelity, even in the glassy phase, where optimal state manipulation is exponentially hard. This study highlights the potential usefulness of RL for applications in out-of-equilibrium quantum physics.

•May 1, 2017•4 citations

An Apparatus for Dynamical Quantum Emulation Using Ultracold Lithium

Zachary A. Geiger

Quantum emulation of extreme non-equilibrium phenomena is a natural application of the precision and control of ultracold atomic physics experiments. In particular, ``Floquet engineering" with an explicitly time-dependent Hamiltonian enables the creation of states of matter that can only exist in the presence of strong driving. In this work, we create and study such tunable strongly driven quantum systems. We use quantum degenerate lithium as a platform because of its light mass, fast tunneling rates, and broadly tunable Feshbach resonance, which allow the study of a wide variety of systems: noninteracting Bosons, strongly interacting many body systems, and bright solitons, to name a few. We describe the experimental setup used to create a 7Li Bose-Einstein condensate as an archetypal quantum platform, and then discuss three key results: the first direct measurement of position-space Bloch oscillations in a cold atom system, the creation of a massive relativistic harmonic oscillator, and finally the demonstration of a new method for controlled long-range quantum coherent transport enabled by Floquet band engineering.

•Jan 1, 2017•1 citation

Self-correcting quantum many-body control using reinforcement learning with tensor networks

Friederike Metz, Marin Bukov

Abstract Quantum many-body control is a central milestone en route to harnessing quantum technologies. However, the exponential growth of the Hilbert space dimension with the number of qubits makes it challenging to classically simulate quantum many-body systems and, consequently, to devise reliable and robust optimal control protocols. Here we present a framework for efficiently controlling quantum many-body systems based on reinforcement learning (RL). We tackle the quantum-control problem by leveraging matrix product states (1) for representing the many-body state and (2) as part of the trainable machine learning architecture for our RL agent. The framework is applied to prepare ground states of the quantum Ising chain, including states in the critical region. It allows us to control systems far larger than neural-network-only architectures permit, while retaining the advantages of deep learning algorithms, such as generalizability and trainable robustness to noise. In particular, we demonstrate that RL agents are capable of finding universal controls, of learning how to optimally steer previously unseen many-body states and of adapting control protocols on the fly when the quantum dynamics is subject to stochastic perturbations. Furthermore, we map our RL framework to a hybrid quantum–classical algorithm that can be performed on noisy intermediate-scale quantum devices and test it under the presence of experimentally relevant sources of noise.

arXiv:2201.11790•Nature Machine Intelligence•Jul 21, 2023•45 citations

Robust quantum control using reinforcement learning from demonstration

Shengyong Li, Yidian Fan, Xiangyang Li +6 more

Quantum control requires high-precision and robust control pulses to ensure optimal system performance. However, control sequences generated with a system model may suffer from model bias, leading to low fidelity. While model-free reinforcement learning (RL) methods have been developed to avoid such biases, training an RL agent from scratch can be time-consuming, often taking hours to gather enough samples for convergence. This challenge has hindered the broad application of RL techniques to larger and more complex quantum control issues, limiting their adaptability. In this work, we use Reinforcement Learning from Demonstration (RLfD) to leverage the control sequences generated with system models and further optimize them with RL to avoid model bias. By avoiding learning from scratch and starting with reasonable control pulse shapes, this approach can increase sample efficiency by reducing the number of samples, which can significantly reduce the training time. Thus, this method can effectively handle pulse shapes that are discretized into more than 1000 pieces without compromising final fidelity. We have simulated the preparation of several high-fidelity non-classical states using the RLfD method. We also find that the training process is more stable when using RLfD. In addition, this method is suitable for fast gate calibration using reinforcement learning.

arXiv:2503.21085•npj Quantum Information•Jul 25, 2025•12 citations

Quantum control of solid state qubits via Reinforcement Learning

Dennis Herb, Mirko Rossini, Sabine Wölk +3 more

The use of reinforcement learning (RL) in the design of quantum control strategies for pulse level control of NISQ era quantum devices is attracting growing interest. Model-free RL, not requiring preexisting knowledge of the physical system, acts as a black-box optimization tool well suited for noisy devices. In this work, we provide a simulation-based analysis of the performance of model-free RL for state preparation and qubit gate design in a challenging setup: indirect control of NV nuclear spins in a spin-bath. In particular, the spin-bath is modeled via the generalized cluster correlation expansion accounting for magnetic impurities and intra-bath interactions. We analyze the performance of the RL agents by performing simulations with minimally observable environments and noisy, sparse rewards as introduced in (Phys. Rev. X 12, 011059), and compare them to other state-of-the-art black-box optimization tools.

quant-pharXiv:2610.21987•Oct 16, 2026•0 citations
I demonstrate the potential of reinforcement learning (RL) to prepare quantum states of strongly periodically driven nonlinear single-particle models. The ability of Q-learning to control systems far away from equilibrium is exhibited by steering the quantum Kapitza oscillator to the Floquet-engineered stable inverted position in the presence of a strong periodic drive within several shaking cycles. The study reveals the potential of the intraperiod (micromotion) dynamics, often neglected in Floquet engineering, to take advantage over pure stroboscopic control at moderate drive frequencies. Without any knowledge about the underlying physical system, the algorithm is capable of learning solely from tried protocols and directly from simulated noisy quantum measurement data, and is stable to noise in the initial state and sources of random failure events in the control sequence. Model-free RL can provide new insights into automating experimental setups for out-of-equilibrium systems undergoing complex dynamics, with potential applications in quantum information, quantum optics, ultracold atoms, trapped ions, and condensed matter.
Quantum control and noise protection of a Floquet 0−π0-π0−π qubit
Time-periodic systems allow engineering new effective Hamiltonians from limited physical interactions. For example, the inverted position of the Kapitza pendulum emerges as a stable equilibrium with rapid drive of its pivot point. In this work, we propose the Kapitzonium\textit{Kapitzonium}Kapitzonium: a Floquet qubit that is the superconducting circuit analog of a mechanical Kapitza pendulum. Under periodic driving, the emerging qubit states are exponentially protected against bit and phase flips caused by dissipation, which is the primary source of decoherence of current qubits. However, we find that dissipation causes leakage out of the Floquet qubit subspace. We engineer a passive cooling scheme to stabilize the qubit subspace, which is crucial for high fidelity quantum control under dissipation. Furthermore, we introduce a hardware-efficient fluorescence-based method for qubit measurement and discuss the experimental implementation of the Floquet qubit. The proposed Kapitzonium is one of the simplest Floquet qubits that can be realized with current technology -- and it already has many intriguing features and capabilities. Our work provides the first steps to develop more complex Floquet quantum systems from the ground up to realize large-scale protected engineered dynamics.
The ability to prepare a physical system in a desired quantum state is central to many areas of physics such as nuclear magnetic resonance, cold atoms, and quantum computing. Yet, preparing states quickly and with high fidelity remains a formidable challenge. In this work we implement cutting-edge Reinforcement Learning (RL) techniques and show that their performance is comparable to optimal control methods in the task of finding short, high-fidelity driving protocol from an initial to a target state in non-integrable many-body quantum systems of interacting qubits. RL methods learn about the underlying physical system solely through a single scalar reward (the fidelity of the resulting state) calculated from numerical simulations of the physical system. We further show that quantum state manipulation, viewed as an optimization problem, exhibits a spin-glass-like phase transition in the space of protocols as a function of the protocol duration. Our RL-aided approach helps identify variational protocols with nearly optimal fidelity, even in the glassy phase, where optimal state manipulation is exponentially hard. This study highlights the potential usefulness of RL for applications in out-of-equilibrium quantum physics.
Quantum emulation of extreme non-equilibrium phenomena is a natural application of the precision and control of ultracold atomic physics experiments. In particular, ``Floquet engineering" with an explicitly time-dependent Hamiltonian enables the creation of states of matter that can only exist in the presence of strong driving. In this work, we create and study such tunable strongly driven quantum systems. We use quantum degenerate lithium as a platform because of its light mass, fast tunneling rates, and broadly tunable Feshbach resonance, which allow the study of a wide variety of systems: noninteracting Bosons, strongly interacting many body systems, and bright solitons, to name a few. We describe the experimental setup used to create a 7Li Bose-Einstein condensate as an archetypal quantum platform, and then discuss three key results: the first direct measurement of position-space Bloch oscillations in a cold atom system, the creation of a massive relativistic harmonic oscillator, and finally the demonstration of a new method for controlled long-range quantum coherent transport enabled by Floquet band engineering.
Abstract Quantum many-body control is a central milestone en route to harnessing quantum technologies. However, the exponential growth of the Hilbert space dimension with the number of qubits makes it challenging to classically simulate quantum many-body systems and, consequently, to devise reliable and robust optimal control protocols. Here we present a framework for efficiently controlling quantum many-body systems based on reinforcement learning (RL). We tackle the quantum-control problem by leveraging matrix product states (1) for representing the many-body state and (2) as part of the trainable machine learning architecture for our RL agent. The framework is applied to prepare ground states of the quantum Ising chain, including states in the critical region. It allows us to control systems far larger than neural-network-only architectures permit, while retaining the advantages of deep learning algorithms, such as generalizability and trainable robustness to noise. In particular, we demonstrate that RL agents are capable of finding universal controls, of learning how to optimally steer previously unseen many-body states and of adapting control protocols on the fly when the quantum dynamics is subject to stochastic perturbations. Furthermore, we map our RL framework to a hybrid quantum–classical algorithm that can be performed on noisy intermediate-scale quantum devices and test it under the presence of experimentally relevant sources of noise.
Quantum control requires high-precision and robust control pulses to ensure optimal system performance. However, control sequences generated with a system model may suffer from model bias, leading to low fidelity. While model-free reinforcement learning (RL) methods have been developed to avoid such biases, training an RL agent from scratch can be time-consuming, often taking hours to gather enough samples for convergence. This challenge has hindered the broad application of RL techniques to larger and more complex quantum control issues, limiting their adaptability. In this work, we use Reinforcement Learning from Demonstration (RLfD) to leverage the control sequences generated with system models and further optimize them with RL to avoid model bias. By avoiding learning from scratch and starting with reasonable control pulse shapes, this approach can increase sample efficiency by reducing the number of samples, which can significantly reduce the training time. Thus, this method can effectively handle pulse shapes that are discretized into more than 1000 pieces without compromising final fidelity. We have simulated the preparation of several high-fidelity non-classical states using the RLfD method. We also find that the training process is more stable when using RLfD. In addition, this method is suitable for fast gate calibration using reinforcement learning.
The use of reinforcement learning (RL) in the design of quantum control strategies for pulse level control of NISQ era quantum devices is attracting growing interest. Model-free RL, not requiring preexisting knowledge of the physical system, acts as a black-box optimization tool well suited for noisy devices. In this work, we provide a simulation-based analysis of the performance of model-free RL for state preparation and qubit gate design in a challenging setup: indirect control of NV nuclear spins in a spin-bath. In particular, the spin-bath is modeled via the generalized cluster correlation expansion accounting for magnetic impurities and intra-bath interactions. We analyze the performance of the RL agents by performing simulations with minimally observable environments and noisy, sparse rewards as introduced in (Phys. Rev. X 12, 011059), and compare them to other state-of-the-art black-box optimization tools.