Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2207.12656 · Physical Review Applied

Deep Reinforcement Learning for Preparation of Thermal and Prethermal Quantum States

S. Baba, Nobuyuki Yoshioka, Yuto Ashida, Takahiro Sagawa

Published Jan 26, 2023• 10 citations
arXivHTMLPDF

Abstract

We propose a method based on deep reinforcement learning that efficiently prepares a quantum many-body pure state in thermal or prethermal equilibrium. The main physical intuition underlying the method is that the information on the equilibrium states can be efficiently encoded and/or extracted by focusing on only a few local observables, relying on the typicality of equilibrium states. Instead of resorting to the expensive preparation protocol that adopts global features such as the quantum state fidelity, we show that the equilibrium states can be efficiently prepared only by learning the expectation values of local observables. We demonstrate our method by preparing two illustrative examples: Gibbs ensembles in nonintegrable systems and generalized Gibbs ensembles in integrable systems. Pure states prepared solely from local observables are numerically shown to successfully encode the macroscopic properties of the equilibrium states. Furthermore, we find that the preparation errors, with respect to the system size, decay exponentially for Gibbs ensembles and polynomially for generalized Gibbs ensembles, which is in agreement with the finite-size fluctuation within thermodynamic ensembles. Our method paves a path toward studying the thermodynamic and statistical properties of quantum many-body systems in quantum hardware.

DOI
10.1103/physrevapplied.19.014068

Related papers

Optimal statistical ensembles for quantum thermal state preparation within the quantum singular value transformation framework

Yasushi Yoneta

Preparing thermal equilibrium states is an essential task for finite-temperature quantum simulations. In statistical mechanics, microstates in thermal equilibrium can be obtained from statistical ensembles. To date, numerous ensembles have been devised, ranging from Gibbs ensembles such as the canonical and microcanonical ensembles to a variety of generalized ensembles. Since these ensembles yield equivalent thermodynamic predictions, one can freely choose an ensemble for computational convenience. In this paper, we exploit this flexibility to develop an efficient quantum algorithm for preparing thermal equilibrium states. We first present a quantum algorithm for implementing generalized ensembles within the framework of quantum singular value transformation. We then perform a detailed analysis of the computational cost and elucidate its dependence on the choice of the ensemble. Our analysis shows that employing an appropriate ensemble can significantly mitigate ensemble-dependent overhead and yield improved scaling of the computational cost with system size compared to existing methods based on the canonical ensemble. We also numerically demonstrate that our approach achieves a significant reduction in the computational cost even for small finite-size systems. Our algorithm applies to arbitrary thermodynamic systems at any temperature and is thus expected to offer a practical and versatile method for computing finite-temperature properties of quantum many-body systems. These results highlight the potential of ensemble design as a powerful tool for enhancing the efficiency of a broad class of quantum algorithms.

arXiv:2505.06216•May 9, 2025•0 citations

Machine Learning Meets Quantum State Preparation. The Phase Diagram of Quantum Control

Marin Bukov, Alexandre G. R. Day, Dries Sels +3 more

The ability to prepare a physical system in a desired quantum state is central to many areas of physics such as nuclear magnetic resonance, cold atoms, and quantum computing. Yet, preparing states quickly and with high fidelity remains a formidable challenge. In this work we implement cutting-edge Reinforcement Learning (RL) techniques and show that their performance is comparable to optimal control methods in the task of finding short, high-fidelity driving protocol from an initial to a target state in non-integrable many-body quantum systems of interacting qubits. RL methods learn about the underlying physical system solely through a single scalar reward (the fidelity of the resulting state) calculated from numerical simulations of the physical system. We further show that quantum state manipulation, viewed as an optimization problem, exhibits a spin-glass-like phase transition in the space of protocols as a function of the protocol duration. Our RL-aided approach helps identify variational protocols with nearly optimal fidelity, even in the glassy phase, where optimal state manipulation is exponentially hard. This study highlights the potential usefulness of RL for applications in out-of-equilibrium quantum physics.

•May 1, 2017•4 citations

Efficient learning of ground and thermal states within phases of matter

Cambyse Rouzé, Daniel Stilck França, Emilio Onorati +1 more

We consider two related tasks: (a) estimating a parameterisation of a given Gibbs state and expectation values of Lipschitz observables on this state; (b) learning the expectation values of local observables within a thermal or quantum phase of matter. In both cases, we present sample-efficient ways to learn these properties to high precision. For the first task, we develop techniques to learn parameterisations of classes of systems, including quantum Gibbs states for classes of non-commuting Hamiltonians. We then give methods to sample-efficiently infer expectation values of extensive properties of the state, including quasi-local observables and entropies. For the second task, we exploit the locality of Hamiltonians to show that M local observables can be learned with probability 1 − δ and precision ε using $$N={\mathcal{O}}\left(\log \left(\frac{M}{\delta }\right){e}^{{\rm{polylog}}({\varepsilon }^{-1})}\right)$$ samples — exponentially improving previous bounds. Our results apply to both families of ground states of Hamiltonians displaying local topological quantum order, and thermal phases of matter with exponentially decaying correlations. Scalable characterisation of quantum states can be achieved by leveraging on simplifications valid for specific classes of states. Here, the authors show how to combine the strengths of shadows and many-body tomography for all states exhibiting exponential decay of correlations and the approximate Markov property.

Nature Communications•Sep 5, 2024•22 citations

Sample-efficient learning of quantum many-body systems

Anurag Anshu, Srinivasan Arunachalam, Tomotaka Kuwahara +1 more

We study the problem of learning the Hamiltonian of a quantum many-body system given samples from its Gibbs (thermal) state. The classical analog of this problem, known as learning graphical models or Boltzmann machines, is a well-studied question in machine learning and statistics. In this work, we give the first sample-efficient algorithm for the quantum Hamiltonian learning problem. In particular, we prove that polynomially many samples in the number of particles (qudits) are necessary and sufficient for learning the parameters of a spatially local Hamiltonian in l_2-norm. Our main contribution is in establishing the strong convexity of the log-partition function of quantum many-body systems, which along with the maximum entropy estimation yields our sample-efficient algorithm. Classically, the strong convexity for partition functions follows from the Markov property of Gibbs distributions. This is, however, known to be violated in its exact form in the quantum case. We introduce several new ideas to obtain an unconditional result that avoids relying on the Markov property of quantum systems, at the cost of a slightly weaker bound. In particular, we prove a lower bound on the variance of quasi-local operators with respect to the Gibbs state, which might be of independent interest. Our work paves the way toward a more rigorous application of machine learning techniques to quantum many-body problems.

Annual Symposium on Foundations of Computer Science•Nov 1, 2020•10 citations

Classifying global state preparation via deep reinforcement learning

Tobias Haug, Wai-Keong Mok, Jia-Bin You +3 more

Abstract Quantum information processing often requires the preparation of arbitrary quantum states, such as all the states on the Bloch sphere for two-level systems. While numerical optimization can prepare individual target states, they lack the ability to find general control protocols that can generate many different target states. Here, we demonstrate global quantum control by preparing a continuous set of states with deep reinforcement learning. The protocols are represented using neural networks, which automatically groups the protocols into similar types, which could be useful for finding classes of protocols and extracting physical insights. As application, we generate arbitrary superposition states for the electron spin in complex multi-level nitrogen-vacancy centers, revealing classes of protocols characterized by specific preparation timescales. Our method could help improve control of near-term quantum computers, quantum sensing devices and quantum simulations.

arXiv:2005.12759•Machine Learning Science and Technology•Nov 5, 2020•36 citations

Efficient learning of ground & thermal states within phases of matter

Emilio Onorati, Cambyse Rouzé, Daniel Stilck França +1 more

We consider two related tasks: (a) estimating a parameterisation of a given Gibbs state and expectation values of Lipschitz observables on this state; and (b) learning the expectation values of local observables within a thermal or quantum phase of matter. In both cases, we wish to minimise the number of samples we use to learn these properties to a given precision. For the first task, we develop new techniques to learn parameterisations of classes of systems, including quantum Gibbs states of non-commuting Hamiltonians with exponential decay of correlations and the approximate Markov property. We show it is possible to infer the expectation values of all extensive properties of the state from a number of copies that not only scales polylogarithmically with the system size, but polynomially in the observable's locality -- an exponential improvement. This set of properties includes expected values of quasi-local observables and entropies. For the second task, we develop efficient algorithms for learning observables in a phase of matter of a quantum system. By exploiting the locality of the Hamiltonian, we show that $M$ local observables can be learned with probability $1-δ$ to precision $ε$ with using only $N=O\big(\log\big(\frac{M}δ\big)e^{polylog(ε^{-1})}\big)$ samples -- an exponential improvement on the precision over previous bounds. Our results apply to both families of ground states of Hamiltonians displaying local topological quantum order, and thermal phases of matter with exponential decay of correlations. In addition, our sample complexity applies to the worse case setting whereas previous results only applied on average. Furthermore, we develop tools of independent interest, such as robust shadow tomography algorithms, Gibbs approximations to ground states, and generalisations of transportation cost inequalities for Gibbs states.

arXiv:2301.12946•Jan 30, 2023•3 citations
We propose a method based on deep reinforcement learning that efficiently prepares a quantum many-body pure state in thermal or prethermal equilibrium. The main physical intuition underlying the method is that the information on the equilibrium states can be efficiently encoded and/or extracted by focusing on only a few local observables, relying on the typicality of equilibrium states. Instead of resorting to the expensive preparation protocol that adopts global features such as the quantum state fidelity, we show that the equilibrium states can be efficiently prepared only by learning the expectation values of local observables. We demonstrate our method by preparing two illustrative examples: Gibbs ensembles in nonintegrable systems and generalized Gibbs ensembles in integrable systems. Pure states prepared solely from local observables are numerically shown to successfully encode the macroscopic properties of the equilibrium states. Furthermore, we find that the preparation errors, with respect to the system size, decay exponentially for Gibbs ensembles and polynomially for generalized Gibbs ensembles, which is in agreement with the finite-size fluctuation within thermodynamic ensembles. Our method paves a path toward studying the thermodynamic and statistical properties of quantum many-body systems in quantum hardware.
Preparing thermal equilibrium states is an essential task for finite-temperature quantum simulations. In statistical mechanics, microstates in thermal equilibrium can be obtained from statistical ensembles. To date, numerous ensembles have been devised, ranging from Gibbs ensembles such as the canonical and microcanonical ensembles to a variety of generalized ensembles. Since these ensembles yield equivalent thermodynamic predictions, one can freely choose an ensemble for computational convenience. In this paper, we exploit this flexibility to develop an efficient quantum algorithm for preparing thermal equilibrium states. We first present a quantum algorithm for implementing generalized ensembles within the framework of quantum singular value transformation. We then perform a detailed analysis of the computational cost and elucidate its dependence on the choice of the ensemble. Our analysis shows that employing an appropriate ensemble can significantly mitigate ensemble-dependent overhead and yield improved scaling of the computational cost with system size compared to existing methods based on the canonical ensemble. We also numerically demonstrate that our approach achieves a significant reduction in the computational cost even for small finite-size systems. Our algorithm applies to arbitrary thermodynamic systems at any temperature and is thus expected to offer a practical and versatile method for computing finite-temperature properties of quantum many-body systems. These results highlight the potential of ensemble design as a powerful tool for enhancing the efficiency of a broad class of quantum algorithms.
The ability to prepare a physical system in a desired quantum state is central to many areas of physics such as nuclear magnetic resonance, cold atoms, and quantum computing. Yet, preparing states quickly and with high fidelity remains a formidable challenge. In this work we implement cutting-edge Reinforcement Learning (RL) techniques and show that their performance is comparable to optimal control methods in the task of finding short, high-fidelity driving protocol from an initial to a target state in non-integrable many-body quantum systems of interacting qubits. RL methods learn about the underlying physical system solely through a single scalar reward (the fidelity of the resulting state) calculated from numerical simulations of the physical system. We further show that quantum state manipulation, viewed as an optimization problem, exhibits a spin-glass-like phase transition in the space of protocols as a function of the protocol duration. Our RL-aided approach helps identify variational protocols with nearly optimal fidelity, even in the glassy phase, where optimal state manipulation is exponentially hard. This study highlights the potential usefulness of RL for applications in out-of-equilibrium quantum physics.
We consider two related tasks: (a) estimating a parameterisation of a given Gibbs state and expectation values of Lipschitz observables on this state; (b) learning the expectation values of local observables within a thermal or quantum phase of matter. In both cases, we present sample-efficient ways to learn these properties to high precision. For the first task, we develop techniques to learn parameterisations of classes of systems, including quantum Gibbs states for classes of non-commuting Hamiltonians. We then give methods to sample-efficiently infer expectation values of extensive properties of the state, including quasi-local observables and entropies. For the second task, we exploit the locality of Hamiltonians to show that M local observables can be learned with probability 1 − δ and precision ε using N=O(log⁡(Mδ)epolylog(ε−1))N={\mathcal{O}}\left(\log \left(\frac{M}{\delta }\right){e}^{{\rm{polylog}}({\varepsilon }^{-1})}\right)N=O(log(δM​)epolylog(ε−1)) samples — exponentially improving previous bounds. Our results apply to both families of ground states of Hamiltonians displaying local topological quantum order, and thermal phases of matter with exponentially decaying correlations. Scalable characterisation of quantum states can be achieved by leveraging on simplifications valid for specific classes of states. Here, the authors show how to combine the strengths of shadows and many-body tomography for all states exhibiting exponential decay of correlations and the approximate Markov property.
We study the problem of learning the Hamiltonian of a quantum many-body system given samples from its Gibbs (thermal) state. The classical analog of this problem, known as learning graphical models or Boltzmann machines, is a well-studied question in machine learning and statistics. In this work, we give the first sample-efficient algorithm for the quantum Hamiltonian learning problem. In particular, we prove that polynomially many samples in the number of particles (qudits) are necessary and sufficient for learning the parameters of a spatially local Hamiltonian in l_2-norm. Our main contribution is in establishing the strong convexity of the log-partition function of quantum many-body systems, which along with the maximum entropy estimation yields our sample-efficient algorithm. Classically, the strong convexity for partition functions follows from the Markov property of Gibbs distributions. This is, however, known to be violated in its exact form in the quantum case. We introduce several new ideas to obtain an unconditional result that avoids relying on the Markov property of quantum systems, at the cost of a slightly weaker bound. In particular, we prove a lower bound on the variance of quasi-local operators with respect to the Gibbs state, which might be of independent interest. Our work paves the way toward a more rigorous application of machine learning techniques to quantum many-body problems.
Abstract Quantum information processing often requires the preparation of arbitrary quantum states, such as all the states on the Bloch sphere for two-level systems. While numerical optimization can prepare individual target states, they lack the ability to find general control protocols that can generate many different target states. Here, we demonstrate global quantum control by preparing a continuous set of states with deep reinforcement learning. The protocols are represented using neural networks, which automatically groups the protocols into similar types, which could be useful for finding classes of protocols and extracting physical insights. As application, we generate arbitrary superposition states for the electron spin in complex multi-level nitrogen-vacancy centers, revealing classes of protocols characterized by specific preparation timescales. Our method could help improve control of near-term quantum computers, quantum sensing devices and quantum simulations.
We consider two related tasks: (a) estimating a parameterisation of a given Gibbs state and expectation values of Lipschitz observables on this state; and (b) learning the expectation values of local observables within a thermal or quantum phase of matter. In both cases, we wish to minimise the number of samples we use to learn these properties to a given precision. For the first task, we develop new techniques to learn parameterisations of classes of systems, including quantum Gibbs states of non-commuting Hamiltonians with exponential decay of correlations and the approximate Markov property. We show it is possible to infer the expectation values of all extensive properties of the state from a number of copies that not only scales polylogarithmically with the system size, but polynomially in the observable's locality -- an exponential improvement. This set of properties includes expected values of quasi-local observables and entropies. For the second task, we develop efficient algorithms for learning observables in a phase of matter of a quantum system. By exploiting the locality of the Hamiltonian, we show that MMM local observables can be learned with probability 1−δ1-δ1−δ to precision εεε with using only N=O(log⁡(Mδ)epolylog(ε−1))N=O\big(\log\big(\frac{M}δ\big)e^{polylog(ε^{-1})}\big)N=O(log(δM​)epolylog(ε−1)) samples -- an exponential improvement on the precision over previous bounds. Our results apply to both families of ground states of Hamiltonians displaying local topological quantum order, and thermal phases of matter with exponential decay of correlations. In addition, our sample complexity applies to the worse case setting whereas previous results only applied on average. Furthermore, we develop tools of independent interest, such as robust shadow tomography algorithms, Gibbs approximations to ground states, and generalisations of transportation cost inequalities for Gibbs states.