Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2408.15328 · Quantum Science and Technology

Artificially intelligent Maxwell’s demon for optimal control of open quantum systems

Paolo Andrea Erdman, Robert Czupryniak, Bibek Bhandari, Andrew N. Jordan, Frank Noé, Jens Eisert, Giacomo Guarnieri

Published Mar 5, 2025• 4 citations
arXivHTMLPDF

Abstract

Abstract Feedback control of open quantum systems is of fundamental importance for practical applications in various contexts, ranging from quantum computation to quantum error correction and quantum metrology. Its use in the context of thermodynamics further enables the study of the interplay between information and energy. However, deriving optimal feedback control strategies is highly challenging, as it involves the optimal control of open quantum systems, the stochastic nature of quantum measurement, and the inclusion of policies that maximize a long-term time- and trajectory-averaged goal. In this work, we employ a reinforcement learning approach to automate and capture the role of a quantum Maxwell’s demon: the agent takes the literal role of discovering optimal feedback control strategies in qubit-based systems that maximize a trade-off between measurement-powered cooling and measurement efficiency. Considering weak or projective quantum measurements, we explore different regimes based on the ordering between the thermalization, the measurement, and the unitary feedback timescales, finding different and highly non-intuitive, yet interpretable, strategies. In the thermalization-dominated regime, we find strategies with elaborate finite-time thermalization protocols conditioned on measurement outcomes. In the measurement-dominated regime, we find that optimal strategies involve adaptively measuring different qubit observables reflecting the acquired information, and repeating multiple weak measurements until the quantum state is ‘sufficiently pure’, leading to random walks in state space. Finally, we study the case when all timescales are comparable, finding new feedback control strategies that considerably outperform more intuitive ones. We discuss a two-qubit example where we explore the role of entanglement and conclude discussing the scaling of our results to quantum many-body systems.

DOI
10.1088/2058-9565/adbccf

Related papers

Maxwell’s Demon Meets Nonequilibrium Quantum Thermodynamics

John Goold

Maxwell's demon explores the role of information in physical processes.Employing information about microscopic degrees of freedom, this "intelligent observer" is capable of compensating entropy production (or extracting work), apparently challenging the second law of thermodynamics.In a modern standpoint, it is regarded as a feedback control mechanism and the limits of thermodynamics are recast incorporating information-to-energy conversion.We derive a trade-off relation between information-theoretic quantities empowering the design of an efficient Maxwell's demon in a quantum system.The demon is experimentally implemented as a spin-1=2 quantum memory that acquires information, and employs it to control the dynamics of another spin-1=2 system, through a natural interaction.Noise and imperfections in this protocol are investigated by the assessment of its effectiveness.This realization provides experimental evidence that the irreversibility in a nonequilibrium dynamics can be mitigated by assessing microscopic information and applying a feed-forward strategy at the quantum scale.

Physics•Dec 5, 2016•2 citations

Experimental Rectification of Entropy Production by Maxwell’s Demon in a Quantum System

Patrice A. Camati, John P. S. Peterson, Tiago B. Batalhão +5 more

Maxwell's demon explores the role of information in physical processes. Employing information about microscopic degrees of freedom, this "intelligent observer" is capable of compensating entropy production (or extracting work), apparently challenging the second law of thermodynamics. In a modern standpoint, it is regarded as a feedback control mechanism and the limits of thermodynamics are recast incorporating information-to-energy conversion. We derive a trade-off relation between information-theoretic quantities empowering the design of an efficient Maxwell's demon in a quantum system. The demon is experimentally implemented as a spin-1/2 quantum memory that acquires information, and employs it to control the dynamics of another spin-1/2 system, through a natural interaction. Noise and imperfections in this protocol are investigated by the assessment of its effectiveness. This realization provides experimental evidence that the irreversibility in a nonequilibrium dynamics can be mitigated by assessing microscopic information and applying a feed-forward strategy at the quantum scale.

arXiv:1605.08821•Physical Review Letters•Dec 5, 2016•167 citations

Quantum Control based on Deep Reinforcement Learning

Zhikang Wang

In this thesis, we consider two simple but typical control problems and apply deep reinforcement learning to them, i.e., to cool and control a particle which is subject to continuous position measurement in a one-dimensional quadratic potential or in a quartic potential. We compare the performance of reinforcement learning control and conventional control strategies on the two problems, and show that the reinforcement learning achieves a performance comparable to the optimal control for the quadratic case, and outperforms conventional control strategies for the quartic case for which the optimal control strategy is unknown. To our knowledge, this is the first time deep reinforcement learning is applied to quantum control problems in continuous real space. Our research demonstrates that deep reinforcement learning can be used to control a stochastic quantum system in real space effectively as a measurement-feedback closed-loop controller, and our research also shows the ability of AI to discover new control strategies and properties of the quantum systems that are not well understood, and we can gain insights into these problems by learning from the AI, which opens up a new regime for scientific research.

arXiv:2212.07385•Dec 14, 2022•1 citation

Thermodynamics of Quantum Measurement and the Demon's Arrow of Time

Kagan Yanik, Bibek Bhandari, Sreenath K. Manikandan +1 more

We discuss the thermodynamic aspects of a single qubit based device, powered by weak quantum measurements, and feedback controlled by a quantum Maxwell's demon. We discuss both discrete and time-continuous operation of the measurement based device at finite temperature of the reservoir. In the discrete example where a demon acquires information via discrete weak measurements, we find that the thermodynamic variables including the heat exchanged, extractable work, and the entropy produced are completely determined by an information theoretic measure of the demon's perceived arrow of time. We also discuss a realistic time-continuous operation of the device where the feedback is applied after a sequence of weak measurements. In the time-continuous limit, we derive the exact finite-time statistics of work, heat and entropy changes along individual quantum trajectories of the quantum measurement process, and relate them to the demon's arrow of time.

arXiv:2205.08031•May 17, 2022•0 citations

Impact of Information on Quantum Heat Engines

Lindsay Bassman Oftelie, Michele Campisi

The emerging field of quantum thermodynamics is beginning to reveal the intriguing role that information can play in quantum thermal engines. Information enters as a resource when considering feedback-controlled thermal machines. While both a general theory of quantum feedback control as well as specific examples of quantum feedback-controlled engines have been presented, still lacking is a general framework for such machines. Here, we present a framework for a generic, two-stroke quantum heat engine interacting with $N$ thermal baths and Maxwell's demon. The demon performs projective measurements on the engine working substance, the outcome of which is recorded in a classical memory, embedded in its own thermal bath. To perform feedback control, the demon enacts unitary operations on the working substance, conditioned on the recorded outcome. By considering the compound machine-memory as a hybrid (classical-quantum) standard thermal machine interacting with $N+1$ thermal baths, our framework puts the working substance and memory on equal footing, thereby enabling a comprehensible resolution to Maxwell's paradox and elucidating the intricate manner in which information impacts the performance of quantum engines. We illustrate the application of our framework with a two-qubit engine. A remarkable observation is that more information does not necessarily result in better thermodynamic performance: sometimes knowing less is better.

quant-pharXiv:2512.13371•Physical Review A•Dec 16, 2025•0 citations

Measurement-Based Quantum Thermal Machines with Feedback Control

Bibek Bhandari, Robert Czupryniak, Paolo Andrea Erdman +1 more

We investigated coupled-qubit-based thermal machines powered by quantum measurements and feedback. We considered two different versions of the machine: (1) a quantum Maxwell's demon, where the coupled-qubit system is connected to a detachable single shared bath, and (2) a measurement-assisted refrigerator, where the coupled-qubit system is in contact with a hot and cold bath. In the quantum Maxwell's demon case, we discuss both discrete and continuous measurements. We found that the power output from a single qubit-based device can be improved by coupling it to the second qubit. We further found that the simultaneous measurement of both qubits can produce higher net heat extraction compared to two setups operated in parallel where only single-qubit measurements are performed. In the refrigerator case, we used continuous measurement and unitary operations to power the coupled-qubit-based refrigerator. We found that the cooling power of a refrigerator operated with swap operations can be enhanced by performing suitable measurements.

arXiv:2212.01502•Entropy•Jan 20, 2023•19 citations
Abstract Feedback control of open quantum systems is of fundamental importance for practical applications in various contexts, ranging from quantum computation to quantum error correction and quantum metrology. Its use in the context of thermodynamics further enables the study of the interplay between information and energy. However, deriving optimal feedback control strategies is highly challenging, as it involves the optimal control of open quantum systems, the stochastic nature of quantum measurement, and the inclusion of policies that maximize a long-term time- and trajectory-averaged goal. In this work, we employ a reinforcement learning approach to automate and capture the role of a quantum Maxwell’s demon: the agent takes the literal role of discovering optimal feedback control strategies in qubit-based systems that maximize a trade-off between measurement-powered cooling and measurement efficiency. Considering weak or projective quantum measurements, we explore different regimes based on the ordering between the thermalization, the measurement, and the unitary feedback timescales, finding different and highly non-intuitive, yet interpretable, strategies. In the thermalization-dominated regime, we find strategies with elaborate finite-time thermalization protocols conditioned on measurement outcomes. In the measurement-dominated regime, we find that optimal strategies involve adaptively measuring different qubit observables reflecting the acquired information, and repeating multiple weak measurements until the quantum state is ‘sufficiently pure’, leading to random walks in state space. Finally, we study the case when all timescales are comparable, finding new feedback control strategies that considerably outperform more intuitive ones. We discuss a two-qubit example where we explore the role of entanglement and conclude discussing the scaling of our results to quantum many-body systems.
Maxwell's demon explores the role of information in physical processes.Employing information about microscopic degrees of freedom, this "intelligent observer" is capable of compensating entropy production (or extracting work), apparently challenging the second law of thermodynamics.In a modern standpoint, it is regarded as a feedback control mechanism and the limits of thermodynamics are recast incorporating information-to-energy conversion.We derive a trade-off relation between information-theoretic quantities empowering the design of an efficient Maxwell's demon in a quantum system.The demon is experimentally implemented as a spin-1=2 quantum memory that acquires information, and employs it to control the dynamics of another spin-1=2 system, through a natural interaction.Noise and imperfections in this protocol are investigated by the assessment of its effectiveness.This realization provides experimental evidence that the irreversibility in a nonequilibrium dynamics can be mitigated by assessing microscopic information and applying a feed-forward strategy at the quantum scale.
Maxwell's demon explores the role of information in physical processes. Employing information about microscopic degrees of freedom, this "intelligent observer" is capable of compensating entropy production (or extracting work), apparently challenging the second law of thermodynamics. In a modern standpoint, it is regarded as a feedback control mechanism and the limits of thermodynamics are recast incorporating information-to-energy conversion. We derive a trade-off relation between information-theoretic quantities empowering the design of an efficient Maxwell's demon in a quantum system. The demon is experimentally implemented as a spin-1/2 quantum memory that acquires information, and employs it to control the dynamics of another spin-1/2 system, through a natural interaction. Noise and imperfections in this protocol are investigated by the assessment of its effectiveness. This realization provides experimental evidence that the irreversibility in a nonequilibrium dynamics can be mitigated by assessing microscopic information and applying a feed-forward strategy at the quantum scale.
In this thesis, we consider two simple but typical control problems and apply deep reinforcement learning to them, i.e., to cool and control a particle which is subject to continuous position measurement in a one-dimensional quadratic potential or in a quartic potential. We compare the performance of reinforcement learning control and conventional control strategies on the two problems, and show that the reinforcement learning achieves a performance comparable to the optimal control for the quadratic case, and outperforms conventional control strategies for the quartic case for which the optimal control strategy is unknown. To our knowledge, this is the first time deep reinforcement learning is applied to quantum control problems in continuous real space. Our research demonstrates that deep reinforcement learning can be used to control a stochastic quantum system in real space effectively as a measurement-feedback closed-loop controller, and our research also shows the ability of AI to discover new control strategies and properties of the quantum systems that are not well understood, and we can gain insights into these problems by learning from the AI, which opens up a new regime for scientific research.
We discuss the thermodynamic aspects of a single qubit based device, powered by weak quantum measurements, and feedback controlled by a quantum Maxwell's demon. We discuss both discrete and time-continuous operation of the measurement based device at finite temperature of the reservoir. In the discrete example where a demon acquires information via discrete weak measurements, we find that the thermodynamic variables including the heat exchanged, extractable work, and the entropy produced are completely determined by an information theoretic measure of the demon's perceived arrow of time. We also discuss a realistic time-continuous operation of the device where the feedback is applied after a sequence of weak measurements. In the time-continuous limit, we derive the exact finite-time statistics of work, heat and entropy changes along individual quantum trajectories of the quantum measurement process, and relate them to the demon's arrow of time.
The emerging field of quantum thermodynamics is beginning to reveal the intriguing role that information can play in quantum thermal engines. Information enters as a resource when considering feedback-controlled thermal machines. While both a general theory of quantum feedback control as well as specific examples of quantum feedback-controlled engines have been presented, still lacking is a general framework for such machines. Here, we present a framework for a generic, two-stroke quantum heat engine interacting with NNN thermal baths and Maxwell's demon. The demon performs projective measurements on the engine working substance, the outcome of which is recorded in a classical memory, embedded in its own thermal bath. To perform feedback control, the demon enacts unitary operations on the working substance, conditioned on the recorded outcome. By considering the compound machine-memory as a hybrid (classical-quantum) standard thermal machine interacting with N+1N+1N+1 thermal baths, our framework puts the working substance and memory on equal footing, thereby enabling a comprehensible resolution to Maxwell's paradox and elucidating the intricate manner in which information impacts the performance of quantum engines. We illustrate the application of our framework with a two-qubit engine. A remarkable observation is that more information does not necessarily result in better thermodynamic performance: sometimes knowing less is better.
We investigated coupled-qubit-based thermal machines powered by quantum measurements and feedback. We considered two different versions of the machine: (1) a quantum Maxwell's demon, where the coupled-qubit system is connected to a detachable single shared bath, and (2) a measurement-assisted refrigerator, where the coupled-qubit system is in contact with a hot and cold bath. In the quantum Maxwell's demon case, we discuss both discrete and continuous measurements. We found that the power output from a single qubit-based device can be improved by coupling it to the second qubit. We further found that the simultaneous measurement of both qubits can produce higher net heat extraction compared to two setups operated in parallel where only single-qubit measurements are performed. In the refrigerator case, we used continuous measurement and unitary operations to power the coupled-qubit-based refrigerator. We found that the cooling power of a refrigerator operated with swap operations can be enhanced by performing suitable measurements.