A quantum thermal machine is an open quantum system that enables the conversion between heat and work at the micro or nano-scale. Optimally controlling such out-of-equilibrium systems is a crucial yet challenging task with applications to quantum technologies and devices. We introduce a general model-free framework based on Reinforcement Learning to identify out-of-equilibrium thermodynamic cycles that are Pareto optimal trade-offs between power and efficiency for quantum heat engines and refrigerators. The method does not require any knowledge of the quantum thermal machine, nor of the system model, nor of the quantum state. Instead, it only observes the heat fluxes, so it is both applicable to simulations and experimental devices. We test our method on a model of an experimentally realistic refrigerator based on a superconducting qubit, and on a heat engine based on a quantum harmonic oscillator. In both cases, we identify the Pareto-front representing optimal power-efficiency tradeoffs, and the corresponding cycles. Such solutions outperform previous proposals made in the literature, such as optimized Otto cycles, reducing quantum friction.
A quantum thermal machine is an open quantum system that enables the conversion between heat and work at the micro or nano-scale. Optimally controlling such out-of-equilibrium systems is a crucial yet challenging task with applications to quantum technologies and devices. We introduce a general model-free framework based on Reinforcement Learning to identify out-of-equilibrium thermodynamic cycles that are Pareto optimal trade-offs between power and efficiency for quantum heat engines and refrigerators. The method does not require any knowledge of the quantum thermal machine, nor of the system model, nor of the quantum state. Instead, it only observes the heat fluxes, so it is both applicable to simulations and experimental devices. We test our method on a model of an experimentally realistic refrigerator based on a superconducting qubit, and on a heat engine based on a quantum harmonic oscillator. In both cases, we identify the Pareto-front representing optimal power-efficiency tradeoffs, and the corresponding cycles. Such solutions outperform previous proposals made in the literature, such as optimized Otto cycles, reducing quantum friction.
A quantum thermal machine is an open quantum system that enables the conversion between heat and work at the micro or nano-scale. Optimally controlling such out-of-equilibrium systems is a crucial yet challenging task with applications to quantum technologies and devices. We introduce a general model-free framework based on reinforcement learning to identify out-of-equilibrium thermodynamic cycles that are Pareto optimal tradeoffs between power and efficiency for quantum heat engines and refrigerators. The method does not require any knowledge of the quantum thermal machine, nor of the system model, nor of the quantum state. Instead, it only observes the heat fluxes, so it is both applicable to simulations and experimental devices. We test our method on a model of an experimentally realistic refrigerator based on a superconducting qubit, and on a heat engine based on a quantum harmonic oscillator. In both cases, we identify the Pareto-front representing optimal power-efficiency tradeoffs, and the corresponding cycles. Such solutions outperform previous proposals made in the literature, such as optimized Otto cycles, reducing quantum friction.
Abstract The optimal control of open quantum systems is a challenging task but has a key role in improving existing quantum information processing technologies. We introduce a general framework based on reinforcement learning to discover optimal thermodynamic cycles that maximize the power of out-of-equilibrium quantum heat engines and refrigerators. We apply our method, based on the soft actor-critic algorithm, to three systems: a benchmark two-level system heat engine, where we find the optimal known cycle; an experimentally realistic refrigerator based on a superconducting qubit that generates coherence, where we find a non-intuitive control sequence that outperforms previous cycles proposed in literature; a heat engine based on a quantum harmonic oscillator, where we find a cycle with an elaborate structure that outperforms the optimized Otto cycle. We then evaluate the corresponding efficiency at maximum power.
Quantum thermodynamic relationships in emerging nanodevices are significant but often complex to deal with. The application of machine learning in quantum thermodynamics has provided a new perspective. This study employs reinforcement learning to output the optimal cycle of a quantum heat engine. Specifically, the soft actor-critic algorithm is adopted to optimize the cycle of a three-level coherent quantum heat engine with the aim of maximal average power. The results show that the optimal average output power of the coherent three-level heat engine is 1.28 times greater than the original cycle (steady limit). Meanwhile, the efficiency of the optimal cycle is greater than the Curzon-Ahlborn efficiency as well as efficiencies reported by other researchers. Notably, this optimal cycle can be fitted as an Otto-like cycle, which illustrates the effectiveness of the method.
Abstract This paper presents an analysis of quantum Otto heat engines operating in non-equilibrium. It explores the fundamental thermodynamic principles that govern these engines, focusing on efficiency, work output, and the impact of environmental factors such as thermal gradients, external fields, noise, and decoherence. The study investigates the effects of non-equilibrium conditions on engine performance, highlighting challenges and opportunities in practical realizations. Through numerical simulations, the article examines the power, efficiency, and performance coefficients, revealing trade-offs between these metrics and the influence of temperature differences and internal coupling strength. The results demonstrate that non-equilibrium effects can significantly reduce efficiency compared to ideal scenarios, emphasizing the importance of considering both quantum effects and real-world limitations in the design and optimization of quantum heat engines. This work contributes to the growing body of knowledge in quantum thermodynamics, offering insights for quantum computing, sustainable energy technologies, and thermodynamic cycles at the quantum scale.
This paper presents an analysis of quantum Otto heat engines operating out of equilibrium. It explores the fundamental thermodynamic principles governing these engines, focusing on efficiency, work output, and the impact of environmental factors, including thermal gradients, external fields, noise, and decoherence. The study investigates the effects of non-equilibrium conditions on engine performance, highlighting challenges and opportunities in practical realizations. Through numerical simulations, the article examines the power, efficiency, and performance coefficients, revealing trade-offs between these metrics and the influence of temperature differences and internal coupling strength. The results demonstrate that non-equilibrium effects can significantly reduce efficiency compared to idealized scenarios, underscoring the importance of accounting for both quantum effects and real-world constraints in the design and optimization of quantum heat engines. This work contributes to the growing body of knowledge in quantum thermodynamics, offering insights for quantum computing, sustainable energy technologies, and thermodynamic cycles at the quantum scale.
Quantum heat machines, encompassing heat engines, refrigerators, heaters, and accelerators, represent the forefront of quantum thermodynamics, offering a novel paradigm for converting heat energy into useful mechanical work. Leveraging quantum mechanical principles, these machines promise superior efficiency and performance compared to classical counterparts, with potential applications in renewable energy and quantum computing. This paper investigates a quantum Otto engine operating in both ideal and finite-time scenarios, employing a two-level system interacting with a harmonic oscillator within the framework of the anisotropic quantum Rabi-Stark model (AQRSM) as the working medium. This model is notable for exhibiting both first-order and continuous quantum phase transitions. By focusing on quantum heat engines, our study reveals that these phase transitions critically modulate the efficiency and power of AQRSM-based engines, outperforming quantum engines fueled by working medium with harmonic spectrum. Additionally, we explore the impacts of quantum friction and conduct limit cycle analysis in finite-time operations, providing insights into optimizing quantum heat engines for practical implementation.