Abstract The optimal control of open quantum systems is a challenging task but has a key role in improving existing quantum information processing technologies. We introduce a general framework based on reinforcement learning to discover optimal thermodynamic cycles that maximize the power of out-of-equilibrium quantum heat engines and refrigerators. We apply our method, based on the soft actor-critic algorithm, to three systems: a benchmark two-level system heat engine, where we find the optimal known cycle; an experimentally realistic refrigerator based on a superconducting qubit that generates coherence, where we find a non-intuitive control sequence that outperforms previous cycles proposed in literature; a heat engine based on a quantum harmonic oscillator, where we find a cycle with an elaborate structure that outperforms the optimized Otto cycle. We then evaluate the corresponding efficiency at maximum power.
Abstract The optimal control of open quantum systems is a challenging task but has a key role in improving existing quantum information processing technologies. We introduce a general framework based on reinforcement learning to discover optimal thermodynamic cycles that maximize the power of out-of-equilibrium quantum heat engines and refrigerators. We apply our method, based on the soft actor-critic algorithm, to three systems: a benchmark two-level system heat engine, where we find the optimal known cycle; an experimentally realistic refrigerator based on a superconducting qubit that generates coherence, where we find a non-intuitive control sequence that outperforms previous cycles proposed in literature; a heat engine based on a quantum harmonic oscillator, where we find a cycle with an elaborate structure that outperforms the optimized Otto cycle. We then evaluate the corresponding efficiency at maximum power.
A quantum thermal machine is an open quantum system that enables the conversion between heat and work at the micro or nano-scale. Optimally controlling such out-of-equilibrium systems is a crucial yet challenging task with applications to quantum technologies and devices. We introduce a general model-free framework based on reinforcement learning to identify out-of-equilibrium thermodynamic cycles that are Pareto optimal tradeoffs between power and efficiency for quantum heat engines and refrigerators. The method does not require any knowledge of the quantum thermal machine, nor of the system model, nor of the quantum state. Instead, it only observes the heat fluxes, so it is both applicable to simulations and experimental devices. We test our method on a model of an experimentally realistic refrigerator based on a superconducting qubit, and on a heat engine based on a quantum harmonic oscillator. In both cases, we identify the Pareto-front representing optimal power-efficiency tradeoffs, and the corresponding cycles. Such solutions outperform previous proposals made in the literature, such as optimized Otto cycles, reducing quantum friction.
A quantum thermal machine is an open quantum system that enables the conversion between heat and work at the micro or nano-scale. Optimally controlling such out-of-equilibrium systems is a crucial yet challenging task with applications to quantum technologies and devices. We introduce a general model-free framework based on Reinforcement Learning to identify out-of-equilibrium thermodynamic cycles that are Pareto optimal trade-offs between power and efficiency for quantum heat engines and refrigerators. The method does not require any knowledge of the quantum thermal machine, nor of the system model, nor of the quantum state. Instead, it only observes the heat fluxes, so it is both applicable to simulations and experimental devices. We test our method on a model of an experimentally realistic refrigerator based on a superconducting qubit, and on a heat engine based on a quantum harmonic oscillator. In both cases, we identify the Pareto-front representing optimal power-efficiency tradeoffs, and the corresponding cycles. Such solutions outperform previous proposals made in the literature, such as optimized Otto cycles, reducing quantum friction.
Quantum thermodynamic relationships in emerging nanodevices are significant but often complex to deal with. The application of machine learning in quantum thermodynamics has provided a new perspective. This study employs reinforcement learning to output the optimal cycle of a quantum heat engine. Specifically, the soft actor-critic algorithm is adopted to optimize the cycle of a three-level coherent quantum heat engine with the aim of maximal average power. The results show that the optimal average output power of the coherent three-level heat engine is 1.28 times greater than the original cycle (steady limit). Meanwhile, the efficiency of the optimal cycle is greater than the Curzon-Ahlborn efficiency as well as efficiencies reported by other researchers. Notably, this optimal cycle can be fitted as an Otto-like cycle, which illustrates the effectiveness of the method.
We study the optimization of the performance of arbitrary periodically driven thermal machines. Within the assumption of fast modulation of the driving parameters, we derive the optimal cycle that universally maximizes the extracted power of heat engines, the cooling power of refrigerators, and in general any linear combination of the heat currents. We denote this optimal solution as ``generalized Otto cycle'' since it shares the basic structure with the standard Otto cycle, but it is characterized by a greater number of fast strokes. We bound this number in terms of the dimension of the Hilbert space of the system used as working fluid. The generality of these results allows for a widespread range of applications, such as reducing the computational complexity for numerical approaches, or obtaining the explicit form of the optimal protocols when the system-baths interactions are characterized by a single thermalization scale. In this case, we compare the thermodynamic performance of a collection of optimally driven noninteracting and interacting qubits. Remarkably, for refrigerators the noninteracting qubits perform almost as well as the interacting ones, while in the heat engine case there is a many-body advantage both in the maximum power, and in the efficiency at maximum power. Additionally, we illustrate our general results studying the paradigmatic model of a qutrit-based heat engine. Our results strictly hold in the semiclassical case in which no coherence is generated by the driving, and finally we discuss the noncommuting case.
We apply advanced methods of control theory to open quantum systems and we determine finite-time processes which are optimal with respect to thermodynamic performances. General properties and necessary conditions characterizing optimal drivings are derived, obtaining bang-bang-type solutions corresponding to control strategies switching between adiabatic and isothermal transformations. A direct application of these results is the maximization of the work produced by a generic quantum heat engine, where we show that the maximum power is directly linked to a particular conserved quantity naturally emerging from the control problem. Finally we apply our general approach to the specific case of a two-level system, which can be put in contact with two different baths at fixed temperatures, identifying the processes that minimize heat dissipation. Moreover, we explicitly solve the optimization problem for a cyclic two-level heat engine driven beyond the linear-response regime, determining the corresponding optimal cycle, the maximum power, and the efficiency at maximum power.
Abstract We study how to achieve the ultimate power in the simplest, yet non-trivial, model of a thermal machine, namely a two-level quantum system coupled to two thermal baths. Without making any prior assumption on the protocol, via optimal control we show that, regardless of the microscopic details and of the operating mode of the thermal machine, the maximum power is universally achieved by a fast Otto-cycle like structure in which the controls are rapidly switched between two extremal values. A closed formula for the maximum power is derived, and finite-speed effects are discussed. We also analyze the associated efficiency at maximum power showing that, contrary to universal results derived in the slow-driving regime, it can approach Carnot’s efficiency, no other universal bounds being allowed.