We use a reinforcement learning approach to reduce entropy production in a closed quantum system brought out of equilibrium. Our strategy makes use of an external control Hamiltonian and a policy gradient technique. Our approach bears no dependence on the quantitative tool chosen to characterize the degree of thermodynamic irreversibility induced by the dynamical process being considered, requires little knowledge of the dynamics itself, and does not need the tracking of the quantum state of the system during the evolution, thus embodying an experimentally nondemanding approach to the control of nonequilibrium quantum thermodynamics. We successfully apply our methods to the case of single- and two-particle systems subjected to time-dependent driving potentials.
We use a reinforcement learning approach to reduce entropy production in a closed quantum system brought out of equilibrium. Our strategy makes use of an external control Hamiltonian and a policy gradient technique. Our approach bears no dependence on the quantitative tool chosen to characterize the degree of thermodynamic irreversibility induced by the dynamical process being considered, requires little knowledge of the dynamics itself, and does not need the tracking of the quantum state of the system during the evolution, thus embodying an experimentally nondemanding approach to the control of nonequilibrium quantum thermodynamics. We successfully apply our methods to the case of single- and two-particle systems subjected to time-dependent driving potentials.
Abstract Exploring optimized processes of thermodynamics at microscale is vital to exploitation of quantum advantages relevant to microscopic machines and quantum information processing. Here, we experimentally execute a reinforcement learning strategy, using a single trapped 40Ca+ ion, for engineering quantum state evolution out of thermal equilibrium. We consider a qubit system coupled to classical and quantum baths, respectively, the former of which is achieved by switching on the spontaneous emission relevant to the qubit and the latter of which is made based on a Jaynes-Cummings model involving the qubit and the vibrational degree of freedom of the ion. Our optimized operations make use of the external control on the qubit, designed by the reinforcement learning approach. In comparison to the conventional situation of free evolution subject to the same Hamiltonian of interest, our experimental implementation presents the evolution of the states with higher fidelity while with less consumption of entropy production and work, highlighting the potential of reinforcement learning in accomplishment of optimized nonequilibrium thermodynamic processes at atomic level.
In this thesis, we consider two simple but typical control problems and apply deep reinforcement learning to them, i.e., to cool and control a particle which is subject to continuous position measurement in a one-dimensional quadratic potential or in a quartic potential. We compare the performance of reinforcement learning control and conventional control strategies on the two problems, and show that the reinforcement learning achieves a performance comparable to the optimal control for the quadratic case, and outperforms conventional control strategies for the quartic case for which the optimal control strategy is unknown. To our knowledge, this is the first time deep reinforcement learning is applied to quantum control problems in continuous real space. Our research demonstrates that deep reinforcement learning can be used to control a stochastic quantum system in real space effectively as a measurement-feedback closed-loop controller, and our research also shows the ability of AI to discover new control strategies and properties of the quantum systems that are not well understood, and we can gain insights into these problems by learning from the AI, which opens up a new regime for scientific research.
Reinforcement learning (RL) is considered a powerful technology with the potential to revolutionize quantum control. However, the application effectiveness of traditional RL is often limited by some insurmountable experimental conditions. Thus, developing new RL algorithms that can efficiently manipulate the quantum system dynamics is a crucial task. Prior research has shown that incorporating quantum mechanical properties into RL can improve learning performance. In this article, we consider the quantum control problem where only the target state can be accurately identified and introduce a quantum-inspired RL (QiRL) method. In particular, we propose a quantum-inspired exploration strategy to replace a commonly used$\epsilon $-greedy strategy, as well as a quantum-inspired reward scheme to incentivize the learning agent. Numerical results on three quantum system control problems, i.e., one-qubit closed quantum system, two-level open quantum system, and many-qubit closed quantum system, verify the effectiveness of QiRL. Comparison results show that the proposed QiRL outperforms existing RL algorithms (deep Q-network and proximal policy optimization) in terms of stability and efficiency for solving quantum control problems.
The progress in machine learning during the last decade has had a considerable impact on many areas of science and technology, including quantum technology. This work explores the application of reinforcement learning (RL) methods to the quantum control problem of state transfer in a single qubit. The goal is to create an RL agent that learns an optimal policy and thus discovers optimal pulses to control the qubit. The most crucial step is to mathematically formulate the problem of interest as a Markov decision process (MDP). This enables the use of RL algorithms to solve the quantum control problem. Deep learning and the use of deep neural networks provide the freedom to employ continuous action and state spaces, offering the expressivity and generalization of the process. This flexibility helps to formulate the quantum state transfer problem as an MDP in several different ways. All the developed methodologies are applied to the fundamental problem of population inversion in a qubit. In most cases, the derived optimal pulses achieve fidelity equal to or higher than 0.9999, as required by quantum computing applications. The present methods can be easily extended to quantum systems with more energy levels and may be used for the efficient control of collections of qubits and to counteract the effect of noise, which are important topics for quantum sensing applications.
Abstract Machine learning technology based on artificial neural network has been successfully applied to solve many scientific problems. One of the most interesting areas of machine learning is reinforcement learning, which has natural applicability to optimization problems in physics. In the quantum control task, it is necessary to find a set of optimal control functions to transfer a quantum system from the initial state to the target state with the highest fidelity possible, which is essentially an optimization task. In this paper, we use Deep Deterministic Policy Gradient algorithm (DDPG) to study the quantum control tasks. We use the algorithm to control the transfer of several quantum systems from one state to another. The results show that DDPG algorithm can find a control strategy to make the fidelity of the final state and the target state of the quantum system be maximum value 1. The results show the potential of DDPG in quantum control.
A central open question in quantum thermodynamics is the identification of thermodynamic entropy away from equilibrium. Although thermodynamic entropy coincides with the von Neumann entropy for a Gibbs state at equilibrium, the thermodynamic role of quantum coherence generated during nonequilibrium evolution remains unclear. Here, we show that von Neumann entropy does not, in general, represent the thermodynamic entropy governing heat exchange in coherent nonequilibrium quantum systems. By integrating the resource theory of quantum coherence with thermodynamic principles, we develop a general framework for nonequilibrium quantum thermodynamics. We identify thermodynamic entropy with the energy entropy derived from energy-measurement statistics and introduce a dynamical temperature extending the equilibrium concept to nonequilibrium states. Consequently, we obtain an exact entropy-balance relation separating the entropy flux associated with heat exchange from the coherence-loss contribution to entropy production, establishing an operational connection between quantum coherence, heat flow, and total entropy change. Unlike approaches relying on equilibrium reference states or stationary Gibbs ensembles, our framework uses instantaneous dynamical quantities and remains valid throughout transient nonequilibrium evolution. It provides consistent dynamical definitions of temperature, work, heat, and free energy. Applying the formalism to an exactly solvable open quantum system, we show that the first and second laws hold throughout transient evolution and that equilibrium thermodynamics emerges dynamically in the weak-coupling long-time limit. Our results establish a unified framework for coherent nonequilibrium quantum thermodynamics and suggest opportunities for controlling heat dissipation through coherence engineering in quantum thermal technologies.