Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:1904.04712 · Physical Review A

Deep reinforcement learning for quantum Szilard engine optimization

Vegard B. Sørdal, Joakim Bergli

Published Oct 16, 2019• 18 citations
arXivHTMLPDF

Abstract

Machine learning techniques based on artificial neural networks have been successfully applied to solve many problems in science. One of the most interesting domains of machine learning, reinforcement learning, has a natural applicability for optimization problems in physics. In this paper, we use deep reinforcement learning and chopped random basis optimization to solve an optimization problem based on the insertion of an off-center barrier in a quantum Szilard engine. We show that by using designed protocols for the time dependence of the barrier strength, we can achieve an equal splitting of the wave function (1/2 probability to find the particle on either side of the barrier) even for an asymmetric Szilard engine in such a way that no information is lost when measuring which side the particle is found. This implies that the asymmetric nonadiabatic Szilard engine can operate with the same efficiency as the traditional Szilard engine with adiabatic insertion of a central barrier. We compare the two optimization methods and demonstrate the advantage of reinforcement learning when it comes to constructing robust and noise-resistant protocols.

DOI
10.1103/physreva.100.042314

Related papers

Control of Qubit Dynamics Using Reinforcement Learning

Dimitris Koutromanos, Dionisis Stefanatos, Emmanuel Paspalakis

The progress in machine learning during the last decade has had a considerable impact on many areas of science and technology, including quantum technology. This work explores the application of reinforcement learning (RL) methods to the quantum control problem of state transfer in a single qubit. The goal is to create an RL agent that learns an optimal policy and thus discovers optimal pulses to control the qubit. The most crucial step is to mathematically formulate the problem of interest as a Markov decision process (MDP). This enables the use of RL algorithms to solve the quantum control problem. Deep learning and the use of deep neural networks provide the freedom to employ continuous action and state spaces, offering the expressivity and generalization of the process. This flexibility helps to formulate the quantum state transfer problem as an MDP in several different ways. All the developed methodologies are applied to the fundamental problem of population inversion in a qubit. In most cases, the derived optimal pulses achieve fidelity equal to or higher than 0.9999, as required by quantum computing applications. The present methods can be easily extended to quantum systems with more energy levels and may be used for the efficient control of collections of qubits and to counteract the effect of noise, which are important topics for quantum sensing applications.

Information•May 11, 2024•13 citations

A framework for deep energy-based reinforcement learning with quantum speed-up

Sofiène Jerbi, Hendrik Poulsen Nautrup, Lea M. Trenkwalder +2 more

In the past decade, deep learning methods have seen tremendous success in various supervised and unsupervised learning tasks such as classification and generative modeling. More recently, deep neural networks have emerged in the domain of reinforcement learning as a tool to solve decision-making problems of unprecedented complexity, e.g., navigation problems or game-playing AI. Despite the successful combinations of ideas from quantum computing with machine learning methods, there have been relatively few attempts to design quantum algorithms that would enhance deep reinforcement learning. This is partly due to the fact that quantum enhancements of deep neural networks, in general, have not been as extensively investigated as other quantum machine learning methods. In contrast, projective simulation is a reinforcement learning model inspired by the stochastic evolution of physical systems that enables a quantum speed-up in decision making. In this paper, we develop a unifying framework that connects deep learning and projective simulation, opening the route to quantum improvements in deep reinforcement learning. Our approach is based on so-called generative energy-based models to design reinforcement learning methods with a computational advantage in solving complex and large-scale decision-making problems.

•Oct 28, 2019•4 citations

Deep reinforcement learning for quantum gate control

Zheng An, D. L. Zhou

How to implement multi-qubit gates efficiently with high precision is essential for realizing universal fault-tolerant computing. For a physical system with some external controllable parameters, it is a great challenge to control the time dependence of these parameters to achieve a target multi-qubit gate efficiently and precisely. Here we construct a dueling double deep Q-learning neural network (DDDQN) to find out the optimized time dependence of controllable parameters to implement two typical quantum gates: a single-qubit Hadamard gate and a two-qubit CNOT gate. Compared with traditional optimal control methods, this deep reinforcement learning method can realize efficient and precise gate control without requiring any gradient information during the learning process. This work attempts to pave the way to investigate more quantum control problems with deep reinforcement learning techniques.

arXiv:1902.08418•Europhysics Letters (EPL)•Jul 22, 2019•132 citations

On the Use of Quantum Reinforcement Learning in Energy-Efficiency Scenarios

Eva Andrés, Manuel Pegalájar Cuéllar, Gabriel Navarro

In the last few years, deep reinforcement learning has been proposed as a method to perform online learning in energy-efficiency scenarios such as HVAC control, electric car energy management, or building energy management, just to mention a few. On the other hand, quantum machine learning was born during the last decade to extend classic machine learning to a quantum level. In this work, we propose to study the benefits and limitations of quantum reinforcement learning to solve energy-efficiency scenarios. As a testbed, we use existing energy-efficiency-based reinforcement learning simulators and compare classic algorithms with the quantum proposal. Results in HVAC control, electric vehicle fuel consumption, and profit optimization of electrical charging stations applications suggest that quantum neural networks are able to solve problems in reinforcement learning scenarios with better accuracy than their classical counterpart, obtaining a better cumulative reward with fewer parameters to be learned.

Energies•Aug 19, 2022•28 citations

Reinforcement Learning with Deep Quantum Neural Networks

Wei Hu, James Lee Hu

The advantage of quantum computers over classical computers fuels the recent trend of developing machine learning algorithms on quantum computers, which can potentially lead to breakthroughs and new learning models in this area. The aim of our study is to explore deep quantum reinforcement learning (RL) on photonic quantum computers, which can process information stored in the quantum states of light. These quantum computers can naturally represent continuous variables, making them an ideal platform to create quantum versions of neural networks. Using quantum photonic circuits, we implement Q learning and actor-critic algorithms with multilayer quantum neural networks and test them in the grid world environment. Our experiments show that 1) these quantum algorithms can solve the RL problem and 2) compared to one layer, using three layer quantum networks improves the learning of both algorithms in terms of rewards collected. In summary, our findings suggest that having more layers in deep quantum RL can enhance the learning outcome.

Journal of Quantum Information Science•Jan 1, 2019•13 citations

Quantum reinforcement learning: the maze problem

Nicola Dalla Pozza, Lorenzo Buffoni, Stefano Martina +1 more

Abstract Quantum machine learning (QML) is a young but rapidly growing field where quantum information meets machine learning. Here, we will introduce a new QML model generalising the classical concept of reinforcement learning to the quantum domain, i.e. quantum reinforcement learning (QRL). In particular, we apply this idea to the maze problem, where an agent has to learn the optimal set of actions in order to escape from a maze with the highest success probability. To perform the strategy optimisation, we consider a hybrid protocol where QRL is combined with classical deep neural networks. In particular, we find that the agent learns the optimal strategy in both the classical and quantum regimes, and we also investigate its behaviour in a noisy environment. It turns out that the quantum speedup does robustly allow the agent to exploit useful actions also at very short time scales, with key roles played by the quantum coherence and the external noise. This new framework has the high potential to be applied to perform different tasks (e.g. high transmission/processing rates and quantum error correction) in the new-generation noisy intermediate-scale quantum (NISQ) devices whose topology engineering is starting to become a new and crucial control knob for practical applications in real-world problems. This work is dedicated to the memory of Peter Wittek.

arXiv:2108.04490•Quantum Machine Intelligence•May 27, 2022•42 citations
Machine learning techniques based on artificial neural networks have been successfully applied to solve many problems in science. One of the most interesting domains of machine learning, reinforcement learning, has a natural applicability for optimization problems in physics. In this paper, we use deep reinforcement learning and chopped random basis optimization to solve an optimization problem based on the insertion of an off-center barrier in a quantum Szilard engine. We show that by using designed protocols for the time dependence of the barrier strength, we can achieve an equal splitting of the wave function (1/2 probability to find the particle on either side of the barrier) even for an asymmetric Szilard engine in such a way that no information is lost when measuring which side the particle is found. This implies that the asymmetric nonadiabatic Szilard engine can operate with the same efficiency as the traditional Szilard engine with adiabatic insertion of a central barrier. We compare the two optimization methods and demonstrate the advantage of reinforcement learning when it comes to constructing robust and noise-resistant protocols.
The progress in machine learning during the last decade has had a considerable impact on many areas of science and technology, including quantum technology. This work explores the application of reinforcement learning (RL) methods to the quantum control problem of state transfer in a single qubit. The goal is to create an RL agent that learns an optimal policy and thus discovers optimal pulses to control the qubit. The most crucial step is to mathematically formulate the problem of interest as a Markov decision process (MDP). This enables the use of RL algorithms to solve the quantum control problem. Deep learning and the use of deep neural networks provide the freedom to employ continuous action and state spaces, offering the expressivity and generalization of the process. This flexibility helps to formulate the quantum state transfer problem as an MDP in several different ways. All the developed methodologies are applied to the fundamental problem of population inversion in a qubit. In most cases, the derived optimal pulses achieve fidelity equal to or higher than 0.9999, as required by quantum computing applications. The present methods can be easily extended to quantum systems with more energy levels and may be used for the efficient control of collections of qubits and to counteract the effect of noise, which are important topics for quantum sensing applications.
In the past decade, deep learning methods have seen tremendous success in various supervised and unsupervised learning tasks such as classification and generative modeling. More recently, deep neural networks have emerged in the domain of reinforcement learning as a tool to solve decision-making problems of unprecedented complexity, e.g., navigation problems or game-playing AI. Despite the successful combinations of ideas from quantum computing with machine learning methods, there have been relatively few attempts to design quantum algorithms that would enhance deep reinforcement learning. This is partly due to the fact that quantum enhancements of deep neural networks, in general, have not been as extensively investigated as other quantum machine learning methods. In contrast, projective simulation is a reinforcement learning model inspired by the stochastic evolution of physical systems that enables a quantum speed-up in decision making. In this paper, we develop a unifying framework that connects deep learning and projective simulation, opening the route to quantum improvements in deep reinforcement learning. Our approach is based on so-called generative energy-based models to design reinforcement learning methods with a computational advantage in solving complex and large-scale decision-making problems.
How to implement multi-qubit gates efficiently with high precision is essential for realizing universal fault-tolerant computing. For a physical system with some external controllable parameters, it is a great challenge to control the time dependence of these parameters to achieve a target multi-qubit gate efficiently and precisely. Here we construct a dueling double deep Q-learning neural network (DDDQN) to find out the optimized time dependence of controllable parameters to implement two typical quantum gates: a single-qubit Hadamard gate and a two-qubit CNOT gate. Compared with traditional optimal control methods, this deep reinforcement learning method can realize efficient and precise gate control without requiring any gradient information during the learning process. This work attempts to pave the way to investigate more quantum control problems with deep reinforcement learning techniques.
In the last few years, deep reinforcement learning has been proposed as a method to perform online learning in energy-efficiency scenarios such as HVAC control, electric car energy management, or building energy management, just to mention a few. On the other hand, quantum machine learning was born during the last decade to extend classic machine learning to a quantum level. In this work, we propose to study the benefits and limitations of quantum reinforcement learning to solve energy-efficiency scenarios. As a testbed, we use existing energy-efficiency-based reinforcement learning simulators and compare classic algorithms with the quantum proposal. Results in HVAC control, electric vehicle fuel consumption, and profit optimization of electrical charging stations applications suggest that quantum neural networks are able to solve problems in reinforcement learning scenarios with better accuracy than their classical counterpart, obtaining a better cumulative reward with fewer parameters to be learned.
The advantage of quantum computers over classical computers fuels the recent trend of developing machine learning algorithms on quantum computers, which can potentially lead to breakthroughs and new learning models in this area. The aim of our study is to explore deep quantum reinforcement learning (RL) on photonic quantum computers, which can process information stored in the quantum states of light. These quantum computers can naturally represent continuous variables, making them an ideal platform to create quantum versions of neural networks. Using quantum photonic circuits, we implement Q learning and actor-critic algorithms with multilayer quantum neural networks and test them in the grid world environment. Our experiments show that 1) these quantum algorithms can solve the RL problem and 2) compared to one layer, using three layer quantum networks improves the learning of both algorithms in terms of rewards collected. In summary, our findings suggest that having more layers in deep quantum RL can enhance the learning outcome.
Abstract Quantum machine learning (QML) is a young but rapidly growing field where quantum information meets machine learning. Here, we will introduce a new QML model generalising the classical concept of reinforcement learning to the quantum domain, i.e. quantum reinforcement learning (QRL). In particular, we apply this idea to the maze problem, where an agent has to learn the optimal set of actions in order to escape from a maze with the highest success probability. To perform the strategy optimisation, we consider a hybrid protocol where QRL is combined with classical deep neural networks. In particular, we find that the agent learns the optimal strategy in both the classical and quantum regimes, and we also investigate its behaviour in a noisy environment. It turns out that the quantum speedup does robustly allow the agent to exploit useful actions also at very short time scales, with key roles played by the quantum coherence and the external noise. This new framework has the high potential to be applied to perform different tasks (e.g. high transmission/processing rates and quantum error correction) in the new-generation noisy intermediate-scale quantum (NISQ) devices whose topology engineering is starting to become a new and crucial control knob for practical applications in real-world problems. This work is dedicated to the memory of Peter Wittek.