Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Quantum Information and Computation

Reinforcement learning using quantum Boltzmann machines

Daniel Crawford, Anna Levit, Navid Ghadermarzy, Jaspreet S. Oberoi, Pooya Ronagh

Published Feb 1, 2018• 72 citations
Publisher

Abstract

We investigate whether quantum annealers with select chip layouts can outperform classical computers in reinforcement learning tasks. We associate a transverse field Ising spin Hamiltonian with a layout of qubits similar to that of a deep Boltzmann machine (DBM) and use simulated quantum annealing (SQA) to numerically simulate quantum sampling from this system. We design a reinforcement learning algorithm in which the set of visible nodes representing the states and actions of an optimal policy are the first and last layers of the deep network. In absence of a transverse field, our simulations show that DBMs are trained more effectively than restricted Boltzmann machines (RBM) with the same number of nodes. We then develop a framework for training the network as a quantum Boltzmann machine (QBM) in the presence of a significant transverse field for reinforcement learning. This method also outperforms the reinforcement learning method that uses RBMs.

DOI
10.26421/qic18.1-2-3

Related papers

Reinforcement Learning Using Quantum Boltzmann Machines

Crawford, Daniel, Anna Levit, Navid Ghadermarzy +2 more

We investigate whether quantum annealers with select chip layouts can outperform classical computers in reinforcement learning tasks. We associate a transverse field Ising spin Hamiltonian with a layout of qubits similar to that of a deep Boltzmann machine (DBM) and use simulated quantum annealing (SQA) to numerically simulate quantum sampling from this system. We design a reinforcement learning algorithm in which the set of visible nodes representing the states and actions of an optimal policy are the first and last layers of the deep network. In absence of a transverse field, our simulations show that DBMs are trained more effectively than restricted Boltzmann machines (RBM) with the same number of nodes. We then develop a framework for training the network as a quantum Boltzmann machine (QBM) in the presence of a significant transverse field for reinforcement learning. This method also outperforms the reinforcement learning method that uses RBMs.

arXiv:1612.05695•Dec 17, 2016•14 citations

Quantum Reinforcement Learning

Daniel Crawford, Anna Levit, Navid Ghadermarzy +2 more

We investigate whether quantum annealers with select chip layouts can outperform classical computers in reinforcement learning tasks. We associate a transverse field Ising spin Hamiltonian with a layout of qubits similar to that of a deep Boltzmann machine (DBM) and use simulated quantum annealing (SQA) to numerically simulate quantum sampling from this system. We design a reinforcement learning algorithm in which the set of visible nodes representing the states and actions of an optimal policy are the first and last layers of the deep network. In absence of a transverse field, our simulations show that DBMs are trained more effectively than restricted Boltzmann machines (RBM) with the same number of nodes. We then develop a framework for training the network as a quantum Boltzmann machine (QBM) in the presence of a significant transverse field for reinforcement learning. This method also outperforms the reinforcement learning method that uses RBMs.

•Dec 17, 2016•0 citations

Towards Multi-agent Reinforcement Learning using Quantum Boltzmann Machines

Tobias Müller, Christoph Roch, Kyrill Schmid +1 more

Reinforcement learning has driven impressive advances in machine learning. Simultaneously, quantum-enhanced machine learning algorithms using quantum annealing underlie heavy developments. Recently, a multi-agent reinforcement learning (MARL) architecture combining both paradigms has been proposed. This novel algorithm, which utilizes Quantum Boltzmann Machines (QBMs) for Q-value approximation has outperformed regular deep reinforcement learning in terms of time-steps needed to converge. However, this algorithm was restricted to single-agent and small 2x2 multi-agent grid domains. In this work, we propose an extension to the original concept in order to solve more challenging problems. Similar to classic DQNs, we add an experience replay buffer and use different networks for approximating the target and policy values. The experimental results show that learning becomes more stable and enables agents to find optimal policies in grid-domains with higher complexity. Additionally, we assess how parameter sharing influences the agents behavior in multi-agent domains. Quantum sampling proves to be a promising method for reinforcement learning tasks, but is currently limited by the QPU size and therefore by the size of the input and Boltzmann machine.

arXiv:2109.10900•International Conference on Agents and Artificial Intelligence (ICAART)•Jan 1, 2022•4 citations

Machine learning in quantum computers via general Boltzmann Machines: Generative and Discriminative training through annealing.

Siddhartha Srivastava, Veera Sundararaghavan

We present a Hybrid-Quantum-classical method for learning Boltzmann machines (BM) for generative and discriminative tasks. Boltzmann machines are undirected graphs that form the building block of many learning architectures such as Restricted Boltzmann machines (RBM's) and Deep Boltzmann machines (DBM's). They have a network of visible and hidden nodes where the former are used as the reading sites while the latter are used to manipulate the probability of the visible states. BM's are versatile machines that can be used for both learning distributions as a generative task as well as for performing classification or function approximation as a discriminative task. We show that minimizing KL-divergence works best for training BM for applications of function approximation. In our approach, we use Quantum annealers for sampling Boltzmann states. These states are used to approximate gradients in a stochastic gradient descent scheme. The approach is used to demonstrate logic circuits in the discriminative sense and a specialized two-phase distribution using generative BM.

•Feb 3, 2020•4 citations

Reinforcement Learning via Replica Stacking of Quantum Measurements for the Training of Quantum Boltzmann Machines

Anna Levit, Daniel Crawford, Navid Ghadermarzy +3 more

Recent theoretical and experimental results suggest the possibility of using current and near-future quantum hardware in challenging sampling tasks. In this paper, we introduce free-energy-based reinforcement learning (FERL) as an application of quantum hardware. We propose a method for processing a quantum annealer’s measured qubit spin configurations in approximating the free energy of a quantum Boltzmann machine (QBM). We then apply this method to perform reinforcement learning on the grid-world problem using the D-Wave 2000Q quantum annealer. The experimental results show that our technique is a promising method for harnessing the power of quantum sampling in reinforcement learning tasks.

•Feb 15, 2018•0 citations

Quantum boltzmann machine :emergence & applications

Miguel Francisco Aldana Páez

Lately, there has been an extensive state-of-the-art research in Machine Learning methods, due to their important features such as universality in approximations and dimensional reduction. In this way, the present work aims at exploiting these properties of Machine Learning on physical many-body problems, in which a restricted Boltzmann Machine (RBM) is trained with quantum Monte Carlo data to best represent the ground state of an N spin system under the transverse field Ising model Hamiltonian in one dimension. During the path coursed along the development of this project, we first review classical well-implemented methods such as the single-layer perceptron, multi-layer perceptron for the XOR problem with back-propagation algorithm, discrete Hopfield network for pattern reconstruction, continuous Hopfield network with Simulated Annealing for solving the traveling salesman problem with 10 cities and the Boltzmann machine for feature extraction. Then, we review a few aspects of what is called a Quantum Boltzmann Machine and the basics of the transverse field Ising model. Finally, we train an RBM consisting of N visible neurons, hidden density neurons alpha=M/N=2 and 1-step contrastive divergence (CD_1), by minimizing the Kullbach-Liebler divergence of the RBM and the training data set coming from the algorithmic variant of quantum Monte Carlo, known as discrete Path-Integral Monte Carlo. For this last step, we use Trotter-Suzuki decomposition with m Trotter slices and the Wolff-s cluster algorithm. After training the RBM, we calculate magnetic observables based on the RBM-s wave-function such as longitudinal and transverse magnetization, for different values of transverse field and we do an estimation of the critical field at which the transverse Ising model system suffers a quantum phase transition from ferromagnet to paramagnet

•Jan 1, 2018•0 citations
We investigate whether quantum annealers with select chip layouts can outperform classical computers in reinforcement learning tasks. We associate a transverse field Ising spin Hamiltonian with a layout of qubits similar to that of a deep Boltzmann machine (DBM) and use simulated quantum annealing (SQA) to numerically simulate quantum sampling from this system. We design a reinforcement learning algorithm in which the set of visible nodes representing the states and actions of an optimal policy are the first and last layers of the deep network. In absence of a transverse field, our simulations show that DBMs are trained more effectively than restricted Boltzmann machines (RBM) with the same number of nodes. We then develop a framework for training the network as a quantum Boltzmann machine (QBM) in the presence of a significant transverse field for reinforcement learning. This method also outperforms the reinforcement learning method that uses RBMs.
We investigate whether quantum annealers with select chip layouts can outperform classical computers in reinforcement learning tasks. We associate a transverse field Ising spin Hamiltonian with a layout of qubits similar to that of a deep Boltzmann machine (DBM) and use simulated quantum annealing (SQA) to numerically simulate quantum sampling from this system. We design a reinforcement learning algorithm in which the set of visible nodes representing the states and actions of an optimal policy are the first and last layers of the deep network. In absence of a transverse field, our simulations show that DBMs are trained more effectively than restricted Boltzmann machines (RBM) with the same number of nodes. We then develop a framework for training the network as a quantum Boltzmann machine (QBM) in the presence of a significant transverse field for reinforcement learning. This method also outperforms the reinforcement learning method that uses RBMs.
We investigate whether quantum annealers with select chip layouts can outperform classical computers in reinforcement learning tasks. We associate a transverse field Ising spin Hamiltonian with a layout of qubits similar to that of a deep Boltzmann machine (DBM) and use simulated quantum annealing (SQA) to numerically simulate quantum sampling from this system. We design a reinforcement learning algorithm in which the set of visible nodes representing the states and actions of an optimal policy are the first and last layers of the deep network. In absence of a transverse field, our simulations show that DBMs are trained more effectively than restricted Boltzmann machines (RBM) with the same number of nodes. We then develop a framework for training the network as a quantum Boltzmann machine (QBM) in the presence of a significant transverse field for reinforcement learning. This method also outperforms the reinforcement learning method that uses RBMs.
Reinforcement learning has driven impressive advances in machine learning. Simultaneously, quantum-enhanced machine learning algorithms using quantum annealing underlie heavy developments. Recently, a multi-agent reinforcement learning (MARL) architecture combining both paradigms has been proposed. This novel algorithm, which utilizes Quantum Boltzmann Machines (QBMs) for Q-value approximation has outperformed regular deep reinforcement learning in terms of time-steps needed to converge. However, this algorithm was restricted to single-agent and small 2x2 multi-agent grid domains. In this work, we propose an extension to the original concept in order to solve more challenging problems. Similar to classic DQNs, we add an experience replay buffer and use different networks for approximating the target and policy values. The experimental results show that learning becomes more stable and enables agents to find optimal policies in grid-domains with higher complexity. Additionally, we assess how parameter sharing influences the agents behavior in multi-agent domains. Quantum sampling proves to be a promising method for reinforcement learning tasks, but is currently limited by the QPU size and therefore by the size of the input and Boltzmann machine.
We present a Hybrid-Quantum-classical method for learning Boltzmann machines (BM) for generative and discriminative tasks. Boltzmann machines are undirected graphs that form the building block of many learning architectures such as Restricted Boltzmann machines (RBM's) and Deep Boltzmann machines (DBM's). They have a network of visible and hidden nodes where the former are used as the reading sites while the latter are used to manipulate the probability of the visible states. BM's are versatile machines that can be used for both learning distributions as a generative task as well as for performing classification or function approximation as a discriminative task. We show that minimizing KL-divergence works best for training BM for applications of function approximation. In our approach, we use Quantum annealers for sampling Boltzmann states. These states are used to approximate gradients in a stochastic gradient descent scheme. The approach is used to demonstrate logic circuits in the discriminative sense and a specialized two-phase distribution using generative BM.
Recent theoretical and experimental results suggest the possibility of using current and near-future quantum hardware in challenging sampling tasks. In this paper, we introduce free-energy-based reinforcement learning (FERL) as an application of quantum hardware. We propose a method for processing a quantum annealer’s measured qubit spin configurations in approximating the free energy of a quantum Boltzmann machine (QBM). We then apply this method to perform reinforcement learning on the grid-world problem using the D-Wave 2000Q quantum annealer. The experimental results show that our technique is a promising method for harnessing the power of quantum sampling in reinforcement learning tasks.
Lately, there has been an extensive state-of-the-art research in Machine Learning methods, due to their important features such as universality in approximations and dimensional reduction. In this way, the present work aims at exploiting these properties of Machine Learning on physical many-body problems, in which a restricted Boltzmann Machine (RBM) is trained with quantum Monte Carlo data to best represent the ground state of an N spin system under the transverse field Ising model Hamiltonian in one dimension. During the path coursed along the development of this project, we first review classical well-implemented methods such as the single-layer perceptron, multi-layer perceptron for the XOR problem with back-propagation algorithm, discrete Hopfield network for pattern reconstruction, continuous Hopfield network with Simulated Annealing for solving the traveling salesman problem with 10 cities and the Boltzmann machine for feature extraction. Then, we review a few aspects of what is called a Quantum Boltzmann Machine and the basics of the transverse field Ising model. Finally, we train an RBM consisting of N visible neurons, hidden density neurons alpha=M/N=2 and 1-step contrastive divergence (CD_1), by minimizing the Kullbach-Liebler divergence of the RBM and the training data set coming from the algorithmic variant of quantum Monte Carlo, known as discrete Path-Integral Monte Carlo. For this last step, we use Trotter-Suzuki decomposition with m Trotter slices and the Wolff-s cluster algorithm. After training the RBM, we calculate magnetic observables based on the RBM-s wave-function such as longitudinal and transverse magnetization, for different values of transverse field and we do an estimation of the critical field at which the transverse Ising model system suffers a quantum phase transition from ferromagnet to paramagnet