Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Scientific Reports

Reinforcement Quantum Annealing: A Hybrid Quantum Learning Automata

Ramin Ayanzadeh, Milton Halem, Tim Finin

Published May 14, 2020• 57 citations
Publisher

Abstract

We introduce the notion of reinforcement quantum annealing (RQA) scheme in which an intelligent agent searches in the space of Hamiltonians and interacts with a quantum annealer that plays the stochastic environment role of learning automata. At each iteration of RQA, after analyzing results (samples) from the previous iteration, the agent adjusts the penalty of unsatisfied constraints and re-casts the given problem to a new Ising Hamiltonian. As a proof-of-concept, we propose a novel approach for casting the problem of Boolean satisfiability (SAT) to Ising Hamiltonians and show how to apply the RQA for increasing the probability of finding the global optimum. Our experimental results on two different benchmark SAT problems (namely factoring pseudo-prime numbers and random SAT with phase transitions), using a D-Wave 2000Q quantum processor, demonstrated that RQA finds notably better solutions with fewer samples, compared to the best-known techniques in the realm of quantum annealing.

DOI
10.1038/s41598-020-64078-1

Related papers

Reinforcement Quantum Annealing: A Quantum-Assisted Learning Automata Approach

Ramin Ayanzadeh, Milton Halem, Tim Finin

We introduce the reinforcement quantum annealing (RQA) scheme in which an intelligent agent interacts with a quantum annealer that plays the stochastic environment role of learning automata and tries to iteratively find better Ising Hamiltonians for the given problem of interest. As a proof-of-concept, we propose a novel approach for reducing the NP-complete problem of Boolean satisfiability (SAT) to minimizing Ising Hamiltonians and show how to apply the RQA for increasing the probability of finding the global optimum. Our experimental results on two different benchmark SAT problems (namely factoring pseudo-prime numbers and random SAT with phase transitions), using a D-Wave 2000Q quantum processor, demonstrated that RQA finds notably better solutions with fewer samples, compared to state-of-the-art techniques in the realm of quantum annealing.

arXiv:2001.00234•Jan 1, 2020•3 citations

Leveraging Artificial Intelligence to Advance Problem-Solving with Quantum Annealers

Ramin Ayanzadeh

We show how to advance quantum information processing, specifically problem-solving with quantum annealers, in the realm of artificial intelligence. We introduce SAT++, as a novel quantum programming paradigm, that can compile classical algorithms (implemented in classical programming languages) and execute them on quantum annealers. Moreover, we introduce a post-quantum error correction method that can find samples with significantly lower energy values, compared to the state-of-the-art techniques in quantum annealing. We also demonstrate that performing post-quantum error correction methods can make results of quantum annealers reproducible?i.e., more robust behavior of quantum annealers, compared to recent software and hardware advancements in quantum annealing. In addition, we offer to conjugate both optimization and sampling aspects of the quantum annealers and introduce two AI hybrid approaches. In reinforcement quantum annealing (RQA) scheme, an intelligent agent interacts with a quantum annealer that plays the stochastic environment role of learning automata and searches in the space of Hamiltonians, rather than exploring the Hilbert space of a given Ising model. In the same manner, greedy quantum annealing (GQA) is a novel model that utilizes quantum annealers to better select candidates in greedy algorithms. We experimentally demonstrate that our proposed AI hybrid approaches outperform the best-known techniques in quantum annealing, specifically when problems require longer chains for forming virtual qubits with higher connectivity. Furthermore, we introduce the theory of ensemble quantum annealing (EQA) that generates multiple (distinct) Ising Hamiltonians whose ground states are all identical to a solution to the problem of interest. Our experimental results reveal that applying EQA can significantly boost the performance of the quantum annealers. After benchmarking the D-Wave 2000Q quantum processors, as a proof-of-concept, we apply our proposed models on two real-world problems. As our first study case, we use SAT++ for factoring pseudo-prime numbers on a D-Wave quantum processor that can jeopardize the security of modern public-key cryptography systems. As our second case of study, we address the NP-hard problem of compressive sensing (i.e., the $\ell_0$-norm problem of sparse recovery) through reducing the original problem of compressive sensing to Weighted-MAX-SAT instances, casting the $\ell_0$-norm problem of binary compressive sensing to quadratic unconstrained binary optimization (QUBO), and proposing to solve the original problems of binary compressive sensing and binary compressive sensing with matrix uncertainty on quantum annealers.

•Jan 20, 2020•6 citations

Enhancing the efficiency of quantum annealing via reinforcement: A path-integral Monte Carlo simulation of the quantum reinforcement algorithm

Abolfazl Ramezanpour

The standard quantum annealing algorithm tries to approach the ground state of a classical system by slowly decreasing the hopping rates of a quantum random walk in the configuration space of the problem, where the on-site energies are provided by the classical energy function. In a quantum reinforcement algorithm, the annealing works instead by increasing gradually the strength of the on-site energies according to the probability of finding the walker on each site of the configuration space. Here, by using the path-integral Monte Carlo simulations of the quantum algorithms, we show that annealing via reinforcement can significantly enhance the success probability of the quantum walker. More precisely, we implement a local version of the quantum reinforcement algorithm, where the system wave function is replaced by an approximate wave function using the local expectation values of the system. We use this algorithm to find solutions to a prototypical constraint satisfaction problem (XORSAT) close to the satisfiability to unsatisfiability phase transition. The study is limited to small problem sizes (a few hundreds of variables), nevertheless, the numerical results suggest that quantum reinforcement may provide a useful strategy to deal with other computationally hard problems and larger problem sizes even as a classical optimization algorithm.

arXiv:1812.02569•Physical Review A•Dec 6, 2018•8 citations

Quantum walk in a reinforced free-energy landscape: Quantum annealing with reinforcement

Abolfazl Ramezanpour

Providing an optimal path to a quantum annealing algorithm is key to finding good approximate solutions to computationally hard optimization problems. Reinforcement is one of the strategies that can be used to circumvent the exponentially small energy gaps of the system in the annealing process. Here a time-dependent reinforcement term is added to the Hamiltonian in order to give lower energies to the most probable states of the evolving system. In this study, we take local entropy in the configuration space for the reinforcement and apply the algorithm to a number of easy and hard optimization problems. The reinforced algorithm performs better than the standard quantum annealing algorithm in the quantum search problem, where the optimal parameters behave very differently depending on the number of solutions. Moreover, the reinforcements can change the discontinuous phase transitions of the mean-field $p$-spin model ($p>2$) to a continuous transition. The algorithm's performance in the binary perceptron problem is also superior to that of the standard quantum annealing algorithm, which already works better than a classical simulated annealing algorithm.

arXiv:2202.10908•Physical Review A•Jul 14, 2022•3 citations

Quantum Annealing with Anneal Path Control: Application to 2-SAT Problems with Known Energy Landscapes

Ting-Jui Hsu, Fengping Jin, Christian Seidel +3 more

We study the effect of the anneal path control per qubit, a new user control feature offered on the D-Wave 2000Q quantum annealer, on the performance of quantum annealing for solving optimization problems by numerically solving the time-dependent Schrödinger equation for the time-dependent Hamiltonian modeling the annealing problems. The anneal path control is thereby modeled as a modified linear annealing scheme, resulting in an advanced and retarded scheme. The considered optimization problems are 2-SAT problems with 12 Boolean variables, a known unique ground state and a highly degenerate first excited state. We show that adjustment of the anneal path control can result in a widening of the minimal spectral gap by one or two orders of magnitude and an enhancement of the success probability of finding the solution of the optimization problem. We scrutinize various iterative methods based on the spin floppiness, the average spin value, and on the average energy and describe their performance in boosting the quantum annealing process.

arXiv:1810.00194•Communications in Computational Physics•Apr 28, 2019•10 citations

Adaptive Quantum Annealing for Constraint Satisfaction Problems

Jincheng Zhang

Constraint Satisfaction Problems (CSPs) represent a significant class of computationally challenging problems across diverse fields, including scheduling, routing, and artificial intelligence. Traditional quantum annealing (QA) approaches often face difficulties in effectively tackling complex CSPs, primarily due to challenges in accurately embedding the problem's structure onto the quantum hardware and efficiently exploring the solution space. This research proposes an adaptive quantum annealing (AQA) algorithm designed to mitigate these limitations. The AQA algorithm employs reinforcement learning (RL) to dynamically adjust the embedding strategy and refine the annealing schedule during the QA process. Specifically, the RL agent learns to optimize the problem's representation on the quantum hardware and adapt the annealing parameters based on real-time feedback from the observed solution landscape. The core innovation lies in the integration of adaptive control within the QA framework, potentially unlocking significant improvements in performance for complex CSPs. We present a theoretical framework and outline the key components of the AQA algorithm, demonstrating its potential to overcome the limitations associated with conventional QA methods. The algorithm's ability to dynamically respond to the problem's characteristics offers a promising avenue for enhancing the effectiveness of QA in solving challenging CSP instances.

•Sep 3, 2026•0 citations
We introduce the notion of reinforcement quantum annealing (RQA) scheme in which an intelligent agent searches in the space of Hamiltonians and interacts with a quantum annealer that plays the stochastic environment role of learning automata. At each iteration of RQA, after analyzing results (samples) from the previous iteration, the agent adjusts the penalty of unsatisfied constraints and re-casts the given problem to a new Ising Hamiltonian. As a proof-of-concept, we propose a novel approach for casting the problem of Boolean satisfiability (SAT) to Ising Hamiltonians and show how to apply the RQA for increasing the probability of finding the global optimum. Our experimental results on two different benchmark SAT problems (namely factoring pseudo-prime numbers and random SAT with phase transitions), using a D-Wave 2000Q quantum processor, demonstrated that RQA finds notably better solutions with fewer samples, compared to the best-known techniques in the realm of quantum annealing.
We introduce the reinforcement quantum annealing (RQA) scheme in which an intelligent agent interacts with a quantum annealer that plays the stochastic environment role of learning automata and tries to iteratively find better Ising Hamiltonians for the given problem of interest. As a proof-of-concept, we propose a novel approach for reducing the NP-complete problem of Boolean satisfiability (SAT) to minimizing Ising Hamiltonians and show how to apply the RQA for increasing the probability of finding the global optimum. Our experimental results on two different benchmark SAT problems (namely factoring pseudo-prime numbers and random SAT with phase transitions), using a D-Wave 2000Q quantum processor, demonstrated that RQA finds notably better solutions with fewer samples, compared to state-of-the-art techniques in the realm of quantum annealing.
We show how to advance quantum information processing, specifically problem-solving with quantum annealers, in the realm of artificial intelligence. We introduce SAT++, as a novel quantum programming paradigm, that can compile classical algorithms (implemented in classical programming languages) and execute them on quantum annealers. Moreover, we introduce a post-quantum error correction method that can find samples with significantly lower energy values, compared to the state-of-the-art techniques in quantum annealing. We also demonstrate that performing post-quantum error correction methods can make results of quantum annealers reproducible?i.e., more robust behavior of quantum annealers, compared to recent software and hardware advancements in quantum annealing. In addition, we offer to conjugate both optimization and sampling aspects of the quantum annealers and introduce two AI hybrid approaches. In reinforcement quantum annealing (RQA) scheme, an intelligent agent interacts with a quantum annealer that plays the stochastic environment role of learning automata and searches in the space of Hamiltonians, rather than exploring the Hilbert space of a given Ising model. In the same manner, greedy quantum annealing (GQA) is a novel model that utilizes quantum annealers to better select candidates in greedy algorithms. We experimentally demonstrate that our proposed AI hybrid approaches outperform the best-known techniques in quantum annealing, specifically when problems require longer chains for forming virtual qubits with higher connectivity. Furthermore, we introduce the theory of ensemble quantum annealing (EQA) that generates multiple (distinct) Ising Hamiltonians whose ground states are all identical to a solution to the problem of interest. Our experimental results reveal that applying EQA can significantly boost the performance of the quantum annealers. After benchmarking the D-Wave 2000Q quantum processors, as a proof-of-concept, we apply our proposed models on two real-world problems. As our first study case, we use SAT++ for factoring pseudo-prime numbers on a D-Wave quantum processor that can jeopardize the security of modern public-key cryptography systems. As our second case of study, we address the NP-hard problem of compressive sensing (i.e., the ℓ0\ell_0ℓ0​-norm problem of sparse recovery) through reducing the original problem of compressive sensing to Weighted-MAX-SAT instances, casting the ℓ0\ell_0ℓ0​-norm problem of binary compressive sensing to quadratic unconstrained binary optimization (QUBO), and proposing to solve the original problems of binary compressive sensing and binary compressive sensing with matrix uncertainty on quantum annealers.
The standard quantum annealing algorithm tries to approach the ground state of a classical system by slowly decreasing the hopping rates of a quantum random walk in the configuration space of the problem, where the on-site energies are provided by the classical energy function. In a quantum reinforcement algorithm, the annealing works instead by increasing gradually the strength of the on-site energies according to the probability of finding the walker on each site of the configuration space. Here, by using the path-integral Monte Carlo simulations of the quantum algorithms, we show that annealing via reinforcement can significantly enhance the success probability of the quantum walker. More precisely, we implement a local version of the quantum reinforcement algorithm, where the system wave function is replaced by an approximate wave function using the local expectation values of the system. We use this algorithm to find solutions to a prototypical constraint satisfaction problem (XORSAT) close to the satisfiability to unsatisfiability phase transition. The study is limited to small problem sizes (a few hundreds of variables), nevertheless, the numerical results suggest that quantum reinforcement may provide a useful strategy to deal with other computationally hard problems and larger problem sizes even as a classical optimization algorithm.
Providing an optimal path to a quantum annealing algorithm is key to finding good approximate solutions to computationally hard optimization problems. Reinforcement is one of the strategies that can be used to circumvent the exponentially small energy gaps of the system in the annealing process. Here a time-dependent reinforcement term is added to the Hamiltonian in order to give lower energies to the most probable states of the evolving system. In this study, we take local entropy in the configuration space for the reinforcement and apply the algorithm to a number of easy and hard optimization problems. The reinforced algorithm performs better than the standard quantum annealing algorithm in the quantum search problem, where the optimal parameters behave very differently depending on the number of solutions. Moreover, the reinforcements can change the discontinuous phase transitions of the mean-field ppp-spin model (p>2p>2p>2) to a continuous transition. The algorithm's performance in the binary perceptron problem is also superior to that of the standard quantum annealing algorithm, which already works better than a classical simulated annealing algorithm.
We study the effect of the anneal path control per qubit, a new user control feature offered on the D-Wave 2000Q quantum annealer, on the performance of quantum annealing for solving optimization problems by numerically solving the time-dependent Schrödinger equation for the time-dependent Hamiltonian modeling the annealing problems. The anneal path control is thereby modeled as a modified linear annealing scheme, resulting in an advanced and retarded scheme. The considered optimization problems are 2-SAT problems with 12 Boolean variables, a known unique ground state and a highly degenerate first excited state. We show that adjustment of the anneal path control can result in a widening of the minimal spectral gap by one or two orders of magnitude and an enhancement of the success probability of finding the solution of the optimization problem. We scrutinize various iterative methods based on the spin floppiness, the average spin value, and on the average energy and describe their performance in boosting the quantum annealing process.
Constraint Satisfaction Problems (CSPs) represent a significant class of computationally challenging problems across diverse fields, including scheduling, routing, and artificial intelligence. Traditional quantum annealing (QA) approaches often face difficulties in effectively tackling complex CSPs, primarily due to challenges in accurately embedding the problem's structure onto the quantum hardware and efficiently exploring the solution space. This research proposes an adaptive quantum annealing (AQA) algorithm designed to mitigate these limitations. The AQA algorithm employs reinforcement learning (RL) to dynamically adjust the embedding strategy and refine the annealing schedule during the QA process. Specifically, the RL agent learns to optimize the problem's representation on the quantum hardware and adapt the annealing parameters based on real-time feedback from the observed solution landscape. The core innovation lies in the integration of adaptive control within the QA framework, potentially unlocking significant improvements in performance for complex CSPs. We present a theoretical framework and outline the key components of the AQA algorithm, demonstrating its potential to overcome the limitations associated with conventional QA methods. The algorithm's ability to dynamically respond to the problem's characteristics offers a promising avenue for enhancing the effectiveness of QA in solving challenging CSP instances.