Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:1812.02569 · Physical Review A

Enhancing the efficiency of quantum annealing via reinforcement: A path-integral Monte Carlo simulation of the quantum reinforcement algorithm

Abolfazl Ramezanpour

Published Dec 6, 2018• 8 citations
arXivHTMLPDF

Abstract

The standard quantum annealing algorithm tries to approach the ground state of a classical system by slowly decreasing the hopping rates of a quantum random walk in the configuration space of the problem, where the on-site energies are provided by the classical energy function. In a quantum reinforcement algorithm, the annealing works instead by increasing gradually the strength of the on-site energies according to the probability of finding the walker on each site of the configuration space. Here, by using the path-integral Monte Carlo simulations of the quantum algorithms, we show that annealing via reinforcement can significantly enhance the success probability of the quantum walker. More precisely, we implement a local version of the quantum reinforcement algorithm, where the system wave function is replaced by an approximate wave function using the local expectation values of the system. We use this algorithm to find solutions to a prototypical constraint satisfaction problem (XORSAT) close to the satisfiability to unsatisfiability phase transition. The study is limited to small problem sizes (a few hundreds of variables), nevertheless, the numerical results suggest that quantum reinforcement may provide a useful strategy to deal with other computationally hard problems and larger problem sizes even as a classical optimization algorithm.

DOI
10.1103/physreva.98.062309

Related papers

Quantum walk in a reinforced free-energy landscape: Quantum annealing with reinforcement

Abolfazl Ramezanpour

Providing an optimal path to a quantum annealing algorithm is key to finding good approximate solutions to computationally hard optimization problems. Reinforcement is one of the strategies that can be used to circumvent the exponentially small energy gaps of the system in the annealing process. Here a time-dependent reinforcement term is added to the Hamiltonian in order to give lower energies to the most probable states of the evolving system. In this study, we take local entropy in the configuration space for the reinforcement and apply the algorithm to a number of easy and hard optimization problems. The reinforced algorithm performs better than the standard quantum annealing algorithm in the quantum search problem, where the optimal parameters behave very differently depending on the number of solutions. Moreover, the reinforcements can change the discontinuous phase transitions of the mean-field $p$-spin model ($p>2$) to a continuous transition. The algorithm's performance in the binary perceptron problem is also superior to that of the standard quantum annealing algorithm, which already works better than a classical simulated annealing algorithm.

arXiv:2202.10908•Physical Review A•Jul 14, 2022•3 citations

Optimization by a quantum reinforcement algorithm

Abolfazl Ramezanpour

A reinforcement algorithm solves a classical optimization problem by introducing a feedback to the system, which slowly changes the energy landscape and converges the algorithm to an optimal solution in the configuration space. Here, we use this strategy to concentrate (localize) the wave function of a quantum particle, which explores the configuration space of the problem, preferentially on an optimal configuration. We examine the method by solving numerically the equations governing the evolution of the system, which are similar to the nonlinear Schr\"odinger equations, for small problem sizes. In particular, we observe that reinforcement increases the minimal energy gap of the system in a quantum annealing algorithm. Our numerical simulations and the latter observation show that such kind of quantum feedback might be helpful in solving a computationally hard optimization problem by a quantum reinforcement algorithm.

arXiv:1706.04262•Physical Review A•Nov 3, 2017•11 citations

Optimization by quantum annealing: Lessons from hard satisfiability problems

Demian A. Battaglia, Giuseppe E. Santoro, Erio Tosatti

The path integral Monte Carlo simulated quantum annealing algorithm is applied to the optimization of a large hard instance of the random satisfiability problem (N = 10,000). The dynamical behavior of the quantum and the classical annealing are compared, showing important qualitative differences in the way of exploring the complex energy landscape of the combinatorial optimization problem. At variance with the results obtained for the Ising spin glass and for the traveling salesman problem, in the present case the linear-schedule quantum annealing performance is definitely worse than classical annealing. Nevertheless, a quantum cooling protocol based on field-cycling and able to outperform standard classical simulated annealing over short time scales is introduced.

arXiv:cond-mat/0502468•Physical Review E•Jun 29, 2005•107 citations

Reinforcement Quantum Annealing: A Hybrid Quantum Learning Automata

Ramin Ayanzadeh, Milton Halem, Tim Finin

We introduce the notion of reinforcement quantum annealing (RQA) scheme in which an intelligent agent searches in the space of Hamiltonians and interacts with a quantum annealer that plays the stochastic environment role of learning automata. At each iteration of RQA, after analyzing results (samples) from the previous iteration, the agent adjusts the penalty of unsatisfied constraints and re-casts the given problem to a new Ising Hamiltonian. As a proof-of-concept, we propose a novel approach for casting the problem of Boolean satisfiability (SAT) to Ising Hamiltonians and show how to apply the RQA for increasing the probability of finding the global optimum. Our experimental results on two different benchmark SAT problems (namely factoring pseudo-prime numbers and random SAT with phase transitions), using a D-Wave 2000Q quantum processor, demonstrated that RQA finds notably better solutions with fewer samples, compared to the best-known techniques in the realm of quantum annealing.

Scientific Reports•May 14, 2020•57 citations

Quantum annealing: A new method for minimizing multidimensional functions

A B Finnila, M A Gomez, C. Sebenik +2 more

Abstract Quantum Monte Carlo has found a new application far afield from the solution of the Schrodinger equation for many-body problems. In this paper the authors report a method for solving multidimensional optimization problems that is superior, in many cases, to other methods such as conjugate gradient methods, the simplex method, direction-set methods, genetic methods, and classical simulated annealing. The new approach, aptly titled “quantum annealing,” is closely related to classical annealing, in which a physical system such as a metal cluster is slowly cooled and finds its lowest-energy configuration as the temperature drops to zero. In classical annealing the system follows classical-mechanical dynamics as the temperature drops. In quantum annealing a collection of configurations evolves in the the same way as walkers undergoing diffusion and multiplication in diffusion QMC with the added feature of a slowly dropping value for Planck’s constant h. Classical annealing allows the escape from local minima with the aid of thermal fluctuations. Quantum annealing allows such escape by delocalization and tunneling. Since a QMC walker can multiply in number on entering a low-energy region, a single walker can populate an entire region.

•Jun 18, 2007•23 citations

Optimization using quantum mechanics: quantum annealing through adiabatic evolution

Giuseppe E. Santoro, Erio Tosatti

We review here some recent work in the field of quantum annealing, alias adiabatic quantum computation. The idea of quantum annealing is to perform optimization by a quantum adiabatic evolution which tracks the ground state of a suitable time-dependent Hamiltonian, where 'ℏ' is slowly switched off. We illustrate several applications of quantum annealing strategies, starting from textbook toy-models—double-well potentials and other one-dimensional examples, with and without disorder. These examples display in a clear way the crucial differences between classical and quantum annealing. We then discuss applications of quantum annealing to challenging hard optimization problems, such as the random Ising model, the travelling salesman problem and Boolean satisfiability problems. The techniques used to implement quantum annealing are either deterministic Schrödinger's evolutions, for the toy models, or path-integral Monte Carlo and Green's function Monte Carlo approaches, for the hard optimization problems. The crucial role played by disorder and the associated non-trivial Landau–Zener tunnelling phenomena is discussed and emphasized.

Journal of Physics A Mathematical and General•Aug 18, 2006•354 citations
The standard quantum annealing algorithm tries to approach the ground state of a classical system by slowly decreasing the hopping rates of a quantum random walk in the configuration space of the problem, where the on-site energies are provided by the classical energy function. In a quantum reinforcement algorithm, the annealing works instead by increasing gradually the strength of the on-site energies according to the probability of finding the walker on each site of the configuration space. Here, by using the path-integral Monte Carlo simulations of the quantum algorithms, we show that annealing via reinforcement can significantly enhance the success probability of the quantum walker. More precisely, we implement a local version of the quantum reinforcement algorithm, where the system wave function is replaced by an approximate wave function using the local expectation values of the system. We use this algorithm to find solutions to a prototypical constraint satisfaction problem (XORSAT) close to the satisfiability to unsatisfiability phase transition. The study is limited to small problem sizes (a few hundreds of variables), nevertheless, the numerical results suggest that quantum reinforcement may provide a useful strategy to deal with other computationally hard problems and larger problem sizes even as a classical optimization algorithm.
Providing an optimal path to a quantum annealing algorithm is key to finding good approximate solutions to computationally hard optimization problems. Reinforcement is one of the strategies that can be used to circumvent the exponentially small energy gaps of the system in the annealing process. Here a time-dependent reinforcement term is added to the Hamiltonian in order to give lower energies to the most probable states of the evolving system. In this study, we take local entropy in the configuration space for the reinforcement and apply the algorithm to a number of easy and hard optimization problems. The reinforced algorithm performs better than the standard quantum annealing algorithm in the quantum search problem, where the optimal parameters behave very differently depending on the number of solutions. Moreover, the reinforcements can change the discontinuous phase transitions of the mean-field ppp-spin model (p>2p>2p>2) to a continuous transition. The algorithm's performance in the binary perceptron problem is also superior to that of the standard quantum annealing algorithm, which already works better than a classical simulated annealing algorithm.
A reinforcement algorithm solves a classical optimization problem by introducing a feedback to the system, which slowly changes the energy landscape and converges the algorithm to an optimal solution in the configuration space. Here, we use this strategy to concentrate (localize) the wave function of a quantum particle, which explores the configuration space of the problem, preferentially on an optimal configuration. We examine the method by solving numerically the equations governing the evolution of the system, which are similar to the nonlinear Schr\"odinger equations, for small problem sizes. In particular, we observe that reinforcement increases the minimal energy gap of the system in a quantum annealing algorithm. Our numerical simulations and the latter observation show that such kind of quantum feedback might be helpful in solving a computationally hard optimization problem by a quantum reinforcement algorithm.
The path integral Monte Carlo simulated quantum annealing algorithm is applied to the optimization of a large hard instance of the random satisfiability problem (N = 10,000). The dynamical behavior of the quantum and the classical annealing are compared, showing important qualitative differences in the way of exploring the complex energy landscape of the combinatorial optimization problem. At variance with the results obtained for the Ising spin glass and for the traveling salesman problem, in the present case the linear-schedule quantum annealing performance is definitely worse than classical annealing. Nevertheless, a quantum cooling protocol based on field-cycling and able to outperform standard classical simulated annealing over short time scales is introduced.
We introduce the notion of reinforcement quantum annealing (RQA) scheme in which an intelligent agent searches in the space of Hamiltonians and interacts with a quantum annealer that plays the stochastic environment role of learning automata. At each iteration of RQA, after analyzing results (samples) from the previous iteration, the agent adjusts the penalty of unsatisfied constraints and re-casts the given problem to a new Ising Hamiltonian. As a proof-of-concept, we propose a novel approach for casting the problem of Boolean satisfiability (SAT) to Ising Hamiltonians and show how to apply the RQA for increasing the probability of finding the global optimum. Our experimental results on two different benchmark SAT problems (namely factoring pseudo-prime numbers and random SAT with phase transitions), using a D-Wave 2000Q quantum processor, demonstrated that RQA finds notably better solutions with fewer samples, compared to the best-known techniques in the realm of quantum annealing.
Abstract Quantum Monte Carlo has found a new application far afield from the solution of the Schrodinger equation for many-body problems. In this paper the authors report a method for solving multidimensional optimization problems that is superior, in many cases, to other methods such as conjugate gradient methods, the simplex method, direction-set methods, genetic methods, and classical simulated annealing. The new approach, aptly titled “quantum annealing,” is closely related to classical annealing, in which a physical system such as a metal cluster is slowly cooled and finds its lowest-energy configuration as the temperature drops to zero. In classical annealing the system follows classical-mechanical dynamics as the temperature drops. In quantum annealing a collection of configurations evolves in the the same way as walkers undergoing diffusion and multiplication in diffusion QMC with the added feature of a slowly dropping value for Planck’s constant h. Classical annealing allows the escape from local minima with the aid of thermal fluctuations. Quantum annealing allows such escape by delocalization and tunneling. Since a QMC walker can multiply in number on entering a low-energy region, a single walker can populate an entire region.
We review here some recent work in the field of quantum annealing, alias adiabatic quantum computation. The idea of quantum annealing is to perform optimization by a quantum adiabatic evolution which tracks the ground state of a suitable time-dependent Hamiltonian, where 'ℏ' is slowly switched off. We illustrate several applications of quantum annealing strategies, starting from textbook toy-models—double-well potentials and other one-dimensional examples, with and without disorder. These examples display in a clear way the crucial differences between classical and quantum annealing. We then discuss applications of quantum annealing to challenging hard optimization problems, such as the random Ising model, the travelling salesman problem and Boolean satisfiability problems. The techniques used to implement quantum annealing are either deterministic Schrödinger's evolutions, for the toy models, or path-integral Monte Carlo and Green's function Monte Carlo approaches, for the hard optimization problems. The crucial role played by disorder and the associated non-trivial Landau–Zener tunnelling phenomena is discussed and emphasized.