Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum

RL-assisted Annealing for QUBO on a Multi-GPU System

Reo Gakumi, Ryota Yasudo

Published Jun 3, 2025• 0 citations
Publisher

Abstract

Quadratic Unconstrained Binary Optimization (QUBO) has emerged as a unifying framework for diverse NP-hard combinatorial optimization problems. However, a major challenge in existing QUBO solvers is the need for extensive manual tuning of algorithmic hyperparameters, such as temperature schedules in simulated annealing, which can vary greatly in effectiveness depending on the problem instance. In this paper, we propose RL-assisted annealing(RLA), a novel approach that integrates reinforcement learning (RL) with annealing-based local searches. Rather than relying on hand-crafted heuristics, RLA trains an agent to adaptively determine a bit-flip policy by observing the statistical properties of the energy differences in the objective function. Crucially, RLA encodes QUBO states as fixed-dimensional statistics, making the method scalable to various problem sizes. To further support large-scale problems, we employ distributed training on multi-GPU platforms using JAX’s pmap, parallelizing both environment simulation and policy updates. Experimental evaluations on benchmark datasets, including TSP, QAP, Max-Cut, and randomly generated QUBO instances, demonstrate that RLA achieves solution qualities on par with or better than conventional annealing-based methods, while maintaining robust performance across diverse problem instances without extensive hyperparameter tuning. These results highlight RLA as a promising step toward a flexible and practical solver for QUBO-based applications.

DOI
10.1109/ipdpsw66978.2025.00095

Related papers

RL-driven Annealing Computation for QUBO on Multi-GPU System

Reo Gakumi, Ryota Yasudo

Quadratic Unconstrained Binary Optimization (QUBO) has emerged as a unifying framework for diverse NP-hard combinatorial optimization problems.However, a major challenge in existing QUBO solvers is the need for extensive manual tuning of algorithmic hyperparameters, such as temperature schedules in simulated annealing, which can vary greatly in effectiveness depending on the problem instance.In this paper, we propose RL-driven annealing (RLA), a novel approach that integrates reinforcement learning (RL) with annealing-based local searches.Rather than relying on hand-crafted heuristics, RLA trains an agent to adaptively determine a bit-flip policy by observing the statistical properties of the energy differences in the objective function.Crucially, RLA encodes QUBO states as fixed-dimensional statistics, making the method scalable to various problem sizes.To further support large-scale problems, we employ distributed training on multi-GPU platforms using JAX's pmap, parallelizing both environment simulation and policy updates.Experimental evaluations on benchmark datasets, including TSP, QAP, Max-Cut, and randomly generated QUBO instances, demonstrate that RLA achieves solution qualities on par with or better than conventional annealing-based methods, while maintaining robust performance across diverse problem instances without extensive hyperparameter tuning.These results highlight RLA as a promising step toward a flexible and practical solver for QUBO-based applications.

International Journal of Networking and Computing•Jan 5, 2026•0 citations

Hybrid Heuristic Algorithms for Adiabatic Quantum Machine Learning Models

Bahram Alidaee, Haibo Wang, Lutfu Sua +1 more

Numerous established machine learning models and various neural network architectures can be restructured as Quadratic Unconstrained Binary Optimization (QUBO) problems. A significant challenge in Adiabatic Quantum Machine Learning (AQML) is the computational demand of the training phase. To mitigate this, approximation techniques inspired by quantum annealing, like Simulated Annealing and Multiple Start Tabu Search (MSTS), have been employed to expedite QUBO-based AQML training. This paper introduces a novel hybrid algorithm that incorporates an "r-flip" strategy. This strategy is aimed at solving large-scale QUBO problems more effectively, offering better solution quality and lower computational costs compared to existing MSTS methods. The r-flip approach has practical applications in diverse fields, including cross-docking, supply chain management, machine scheduling, and fraud detection. The paper details extensive computational experiments comparing this r-flip enhanced hybrid heuristic against a standard MSTS approach. These tests utilize both standard benchmark problems and three particularly large QUBO instances. The results indicate that the r-flip enhanced method consistently produces high-quality solutions efficiently, operating within practical time constraints.

arXiv:2407.21062•Jul 26, 2024•0 citations

Minor embedding for quantum annealing with reinforcement learning

Riccardo Nembrini, Maurizio Ferrari Dacrema, Paolo Cremonesi

Quantum Annealing (QA) is a quantum computing paradigm for solving combinatorial optimization problems formulated as Quadratic Unconstrained Binary Optimization (QUBO) problems. An essential step in QA is minor embedding, which maps the problem graph onto the sparse topology of the quantum processor and then adjusts the problem weights. The process of mapping the problem variables to the hardware is computationally expensive and scales poorly with increasing problem size and hardware complexity. Existing heuristics are often developed for specific problem graphs or hardware topologies and are difficult to generalize. To address this limitation, we explore the use of machine learning methods, which would allow a much greater degree of flexibility, in particular of Reinforcement Learning (RL). RL offers a promising alternative by treating minor embedding as a sequential decision-making problem, where an agent learns to construct minor embeddings by iteratively mapping the problem variables to the hardware qubits. We propose a RL-based approach to minor embedding using a Proximal Policy Optimization agent, testing its ability to embed both fully connected and randomly generated problem graphs on two hardware topologies, Chimera and Zephyr. The results show that our agent consistently produces valid minor embeddings even when they span over more than a thousand qubits, in particular on the more modern Zephyr topology. Our proposed approach is also able to scale to moderate problem sizes and adapts well to different graph structures, highlighting RL’s potential as a flexible and general-purpose framework for minor embedding in QA but also pointing to limitations that will need to be addressed, for example in reducing the number of qubits required.

arXiv:2507.16004•Quantum Machine Intelligence•Feb 19, 2026•3 citations

Diverse Adaptive Bulk Search: a Framework for Solving QUBO Problems on Multiple GPUs

Koji Nakano, Daisuke Takafuji, Yasuaki Ito +5 more

Quadratic Unconstrained Binary Optimization (QUBO) is a combinatorial optimization to find an optimal binary solution vector that minimizes the energy value defined by a quadratic formula of binary variables in the vector. As many NP-hard problems can be reduced to QUBO problems, considerable research has gone into developing QUBO solvers running on various computing platforms such as quantum devices, ASICs, FPGAs, GPUs, and optical fibers. This paper presents a framework called Diverse Adaptive Bulk Search (DABS), which has the potential to find optimal solutions of many types of QUBO problems. Our DABS solver employs a genetic algorithm-based search algorithm featuring three diverse strategies: multiple search algorithms, multiple genetic operations, and multiple solution pools. During the execution of the solver, search algorithms and genetic operations that succeeded in finding good solutions are automatically selected to obtain better solutions. Moreover, search algorithms traverse between different solution pools to find good solutions. We have implemented our DABS solver to run on multiple GPUs. Experimental evaluations using eight NVIDIA A100 GPUs confirm that our DABS solver succeeds in finding optimal or potentially optimal solutions for three types of QUBO problems.

arXiv:2207.03069•IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum•May 1, 2023•20 citations

Techniques to enhance a QUBO solver for permutation-based combinatorial optimization

Siong Thye Goh, Jianyuan Bo, Sabrish Gopalakrishnan +1 more

Many combinatorial optimization problems can be formulated as a problem to determine the order of sequence or to find a corresponding mapping of the objects. We call such problems permutation-based optimization problems. Many such problems can be formulated as a quadratic unconstrained binary optimization (QUBO) or Ising model by introducing a penalty coefficient to the permutation constraint terms. While classical and quantum annealing approaches have been proposed to solve QUBOs to date, they face issues with optimality and feasibility. Here we treat a given QUBO solver as a black box and propose techniques to enhance its performance. First, to ensure an effective search for good quality solutions, a smooth energy landscape is needed; we propose a data scaling approach that reduces the amplitudes of the input without compromising optimality. Second, we need to tune the penalty coefficient. In this paper, we illustrate that for certain problems, it suffices to tune the parameter by performing random sampling on the penalty coefficients to achieve good performance. Finally, to handle possible infeasibility of the solution, we introduce a polynomial-time projection algorithm. We apply these techniques along with a divide-and-conquer strategy to solve some large-scale permutation-based problems and present results for TSP and QAP.

Genetic and Evolutionary Computation Conference Companion (GECCO Companion)•Jul 9, 2022•10 citations

QUBO-Driven Quantum Annealing: A Unified Cloud-Scale Framework for Resource Forecasting, Classification, and Deep Learning Training

Yulian Song, Guozheng Zhao, Nan Wang +3 more

This paper proposes an integrated QUBO-based quantum-annealing optimization framework for next-generation cloud computing. First, for resource-demand forecasting, we blend Autoregressive (AR) models with Quadratic Unconstrained Binary Optimization (QUBO) and solve the resulting problem via Simulated Annealing; monthly-scale tests reveal superior trend-capture accuracy compared with classical predictors. Second, we embed quantum optimization into Support Vector Machines by casting kernel selection and penalty-parameter tuning as QUBO instances; evaluations on the Iris benchmark—and on synthetically enlarged variants—show marked gains in both classification accuracy and training speed, especially under large-data regimes. Third, we extend the same paradigm to deep learning: a QUBO-formulated quantum-annealing routine optimizes weight initialization and layer-wise learning rates in convolutional networks for image classification, yielding higher top-1 accuracy and reduced wall-clock training time. Across all tasks the hybrid quantum-classical solver consistently outperforms its classical counterparts while maintaining linear scalability in cloud environments. These results show the practical potential of quantum - enhanced algorithms for resource prediction, classification, and deep learning, and pave the way for wider use of quantum technologies in cloud - based AI services.

Applied and Computational Engineering•Aug 13, 2025•0 citations
Quadratic Unconstrained Binary Optimization (QUBO) has emerged as a unifying framework for diverse NP-hard combinatorial optimization problems. However, a major challenge in existing QUBO solvers is the need for extensive manual tuning of algorithmic hyperparameters, such as temperature schedules in simulated annealing, which can vary greatly in effectiveness depending on the problem instance. In this paper, we propose RL-assisted annealing(RLA), a novel approach that integrates reinforcement learning (RL) with annealing-based local searches. Rather than relying on hand-crafted heuristics, RLA trains an agent to adaptively determine a bit-flip policy by observing the statistical properties of the energy differences in the objective function. Crucially, RLA encodes QUBO states as fixed-dimensional statistics, making the method scalable to various problem sizes. To further support large-scale problems, we employ distributed training on multi-GPU platforms using JAX’s pmap, parallelizing both environment simulation and policy updates. Experimental evaluations on benchmark datasets, including TSP, QAP, Max-Cut, and randomly generated QUBO instances, demonstrate that RLA achieves solution qualities on par with or better than conventional annealing-based methods, while maintaining robust performance across diverse problem instances without extensive hyperparameter tuning. These results highlight RLA as a promising step toward a flexible and practical solver for QUBO-based applications.
Quadratic Unconstrained Binary Optimization (QUBO) has emerged as a unifying framework for diverse NP-hard combinatorial optimization problems.However, a major challenge in existing QUBO solvers is the need for extensive manual tuning of algorithmic hyperparameters, such as temperature schedules in simulated annealing, which can vary greatly in effectiveness depending on the problem instance.In this paper, we propose RL-driven annealing (RLA), a novel approach that integrates reinforcement learning (RL) with annealing-based local searches.Rather than relying on hand-crafted heuristics, RLA trains an agent to adaptively determine a bit-flip policy by observing the statistical properties of the energy differences in the objective function.Crucially, RLA encodes QUBO states as fixed-dimensional statistics, making the method scalable to various problem sizes.To further support large-scale problems, we employ distributed training on multi-GPU platforms using JAX's pmap, parallelizing both environment simulation and policy updates.Experimental evaluations on benchmark datasets, including TSP, QAP, Max-Cut, and randomly generated QUBO instances, demonstrate that RLA achieves solution qualities on par with or better than conventional annealing-based methods, while maintaining robust performance across diverse problem instances without extensive hyperparameter tuning.These results highlight RLA as a promising step toward a flexible and practical solver for QUBO-based applications.
Numerous established machine learning models and various neural network architectures can be restructured as Quadratic Unconstrained Binary Optimization (QUBO) problems. A significant challenge in Adiabatic Quantum Machine Learning (AQML) is the computational demand of the training phase. To mitigate this, approximation techniques inspired by quantum annealing, like Simulated Annealing and Multiple Start Tabu Search (MSTS), have been employed to expedite QUBO-based AQML training. This paper introduces a novel hybrid algorithm that incorporates an "r-flip" strategy. This strategy is aimed at solving large-scale QUBO problems more effectively, offering better solution quality and lower computational costs compared to existing MSTS methods. The r-flip approach has practical applications in diverse fields, including cross-docking, supply chain management, machine scheduling, and fraud detection. The paper details extensive computational experiments comparing this r-flip enhanced hybrid heuristic against a standard MSTS approach. These tests utilize both standard benchmark problems and three particularly large QUBO instances. The results indicate that the r-flip enhanced method consistently produces high-quality solutions efficiently, operating within practical time constraints.
Quantum Annealing (QA) is a quantum computing paradigm for solving combinatorial optimization problems formulated as Quadratic Unconstrained Binary Optimization (QUBO) problems. An essential step in QA is minor embedding, which maps the problem graph onto the sparse topology of the quantum processor and then adjusts the problem weights. The process of mapping the problem variables to the hardware is computationally expensive and scales poorly with increasing problem size and hardware complexity. Existing heuristics are often developed for specific problem graphs or hardware topologies and are difficult to generalize. To address this limitation, we explore the use of machine learning methods, which would allow a much greater degree of flexibility, in particular of Reinforcement Learning (RL). RL offers a promising alternative by treating minor embedding as a sequential decision-making problem, where an agent learns to construct minor embeddings by iteratively mapping the problem variables to the hardware qubits. We propose a RL-based approach to minor embedding using a Proximal Policy Optimization agent, testing its ability to embed both fully connected and randomly generated problem graphs on two hardware topologies, Chimera and Zephyr. The results show that our agent consistently produces valid minor embeddings even when they span over more than a thousand qubits, in particular on the more modern Zephyr topology. Our proposed approach is also able to scale to moderate problem sizes and adapts well to different graph structures, highlighting RL’s potential as a flexible and general-purpose framework for minor embedding in QA but also pointing to limitations that will need to be addressed, for example in reducing the number of qubits required.
Quadratic Unconstrained Binary Optimization (QUBO) is a combinatorial optimization to find an optimal binary solution vector that minimizes the energy value defined by a quadratic formula of binary variables in the vector. As many NP-hard problems can be reduced to QUBO problems, considerable research has gone into developing QUBO solvers running on various computing platforms such as quantum devices, ASICs, FPGAs, GPUs, and optical fibers. This paper presents a framework called Diverse Adaptive Bulk Search (DABS), which has the potential to find optimal solutions of many types of QUBO problems. Our DABS solver employs a genetic algorithm-based search algorithm featuring three diverse strategies: multiple search algorithms, multiple genetic operations, and multiple solution pools. During the execution of the solver, search algorithms and genetic operations that succeeded in finding good solutions are automatically selected to obtain better solutions. Moreover, search algorithms traverse between different solution pools to find good solutions. We have implemented our DABS solver to run on multiple GPUs. Experimental evaluations using eight NVIDIA A100 GPUs confirm that our DABS solver succeeds in finding optimal or potentially optimal solutions for three types of QUBO problems.
Many combinatorial optimization problems can be formulated as a problem to determine the order of sequence or to find a corresponding mapping of the objects. We call such problems permutation-based optimization problems. Many such problems can be formulated as a quadratic unconstrained binary optimization (QUBO) or Ising model by introducing a penalty coefficient to the permutation constraint terms. While classical and quantum annealing approaches have been proposed to solve QUBOs to date, they face issues with optimality and feasibility. Here we treat a given QUBO solver as a black box and propose techniques to enhance its performance. First, to ensure an effective search for good quality solutions, a smooth energy landscape is needed; we propose a data scaling approach that reduces the amplitudes of the input without compromising optimality. Second, we need to tune the penalty coefficient. In this paper, we illustrate that for certain problems, it suffices to tune the parameter by performing random sampling on the penalty coefficients to achieve good performance. Finally, to handle possible infeasibility of the solution, we introduce a polynomial-time projection algorithm. We apply these techniques along with a divide-and-conquer strategy to solve some large-scale permutation-based problems and present results for TSP and QAP.
This paper proposes an integrated QUBO-based quantum-annealing optimization framework for next-generation cloud computing. First, for resource-demand forecasting, we blend Autoregressive (AR) models with Quadratic Unconstrained Binary Optimization (QUBO) and solve the resulting problem via Simulated Annealing; monthly-scale tests reveal superior trend-capture accuracy compared with classical predictors. Second, we embed quantum optimization into Support Vector Machines by casting kernel selection and penalty-parameter tuning as QUBO instances; evaluations on the Iris benchmark—and on synthetically enlarged variants—show marked gains in both classification accuracy and training speed, especially under large-data regimes. Third, we extend the same paradigm to deep learning: a QUBO-formulated quantum-annealing routine optimizes weight initialization and layer-wise learning rates in convolutional networks for image classification, yielding higher top-1 accuracy and reduced wall-clock training time. Across all tasks the hybrid quantum-classical solver consistently outperforms its classical counterparts while maintaining linear scalability in cloud environments. These results show the practical potential of quantum - enhanced algorithms for resource prediction, classification, and deep learning, and pave the way for wider use of quantum technologies in cloud - based AI services.