Recent advances in deep learning techniques pose a question of whether they can facilitate the task of finding good quality solutions to combinatorial optimization (CO) problems in a practically relevant solution time. Specifically, it is of practical relevance to determine to which extent graph neural networks (GNNs) can be applied to CO problems that can be formulated as QUBO’s and thus be naturally interpreted as graph problems. In this research a GNNsolver is applied to two classical CO problems–the maximum cut problem and maximum independent set problem–in an unsupervised learning setting. We show that while GNN solver consistently finds good quality solutions for the Max Cut problem irrespective of the size and density of the graph, solving MIS problems is challenging for all but very sparse graphs. We further show how this problem can be addressed by embedding transfer between these two problems and compare two different GNN architectures–GCN and GraphSAGE on their robustness with respect to graph density and symmetry. Finally we demonstrate that change of widely used Adam optimizer to Rprop optimizer can lead to considerable reduction of solution times. • Previously suggested GNNsolver is effective on sparse graphs only. • Replacing Adam with RPROP reduced solution times. • GraphSAGE can handle denser graphs, GCN performs better on sparse graphs. • Transfer learning between MaxCut and MIS problems improves solvers performance.
Recent advances in deep learning techniques pose a question of whether they can facilitate the task of finding good quality solutions to combinatorial optimization (CO) problems in a practically relevant solution time. Specifically, it is of practical relevance to determine to which extent graph neural networks (GNNs) can be applied to CO problems that can be formulated as QUBO’s and thus be naturally interpreted as graph problems. In this research a GNNsolver is applied to two classical CO problems–the maximum cut problem and maximum independent set problem–in an unsupervised learning setting. We show that while GNN solver consistently finds good quality solutions for the Max Cut problem irrespective of the size and density of the graph, solving MIS problems is challenging for all but very sparse graphs. We further show how this problem can be addressed by embedding transfer between these two problems and compare two different GNN architectures–GCN and GraphSAGE on their robustness with respect to graph density and symmetry. Finally we demonstrate that change of widely used Adam optimizer to Rprop optimizer can lead to considerable reduction of solution times. • Previously suggested GNNsolver is effective on sparse graphs only. • Replacing Adam with RPROP reduced solution times. • GraphSAGE can handle denser graphs, GCN performs better on sparse graphs. • Transfer learning between MaxCut and MIS problems improves solvers performance.
In recent years, there has been notable interest in investigating combinatorial optimization (CO) problems by neural-based framework. An emerging strategy to tackle these challenging problems involves the adoption of graph neural networks (GNNs) as an alternative to traditional algorithms, a subject that has attracted considerable attention. Despite the growing popularity of GNNs and traditional algorithm solvers in the realm of CO, there is limited research on their integrated use and the correlation between them within an end-to-end framework. The primary focus of our work is to formulate a more efficient and precise framework for CO by employing decision-focused learning on graphs. Additionally, we introduce a decision-focused framework that utilizes GNNs to address CO problems with auxiliary support. To realize an end-to-end approach, we have designed two cascaded modules: (a) an unsupervised trained graph predictive model, and (b) a solver for quadratic binary unconstrained optimization. Empirical evaluations are conducted on various classical tasks, including maximum cut, maximum independent set, and minimum vertex cover. The experimental results on classical CO problems (i.e. MaxCut, MIS, and MVC) demonstrate the superiority of our method over both the standalone GNN approach and classical methods.
In recent years, graph neural networks (GNNs) have become increasingly popular for solving NP-hard combinatorial optimization (CO) problems, such as maximum cut and maximum independent set. The core idea behind these methods is to represent a CO problem as a graph and then use GNNs to learn the node/graph embedding with combinatorial information. Although these methods have achieved promising results, given a specific CO problem, the design of GNN architectures still requires heavy manual work with domain knowledge. Existing automated GNNs are mostly focused on traditional graph learning problems, which is inapplicable to solving NP-hard CO problems. To this end, we present a new class of \textbf{AUTO}mated \textbf{G}NNs for solving \textbf{NP}-hard problems, namely \textbf{AutoGNP}. We represent CO problems by GNNs and focus on two specific problems, i.e., mixed integer linear programming and quadratic unconstrained binary optimization. The idea of AutoGNP is to use graph neural architecture search algorithms to automatically find the best GNNs for a given NP-hard combinatorial optimization problem. Compared with existing graph neural architecture search algorithms, AutoGNP utilizes two-hop operators in the architecture search space. Moreover, AutoGNP utilizes simulated annealing and a strict early stopping policy to avoid local optimal solutions. Empirical results on benchmark combinatorial problems demonstrate the superiority of our proposed model.
Quadratic Unconstrained Binary Optimization (QUBO) is a generic technique to model various NP-hard combinatorial optimization problems in the form of binary variables. The Hamiltonian function is often used to formulate QUBO problems where it is used as the objective function in the context of optimization. Recently, PI-GNN, a generic scalable framework, has been proposed to address the Combinatorial Optimization (CO) problems over graphs based on a simple Graph Neural Network (GNN) architecture. Their novel contribution was a generic QUBO-formulated Hamiltonian-inspired loss function that was optimized using GNN. In this study, we address a crucial issue related to the aforementioned setup especially observed in denser graphs. The reinforcement learning-based paradigm has also been widely used to address numerous CO problems. Here we also formulate and empirically evaluate the compatibility of the QUBO-formulated Hamiltonian as the generic reward function in the Reinforcement Learning paradigm to directly integrate the actual node projection status during training as the form of rewards. In our experiments, we observed up to 44% improvement in the RL-based setup compared to the PI-GNN algorithm. Our implementation can be found in https://github.com/rizveeredwan/learning-graph-structure.
Quadratic Unconstrained Binary Optimization (QUBO) is a versatile approach used to represent a wide range of NP-hard Combinatorial Optimization (CO) problems through binary variables. The transformation of QUBO to an Ising Hamiltonian is recognized as an effective method for solving key optimization problems using quantum algorithms. Recently, PI-GNN, a generic framework, has been proposed to address CO problems over graphs based on QUBO with Hamiltonian loss function to train the underlying GNN architecture. Though PI-GNN is highly scalable, it exhibits a noticeable decrease in terms of the number of satisfied constraints with higher graph densities. In this paper, firstly, we identify the limitations and empirically present our strategy to improve PI-GNN’s performance. Secondly, we formulate and evaluate two strategies to integrate QUBO-Hamiltonian as the generic loss function in Reinforcement learning-based (RL) frameworks. The major contribution of our work lies in understanding the feasibility and quality of the QUBO-based generic reward function in an unsupervised RL setup in addressing graph-based CO problems. Empirically, through our empirical evaluation (Our implementation can be found inhttps://tinyurl.com/5apnymz7), we have observed up to 44% improvement in terms of the number of satisfied constraints over PI-GNN in a representative Max-Cut problem.
Combinatorial optimization is essential across numerous disciplines. Traditional metaheuristics excel at exploring complex solution spaces efficiently, yet they often struggle with scalability. Deep learning has become a viable alternative for quickly generating high-quality solutions, particularly when metaheuristics underperform. In recent years, quantum-inspired approaches such as tensor networks have shown promise in addressing these challenges. Despite these advancements, a thorough comparison of the different paradigms is missing. This study evaluates eight algorithms on Weighted Max-Cut graphs ranging from 10 to 250 nodes. Specifically, we compare a Genetic Algorithm representing metaheuristics, a Graph Neural Network for deep learning, and the Density Matrix Renormalization Group as a tensor network approach. Our analysis focuses on solution quality and computational efficiency (i.e., time and memory usage). Numerical results show that the Genetic Algorithm achieves near-optimal results for small graphs, although its computation time grows significantly with problem size. The Graph Neural Network offers a balanced solution for medium-sized instances with low memory demands and rapid inference, yet it exhibits more significant variability on larger graphs. Meanwhile, the Tensor Network approach consistently yields high approximation ratios and efficient execution on larger graphs, albeit with increased memory consumption.
Many complex problems encountered in both production and daily life can be conceptualized as combinatorial optimization problems (COPs). Many ad-hoc deep learning methods have been proposed to solve these problems, but there still lacks an effective unified framework. In this work, we take a step towards this goal by designing an unified end-to-end deep reinforcement learning framework named CONQUER. CONQUER first translates various COPs into the unified QUBO (Quadratic Unconstrained Binary Optimization) formalization, and afterwards trains a neural QUBO solver to search high-quality strategies, which are then checked and repaired to obtain the solutions for original COPs. The QUBO solver is a purely data-driven neural model, which does not rely on any expert inputs. We adopt the graph transformer network to represent the QUBO graph, so as to effectively capture the features of problem objective and constraints. The Deep Q-network is utilized to train the QUBO solver to find long-sighted solution strategies. Despite being trained solely on small synthetic graphs, CONQUER exhibits promising generality on much larger instances, and can be easily adapted to real-world datasets. Experimentally, we show that CONQUER achieves the state-of-the-art performances on four COPs over graphs, including minimum vertex cover, maximum independent set, maximum clique and maximum cut problem.