There are two common ways to evaluate algorithms: performance on benchmark problems derived from real applications and analysis of performance on parametrized families of problems. The two approaches complement each other, each having its advantages and disadvantages. The planning community has concentrated on the first approach, with few ways of generating parametrized families of hard problems known prior to this work. Our group's main interest is in comparing approaches to solving planning problems using a novel type of computational device - a quantum annealer - to existing state-of-the-art planning algorithms. Because only small-scale quantum annealers are available, we must compare on small problem sizes. Small problems are primarily useful for comparison only if they are instances of parametrized families of problems for which scaling analysis can be done. In this technical report, we discuss our approach to the generation of hard planning problems from classes of well-studied NP-complete problems that map naturally to planning problems or to aspects of planning problems that many practical planning problems share. These problem classes exhibit a phase transition between easy-to-solve and easy-to-show-unsolvable planning problems. The parametrized families of hard planning problems lie at the phase transition. The exponential scaling of hardness with problem size is apparent in these families even at very small problem sizes, thus enabling us to characterize even very small problems as hard. The families we developed will prove generally useful to the planning community in analyzing the performance of planning algorithms, providing a complementary approach to existing evaluation methods. We illustrate the hardness of these problems and their scaling with results on four state-of-the-art planners, observing significant differences between these planners on these problem families. Finally, we describe two general, and quite different, mappings of planning problems to QUBOs, the form of input required for a quantum annealing machine such as the D-Wave II.
There are two common ways to evaluate algorithms: performance on benchmark problems derived from real applications and analysis of performance on parametrized families of problems. The two approaches complement each other, each having its advantages and disadvantages. The planning community has concentrated on the first approach, with few ways of generating parametrized families of hard problems known prior to this work. Our group's main interest is in comparing approaches to solving planning problems using a novel type of computational device - a quantum annealer - to existing state-of-the-art planning algorithms. Because only small-scale quantum annealers are available, we must compare on small problem sizes. Small problems are primarily useful for comparison only if they are instances of parametrized families of problems for which scaling analysis can be done. In this technical report, we discuss our approach to the generation of hard planning problems from classes of well-studied NP-complete problems that map naturally to planning problems or to aspects of planning problems that many practical planning problems share. These problem classes exhibit a phase transition between easy-to-solve and easy-to-show-unsolvable planning problems. The parametrized families of hard planning problems lie at the phase transition. The exponential scaling of hardness with problem size is apparent in these families even at very small problem sizes, thus enabling us to characterize even very small problems as hard. The families we developed will prove generally useful to the planning community in analyzing the performance of planning algorithms, providing a complementary approach to existing evaluation methods. We illustrate the hardness of these problems and their scaling with results on four state-of-the-art planners, observing significant differences between these planners on these problem families. Finally, we describe two general, and quite different, mappings of planning problems to QUBOs, the form of input required for a quantum annealing machine such as the D-Wave II.
Abstract One approach to solving planning problems is to compile them to other problems for which powerful off-the-shelf solvers are available; common targets include SAT, CSP, and MILP. Recently, a novel optimization technique has become available: quantum annealing (QA). QA takes as input problem instances of quadratic unconstrained binary optimization (QUBO) problem. Early quantum annealers are now available, though their constraints restrict the types of QUBOs they can take as input. Here, we introduce the planning community to the key steps in compiling planning problems to QA hardware: a hardware-independent step, mapping, and a hardware-dependent step, embedding. After describing two approaches to mapping general planning problems to QUBO, we describe preliminary results from running an early quantum annealer on a parametrized family of hard planning problems. The results show that different mappings can substantially affect performance, even when many features of the resulting instances are similar. We conclude with insights gained from this early study that suggest directions for future work.
Abstract : One approach to solving planning problems is to compile them to another problem for which powerful off-the-shelf solvers are available; common targets include SAT, CSP, and MILP. Recently, a novel optimization technique has become available: quantum annealing (QA). QA takes as input problem instances encoded as Quadratic Unconstrained Binary Optimization (QUBO). Early quantum annealers are now available, and more sophisticated quantum annealers will likely be built over the next decades. Specific quantum annealing hardware implementations have specific constraints, restricting the types of QUBOs each can take as input. In this paper, we introduce the planning community to the key steps involved in compiling planning problems to quantum annealing hardware: a hardware-independent step, mapping, and a hardware-dependent step, embedding. After describing two approaches to mapping general planning problems to QUBO, we provide preliminary results from running an early quantum annealer on a parameterized family of hard planning problems. The results show that different mappings can lead to a substantial difference in performance, even when many superficial features of the resulting instances are similar. We also provide some insights gained from this early study, and suggest directions for future work.
Abstract We compare the performance of four quantum annealers, the D-Wave Two, 2X, 2000Q, and Advantage in solving an identical ensemble of a parametrized family of scheduling problems. These problems are NP-complete and, in fact, equivalent to vertex coloring problems. They are also practically motivated and closely connected to planning problems from artificial intelligence. We examine factors contributing to the performance differences while separating the contributions from hardware upgrades, support for shorter anneal times, and possible optimization of ferromagnetic couplings. While shorter anneal times can improve the time to solution (TTS) at any given problem size, the scaling of TTS with respect to the problem size worsens for shorter anneal times. In contrast, optimizing the ferromagnetic coupling improves both the absolute TTS and the scaling. There is a statistically significant improvement in performance between D-Wave Two and 2X and from all older generation annealers to Advantage, even when operated under identical anneal time and ferromagnetic couplings. However, the performance improvement from 2X to 2000Q requires the anneal time and ferromagnetic couplings to be optimized. Overall, owing to these inter-generational hardware improvements and optimizations, the scaling exponent reduces from 1.01±0.01 1.01 ± 0.01 on Two to 0.259±0.008 0.259 ± 0.008 on Advantage.
Quantum annealing is a promising technique to solve NP-hard problems more efficiently than state-of-the-art classical computing algorithms. Current generation quantum annealers require users to formulate problems following specific models: either as an Ising Hamiltonian, or as a quadratic unconstrained binary optimization (QUBO) polynomial. In this paper, we show that a problem may be formulated in multiple, fundamentally different ways using these models, and that the choice of problem formulation drastically impacts the scope of problem instances that the quantum annealer is able to solve. Using Boolean satisfiability (SAT) solving as an example, we investigate which characteristics of problem formulation are determinant in solution viability. Drawing inspiration from classical algorithmic asymptotic analysis, we establish a framework for measuring the complexity of any given formulation using the QUBO or Ising models, which we argue can be useful for practitioners to evaluate and compare the different ways one may tackle a problem using quantum annealers.
The beginning of the quantum era has increased the hype around computationally demanding problems and, particularly, NP-complete problems. One of the most famous is the Set Packing, which has several applications in different management contexts. This work presents a solver for Set Packing Problem on quantum annealers exploiting the QUBO model. We introduce a general format to express instances of the Set Packing Problem that enables an automatic characterization, lowering the barriers to adopting quantum annealers. Finally, we compare the performance of two quantum annealer on the Set Packing, namely D-Wave 2000Q and Advantage, highlighting their pros and cons. As key evaluation indicators, this work uses spatial features, such as the number of qubits and maximum chain length, and the execution time required for a reliable problem resolution and its breakdown. In particular, we can scale the solutions to 60 and 150 subsets on 2000Q and Advantage, respectively, with 0 chain breaks. Moreover, our analysis illustrates that 2000Q has lower mean energy required but higher execution times on the same problem complexity.
In this thesis, we focus on the problem of validating and benchmarking quantum annealers. To this end, we propose two algorithms for solving real-world problems and test how they perform on the current generation of quantum annealers. The first algorithm allows for solving the dynamics of quantum systems (or, in fact, any dynamical systems). The second of the proposed algorithms is suitable for solving a particular family of railway dispatching problems. We assess the performance of those algorithms on the current generation of D-Wave quantum annealers with the assistance of two novel, classical strategies for solving an Ising model also presented in the thesis. The first, tensor network-based approach is a heuristic algorithm tailored for solving instances defined on Chimera-like graphs, thus making it ideal for providing a baseline with which the results from physical annealers can be compared. The other presented approach is a massively parallel implementation of the exhaustive (brute-force) search through the whole solution space. Although the brute-force approach is limited to moderate instance sizes, it has the advantage of being able to compute the low energy spectrum and certify the solutions. Our results suggest that present-day quantum annealers are able to solve a subset of the aforementioned problems. In particular, we show that the D-Wave annealers are capable of capturing the dynamics of a simple quantum system in a specific regime of parameters, and can be used to obtain good-quality solutions for instances of railway conflict management problems. Finally, our findings indicate that the current generation of D-Wave annealers is far from perfect. We discuss problem instances for which the annealers failed to find a good or even feasible solution. We also provide, where possible, a plausible explanation of why some of the presented problems might be hard for the annealers.