We introduce universal, easy-to-reproduce generative models for the QUBO instances to differentiate the performance of the hardware/solvers effectively. Our benchmark process extends the well-known Hebb's rule of associative memory with the asymmetric pattern weights. We provide a comprehensive overview of calculations conducted across various scales and using different classes of dynamical equations. Our aim is to analyze their results, including factors such as the probability of encountering the ground state, planted state, spurious state, or states falling outside the predetermined energy range. Moreover, the generated problems show additional properties, such as the easy-hard-easy complexity transition and complicated cluster structures of planted solutions. Our method establishes a prospective platform to potentially address other questions related to the fundamental principles behind device physics and algorithms for novel computing machines.
We introduce universal, easy-to-reproduce generative models for the QUBO instances to differentiate the performance of the hardware/solvers effectively. Our benchmark process extends the well-known Hebb's rule of associative memory with the asymmetric pattern weights. We provide a comprehensive overview of calculations conducted across various scales and using different classes of dynamical equations. Our aim is to analyze their results, including factors such as the probability of encountering the ground state, planted state, spurious state, or states falling outside the predetermined energy range. Moreover, the generated problems show additional properties, such as the easy-hard-easy complexity transition and complicated cluster structures of planted solutions. Our method establishes a prospective platform to potentially address other questions related to the fundamental principles behind device physics and algorithms for novel computing machines.
This research focuses on developing effective benchmarks for quadratic unconstrained binary optimization instances, crucial for evaluating the performance of Ising hardware and solvers. Currently, the field lacks accessible and reproducible models for systematically testing such systems, particularly in terms of detailed phase space characterization. Here, we introduce universal generative models based on an extension of Hebb’s rule of associative memory with asymmetric pattern weights. We conduct comprehensive calculations across different scales and dynamical equations, examining outcomes like the probabilities of reaching the ground state, planted state, spurious state, or other energy levels. Additionally, the generated problems reveal properties such as the easy-hard-easy complexity transition and complex solution cluster structures. This method offers a promising platform for analyzing and understanding the behavior of physical hardware and its simulations, contributing to future advancements in optimization technologies. The authors present a method for creating generative models that produce test instances for quadratic unconstrained binary optimization (QUBO) problems, inspired by Hebb’s rule. Their approach identifies key solution patterns and complexity transitions, providing a platform to evaluate the performance of Ising hardware and solvers efficiently.
We are interested in benchmarking both quantum annealing and classical algorithms for minimizing quadratic unconstrained binary optimization (QUBO) problems. Such problems are NP-hard in general, implying that the exact minima of randomly generated instances are hard to find and thus typically unknown. While brute forcing smaller instances is possible, such instances are typically not interesting due to being too easy for both quantum and classical algorithms. In this contribution, we propose a novel method, called posiform planting, for generating random QUBO instances of arbitrary size with known optimal solutions, and use those instances to benchmark the sampling quality of four D-Wave quantum annealers utilizing different interconnection structures (Chimera, Pegasus, and Zephyr hardware graphs) and the simulated annealing algorithm. Posiform planting differs from many existing methods in two key ways. It ensures the uniqueness of the planted optimal solution, thus avoiding groundstate degeneracy, and it enables the generation of QUBOs that are tailored to a given hardware connectivity structure, provided that the connectivity is not too sparse. Posiform planted QUBOs are a type of 2-SAT boolean satisfiability combinatorial optimization problems. Our experiments demonstrate the capability of the D-Wave quantum annealers to sample the optimal planted solution of combinatorial optimization problems with up to 5, 627 qubits.
Abstract Posiform planting is a method for constructing QUBO instances with a unique planted solution that can be tailored to arbitrary connectivity graphs. In this study we investigate making posiform planted QUBOs computationally harder by fusing many smaller random Ising models, whose global minimum is computed classically, with posiform planted QUBOs. The unique ground state of the resulting QUBO is the concatenation of (exactly one of) the ground states of each smaller problem. Our method generates QUBO instances that have a unique solution, are native to the hardware graph, and have tunable computational hardness. We use our QUBOs to benchmark three D-Wave quantum annealing processors (with 563–5627 qubits), and compare them against simulated annealing and Gurobi. Surprisingly, we find that the D-Wave ground state sampling success rate is not dependent on the glued random QUBO size, and that some QUBO classes are solved at high success rates at short annealing times on the Zephyr processors.
Recent years have witnessed the flourishing of experimental $I\phantom{\rule{0}{0ex}}s\phantom{\rule{0}{0ex}}i\phantom{\rule{0}{0ex}}n\phantom{\rule{0}{0ex}}g$ $m\phantom{\rule{0}{0ex}}a\phantom{\rule{0}{0ex}}c\phantom{\rule{0}{0ex}}h\phantom{\rule{0}{0ex}}i\phantom{\rule{0}{0ex}}n\phantom{\rule{0}{0ex}}e\phantom{\rule{0}{0ex}}s$, special-purpose computational devices that promise to solve the world's toughest optimization problems in record times. Evaluating an Ising machine's performance is problematic, though, as it poses two seemingly contradictory requirements: On the one hand, the generated problem instances should be hard to solve, yet on the other hand they should have verifiable solutions. This study provides a methodology for generating random optimization-problem sets from linear systems of equations that possess both desired properties, thereby allowing direct, unbiased benchmarking of these physical optimization devices.
Abstract With current semiconductor technology reaching its physical limits, special-purpose hardware has emerged as an option to tackle specific computing-intensive challenges. Optimization in the form of solving quadratic unconstrained binary optimization problems, or equivalently Ising spin glasses, has been the focus of several new dedicated hardware platforms. These platforms come in many different flavors, from highly-efficient hardware implementations on digital-logic of established algorithms to proposals of analog hardware implementing new algorithms. In this work, we use a mapping of a specific class of linear equations whose solutions can be found efficiently, to a hard constraint satisfaction problem (three-regular three-XORSAT, or an Ising spin glass) with a ‘golf-course’ shaped energy landscape, to benchmark several of these different approaches. We perform a scaling and prefactor analysis of the performance of Fujitsu’s digital annealer unit (DAU), the D-Wave advantage quantum annealer, a virtual MemComputing machine, Toshiba’s simulated bifurcation machine (SBM), the SATonGPU algorithm from Bernaschi et al , and our implementation of parallel tempering. We identify the SATonGPU and DAU as currently having the smallest scaling exponent for this benchmark, with SATonGPU having a small scaling advantage and in addition having by far the smallest prefactor thanks to its use of massive parallelism. Our work provides an objective assessment and a snapshot of the promise and limitations of dedicated optimization hardware relative to a particular class of optimization problems.
Recent progress has brought benchmarking of (heuristic) quantum algorithms at scale within reach. Particularly in combinatorial optimization, it is key to empirically analyze and track progress towards quantum advantage. This work introduces a systematic, fair and comparable benchmarking framework for quantum optimization methods by presenting ten model-independent problem classes that are challenging for classical methods. Track records of specific instances and solutions are given in an accompanying open-source repository. While the individual properties of the problem classes vary, they all become challenging from less than 100 to, at most, an order of 100,000 decision variables. We reference results from state-of-the-art solvers for instances across all problem classes and demonstrate exemplary baseline results obtained with quantum solvers for selected problems, which illustrate standardized benchmark reporting. The presented problem instances may be approached with classical or quantum algorithms executed on varying hardware platforms to drive the field towards quantum advantage.