As a variety of quantum computing models and platforms become available, methods for assessing and comparing the performance of these devices are of increasing interest and importance. Despite being built of the same fundamental computational unit, radically different approaches have emerged for characterizing the performance of qubits in gate-based and quantum annealing computers, limiting and complicating consistent cross-platform comparisons. To fill this gap, this work proposes a single-qubit protocol (Q-RBPN) for measuring some basic performance characteristics of individual qubits in both models of quantum computation. The proposed protocol scales to large quantum computers with thousands of qubits and provides insights into the distribution of qubit properties within a particular hardware device and across families of devices. The efficacy of the Q-RBPN protocol is demonstrated through the analysis of more than 300 gate-based qubits spanning eighteen machines and 2000 annealing-based qubits from one machine, revealing some unexpected differences in qubit performance. Overall, the proposed Q-RBPN protocol provides a new platform-agnostic tool for assessing the performance of a wide range of emerging quantum computing devices.
As a variety of quantum computing models and platforms become available, methods for assessing and comparing the performance of these devices are of increasing interest and importance. Despite being built of the same fundamental computational unit, radically different approaches have emerged for characterizing the performance of qubits in gate-based and quantum annealing computers, limiting and complicating consistent cross-platform comparisons. To fill this gap, this work proposes a single-qubit protocol (Q-RBPN) for measuring some basic performance characteristics of individual qubits in both models of quantum computation. The proposed protocol scales to large quantum computers with thousands of qubits and provides insights into the distribution of qubit properties within a particular hardware device and across families of devices. The efficacy of the Q-RBPN protocol is demonstrated through the analysis of more than 300 gate-based qubits spanning eighteen machines and 2000 annealing-based qubits from one machine, revealing some unexpected differences in qubit performance. Overall, the proposed Q-RBPN protocol provides a new platform-agnostic tool for assessing the performance of a wide range of emerging quantum computing devices.
As a wide variety of quantum computing platforms become available, methods for assessing and comparing the performance of these devices are of increasing interest and importance. Inspired by the success of single-qubit error rate computations for tracking the progress of gate-based quantum computers, this work proposes a quantum annealing single-qubit assessment (QASA) protocol for quantifying the performance of individual qubits in quantum annealing computers. The proposed protocol scales to large quantum annealers with thousands of qubits and provides unique insights into the distribution of qubit properties within a particular hardware device. The efficacy of the QASA protocol is demonstrated by analyzing the properties of a D-Wave 2000Q system, revealing unanticipated correlations in the qubit performance of that device. A study repeating the QASA protocol at different annealing times highlights how the method can be utilized to understand the impact of annealing parameters on qubit performance. Overall, the proposed QASA protocol provides a useful tool for assessing the performance of current and emerging quantum annealing devices.
As quantum technologies continue to advance, the proliferation of hardware architectures with diverse capabilities and limitations has underscored the importance of benchmarking as a tool to compare performance across platforms. Achieving fair, scalable and consistent evaluations is a key open problem in quantum computing, particularly in the pre-fault-tolerant era. To address this challenge, we introduce QuSquare, a quality-oriented benchmark suite designed to provide a scalable, fair, reproducible, and well-defined framework for assessing the performance of quantum devices across hardware architectures. QuSquare consists of four benchmark tests that evaluate quantum hardware performance at both the system and application levels: Partial Clifford Randomized, Multipartite Entanglement, Transverse Field Ising Model (TFIM) Hamiltonian Simulation, and Data Re-Uploading Quantum Neural Network (QNN). Together, these benchmarks offer an integral, hardware-agnostic, and impartial methodology to quantify the quality and capabilities of current quantum computers, supporting fair cross-platform comparisons and fostering the development of future performance standards.
In the near-term "NISQ"-era of noisy, intermediate-scale, quantum hardware and beyond, reliably determining the quality of quantum devices becomes increasingly important: users need to be able to compare them with one another, and make an estimate whether they are capable of performing a given task ahead of time. In this work, we develop and release an advanced quantum benchmarking framework in order to help assess the state of the art of current quantum devices. Our testing framework measures the performance of universal quantum devices in a hardware-agnostic way, with metrics that are aimed to facilitate an intuitive understanding of which device is likely to outperform others on a given task. This is achieved through six structured tests that allow for an immediate, visual assessment of how devices compare. Each test is designed with scalability in mind, making this framework not only suitable for testing the performance of present-day quantum devices, but also of those released in the foreseeable future. The series of tests are motivated by real-life scenarios, and therefore emphasise the interplay between various relevant characteristics of quantum devices, such as qubit count, connectivity, and gate and measurement fidelity. We present the benchmark results of twenty-one different quantum devices from IBM, Rigetti and IonQ.
The fragmented landscape of quantum computer benchmarks, characterized by system-specific tools and inconsistent evaluation methodologies, hinders reliable cross-platform performance assessment. We introduce Metriq, an open-source collaborative platform for reproducible cross-platform quantum benchmarking that integrates benchmark definition and execution, data collection, and public presentation into a unified workflow. The Metriq benchmark suite spans both system-level metrics that characterize fundamental device properties such as entanglement quality, gate performance, and circuit speed, as well as application-inspired protocols that assess performance on quantum machine learning, optimization, and quantum simulation tasks. Benchmarks are chosen to scale with processor size, and the framework incorporates cost and resource estimation to support practical evaluation. Using Metriq, we collect and publicly release results from more than ten quantum computers across multiple hardware vendors, enabling systematic cross-platform comparison. The resulting curated dataset also reveals the practical strengths and limitations of individual benchmarks, creating a feedback loop that informs the ongoing refinement of the suite. To summarize performance across the benchmark suite, we introduce the Metriq Score, a composite index aggregating benchmark outcomes. We further present cross-benchmark analyses enabled by the shared dataset and their correlations with hardware calibration metrics. Through open development and data sharing, Metriq provides a practical foundation for reproducible benchmarking of quantum computers as hardware and benchmarking methods continue to evolve.
Quantum computing hardware is advancing rapidly toward utility-scale machines that will enable scientific breakthroughs. Many teams are pursuing distinct and difficult-to-compare routes to this goal, using different qubit technologies and logical architectures. Tracking progress toward quantum utility therefore requires rigorous benchmarks that measure computational capability relative to utility-scale challenge problems and enable fair comparison across disparate platforms. Here we demonstrate direct, cross-platform measurement of quantum computational capability using a new benchmark that quantifies the size of the largest computationally relevant quantum circuits that a machine can execute successfully and the speed at which it can execute them. We apply this quantum universal operation performance system (QUOPS) experimentally to leading processors from Quantinuum, Google, and IBM, computing directly on physical qubits. Translating state-of-the-art resource requirements for recognized challenge problems that represent useful quantum computation into effective QUOPS circuit sizes shows that computational capability must grow by 5 orders of magnitude, motivating fault-tolerant approaches. We use the same benchmark to assess the performance of a simple fault-tolerant logical-qubit processor implemented on up to eight [[7,1,3]]-encoded logical qubits using Quantinuum Helios-1, and project the growth of capability across successive generations of fault-tolerant quantum computers to show how QUOPS can track progress toward quantum scientific utility.
Large-scale quantum computers are expected to benefit from modular architectures. Validating the capabilities of modular devices requires benchmarking strategies that assess performance within and between modules. In this work, we evaluate cross-platform verification protocols, which are critical for quantifying how accurately different modules prepare the same quantum state—a key requirement for modular scalability and system-wide consistency. We demonstrate these algorithms using a six-qubit flip-chip superconducting quantum device consisting of two 3-qubit modules on a single carrier chip, with connectivity for intra- and intermodule entanglement. We examine how the resource requirements of protocols relying solely on classical communication between modules scale exponentially with qubit number, and demonstrate that introducing an intermodule two-qubit gate enables subexponential scaling in cross-platform verification. This approach reduces the number of repetitions required by a factor of 4 for three-qubit states, with greater reductions projected for larger and higher-fidelity devices.