Existing protocols for benchmarking current quantum coprocessors fail to meet the usual standards for assessing the performance of high-performance-computing platforms. After a synthetic review of these protocols—whether at the gate, circuit, or application level—we introduce a new benchmark, dubbed Atos Q-score, which is application-centric, hardware-agnostic, and scalable to quantum advantage processor sizes and beyond. The Q-score measures the maximum number of qubits that can be usedeffectivelyto solve the MaxCut combinatorial optimization problem with the quantum approximate optimization algorithm. We give a robust definition of the notion of effective performance by introducing an improved approximation ratio based on the scaling of random and optimal algorithms. We illustrate the behavior of Q-score using perfect and noisy simulations of quantum processors. Finally, we provide an open-source implementation of Q-score that makes it easy to compute the Q-score of any quantum hardware.
Existing protocols for benchmarking current quantum coprocessors fail to meet the usual standards for assessing the performance of high-performance-computing platforms. After a synthetic review of these protocols—whether at the gate, circuit, or application level—we introduce a new benchmark, dubbed Atos Q-score, which is application-centric, hardware-agnostic, and scalable to quantum advantage processor sizes and beyond. The Q-score measures the maximum number of qubits that can be usedeffectivelyto solve the MaxCut combinatorial optimization problem with the quantum approximate optimization algorithm. We give a robust definition of the notion of effective performance by introducing an improved approximation ratio based on the scaling of random and optimal algorithms. We illustrate the behavior of Q-score using perfect and noisy simulations of quantum processors. Finally, we provide an open-source implementation of Q-score that makes it easy to compute the Q-score of any quantum hardware.
This paper presents the benchmark score definitions of QPack, an application-oriented cross-platform benchmarking suite for quantum computers and simulators, which makes use of scalable Quantum Approximate Optimization Algorithm and Variational Quantum Eigensolver applications. Using a varied set of benchmark applications, an insight of how well a quantum computer or its simulator performs on a general NISQ-era application can be quantitatively made. This paper presents what quantum execution data can be collected and transformed into benchmark scores for application-oriented quantum benchmarking. Definitions are given for an overall benchmark score, as well as sub-scores based on runtime, accuracy, scalability and capacity performance. Using these scores, a comparison is made between various quantum computer simulators, running both locally and on vendors' remote cloud services. We also use the QPack benchmark to collect a small set of quantum execution data of the IBMQ Nairobi quantum processor. The goal of the QPack benchmark scores is to give a holistic insight into quantum performance and the ability to make easy and quick comparisons between different quantum computers
In recent years, neutral atom-based quantum computation has been established as a competing alternative for the realization of fault-tolerant quantum computation. However, as with other quantum technologies, various sources of noise limit their performance. With processors continuing to scale up, new techniques are needed to characterize and compare them in order to track their progress. In this work, we present two systematic benchmarks that evaluate these quantum processors at scale. We use the quantum adiabatic algorithm (QAA) and the quantum approximate optimization algorithm (QAOA) to solve maximal independent set (MIS) instances of random unit-disk graphs. These benchmarks are scalable, relying not on prior knowledge of the system's evolution but on the quality of the MIS solutions obtained. Rather than isolating individual sources of noise, they provide an application-level figure of merit: improvements in a QPU and its implementation of the protocols should be reflected in higher-quality MIS solutions. We benchmark quera_aquila and pasqal_fresnel on problem sizes up to 102 and 85 qubits, respectively. Overall, quera_aquila performs better on QAOA and QAA instances. Finally, we generate MIS instances of up to 1000 qubits, providing scalable benchmarks for evaluating future, larger processors as they become available. The proposed protocols can therefore serve as a common reference for comparisons, allowing future work to assess whether advances in hardware translate into measurable improvements in MIS solution quality.
Quantum computers have now surpassed classical simulation limits, yet noise continues to limit their practical utility. As the field shifts from proof-of-principle demonstrations to early deployments, there is no standard method for meaningfully and scalably comparing heterogeneous quantum hardware. Existing benchmarks typically focus on gate-level fidelity or constant-depth circuits, offering limited insight into algorithmic performance at depth. Here we introduce a benchmarking protocol based on the linear ramp quantum approximate optimization algorithm (LR-QAOA), a fixed-parameter, deterministic variant of QAOA. LR-QAOA quantifies a QPU's ability to preserve a coherent signal as circuit depth increases, identifying when performance becomes statistically indistinguishable from random sampling. We apply this protocol to 24 quantum processors from six vendors, testing problems with up to 156 qubits and 10,000 layers across 1D-chains, native layouts, and fully connected topologies. This constitutes the most extensive cross-platform quantum benchmarking effort to date, with circuits reaching a million two-qubit gates. LR-QAOA offers a scalable, unified benchmark across platforms and architectures, making it a tool for tracking performance in quantum computing.
As quantum processors continue to scale in size and complexity, the need for well-defined, reproducible, and technology-agnostic performance metrics becomes increasingly critical. Here we present a suite of scalable quantum computing benchmarks developed as key performance indicators (KPIs) within the EU Quantum Flagship. These proposed benchmarks are designed to assess holistic system performance rather than isolated components, and to remain applicable across both noisy intermediate-scale quantum (NISQ) devices and future fault-tolerant architectures. We introduce four core benchmarks addressing complementary aspects of quantum computing capability: large multi-qubit circuit execution via a Clifford Volume benchmark, scalable multipartite entanglement generation through GHZ-state preparation, a benchmark based on the application of Shor's period-finding subroutine to simple functions, and a protocol quantifying the benefit of quantum error correction using Bell states. Each benchmark is accompanied by clearly specified protocols, reporting standards, and scalable evaluation methods. Together, these KPIs provide a coherent framework for transparent and fair performance assessment across quantum hardware platforms and for tracking progress late-NISQ toward early fault-tolerant quantum computation.
In this paper, we present QPack, a universal benchmark for Noisy Intermediate-Scale Quantum (NISQ) computers based on Quantum Approximate Optimization Algorithms (QAOA). Unlike other evaluation metrics in the field, this benchmark evaluates not only one, but multiple important aspects of quantum computing hardware: the maximum problem size a quantum computer can solve, the required runtime, as well as the achieved accuracy. The applications MaxCut, dominating set and traveling salesman are included to provide variation in resource requirements. This will allow for a diverse benchmark that promotes optimal design considerations, avoiding hardware implementations for specific applications. We also discuss the design aspects that are taken in consideration for the QPack benchmark, with critical quantum benchmark requirements in mind. An implementation is presented, providing practical metrics. QPack is presented as a hardware agnostic benchmark by making use of the XACC library. We demonstrate the application of the benchmark on various IBM machines, as well as a range of simulators.
Quantum computing technologies are undergoing rapid development. The different qubit modalities being considered for quantum computing each have their strengths and weaknesses, making it challenging to compare their performance relative to each other and the state-of-the-art in classical high-performance computing. To better understand the utility of a given quantum processor and to assess when and how it will be able to advance the frontiers of computational science, researchers need a robust approach to quantum benchmarking. A variety of approaches have been proposed, many of which characterize the presence of noise in current quantum devices. These efforts include component-level performance metrics, such as randomized benchmarking and gate set tomography; high-level application-dependent metrics; and devicelevel metrics, such as the Quantum Volume. However, it remains unclear how low-level metrics, such as fidelities and decoherence times, and global device metrics, such as Quantum Volume, relate to the computational utility and practical limitations of quantum processors to solve useful problems. In this paper, we describe our Hamiltonian-oriented approach to quantum benchmarking called HamPerf. Where previous application-dependent approaches specify a suite of benchmarking circuits inspired by applications, we place the problem Hamiltonian at the center. Our strategy allows us to probe the computational performance of a quantum processor on standardized and relevant problem sets, agnostic of the algorithms and hardware used to solve them; it also provides fundamental insights into how device characteristics correlate with computational utility.