We present a method for characterizing the performance of noisy quantum processors using discrete time crystals. Deviations from ideal persistent oscillatory behavior give rise to numerical scores by which relative quantum processor capabilities can be measured. We construct small sets of qubit layouts that cover the full topology of a target system, and execute our metric over these sets on a wide range of IBM Quantum processors. We show that there is a large variability in scores, not only across multiple processors, but between different circuit layouts over individual devices. The stability of results is also examined. Our results suggest that capturing the true performance characteristics of a quantum system requires interrogation over the full device, rather than isolated subgraphs. Moreover, the disagreement between our results and other metrics indicates that benchmarks computed infrequently are not indicative of the real-world performance of a quantum processor. This method is platform agnostic, simple to implement, and scalable to any number of qubits forming a linear-chain, while simultaneously allowing for identifying ill-performing regions of a device at the individual qubit level.
We present a method for characterizing the performance of noisy quantum processors using discrete time crystals. Deviations from ideal persistent oscillatory behavior give rise to numerical scores by which relative quantum processor capabilities can be measured. We construct small sets of qubit layouts that cover the full topology of a target system, and execute our metric over these sets on a wide range of IBM Quantum processors. We show that there is a large variability in scores, not only across multiple processors, but between different circuit layouts over individual devices. The stability of results is also examined. Our results suggest that capturing the true performance characteristics of a quantum system requires interrogation over the full device, rather than isolated subgraphs. Moreover, the disagreement between our results and other metrics indicates that benchmarks computed infrequently are not indicative of the real-world performance of a quantum processor. This method is platform agnostic, simple to implement, and scalable to any number of qubits forming a linear-chain, while simultaneously allowing for identifying ill-performing regions of a device at the individual qubit level.
As quantum processors grow, new performance benchmarks are required to capture the full quality of the devices at scale. While quantum volume is an excellent benchmark, it focuses on the highest quality subset of the device and so is unable to indicate the average performance over a large number of connected qubits. Furthermore, it is a discrete pass/fail and so is not reflective of continuous improvements in hardware nor does it provide quantitative direction to large-scale algorithms. For example, there may be value in error mitigated Hamiltonian simulation at scale with devices unable to pass strict quantum volume tests. Here we discuss a scalable benchmark which measures the fidelity of a connecting set of two-qubit gates over N qubits by measuring gate errors using simultaneous direct randomized benchmarking in disjoint layers. Our layer fidelity can be easily related to algorithmic run time, via γ defined in Ref.\cite{berg2022probabilistic} that can be used to estimate the number of circuits required for error mitigation. The protocol is efficient and obtains all the pair rates in the layered structure. Compared to regular (isolated) RB this approach is sensitive to crosstalk. As an example we measure a N=80(100) qubit layer fidelity on a 127 qubit fixed-coupling "Eagle" processor (ibm\_sherbrooke) of 0.26(0.19) and on the 133 qubit tunable-coupling "Heron" processor (ibm\_montecarlo) of 0.61(0.26). This can easily be expressed as a layer size independent quantity, error per layered gate (EPLG), which is here 1.7×10−2(1.7×10−2) for ibm\_sherbrooke and 6.2×10−3(1.2×10−2) for ibm\_montecarlo.
As quantum processors continue to scale in size and complexity, the need for well-defined, reproducible, and technology-agnostic performance metrics becomes increasingly critical. Here we present a suite of scalable quantum computing benchmarks developed as key performance indicators (KPIs) within the EU Quantum Flagship. These proposed benchmarks are designed to assess holistic system performance rather than isolated components, and to remain applicable across both noisy intermediate-scale quantum (NISQ) devices and future fault-tolerant architectures. We introduce four core benchmarks addressing complementary aspects of quantum computing capability: large multi-qubit circuit execution via a Clifford Volume benchmark, scalable multipartite entanglement generation through GHZ-state preparation, a benchmark based on the application of Shor's period-finding subroutine to simple functions, and a protocol quantifying the benefit of quantum error correction using Bell states. Each benchmark is accompanied by clearly specified protocols, reporting standards, and scalable evaluation methods. Together, these KPIs provide a coherent framework for transparent and fair performance assessment across quantum hardware platforms and for tracking progress late-NISQ toward early fault-tolerant quantum computation.
Existing protocols for benchmarking current quantum coprocessors fail to meet the usual standards for assessing the performance of high-performance-computing platforms. After a synthetic review of these protocols—whether at the gate, circuit, or application level—we introduce a new benchmark, dubbed Atos Q-score, which is application-centric, hardware-agnostic, and scalable to quantum advantage processor sizes and beyond. The Q-score measures the maximum number of qubits that can be usedeffectivelyto solve the MaxCut combinatorial optimization problem with the quantum approximate optimization algorithm. We give a robust definition of the notion of effective performance by introducing an improved approximation ratio based on the scaling of random and optimal algorithms. We illustrate the behavior of Q-score using perfect and noisy simulations of quantum processors. Finally, we provide an open-source implementation of Q-score that makes it easy to compute the Q-score of any quantum hardware.
Quantum computing technologies are undergoing rapid development. The different qubit modalities being considered for quantum computing each have their strengths and weaknesses, making it challenging to compare their performance relative to each other and the state-of-the-art in classical high-performance computing. To better understand the utility of a given quantum processor and to assess when and how it will be able to advance the frontiers of computational science, researchers need a robust approach to quantum benchmarking. A variety of approaches have been proposed, many of which characterize the presence of noise in current quantum devices. These efforts include component-level performance metrics, such as randomized benchmarking and gate set tomography; high-level application-dependent metrics; and devicelevel metrics, such as the Quantum Volume. However, it remains unclear how low-level metrics, such as fidelities and decoherence times, and global device metrics, such as Quantum Volume, relate to the computational utility and practical limitations of quantum processors to solve useful problems. In this paper, we describe our Hamiltonian-oriented approach to quantum benchmarking called HamPerf. Where previous application-dependent approaches specify a suite of benchmarking circuits inspired by applications, we place the problem Hamiltonian at the center. Our strategy allows us to probe the computational performance of a quantum processor on standardized and relevant problem sets, agnostic of the algorithms and hardware used to solve them; it also provides fundamental insights into how device characteristics correlate with computational utility.
Quantum processors are now able to run quantum circuits that are infeasible to simulate classically, creating a need for benchmarks that assess a quantum processor's rate of errors when running these circuits. Here, we introduce a general technique for creating efficient benchmarks from any set of quantum computations, specified by unitary circuits. Our benchmarks assess the integrated performance of a quantum processor's classical compilation algorithms and its low-level quantum operations. Unlike existing “full-stack benchmarks”, our benchmarks do not require classical simulations of quantum circuits, and they use only efficient classical computations. We use our method to create randomized circuit benchmarks, including a computationally efficient version of the quantum volume benchmark, and an algorithm-based benchmark that uses Hamiltonian simulation circuits. We perform these benchmarks on IBM Q devices and in simulations, and we compare their results to the results of existing benchmarking methods.
In the near-term "NISQ"-era of noisy, intermediate-scale, quantum hardware and beyond, reliably determining the quality of quantum devices becomes increasingly important: users need to be able to compare them with one another, and make an estimate whether they are capable of performing a given task ahead of time. In this work, we develop and release an advanced quantum benchmarking framework in order to help assess the state of the art of current quantum devices. Our testing framework measures the performance of universal quantum devices in a hardware-agnostic way, with metrics that are aimed to facilitate an intuitive understanding of which device is likely to outperform others on a given task. This is achieved through six structured tests that allow for an immediate, visual assessment of how devices compare. Each test is designed with scalability in mind, making this framework not only suitable for testing the performance of present-day quantum devices, but also of those released in the foreseeable future. The series of tests are motivated by real-life scenarios, and therefore emphasise the interplay between various relevant characteristics of quantum devices, such as qubit count, connectivity, and gate and measurement fidelity. We present the benchmark results of twenty-one different quantum devices from IBM, Rigetti and IonQ.