Quantum processors with sizes in the 10-100 qubit range are now increasingly common. However, with increased size comes increased complexity for benchmarking. The effectiveness of a given device may vary greatly between different tasks, and will not always be easy to predict from single and two qubit gate fidelities. For this reason, it is important to assess processor quality for a range of important tasks. In this work we propose and implement tests based on random quantum circuits. These are used to evaluate multiple different superconducting qubit devices, with sizes from 5 to 19 qubits, from two hardware manufacturers: IBM Research and Rigetti. The data is analyzed to give a quantitive description of how the devices perform. We also describe how it can be used for a qualititive description accessible to the layperson, by being played as a game.
As quantum processors grow, new performance benchmarks are required to capture the full quality of the devices at scale. While quantum volume is an excellent benchmark, it focuses on the highest quality subset of the device and so is unable to indicate the average performance over a large number of connected qubits. Furthermore, it is a discrete pass/fail and so is not reflective of continuous improvements in hardware nor does it provide quantitative direction to large-scale algorithms. For example, there may be value in error mitigated Hamiltonian simulation at scale with devices unable to pass strict quantum volume tests. Here we discuss a scalable benchmark which measures the fidelity of a connecting set of two-qubit gates over
N qubits by measuring gate errors using simultaneous direct randomized benchmarking in disjoint layers. Our layer fidelity can be easily related to algorithmic run time, via
γ defined in Ref.\cite{berg2022probabilistic} that can be used to estimate the number of circuits required for error mitigation. The protocol is efficient and obtains all the pair rates in the layered structure. Compared to regular (isolated) RB this approach is sensitive to crosstalk. As an example we measure a
N=80 (100) qubit layer fidelity on a 127 qubit fixed-coupling "Eagle" processor (ibm\_sherbrooke) of 0.26(0.19) and on the 133 qubit tunable-coupling "Heron" processor (ibm\_montecarlo) of 0.61(0.26). This can easily be expressed as a layer size independent quantity, error per layered gate (EPLG), which is here
1.7×10−2(1.7×10−2) for ibm\_sherbrooke and
6.2×10−3(1.2×10−2) for ibm\_montecarlo.
In the near-term "NISQ"-era of noisy, intermediate-scale, quantum hardware and beyond, reliably determining the quality of quantum devices becomes increasingly important: users need to be able to compare them with one another, and make an estimate whether they are capable of performing a given task ahead of time. In this work, we develop and release an advanced quantum benchmarking framework in order to help assess the state of the art of current quantum devices. Our testing framework measures the performance of universal quantum devices in a hardware-agnostic way, with metrics that are aimed to facilitate an intuitive understanding of which device is likely to outperform others on a given task. This is achieved through six structured tests that allow for an immediate, visual assessment of how devices compare. Each test is designed with scalability in mind, making this framework not only suitable for testing the performance of present-day quantum devices, but also of those released in the foreseeable future. The series of tests are motivated by real-life scenarios, and therefore emphasise the interplay between various relevant characteristics of quantum devices, such as qubit count, connectivity, and gate and measurement fidelity. We present the benchmark results of twenty-one different quantum devices from IBM, Rigetti and IonQ.
Being able to quantify the level of coherent control in a proposed device implementing a quantum information processor (QIP) is an important task for both comparing different devices and assessing a device's prospects with regards to achieving fault-tolerant quantum control. We implement in a liquid-state nuclear magnetic resonance QIP the randomized benchmarking protocol presented by Knill et al (2008 Phys. Rev. A 77 012307). We report an error per randomized π/2 pulse of 1.3±0.1 × 10 −4 with a single-qubit QIP and show an experimentally relevant error model where the randomized benchmarking gives a signature fidelity decay which is not possible to interpret as a single error per gate. We explore and experimentally investigate multi-qubit extensions of this protocol and report an average error rate for one- and two-qubit gates of 4.7±0.3 × 10 −3 for a three-qubit QIP. We estimate that these error rates are still not decoherence limited and thus can be improved with modifications to the control hardware and software.
Abstract Quantum computers have the potential to outperform classical computers in a range of computational tasks, such as prime factorisation and unstructured searching. However, real-world quantum computers are subject to noise. Quantifying noise is of vital importance, since it is often the dominant factor preventing the successful realisation of advanced quantum computations. Here we propose and demonstrate an interleaved randomised benchmarking protocol for measurement-based quantum computers that can be used to estimate the fidelity of any single-qubit measurement-based gate. We tested the protocol on IBM superconducting quantum processors by estimating the fidelity of the Hadamard and T gates—a universal single-qubit gate set. Measurements were performed on entangled cluster states of up to 31 qubits. Our estimated gate fidelities show good agreement with those calculated from quantum process tomography. By artificially increasing noise, we were able to show that our protocol detects large noise variations in different implementations of a gate.
Despite rapid progress in quantum optimization, broad real-hardware benchmarks comparing multiple algorithmic families across diverse combinatorial problems under a common protocol remain limited. We benchmark gate-based quantum optimization on four NP-hard problems: multi-dimensional knapsack (MDKP), maximum independent set (MIS), quadratic assignment (QAP), and market-share (MSP). We study VQE, CVaR-VQE, standard, multi-angle, and warm-start QAOA, together with qubit-efficient PCE and QRAO, on IBM Heron r1/r2 processors using resilience-level-2 mitigation. To our knowledge, this includes the first real-hardware QRAO results and the first multi-problem PCE hardware benchmark.
Across 247 method-instance combinations, we report transpiled circuit size, hardware outcomes, and an independent-error gate-count fidelity proxy,
Fest. For MDKP and MIS, an empirical operating point near
Fest≈0.1, corresponding to about 770 two-qubit gates at the median Heron-r2 CZ error rate, marks the onset of noise-dominated execution. QAP exposes a separate bottleneck: dense one-hot encodings and an exponentially sparse feasible manifold, with feasible fraction
10!/2100 at
n=10; no tested hardware method produces a feasible assignment. Compiled QAOA-family circuits are generally noise dominated, and a matched uniform-random control shows that most feasible low-fidelity outcomes fall within the random range, apart from one finite-sample MIS warm-start exception. A SWAP-aware, fractional-gate, Nighthawk-topology compilation counterfactual reduces two-qubit counts but leaves all circuits below
Fest=10−3. These conclusions apply to the tested implementations rather than QAOA in general. Qubit-efficient methods extend runnable instance sizes, but only within the empirical fidelity budget.
Random quantum circuit sampling, a task to sample bit strings from a random quantum circuit, is considered a suitable benchmark task to demonstrate the outperformance of quantum computers even with noisy qubits. Recently, random quantum circuit sampling was performed on the Sycamore quantum processor with 53 qubits [Nature (London) 574, 505 (2019)] and on the Zuchongzhi quantum processor with 56 qubits [Phys. Rev. Lett. 127, 180501 (2021)]. Here, we analyze and compare the statistical properties of the outputs of the random quantum circuit sampling by the Sycamore and Zuchongzhi processors. Using the Marchenko-Pastur law of random matrices of bit strings and the Wasssertein distances between bit strings, we find that the statistical properties of Sycamore bit strings are quite different from those of Zuchongzhi bit strings, while both processors score similar values of linear cross-entropy fidelity for random circuit sampling. Some bit strings sampled by the Zuchongzhi processor pass the NIST random number tests while both Sycamore and Zuchongzhi processors show similar patterns in the heat maps of bit strings. Zuchongzhi bit strings are much closer to classical uniform random bits than those of Sycamore. It is shown that the statistical properties of bit strings of both random quantum circuits change little as the depth of the random quantum circuits increases. Our findings raise a question about the computational reliability of noisy quantum processors because two quantum processors with similar noise levels and similar qubit structures produced statistically different outputs for the same random quantum circuit sampling.