Abstract In this report, we explore benchmarking gate-based quantum devices based on the certification of quantum measurements, which can be viewed as an extension of quantum hypothesis testing. Here, we are interested in a binary certification scheme in which the null and alternative hypotheses are qubit von Neumann measurements. The certification procedure can be used to evaluate the accuracy of Noisy Intermediate-Scale Quantum devices. To support this methodology, we contribute benchmarks to PyQBench, an open-source Python library for benchmarking gate-based quantum computers. We provide a user-friendly command-line interface for certifying measurements from the Fourier family. For more advanced scenarios, PyQBench can also be used as a Python library, allowing users to define custom measurement schemes and error models. The latest version of PyQBench is designed to support IBM Q devices through integration with the Qiskit library, enabling seamless benchmarking on real quantum hardware. By providing flexible benchmarking tools, PyQBench addresses the critical need for reliable performance metrics in the NISQ era, contributing to the development of error mitigation techniques and the verification of quantum measurement fidelity.
Abstract In this report, we explore benchmarking gate-based quantum devices based on the certification of quantum measurements, which can be viewed as an extension of quantum hypothesis testing. Here, we are interested in a binary certification scheme in which the null and alternative hypotheses are qubit von Neumann measurements. The certification procedure can be used to evaluate the accuracy of Noisy Intermediate-Scale Quantum devices. To support this methodology, we contribute benchmarks to PyQBench, an open-source Python library for benchmarking gate-based quantum computers. We provide a user-friendly command-line interface for certifying measurements from the Fourier family. For more advanced scenarios, PyQBench can also be used as a Python library, allowing users to define custom measurement schemes and error models. The latest version of PyQBench is designed to support IBM Q devices through integration with the Qiskit library, enabling seamless benchmarking on real quantum hardware. By providing flexible benchmarking tools, PyQBench addresses the critical need for reliable performance metrics in the NISQ era, contributing to the development of error mitigation techniques and the verification of quantum measurement fidelity.
We present an updated version of PyQBench, an open-source Python library designed for benchmarking gate-based quantum computers, with a focus on certifying qubit von Neumann measurements. This version extends PyQBench's capabilities by incorporating a certification scheme of quantum measurements that evaluates the accuracy on Noisy Intermediate-Scale Quantum devices, alongside its original functionality of von Neumann measurements' discrimination. PyQBench offers a user-friendly command-line interface and Python library integration, allowing users to implement custom measurement schemes and error models for more advanced benchmarking tasks. The new version is specifically designed to support IBM Q devices through integration with the Qiskit library, enabling seamless benchmarking on real quantum hardware. By providing flexible benchmarking tools, PyQBench addresses the critical need for reliable performance metrics in the NISQ era, contributing to the development of error mitigation techniques and the verification of quantum measurement fidelity. The source code is available on GitHub under an open-source license, encouraging community collaboration and further advancements in quantum hardware benchmarking.
In the near-term "NISQ"-era of noisy, intermediate-scale, quantum hardware and beyond, reliably determining the quality of quantum devices becomes increasingly important: users need to be able to compare them with one another, and make an estimate whether they are capable of performing a given task ahead of time. In this work, we develop and release an advanced quantum benchmarking framework in order to help assess the state of the art of current quantum devices. Our testing framework measures the performance of universal quantum devices in a hardware-agnostic way, with metrics that are aimed to facilitate an intuitive understanding of which device is likely to outperform others on a given task. This is achieved through six structured tests that allow for an immediate, visual assessment of how devices compare. Each test is designed with scalability in mind, making this framework not only suitable for testing the performance of present-day quantum devices, but also of those released in the foreseeable future. The series of tests are motivated by real-life scenarios, and therefore emphasise the interplay between various relevant characteristics of quantum devices, such as qubit count, connectivity, and gate and measurement fidelity. We present the benchmark results of twenty-one different quantum devices from IBM, Rigetti and IonQ.
As quantum computers continue to increase in size and topological complexity, benchmarking crosstalk becomes more complex and resource-intensive. This limits the ability to obtain relevant crosstalk metrics for applications such as error mitigation, quantum computer security, and circuit transpilation. There applications benefit from accessible metrics on how each gate contributes to crosstalk. However, crosstalk metrics are rarely provided by quantum computer providers and can be expensive to obtain on modern large NISQ devices. In this work, we propose CrossBench, a customizable system that generates crosstalk benchmarks for finitely large NISQ devices with arbitrary topologies. CrossBench uses a custom graph labeling algorithm to generate crosstalk benchmarks for a given quantum topology and gate set. These benchmarks can be used to estimate the average contribution of each gate's crosstalk to qubit error rates. We evaluate the effectiveness of CrossBench by generating and running benchmarks on multiple IBM quantum computers with different topologies. Our results show that CrossBench can identify gates that introduce significant crosstalk across all tested devices, with strong statistical significance (p < 0.05). These promising results show that CrossBench can give simple and accessible crosstalk benchmarks for modern NISQ systems.
In this report we study certification of quantum measurements, which can be viewed as the extension of quantum hypotheses testing. This extensions involves the study of the input state and the measurement procedure. Here, we will be interested in two-point (binary) certification scheme in which the null alternative hypotheses are single element sets. Our goal is to minimize the probability of the type II error given some fixed statistical significance. In this report, we begin with studying the two-point certification of pure quantum states and unitary channels to later use them to prove our main result, which is the certification of von Neumann measurements in single-shot and parallel scenarios. From our main result follow the conditions when two pure states, unitary operations and von Neumann measurements cannot be distinguished perfectly but still can be certified with a given statistical significance. Moreover, we show the connection between the certification of quantum channels and the notion of q-numerical range.
The rapid advancement of quantum hardware calls for the development of reliable methods to certify its correct functioning. However, existing certification tests often fall short: they either rely on flawless state preparation and measurement or lack soundness guarantees, meaning that they do not rule out incorrect implementations of the target operations by a quantum device. We introduce an approach, which we call quantum system quizzing, for the certification of quantum gates in a practical server-user scenario, where a classical user tests the results of quantum computation performed by a quantum server by checking its responses to a set of predesigned small-sized computational problems. Importantly, this approach does not require trusted state preparation and measurement and is thus inherently free from the associated systematic errors. For a wide range of relevant gate sets, including a universal one, we prove our certification protocol to be sound; i.e., it is guaranteed to reject any incorrect gate implementation, under the assumptions of a known Hilbert space dimension and context independence of error. A major technical challenge that we are first to resolve is recovering the tensor product structure of a multi-qubit system in the memory-bounded single-device setup. Finally, we prove the robustness of our protocol and validate its sample and computational efficiency through extensive numerical experiments. Our protocol is platform-agnostic and introduces a new paradigm for benchmarking and comparing diverse quantum architectures.
Context: The main obstacle of quantum computing on its way to scalability is the noisy nature of current hardware. Various types of errors lead to incorrect computational results,and measures such as quantum error mitigation are needed to counteract these errors. Many error mitigation techniques require a thorough understanding of the noise and its impact on computations. Therefore, accurate noise models are indispensable to facilitate meaningful quantum computing in the near future. They also enable realistic simulations of quantum devices. Since these devices are currently limited and quantum computing is expensive, researchers typically rely on simulations for testing their algorithms.Objective: In this thesis, we aim to construct a realistic noise model for quantum computing and optimise its model parameters. The model construction should build on the underlying physical processes, and the number of parameters should scale well with the system size. We also address the evaluation and comparison of different noise models by benchmarks using quantum circuits specific to real-world applications. Finally, we aim to develop useful graphical representations of noise models based on the existing quantum circuit model.Method: We present a benchmarking framework for quantum computing noise models and evaluate it based on benchmarking quality attributes from the literature. Moreover, we construct a quantum computing noise model and develop a training procedure for its parameters. We analyse the parameters of the trained noise model and the impact ofdifferent types of errors on computations. We benchmark the noise model with the above approach and compare it to a noise model provided by IBM’s software development kit Qiskit. Finally, we develop an extension of the quantum circuit model to represent noise channels.Result: The benchmarks performed in this thesis show that our noise model predicts noisy hardware behaviour of IBM’s ibmq_manila quantum device equally well as the Qiskit model, if not better, based on quantum circuits commonly used for variational quantum algorithms. Within the trained noise model, readout error has the most detrimental impact on computations. Our benchmarking approach satisfies relevant quality criteria by choosing suitable quantum circuits and objective functions to compare model predictionsto hardware data. Moreover, the quantum circuit model can be extended meaningfully by noise channels.Conclusion: Our benchmarking approach is suitable for evaluating and comparing quantum computing noise models. In future work, volumetric benchmarks should be performed for various noise models and application contexts. The parameter optimisation based on learning with training circuits yields accurate noise models and shows the importance of mitigating measurement error. Different optimisation algorithms and quantum circuits should be explored, and more types of noise should be incorporated into the noise model.