Current technological advancements of quantum computers highlight the need for application-driven, practical and well-defined methods of benchmarking their performance. As the existing NISQ device's quality of two-qubit gate errors rate is even around one percent and the number of qubits is still limited to a few or several dozen, naturally, we need to propose rather small algorithms instances taken from key promising application areas, such as quantum chemistry, combinatorial optimisation or machine learning. While many techniques for assessing the performance of logical components, such as gate fidelity and qubit coherence exist, it is still challenging to extrapolate those values onto the performance of different quantum algorithms and subroutines. This work aims to introduce a series of initial quantum application benchmarks together with a methodology of execution for measuring the performance and fidelity of the results. The proposed suite refers to several variational algorithms widely used on available NISQ devices but also includes examples of quantum circuits designed for a fault-tolerant quantum computer.
Current technological advancements of quantum computers highlight the need for application-driven, practical and well-defined methods of benchmarking their performance. As the existing NISQ device's quality of two-qubit gate errors rate is even around one percent and the number of qubits is still limited to a few or several dozen, naturally, we need to propose rather small algorithms instances taken from key promising application areas, such as quantum chemistry, combinatorial optimisation or machine learning. While many techniques for assessing the performance of logical components, such as gate fidelity and qubit coherence exist, it is still challenging to extrapolate those values onto the performance of different quantum algorithms and subroutines. This work aims to introduce a series of initial quantum application benchmarks together with a methodology of execution for measuring the performance and fidelity of the results. The proposed suite refers to several variational algorithms widely used on available NISQ devices but also includes examples of quantum circuits designed for a fault-tolerant quantum computer.
As quantum processors continue to scale in size and complexity, the need for well-defined, reproducible, and technology-agnostic performance metrics becomes increasingly critical. Here we present a suite of scalable quantum computing benchmarks developed as key performance indicators (KPIs) within the EU Quantum Flagship. These proposed benchmarks are designed to assess holistic system performance rather than isolated components, and to remain applicable across both noisy intermediate-scale quantum (NISQ) devices and future fault-tolerant architectures. We introduce four core benchmarks addressing complementary aspects of quantum computing capability: large multi-qubit circuit execution via a Clifford Volume benchmark, scalable multipartite entanglement generation through GHZ-state preparation, a benchmark based on the application of Shor's period-finding subroutine to simple functions, and a protocol quantifying the benefit of quantum error correction using Bell states. Each benchmark is accompanied by clearly specified protocols, reporting standards, and scalable evaluation methods. Together, these KPIs provide a coherent framework for transparent and fair performance assessment across quantum hardware platforms and for tracking progress late-NISQ toward early fault-tolerant quantum computation.
It is widely recognized that quantum computing has profound impacts on multiple fields, including but not limited to cryptography, machine learning, materials science, and so on. To run quantum algorithms, it is essential to develop scalable quantum hardware with low noise levels and to design efficient fault-tolerant quantum computing (FTQC) schemes. Currently, various FTQC schemes have been developed for different hardware platforms. However, a comprehensive framework for the analysis and evaluation of these schemes is still lacking. In this work, we propose Quantum Logical Operations Per Second (QLOPS) as a metric for assessing the performance of FTQC schemes on quantum hardware platforms. This benchmarking framework will integrate essential relevant factors, e.g., the code rates of quantum error-correcting codes, the accuracy, throughput, and latency of the decoder. Through a resource analysis of factoring RSA-2048, we demonstrate that QLOPS reflects the practical requirements of quantum algorithm execution. This framework will enable the identification of bottlenecks in quantum hardware, providing potential directions for their development. Moreover, our results will help establish a comparative framework for evaluating FTQC designs. As this benchmarking approach considers practical applications, it may assist in estimating the hardware resources needed to implement quantum algorithms and offers preliminary insights into potential timelines.
The development of near-term quantum computers, referred to as Noisy Intermediate-Scale Quantum (NISQ) computers, has progressed rapidly in the past few years resulting in several quantum computers which vary in their underlying technology and physical constraints. The performance of these computers also varies from one quantum algorithm to another. To enable efficient selection of the quantum computer that provides the highest output fidelity for a given application, an accurate noise modeling of each quantum hardware is required. However, noise modeling for a given application is a complex problem because of the unknown interaction between the quantum circuit parameters and the noise parameters of NISQ devices. We propose the use of Machine Learning (ML) to model the performance of different quantum computers at the application level. The ML models predict the output fidelity of the quantum application executed on different quantum computers given their publicly available physical constraints. We use a diverse training dataset to cover the key features for application-level benchmarking of the quantum hardware. Our results obtained from different superconducting quantum devices show that our proposed ML models enable cost-effective quantum computer selection for different quantum applications with different fidelity metrics.
In this work we introduce an open source suite of quantum application-oriented performance benchmarks that is designed to measure the effectiveness of quantum computing hardware at executing quantum applications. These benchmarks probe a quantum computer's performance on various algorithms and small applications as the problem size is varied, by mapping out the fidelity of the results as a function of circuit width and depth using the framework of volumetric benchmarking. In addition to estimating the fidelity of results generated by quantum execution, the suite is designed to benchmark certain aspects of the execution pipeline in order to provide end-users with a practical measure of both the quality of and the time to solution. Our methodology is constructed to anticipate advances in quantum computing hardware that are likely to emerge in the next five years. This benchmarking suite is designed to be readily accessible to a broad audience of users and provides benchmarks that correspond to many well-known quantum computing algorithms.
The major advances in quantum computing over the last few decades have sparked great interest in applying it to solve the most challenging computational problems in a wide variety of areas. One of the most pronounced domains here are optimization problems and a number of algorithmic approaches have been proposed for their solution. For the current noisy intermediate-scale quantum (NISQ) computers the quantum approximate optimization algorithm (QAOA), the variational quantum eigensolver (VQE), and quantum annealing (QA) are the central algorithms for this problem class. The two former can be executed on digital gate-model quantum computers, whereas the latter requires a quantum annealer. Across all hardware architectures and manufactures, the quantum computers available today share the property of being too error-prone to reliably execute involved quantum circuits as they typically arise from quantum optimization algorithms. In order to characterize the limits of existing quantum computers, many component and system level benchmarks have been proposed. However, owing to the complex nature of the errors in quantum systems these benchmark fail to provide predictive power beyond simple quantum circuits and small examples. Application oriented benchmarks have been proposed to remedy this problem, but both, results from real quantum systems as well as use cases beyond constructed academic examples, remain very rare. This paper addresses precisely this gap by considering two industrial relevant use cases: one in the realm of optimizing charging schedules for electric vehicles, the other concerned with the optimization of truck routes. Our central contribution are systematic series of examples derived from these uses cases that we execute on different processors of the gate-based quantum computers of IBM as well as on the quantum annealer of D-Wave.
As quantum technologies continue to advance, the proliferation of hardware architectures with diverse capabilities and limitations has underscored the importance of benchmarking as a tool to compare performance across platforms. Achieving fair, scalable and consistent evaluations is a key open problem in quantum computing, particularly in the pre-fault-tolerant era. To address this challenge, we introduce QuSquare, a quality-oriented benchmark suite designed to provide a scalable, fair, reproducible, and well-defined framework for assessing the performance of quantum devices across hardware architectures. QuSquare consists of four benchmark tests that evaluate quantum hardware performance at both the system and application levels: Partial Clifford Randomized, Multipartite Entanglement, Transverse Field Ising Model (TFIM) Hamiltonian Simulation, and Data Re-Uploading Quantum Neural Network (QNN). Together, these benchmarks offer an integral, hardware-agnostic, and impartial methodology to quantify the quality and capabilities of current quantum computers, supporting fair cross-platform comparisons and fostering the development of future performance standards.