Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

NISQ Benchmarking

Elijah Pelofske, Andreas Bärtschi, Stephan Eidenbenz, John K. Golden

Published Jan 1, 2022• 0 citations
Publisher

Abstract

Test suite of quantum algorithms for Noisy Intermediate Scale Quantum (NISQ) computers. The test suite includes benchmark-style code for quantum volume circuits (QV), fairness sampling circuits, quantum telecloning circuits, and other NISQ benchmark style algorithms on small problems (i.e., up to 100 qubits), such as Variational Quantum Eigensolver (VQE), Hamiltonian Simulation, and Grover unstructured search example circuits. These benchmark-style applications are implemented in quantum software packages, mostly IBM's QISKIT, but may include vendor-specific frameworks, such as PyQuil (for Rigetti) or Q\# for Microsoft, or CirQ (for Google) as the test suite grows with the vendor sample. The test suite also includes numerical simulation code for Quantum Alternating Operator Ansatz (QAOA) algorithms, VQE, Hamiltonian Simulation and search examples. Numerical simulation code simulates quantum computers on classical computers, which is only possible for small problem instances; the implementation framework of choice is typically within Python, using the numpy/scipy libraries as well as extensions to the Julia language.

DOI
10.11578/dc.20220715.2

Related papers

Quantum Volume in Practice: What Users Can Expect From NISQ Devices

Elijah Pelofske, Andreas Bärtschi, Stephan Eidenbenz

Quantum volume (QV) has become the de-facto standard benchmark to quantify the capability of Noisy Intermediate-Scale Quantum (NISQ) devices. While QV values are often reported by NISQ providers for their systems, we perform our own series of QV calculations on 24 NISQ devices currently offered by IBM Q, IonQ, Rigetti, Oxford Quantum Circuits, and Quantinuum (formerly Honeywell). Our approach characterizes the performances that an advanced user of these NISQ devices can expect to achieve with a reasonable amount of optimization, but without white-box access to the device. In particular, we compile QV circuits to standard gate sets of the vendor using compiler optimization routines where available, and we perform experiments across different qubit subsets. We find that running QV tests requires very significant compilation cycles, QV values achieved in our tests typically lag behind officially reported results and also depend significantly on the classical compilation effort invested.

arXiv:2203.03816•IEEE Transactions on Quantum Engineering•Jan 1, 2022•75 citations

QAOA-based Fair Sampling on NISQ Devices.

John K. Golden, Andreas Bärtschi, Daniel O’Malley +1 more

We study the status of fair sampling on Noisy Intermediate Scale Quantum (NISQ) devices, in particular the IBM Q family of backends. Using the recently introduced Grover Mixer-QAOA algorithm for discrete optimization, we generate fair sampling circuits to solve six problems of varying difficulty, each with several optimal solutions, which we then run on ten different backends available on the IBM Q system. For a given circuit evaluated on a specific set of qubits, we evaluate: how frequently the qubits return an optimal solution to the problem, the fairness with which the qubits sample from all optimal solutions, and the reported hardware error rate of the qubits. To quantify fairness, we define a novel metric based on Pearson's $\chi^2$ test. We find that fairness is relatively high for circuits with small and large error rates, but drops for circuits with medium error rates. This indicates that structured errors dominate in this regime, while unstructured errors, which are random and thus inherently fair, dominate in noisier qubits and longer circuits. Our results provide a simple, intuitive means of quantifying fairness in quantum circuits, and show that reducing structured errors is necessary to improve fair sampling on NISQ hardware.

•Jan 8, 2021•2 citations

Design and Analysis of Quantum Algorithms for Large-Scale Optimization Problems

Laura Bianchi

Large-scale combinatorial and continuous optimisation problems remain intractable for classical algorithms as problem dimensionality scales. Quantum computing offers advantages through parallelism, amplitude amplification, and tunnelling, but practical realisation on near-term noisy intermediate-scale quantum (NISQ) devices demands careful algorithm design balancing advantage against noise resilience and circuit depth constraints. This paper presents the Quantum Optimisation Algorithm Suite (QOAS), a systematic framework across four algorithmic paradigms: Adaptive QAOA with dynamic depth scheduling; Variational Quantum Eigensolver (VQE) for QUBO problems; hybrid Quantum Annealing (QA); and Grover-enhanced Branch and Bound (GBB) for constrained integer programming. QOAS is benchmarked on five problem classes (MaxCut, TSP, Portfolio, Drug-Target, Feature Selection) across n = 12-84 qubits on IBM Quantum Eagle (127-qubit) and Falcon (27-qubit). Adaptive QAOA achieves approximation ratio 0.924 (SD = 0.018) for MaxCut at n = 48 -- outperforming standard QAOA (0.848) and Goemans-Williamson (0.878) at equivalent wall-clock time. GBB achieves 28.4x speedup over classical branch-and-bound for the n = 24 drug-target problem. QOAS provides systematic methodology for algorithm selection, circuit depth optimisation, and noise mitigation for NISQ workloads.

Quantum Frontiers Journal•Mar 28, 2024•0 citations

Fair Sampling Error Analysis on NISQ Devices

John K. Golden, Andreas Bärtschi, Daniel O’Malley +1 more

We study the status of fair sampling on Noisy Intermediate Scale Quantum (NISQ) devices, in particular the IBM Q family of backends. Using the recently introduced Grover Mixer-QAOA algorithm for discrete optimization, we generate fair sampling circuits to solve six problems of varying difficulty, each with several optimal solutions, which we then run on twenty backends across the IBM Q system. For a given circuit evaluated on a specific set of qubits, we evaluate: how frequently the qubits return an optimal solution to the problem, the fairness with which the qubits sample from all optimal solutions, and the reported hardware error rate of the qubits. To quantify fairness, we define a novel metric based on Pearson’s χ 2 test. We find that fairness is relatively high for circuits with small and large error rates, but drops for circuits with medium error rates. This indicates that structured errors dominate in this regime, while unstructured errors, which are random and thus inherently fair, dominate in noisier qubits and longer circuits. Our results show that fairness can be a powerful tool for understanding the intricate web of errors affecting current NISQ hardware.

arXiv:2101.03258•ACM Transactions on Quantum Computing•Mar 7, 2022•16 citations

Hybrid Classical-Quantum Algorithms for Optimization: A Multi-SDK Framework

Murali Krishna Pasupuleti

Abstract: This paper explores hybrid quantum-classical optimization algorithms implemented across multiple quantum software development kits (SDKs): Qiskit and Cirq,. We evaluate the Variational Quantum Eigensolver (VQE) and Quantum Approximate Optimization Algorithm (QAOA) in terms of convergence rate, final cost value, and fidelity, using both simulators and real quantum hardware backends. Our results offer comparative insights into SDK performance and propose a standardized benchmarking approach for future quantum algorithm research. Keywords: Hybrid Quantum Algorithms, Qiskit, Cirq, VQE, QAOA, Quantum SDKs, Quantum Optimization, NISQ

International Journal of Academic and Industrial Research Innovations(IJAIRI)•May 20, 2025•0 citations

Quantum Benchmark Suite for IBM's Backend Based on SupermarQ

Chalothorn Yooyuen, Ruchipas Bavontaweepanya, Prapong Prechaprapranwong +1 more

The benchmark of Noisy Intermediate-Scale Quantum (NISQ) computers plays a major role in the research and development of quantum devices and algorithms. Our study aims to investigate a single metric of benchmarking for the Noisy Intermediate-Scale Quantum device. We use state-of-the-art applications to generate a device-score according to our proposed formula. Our study utilizes Quantum Volume (QV) to determine the qubit circuit size for each device. Furthermore, we utilize SupermarQ to determine each application's benchmark score, utilizing their 5 feature vectors to indicate the circuit performance, and applying their circuit optimization techniques to configure the transpiler. Our experiment shows that device-score of the simulated IBM backend are directly correlated to Quantum Volume. However, it should be noted that topology and gate error are also embedded in the benchmark scores of the applications. Because of this, our study introduces the device-score, a single number metric that provides a more precise scale with the assistance of Quantum Volume and SupermarQ.

International Joint Conference on Computer Science and Software Engineering•Jun 19, 2024•0 citations
Test suite of quantum algorithms for Noisy Intermediate Scale Quantum (NISQ) computers. The test suite includes benchmark-style code for quantum volume circuits (QV), fairness sampling circuits, quantum telecloning circuits, and other NISQ benchmark style algorithms on small problems (i.e., up to 100 qubits), such as Variational Quantum Eigensolver (VQE), Hamiltonian Simulation, and Grover unstructured search example circuits. These benchmark-style applications are implemented in quantum software packages, mostly IBM's QISKIT, but may include vendor-specific frameworks, such as PyQuil (for Rigetti) or Q# for Microsoft, or CirQ (for Google) as the test suite grows with the vendor sample. The test suite also includes numerical simulation code for Quantum Alternating Operator Ansatz (QAOA) algorithms, VQE, Hamiltonian Simulation and search examples. Numerical simulation code simulates quantum computers on classical computers, which is only possible for small problem instances; the implementation framework of choice is typically within Python, using the numpy/scipy libraries as well as extensions to the Julia language.
Quantum volume (QV) has become the de-facto standard benchmark to quantify the capability of Noisy Intermediate-Scale Quantum (NISQ) devices. While QV values are often reported by NISQ providers for their systems, we perform our own series of QV calculations on 24 NISQ devices currently offered by IBM Q, IonQ, Rigetti, Oxford Quantum Circuits, and Quantinuum (formerly Honeywell). Our approach characterizes the performances that an advanced user of these NISQ devices can expect to achieve with a reasonable amount of optimization, but without white-box access to the device. In particular, we compile QV circuits to standard gate sets of the vendor using compiler optimization routines where available, and we perform experiments across different qubit subsets. We find that running QV tests requires very significant compilation cycles, QV values achieved in our tests typically lag behind officially reported results and also depend significantly on the classical compilation effort invested.
We study the status of fair sampling on Noisy Intermediate Scale Quantum (NISQ) devices, in particular the IBM Q family of backends. Using the recently introduced Grover Mixer-QAOA algorithm for discrete optimization, we generate fair sampling circuits to solve six problems of varying difficulty, each with several optimal solutions, which we then run on ten different backends available on the IBM Q system. For a given circuit evaluated on a specific set of qubits, we evaluate: how frequently the qubits return an optimal solution to the problem, the fairness with which the qubits sample from all optimal solutions, and the reported hardware error rate of the qubits. To quantify fairness, we define a novel metric based on Pearson's χ2\chi^2χ2 test. We find that fairness is relatively high for circuits with small and large error rates, but drops for circuits with medium error rates. This indicates that structured errors dominate in this regime, while unstructured errors, which are random and thus inherently fair, dominate in noisier qubits and longer circuits. Our results provide a simple, intuitive means of quantifying fairness in quantum circuits, and show that reducing structured errors is necessary to improve fair sampling on NISQ hardware.
Large-scale combinatorial and continuous optimisation problems remain intractable for classical algorithms as problem dimensionality scales. Quantum computing offers advantages through parallelism, amplitude amplification, and tunnelling, but practical realisation on near-term noisy intermediate-scale quantum (NISQ) devices demands careful algorithm design balancing advantage against noise resilience and circuit depth constraints. This paper presents the Quantum Optimisation Algorithm Suite (QOAS), a systematic framework across four algorithmic paradigms: Adaptive QAOA with dynamic depth scheduling; Variational Quantum Eigensolver (VQE) for QUBO problems; hybrid Quantum Annealing (QA); and Grover-enhanced Branch and Bound (GBB) for constrained integer programming. QOAS is benchmarked on five problem classes (MaxCut, TSP, Portfolio, Drug-Target, Feature Selection) across n = 12-84 qubits on IBM Quantum Eagle (127-qubit) and Falcon (27-qubit). Adaptive QAOA achieves approximation ratio 0.924 (SD = 0.018) for MaxCut at n = 48 -- outperforming standard QAOA (0.848) and Goemans-Williamson (0.878) at equivalent wall-clock time. GBB achieves 28.4x speedup over classical branch-and-bound for the n = 24 drug-target problem. QOAS provides systematic methodology for algorithm selection, circuit depth optimisation, and noise mitigation for NISQ workloads.
We study the status of fair sampling on Noisy Intermediate Scale Quantum (NISQ) devices, in particular the IBM Q family of backends. Using the recently introduced Grover Mixer-QAOA algorithm for discrete optimization, we generate fair sampling circuits to solve six problems of varying difficulty, each with several optimal solutions, which we then run on twenty backends across the IBM Q system. For a given circuit evaluated on a specific set of qubits, we evaluate: how frequently the qubits return an optimal solution to the problem, the fairness with which the qubits sample from all optimal solutions, and the reported hardware error rate of the qubits. To quantify fairness, we define a novel metric based on Pearson’s χ 2 test. We find that fairness is relatively high for circuits with small and large error rates, but drops for circuits with medium error rates. This indicates that structured errors dominate in this regime, while unstructured errors, which are random and thus inherently fair, dominate in noisier qubits and longer circuits. Our results show that fairness can be a powerful tool for understanding the intricate web of errors affecting current NISQ hardware.
Abstract: This paper explores hybrid quantum-classical optimization algorithms implemented across multiple quantum software development kits (SDKs): Qiskit and Cirq,. We evaluate the Variational Quantum Eigensolver (VQE) and Quantum Approximate Optimization Algorithm (QAOA) in terms of convergence rate, final cost value, and fidelity, using both simulators and real quantum hardware backends. Our results offer comparative insights into SDK performance and propose a standardized benchmarking approach for future quantum algorithm research. Keywords: Hybrid Quantum Algorithms, Qiskit, Cirq, VQE, QAOA, Quantum SDKs, Quantum Optimization, NISQ
The benchmark of Noisy Intermediate-Scale Quantum (NISQ) computers plays a major role in the research and development of quantum devices and algorithms. Our study aims to investigate a single metric of benchmarking for the Noisy Intermediate-Scale Quantum device. We use state-of-the-art applications to generate a device-score according to our proposed formula. Our study utilizes Quantum Volume (QV) to determine the qubit circuit size for each device. Furthermore, we utilize SupermarQ to determine each application's benchmark score, utilizing their 5 feature vectors to indicate the circuit performance, and applying their circuit optimization techniques to configure the transpiler. Our experiment shows that device-score of the simulated IBM backend are directly correlated to Quantum Volume. However, it should be noted that topology and gate error are also embedded in the benchmark scores of the applications. Because of this, our study introduces the device-score, a single number metric that provides a more precise scale with the assistance of Quantum Volume and SupermarQ.