Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2405.08617 · Lecture notes in computer science

A Methodology for Comparing and Benchmarking Quantum Devices

Jessica Park, Susan Stepney, Irene D’Amico

Published Jan 1, 2024• 1 citation
arXivHTMLPDF
DOI
10.1007/978-3-031-63742-1_3

Related papers

Seeking Quantum Speedup Through Spin Glasses: The Good, the Bad, and the Ugly

Helmut G. Katzgraber, Firas Hamze, Zhu, Zheng +2 more

There has been considerable progress in the design and construction of quantum annealing devices. However, a conclusive detection of quantum speedup over traditional silicon-based machines remains elusive, despite multiple careful studies. In this work we outline strategies to design hard tunable benchmark instances based on insights from the study of spin glasses - the archetypal random benchmark problem for novel algorithms and optimization devices. We propose to complement head-to-head scaling studies that compare quantum annealing machines to state-of-the-art classical codes with an approach that compares the performance of different algorithms and/or computing architectures on different classes of computationally hard tunable spin-glass instances. The advantage of such an approach lies in having to only compare the performance hit felt by a given algorithm and/or architecture when the instance complexity is increased. Furthermore, we propose a methodology that might not directly translate into the detection of quantum speedup, but might elucidate whether quantum annealing has a "`quantum advantage" over corresponding classical algorithms like simulated annealing. Our results on a 496 qubit D-Wave Two quantum annealing device are compared to recently-used state-of-the-art thermal simulated annealing codes.

•Jan 1, 2015•86 citations

Benchmarking quantum computers

Timothy J Proctor, Kevin C Young, Andrew Baczewski +1 more

arXiv:2407.08828•Nature Reviews Physics•Jan 7, 2025•71 citations

QuSquare: Scalable Quality-Oriented Benchmark Suite for Pre-Fault-Tolerant Quantum Devices

David Aguirre, Rubén Peña, Mikel Sanz

As quantum technologies continue to advance, the proliferation of hardware architectures with diverse capabilities and limitations has underscored the importance of benchmarking as a tool to compare performance across platforms. Achieving fair, scalable and consistent evaluations is a key open problem in quantum computing, particularly in the pre-fault-tolerant era. To address this challenge, we introduce QuSquare, a quality-oriented benchmark suite designed to provide a scalable, fair, reproducible, and well-defined framework for assessing the performance of quantum devices across hardware architectures. QuSquare consists of four benchmark tests that evaluate quantum hardware performance at both the system and application levels: Partial Clifford Randomized, Multipartite Entanglement, Transverse Field Ising Model (TFIM) Hamiltonian Simulation, and Data Re-Uploading Quantum Neural Network (QNN). Together, these benchmarks offer an integral, hardware-agnostic, and impartial methodology to quantify the quality and capabilities of current quantum computers, supporting fair cross-platform comparisons and fostering the development of future performance standards.

quant-pharXiv:2512.19665•Quantum Science and Technology•Dec 23, 2025•0 citations

Benchmarking Quantum Computers

Robson Giles

•Jan 1, 2024•0 citations

Scalable and targeted benchmarking of quantum computers

Timothy J Proctor, Kenneth Rudinger, Mohan Sarovar +3 more

Bulletin of the American Physical Society•Mar 15, 2021•0 citations

A Comprehensive Cross-Model Framework for Benchmarking the Performance of Quantum Hamiltonian Simulations

Avimita Chatterjee, Sonny Rappaport, Anish Giri +5 more

Quantum Hamiltonian simulation is one of the most promising applications of quantum computing and forms the basis for many quantum algorithms. Benchmarking them is an important gauge of progress in quantum computing technology. We present a methodology and software framework to evaluate various facets of the performance of gate-based quantum computers on Trotterized quantum Hamiltonian evolution. We propose three distinct modes for benchmarking: (i) comparing simulation on a real device to that on a noiseless classical simulator, (ii) comparing simulation on a real device with exact diagonalization results, and (iii) using scalable mirror circuit techniques to assess hardware performance in scenarios beyond classical simulation methods. We demonstrate this framework on five Hamiltonian models from the HamLib library: the Fermi and Bose-Hubbard models, the transverse field Ising model, the Heisenberg model, and the Max3SAT problem. Experiments were conducted using Qiskit's Aer simulator, BlueQubit's CPU cluster and GPU simulators, and IBM's quantum hardware. Our framework, extendable to other Hamiltonians, provides comprehensive performance profiles that reveal hardware and algorithmic limitations and measure both fidelity and execution times, identifying crossover points where quantum hardware outperforms CPU/GPU simulators.

arXiv:2409.06919•IEEE Transactions on Quantum Engineering•Jan 1, 2025•7 citations
There has been considerable progress in the design and construction of quantum annealing devices. However, a conclusive detection of quantum speedup over traditional silicon-based machines remains elusive, despite multiple careful studies. In this work we outline strategies to design hard tunable benchmark instances based on insights from the study of spin glasses - the archetypal random benchmark problem for novel algorithms and optimization devices. We propose to complement head-to-head scaling studies that compare quantum annealing machines to state-of-the-art classical codes with an approach that compares the performance of different algorithms and/or computing architectures on different classes of computationally hard tunable spin-glass instances. The advantage of such an approach lies in having to only compare the performance hit felt by a given algorithm and/or architecture when the instance complexity is increased. Furthermore, we propose a methodology that might not directly translate into the detection of quantum speedup, but might elucidate whether quantum annealing has a "`quantum advantage" over corresponding classical algorithms like simulated annealing. Our results on a 496 qubit D-Wave Two quantum annealing device are compared to recently-used state-of-the-art thermal simulated annealing codes.
As quantum technologies continue to advance, the proliferation of hardware architectures with diverse capabilities and limitations has underscored the importance of benchmarking as a tool to compare performance across platforms. Achieving fair, scalable and consistent evaluations is a key open problem in quantum computing, particularly in the pre-fault-tolerant era. To address this challenge, we introduce QuSquare, a quality-oriented benchmark suite designed to provide a scalable, fair, reproducible, and well-defined framework for assessing the performance of quantum devices across hardware architectures. QuSquare consists of four benchmark tests that evaluate quantum hardware performance at both the system and application levels: Partial Clifford Randomized, Multipartite Entanglement, Transverse Field Ising Model (TFIM) Hamiltonian Simulation, and Data Re-Uploading Quantum Neural Network (QNN). Together, these benchmarks offer an integral, hardware-agnostic, and impartial methodology to quantify the quality and capabilities of current quantum computers, supporting fair cross-platform comparisons and fostering the development of future performance standards.
Quantum Hamiltonian simulation is one of the most promising applications of quantum computing and forms the basis for many quantum algorithms. Benchmarking them is an important gauge of progress in quantum computing technology. We present a methodology and software framework to evaluate various facets of the performance of gate-based quantum computers on Trotterized quantum Hamiltonian evolution. We propose three distinct modes for benchmarking: (i) comparing simulation on a real device to that on a noiseless classical simulator, (ii) comparing simulation on a real device with exact diagonalization results, and (iii) using scalable mirror circuit techniques to assess hardware performance in scenarios beyond classical simulation methods. We demonstrate this framework on five Hamiltonian models from the HamLib library: the Fermi and Bose-Hubbard models, the transverse field Ising model, the Heisenberg model, and the Max3SAT problem. Experiments were conducted using Qiskit's Aer simulator, BlueQubit's CPU cluster and GPU simulators, and IBM's quantum hardware. Our framework, extendable to other Hamiltonians, provides comprehensive performance profiles that reveal hardware and algorithmic limitations and measure both fidelity and execution times, identifying crossover points where quantum hardware outperforms CPU/GPU simulators.