Quantum computing holds the potential to provide speedups in solving complex problems that are currently difficult for classical computers. However, the realization of this potential is hindered by the issue of current hardware reliability, primarily due to noise and architectural imperfections. As quantum computing systems rapidly advance, there exists a need to create a generalizable benchmarking tool that can assess reliability across different hardware platforms. In this paper, we introduce GRADE (Grover-based Reliability Assessment for Device Evaluation), an open-source benchmarking toolkit to evaluate the reliability of quantum hardware using a generalized form of Grover's algorithm. GRADE operates by implementing Grover's algorithm to search through a variety of primitive collections that are customizable by the user, analyzing the probability distribution of the results to assess accuracy and stability. This approach aims to evaluate the hardware performance of Grover's algorithm, which is fundamental in unordered search problems, making it an ideal candidate for benchmarking purposes, as it is one of the few generalizable quantum algorithms that provide a direct speedup. Importantly, GRADE can adapt to a wide range of quantum computing platforms, ensuring it can be applied across different hardware architectures. Additionally, our work provides a scoring function for evaluating the hardware performance for multi-target Grover's implementation. Our work validates this approach across a multitude of simulators and hardware platforms from varying providers, demonstrating its adaptability to differing backend providers.
Quantum computing holds the potential to provide speedups in solving complex problems that are currently difficult for classical computers. However, the realization of this potential is hindered by the issue of current hardware reliability, primarily due to noise and architectural imperfections. As quantum computing systems rapidly advance, there exists a need to create a generalizable benchmarking tool that can assess reliability across different hardware platforms. In this paper, we introduce GRADE (Grover-based Reliability Assessment for Device Evaluation), an open-source benchmarking toolkit to evaluate the reliability of quantum hardware using a generalized form of Grover's algorithm. GRADE operates by implementing Grover's algorithm to search through a variety of primitive collections that are customizable by the user, analyzing the probability distribution of the results to assess accuracy and stability. This approach aims to evaluate the hardware performance of Grover's algorithm, which is fundamental in unordered search problems, making it an ideal candidate for benchmarking purposes, as it is one of the few generalizable quantum algorithms that provide a direct speedup. Importantly, GRADE can adapt to a wide range of quantum computing platforms, ensuring it can be applied across different hardware architectures. Additionally, our work provides a scoring function for evaluating the hardware performance for multi-target Grover's implementation. Our work validates this approach across a multitude of simulators and hardware platforms from varying providers, demonstrating its adaptability to differing backend providers.
This paper introduces HamilToniQ, an open-source benchmarking toolkit for Quantum Processing Units (QPUs). It addresses the complexities of quantum computations by providing a methodological framework to assess QPU types, topologies, and systems. HamilToniQ facilitates performance evaluations through steps like circuit compilation and quantum error mitigation, with strategies tailored for each stage. The toolkit's H-Score measures QPU fidelity and reliability, offering a comprehensive view of performance. Focused on the Quantum Approximate Optimization Algorithm (QAOA), HamilToniQ enables consistent QPU comparisons, enhancing benchmarking transparency. Validated on various IBM QPUs, the toolkit proves effective and robust, advancing quantum computing with precise benchmarking metrics.
It is widely recognized that quantum computing has profound impacts on multiple fields, including but not limited to cryptography, machine learning, materials science, and so on. To run quantum algorithms, it is essential to develop scalable quantum hardware with low noise levels and to design efficient fault-tolerant quantum computing (FTQC) schemes. Currently, various FTQC schemes have been developed for different hardware platforms. However, a comprehensive framework for the analysis and evaluation of these schemes is still lacking. In this work, we propose Quantum Logical Operations Per Second (QLOPS) as a metric for assessing the performance of FTQC schemes on quantum hardware platforms. This benchmarking framework will integrate essential relevant factors, e.g., the code rates of quantum error-correcting codes, the accuracy, throughput, and latency of the decoder. Through a resource analysis of factoring RSA-2048, we demonstrate that QLOPS reflects the practical requirements of quantum algorithm execution. This framework will enable the identification of bottlenecks in quantum hardware, providing potential directions for their development. Moreover, our results will help establish a comparative framework for evaluating FTQC designs. As this benchmarking approach considers practical applications, it may assist in estimating the hardware resources needed to implement quantum algorithms and offers preliminary insights into potential timelines.
As quantum technologies continue to advance, the proliferation of hardware architectures with diverse capabilities and limitations has underscored the importance of benchmarking as a tool to compare performance across platforms. Achieving fair, scalable and consistent evaluations is a key open problem in quantum computing, particularly in the pre-fault-tolerant era. To address this challenge, we introduce QuSquare, a quality-oriented benchmark suite designed to provide a scalable, fair, reproducible, and well-defined framework for assessing the performance of quantum devices across hardware architectures. QuSquare consists of four benchmark tests that evaluate quantum hardware performance at both the system and application levels: Partial Clifford Randomized, Multipartite Entanglement, Transverse Field Ising Model (TFIM) Hamiltonian Simulation, and Data Re-Uploading Quantum Neural Network (QNN). Together, these benchmarks offer an integral, hardware-agnostic, and impartial methodology to quantify the quality and capabilities of current quantum computers, supporting fair cross-platform comparisons and fostering the development of future performance standards.
Quantum computers are emerging as a major tool in the computation field, leveraging the principles of quantum mechanics to solve specific problems currently beyond the capability of classical computers. This technology holds significant promise in edge-main cloud deployments, where it can enable low-latency data processing and secure communication. This paper aims to establish a research foundation by integrating quantum computing with classical edge-cloud environments to promote performance across a range of applications that scientists are actively investigating. However, the successful deployment of hybrid quantum–classical edge-clouds requires a comprehensive evaluation framework to ensure their alignment with the performance requirements. This study first proposes a novel quantum benchmarking framework, including two distinct methods to evaluate latency scores based on the quantum transpilation levels across different quantum-edge-cloud platforms. The framework is then validated for the edge-cloud environment by benchmarking several well-known and useful quantum algorithms potentially useful in this domain, including Shor’s, Grover’s, and the Quantum Walks algorithm. An optimal transpilation level is eventually suggested to achieve maximum performance in quantum-edge-cloud environments. In summary, this research paper provides critical insights into the current and prospective capabilities of QPU integration, offering a novel benchmarking framework and providing a comprehensive assessment of their potential to enhance edge-cloud performance under varying parameters, including fidelity and transpilation levels.
The fragmented landscape of quantum computer benchmarks, characterized by system-specific tools and inconsistent evaluation methodologies, hinders reliable cross-platform performance assessment. We introduce Metriq, an open-source collaborative platform for reproducible cross-platform quantum benchmarking that integrates benchmark definition and execution, data collection, and public presentation into a unified workflow. The Metriq benchmark suite spans both system-level metrics that characterize fundamental device properties such as entanglement quality, gate performance, and circuit speed, as well as application-inspired protocols that assess performance on quantum machine learning, optimization, and quantum simulation tasks. Benchmarks are chosen to scale with processor size, and the framework incorporates cost and resource estimation to support practical evaluation. Using Metriq, we collect and publicly release results from more than ten quantum computers across multiple hardware vendors, enabling systematic cross-platform comparison. The resulting curated dataset also reveals the practical strengths and limitations of individual benchmarks, creating a feedback loop that informs the ongoing refinement of the suite. To summarize performance across the benchmark suite, we introduce the Metriq Score, a composite index aggregating benchmark outcomes. We further present cross-benchmark analyses enabled by the shared dataset and their correlations with hardware calibration metrics. Through open development and data sharing, Metriq provides a practical foundation for reproducible benchmarking of quantum computers as hardware and benchmarking methods continue to evolve.
This study explores how well Grover's Algorithm performs in solving the Boolean Satisfiability Problem (SAT) using quantum circuits. The algorithm is implemented with IBM's Qiskit framework and compared to classical brute-force methods. Experiments focus on 3-SAT, 4-SAT, and 5-SAT problems, using quantum simulators and IBM quantum hardware. The results show that Grover's Algorithm is more efficient, offering a theoretical quadratic speedup over classical methods. However, practical issues like limited qubit availability, hardware noise, and optimization challenges impact its current performance. The data highlights the potential for quantum computing to scale and solve NP-complete problems. This research shows how quantum computing can improve problem-solving and lays the groundwork for future studies on more complex SAT problems and advanced quantum hardware.