The rapid advancement of quantum technologies has led to a growing and diverse landscape of quantum computing devices, each with varying architectures and capabilities. Most of them are evaluated using hardware-centric benchmarks such as gate fidelity, quantum volume, or coherence time. However, these metrics often fail to capture performance in problem-specific contexts, particularly in applications involving quantum image processing. In this work, we address this gap by introducing the Quantum Image Encoding Fidelity (QIEF) metric, a novel task-specific benchmark designed to evaluate how accurately quantum devices can encode classical image data into quantum representations. By focusing on image-specific criteria rather than device-agnostic parameters, QIEF offers a more application-relevant assessment of device capability, enabling a direct comparison of quantum hardware based on their effectiveness in real-world image-related tasks. We present the theoretical formulation of the QIEF, validate its relevance through quantum simulators and experiments on real quantum devices, and discuss its implications for guiding future hardware development tailored to quantum image processing applications.
The rapid advancement of quantum technologies has led to a growing and diverse landscape of quantum computing devices, each with varying architectures and capabilities. Most of them are evaluated using hardware-centric benchmarks such as gate fidelity, quantum volume, or coherence time. However, these metrics often fail to capture performance in problem-specific contexts, particularly in applications involving quantum image processing. In this work, we address this gap by introducing the Quantum Image Encoding Fidelity (QIEF) metric, a novel task-specific benchmark designed to evaluate how accurately quantum devices can encode classical image data into quantum representations. By focusing on image-specific criteria rather than device-agnostic parameters, QIEF offers a more application-relevant assessment of device capability, enabling a direct comparison of quantum hardware based on their effectiveness in real-world image-related tasks. We present the theoretical formulation of the QIEF, validate its relevance through quantum simulators and experiments on real quantum devices, and discuss its implications for guiding future hardware development tailored to quantum image processing applications.
In the near-term "NISQ"-era of noisy, intermediate-scale, quantum hardware and beyond, reliably determining the quality of quantum devices becomes increasingly important: users need to be able to compare them with one another, and make an estimate whether they are capable of performing a given task ahead of time. In this work, we develop and release an advanced quantum benchmarking framework in order to help assess the state of the art of current quantum devices. Our testing framework measures the performance of universal quantum devices in a hardware-agnostic way, with metrics that are aimed to facilitate an intuitive understanding of which device is likely to outperform others on a given task. This is achieved through six structured tests that allow for an immediate, visual assessment of how devices compare. Each test is designed with scalability in mind, making this framework not only suitable for testing the performance of present-day quantum devices, but also of those released in the foreseeable future. The series of tests are motivated by real-life scenarios, and therefore emphasise the interplay between various relevant characteristics of quantum devices, such as qubit count, connectivity, and gate and measurement fidelity. We present the benchmark results of twenty-one different quantum devices from IBM, Rigetti and IonQ.
As quantum technologies continue to advance, the proliferation of hardware architectures with diverse capabilities and limitations has underscored the importance of benchmarking as a tool to compare performance across platforms. Achieving fair, scalable and consistent evaluations is a key open problem in quantum computing, particularly in the pre-fault-tolerant era. To address this challenge, we introduce QuSquare, a quality-oriented benchmark suite designed to provide a scalable, fair, reproducible, and well-defined framework for assessing the performance of quantum devices across hardware architectures. QuSquare consists of four benchmark tests that evaluate quantum hardware performance at both the system and application levels: Partial Clifford Randomized, Multipartite Entanglement, Transverse Field Ising Model (TFIM) Hamiltonian Simulation, and Data Re-Uploading Quantum Neural Network (QNN). Together, these benchmarks offer an integral, hardware-agnostic, and impartial methodology to quantify the quality and capabilities of current quantum computers, supporting fair cross-platform comparisons and fostering the development of future performance standards.
Quantum AI is a new way to think about the future of Artificial Intelligence through the merging of two technologies: quantum computing and advanced machine learning. Quantum AI has the potential to provide solutions to computationally difficult problems that traditional systems cannot solve. Quantum AI's continued success will rely on advancements in hybrid quantum-classical solutions, as well as the expansion of quantum hardware; hence, research remains necessary to conduct systematic benchmarks as well as performance analysis in order to determine both how well quantum AI algorithms perform practically, and where their limitations lie. In this chapter, detailed examinations of benchmarks that have been developed specifically for quantum AI are presented alongside evaluative metrics across various key areas of performance including algorithmic efficiency, scalability, noise immunity, and resource use for various types of quantum architectures. The chapter integrates dialogue between theory and practice to demonstrate ways in which performance metrics (i.e., quantum circuit depth, fidelity, convergence behaviour and computational complexity) can effectively compare Quantum AI algorithms to classical algorithms. Particular attention is given to hybrid variational algorithms, quantum-enhanced optimization algorithms, and quantum machine learning. Using benchmark datasets, simulation frameworks and new evaluation methods, there are several challenges in terms of reproducibility, hardware variability and algorithm/hardware co-design that emerge through analysis of the above metrics as well. Aside from focusing on a more broad view of how Performance Evaluation relates to developing more effective Quantum AI Solutions. In addition to viewing Benchmarking as a technical exercise solely, this chapter provides a more holistic view in relation to a human perspective of the situation. This paper outlines how performance evaluation practices can be used as a considerable and well-designed means of bridging the gap between the promise of Quantum Technology at an abstract level, and the ability of Quantum Technology to produce tangible outcomes in the real world by adopting a more comprehensive Governance-oriented approach to Benchmarking Quantum AI Systems. The purpose of this chapter is to assist the growing community of researchers, practitioners, and Strategic Technology Professionals by providing them with a clear and structured framework for Evaluation of Quantum AI Algorithms. There is a significant focus of the chapter on establishing important reference points that organizations can use to determine when an intelligent system has successfully achieved a true and validated quantum advantage. The ultimate goal is to assist in guiding the responsible development and implementation of Quantum AI to enable the Potential Benefits of Quantum AI to be realized, in ways that are meaningful and impactful.
Quantum computers have the potential to provide an advantage over classical computers in a number of areas. Numerous metrics to benchmark the performance of quantum computers, ranging from their individual hardware components to entire applications, have been proposed over the years. Objective comparisons are further hampered in practice as different variations of the same metric are used, and the data disclosed together with a reported metric value is often not sufficient to reproduce the measurements. In this poster, we present our recent article, where we address these challenges by providing a review of metrics and benchmarks for quantum computers [1], including 1) a comprehensive collection of benchmarks allowing holistic comparisons of quantum computers, 2) a consistent format of the definitions across all metrics including a transparent description of the methodology and of the main assumptions and limitations, and 3) a reproducible approach by linking the metrics to open-source software used to evaluate them . We identify five areas where international standardization working groups could be established, namely: i) the identification and agreement on the categories of metrics that comprehensively benchmark device performance; ii) the identification and agreement on a set of well-established metrics that together comprehensively benchmark performance; iii) the identification of metrics specific to hardware platforms, including non-gate-based quantum computers; iv) inter-laboratory comparison studies to develop best practice guides for measurement methodology; and v) agreement on what data and software should be reported together with a metric value to ensure trust, transparency and reproducibility. We provide potential routes to advancing these areas. We expect that this work will accelerate the progress of quantum computing hardware towards quantum advantage. [1] D. Lall et al., A Review and Collection of Metrics and Benchmarks for Quantum Computers: Definitions, Methodologies and Software, arXiv:2502.06717.
A wholly quantum-based metric to assess fidelity between quantum images (QIFM) is proposed. Unlike traditional image quality measures, such as the PSNR metric, the proposed QIFM is formulated as a pixel difference-based measure whose design conforms to the intricacies inherent to quantum image processing (QIP). Results from simulation-based implementation of the QIFM suggest that it is a better technique to quantify fidelity between quantum images. By using m qubits to encode a strip containing multiple cover and test images (each encoded using an additional n qubits) the implementation of the proposed QIFM framework requires 326n basic quantum gates and 26m Toffoli gates to execute. The proposed metric has the potential to offer a better and more effective tool to assess fidelity between images in quantum watermarking, quantum movie aggregation and other applications in QIP.
As quantum computing systems continue to mature, there is an increasing need for benchmarking methodologies that capture performance in terms of meaningful, application-level metrics. In this work, we present a scalable framework for application-level quantum benchmarking that is designed to support internal system evaluation and cross-platform comparison across technology providers. Our framework is guided by a set of core principles, including measurability, simplicity, scalability, and extensibility. We present 13 benchmark families that reflect realistic workloads across multiple domains. This enables the systematic evaluation of the quality of solutions, the total execution time, total used energy, as well as Time-to-Solution. The benchmarks are designed to be reproducible, interpretable across stakeholder groups, and adaptable to evolving system capabilities. The framework aims to bridge the gap between low-level performance metrics and real-world value, providing a unified approach to assessing quantum systems. The resulting benchmarks support development and validation and contribute to the foundation of industry-wide benchmarking standards.