Both Superconducting and Ion-Trap are leading quantum architectures common in the current landscape of the quantum computing field, each with distinct characteristics and operational constraints. Understanding and measuring the underlying \underline{quantumness} of these devices is essential for assessing their readiness for practical applications and guiding future progress and research. Building on earlier work (Meirom, Mor and Weinstein Arxiv 2505.12441), we utilize a benchmarking strategy applicable for comparing these two architectures by measuring "quantumness" directly on optimal sub-chips. Distinct from existing metrics, our approach employs rigorous binary fidelity thresholds derived from the classical limits of state transfer. This enables us to definitively establish quantum advantage of a designated sub-region. Here we apply this quality assurance methodology to platforms from both technologies. This comparison provides a protocol-based evaluation of quantumness advantage, revealing not only the strengths and weaknesses of each tested chip and its sub-chips but also offering a common language for their assessment. By abstracting away technical differences in the final result, we demonstrate a benchmarking strategy that bridges the gap between disparate quantum-circuit technologies, enabling fair performance comparisons and establishing a critical foundation for evaluating future claims of quantum advantage. This work was made possible by policies of two companies who enable independent and objective assessment on their quantum computers and sub-chips. In the name of science, we encourage other companies to emulate the independent qubit availability and the fair pricing which allow researchers to preform such assessments.
Both Superconducting and Ion-Trap are leading quantum architectures common in the current landscape of the quantum computing field, each with distinct characteristics and operational constraints. Understanding and measuring the underlying \underline{quantumness} of these devices is essential for assessing their readiness for practical applications and guiding future progress and research. Building on earlier work (Meirom, Mor and Weinstein Arxiv 2505.12441), we utilize a benchmarking strategy applicable for comparing these two architectures by measuring "quantumness" directly on optimal sub-chips. Distinct from existing metrics, our approach employs rigorous binary fidelity thresholds derived from the classical limits of state transfer. This enables us to definitively establish quantum advantage of a designated sub-region. Here we apply this quality assurance methodology to platforms from both technologies. This comparison provides a protocol-based evaluation of quantumness advantage, revealing not only the strengths and weaknesses of each tested chip and its sub-chips but also offering a common language for their assessment. By abstracting away technical differences in the final result, we demonstrate a benchmarking strategy that bridges the gap between disparate quantum-circuit technologies, enabling fair performance comparisons and establishing a critical foundation for evaluating future claims of quantum advantage. This work was made possible by policies of two companies who enable independent and objective assessment on their quantum computers and sub-chips. In the name of science, we encourage other companies to emulate the independent qubit availability and the fair pricing which allow researchers to preform such assessments.
As quantum computing hardware rapidly advances, objectively evaluating the capabilities and error rates of new processors remains a critical challenge for the field. A clear and realistic understanding of current quantum performance is essential for guiding research priorities and driving meaningful progress. In this work, we apply and extend a protocol-based benchmarking methodology (Meirom, Mor, Weinstein Arxiv 2505.12441) that utilizes well-defined \underline{quantumness} thresholds. By evaluating performance at protocol level rather than the gate level, this approach provides a transparent and intuitive assessment of whether specific quantum processors, or isolated sub-chips within them, can demonstrate a practical quantum advantage. To illustrate the utility of this method, we compare two generations of IBM quantum computers: the older Eagle architecture and the newer Heron architecture. Our findings reveal the genuine operational strengths and limitations of these devices, demonstrating substantial performance improvements in the newer Heron generation. This work was made possible by IBM Quantum policies that enable independent and objective assessment of its quantum computers and sub-chips. We strongly encourage other companies to emulate the independent qubit availability and the fair pricing that allow researchers to perform such assessments.
Abstract The technological development of increasingly larger quantum processors on different quantum platforms raises the problem of how to fairly compare their performance, known as quantum benchmarking of quantum processors. This is a challenge that computer scientists have already faced when comparing classical processors, leading to the development of various mathematical tools to address it, but also to the identification of the limits of this problem. In this work, we briefly review the most important aspects of both classical processor benchmarks and the metrics comprising them, providing precise definitions and analyzing the quality attributes that they should exhibit. Subsequently, we analyze the intrinsic properties that characterize the paradigm of quantum computing and hinder the naive transfer of strategies from classical benchmarking. However, we can still leverage some of the lessons learned such as the quality attributes of a good benchmark such as relevance, reproducibility, fairness, verifiability and usability. Additionally, we review some of the most important metrics and benchmarks for quantum processors proposed in the literature, assessing what quality attributes they fulfill. Finally, we propose general guidelines for quantum benchmarking. These guidelines aim to pave the way for establishing a roadmap towards standardizing the performance evaluation of quantum devices, ultimately leading to the creation of an organization akin to the Standard Performance Evaluation Corporation (SPEC).
The pursuit of a universal, fault-tolerant quantum computer has transformed quantum computing from a predominantly theoretical discipline into a rapidly diversifying hardware ecosystem. This review presents a comprehensive and technically grounded analysis of the principal physical qubit modalities: superconducting circuits, trapped ions, photonic platforms, neutral atoms, semiconductor spin qubits, and topological approaches, focusing on their underlying operating principles, performance characteristics, and system-level constraints. We examine how these qubit technologies are embedded within scalable quantum processor architectures, including control and readout infrastructures, connectivity topologies, and quantum error correction strategies. The review provides a comparative study of key performance metrics, such as coherence times, gate fidelities, error rates, and scalability, reveals fundamental trade-offs between speed, robustness, and operational complexity that shape distinct technological roadmaps and target applications. We further highlight the ongoing transition from noisy intermediate-scale quantum (NISQ) devices toward architectures centred on logical qubits, driven by advances in materials engineering, precision control, and quantum information theory. Together, these developments sketch the emerging routes toward fault-tolerant quantum computing and practical quantum advantage, while offering readers a coherent entry point and reference guide to the current quantum hardware landscape.
Quantum computers are rapidly becoming more capable, with dramatic increases in both qubit count \cite{kim2023evidence} and quality \cite{moses2023race}. Among different hardware approaches, trapped-ion quantum processors are a leading technology for quantum computing, with established high-fidelity operations and architectures with promising scaling. Here, we demonstrate and thoroughly benchmark the IonQ Forte system: configured as a single-chain 30-qubit trapped-ion quantum computer with all-to-all operations. We assess the performance of our quantum computer operation at the component level via direct randomized benchmarking (DRB) across all 30 choose 2 = 435 gate pairs. We then show the results of application-oriented \cite{IonQ_AQ20_2022}\cite{qedcPeerReviewed} benchmarks and show that the system passes the suite of algorithmic qubit (AQ) benchmarks up to #AQ 29. Finally, we use our component-level benchmarking to build a system-level model to predict the application benchmarking data through direct simulation. While we find that the system-level model correlates with the experiment in predicting application circuit performance, we note quantitative discrepancies indicating significant out-of-model errors, leading to higher predicted performance than what is observed. This highlights that as quantum computers move toward larger and higher-quality devices, characterization becomes more challenging, suggesting future work required to push performance further.
Quantum computers have the potential to provide an advantage over classical computers in a number of areas. Numerous metrics to benchmark the performance of quantum computers, ranging from their individual hardware components to entire applications, have been proposed over the years. Objective comparisons are further hampered in practice as different variations of the same metric are used, and the data disclosed together with a reported metric value is often not sufficient to reproduce the measurements. In this poster, we present our recent article, where we address these challenges by providing a review of metrics and benchmarks for quantum computers [1], including 1) a comprehensive collection of benchmarks allowing holistic comparisons of quantum computers, 2) a consistent format of the definitions across all metrics including a transparent description of the methodology and of the main assumptions and limitations, and 3) a reproducible approach by linking the metrics to open-source software used to evaluate them . We identify five areas where international standardization working groups could be established, namely: i) the identification and agreement on the categories of metrics that comprehensively benchmark device performance; ii) the identification and agreement on a set of well-established metrics that together comprehensively benchmark performance; iii) the identification of metrics specific to hardware platforms, including non-gate-based quantum computers; iv) inter-laboratory comparison studies to develop best practice guides for measurement methodology; and v) agreement on what data and software should be reported together with a metric value to ensure trust, transparency and reproducibility. We provide potential routes to advancing these areas. We expect that this work will accelerate the progress of quantum computing hardware towards quantum advantage. [1] D. Lall et al., A Review and Collection of Metrics and Benchmarks for Quantum Computers: Definitions, Methodologies and Software, arXiv:2502.06717.
The limitations of classical computing in solving complex problems in cryptography, materials science, and optimization necessitate the development of a new computational paradigm based on the principles of quantum mechanics. This article aimed to analyze the current state of quantum computing hardware, evaluate the primary challenges to achieving fault tolerance, and project a realistic timeline for its practical application. The methodology involved a systematic review and comparative analysis of publicly available empirical data from peer-reviewed literature and corporate technical roadmaps, employing a framework of key performance indicators such as coherence times, gate fidelities, and qubit counts to assess leading qubit modalities, including superconducting circuits, trapped ions, and photonic systems. The analysis confirmed that while superconducting qubits currently lead in scalability, with demonstrations of quantum supremacy using 53-qubit processors, trapped ion platforms maintain a significant advantage in gate fidelity and coherence times. The central finding identifies decoherence and high error rates as the fundamental barriers, necessitating that current Noisy Intermediate-Scale Quantum (NISQ) devices rely on error mitigation techniques rather than robust quantum error correction. The comparative assessment concludes that no single qubit modality yet fulfills all DiVincenzo criteria for fault tolerance simultaneously. The path to scalable quantum computing is shown to depend on the successful implementation of topological error-correcting codes like the surface code, which currently requires thousands of physical qubits to create a single stable logical qubit. Projections based on current progress suggest that while demonstrations of quantum utility on specific problems are imminent, fully fault-tolerant quantum computers capable of breaking RSA encryption or revolutionizing drug discovery remain a long-term endeavor, likely requiring several more decades of intensive research and development. The practical value of this research lies in its synthesized technical overview, which provides a clear, evidence-based roadmap for researchers, engineers, and policymakers to navigate the technological hurdles and strategic investments required to realize the transformative potential of quantum computing.