Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2305.08796 · IEEE International Conference on Quantum Computing and Engineering (QCE)

Predictive Models from Quantum Computer Benchmarks

Daniel Hothem, Jordan Hines, Karthik Nataraj, Robin Blume-Kohout, Timothy J Proctor

Published Sep 17, 2023• 7 citations
arXivHTMLPDF

Abstract

Holistic benchmarks for quantum computers are essential for testing and summarizing the performance of quantum hardware. However, holistic benchmarks-such as algorithmic or randomized benchmarks-typically do not predict a processor's performance on circuits outside the benchmark's necessarily very limited set of test circuits. In this paper, we introduce a general framework for building predictive models from benchmarking data using capability models. Capability models can be fit to many kinds of benchmarking data and used for a variety of predictive tasks. We demonstrate this flexibility with two case studies. In the first case study, we predict circuit (i) process fidelities and (ii) success probabilities by fitting error rates models to two kinds of volumetric benchmarking data. Error rates models are simple yet versatile capability models which assign effective error rates to individual gates, or more general circuit components. In the second case study, we construct a capability model for predicting circuit success probabilities by applying transfer learning to ResNet50, a neural network trained for image classification. Our case studies use data from cloud-accessible quantum computers and simulations of noisy quantum computers.

DOI
10.1109/qce57702.2023.00086

Related papers

Learning a Quantum Computer's Capability

Daniel Hothem, Kevin C Young, Tommie Catanach +1 more

Accurately predicting a quantum computer's capability—which circuits it can run and how well it can run them—is a foundational goal of quantum characterization and benchmarking. As modern quantum computers become increasingly hard to simulate, we must develop accurate and scalable predictive capability models to help researchers and stakeholders decide which quantum computers to build and use. In this work, we propose a hardware-agnostic method to efficiently construct scalable predictive models of a quantum computer's capability for almost any class of circuits, and demonstrate our method using convolutional neural networks (CNNs). Our CNN-based approach works by efficiently representing a circuit as a three-dimensional tensor and then using a CNN to predict its success rate. Our CNN capability models obtain approximately a 1% average absolute prediction error when modeling processors experiencing both Markovian and non-Markovian stochastic Pauli errors. We also apply our CNNs to model the capabilities of cloud-access quantum computing systems, obtaining moderate prediction accuracy (average absolute error around$2-5\%$), and we highlight the challenges to building better neural network capability models.

arXiv:2304.10650•IEEE Transactions on Quantum Engineering•Jan 1, 2024•2 citations

Featuremetric benchmarking: Quantum computer benchmarks based on circuit features

Timothy J Proctor, Tran, Anh, Xingxin Liu +4 more

Benchmarks that concisely summarize the performance of many-qubit quantum computers are essential for measuring progress towards the goal of useful quantum computation. In this work, we present a benchmarking framework that is based on quantifying how a quantum computer's performance on quantum circuits varies as a function of features of those circuits, such as circuit depth, width, two-qubit gate density, problem input size, or algorithmic depth. Our featuremetric benchmarking framework generalizes volumetric benchmarking -- a widely-used methodology that quantifies performance versus circuit width and depth -- and we show that it enables richer and more faithful models of quantum computer performance. We demonstrate featuremetric benchmarking with example benchmarks run on IBM Q and IonQ systems of up to 27 qubits, and we show how to produce performance summaries from the data using Gaussian process regression. Our data analysis methods are also of interest in the special case of volumetric benchmarking, as they enable the creation of intuitive two-dimensional capability regions using data from few circuits.

arXiv:2504.12575•Apr 17, 2025•0 citations

Constructing and Benchmarking Noise Models for Quantum Computing

Tom Weber

Context: The main obstacle of quantum computing on its way to scalability is the noisy nature of current hardware. Various types of errors lead to incorrect computational results,and measures such as quantum error mitigation are needed to counteract these errors. Many error mitigation techniques require a thorough understanding of the noise and its impact on computations. Therefore, accurate noise models are indispensable to facilitate meaningful quantum computing in the near future. They also enable realistic simulations of quantum devices. Since these devices are currently limited and quantum computing is expensive, researchers typically rely on simulations for testing their algorithms.Objective: In this thesis, we aim to construct a realistic noise model for quantum computing and optimise its model parameters. The model construction should build on the underlying physical processes, and the number of parameters should scale well with the system size. We also address the evaluation and comparison of different noise models by benchmarks using quantum circuits specific to real-world applications. Finally, we aim to develop useful graphical representations of noise models based on the existing quantum circuit model.Method: We present a benchmarking framework for quantum computing noise models and evaluate it based on benchmarking quality attributes from the literature. Moreover, we construct a quantum computing noise model and develop a training procedure for its parameters. We analyse the parameters of the trained noise model and the impact ofdifferent types of errors on computations. We benchmark the noise model with the above approach and compare it to a noise model provided by IBM’s software development kit Qiskit. Finally, we develop an extension of the quantum circuit model to represent noise channels.Result: The benchmarks performed in this thesis show that our noise model predicts noisy hardware behaviour of IBM’s ibmq_manila quantum device equally well as the Qiskit model, if not better, based on quantum circuits commonly used for variational quantum algorithms. Within the trained noise model, readout error has the most detrimental impact on computations. Our benchmarking approach satisfies relevant quality criteria by choosing suitable quantum circuits and objective functions to compare model predictionsto hardware data. Moreover, the quantum circuit model can be extended meaningfully by noise channels.Conclusion: Our benchmarking approach is suitable for evaluating and comparing quantum computing noise models. In future work, volumetric benchmarks should be performed for various noise models and application contexts. The parameter optimisation based on learning with training circuits yields accurate noise models and shows the importance of mitigating measurement error. Different optimisation algorithms and quantum circuits should be explored, and more types of noise should be incorporated into the noise model.

•Jan 12, 2026•0 citations

Predicting the success rates of quantum circuits with artificial neural networks.

Daniel Hothem, Thomas Catanach, Kevin C Young +1 more

Current quantum computers are noisy and error-prone.As devices grow in size, we need scalable methods for predicting the performance of a given device.In this work, we explore the potential of neural networks to play this role.We demonstrate a convolutional neural network's ability to predict circuit success rates under both simulated and experimental error models; in each case outperforming non-neural network models that are based on per gate error rates.We also experimentally investigate how a convolutional network's ability to predict success rates varies as a function of dataset size and measurement accuracy, achieving noticeable improvements in performance on simulated data as measurement precision increases.Finally, we present proof-of-concept work detailing a convolutional neural network's ability to handle non-Markovian noise on wide circuits (greater than 25 qubits), a regime in which traditional techniques become inaccurate and highly expensive.

•Oct 1, 2022•0 citations

Predicting the success rates of quantum circuits with artificial neural networks.

Daniel Hothem, Thomas Catanach, Timothy J Proctor +1 more

Current quantum computers are noisy and error-prone.As devices grow in size, we need scalable methods for predicting the performance of a given device.In this work, we explore the potential of neural networks to play this role.We demonstrate a convolutional neural network's ability to predict circuit success rates under both simulated and experimental error models; in each case outperforming non-neural network models that are based on per gate error rates.We also experimentally investigate how a convolutional network's ability to predict success rates varies as a function of dataset size and measurement accuracy, achieving noticeable improvements in performance on simulated data as measurement precision increases.Finally, we present proof-of-concept work detailing a convolutional neural network's ability to handle non-Markovian noise on wide circuits (greater than 25 qubits), a regime in which traditional techniques become inaccurate and highly expensive.

•Nov 1, 2022•0 citations

Reliability Modeling of NISQ- Era Quantum Computers

Ji Liu, Huiyang Zhou

Recent developments in quantum computers have been pushing up the number of qubits. However, the state-of-the-art Noisy Intermediate Scale Quantum (NISQ) computers still do not have enough qubits to accommodate the error correction circuit. Noise in quantum gates limits the reliability of quantum circuits. To characterize the noise effects, prior methods such as process tomography, gateset tomography and randomized benchmarking have been proposed. However, the challenge is that these methods do not scale well with the number of qubits. Noise models based on the understanding of underneath physics have also been proposed to study different kinds of noise in quantum computers. The difficulty is that there is no widely accepted noise model that incorporates all different kinds of errors. The realworld errors can be very complicated and it remains an active area of research to produce accurate noise models. In this paper, instead of using noise models to estimate the reliability, which is measured with success rates or inference strength, we treat the NISQ quantum computer as a black box. We use several quantum circuit characteristics such as the number of qubits, circuit depth, the number of CNOT gates, and the connection topology of the quantum computer as inputs to the black box and derive a reliability estimation model using (1) polynomial fitting and (2) a shallow neural network. We propose randomized benchmarks with random numbers of qubits and basic gates to generate a large data set for neural network training. We show that the estimated reliability from our black-box model outperforms the noise models from Qiskit. We also showcase that our black-box model can be used to guide quantum circuit optimization at compile time.

IEEE International Symposium on Workload Characterization•Oct 1, 2020•45 citations
Holistic benchmarks for quantum computers are essential for testing and summarizing the performance of quantum hardware. However, holistic benchmarks-such as algorithmic or randomized benchmarks-typically do not predict a processor's performance on circuits outside the benchmark's necessarily very limited set of test circuits. In this paper, we introduce a general framework for building predictive models from benchmarking data using capability models. Capability models can be fit to many kinds of benchmarking data and used for a variety of predictive tasks. We demonstrate this flexibility with two case studies. In the first case study, we predict circuit (i) process fidelities and (ii) success probabilities by fitting error rates models to two kinds of volumetric benchmarking data. Error rates models are simple yet versatile capability models which assign effective error rates to individual gates, or more general circuit components. In the second case study, we construct a capability model for predicting circuit success probabilities by applying transfer learning to ResNet50, a neural network trained for image classification. Our case studies use data from cloud-accessible quantum computers and simulations of noisy quantum computers.
Accurately predicting a quantum computer's capability—which circuits it can run and how well it can run them—is a foundational goal of quantum characterization and benchmarking. As modern quantum computers become increasingly hard to simulate, we must develop accurate and scalable predictive capability models to help researchers and stakeholders decide which quantum computers to build and use. In this work, we propose a hardware-agnostic method to efficiently construct scalable predictive models of a quantum computer's capability for almost any class of circuits, and demonstrate our method using convolutional neural networks (CNNs). Our CNN-based approach works by efficiently representing a circuit as a three-dimensional tensor and then using a CNN to predict its success rate. Our CNN capability models obtain approximately a 1% average absolute prediction error when modeling processors experiencing both Markovian and non-Markovian stochastic Pauli errors. We also apply our CNNs to model the capabilities of cloud-access quantum computing systems, obtaining moderate prediction accuracy (average absolute error around2−5%2-5\%2−5%), and we highlight the challenges to building better neural network capability models.
Benchmarks that concisely summarize the performance of many-qubit quantum computers are essential for measuring progress towards the goal of useful quantum computation. In this work, we present a benchmarking framework that is based on quantifying how a quantum computer's performance on quantum circuits varies as a function of features of those circuits, such as circuit depth, width, two-qubit gate density, problem input size, or algorithmic depth. Our featuremetric benchmarking framework generalizes volumetric benchmarking -- a widely-used methodology that quantifies performance versus circuit width and depth -- and we show that it enables richer and more faithful models of quantum computer performance. We demonstrate featuremetric benchmarking with example benchmarks run on IBM Q and IonQ systems of up to 27 qubits, and we show how to produce performance summaries from the data using Gaussian process regression. Our data analysis methods are also of interest in the special case of volumetric benchmarking, as they enable the creation of intuitive two-dimensional capability regions using data from few circuits.
Context: The main obstacle of quantum computing on its way to scalability is the noisy nature of current hardware. Various types of errors lead to incorrect computational results,and measures such as quantum error mitigation are needed to counteract these errors. Many error mitigation techniques require a thorough understanding of the noise and its impact on computations. Therefore, accurate noise models are indispensable to facilitate meaningful quantum computing in the near future. They also enable realistic simulations of quantum devices. Since these devices are currently limited and quantum computing is expensive, researchers typically rely on simulations for testing their algorithms.Objective: In this thesis, we aim to construct a realistic noise model for quantum computing and optimise its model parameters. The model construction should build on the underlying physical processes, and the number of parameters should scale well with the system size. We also address the evaluation and comparison of different noise models by benchmarks using quantum circuits specific to real-world applications. Finally, we aim to develop useful graphical representations of noise models based on the existing quantum circuit model.Method: We present a benchmarking framework for quantum computing noise models and evaluate it based on benchmarking quality attributes from the literature. Moreover, we construct a quantum computing noise model and develop a training procedure for its parameters. We analyse the parameters of the trained noise model and the impact ofdifferent types of errors on computations. We benchmark the noise model with the above approach and compare it to a noise model provided by IBM’s software development kit Qiskit. Finally, we develop an extension of the quantum circuit model to represent noise channels.Result: The benchmarks performed in this thesis show that our noise model predicts noisy hardware behaviour of IBM’s ibmq_manila quantum device equally well as the Qiskit model, if not better, based on quantum circuits commonly used for variational quantum algorithms. Within the trained noise model, readout error has the most detrimental impact on computations. Our benchmarking approach satisfies relevant quality criteria by choosing suitable quantum circuits and objective functions to compare model predictionsto hardware data. Moreover, the quantum circuit model can be extended meaningfully by noise channels.Conclusion: Our benchmarking approach is suitable for evaluating and comparing quantum computing noise models. In future work, volumetric benchmarks should be performed for various noise models and application contexts. The parameter optimisation based on learning with training circuits yields accurate noise models and shows the importance of mitigating measurement error. Different optimisation algorithms and quantum circuits should be explored, and more types of noise should be incorporated into the noise model.
Current quantum computers are noisy and error-prone.As devices grow in size, we need scalable methods for predicting the performance of a given device.In this work, we explore the potential of neural networks to play this role.We demonstrate a convolutional neural network's ability to predict circuit success rates under both simulated and experimental error models; in each case outperforming non-neural network models that are based on per gate error rates.We also experimentally investigate how a convolutional network's ability to predict success rates varies as a function of dataset size and measurement accuracy, achieving noticeable improvements in performance on simulated data as measurement precision increases.Finally, we present proof-of-concept work detailing a convolutional neural network's ability to handle non-Markovian noise on wide circuits (greater than 25 qubits), a regime in which traditional techniques become inaccurate and highly expensive.
Current quantum computers are noisy and error-prone.As devices grow in size, we need scalable methods for predicting the performance of a given device.In this work, we explore the potential of neural networks to play this role.We demonstrate a convolutional neural network's ability to predict circuit success rates under both simulated and experimental error models; in each case outperforming non-neural network models that are based on per gate error rates.We also experimentally investigate how a convolutional network's ability to predict success rates varies as a function of dataset size and measurement accuracy, achieving noticeable improvements in performance on simulated data as measurement precision increases.Finally, we present proof-of-concept work detailing a convolutional neural network's ability to handle non-Markovian noise on wide circuits (greater than 25 qubits), a regime in which traditional techniques become inaccurate and highly expensive.
Recent developments in quantum computers have been pushing up the number of qubits. However, the state-of-the-art Noisy Intermediate Scale Quantum (NISQ) computers still do not have enough qubits to accommodate the error correction circuit. Noise in quantum gates limits the reliability of quantum circuits. To characterize the noise effects, prior methods such as process tomography, gateset tomography and randomized benchmarking have been proposed. However, the challenge is that these methods do not scale well with the number of qubits. Noise models based on the understanding of underneath physics have also been proposed to study different kinds of noise in quantum computers. The difficulty is that there is no widely accepted noise model that incorporates all different kinds of errors. The realworld errors can be very complicated and it remains an active area of research to produce accurate noise models. In this paper, instead of using noise models to estimate the reliability, which is measured with success rates or inference strength, we treat the NISQ quantum computer as a black box. We use several quantum circuit characteristics such as the number of qubits, circuit depth, the number of CNOT gates, and the connection topology of the quantum computer as inputs to the black box and derive a reliability estimation model using (1) polynomial fitting and (2) a shallow neural network. We propose randomized benchmarks with random numbers of qubits and basic gates to generate a large data set for neural network training. We show that the estimated reliability from our black-box model outperforms the noise models from Qiskit. We also showcase that our black-box model can be used to guide quantum circuit optimization at compile time.