Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

IEEE Computer Society Annual Symposium on VLSI

Reproducibility in Quantum Computing

Samudra Dasgupta, Travis S. Humble

Published Jul 1, 2021• 6 citations
Publisher

Abstract

Reproducibility is important for validating the performance of applications in quantum computing as a measure of consistency in computation. Current noisy, intermediate-scale devices quantum (NISQ) devices are strongly affected by intrinsic noise that leads to a variety of computational error mechanisms. Here we assess reproducibility of NISQ computing by focusing on a specific simple error mechanism that arises during noisy readout. Using an asymmetric channel for binary readout, we develop an analytic bound on the Hellinger distance between computational outputs for different readout parameters. We validate this model using characterization and testing of the IBM toronto device, which displays a range of readout parameters. We find that to ensure reproducibility, one must avoid using register elements characterized by fidelity asymmetry exceeding a threshold.

DOI
10.1109/isvlsi51109.2021.00090

Related papers

Characterizing the Reproducibility of Noisy Quantum Circuits

Samudra Dasgupta, Travis S. Humble

The ability of a quantum computer to reproduce or replicate the results of a quantum circuit is a key concern for verifying and validating applications of quantum computing. Statistical variations in circuit outcomes that arise from ill-characterized fluctuations in device noise may lead to computational errors and irreproducible results. While device characterization offers a direct assessment of noise, an outstanding concern is how such metrics bound the reproducibility of a given quantum circuit. Here, we first directly assess the reproducibility of a noisy quantum circuit, in terms of the Hellinger distance between the computational results, and then we show that device characterization offers an analytic bound on the observed variability. We validate the method using an ensemble of single qubit test circuits, executed on a superconducting transmon processor with well-characterized readout and gate error rates. The resulting description for circuit reproducibility, in terms of a composite device parameter, is confirmed to define an upper bound on the observed Hellinger distance, across the variable test circuits. This predictive correlation between circuit outcomes and device characterization offers an efficient method for assessing the reproducibility of noisy quantum circuits.

arXiv:2204.01529•Entropy•Feb 5, 2022•31 citations

Stability of noisy quantum computing devices

Samudra Dasgupta, Travis S. Humble

Noisy, intermediate-scale quantum (NISQ) computing devices offer opportunities to test the principles of quantum computing but are prone to errors arising from various sources of noise. Fluctuations in the noise itself lead to unstable devices that undermine the reproducibility of NISQ results. Here we characterize the reliability of NISQ devices by quantifying the stability of essential performance metrics. Using the Hellinger distance, we quantify the similarity between experimental characterizations of several NISQ devices by comparing gate fidelities, duty cycles, and register addressability across temporal and spatial scales. Our observations collected over 22 months reveal large fluctuations in each metric that underscore the limited scales on which current NISQ devices may be considered reliable.

arXiv:2105.09472•May 20, 2021•17 citations

Impact of Unreliable Devices on Stability of Quantum Computations

Samudra Dasgupta, Travis S. Humble

Noisy intermediate-scale quantum (NISQ) devices are valuable platforms for testing the tenets of quantum computing, but these devices are susceptible to errors arising from de-coherence, leakage, cross-talk, and other sources of noise. This raises concerns regarding the stability of results when using NISQ devices since strategies for mitigating errors generally require well-characterized and stationary error models. Here, we quantify the reliability of NISQ devices by assessing the necessary conditions for generating stable results within a given tolerance. We use similarity metrics derived from device characterization data to derive and validate bounds on the stability of a 5-qubit implementation of the Bernstein-Vazirani algorithm. Simulation experiments conducted with noise data from IBM washington, spanning January 2022 to April 2023, revealed that the reliability metric fluctuated between 41% and 92%. This variation significantly surpasses the maximum allowable threshold of 2.2% needed for stable outcomes. Consequently, the device proved unreliable for consistently reproducing the statistical mean in the context of the Bernstein-Vazirani circuit.

arXiv:2307.06833•ACM Transactions on Quantum Computing•Jul 29, 2024•7 citations

Experimental Evaluation of NISQ Quantum Computers: Error Measurement, Characterization, and Implications

Tirthak Patel, Abhay Potharaju, Baolin Li +2 more

Noisy Intermediate-Scale Quantum (NISQ) computers are being increasingly used for executing early-stage quantum programs to establish the practical realizability of existing quantum algorithms. These quantum programs have uses cases in the realm of high-performance computing ranging from molecular chemistry and physics simulations to addressing NP-complete optimization problems. However, NISQ devices are prone to multiple types of errors, which affect the fidelity and reproducibility of the program execution. As the technology is still primitive, our understanding of these quantum machines and their error characteristics is limited. To bridge that understanding gap, this is the first work to provide a systematic and rich experimental evaluation of IBM Quantum Experience (QX) quantum computers of different scales and topologies. Our experimental evaluation uncovers multiple important and interesting aspects of benchmarking and evaluating quantum program on NISQ machines. We have open-sourced our experimental framework and dataset to help accelerate the evaluation of quantum computing systems.

International Conference for High Performance Computing, Networking, Storage and Analysis (SC)•Nov 1, 2020•37 citations

Quantum Noise And Measuring Quantum Distance For NISQ Circuits

Sakibul Islam Sazzad, Protik Nag

In this article, we analyzed the performance of several classes of quantum circuits under various errors. We executed NISQ circuits on both noisy simulator and real hardware backend publicly available from IBM Quantum. On simulation, we created noise models with both individual errors and combination of multiple errors to quantify how each type of error impact performance of quantum circuits. We utilized statistical distances such as total variation distance (TVD), Jensen-Shannon divergence (JSD), and Hellinger distance, to measure the deviation of noisy output from ideal output. Finally, we compare the noisy outputs from the simulator and outputs from the real hardware to showcase the gap between simulation and real device experiment. This gap points to the limitation of present noise models failure to exhibit the behavior of real hardware.

•Dec 30, 2022•1 citation

Quantum Noise And Measuring Quantum Distance For NISQ Circuits

Sakibul Islam Sazzad, Protik Nag

In this article, we analyzed the performance of several classes of quantum circuits under various errors. We executed NISQ circuits on both noisy simulator and real hardware backend publicly available from IBM Quantum. On simulation, we created noise models with both individual errors and combination of multiple errors to quantify how each type of error impact performance of quantum circuits. We utilized statistical distances such as total variation distance (TVD), Jensen-Shannon divergence (JSD), and Hellinger distance, to measure the deviation of noisy output from ideal output. Finally, we compare the noisy outputs from the simulator and outputs from the real hardware to showcase the gap between simulation and real device experiment. This gap points to the limitation of present noise models failure to exhibit the behavior of real hardware.

•Dec 30, 2022•0 citations
Reproducibility is important for validating the performance of applications in quantum computing as a measure of consistency in computation. Current noisy, intermediate-scale devices quantum (NISQ) devices are strongly affected by intrinsic noise that leads to a variety of computational error mechanisms. Here we assess reproducibility of NISQ computing by focusing on a specific simple error mechanism that arises during noisy readout. Using an asymmetric channel for binary readout, we develop an analytic bound on the Hellinger distance between computational outputs for different readout parameters. We validate this model using characterization and testing of the IBM toronto device, which displays a range of readout parameters. We find that to ensure reproducibility, one must avoid using register elements characterized by fidelity asymmetry exceeding a threshold.
The ability of a quantum computer to reproduce or replicate the results of a quantum circuit is a key concern for verifying and validating applications of quantum computing. Statistical variations in circuit outcomes that arise from ill-characterized fluctuations in device noise may lead to computational errors and irreproducible results. While device characterization offers a direct assessment of noise, an outstanding concern is how such metrics bound the reproducibility of a given quantum circuit. Here, we first directly assess the reproducibility of a noisy quantum circuit, in terms of the Hellinger distance between the computational results, and then we show that device characterization offers an analytic bound on the observed variability. We validate the method using an ensemble of single qubit test circuits, executed on a superconducting transmon processor with well-characterized readout and gate error rates. The resulting description for circuit reproducibility, in terms of a composite device parameter, is confirmed to define an upper bound on the observed Hellinger distance, across the variable test circuits. This predictive correlation between circuit outcomes and device characterization offers an efficient method for assessing the reproducibility of noisy quantum circuits.
Noisy, intermediate-scale quantum (NISQ) computing devices offer opportunities to test the principles of quantum computing but are prone to errors arising from various sources of noise. Fluctuations in the noise itself lead to unstable devices that undermine the reproducibility of NISQ results. Here we characterize the reliability of NISQ devices by quantifying the stability of essential performance metrics. Using the Hellinger distance, we quantify the similarity between experimental characterizations of several NISQ devices by comparing gate fidelities, duty cycles, and register addressability across temporal and spatial scales. Our observations collected over 22 months reveal large fluctuations in each metric that underscore the limited scales on which current NISQ devices may be considered reliable.
Noisy intermediate-scale quantum (NISQ) devices are valuable platforms for testing the tenets of quantum computing, but these devices are susceptible to errors arising from de-coherence, leakage, cross-talk, and other sources of noise. This raises concerns regarding the stability of results when using NISQ devices since strategies for mitigating errors generally require well-characterized and stationary error models. Here, we quantify the reliability of NISQ devices by assessing the necessary conditions for generating stable results within a given tolerance. We use similarity metrics derived from device characterization data to derive and validate bounds on the stability of a 5-qubit implementation of the Bernstein-Vazirani algorithm. Simulation experiments conducted with noise data from IBM washington, spanning January 2022 to April 2023, revealed that the reliability metric fluctuated between 41% and 92%. This variation significantly surpasses the maximum allowable threshold of 2.2% needed for stable outcomes. Consequently, the device proved unreliable for consistently reproducing the statistical mean in the context of the Bernstein-Vazirani circuit.
Noisy Intermediate-Scale Quantum (NISQ) computers are being increasingly used for executing early-stage quantum programs to establish the practical realizability of existing quantum algorithms. These quantum programs have uses cases in the realm of high-performance computing ranging from molecular chemistry and physics simulations to addressing NP-complete optimization problems. However, NISQ devices are prone to multiple types of errors, which affect the fidelity and reproducibility of the program execution. As the technology is still primitive, our understanding of these quantum machines and their error characteristics is limited. To bridge that understanding gap, this is the first work to provide a systematic and rich experimental evaluation of IBM Quantum Experience (QX) quantum computers of different scales and topologies. Our experimental evaluation uncovers multiple important and interesting aspects of benchmarking and evaluating quantum program on NISQ machines. We have open-sourced our experimental framework and dataset to help accelerate the evaluation of quantum computing systems.
In this article, we analyzed the performance of several classes of quantum circuits under various errors. We executed NISQ circuits on both noisy simulator and real hardware backend publicly available from IBM Quantum. On simulation, we created noise models with both individual errors and combination of multiple errors to quantify how each type of error impact performance of quantum circuits. We utilized statistical distances such as total variation distance (TVD), Jensen-Shannon divergence (JSD), and Hellinger distance, to measure the deviation of noisy output from ideal output. Finally, we compare the noisy outputs from the simulator and outputs from the real hardware to showcase the gap between simulation and real device experiment. This gap points to the limitation of present noise models failure to exhibit the behavior of real hardware.
In this article, we analyzed the performance of several classes of quantum circuits under various errors. We executed NISQ circuits on both noisy simulator and real hardware backend publicly available from IBM Quantum. On simulation, we created noise models with both individual errors and combination of multiple errors to quantify how each type of error impact performance of quantum circuits. We utilized statistical distances such as total variation distance (TVD), Jensen-Shannon divergence (JSD), and Hellinger distance, to measure the deviation of noisy output from ideal output. Finally, we compare the noisy outputs from the simulator and outputs from the real hardware to showcase the gap between simulation and real device experiment. This gap points to the limitation of present noise models failure to exhibit the behavior of real hardware.