Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2312.15836

Optimized experiment design and analysis for fully randomized benchmarking

Alex Kwiatkowski, L. J. Stephenson, H. M. Knaack, Alejandra Collopy, Christina M. Bowers, Dietrich G. Leibfried, D. H. Slichter, Scott Glancy, Emanuel H. Knill

Published Dec 26, 2023• 0 citations
arXivHTMLPDF

Abstract

Randomized benchmarking (RB) is a widely used strategy to assess the quality of available quantum gates in a computational context. RB involves applying known random sequences of gates to an initial state and using the statistics of a final measurement step to determine an effective depolarizing error per step of the sequence, which is a metric of gate quality. Here we investigate the advantages of fully randomized benchmarking, where a new random sequence is drawn for each experimental trial. The advantages of full randomization include smaller confidence intervals on the inferred step error, the ability to use maximum likelihood analysis without heuristics, straightforward optimization of the sequence lengths, and the ability to model and measure behaviors that go beyond the typical assumption of time-independent error rates. We discuss models of time-dependent or non-Markovian errors that generalize the basic RB model of a single exponential decay of the success probability. For any of these models, we implement a concrete protocol to minimize the uncertainty of the estimated parameters given a fixed time constraint on the complete experiment, and we implement a maximum likelihood analysis. We consider several previously published experiments and determine the potential for improvements with optimized full randomization. We experimentally observe such improvements in Clifford randomized benchmarking experiments on a single trapped ion qubit at the National Institute of Standards and Technology (NIST). For an experiment with uniform lengths and intentionally repeated sequences the step error was $2.42^{+0.30}_{-0.22}\times 10^{-5}$, and for an optimized fully randomized experiment of the same total duration the step error was $2.57^{+0.07}_{-0.06}\times 10^{-5}$. We find a substantial decrease in the uncertainty of the step error as a result of optimized fully randomized benchmarking.

DOI
10.48550/arxiv.2312.15836

Related papers

Sampling Strategy Optimization for Randomized Benchmarking

Toshinari Itoko, Rudy Raymond

Randomized benchmarking (RB) is a widely used method for estimating the average fidelity of gates implemented on a quantum computing device. The stochastic error of the average gate fidelity estimated by RB depends on the sampling strategy (i.e., how to sample sequences to be run in the protocol). The sampling strategy is determined by a set of configurable parameters (an RB configuration) that includes Clifford lengths (a list of the number of independent Clifford gates in a sequence) and the number of sequences for each Clifford length. The RB configuration is often chosen heuristically and there has been little research on its best configuration. Therefore, we propose a method for fully optimizing an RB configuration so that the confidence interval of the estimated fidelity is minimized while not increasing the total execution time of sequences. By experiments on real devices, we demonstrate the efficacy of the optimization method against heuristic selection in reducing the variance of the estimated fidelity.

arXiv:2109.07653•IEEE International Conference on Quantum Computing and Engineering (QCE)•Oct 1, 2021•0 citations

Multiqubit randomized benchmarking using few samples

Jonas Helsen, Joel J. Wallman, Steven T. Flammia +1 more

Randomized benchmarking (RB) is an efficient and robust method to characterize gate errors in quantum circuits. Averaging over random sequences of gates leads to estimates of gate errors in terms of the average fidelity. These estimates are isolated from the state preparation and measurement errors that plague other methods such as channel tomography and direct fidelity estimation. A decisive factor in the feasibility of randomized benchmarking is the number of sampled sequences required to obtain rigorous confidence intervals. Previous bounds were either prohibitively loose or required the number of sampled sequences to scale exponentially with the number of qubits in order to obtain a fixed confidence interval at a fixed error rate. Here, we show that, with a small adaptation to the randomized benchmarking procedure, the number of sampled sequences required for a fixed confidence interval is dramatically smaller than could previously be justified. In particular, we show that the number of sampled sequences required is essentially independent of the number of qubits and scales favorably with the average error rate of the system under investigation. We also investigate the fitting procedure inherent to randomized benchmarking in light of our results and find that standard methods such as ordinary least squares optimization can give misleading results. We therefore recommend moving to more sophisticated fitting methods such as iteratively reweighted least squares optimization. Our results bring rigorous randomized benchmarking on systems with many qubits into the realm of experimental feasibility.

arXiv:1701.04299•Physical Review A•Sep 3, 2019•49 citations

A framework for randomized benchmarking over compact groups

Linghang Kong

Characterization of experimental systems is an essential step in developing and improving quantum hardware. A collection of protocols known as Randomized Benchmarking (RB) was developed in the past decade, which provides an efficient way to measure error rates in quantum systems. In a recent paper (arxiv:2010.07974), a general framework for RB was proposed, which encompassed most of the known RB protocols and overcame the limitation on error models in previous works. However, even this general framework has a restriction: it can only be applied to a finite group of gates. This does not meet the need posed by experiments, in particular the demand for benchmarking non-Clifford gates and continuous gate sets on quantum devices. In this work we generalize the RB framework to continuous groups of gates and show that as long as the noise level is reasonably small, the output can be approximated as a linear combination of matrix exponential decays. As an application, we numerically study the fully randomized benchmarking protocol (i.e. RB with the entire unitary group as the gate set) enabled by our proof. This provides a unified way to estimate the gate fidelity for any quantum gate in an experiment.

arXiv:2111.10357•Nov 19, 2021•4 citations

A Theory of Direct Randomized Benchmarking

Anthony Polloreno, Arnaud Carignan-Dugas, Jordan Hines +3 more

Randomized benchmarking (RB) protocols are widely used to measure an average error rate for a set of quantum logic gates. However, the standard version of RB is limited because it only benchmarks a processor's native gates indirectly, by using them in composite n -qubit Clifford gates. Standard RB's reliance on n -qubit Clifford gates restricts it to the few-qubit regime, because the fidelity of a typical composite n -qubit Clifford gate decreases rapidly with increasing n . Furthermore, although standard RB is often used to infer the error rate of native gates, by rescaling standard RB's error per Clifford to an error per native gate, this is an unreliable extrapolation. Direct RB is a method that addresses these limitations of standard RB, by directly benchmarking a customizable gate set, such as a processor's native gates. Here we provide a detailed introduction to direct RB, we discuss how to design direct RB experiments, and we present two complementary theories for direct RB. The first of these theories uses the concept of error propagation or scrambling in random circuits to show that direct RB is reliable for gates that experience stochastic Pauli errors. We prove that the direct RB decay is a single exponential, and that the decay rate is equal to the average infidelity of the benchmarked gates, under broad circumstances. This theory shows that group twirling is not required for reliable RB. Our second theory proves that direct RB is reliable for gates that experience general gate-dependent Markovian errors, using similar techniques to contemporary theories for standard RB. Our two theories for direct RB have complementary regimes of applicability, and they provide complementary perspectives on why direct RB works. Together these theories provide comprehensive guarantees on the reliability of direct RB.

arXiv:2302.13853•Quantum•Sep 5, 2025•5 citations

Concepts and methods for benchmarking quantum computers

Kristine Boone

Quantum computers have the potential to bring about a new age of technology, but in order for them to become a reality, the elephant in the room must be addressed, quantum errors. The complexity of quantum errors grows exponentially as the system size is increased; scalable characterization protocols which work under physically realistic noise are therefore needed to assess and optimize control. \n \nRandomized Benchmarking (RB) is a family of characterization protocols with the requisite qualities. RB protocols are independent of state preparation and measurement (SPAM) errors, and because they extract partial information about the error profiles in the form of figures of merit, they are scalable in the number of qubits. In this thesis, we study these figures of merit under various error models, gate-sets, and sample distributions, to further our understanding of their physical meaning, and to fine-tune the best practice for diagnosing errors in basic gate-sets. \n \nThe first study presented in this thesis looks at the physical meaning of the unitarity under physically realistic gate-dependent noise. Numerically, we demonstrate that under these realistic noise models, the unitarity obtained via Extended RB (XRB) overlaps strongly with the average of the unitarity of the errors over the set of individual operations. \n \nThe second study numerically and analytically proves that the Standard RB (SRB) decay parameter p coincides with the decay parameter of the novel figure of merit gate-set circuit fidelity. We also prove that, in the limit of high-fidelity single-qubit experiments, the disconnect between the average gate fidelity and SRB experimental results is caused by a basis mismatch between the gates and the state-preparation and measurement procedures. \n \nIn the third part of this thesis, we compare the results of two variations of the SRB protocol, one that samples uniformly (SRB) and one that samples non-uniformly (NIST RB) from the Clifford group. We prove that NIST RB produces an exponential fidelity curve that gives different infidelity results than SRB. \n \nThe fourth and final study gives a standard for assessing basic gate-sets. Experimental results are given from the assessment run on super-conducting and ion-trap devices. We also experimentally show how the systematic error bounds can be tightened by accounting for the error's degree of coherence, as well as using weaker randomizing sets.

•Jul 9, 2021•0 citations

What Randomized Benchmarking Actually Measures

Timothy J Proctor, Kenneth Rudinger, Kevin C Young +2 more

Randomized benchmarking (RB) is widely used to measure an error rate of a set of quantum gates, by performing random circuits that would do nothing if the gates were perfect. In the limit of no finite-sampling error, the exponential decay rate of the observable survival probabilities, versus circuit length, yields a single error metric r. For Clifford gates with arbitrary small errors described by process matrices, r was believed to reliably correspond to the mean, over all Clifford gates, of the average gate infidelity between the imperfect gates and their ideal counterparts. We show that this quantity is not a well-defined property of a physical gate set. It depends on the representations used for the imperfect and ideal gates, and the variant typically computed in the literature can differ from r by orders of magnitude. We present new theories of the RB decay that are accurate for all small errors describable by process matrices, and show that the RB decay curve is a simple exponential for all such errors. These theories allow explicit computation of the error rate that RB measures (r), but as far as we can tell it does not correspond to the infidelity of a physically allowed (completely positive) representation of the imperfect gates.

arXiv:1702.01853•Physical Review Letters•Sep 28, 2017•122 citations
Randomized benchmarking (RB) is a widely used strategy to assess the quality of available quantum gates in a computational context. RB involves applying known random sequences of gates to an initial state and using the statistics of a final measurement step to determine an effective depolarizing error per step of the sequence, which is a metric of gate quality. Here we investigate the advantages of fully randomized benchmarking, where a new random sequence is drawn for each experimental trial. The advantages of full randomization include smaller confidence intervals on the inferred step error, the ability to use maximum likelihood analysis without heuristics, straightforward optimization of the sequence lengths, and the ability to model and measure behaviors that go beyond the typical assumption of time-independent error rates. We discuss models of time-dependent or non-Markovian errors that generalize the basic RB model of a single exponential decay of the success probability. For any of these models, we implement a concrete protocol to minimize the uncertainty of the estimated parameters given a fixed time constraint on the complete experiment, and we implement a maximum likelihood analysis. We consider several previously published experiments and determine the potential for improvements with optimized full randomization. We experimentally observe such improvements in Clifford randomized benchmarking experiments on a single trapped ion qubit at the National Institute of Standards and Technology (NIST). For an experiment with uniform lengths and intentionally repeated sequences the step error was 2.42−0.22+0.30×10−52.42^{+0.30}_{-0.22}\times 10^{-5}2.42−0.22+0.30​×10−5, and for an optimized fully randomized experiment of the same total duration the step error was 2.57−0.06+0.07×10−52.57^{+0.07}_{-0.06}\times 10^{-5}2.57−0.06+0.07​×10−5. We find a substantial decrease in the uncertainty of the step error as a result of optimized fully randomized benchmarking.
Randomized benchmarking (RB) is a widely used method for estimating the average fidelity of gates implemented on a quantum computing device. The stochastic error of the average gate fidelity estimated by RB depends on the sampling strategy (i.e., how to sample sequences to be run in the protocol). The sampling strategy is determined by a set of configurable parameters (an RB configuration) that includes Clifford lengths (a list of the number of independent Clifford gates in a sequence) and the number of sequences for each Clifford length. The RB configuration is often chosen heuristically and there has been little research on its best configuration. Therefore, we propose a method for fully optimizing an RB configuration so that the confidence interval of the estimated fidelity is minimized while not increasing the total execution time of sequences. By experiments on real devices, we demonstrate the efficacy of the optimization method against heuristic selection in reducing the variance of the estimated fidelity.
Randomized benchmarking (RB) is an efficient and robust method to characterize gate errors in quantum circuits. Averaging over random sequences of gates leads to estimates of gate errors in terms of the average fidelity. These estimates are isolated from the state preparation and measurement errors that plague other methods such as channel tomography and direct fidelity estimation. A decisive factor in the feasibility of randomized benchmarking is the number of sampled sequences required to obtain rigorous confidence intervals. Previous bounds were either prohibitively loose or required the number of sampled sequences to scale exponentially with the number of qubits in order to obtain a fixed confidence interval at a fixed error rate. Here, we show that, with a small adaptation to the randomized benchmarking procedure, the number of sampled sequences required for a fixed confidence interval is dramatically smaller than could previously be justified. In particular, we show that the number of sampled sequences required is essentially independent of the number of qubits and scales favorably with the average error rate of the system under investigation. We also investigate the fitting procedure inherent to randomized benchmarking in light of our results and find that standard methods such as ordinary least squares optimization can give misleading results. We therefore recommend moving to more sophisticated fitting methods such as iteratively reweighted least squares optimization. Our results bring rigorous randomized benchmarking on systems with many qubits into the realm of experimental feasibility.
Characterization of experimental systems is an essential step in developing and improving quantum hardware. A collection of protocols known as Randomized Benchmarking (RB) was developed in the past decade, which provides an efficient way to measure error rates in quantum systems. In a recent paper (arxiv:2010.07974), a general framework for RB was proposed, which encompassed most of the known RB protocols and overcame the limitation on error models in previous works. However, even this general framework has a restriction: it can only be applied to a finite group of gates. This does not meet the need posed by experiments, in particular the demand for benchmarking non-Clifford gates and continuous gate sets on quantum devices. In this work we generalize the RB framework to continuous groups of gates and show that as long as the noise level is reasonably small, the output can be approximated as a linear combination of matrix exponential decays. As an application, we numerically study the fully randomized benchmarking protocol (i.e. RB with the entire unitary group as the gate set) enabled by our proof. This provides a unified way to estimate the gate fidelity for any quantum gate in an experiment.
Randomized benchmarking (RB) protocols are widely used to measure an average error rate for a set of quantum logic gates. However, the standard version of RB is limited because it only benchmarks a processor's native gates indirectly, by using them in composite n -qubit Clifford gates. Standard RB's reliance on n -qubit Clifford gates restricts it to the few-qubit regime, because the fidelity of a typical composite n -qubit Clifford gate decreases rapidly with increasing n . Furthermore, although standard RB is often used to infer the error rate of native gates, by rescaling standard RB's error per Clifford to an error per native gate, this is an unreliable extrapolation. Direct RB is a method that addresses these limitations of standard RB, by directly benchmarking a customizable gate set, such as a processor's native gates. Here we provide a detailed introduction to direct RB, we discuss how to design direct RB experiments, and we present two complementary theories for direct RB. The first of these theories uses the concept of error propagation or scrambling in random circuits to show that direct RB is reliable for gates that experience stochastic Pauli errors. We prove that the direct RB decay is a single exponential, and that the decay rate is equal to the average infidelity of the benchmarked gates, under broad circumstances. This theory shows that group twirling is not required for reliable RB. Our second theory proves that direct RB is reliable for gates that experience general gate-dependent Markovian errors, using similar techniques to contemporary theories for standard RB. Our two theories for direct RB have complementary regimes of applicability, and they provide complementary perspectives on why direct RB works. Together these theories provide comprehensive guarantees on the reliability of direct RB.
Quantum computers have the potential to bring about a new age of technology, but in order for them to become a reality, the elephant in the room must be addressed, quantum errors. The complexity of quantum errors grows exponentially as the system size is increased; scalable characterization protocols which work under physically realistic noise are therefore needed to assess and optimize control. \n \nRandomized Benchmarking (RB) is a family of characterization protocols with the requisite qualities. RB protocols are independent of state preparation and measurement (SPAM) errors, and because they extract partial information about the error profiles in the form of figures of merit, they are scalable in the number of qubits. In this thesis, we study these figures of merit under various error models, gate-sets, and sample distributions, to further our understanding of their physical meaning, and to fine-tune the best practice for diagnosing errors in basic gate-sets. \n \nThe first study presented in this thesis looks at the physical meaning of the unitarity under physically realistic gate-dependent noise. Numerically, we demonstrate that under these realistic noise models, the unitarity obtained via Extended RB (XRB) overlaps strongly with the average of the unitarity of the errors over the set of individual operations. \n \nThe second study numerically and analytically proves that the Standard RB (SRB) decay parameter p coincides with the decay parameter of the novel figure of merit gate-set circuit fidelity. We also prove that, in the limit of high-fidelity single-qubit experiments, the disconnect between the average gate fidelity and SRB experimental results is caused by a basis mismatch between the gates and the state-preparation and measurement procedures. \n \nIn the third part of this thesis, we compare the results of two variations of the SRB protocol, one that samples uniformly (SRB) and one that samples non-uniformly (NIST RB) from the Clifford group. We prove that NIST RB produces an exponential fidelity curve that gives different infidelity results than SRB. \n \nThe fourth and final study gives a standard for assessing basic gate-sets. Experimental results are given from the assessment run on super-conducting and ion-trap devices. We also experimentally show how the systematic error bounds can be tightened by accounting for the error's degree of coherence, as well as using weaker randomizing sets.
Randomized benchmarking (RB) is widely used to measure an error rate of a set of quantum gates, by performing random circuits that would do nothing if the gates were perfect. In the limit of no finite-sampling error, the exponential decay rate of the observable survival probabilities, versus circuit length, yields a single error metric r. For Clifford gates with arbitrary small errors described by process matrices, r was believed to reliably correspond to the mean, over all Clifford gates, of the average gate infidelity between the imperfect gates and their ideal counterparts. We show that this quantity is not a well-defined property of a physical gate set. It depends on the representations used for the imperfect and ideal gates, and the variant typically computed in the literature can differ from r by orders of magnitude. We present new theories of the RB decay that are accurate for all small errors describable by process matrices, and show that the RB decay curve is a simple exponential for all such errors. These theories allow explicit computation of the error rate that RB measures (r), but as far as we can tell it does not correspond to the infidelity of a physically allowed (completely positive) representation of the imperfect gates.