Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2109.07653 · IEEE International Conference on Quantum Computing and Engineering (QCE)

Sampling Strategy Optimization for Randomized Benchmarking

Toshinari Itoko, Rudy Raymond

Published Oct 1, 2021• 0 citations
arXivHTMLPDF

Abstract

Randomized benchmarking (RB) is a widely used method for estimating the average fidelity of gates implemented on a quantum computing device. The stochastic error of the average gate fidelity estimated by RB depends on the sampling strategy (i.e., how to sample sequences to be run in the protocol). The sampling strategy is determined by a set of configurable parameters (an RB configuration) that includes Clifford lengths (a list of the number of independent Clifford gates in a sequence) and the number of sequences for each Clifford length. The RB configuration is often chosen heuristically and there has been little research on its best configuration. Therefore, we propose a method for fully optimizing an RB configuration so that the confidence interval of the estimated fidelity is minimized while not increasing the total execution time of sequences. By experiments on real devices, we demonstrate the efficacy of the optimization method against heuristic selection in reducing the variance of the estimated fidelity.

DOI
10.1109/qce52317.2021.00036

Related papers

Optimized experiment design and analysis for fully randomized benchmarking

Alex Kwiatkowski, L. J. Stephenson, H. M. Knaack +6 more

Randomized benchmarking (RB) is a widely used strategy to assess the quality of available quantum gates in a computational context. RB involves applying known random sequences of gates to an initial state and using the statistics of a final measurement step to determine an effective depolarizing error per step of the sequence, which is a metric of gate quality. Here we investigate the advantages of fully randomized benchmarking, where a new random sequence is drawn for each experimental trial. The advantages of full randomization include smaller confidence intervals on the inferred step error, the ability to use maximum likelihood analysis without heuristics, straightforward optimization of the sequence lengths, and the ability to model and measure behaviors that go beyond the typical assumption of time-independent error rates. We discuss models of time-dependent or non-Markovian errors that generalize the basic RB model of a single exponential decay of the success probability. For any of these models, we implement a concrete protocol to minimize the uncertainty of the estimated parameters given a fixed time constraint on the complete experiment, and we implement a maximum likelihood analysis. We consider several previously published experiments and determine the potential for improvements with optimized full randomization. We experimentally observe such improvements in Clifford randomized benchmarking experiments on a single trapped ion qubit at the National Institute of Standards and Technology (NIST). For an experiment with uniform lengths and intentionally repeated sequences the step error was $2.42^{+0.30}_{-0.22}\times 10^{-5}$, and for an optimized fully randomized experiment of the same total duration the step error was $2.57^{+0.07}_{-0.06}\times 10^{-5}$. We find a substantial decrease in the uncertainty of the step error as a result of optimized fully randomized benchmarking.

arXiv:2312.15836•Dec 26, 2023•0 citations

Multiqubit randomized benchmarking using few samples

Jonas Helsen, Joel J. Wallman, Steven T. Flammia +1 more

Randomized benchmarking (RB) is an efficient and robust method to characterize gate errors in quantum circuits. Averaging over random sequences of gates leads to estimates of gate errors in terms of the average fidelity. These estimates are isolated from the state preparation and measurement errors that plague other methods such as channel tomography and direct fidelity estimation. A decisive factor in the feasibility of randomized benchmarking is the number of sampled sequences required to obtain rigorous confidence intervals. Previous bounds were either prohibitively loose or required the number of sampled sequences to scale exponentially with the number of qubits in order to obtain a fixed confidence interval at a fixed error rate. Here, we show that, with a small adaptation to the randomized benchmarking procedure, the number of sampled sequences required for a fixed confidence interval is dramatically smaller than could previously be justified. In particular, we show that the number of sampled sequences required is essentially independent of the number of qubits and scales favorably with the average error rate of the system under investigation. We also investigate the fitting procedure inherent to randomized benchmarking in light of our results and find that standard methods such as ordinary least squares optimization can give misleading results. We therefore recommend moving to more sophisticated fitting methods such as iteratively reweighted least squares optimization. Our results bring rigorous randomized benchmarking on systems with many qubits into the realm of experimental feasibility.

arXiv:1701.04299•Physical Review A•Sep 3, 2019•49 citations

Complete randomized benchmarking protocol accounting for leakage errors

Tobias Chasseur, Frank K. Wilhelm

Randomized benchmarking allows one to efficiently and scalably characterize the average error of a unitary 2-design such as the Clifford group $\mathcal{C}$ on a physical candidate for quantum computation, as long as there are no noncomputational leakage levels in the system. We investigate the effect of leakage errors on randomized benchmarking induced from an additional level per physical qubit and provide a modified protocol that allows us to derive reliable estimates for the error per gate in their presence. We assess the variance of the sequence fidelity corresponding to the number of random sequences needed for valid fidelity estimation. Our protocol allows for gate-dependent error channels without being restricted to perturbations. We show that our protocol is compatible with interleaved randomized benchmarking and expand to benchmarking of arbitrary gates. This setting is relevant for superconducting transmon qubits, among other systems.

arXiv:1505.00580•Physical Review A•Oct 30, 2015•38 citations

Direct Randomized Benchmarking for Multiqubit Devices

Timothy J Proctor, Arnaud Carignan-Dugas, Kenneth Rudinger +3 more

Benchmarking methods that can be adapted to multiqubit systems are essential for assessing the overall or "holistic" performance of nascent quantum processors. The current industry standard is Clifford randomized benchmarking (RB), which measures a single error rate that quantifies overall performance. But, scaling Clifford RB to many qubits is surprisingly hard. It has only been performed on one, two, and three qubits as of this writing. This reflects a fundamental inefficiency in Clifford RB: the n-qubit Clifford gates at its core have to be compiled into large circuits over the one- and two-qubit gates native to a device. As n grows, the quality of these Clifford gates quickly degrades, making Clifford RB impractical at relatively low n. In this Letter, we propose a direct RB protocol that mostly avoids compiling. Instead, it uses random circuits over the native gates in a device, which are seeded by an initial layer of Clifford-like randomization. We demonstrate this protocol experimentally on two to five qubits using the publicly available ibmqx5. We believe this to be the greatest number of qubits holistically benchmarked, and this was achieved on a freely available device without any special tuning up. Our protocol retains the simplicity and convenient properties of Clifford RB: it estimates an error rate from an exponential decay. But, it can be extended to processors with more qubits-we present simulations on 10+ qubits-and it reports a more directly informative and flexible error rate than the one reported by Clifford RB. We show how to use this flexibility to measure separate error rates for distinct sets of gates, and we use this method to estimate the average error rate of a set of cnot gates.

arXiv:1807.07975•Physical Review Letters•Jul 19, 2019•118 citations

A Theory of Direct Randomized Benchmarking

Anthony Polloreno, Arnaud Carignan-Dugas, Jordan Hines +3 more

Randomized benchmarking (RB) protocols are widely used to measure an average error rate for a set of quantum logic gates. However, the standard version of RB is limited because it only benchmarks a processor's native gates indirectly, by using them in composite n -qubit Clifford gates. Standard RB's reliance on n -qubit Clifford gates restricts it to the few-qubit regime, because the fidelity of a typical composite n -qubit Clifford gate decreases rapidly with increasing n . Furthermore, although standard RB is often used to infer the error rate of native gates, by rescaling standard RB's error per Clifford to an error per native gate, this is an unreliable extrapolation. Direct RB is a method that addresses these limitations of standard RB, by directly benchmarking a customizable gate set, such as a processor's native gates. Here we provide a detailed introduction to direct RB, we discuss how to design direct RB experiments, and we present two complementary theories for direct RB. The first of these theories uses the concept of error propagation or scrambling in random circuits to show that direct RB is reliable for gates that experience stochastic Pauli errors. We prove that the direct RB decay is a single exponential, and that the decay rate is equal to the average infidelity of the benchmarked gates, under broad circumstances. This theory shows that group twirling is not required for reliable RB. Our second theory proves that direct RB is reliable for gates that experience general gate-dependent Markovian errors, using similar techniques to contemporary theories for standard RB. Our two theories for direct RB have complementary regimes of applicability, and they provide complementary perspectives on why direct RB works. Together these theories provide comprehensive guarantees on the reliability of direct RB.

arXiv:2302.13853•Quantum•Sep 5, 2025•5 citations

Characterization of Quantum Gate Noise Using Randomized Benchmarking

Goutam Paul

Benchmarking noise in quantum gates is crucial for developing dependable quantum computers, as noise directly affects gate fidelity and computational accuracy. Randomized benchmarking (RB) protocols, which evaluate performance through random gate sequences, provide a reliable way to quantify average error rates due to their scalability and robustness in averaging out complex, device-specific noise. In this talk, we will discuss several variants of randomized benchmarking protocols including an overview of our recent work in this area.

River Publishers eBooks•Jan 26, 2026•0 citations
Randomized benchmarking (RB) is a widely used method for estimating the average fidelity of gates implemented on a quantum computing device. The stochastic error of the average gate fidelity estimated by RB depends on the sampling strategy (i.e., how to sample sequences to be run in the protocol). The sampling strategy is determined by a set of configurable parameters (an RB configuration) that includes Clifford lengths (a list of the number of independent Clifford gates in a sequence) and the number of sequences for each Clifford length. The RB configuration is often chosen heuristically and there has been little research on its best configuration. Therefore, we propose a method for fully optimizing an RB configuration so that the confidence interval of the estimated fidelity is minimized while not increasing the total execution time of sequences. By experiments on real devices, we demonstrate the efficacy of the optimization method against heuristic selection in reducing the variance of the estimated fidelity.
Randomized benchmarking (RB) is a widely used strategy to assess the quality of available quantum gates in a computational context. RB involves applying known random sequences of gates to an initial state and using the statistics of a final measurement step to determine an effective depolarizing error per step of the sequence, which is a metric of gate quality. Here we investigate the advantages of fully randomized benchmarking, where a new random sequence is drawn for each experimental trial. The advantages of full randomization include smaller confidence intervals on the inferred step error, the ability to use maximum likelihood analysis without heuristics, straightforward optimization of the sequence lengths, and the ability to model and measure behaviors that go beyond the typical assumption of time-independent error rates. We discuss models of time-dependent or non-Markovian errors that generalize the basic RB model of a single exponential decay of the success probability. For any of these models, we implement a concrete protocol to minimize the uncertainty of the estimated parameters given a fixed time constraint on the complete experiment, and we implement a maximum likelihood analysis. We consider several previously published experiments and determine the potential for improvements with optimized full randomization. We experimentally observe such improvements in Clifford randomized benchmarking experiments on a single trapped ion qubit at the National Institute of Standards and Technology (NIST). For an experiment with uniform lengths and intentionally repeated sequences the step error was 2.42−0.22+0.30×10−52.42^{+0.30}_{-0.22}\times 10^{-5}2.42−0.22+0.30​×10−5, and for an optimized fully randomized experiment of the same total duration the step error was 2.57−0.06+0.07×10−52.57^{+0.07}_{-0.06}\times 10^{-5}2.57−0.06+0.07​×10−5. We find a substantial decrease in the uncertainty of the step error as a result of optimized fully randomized benchmarking.
Randomized benchmarking (RB) is an efficient and robust method to characterize gate errors in quantum circuits. Averaging over random sequences of gates leads to estimates of gate errors in terms of the average fidelity. These estimates are isolated from the state preparation and measurement errors that plague other methods such as channel tomography and direct fidelity estimation. A decisive factor in the feasibility of randomized benchmarking is the number of sampled sequences required to obtain rigorous confidence intervals. Previous bounds were either prohibitively loose or required the number of sampled sequences to scale exponentially with the number of qubits in order to obtain a fixed confidence interval at a fixed error rate. Here, we show that, with a small adaptation to the randomized benchmarking procedure, the number of sampled sequences required for a fixed confidence interval is dramatically smaller than could previously be justified. In particular, we show that the number of sampled sequences required is essentially independent of the number of qubits and scales favorably with the average error rate of the system under investigation. We also investigate the fitting procedure inherent to randomized benchmarking in light of our results and find that standard methods such as ordinary least squares optimization can give misleading results. We therefore recommend moving to more sophisticated fitting methods such as iteratively reweighted least squares optimization. Our results bring rigorous randomized benchmarking on systems with many qubits into the realm of experimental feasibility.
Randomized benchmarking allows one to efficiently and scalably characterize the average error of a unitary 2-design such as the Clifford group C\mathcal{C}C on a physical candidate for quantum computation, as long as there are no noncomputational leakage levels in the system. We investigate the effect of leakage errors on randomized benchmarking induced from an additional level per physical qubit and provide a modified protocol that allows us to derive reliable estimates for the error per gate in their presence. We assess the variance of the sequence fidelity corresponding to the number of random sequences needed for valid fidelity estimation. Our protocol allows for gate-dependent error channels without being restricted to perturbations. We show that our protocol is compatible with interleaved randomized benchmarking and expand to benchmarking of arbitrary gates. This setting is relevant for superconducting transmon qubits, among other systems.
Benchmarking methods that can be adapted to multiqubit systems are essential for assessing the overall or "holistic" performance of nascent quantum processors. The current industry standard is Clifford randomized benchmarking (RB), which measures a single error rate that quantifies overall performance. But, scaling Clifford RB to many qubits is surprisingly hard. It has only been performed on one, two, and three qubits as of this writing. This reflects a fundamental inefficiency in Clifford RB: the n-qubit Clifford gates at its core have to be compiled into large circuits over the one- and two-qubit gates native to a device. As n grows, the quality of these Clifford gates quickly degrades, making Clifford RB impractical at relatively low n. In this Letter, we propose a direct RB protocol that mostly avoids compiling. Instead, it uses random circuits over the native gates in a device, which are seeded by an initial layer of Clifford-like randomization. We demonstrate this protocol experimentally on two to five qubits using the publicly available ibmqx5. We believe this to be the greatest number of qubits holistically benchmarked, and this was achieved on a freely available device without any special tuning up. Our protocol retains the simplicity and convenient properties of Clifford RB: it estimates an error rate from an exponential decay. But, it can be extended to processors with more qubits-we present simulations on 10+ qubits-and it reports a more directly informative and flexible error rate than the one reported by Clifford RB. We show how to use this flexibility to measure separate error rates for distinct sets of gates, and we use this method to estimate the average error rate of a set of cnot gates.
Randomized benchmarking (RB) protocols are widely used to measure an average error rate for a set of quantum logic gates. However, the standard version of RB is limited because it only benchmarks a processor's native gates indirectly, by using them in composite n -qubit Clifford gates. Standard RB's reliance on n -qubit Clifford gates restricts it to the few-qubit regime, because the fidelity of a typical composite n -qubit Clifford gate decreases rapidly with increasing n . Furthermore, although standard RB is often used to infer the error rate of native gates, by rescaling standard RB's error per Clifford to an error per native gate, this is an unreliable extrapolation. Direct RB is a method that addresses these limitations of standard RB, by directly benchmarking a customizable gate set, such as a processor's native gates. Here we provide a detailed introduction to direct RB, we discuss how to design direct RB experiments, and we present two complementary theories for direct RB. The first of these theories uses the concept of error propagation or scrambling in random circuits to show that direct RB is reliable for gates that experience stochastic Pauli errors. We prove that the direct RB decay is a single exponential, and that the decay rate is equal to the average infidelity of the benchmarked gates, under broad circumstances. This theory shows that group twirling is not required for reliable RB. Our second theory proves that direct RB is reliable for gates that experience general gate-dependent Markovian errors, using similar techniques to contemporary theories for standard RB. Our two theories for direct RB have complementary regimes of applicability, and they provide complementary perspectives on why direct RB works. Together these theories provide comprehensive guarantees on the reliability of direct RB.
Benchmarking noise in quantum gates is crucial for developing dependable quantum computers, as noise directly affects gate fidelity and computational accuracy. Randomized benchmarking (RB) protocols, which evaluate performance through random gate sequences, provide a reliable way to quantify average error rates due to their scalability and robustness in averaging out complex, device-specific noise. In this talk, we will discuss several variants of randomized benchmarking protocols including an overview of our recent work in this area.