Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:1401.1546 · Physical Review X

Glassy Chimeras Could Be Blind to Quantum Speedup: Designing Better Benchmarks for Quantum Annealing Machines

Helmut G. Katzgraber, Firas Hamze, Ruben S. Andrist

Published Apr 10, 2014• 145 citations
arXivHTMLPDF

Abstract

Recent benchmarking of the computational speedup of quantum ``annealing'' machines of the D-Wave-2 type shows that they do not perform faster than a standard desktop computer. A timely theoretical study of the computational tests used in the benchmarking explains why that may be the case.
DOI
10.1103/physrevx.4.021008

Related papers

Seeking Quantum Speedup Through Spin Glasses: The Good, the Bad, and the Ugly

Helmut G. Katzgraber, Firas Hamze, Zhu, Zheng +2 more

There has been considerable progress in the design and construction of quantum annealing devices. However, a conclusive detection of quantum speedup over traditional silicon-based machines remains elusive, despite multiple careful studies. In this work we outline strategies to design hard tunable benchmark instances based on insights from the study of spin glasses - the archetypal random benchmark problem for novel algorithms and optimization devices. We propose to complement head-to-head scaling studies that compare quantum annealing machines to state-of-the-art classical codes with an approach that compares the performance of different algorithms and/or computing architectures on different classes of computationally hard tunable spin-glass instances. The advantage of such an approach lies in having to only compare the performance hit felt by a given algorithm and/or architecture when the instance complexity is increased. Furthermore, we propose a methodology that might not directly translate into the detection of quantum speedup, but might elucidate whether quantum annealing has a "`quantum advantage" over corresponding classical algorithms like simulated annealing. Our results on a 496 qubit D-Wave Two quantum annealing device are compared to recently-used state-of-the-art thermal simulated annealing codes.

•Jan 1, 2015•86 citations

Evidence for a Limited Quantum Speedup on a Quantum Annealer

Tameem Albash, Daniel A. Lidar

The observation of an unequivocal quantum speedup remains an elusive objective for quantum computing. The D-Wave quantum annealing processors have been at the forefront of experimental attempts to address this goal, given their relatively large numbers of qubits and programmability. A complete determination of the optimal time-to-solution (TTS) using these processors has not been possible to date, preventing definitive conclusions about the presence of a scaling advantage. The main technical obstacle has been the inability to verify an optimal annealing time within the available range. Here we overcome this obstacle and present a class of problem instances for which we observe an optimal annealing time using a D-Wave 2000Q processor over a range spanning up to more than $2000$ qubits. This allows us to perform an optimal TTS benchmarking analysis and perform a comparison to several classical algorithms, including simulated annealing, spin-vector Monte Carlo, and discrete-time simulated quantum annealing. We establish the first example of a scaling advantage for an experimental quantum annealer over classical simulated annealing: we find that the D-Wave device exhibits certifiably better scaling than simulated annealing, with $95\%$ confidence, over the range of problem sizes that we can test. However, we do not find evidence for a quantum speedup: simulated quantum annealing exhibits the best scaling by a significant margin. Our construction of instance classes with verifiably optimal annealing times opens up the possibility of generating many new such classes, paving the way for further definitive assessments of scaling advantages using current and future quantum annealing devices.

•May 21, 2017•6 citations

How to benchmark a quantum computer

Ian S. Osborne

Quantum Computing Quantum machines offer the possibility of performing certain computations much faster than their classical counterparts. However, how to define and measure quantum speedup is a topic of debate. Ronnow et al. describe methods for fairly evaluating the difference in computational power between classical and quantum processors. They define various types of quantum speedup and consider quantum processors that are designed to solve a specific class of problems. Science , this issue p. [420][1] [1]: /lookup/volpage/345/420?iss=6195

Science•Jul 24, 2014•0 citations

Quantum Annealing versus Digital Computing

Michael Jünger, Elisabeth Lobe, Petra Mutzel +4 more

Quantum annealing is getting increasing attention in combinatorial optimization. The quantum processing unit by D-Wave is constructed to approximately solve Ising models on so-called Chimera graphs. Ising models are equivalent to quadratic unconstrained binary optimization (QUBO) problems and maximum cut problems on the associated graphs. We have tailored branch-and-cut as well as semidefinite programming algorithms for solving Ising models for Chimera graphs to provable optimality and use the strength of these approaches for comparing our solution values to those obtained on the current quantum annealing machine, D-Wave 2000Q. This allows for the assessment of the quality of solutions produced by the D-Wave hardware. In addition, we also evaluate the performance of a heuristic by Selby. It has been a matter of discussion in the literature how well the D-Wave hardware performs at its native task, and our experiments shed some more light on this issue. In particular, we examine how reliably the D-Wave computer can deliver true optimum solutions and present some surprising results.

arXiv:1904.11965•ACM Journal of Experimental Algorithmics•Jul 9, 2021•48 citations

Quantum chip flunks landmark speed test

Jacob Aron

The quantum revolution is still in the slow lane. The first head-to-head speed test of the D-Wave Two, the commercial quantum computer recently purchased by Google, suggests that the machine performs no better than an ordinary PC. In theory, quantum computers are capable of calculating answers to problems much faster than standard PCs. Instead of storing information in bits as either 0s or 1s, quantum machines use quantum bits, or qubits, that can be both 0 and 1 at the same time. So far, however, the quantum devices built in labs can only support a handful of qubits, which limits their speed. Now a team led by Matthias Troyer of ETH Zurich in Switzerland has tested a D-Wave Two computer against a conventional machine running an optimized algorithm and found no evidence of superior performance in the quantum device.

The New Scientist•Jan 1, 2014•0 citations

Test-driving 1000 qubits

Joshua Job, Daniel A. Lidar

Abstract Quantum computing is no longer a nascent field. Programmable quantum annealing (QA) devices with more than 1000 qubits are commercially available. How does one know that a putative QA device is indeed quantum? How should one go about benchmarking its performance and compare it to classical algorithms? How can its performance be improved by error correction? In this contribution to the focus collection on ‘What would you do with 1000 qubits?’, we review the work we and others have done in this area, since the first D-Wave quantum annealer with 108 qubits was made available to us. Some of the lessons we have learned will be useful when other quantum computing platforms reach a similar scale, and practitioners will attempt to demonstrate quantum speedup.

arXiv:1706.07124•Quantum Science and Technology•Jun 19, 2018•34 citations
There has been considerable progress in the design and construction of quantum annealing devices. However, a conclusive detection of quantum speedup over traditional silicon-based machines remains elusive, despite multiple careful studies. In this work we outline strategies to design hard tunable benchmark instances based on insights from the study of spin glasses - the archetypal random benchmark problem for novel algorithms and optimization devices. We propose to complement head-to-head scaling studies that compare quantum annealing machines to state-of-the-art classical codes with an approach that compares the performance of different algorithms and/or computing architectures on different classes of computationally hard tunable spin-glass instances. The advantage of such an approach lies in having to only compare the performance hit felt by a given algorithm and/or architecture when the instance complexity is increased. Furthermore, we propose a methodology that might not directly translate into the detection of quantum speedup, but might elucidate whether quantum annealing has a "`quantum advantage" over corresponding classical algorithms like simulated annealing. Our results on a 496 qubit D-Wave Two quantum annealing device are compared to recently-used state-of-the-art thermal simulated annealing codes.
The observation of an unequivocal quantum speedup remains an elusive objective for quantum computing. The D-Wave quantum annealing processors have been at the forefront of experimental attempts to address this goal, given their relatively large numbers of qubits and programmability. A complete determination of the optimal time-to-solution (TTS) using these processors has not been possible to date, preventing definitive conclusions about the presence of a scaling advantage. The main technical obstacle has been the inability to verify an optimal annealing time within the available range. Here we overcome this obstacle and present a class of problem instances for which we observe an optimal annealing time using a D-Wave 2000Q processor over a range spanning up to more than 200020002000 qubits. This allows us to perform an optimal TTS benchmarking analysis and perform a comparison to several classical algorithms, including simulated annealing, spin-vector Monte Carlo, and discrete-time simulated quantum annealing. We establish the first example of a scaling advantage for an experimental quantum annealer over classical simulated annealing: we find that the D-Wave device exhibits certifiably better scaling than simulated annealing, with 95%95\%95% confidence, over the range of problem sizes that we can test. However, we do not find evidence for a quantum speedup: simulated quantum annealing exhibits the best scaling by a significant margin. Our construction of instance classes with verifiably optimal annealing times opens up the possibility of generating many new such classes, paving the way for further definitive assessments of scaling advantages using current and future quantum annealing devices.
Quantum Computing Quantum machines offer the possibility of performing certain computations much faster than their classical counterparts. However, how to define and measure quantum speedup is a topic of debate. Ronnow et al. describe methods for fairly evaluating the difference in computational power between classical and quantum processors. They define various types of quantum speedup and consider quantum processors that are designed to solve a specific class of problems. Science , this issue p. [420][1] [1]: /lookup/volpage/345/420?iss=6195
Quantum annealing is getting increasing attention in combinatorial optimization. The quantum processing unit by D-Wave is constructed to approximately solve Ising models on so-called Chimera graphs. Ising models are equivalent to quadratic unconstrained binary optimization (QUBO) problems and maximum cut problems on the associated graphs. We have tailored branch-and-cut as well as semidefinite programming algorithms for solving Ising models for Chimera graphs to provable optimality and use the strength of these approaches for comparing our solution values to those obtained on the current quantum annealing machine, D-Wave 2000Q. This allows for the assessment of the quality of solutions produced by the D-Wave hardware. In addition, we also evaluate the performance of a heuristic by Selby. It has been a matter of discussion in the literature how well the D-Wave hardware performs at its native task, and our experiments shed some more light on this issue. In particular, we examine how reliably the D-Wave computer can deliver true optimum solutions and present some surprising results.
The quantum revolution is still in the slow lane. The first head-to-head speed test of the D-Wave Two, the commercial quantum computer recently purchased by Google, suggests that the machine performs no better than an ordinary PC. In theory, quantum computers are capable of calculating answers to problems much faster than standard PCs. Instead of storing information in bits as either 0s or 1s, quantum machines use quantum bits, or qubits, that can be both 0 and 1 at the same time. So far, however, the quantum devices built in labs can only support a handful of qubits, which limits their speed. Now a team led by Matthias Troyer of ETH Zurich in Switzerland has tested a D-Wave Two computer against a conventional machine running an optimized algorithm and found no evidence of superior performance in the quantum device.
Abstract Quantum computing is no longer a nascent field. Programmable quantum annealing (QA) devices with more than 1000 qubits are commercially available. How does one know that a putative QA device is indeed quantum? How should one go about benchmarking its performance and compare it to classical algorithms? How can its performance be improved by error correction? In this contribution to the focus collection on ‘What would you do with 1000 qubits?’, we review the work we and others have done in this area, since the first D-Wave quantum annealer with 108 qubits was made available to us. Some of the lessons we have learned will be useful when other quantum computing platforms reach a similar scale, and practitioners will attempt to demonstrate quantum speedup.