ABSTRACT The application of quantum computing to Data Processing has naturally extended into the realm of quantum machine learning. However, while numerous benchmarks exist for evaluating quantum hardware, there remains a scarcity of tailored benchmarks that assess the intrinsic performance of learning models. This gap makes it difficult for end‐users to determine whether their chosen quantum computing system can effectively perform learning tasks. To address this, we propose a novel and systematic suite for establishing quantum learning benchmarks. In this work, we decompose the learning workflow into three core components: Data Representation, Data Processing, and Model Optimization. Furthermore, instead of considering all methods for each component, we selected fundamental or predominantly used methods. The resulting benchmark suite consists of carefully selected benchmarks, including Basis, Angle, and Amplitude Encoding for Data Representation; Quantum Fourier Transform, Quantum Amplitude Amplification, and Quantum Kernel Methods for Data Processing; and Quantum Gradient Descent for Model Optimization. This component‐centric benchmark suite provides clear guidelines, enabling systematic comparisons, transparent diagnosis of performance bottlenecks, and facilitating the efficient adoption of quantum learning technologies.ABSTRACT The application of quantum computing to Data Processing has naturally extended into the realm of quantum machine learning. However, while numerous benchmarks exist for evaluating quantum hardware, there remains a scarcity of tailored benchmarks that assess the intrinsic performance of learning models. This gap makes it difficult for end‐users to determine whether their chosen quantum computing system can effectively perform learning tasks. To address this, we propose a novel and systematic suite for establishing quantum learning benchmarks. In this work, we decompose the learning workflow into three core components: Data Representation, Data Processing, and Model Optimization. Furthermore, instead of considering all methods for each component, we selected fundamental or predominantly used methods. The resulting benchmark suite consists of carefully selected benchmarks, including Basis, Angle, and Amplitude Encoding for Data Representation; Quantum Fourier Transform, Quantum Amplitude Amplification, and Quantum Kernel Methods for Data Processing; and Quantum Gradient Descent for Model Optimization. This component‐centric benchmark suite provides clear guidelines, enabling systematic comparisons, transparent diagnosis of performance bottlenecks, and facilitating the efficient adoption of quantum learning technologies.

Concurrency and Computation Practice and ExperienceMay 1, 20260 citations

QTestBench: A Test Framework for Evaluating LLM-Generated Quantum Code

Meriem Terki, Nouhaila Innan, Minghao Shao +3 more

Large language models (LLMs) have shown strong results in classical code generation and are increasingly used to write the unit tests that safeguard that code. Whether they can do the same for quantum software is less clear, because quantum programs are probabilistic, depend on numerical tolerances, and are judged by domain-specific checkers, so a well-formed test can still disagree with the benchmark's verdict. Existing quantum benchmarks measure whether a model produces a correct solution, but not whether its tests can distinguish correct implementations from faulty ones. We introduce QTestBench, a framework that evaluates LLM-generated quantum test suites against independent benchmark checkers, never treating the tests themselves as ground truth. For each task, QTestBench prompts models to produce syntactic, semantic, and behavioral tests, injects candidate solutions at runtime, and compares each test verdict with the checker's label under three validation cohorts, each answering a distinct diagnostic question. We benchmark five LLMs on eight PennyLane QHack challenges and ten Qiskit HumanEval tasks. Generated suites almost always execute, but discrimination lags behind: the best generator reaches a test quality of 0.71 on QHack and 0.34 on Qiskit HumanEval, so executability alone overstates test quality. Oracle-verified synthetic mutants stabilize bug-detection estimates when natural faulty solutions are scarce, as on three of ten Qiskit tasks that yield none. QTestBench offers a reproducible, cross-framework benchmark for the trustworthiness of LLM-generated quantum tests.Large language models (LLMs) have shown strong results in classical code generation and are increasingly used to write the unit tests that safeguard that code. Whether they can do the same for quantum software is less clear, because quantum programs are probabilistic, depend on numerical tolerances, and are judged by domain-specific checkers, so a well-formed test can still disagree with the benchmark's verdict. Existing quantum benchmarks measure whether a model produces a correct solution, but not whether its tests can distinguish correct implementations from faulty ones. We introduce QTestBench, a framework that evaluates LLM-generated quantum test suites against independent benchmark checkers, never treating the tests themselves as ground truth. For each task, QTestBench prompts models to produce syntactic, semantic, and behavioral tests, injects candidate solutions at runtime, and compares each test verdict with the checker's label under three validation cohorts, each answering a distinct diagnostic question. We benchmark five LLMs on eight PennyLane QHack challenges and ten Qiskit HumanEval tasks. Generated suites almost always execute, but discrimination lags behind: the best generator reaches a test quality of 0.71 on QHack and 0.34 on Qiskit HumanEval, so executability alone overstates test quality. Oracle-verified synthetic mutants stabilize bug-detection estimates when natural faulty solutions are scarce, as on three of ten Qiskit tasks that yield none. QTestBench offers a reproducible, cross-framework benchmark for the trustworthiness of LLM-generated quantum tests.

cs.LGCross-listedarXiv:2610.13811Oct 12, 20260 citations

Benchmarking the performance of quantum computing software for quantum circuit creation, manipulation and compilation

Paul D. Nation, Abdullah Ash Saki, Sebastian Brandhofer +4 more

We present Benchpress, a benchmarking suite for evaluating the performance and range of functionality of multiple quantum computing software development kits. This suite consists of a collection of over 1,000 tests measuring key performance metrics for a wide variety of operations on quantum circuits composed of up to 930 qubits and ${\mathcal{O}}(1{0}^{6})$ two-qubit gates, as well as an execution framework for running the tests over multiple quantum software packages in a unified manner. Here we give a detailed overview of the benchmark suite and its methodology and generate representative results over seven different quantum software packages. The flexibility of the Benchpress framework enables benchmarking that not only keeps pace with quantum hardware improvements but also can preemptively gauge the quantum circuit processing costs of future device architectures. In this Resource, the authors present an open-source extensible benchmark tool, Benchpress, for evaluating the performance of mainstream quantum computing software. Benchpress was demonstrated to perform over 1,000 tests with up to 930 qubits to compare the performance of quantum software, providing insight into how to best use current programming stacks.We present Benchpress, a benchmarking suite for evaluating the performance and range of functionality of multiple quantum computing software development kits. This suite consists of a collection of over 1,000 tests measuring key performance metrics for a wide variety of operations on quantum circuits composed of up to 930 qubits and $${\mathcal{O}}(1{0}^{6})$$ two-qubit gates, as well as an execution framework for running the tests over multiple quantum software packages in a unified manner. Here we give a detailed overview of the benchmark suite and its methodology and generate representative results over seven different quantum software packages. The flexibility of the Benchpress framework enables benchmarking that not only keeps pace with quantum hardware improvements but also can preemptively gauge the quantum circuit processing costs of future device architectures. In this Resource, the authors present an open-source extensible benchmark tool, Benchpress, for evaluating the performance of mainstream quantum computing software. Benchpress was demonstrated to perform over 1,000 tests with up to 930 qubits to compare the performance of quantum software, providing insight into how to best use current programming stacks.

arXiv:2409.08844Nature Computational ScienceApr 18, 202522 citations

An Effective Framework for Full-Stack Benchmarking of Quantum Computers

Michele Amoretti

Apr 26, 20214 citations

Scalable and targeted benchmarking of quantum computers

Timothy J Proctor, Kenneth Rudinger, Mohan Sarovar +3 more

Bulletin of the American Physical SocietyMar 15, 20210 citations

{\mathcal{O}}(1{0}^{6})

two-qubit gates, as well as an execution framework for running the tests over multiple quantum software packages in a unified manner. Here we give a detailed overview of the benchmark suite and its methodology and generate representative results over seven different quantum software packages. The flexibility of the Benchpress framework enables benchmarking that not only keeps pace with quantum hardware improvements but also can preemptively gauge the quantum circuit processing costs of future device architectures. In this Resource, the authors present an open-source extensible benchmark tool, Benchpress, for evaluating the performance of mainstream quantum computing software. Benchpress was demonstrated to perform over 1,000 tests with up to 930 qubits to compare the performance of quantum software, providing insight into how to best use current programming stacks.