Benchmarking models via classical simulations is one of the main ways to judge ideas in quantum machine learning before noise-free hardware is available. However, the huge impact of the experimental design on the results, the small scales within reach today, as well as narratives influenced by the commercialisation of quantum technologies make it difficult to gain robust insights. To facilitate better decision-making we develop an open-source package based on the PennyLane software framework and use it to conduct a large-scale study that systematically tests 12 popular quantum machine learning models on 6 binary classification tasks used to create 160 individual datasets. We find that overall, out-of-the-box classical machine learning models outperform the quantum classifiers. Moreover, removing entanglement from a quantum model often results in as good or better performance, suggesting that "quantumness" may not be the crucial ingredient for the small learning tasks considered here. Our benchmarks also unlock investigations beyond simplistic leaderboard comparisons, and we identify five important questions for quantum model design that follow from our results.
Benchmarking models via classical simulations is one of the main ways to judge ideas in quantum machine learning before noise-free hardware is available. However, the huge impact of the experimental design on the results, the small scales within reach today, as well as narratives influenced by the commercialisation of quantum technologies make it difficult to gain robust insights. To facilitate better decision-making we develop an open-source package based on the PennyLane software framework and use it to conduct a large-scale study that systematically tests 12 popular quantum machine learning models on 6 binary classification tasks used to create 160 individual datasets. We find that overall, out-of-the-box classical machine learning models outperform the quantum classifiers. Moreover, removing entanglement from a quantum model often results in as good or better performance, suggesting that "quantumness" may not be the crucial ingredient for the small learning tasks considered here. Our benchmarks also unlock investigations beyond simplistic leaderboard comparisons, and we identify five important questions for quantum model design that follow from our results.
<!--HTML-->Abstract: There is no shortage of quantum machine learning papers observing that a particular quantum model "beats its classical counterparts on real-world datasets". However, the subtlety of choices made in benchmark experiments, the small scale of the models and data, as well as narratives influenced by the commercialisation of quantum technologies carry the danger of a strong positivity bias. To judge the true potential of prominent ideas in quantum machine learning we are conducting one of the first large-scale meta-studies that systematically tests 12 popular supervised quantum models at scale using the PennyLane software framework. This talk gives a sneak peek of some surprising preliminary results, and reveals the technical and conceptual difficulty of robust benchmarking, a skill which deserves more attention in the quantum applications literature.
Quantum machine learning (QML) has emerged as a promising field, combining the power of quantum computing with classical machine learning techniques to solve complex computational tasks. As the demand for efficient quantum simulations grows, multiple QML frameworks, including PennyLane, Qiskit, and TensorFlow Quantum (TFQ), have been developed to facilitate hybrid quantum-classical computations. This study aims to evaluate and compare the performance of three leading QML frameworks PennyLane, Qiskit, and TensorFlow Quantum in simulating quantum machine learning models, focusing on accuracy, execution time, and noise tolerance. The study examined three leading quantum machine learning (QML) frameworks PennyLane (v0.24), Qiskit (v0.43.2), and TensorFlow Quantum (TFQ, v0.7) each tested on a 64-bit Ubuntu Linux system with an Intel Core i7 processor, 32 GB RAM, and NVIDIA GeForce RTX 3080 GPU. Quantum algorithms like variational quantum circuits (VQCs) and quantum support vector machines (QSVMs) were simulated using these frameworks. Various quantum gates, including Pauli-X, Y, Z, Hadamard, CNOT, and Rotation gates (Rx, Ry, Rz), were used for parameter optimization and quantum superposition. The Iris and MNIST datasets were adapted for quantum encoding using Amplitude and Angle Encoding for binary classification tasks. Simulations were run on classical computers, while select circuits were executed on IBM Quantum’s Falcon r5.11 processor to compare hardware performance. PennyLane achieved the highest accuracy (92%) in quantum simulations, excelling in hybrid quantum-classical model integration. TensorFlow Quantum provided the fastest execution time, especially in classical simulations, making it suitable for rapid prototyping. Qiskit produced deeper quantum circuits but had higher error rates (6%) on real quantum hardware, highlighting noise and decoherence effects. The VQC models optimized by stochastic gradient descent (SGD) showed significant improvements in model performance, and QSVMs effectively classified both linearly separable and non-separable datasets, demonstrating the potential of quantum algorithms over classical approaches. Statistical analysis confirmed the significance of the results (p < 0.05). PennyLane is the most robust framework for hybrid quantum-classical models, while TensorFlow Quantum is suited for rapid prototyping. Qiskit, despite longer execution times, excels in hardware implementation. The choice of framework should align with the specific needs of quantum machine learning tasks.
This study explores the performance of Quantum Support Vector Classifiers (QSVCs) and Quantum Neural Networks (QNNs) in comparison to classical models for machine learning tasks. By evaluating these models on the Iris and MNIST-PCA datasets, we find that quantum models tend to outperform classical approaches as the problem complexity increases. While QSVCs generally provide more consistent results, QNNs exhibit superior performance in higher-complexity tasks due to their increased quantum load. Additionally, we analyze the impact of hyperparameter tuning, showing that feature maps and ansatz configurations significantly influence model accuracy. We also compare the PennyLane and Qiskit frameworks, concluding that Qiskit provides better optimization and efficiency for our implementation. These findings highlight the potential of Quantum Machine Learning (QML) for complex classification problems and provide insights into model selection and optimization strategies
Machine Learning has been remarkably successful and is transforming the way real life problems are solved. Quantum Computing is a developing field that utilizes different quantum phenomena for computational speedup. Quantum Machine Learning (QML) focuses on combining these two fields in a meaningful manner. Quantum kernels are among the state of the art approaches to QML. This thesis reproduces the existing work on quantum kernels and then expands on its real life application. These applications are addressed through two types of studies, comparison of quantum kernel based models with classical ML models and analysis of their performance on the present day error prone quantum hardware using noise simulation. Both of these studies are implemented using Qiskit in Python. The comparison results show that the quantum kernels can achieve better results than their classical counterpart. Noise simulations show that the model is resilient to lower levels of noise. However, further investigation is needed to explain the overall noise trends.
Quantum machine learning sits at the intersection of quantum computing and artificial intelligence, seeking to exploitquantum mechanical phenomena -- superposition, entanglement, and interference -- to achieve computationaladvantages for learning tasks. This study presents a systematic evaluation of five quantum ML approaches -- variationalquantum classifiers (VQC), quantum kernel methods, quantum reservoir computing, quantum Boltzmann machines, andhybrid quantum-classical neural networks -- benchmarked against classical counterparts on both simulated quantumhardware (up to 30 qubits) and real NISQ devices (IBM Quantum, 27 qubits). Evaluations spanned four task categories:binary classification on structured datasets (6 UCI benchmarks), quantum chemistry property prediction (QM9 molecularenergies), generative modelling (learning probability distributions), and optimisation (Max-Cut on random graphs). A totalof 1,920 experiments were conducted. On classical structured datasets, no quantum method outperformed the bestclassical baseline (XGBoost or SVM) at any problem size accessible to current quantum hardware (up to 30 features),with quantum kernel methods achieving parity at 0.4 +- 0.8% accuracy difference but at 100-1000x higher computationalcost. On quantum chemistry, variational quantum eigensolvers showed a genuine advantage for small molecules (H2,LiH, BeH2): achieving chemical accuracy (1 kcal/mol) with 4-12 qubits where classical methods required exponentiallymore compute for equivalent accuracy. Noise on real NISQ devices degraded all quantum ML methods by 8.4-22.6%relative to noiseless simulation, with error mitigation recovering 42-68% of the noise-induced loss. The quantumadvantage threshold -- the problem size at which quantum methods become faster than classical for equivalent accuracy-- was estimated at 50-100 logical qubits for quantum chemistry and 200+ logical qubits for general ML tasks, placingpractical quantum ML advantage beyond current NISQ capabilities but within plausible reach of fault-tolerant devices. Apractical assessment framework mapping task type, problem structure, and available quantum hardware to expectedquantum advantage is proposed.
Quantum Machine Learning (QML) has emerged as an interdisciplinary field that harnesses the potential of quantum computing to revolutionize classical machine learning paradigms. This paper presents a benchmarking study focused on four quantum classifier algorithms: Quantum Support Vector Classifier (QSVC), Variational Quantum Circuit (VQC), Quantum Neural Network Classifier (QNNC) and Pegasos Quantum Support Vector Classifier (PegasosQSVC). These quantum classifiers hold promise for unlocking quantum advantages in classification tasks. However, a systematic assessment of their performance across diverse datasets is yet to be fully explored. In this research study, we evaluate and compare the classification performance of quantum machine learning classifiers on various datasets (Breast Cancer, Diabetes, and Titanic) and compare them to a classical model. Our findings revealed that PegasosQSVC consistently maintained high performance across all datasets with the highest average accuracy, recall, and F1-score values of 0.81, 0.72, and 0.71, respectively, followed by VQC, followed by VQC, while QSVC and QNNC perform variably. Nevertheless, QSVC was found to balance performance and training time, achieving an average accuracy of 0.81 and a training time of 177 seconds. Overall, the classical model surpassed QML models as it was up to 10% higher in average accuracy.