Quantum Machine Learning (QML) has emerged as a promising paradigm that leverages quantum mechanical principles to accelerate machine learning tasks.Despite its potential, most QML models remain opaque "black boxes," hindering their deployment in high-stakes or regulated domains where trust and accountability are essential.This work introduces two novel interpretability methods tailored for QML: Quantum Shapley Values (QSV) and Quantum Local Interpretable Model-Agnostic Explanations (Q-LIME).QSV adapts cooperative game theory to quantify the contribution of individual qubits, observables, or circuit parameters, while Q-LIME generates perturbed quantum states and trains surrogate classical models to approximate local decision boundaries.Through experiments on variational quantum classifiers and hybrid quantum-classical neural networks, we demonstrate that QSV provides rigorous attribution but suffers from scalability constraints, whereas Q-LIME yields efficient, approximate explanations suitable for near-term devices.By transforming quantum models from opaque "black boxes" into interpretable "glass boxes," this research lays the foundation for trustworthy, transparent, and human centred quantum AI.
Quantum Machine Learning (QML) has emerged as a promising paradigm that leverages quantum mechanical principles to accelerate machine learning tasks.Despite its potential, most QML models remain opaque "black boxes," hindering their deployment in high-stakes or regulated domains where trust and accountability are essential.This work introduces two novel interpretability methods tailored for QML: Quantum Shapley Values (QSV) and Quantum Local Interpretable Model-Agnostic Explanations (Q-LIME).QSV adapts cooperative game theory to quantify the contribution of individual qubits, observables, or circuit parameters, while Q-LIME generates perturbed quantum states and trains surrogate classical models to approximate local decision boundaries.Through experiments on variational quantum classifiers and hybrid quantum-classical neural networks, we demonstrate that QSV provides rigorous attribution but suffers from scalability constraints, whereas Q-LIME yields efficient, approximate explanations suitable for near-term devices.By transforming quantum models from opaque "black boxes" into interpretable "glass boxes," this research lays the foundation for trustworthy, transparent, and human centred quantum AI.
Abstract As machine learning grows increasingly complex due to big data and deep learning, model explainability has become essential to fostering user trust. Quantum machine learning (QML) has emerged as a promising field, leveraging quantum computing to enhance classical machine learning methods, particularly through quantum representation learning (QRL). QRL aims to provide more efficient and powerful machine learning capabilities on noisy intermediate-scale quantum (NISQ) devices. However, interpreting QRL models poses significant challenges due to the reliance on quantum gate-based parameterized circuits, which, while analogous to classical neural network layers, operate in the quantum domain. To address these challenges, we propose an explainable QRL framework combining a quantum autoencoder (QAE) with a variational quantum classifier (VQC) and incorporating theoretical and empirical explainability for image data. Our dual approach enhances model interpretability by integrating visual explanations via local interpretable model-agnostic explanations (LIME) and analytical insights using Shapley Additive Explanations (SHAP). These complementary methods provide a deeper understanding of the model’s decision-making process based on prediction outcomes. Experimental evaluations on simulators and superconducting quantum hardware validate the effectiveness of the proposed framework for classification tasks, underscoring the importance of explainable representation learning in advancing QML towards more transparent and reliable applications.
Abstract The interplay between machine learning and quantum computing can lead to unprecedented prospects for both fundamental research and real-world applications. In this new field of quantum machine learning, quantum classifiers, i.e., quantum algorithms for solving classification problems, have attracted the most attention. However, despite the promising performance of these new models for classification tasks, they are complex black-box models. Therefore, like or even more so than classical machine learning models, their use in critical contexts such as medical diagnosis could be hampered without the support of explanations. This paper addresses this challenge by investigating the use of Local Interpretable Model-agnostic Explanations (LIME) for quantum classifiers, with a specific focus on the stability of the explanations. Stability is a key property for ensuring that explanations are consistent and trustworthy, especially when decisions impact sensitive domains. The paper conducts a systematic stability study of LIME in the quantum setting, validating whether a classical explainable AI technique can be meaningfully applied to quantum models. As shown in experiments involving different datasets belonging to critical applications, LIME provides explanations for quantum support vector machines and quantum neural networks that are not only interpretable but also stable. By highlighting the stability of explanations, these findings demonstrate the suitability of LIME as an explainability tool for quantum classifiers.
Abstract Methods of artificial intelligence (AI) and especially machine learning (ML) have been growing ever more complex, and at the same time have more and more impact on people’s lives. This leads to explainable AI (XAI) manifesting itself as an important research field that helps humans to better comprehend ML systems. In parallel, quantum machine learning (QML) is emerging with the ongoing improvement of quantum computing hardware combined with its increasing availability via cloud services. QML enables quantum-enhanced ML in which quantum mechanics is exploited to facilitate ML tasks, typically in the form of quantum-classical hybrid algorithms that combine quantum and classical resources. Quantum gates constitute the building blocks of gate-based quantum hardware and form circuits that can be used for quantum computations. For QML applications, quantum circuits are typically parameterized and their parameters are optimized classically such that a suitably defined objective function is minimized. Inspired by XAI, we raise the question of the explainability of such circuits by quantifying the importance of (groups of) gates for specific goals. To this end, we apply the well-established concept of Shapley values. The resulting attributions can be interpreted as explanations for why a specific circuit works well for a given task, improving the understanding of how to construct parameterized (or variational) quantum circuits, and fostering their human interpretability in general. An experimental evaluation on simulators and two superconducting quantum hardware devices demonstrates the benefits of the proposed framework for classification, generative modeling, transpilation, and optimization. Furthermore, our results shed some light on the role of specific gates in popular QML approaches.
Many Machine Learning (ML) models are referred to as black box models, providing no real insights into why a prediction is made. Feature importance and explainability are important for increasing transparency and trust in ML models, particularly in settings such as healthcare and finance. With quantum computing's unique capabilities, such as leveraging quantum mechanical phenomena like superposition, which can be combined with ML techniques to create the field of Quantum Machine Learning (QML), and such techniques may be applied to QML models. This article explores feature importance and explainability insights in QML compared to Classical ML models. Utilizing the widely recognized Iris dataset, classical ML algorithms such as SVM and Random Forests, are compared against hybrid quantum counterparts, implemented via IBM's Qiskit platform: the Variational Quantum Classifier (VQC) and Quantum Support Vector Classifier (QSVC). This article aims to provide a comparison of the insights generated in ML by employing permutation and leave one out feature importance methods, alongside ALE (Accumulated Local Effects) and SHAP (SHapley Additive exPlanations) explainers.
Artificial Intelligence (AI) systems have shown good success at classifying. However, the lack of explainability is a true and significant challenge, especially in high-stakes domains, such as health and finance, where understanding is paramount. We propose a new solution to this challenge: an explainable AI framework based on our comparative study with Quantum Boltzmann Machines (QBMs) and Classical Boltzmann Machines (CBMs). We leverage principles of quantum computing within classical machine learning to provide substantive transparency around decision-making. The design involves training both models on a binarised and dimensionally reduced MNIST dataset, where Principal Component Analysis (PCA) is applied for preprocessing. For interpretability, we employ gradient-based saliency maps in QBMs and SHAP (SHapley Additive exPlanations) in CBMs to evaluate feature attributions.QBMs deploy hybrid quantum-classical circuits with strongly entangling layers, allowing for richer latent representations, whereas CBMs serve as a classical baseline that utilises contrastive divergence. Along the way, we found that QBMs outperformed CBMs on classification accuracy (83.5% vs. 54%) and had more concentrated distributions in feature attributions as quantified by entropy (1.27 vs. 1.39). In other words, QBMs not only produced better predictive performance than CBMs, but they also provided clearer identification of "active ingredient" or the most important features behind model predictions. To conclude, our results illustrate that quantum-classical hybrid models can display improvements in both accuracy and interpretability, which leads us toward more trustworthy and explainable AI systems.
Explainable artificial intelligence (XAI) remains underexplored in quantum machine learning (QML) due to limited quantum computational resources and the lack of practical interpretability techniques. In this study, we implement the Local Interpretable Model-Agnostic Explanations (LIME) algorithm within the CUDA-Q framework and propose an interpretable QML analysis method using GPU-accelerated quantum simulation. All experiments are conducted on a GPU-based simulator rather than real quantum hardware, enabling stable large-scale sampling required by LIME and ensuring reproducible results. We design a binary classification model based on a single-qubit variational circuit composed of parameterized RY and RX rotations, and the model is trained with gradient-based optimization. The performance is evaluated on the MNIST, KMNIST, Fashion-MNIST, and CIFAR-10 datasets using training and test accuracy as metrics. By applying LIME, the classification outcomes of the quantum model are visualized and interpreted in a human-understandable form, demonstrating that meaningful interpretability can be achieved even under limited qubit counts and constrained computational environments. Furthermore, we analyze the trade-off between simulator performance and model accuracy, confirming that GPU-based CUDA-Q simulation provides a scalable and practical research environment for QML. This work offers a concrete pathway toward transparent and interpretable quantum machine learning.