Abstract Interpretability of artificial intelligence (AI) methods, particularly deep neural networks, is of great interest. This heightened focus stems from the widespread use of AI-backed systems. These systems, often relying on intricate neural architectures, can exhibit behavior that is challenging to explain and comprehend. The interpretability of such models is a crucial component of building trusted systems. Many methods exist to approach this problem, but they do not apply straightforwardly to the quantum setting. Here, we explore the interpretability of quantum neural networks using local model-agnostic interpretability measures commonly utilized for classical neural networks. Following this analysis, we generalize a classical technique called LIME, introducing Q-LIME, which produces explanations of quantum neural networks. A feature of our explanations is the delineation of the region in which data samples have been given a random label, likely subjects of inherently random quantum measurements. We view this as a step toward understanding how to build responsible and accountable quantum AI models.
Abstract Interpretability of artificial intelligence (AI) methods, particularly deep neural networks, is of great interest. This heightened focus stems from the widespread use of AI-backed systems. These systems, often relying on intricate neural architectures, can exhibit behavior that is challenging to explain and comprehend. The interpretability of such models is a crucial component of building trusted systems. Many methods exist to approach this problem, but they do not apply straightforwardly to the quantum setting. Here, we explore the interpretability of quantum neural networks using local model-agnostic interpretability measures commonly utilized for classical neural networks. Following this analysis, we generalize a classical technique called LIME, introducing Q-LIME, which produces explanations of quantum neural networks. A feature of our explanations is the delineation of the region in which data samples have been given a random label, likely subjects of inherently random quantum measurements. We view this as a step toward understanding how to build responsible and accountable quantum AI models.
Abstract The interplay between machine learning and quantum computing can lead to unprecedented prospects for both fundamental research and real-world applications. In this new field of quantum machine learning, quantum classifiers, i.e., quantum algorithms for solving classification problems, have attracted the most attention. However, despite the promising performance of these new models for classification tasks, they are complex black-box models. Therefore, like or even more so than classical machine learning models, their use in critical contexts such as medical diagnosis could be hampered without the support of explanations. This paper addresses this challenge by investigating the use of Local Interpretable Model-agnostic Explanations (LIME) for quantum classifiers, with a specific focus on the stability of the explanations. Stability is a key property for ensuring that explanations are consistent and trustworthy, especially when decisions impact sensitive domains. The paper conducts a systematic stability study of LIME in the quantum setting, validating whether a classical explainable AI technique can be meaningfully applied to quantum models. As shown in experiments involving different datasets belonging to critical applications, LIME provides explanations for quantum support vector machines and quantum neural networks that are not only interpretable but also stable. By highlighting the stability of explanations, these findings demonstrate the suitability of LIME as an explainability tool for quantum classifiers.
Abstract As machine learning grows increasingly complex due to big data and deep learning, model explainability has become essential to fostering user trust. Quantum machine learning (QML) has emerged as a promising field, leveraging quantum computing to enhance classical machine learning methods, particularly through quantum representation learning (QRL). QRL aims to provide more efficient and powerful machine learning capabilities on noisy intermediate-scale quantum (NISQ) devices. However, interpreting QRL models poses significant challenges due to the reliance on quantum gate-based parameterized circuits, which, while analogous to classical neural network layers, operate in the quantum domain. To address these challenges, we propose an explainable QRL framework combining a quantum autoencoder (QAE) with a variational quantum classifier (VQC) and incorporating theoretical and empirical explainability for image data. Our dual approach enhances model interpretability by integrating visual explanations via local interpretable model-agnostic explanations (LIME) and analytical insights using Shapley Additive Explanations (SHAP). These complementary methods provide a deeper understanding of the model’s decision-making process based on prediction outcomes. Experimental evaluations on simulators and superconducting quantum hardware validate the effectiveness of the proposed framework for classification tasks, underscoring the importance of explainable representation learning in advancing QML towards more transparent and reliable applications.
Machine learning models offer powerful predictive capabilities but often lack transparency. Local Interpretable Model-agnostic Explanations (LIME) addresses this by perturbing features and measuring their impact on a model's output. In text-based tasks, LIME typically removes present words (bits set to 1) to identify high-impact tokens. We propose \textbf{Q-LIME π} (Quantum LIME π), a quantum-inspired extension of LIME that encodes a binary feature vector in a quantum state, leveraging superposition and interference to explore local neighborhoods more efficiently. Our method focuses on flipping bits from 1→0 to emulate LIME's ``removal'' strategy, and can be extended to 0→1 where adding features is relevant. Experiments on subsets of the IMDb dataset demonstrate that Q-LIME π often achieves near-identical top-feature rankings compared to classical LIME while exhibiting lower runtime in small- to moderate-dimensional feature spaces. This quantum-classical hybrid approach thus provides a new pathway for interpretable AI, suggesting that, with further improvements in quantum hardware and methods, quantum parallelism may facilitate more efficient local explanations for high-dimensional data.
Explainable artificial intelligence (XAI) remains underexplored in quantum machine learning (QML) due to limited quantum computational resources and the lack of practical interpretability techniques. In this study, we implement the Local Interpretable Model-Agnostic Explanations (LIME) algorithm within the CUDA-Q framework and propose an interpretable QML analysis method using GPU-accelerated quantum simulation. All experiments are conducted on a GPU-based simulator rather than real quantum hardware, enabling stable large-scale sampling required by LIME and ensuring reproducible results. We design a binary classification model based on a single-qubit variational circuit composed of parameterized RY and RX rotations, and the model is trained with gradient-based optimization. The performance is evaluated on the MNIST, KMNIST, Fashion-MNIST, and CIFAR-10 datasets using training and test accuracy as metrics. By applying LIME, the classification outcomes of the quantum model are visualized and interpreted in a human-understandable form, demonstrating that meaningful interpretability can be achieved even under limited qubit counts and constrained computational environments. Furthermore, we analyze the trade-off between simulator performance and model accuracy, confirming that GPU-based CUDA-Q simulation provides a scalable and practical research environment for QML. This work offers a concrete pathway toward transparent and interpretable quantum machine learning.
Quantum Machine Learning (QML) has emerged as a promising paradigm that leverages quantum mechanical principles to accelerate machine learning tasks.Despite its potential, most QML models remain opaque "black boxes," hindering their deployment in high-stakes or regulated domains where trust and accountability are essential.This work introduces two novel interpretability methods tailored for QML: Quantum Shapley Values (QSV) and Quantum Local Interpretable Model-Agnostic Explanations (Q-LIME).QSV adapts cooperative game theory to quantify the contribution of individual qubits, observables, or circuit parameters, while Q-LIME generates perturbed quantum states and trains surrogate classical models to approximate local decision boundaries.Through experiments on variational quantum classifiers and hybrid quantum-classical neural networks, we demonstrate that QSV provides rigorous attribution but suffers from scalability constraints, whereas Q-LIME yields efficient, approximate explanations suitable for near-term devices.By transforming quantum models from opaque "black boxes" into interpretable "glass boxes," this research lays the foundation for trustworthy, transparent, and human centred quantum AI.
The rapid advancement of Artificial Intelligence (AI) has heightened concerns about the opacity of decision-making processes in complex models, particularly deep neural networks. This research explores a novel paradigm that integrates quantum computing principles into Explainable AI (XAI) to enhance interpretability without compromising predictive accuracy. The study introduces a Quantum-inspired Interpretable Model (QIIM) framework that leverages Hilbert space embeddings, unitary transformations, and operator-based learning to represent knowledge in a mathematically transparent form. The proposed model computes interpretability through the expectation values of observable operators, enabling the decomposition of decisions into quantifiable, human-understandable components. To capture hierarchical relationships, a Tensor Network Interpretable Model (TNIM) is further developed, offering scalable insights into complex dependencies among features. Experimental evaluations—performed on benchmark datasets for decision-making tasks—demonstrate that the quantum-inspired models achieve competitive accuracy while significantly improving local and global interpretability metrics compared to classical XAI techniques. The findings underscore the potential of quantum formalism as a new foundation for transparent AI systems, bridging the gap between computational efficiency and explainability. This study contributes to both theoretical understanding and practical advancement in interpretable machine learning, paving the way for ethically aligned, transparent, and human-trustworthy AI-driven decision support systems.