While Quantum Neural Networks (QNNs) offer the potential for resource-efficient image classification, practical implementations are hindered by qubit instability and finite availability, causing them to often underperform classical counterparts. This performance disparity constitutes a critical research gap. We propose a novel hybrid classical-quantum knowledge distillation approach to bridge this gap. Our method utilizes a pre-trained classical CNN (AlexNet) as a teacher to guide the training of a quantum student model by minimizing the Kullback-Leibler (KL) divergence (DKL) between their output distributions. The quantum student model employs an architecture designed to facilitate intricate feature interactions using iterated CNOT gates. This technique substantially improves quantum model accuracy, demonstrating gains of 0.34% (6-class, 3 qubits) and 0.28% (8-class, 3 qubits) on the MNIST dataset, and 0.74% (8-class, 4 qubits). Our contribution lies in operationalizing a knowledge distillation paradigm for applied Quantum Machine Learning (QML), enabling the effective transfer of robust knowledge from established classical architectures to resource-constrained quantum models, thereby enhancing faster convergence and accelerating practical QML deployment.
While Quantum Neural Networks (QNNs) offer the potential for resource-efficient image classification, practical implementations are hindered by qubit instability and finite availability, causing them to often underperform classical counterparts. This performance disparity constitutes a critical research gap. We propose a novel hybrid classical-quantum knowledge distillation approach to bridge this gap. Our method utilizes a pre-trained classical CNN (AlexNet) as a teacher to guide the training of a quantum student model by minimizing the Kullback-Leibler (KL) divergence (DKL) between their output distributions. The quantum student model employs an architecture designed to facilitate intricate feature interactions using iterated CNOT gates. This technique substantially improves quantum model accuracy, demonstrating gains of 0.34% (6-class, 3 qubits) and 0.28% (8-class, 3 qubits) on the MNIST dataset, and 0.74% (8-class, 4 qubits). Our contribution lies in operationalizing a knowledge distillation paradigm for applied Quantum Machine Learning (QML), enabling the effective transfer of robust knowledge from established classical architectures to resource-constrained quantum models, thereby enhancing faster convergence and accelerating practical QML deployment.
Quantum neural networks (QNNs), harnessing superposition and entanglement, have shown potential to surpass classical methods in complex learning tasks but remain limited by hardware constraints and noisy conditions. In this work, we present a novel framework for transferring knowledge from classical convolutional neural networks (CNNs) to QNNs via knowledge distillation, thereby reducing the need for resource intensive quantum training and error mitigation. We conduct extensive experiments using two parameterized quantum circuits (PQCs) with 4 and 8 qubits on MNIST, Fashion MNIST, and CIFAR10 datasets. The approach demonstrates consistent accuracy improvements attributed to distilled knowledge from larger classical networks. Through ablation studies, we systematically compare the effect of state of the art dimensionality reduction techniques fully connected layers, center cropping, principal component analysis, and pooling to compress high-dimensional image data prior to quantum encoding. Our findings reveal that fully connected layers retain the most salient features for QNN inference, thereby surpassing other down sampling approaches. Additionally, we examine state of the art data encoding methods (amplitude, angle, and qubit encoding) and identify amplitude encoding as the optimal strategy, yielding superior accuracy across all tested datasets and qubit configurations. Through computational analyses, we show that our distilled 4-qubit and 8-qubit QNNs achieve competitive performance while utilizing significantly fewer parameters than their classical counterparts. Our results establish a promising paradigm for bridging classical deep learning and emerging quantum computing, paving the way for more powerful, resource conscious models in quantum machine intelligence.
The rapid advancement of machine learning (ML) and the growing need for computational power have led to the exploration of quantum computing, which offers significant potential for faster complex calculations. However, Quantum Machine Learning (QML) faces challenges due to the limited number of qubits and noise of quantum circuits, particularly with Noisy Intermediate-Scale Quantum (NISQ) devices. These challenges severely limit the current capacity to train accurate and stable Quantum Machine Learning Models. In this paper, we propose a novel framework for QML that employs the knowledge distillation method to harness the power of well-trained classical machine learning (CML) models and enhance the training performance of QML models. In this framework, we utilize the well-trained CML as a teacher model to assist the training of the student QML model using the knowledge distillation method. By distilling knowledge from the robust CML model, our framework can potentially address the problem of the barren plateau which hinders effective model training. Knowledge distillation is well suited for this framework through the transfer of knowledge without parameter sharing. Through empirical tests, our framework has demonstrated not only an increase in the accuracy of QML models but also a notable improvement in training stability.
Quantum Neural Networks (QNNs) are a promising class of quantum machine learning models with potential quantum advantages when implemented on scalable, error-corrected quantum computers. However, as system sizes increase, deploying QNNs becomes challenging. Similar to their classical counterparts, a key obstacle to their practical applications is that large-scale QNNs may not be easily deployed on smaller systems that have limited resources. Here, we tackle this challenge by compressing QNNs via knowledge distillation. We demonstrate how well-trained QNNs on large systems can be distilled into smaller architectures with similar configurations. We numerically show that knowledge distillation helps reduce the training cost of QNNs in terms of the number of qubits and circuit depth. Additionally, we find that a self-knowledge-distillation approach can accelerate training convergence. We believe our results offer new strategies for the efficient compression and practical deployment of QNNs.
Abstract Quantum computing has emerged as a promising paradigm for addressing computational tasks intractable for classical systems, leveraging quantum mechanical principles such as superposition and entanglement to efficiently explore high-dimensional solution spaces. In recent years, hybrid quantum-classical approaches have gained increasing attention as a means to exploit the representational power of quantum systems while preserving the practicality of classical machine learning frameworks. This paper presents a study on quantum-enhanced model compression via feature-based knowledge distillation, in which a large classical neural network (the teacher) transfers knowledge to a substantially smaller hybrid student model. The proposed hybrid quantum-classical student architecture incorporates a parameterized quantum circuit (PQC) implemented in Qiskit, employing a linear $$\varvec{R_y}$$ angle-encoding feature map followed by an EfficientSU2 variational ansatz, with features bounded through a $$\varvec{\tanh (\cdot )\times \frac{\pi }{2}}$$ scaling to ensure stable gradient propagation. Gradient computation is performed via Reverse-Mode (adjoint) differentiation through the StatevectorEstimator primitive, which provides exact, shot-noise-free gradients at the cost of a single forward–backward pass per optimization step. The hybrid student is compared against a parameter-matched purely classical student under strict 1:1 parameter parity across a six-task binary benchmark spanning MNIST and Fashion-MNIST, as well as a four-class multi-class benchmark, totaling eleven classification experiments. A hardware validation experiment is additionally conducted on the IQM Garnet 20-qubit quantum processor using the Parameter-Shift Rule. The results indicate that the hybrid student outperforms its classical counterpart in five of six binary tasks (83%) and in the two hardest multi-class configurations, with advantages most pronounced in high-confusion regimes where compact classical classifiers approach the limits of their expressive capacity. On real quantum hardware, accuracy decreases from 99.04% in noiseless simulation to 80.50%, reflecting the impact of gate noise, readout errors, and transpilation overhead. While the hybrid model demonstrates improved parameter efficiency and expressive capacity, it incurs higher computational overhead and exhibits greater epoch-to-epoch training variability relative to classical baselines. We discuss practical limitations related to noise, scalability, and current hardware constraints, and identify directions for future work including hardware-efficient gradient estimation and advanced error mitigation protocols. Overall, our findings suggest that variational quantum circuits can serve as compact yet expressive components within neural compression schemes, with their representational advantage most effectively realized in binary and high-ambiguity classification scenarios.
While hybrid quantum–classical architectures and knowledge distillation have each been explored independently in medical imaging and model compression, no prior work has unified these domains within a single framework. This study addresses this gap by investigating the integration of quantum–classical neural networks with knowledge distillation for medical image analysis, with a focus on circuit behavior, parameter efficiency, and diagnostic performance. The framework (QuantumMedKD) employs classical CNN architectures (ResNet-50, EfficientNet-B0, Xception) as teacher models, transferring learned representations to parameter-efficient quantum student networks via parameterized quantum circuits across qubit configurations (3-8), demonstrated through pneumonia detection from pediatric chest radiographs. Experimental validation reveals remarkable parameter efficiency, requiring only 24-36 trainable parameters compared to millions for classical counterparts, achieving compression ratios exceeding 1 0 5 while maintaining competitive diagnostic performance. Optimal configurations achieve 84.00% accuracy (EfficientNet-B0, 6-qubit) and 73.33% (Xception, 4-qubit), with knowledge distillation providing statistically significant improvements ( p -vallues < 0.001, Cohen’s d > 2.0). Medical evaluation confirms diagnostic capability with sensitivity 81.4%, specificity 73.9%, and AUC 0.89, establishing quantum–classical knowledge transfer viability for resource-constrained healthcare deployment. QuantumMedKD reveals essential principles and proof-of-concept for quantum-enhanced healthcare AI systems within and beyond the noisy intermediate-scale quantum (NISQ) era for high efficiency and deployability, paving the way for future advancements in the field.
Image classification is a fundamental task in deep learning, and recent advances in quantum computing have generated significant interest in quantum neural networks. Traditionally, Convolutional Neural Networks (CNNs) are employed to extract image features, while Multilayer Perceptrons (MLPs) handle decision making. However, parameterized quantum circuits offer the potential to capture complex image features and define sophisticated decision boundaries. In this paper, we present a novel Hybrid Quantum–Classical Neural Network (H-QNN) for image classification, and demonstrate its effectiveness using the MNIST dataset. Our model combines quantum computing with classical supervised learning to enhance classification accuracy and computational efficiency. In this study, we detail the architecture of the H-QNN, emphasizing its capability in feature learning and image classification. Experimental results demonstrate that the proposed H-QNN model outperforms conventional deep learning methods in various training scenarios, showcasing its effectiveness in high-dimensional image classification tasks. Additionally, we explore the broader applicability of hybrid quantum–classical approaches in other domains. Our findings contribute to the growing body of work in quantum machine learning, and underscore the potential of quantum-enhanced models for image recognition and classification.