Quantum machine learning (QML) has attracted growing interest for its potential to match large classical models with far fewer parameters. This derives from encoding classical data representations into a Hilbert space exponentially larger than the number of qubits. In practice, however, noise mitigation limits these models to few qubits and shallow parameterized quantum circuits (PQCs), shrinking the accessible Hilbert space. This avoids noise but exposes a deeper problem of information loss that varies across samples: encoding compression. We uncover this bottleneck with the Hilbert-space encoding margin (HEM), partitioning inputs into a collapsed-encoding (CE) region and a preserved-encoding (PE) region, and find baseline accuracy on CE lagging PE by 43--58~pp across six setups. We propose quantum region-aware distillation (Q-RAD), a dual-PQC architecture that distills knowledge from a classical teacher separately on CE and PE samples. Under oracle routing, Q-RAD improves overall accuracy by up to +8.9/+8.1~pp over single-PQC and parameter-matched depth-doubled baselines. These results show that the encoding bottleneck can be exploited via its sample-level structure and architectural separation. Our code is available at https://github.com/skwonbb/Q-RAD/.
Quantum machine learning (QML) has attracted growing interest for its potential to match large classical models with far fewer parameters. This derives from encoding classical data representations into a Hilbert space exponentially larger than the number of qubits. In practice, however, noise mitigation limits these models to few qubits and shallow parameterized quantum circuits (PQCs), shrinking the accessible Hilbert space. This avoids noise but exposes a deeper problem of information loss that varies across samples: encoding compression. We uncover this bottleneck with the Hilbert-space encoding margin (HEM), partitioning inputs into a collapsed-encoding (CE) region and a preserved-encoding (PE) region, and find baseline accuracy on CE lagging PE by 43--58pp across six setups. We propose quantum region-aware distillation (Q-RAD), a dual-PQC architecture that distills knowledge from a classical teacher separately on CE and PE samples. Under oracle routing, Q-RAD improves overall accuracy by up to +8.9/+8.1pp over single-PQC and parameter-matched depth-doubled baselines. These results show that the encoding bottleneck can be exploited via its sample-level structure and architectural separation. Our code is available at https://github.com/skwonbb/Q-RAD/.
Abstract Quantum computing has emerged as a promising paradigm for addressing computational tasks intractable for classical systems, leveraging quantum mechanical principles such as superposition and entanglement to efficiently explore high-dimensional solution spaces. In recent years, hybrid quantum-classical approaches have gained increasing attention as a means to exploit the representational power of quantum systems while preserving the practicality of classical machine learning frameworks. This paper presents a study on quantum-enhanced model compression via feature-based knowledge distillation, in which a large classical neural network (the teacher) transfers knowledge to a substantially smaller hybrid student model. The proposed hybrid quantum-classical student architecture incorporates a parameterized quantum circuit (PQC) implemented in Qiskit, employing a linear $$\varvec{R_y}$$ angle-encoding feature map followed by an EfficientSU2 variational ansatz, with features bounded through a $$\varvec{\tanh (\cdot )\times \frac{\pi }{2}}$$ scaling to ensure stable gradient propagation. Gradient computation is performed via Reverse-Mode (adjoint) differentiation through the StatevectorEstimator primitive, which provides exact, shot-noise-free gradients at the cost of a single forward–backward pass per optimization step. The hybrid student is compared against a parameter-matched purely classical student under strict 1:1 parameter parity across a six-task binary benchmark spanning MNIST and Fashion-MNIST, as well as a four-class multi-class benchmark, totaling eleven classification experiments. A hardware validation experiment is additionally conducted on the IQM Garnet 20-qubit quantum processor using the Parameter-Shift Rule. The results indicate that the hybrid student outperforms its classical counterpart in five of six binary tasks (83%) and in the two hardest multi-class configurations, with advantages most pronounced in high-confusion regimes where compact classical classifiers approach the limits of their expressive capacity. On real quantum hardware, accuracy decreases from 99.04% in noiseless simulation to 80.50%, reflecting the impact of gate noise, readout errors, and transpilation overhead. While the hybrid model demonstrates improved parameter efficiency and expressive capacity, it incurs higher computational overhead and exhibits greater epoch-to-epoch training variability relative to classical baselines. We discuss practical limitations related to noise, scalability, and current hardware constraints, and identify directions for future work including hardware-efficient gradient estimation and advanced error mitigation protocols. Overall, our findings suggest that variational quantum circuits can serve as compact yet expressive components within neural compression schemes, with their representational advantage most effectively realized in binary and high-ambiguity classification scenarios.
Designing parameterized quantum circuits (PQCs) that are expressive, trainable, and robust to hardware noise is a central challenge for quantum machine learning (QML) on noisy intermediate-scale quantum (NISQ) devices. We present a Domain-Aware Quantum Circuit (DAQC) that leverages image priors to guide locality-preserving encoding and entanglement via non-overlapping DCT-style zigzag windows. The design employs interleaved encode-entangle-train cycles, where entanglement is applied among qubits hosting neighboring pixels, aligned to device connectivity. This staged, locality-preserving information flow expands the effective receptive field without deep global mixing, enabling efficient use of limited depth and qubits. The design concentrates representational capacity on short-range correlations, reduces long-range two-qubit operations, and encourages stable optimization, thereby mitigating depth-induced and globally entangled barren-plateau effects. We evaluate DAQC on MNIST, FashionMNIST, and PneumoniaMNIST datasets. On quantum hardware, DAQC achieves performance competitive with strong classical baselines (e.g., ResNet-18/50, DenseNet-121, EfficientNet-B0) and substantially outperforming Quantum Circuit Search (QCS) baselines. To the best of our knowledge, DAQC, which uses a quantum feature extractor with only a linear classical readout (no deep classical backbone), currently achieves the best reported performance on real quantum hardware for QML-based image classification tasks. Code and pretrained models are available at: https://github.com/gurinder-hub/DAQC.
Quantum neural networks (QNNs), harnessing superposition and entanglement, have shown potential to surpass classical methods in complex learning tasks but remain limited by hardware constraints and noisy conditions. In this work, we present a novel framework for transferring knowledge from classical convolutional neural networks (CNNs) to QNNs via knowledge distillation, thereby reducing the need for resource intensive quantum training and error mitigation. We conduct extensive experiments using two parameterized quantum circuits (PQCs) with 4 and 8 qubits on MNIST, Fashion MNIST, and CIFAR10 datasets. The approach demonstrates consistent accuracy improvements attributed to distilled knowledge from larger classical networks. Through ablation studies, we systematically compare the effect of state of the art dimensionality reduction techniques fully connected layers, center cropping, principal component analysis, and pooling to compress high-dimensional image data prior to quantum encoding. Our findings reveal that fully connected layers retain the most salient features for QNN inference, thereby surpassing other down sampling approaches. Additionally, we examine state of the art data encoding methods (amplitude, angle, and qubit encoding) and identify amplitude encoding as the optimal strategy, yielding superior accuracy across all tested datasets and qubit configurations. Through computational analyses, we show that our distilled 4-qubit and 8-qubit QNNs achieve competitive performance while utilizing significantly fewer parameters than their classical counterparts. Our results establish a promising paradigm for bridging classical deep learning and emerging quantum computing, paving the way for more powerful, resource conscious models in quantum machine intelligence.
Quantum machine learning (QML) models operating on classical data require an encoding step to map information into the Hilbert space. However, this imposes a computational bottleneck absent in classical frameworks, necessitating rigorous theoretical complexity analysis and algorithmic mitigations. We provide theoretical foundations of different encoding methods that prepare quantum states for both simulators and hardware, analyzing normal and sparse high energy physics data. In addition, we propose a quantum data augmentation method based on randomly sampled circuits to improve sample efficiency and mitigate the data scarcity bottleneck. We demonstrate the effectiveness of this method on a generative quantum diffusionlike model to produce synthetic samples of classical images. Our results demonstrate that choosing domain optimal encoding and applying the novel quantum data augmentation leads to faster convergence and improved model performance, offering a highly sample-efficient and scalable approach to encoding and augmenting sparse classical data for QML.
The rapid advancement of machine learning (ML) and the growing need for computational power have led to the exploration of quantum computing, which offers significant potential for faster complex calculations. However, Quantum Machine Learning (QML) faces challenges due to the limited number of qubits and noise of quantum circuits, particularly with Noisy Intermediate-Scale Quantum (NISQ) devices. These challenges severely limit the current capacity to train accurate and stable Quantum Machine Learning Models. In this paper, we propose a novel framework for QML that employs the knowledge distillation method to harness the power of well-trained classical machine learning (CML) models and enhance the training performance of QML models. In this framework, we utilize the well-trained CML as a teacher model to assist the training of the student QML model using the knowledge distillation method. By distilling knowledge from the robust CML model, our framework can potentially address the problem of the barren plateau which hinders effective model training. Knowledge distillation is well suited for this framework through the transfer of knowledge without parameter sharing. Through empirical tests, our framework has demonstrated not only an increase in the accuracy of QML models but also a notable improvement in training stability.
Knowledge distillation transfers predictions, representations, or other supervisory information between learning models and is increasingly investigated in quantum machine learning. Its applications extend beyond compression to cross-architecture transfer, circuit approximation, distributed learning, and classical deployment of models trained using quantum-derived supervision. This survey reviews these approaches using two complementary classification axes: teacher-student configuration and the role of quantum resources in the distillation process. Quantum-inspired methods are discussed separately from methods that simulate quantum computations classically or execute them on quantum hardware. We explain how measurement probabilities, expectation values, classical readouts, state overlaps, and circuit-level objectives support different transfer mechanisms. We then critically assess the presented results against baseline design, execution conditions, uncertainty, and resource costs. The evidence shows that distillation can improve selected quantum students, but benefits are task- and architecture-dependent, and several studies find negligible gains or even performance degradation. Evaluations on physical quantum hardware remain limited, with some testing individual quantum components rather than complete learning pipelines. Heterogeneous protocols and incomplete resource reporting prevent general conclusions about practical quantum advantage. We identify priorities for controlled benchmarking, noise-aware learning, reproducible evaluation, and efficient transfer between quantum and classical models.