Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

IEEE International Symposium on Circuits and Systems proceedings

Knowledge Distillation in the QAOA Setting for Advantage on Quantum Hardware

Simone Piperno, David Windridge, Giacomo Vittori, Antonello Rosato, Massimo Panella

Published May 24, 2026• 0 citations
Publisher

Abstract

In this work, we investigate the application of Knowledge Distillation (KD) to the Quantum Approximate Optimization Algorithm (QAOA) setting, introducing a framework to distill knowledge from a Multi-angle QAOA teacher to a standard, single-layer QAOA student. We propose three distillation schemes: count-based KD, distillation using MSE, and distillation with Huber loss for enhanced robustness. Numerical experiments on diverse graph families, including complete graphs, Erdős–Rényi graphs, complete binary trees, and random 3-regular graphs, suggest that MSE and Huber-based distillation can improve the student’s performance, while count-based KD exhibits higher variance and limited generalization. Our results indicate that KD can enable a compression of the QAOA parameter space while often maintaining competitive solution quality, providing a practical avenue for efficient deployment on Noisy Intermediate-Scale Quantum hardware with improved training stability.

DOI
10.1109/iscas66217.2026.11562409

Related papers

Quantum Relational Knowledge Distillation

Chen-Yu Liu, Kuan‐Cheng Chen, Keisuke Murota +2 more

Knowledge distillation (KD) is a widely adopted technique for compressing large models into smaller, more efficient student models that can be deployed on devices with limited computational resources. Among various KD methods, Relational Knowledge Distillation (RKD) improves student performance by aligning relational structures in the feature space, such as pairwise distances and angles. In this work, we propose Quantum Relational Knowledge Distillation (QRKD), which extends RKD by incorporating quantum relational information. Specifically, we map classical features into a Hilbert space, interpret them as quantum states, and compute quantum kernel values to capture richer inter-sample relationships. These quantum-informed relations are then used to guide the distillation process. We evaluate QRKD on both vision and language tasks, including CNNs on MNIST and CIFAR-10, and GPT-2 on WikiText-2, Penn Treebank, and IMDB. Across all benchmarks, QRKD consistently improves student model performance compared to classical RKD. Importantly, both teacher and student models remain classical and deployable on standard hardware, with quantum computation required only during training. This work presents the first demonstration of quantum-enhanced knowledge distillation in a fully classical deployment setting.

arXiv:2508.13054•Aug 18, 2025•7 citations

Improved Performance of Multi-Angle Quantum Approximate Optimization Algorithm (ma-QAOA) Compared to QAOA On Simulation and Experimental Hardware Platforms

Vandit Srivastava, P Rohith, Sanyam Singhal +1 more

Multi-angle quantum approximate optimization algorithm (ma-QAOA) has been proposed as an alternative to regular QAOA for solving optimization problems using noisy intermediate scale quantum (NISQ) hardware. Here, we first implement QAOA and ma-QAOA algorithms to solve MaxCut on weighted and unweighted versions of the Erdős–Rényi (ER) and random cubic (RC) graphs of 6 and 10 nodes. Quantum circuit parameters are updated iteratively through classical feedback with the quantum circuit for forward pass modelled on Qiskit simulator without noise models. Then, the quantum circuit for the final run with updated parameters is implemented on Qiskit simulator without noise models, Qiskit simulator with experimentally benchmarked noise models (fake back-end), and also on actual experimental quantum hardware (IBM’s 127-qubit quantum processor ibm brisbane). We show that for most graph instances, ma-QAOA performs better than QAOA (in terms of approximation ratio (AR)) and needs less stages in the quantum circuit for all three aforementioned methods. Finally, we extend our results up to 14-node RC graphs while restricting ourselves to Qiskit simulation without noise (both for parameter update and final run) and show that for most instances, ma-QAOA again needs much less stages compared to QAOA to obtain reasonable AR.

International Conference on Communication Systems & Networks (COMSNETS)•Jan 6, 2025•4 citations

Quantum Knowledge Distillation: State-of-the-art, Applications and Open Problems

Aldana Al-Sulaiti, Kuan‐Cheng Chen, Muhammad Bilal Akram Dastagir +2 more

Knowledge distillation transfers predictions, representations, or other supervisory information between learning models and is increasingly investigated in quantum machine learning. Its applications extend beyond compression to cross-architecture transfer, circuit approximation, distributed learning, and classical deployment of models trained using quantum-derived supervision. This survey reviews these approaches using two complementary classification axes: teacher-student configuration and the role of quantum resources in the distillation process. Quantum-inspired methods are discussed separately from methods that simulate quantum computations classically or execute them on quantum hardware. We explain how measurement probabilities, expectation values, classical readouts, state overlaps, and circuit-level objectives support different transfer mechanisms. We then critically assess the presented results against baseline design, execution conditions, uncertainty, and resource costs. The evidence shows that distillation can improve selected quantum students, but benefits are task- and architecture-dependent, and several studies find negligible gains or even performance degradation. Evaluations on physical quantum hardware remain limited, with some testing individual quantum components rather than complete learning pipelines. Heterogeneous protocols and incomplete resource reporting prevent general conclusions about practical quantum advantage. We identify priorities for controlled benchmarking, noise-aware learning, reproducible evaluation, and efficient transfer between quantum and classical models.

arXiv:2611.08461•Nov 5, 2026•0 citations

Hybrid Quantum Classical Machine Learning with Knowledge Distillation

Mingze Li, Lei Fan, Aaron Cummings +3 more

The rapid advancement of machine learning (ML) and the growing need for computational power have led to the exploration of quantum computing, which offers significant potential for faster complex calculations. However, Quantum Machine Learning (QML) faces challenges due to the limited number of qubits and noise of quantum circuits, particularly with Noisy Intermediate-Scale Quantum (NISQ) devices. These challenges severely limit the current capacity to train accurate and stable Quantum Machine Learning Models. In this paper, we propose a novel framework for QML that employs the knowledge distillation method to harness the power of well-trained classical machine learning (CML) models and enhance the training performance of QML models. In this framework, we utilize the well-trained CML as a teacher model to assist the training of the student QML model using the knowledge distillation method. By distilling knowledge from the robust CML model, our framework can potentially address the problem of the barren plateau which hinders effective model training. Knowledge distillation is well suited for this framework through the transfer of knowledge without parameter sharing. Through empirical tests, our framework has demonstrated not only an increase in the accuracy of QML models but also a notable improvement in training stability.

International Conference on Communications•Jun 9, 2024•8 citations

Staged Hybridisation for Visual Quantum Reinforcement Learning via Knowledge Distillation

Javier Lazaro, Juan-Ignacio Vazquez, Pablo Garcia-Bringas

Visual environments are a demanding setting for quantum reinforcement learning (QRL): high-dimensional observations, unstable RL optimisation, and constrained variational quantum circuits (VQCs) are difficult to train jointly. This paper studies knowledge distillation (KD) as a staged hybridisation strategy for visual QRL. Instead of training a hybrid visual agent end-to-end from pixels, we first train a classical visual teacher, freeze its encoder as a feature interface, and distil the teacher's policy behaviour into compact downstream heads. These heads can be classical or VQC-based, enabling small quantum-compatible students to be evaluated under the same frozen representation as compact classical controls. We evaluate the pipeline on CartPole Pixels and Acrobot Pixels. The results show that staged KD enables shallow VQC heads to acquire non-trivial visual-control behaviour in settings where direct pixel-based training would be substantially more difficult. Angle-encoded VQC heads retain near-teacher performance, while amplitude-encoded heads push compactness to an extreme regime, at the cost of greater fragility, stronger budget sensitivity, and higher simulation time. Overall, staged KD reframes visual QRL as a compact-head learning problem, opening a practical route for training small quantum-compatible policies outside the standard end-to-end RL loop.

quant-pharXiv:2606.30520•Jun 30, 2026•1 citation

SAFE ma-QAOA: Surrogate-Assisted and Fine-Tuning Enhanced Multi-Angle QAOA with Parameter Distillation

Hyunwoo Kim, Youngseok Lee

The multi-angle Quantum Approximate Optimization Algorithm (ma-QAOA) extends the Quantum Approximate Optimization Algorithm (QAOA) by assigning a larger number of independent variational parameters, thereby increasing expressivity and improving performance at low circuit depths. However, this larger parameterization makes training more difficult and requires repeated circuit evaluations for gradient-based optimization. In this work, we propose the Surrogate-Assisted and Fine-tuning Enhanced (SAFE) framework. SAFE first uses Low-Weight Pauli Propagation (LWPP) as a classical surrogate for pre-training ma-QAOA parameters before exact optimization. SAFE then applies parameter distillation, which removes angles that remain near zero after surrogate pre-training. Finally, SAFE performs exact fine-tuning by optimizing the remaining active parameters using the exact energy objective. We evaluate SAFE on instances of the Sherrington-Kirkpatrick model, two-dimensional square-lattice spin glass, and Max-Cut. SAFE with distillation provides the strongest overall results relative to exact-only: (i) a 64.3 percent reduction in active parameter count and (ii) a 94.5 percent reduction in estimated QPU workload. Within the SAFE workflow, adding distillation further reduces the optimizer steps to the near-optimal regime by 44.4 percent relative to without distillation. These results provide evidence that SAFE ma-QAOA can accelerate convergence to high-quality solutions while reducing the required quantum resources for exact fine-tuning, offering a resource-efficient route toward expressive ma-QAOA on NISQ hardware.

quant-pharXiv:2605.23377•May 25, 2026•2 citations
In this work, we investigate the application of Knowledge Distillation (KD) to the Quantum Approximate Optimization Algorithm (QAOA) setting, introducing a framework to distill knowledge from a Multi-angle QAOA teacher to a standard, single-layer QAOA student. We propose three distillation schemes: count-based KD, distillation using MSE, and distillation with Huber loss for enhanced robustness. Numerical experiments on diverse graph families, including complete graphs, Erdős–Rényi graphs, complete binary trees, and random 3-regular graphs, suggest that MSE and Huber-based distillation can improve the student’s performance, while count-based KD exhibits higher variance and limited generalization. Our results indicate that KD can enable a compression of the QAOA parameter space while often maintaining competitive solution quality, providing a practical avenue for efficient deployment on Noisy Intermediate-Scale Quantum hardware with improved training stability.
Knowledge distillation (KD) is a widely adopted technique for compressing large models into smaller, more efficient student models that can be deployed on devices with limited computational resources. Among various KD methods, Relational Knowledge Distillation (RKD) improves student performance by aligning relational structures in the feature space, such as pairwise distances and angles. In this work, we propose Quantum Relational Knowledge Distillation (QRKD), which extends RKD by incorporating quantum relational information. Specifically, we map classical features into a Hilbert space, interpret them as quantum states, and compute quantum kernel values to capture richer inter-sample relationships. These quantum-informed relations are then used to guide the distillation process. We evaluate QRKD on both vision and language tasks, including CNNs on MNIST and CIFAR-10, and GPT-2 on WikiText-2, Penn Treebank, and IMDB. Across all benchmarks, QRKD consistently improves student model performance compared to classical RKD. Importantly, both teacher and student models remain classical and deployable on standard hardware, with quantum computation required only during training. This work presents the first demonstration of quantum-enhanced knowledge distillation in a fully classical deployment setting.
Multi-angle quantum approximate optimization algorithm (ma-QAOA) has been proposed as an alternative to regular QAOA for solving optimization problems using noisy intermediate scale quantum (NISQ) hardware. Here, we first implement QAOA and ma-QAOA algorithms to solve MaxCut on weighted and unweighted versions of the Erdős–Rényi (ER) and random cubic (RC) graphs of 6 and 10 nodes. Quantum circuit parameters are updated iteratively through classical feedback with the quantum circuit for forward pass modelled on Qiskit simulator without noise models. Then, the quantum circuit for the final run with updated parameters is implemented on Qiskit simulator without noise models, Qiskit simulator with experimentally benchmarked noise models (fake back-end), and also on actual experimental quantum hardware (IBM’s 127-qubit quantum processor ibm brisbane). We show that for most graph instances, ma-QAOA performs better than QAOA (in terms of approximation ratio (AR)) and needs less stages in the quantum circuit for all three aforementioned methods. Finally, we extend our results up to 14-node RC graphs while restricting ourselves to Qiskit simulation without noise (both for parameter update and final run) and show that for most instances, ma-QAOA again needs much less stages compared to QAOA to obtain reasonable AR.
Knowledge distillation transfers predictions, representations, or other supervisory information between learning models and is increasingly investigated in quantum machine learning. Its applications extend beyond compression to cross-architecture transfer, circuit approximation, distributed learning, and classical deployment of models trained using quantum-derived supervision. This survey reviews these approaches using two complementary classification axes: teacher-student configuration and the role of quantum resources in the distillation process. Quantum-inspired methods are discussed separately from methods that simulate quantum computations classically or execute them on quantum hardware. We explain how measurement probabilities, expectation values, classical readouts, state overlaps, and circuit-level objectives support different transfer mechanisms. We then critically assess the presented results against baseline design, execution conditions, uncertainty, and resource costs. The evidence shows that distillation can improve selected quantum students, but benefits are task- and architecture-dependent, and several studies find negligible gains or even performance degradation. Evaluations on physical quantum hardware remain limited, with some testing individual quantum components rather than complete learning pipelines. Heterogeneous protocols and incomplete resource reporting prevent general conclusions about practical quantum advantage. We identify priorities for controlled benchmarking, noise-aware learning, reproducible evaluation, and efficient transfer between quantum and classical models.
The rapid advancement of machine learning (ML) and the growing need for computational power have led to the exploration of quantum computing, which offers significant potential for faster complex calculations. However, Quantum Machine Learning (QML) faces challenges due to the limited number of qubits and noise of quantum circuits, particularly with Noisy Intermediate-Scale Quantum (NISQ) devices. These challenges severely limit the current capacity to train accurate and stable Quantum Machine Learning Models. In this paper, we propose a novel framework for QML that employs the knowledge distillation method to harness the power of well-trained classical machine learning (CML) models and enhance the training performance of QML models. In this framework, we utilize the well-trained CML as a teacher model to assist the training of the student QML model using the knowledge distillation method. By distilling knowledge from the robust CML model, our framework can potentially address the problem of the barren plateau which hinders effective model training. Knowledge distillation is well suited for this framework through the transfer of knowledge without parameter sharing. Through empirical tests, our framework has demonstrated not only an increase in the accuracy of QML models but also a notable improvement in training stability.
Visual environments are a demanding setting for quantum reinforcement learning (QRL): high-dimensional observations, unstable RL optimisation, and constrained variational quantum circuits (VQCs) are difficult to train jointly. This paper studies knowledge distillation (KD) as a staged hybridisation strategy for visual QRL. Instead of training a hybrid visual agent end-to-end from pixels, we first train a classical visual teacher, freeze its encoder as a feature interface, and distil the teacher's policy behaviour into compact downstream heads. These heads can be classical or VQC-based, enabling small quantum-compatible students to be evaluated under the same frozen representation as compact classical controls. We evaluate the pipeline on CartPole Pixels and Acrobot Pixels. The results show that staged KD enables shallow VQC heads to acquire non-trivial visual-control behaviour in settings where direct pixel-based training would be substantially more difficult. Angle-encoded VQC heads retain near-teacher performance, while amplitude-encoded heads push compactness to an extreme regime, at the cost of greater fragility, stronger budget sensitivity, and higher simulation time. Overall, staged KD reframes visual QRL as a compact-head learning problem, opening a practical route for training small quantum-compatible policies outside the standard end-to-end RL loop.
The multi-angle Quantum Approximate Optimization Algorithm (ma-QAOA) extends the Quantum Approximate Optimization Algorithm (QAOA) by assigning a larger number of independent variational parameters, thereby increasing expressivity and improving performance at low circuit depths. However, this larger parameterization makes training more difficult and requires repeated circuit evaluations for gradient-based optimization. In this work, we propose the Surrogate-Assisted and Fine-tuning Enhanced (SAFE) framework. SAFE first uses Low-Weight Pauli Propagation (LWPP) as a classical surrogate for pre-training ma-QAOA parameters before exact optimization. SAFE then applies parameter distillation, which removes angles that remain near zero after surrogate pre-training. Finally, SAFE performs exact fine-tuning by optimizing the remaining active parameters using the exact energy objective. We evaluate SAFE on instances of the Sherrington-Kirkpatrick model, two-dimensional square-lattice spin glass, and Max-Cut. SAFE with distillation provides the strongest overall results relative to exact-only: (i) a 64.3 percent reduction in active parameter count and (ii) a 94.5 percent reduction in estimated QPU workload. Within the SAFE workflow, adding distillation further reduces the optimizer steps to the near-optimal regime by 44.4 percent relative to without distillation. These results provide evidence that SAFE ma-QAOA can accelerate convergence to high-quality solutions while reducing the required quantum resources for exact fine-tuning, offering a resource-efficient route toward expressive ma-QAOA on NISQ hardware.