Large Language Models (LLMs) like ChatGPT and LlaMA offer immense opportunities but face challenges due to their vast size, leading to high training/inference costs and energy demands.Traditional compression methods focus on reducing neurons or precision.We introduce CompactifAI, a novel LLM compression using quantum-inspired Tensor Networks to compress the model's correlation space.Testing on LlaMA-2 7B showed a 93% memory and 70% parameter reduction, with minimal accuracy loss (2-3%) and significant training (50%) and inference (25%) speedups.
Large Language Models (LLMs) like ChatGPT and LlaMA offer immense opportunities but face challenges due to their vast size, leading to high training/inference costs and energy demands.Traditional compression methods focus on reducing neurons or precision.We introduce CompactifAI, a novel LLM compression using quantum-inspired Tensor Networks to compress the model's correlation space.Testing on LlaMA-2 7B showed a 93% memory and 70% parameter reduction, with minimal accuracy loss (2-3%) and significant training (50%) and inference (25%) speedups.
As foundational tools in natural language processing, Large Language Models (LLMs) have immense parameter scales, which makes deployment and inference increasingly prohibitive, especially in resource-constrained devices. Therefore, knowledge distillation for LLMs, i.e., compressing the LLM to a smaller model, is meaningful. With strong parameter representation capacity, quantum computing is regarded as a promising solution. Here, we propose a Quantum knowledge Distillation model for LLMs (QD-LLM) that leverages variational quantum circuits to learn from LLMs. In classical simulation, QD-LLM outperforms several mainstream distillation methods on multiple text classification tasks in terms of both accuracy and efficiency using only 11 qubits. The results reveal an interesting phenomenon that the simulation of quantum student models may be regarded as a new class of quantum-inspired classical algorithms. Remarkably, we deploy the obtained circuits on the Baihua superconducting quantum processor via the Quafu platform to assess practical feasibility. The model maintains stable inference performance despite hardware constraints such as decoherence and finite sampling. In summary, QD-LLM marks a foundational step in connecting quantum computing with LLMs, demonstrating the feasibility of quantum-native approaches that aim to compress and deploy models of increasingly larger scales. The code of this article has been open-sourced at https://github.com/Lilingxiao-bupt/QD-LLM.
Large Language Models (LLMs) such as GPT and LLaMA have transformed artificial intelligence, but their rapid growth has brought major challenges in computation, energy use, and scalability. Quantum computing (QC) offers a fundamentally different way to process information, with the potential to accelerate key operations in training, inference, optimization, and representation learning. This survey provides a structured and comprehensive review of how QC can support the future development of LLMs. It examines quantum algorithms, hybrid quantum-classical methods, quantum neural networks, quantum embeddings, and security applications. It also evaluates their feasibility in both the current noisy intermediate-scale quantum (NISQ) era and the future fault-tolerant era. The survey highlights current progress, identifies major technical barriers, and outlines practical research directions needed to build scalable, efficient, and secure quantum-enhanced language models.
This paper investigates the application of quantum-inspired compression algorithms to reduce the size of neural network weights. The core idea is to mimic certain aspects of quantum entanglement to create a novel compression method. Traditional neural network compression techniques often rely on methods like pruning or quantization, which can introduce accuracy loss. Our approach aims to minimize this loss by leveraging concepts from quantum mechanics, specifically the correlation and superposition properties, to identify and eliminate redundant weight information. We present a mathematical framework for this algorithm, detailing the core operations and the objective function used for optimization. The algorithm is designed to achieve a significant reduction in weight size without substantially impacting the accuracy of the neural network. The proposed method offers a potentially more efficient and accurate way to compress neural networks, particularly for large and complex models. Experimental results, although not presented through formal experimentation due to the theoretical nature of the work, would demonstrate the feasibility and effectiveness of the approach. The primary focus is on establishing the theoretical foundations and outlining the algorithmic steps, paving the way for future implementation and validation.
As LLMs grow in size, compression becomes increasingly important for efficient deployment. SVD-based low-rank compression reduces parameter counts but can degrade downstream performance. To improve performance after compression, we introduce QuLoC, a photonic quantum-assisted LLM compression algorithm that uses quantum circuit outputs to gate the retained low-rank components during training. Model performance is recovered through local functional reconstruction followed by end-to-end knowledge distillation. After training, the gating coefficients are absorbed into the low-rank factors, allowing the compressed model to run on classical hardware without executing quantum circuits during inference. Experiments on Qwen3.5-4B show that QuLoC achieves a 9.73\% relative improvement in average accuracy over state-of-the-art baselines across multiple downstream tasks, demonstrating its effectiveness. We further evaluate QuLoC on LLaMA-7B at different parameter compression ratios. It achieves comparable or higher average downstream accuracy than state-of-the-art baselines, supporting its applicability to a larger model across different compression settings. These results motivate further exploration of photonic quantum-assisted compression for larger models and more complex agentic tasks.
Quantum-centric supercomputing presents a compelling framework for large-scale hybrid quantum-classical tasks. Although quantum machine learning (QML) offers theoretical benefits in various applications, challenges such as large-size data encoding in the input stage and the reliance on quantum resources in the inference stage limit its practicality for tasks like fine-tuning large language models (LLMs). Quantum parameter generation, a novel approach of QML, addresses these limitations by using quantum neural networks (QNNs) to generate classical model weights (parameters) exclusively during training, thereby decoupling inference from quantum hardware. In this work, we introduce Quantum Parameter Adaptation (QPA) in the framework of quantum parameter generation, which integrates QNNs with a classical multi-layer perceptron mapping model to generate parameters for fine-tuning methods. Using Gemma-2 and GPT-2 as case studies, QPA demonstrates significant parameter reduction for parameter-efficient fine-tuning methods, such as Low-Rank Adaptation (LoRA), while maintaining comparable or improved performance in text generation tasks. Specifically, QPA reduces the number of parameters to 52.06% of the original LoRA for GPT-2 with a slight performance gain of 0.75%, and to 16.84% for Gemma-2, with a marginal performance improvement of 0.07%. These results highlight QPA's ability to achieve efficient parameter reduction without sacrificing performance in the quantum parameter generation framework. This work showcases the potential of quantum-enhanced parameter reduction, offering a scalable quantum-classical solution for fine-tuning LLMs while preserving the feasibility of inference on classical hardware.
The rapid progress of large language models (LLMs) has transformed natural language processing, yet the challenge of efficient adaptation remains unresolved. Full fine-tuning achieves strong performance but imposes prohibitive computational and memory costs. Parameter-efficient fine-tuning (PEFT) strategies, such as low-rank adaptation (LoRA), Prefix tuning, and sparse low-rank adaptation (SoRA), address this issue by reducing trainable parameters while maintaining competitive accuracy. However, these methods often encounter limitations in scalability, stability, and generalization across diverse tasks. Recent advances in quantum deep learning introduce novel opportunities through quantum-inspired encoding and parameterized quantum circuits (PQCs). In particular, the quantum-amplitude embedded adaptation (QAA) framework demonstrates expressive model updates with minimal overhead. This paper presents a systematic survey and comparative analysis of conventional PEFT methods and QAA. The analysis demonstrates trade-offs in convergence, efficiency, and representational capacity, while providing insight into the potential of quantum approaches for future LLM adaptation.