Large language models (LLMs) require substantial resources for task-specific adaptation, that motivates the development of parameter-efficient fine-tuning (PEFT) methods. This paper presents quantum-amplitude embedded adaptation (QAA), a novel PEFT framework that logarithmically compresses activation vectors using quantum-amplitude embedding and applies expressive non-linear transformations via parameterized quantum circuits (PQCs). By replacing linear adapters in attention modules with compact quantum modules, QAA achieves high expressivity while drastically reducing the number of trainable parameters. Empirical results demonstrate that QAA performs on par with or better than existing PEFT under constrained memory and compute budgets, highlighting its potential for efficient LLM fine-tuning.
Large language models (LLMs) require substantial resources for task-specific adaptation, that motivates the development of parameter-efficient fine-tuning (PEFT) methods. This paper presents quantum-amplitude embedded adaptation (QAA), a novel PEFT framework that logarithmically compresses activation vectors using quantum-amplitude embedding and applies expressive non-linear transformations via parameterized quantum circuits (PQCs). By replacing linear adapters in attention modules with compact quantum modules, QAA achieves high expressivity while drastically reducing the number of trainable parameters. Empirical results demonstrate that QAA performs on par with or better than existing PEFT under constrained memory and compute budgets, highlighting its potential for efficient LLM fine-tuning.
The rapid progress of large language models (LLMs) has transformed natural language processing, yet the challenge of efficient adaptation remains unresolved. Full fine-tuning achieves strong performance but imposes prohibitive computational and memory costs. Parameter-efficient fine-tuning (PEFT) strategies, such as low-rank adaptation (LoRA), Prefix tuning, and sparse low-rank adaptation (SoRA), address this issue by reducing trainable parameters while maintaining competitive accuracy. However, these methods often encounter limitations in scalability, stability, and generalization across diverse tasks. Recent advances in quantum deep learning introduce novel opportunities through quantum-inspired encoding and parameterized quantum circuits (PQCs). In particular, the quantum-amplitude embedded adaptation (QAA) framework demonstrates expressive model updates with minimal overhead. This paper presents a systematic survey and comparative analysis of conventional PEFT methods and QAA. The analysis demonstrates trade-offs in convergence, efficiency, and representational capacity, while providing insight into the potential of quantum approaches for future LLM adaptation.
When full-parameter updates are impractical for large language models (LLMs), parameter-efficient fine-tuning (PEFT) is commonly employed to reduce the number of trainable parameters. Classical PEFT methods, such as low-rank adaptation (LoRA), are limited to linear transformations and may not capture complex, high-order feature interactions under stringent parameter constraints. In contrast, parameterized quantum circuits (PQCs) provide high-order expressivity per qubit rotation degree of freedom. Inspired by PQCs, in this paper, we propose quantum cross-hybrid fine-tuning (QCHFT), a suite of four PQC-augmented adapters: Matrix Product State PQC, Alternating and Hierarchical Pairwise Contraction PQC, and LinearA–Quantum–LinearB PQC. To evaluate our QCHFT adapters, we conduct experiments on a public multi-tier IoT resource-allocation task that serves as a challenging benchmark with intricate cross-tier interactions data and limited parameter budgets. We conduct experiments on a multi-tier IoT resource-allocation task across five causal LLM backbones scaling from 125M to 2B parameters, including Mamba2, GPT-Neo, Pythia, Llama-3.2, and Qwen-3.5. Our results demonstrate that QCHFT consistently enhances performance compared to both standard LoRA. On Mamba2-130M, the APC–PQC variant improves accuracy from 67.33% (LoRA) to 72.66%. These findings suggest that the high expressivity of quantum entanglement can be practically harnessed through shallow circuits, providing a superior, parameter-efficient pathway for adapting foundation models to complex, high-feature tasks.
Quantum-centric supercomputing presents a compelling framework for large-scale hybrid quantum-classical tasks. Although quantum machine learning (QML) offers theoretical benefits in various applications, challenges such as large-size data encoding in the input stage and the reliance on quantum resources in the inference stage limit its practicality for tasks like fine-tuning large language models (LLMs). Quantum parameter generation, a novel approach of QML, addresses these limitations by using quantum neural networks (QNNs) to generate classical model weights (parameters) exclusively during training, thereby decoupling inference from quantum hardware. In this work, we introduce Quantum Parameter Adaptation (QPA) in the framework of quantum parameter generation, which integrates QNNs with a classical multi-layer perceptron mapping model to generate parameters for fine-tuning methods. Using Gemma-2 and GPT-2 as case studies, QPA demonstrates significant parameter reduction for parameter-efficient fine-tuning methods, such as Low-Rank Adaptation (LoRA), while maintaining comparable or improved performance in text generation tasks. Specifically, QPA reduces the number of parameters to 52.06% of the original LoRA for GPT-2 with a slight performance gain of 0.75%, and to 16.84% for Gemma-2, with a marginal performance improvement of 0.07%. These results highlight QPA's ability to achieve efficient parameter reduction without sacrificing performance in the quantum parameter generation framework. This work showcases the potential of quantum-enhanced parameter reduction, offering a scalable quantum-classical solution for fine-tuning LLMs while preserving the feasibility of inference on classical hardware.
Large language models (LLMs) can be adapted to new tasks using parameter-efficient fine-tuning (PEFT) methods that modify only a small number of trainable parameters, often through low-rank updates. In this work, we adopt a quantum-information-inspired perspective to understand their effectiveness. From this perspective, low-rank parameterizations naturally correspond to low-dimensional Matrix Product States (MPS) representations, which enable entanglement-based characterizations of parameter structure. Thereby, we term and measure "Artificial Entanglement", defined as the entanglement entropy of the parameters in artificial neural networks (in particular the LLMs). We first study the representative low-rank adaptation (LoRA) PEFT method, alongside full fine-tuning (FFT), using LLaMA models at the 1B and 8B scales trained on the Tulu3 and OpenThoughts3 datasets, and uncover: (i) Internal artificial entanglement in the updates of query and value projection matrices in LoRA follows a volume law with a central suppression (termed as the "Entanglement Valley"), which is sensitive to hyper-parameters and is distinct from that in FFT; (ii) External artificial entanglement in attention matrices, corresponding to token-token correlations in representation space, follows an area law with logarithmic corrections and remains robust to LoRA hyper-parameters and training steps. Drawing a parallel to the No-Hair Theorem in black hole physics, we propose that although LoRA and FFT induce distinct internal entanglement signatures, such differences do not manifest in the attention outputs, suggesting a "no-hair" property that results in the effectiveness of low rank updates. We further provide theoretical support based on random matrix theory, and extend our analysis to an MPS Adaptation PEFT method, which exhibits qualitatively similar behaviors.
Large language models (LLMs) exhibit abilities beyond natural language modelling and text generation. Recent advances in their reasoning capabilities have spurred interest in applying LLMs to complex scientific tasks requiring deep domain expertise and sophisticated reasoning. Quantum computing, as a highly specialised field with significant knowledge barriers and hardware constraints, could greatly benefit from such advancements. However, a key open question that first must be answered is: How can we develop fine-tuning pipelines that instil genuine quantum reasoning in LLMs, rather than task-specific pattern matching? We study this question through quantum circuit simulation as a training objective, where the model must predict the measurement probability distribution resulting from a sequence of quantum gate operations. We propose and compare two fine-tuning pipelines: (1) Supervised Fine-Tuning (SFT) on explicit gate-by-gate state-vector simulation traces, and (2) a two-stage SFT+Group Relative Policy Optimisation (GRPO) approach that sequentially applies SFT followed by GRPO with verifiable rewards. Our findings show that SFT achieves near-perfect in-distribution and gate-count extrapolation accuracy, significantly outperforming both the base model and the GPT-OSS-120B baseline. SFT+GRPO trades some in-distribution precision for better generalisation to larger qubit systems that SFT alone cannot handle. Both pipelines significantly outperform the baselines, demonstrating that targeted fine-tuning on explicit reasoning traces is an effective strategy for advancing quantum reasoning in LLMs.
Inspired by the power of large language models (LLMs), our research adapts them to quantum federated learning (QFL) to boost efficiency and performance. We propose a federated fine-tuning method that distills an LLM within QFL, allowing each client to locally adapt the model to its own data while preserving privacy and reducing unnecessary global updates. The fine-tuned LLM also acts as a reinforcement agent, optimizing QFL by adjusting optimizer steps, cutting down communication rounds, and intelligently selecting clients. Experiments show significant efficiency gains. We pioneer a synergy between LLM and QFL, offering: i) practical efficiency: Reduced communication costs and faster convergence. ii) theoretical rigor: Provable guarantees for adaptive federated optimization. iii) scalability: PEFT methods (LoRA, QLoRA) enable deployment on resource-constrained quantum devices. Code implementation is available here 1.