Quantum neural architectures are rapidly emerging as a new paradigm for efficient representation learning. Their intrinsic structure, built upon superposition and entanglement, enables expressive transformations with remarkably few parameters. Meanwhile, parameter-efficient fine-tuning (PEFT) has become a key approach for adapting large language models (LLMs), typically by inserting lightweight adapters or low-rank modules such as LoRA. However, strict low-rank constraints may limit representational capacity under complex domain shifts. This raises a natural question: what if quantum parameterization is introduced into the adapter framework? To address this, we propose a Quantum-Enhanced Adapter (QEA), which adds a parameterized quantum circuit branch in parallel to each Transformer feed-forward network while keeping all pretrained weights frozen. QEA encodes token features as rotation angles, reads out Pauli-Z expectations, and fuses quantum and classical paths via a token-wise gate fusion with a soft-to-hard schedule for stable training. On LLaMA-7B and LLaMA-3.2-1B across WikiText-2 and WikiText-103, QEA achieves comparable or lower perplexity than strong PEFT baselines while using only about 0.066-0.068% of the model parameters. These results highlight QEA’s ability to deliver substantial parameter savings without sacrificing performance. Our work demonstrates the promise of quantum-enhanced, parameter-efficient fine-tuning for LLMs, providing a scalable quantum-classical solution.
Quantum neural architectures are rapidly emerging as a new paradigm for efficient representation learning. Their intrinsic structure, built upon superposition and entanglement, enables expressive transformations with remarkably few parameters. Meanwhile, parameter-efficient fine-tuning (PEFT) has become a key approach for adapting large language models (LLMs), typically by inserting lightweight adapters or low-rank modules such as LoRA. However, strict low-rank constraints may limit representational capacity under complex domain shifts. This raises a natural question: what if quantum parameterization is introduced into the adapter framework? To address this, we propose a Quantum-Enhanced Adapter (QEA), which adds a parameterized quantum circuit branch in parallel to each Transformer feed-forward network while keeping all pretrained weights frozen. QEA encodes token features as rotation angles, reads out Pauli-Z expectations, and fuses quantum and classical paths via a token-wise gate fusion with a soft-to-hard schedule for stable training. On LLaMA-7B and LLaMA-3.2-1B across WikiText-2 and WikiText-103, QEA achieves comparable or lower perplexity than strong PEFT baselines while using only about 0.066-0.068% of the model parameters. These results highlight QEA’s ability to deliver substantial parameter savings without sacrificing performance. Our work demonstrates the promise of quantum-enhanced, parameter-efficient fine-tuning for LLMs, providing a scalable quantum-classical solution.
The rapid progress of large language models (LLMs) has transformed natural language processing, yet the challenge of efficient adaptation remains unresolved. Full fine-tuning achieves strong performance but imposes prohibitive computational and memory costs. Parameter-efficient fine-tuning (PEFT) strategies, such as low-rank adaptation (LoRA), Prefix tuning, and sparse low-rank adaptation (SoRA), address this issue by reducing trainable parameters while maintaining competitive accuracy. However, these methods often encounter limitations in scalability, stability, and generalization across diverse tasks. Recent advances in quantum deep learning introduce novel opportunities through quantum-inspired encoding and parameterized quantum circuits (PQCs). In particular, the quantum-amplitude embedded adaptation (QAA) framework demonstrates expressive model updates with minimal overhead. This paper presents a systematic survey and comparative analysis of conventional PEFT methods and QAA. The analysis demonstrates trade-offs in convergence, efficiency, and representational capacity, while providing insight into the potential of quantum approaches for future LLM adaptation.
When full-parameter updates are impractical for large language models (LLMs), parameter-efficient fine-tuning (PEFT) is commonly employed to reduce the number of trainable parameters. Classical PEFT methods, such as low-rank adaptation (LoRA), are limited to linear transformations and may not capture complex, high-order feature interactions under stringent parameter constraints. In contrast, parameterized quantum circuits (PQCs) provide high-order expressivity per qubit rotation degree of freedom. Inspired by PQCs, in this paper, we propose quantum cross-hybrid fine-tuning (QCHFT), a suite of four PQC-augmented adapters: Matrix Product State PQC, Alternating and Hierarchical Pairwise Contraction PQC, and LinearA–Quantum–LinearB PQC. To evaluate our QCHFT adapters, we conduct experiments on a public multi-tier IoT resource-allocation task that serves as a challenging benchmark with intricate cross-tier interactions data and limited parameter budgets. We conduct experiments on a multi-tier IoT resource-allocation task across five causal LLM backbones scaling from 125M to 2B parameters, including Mamba2, GPT-Neo, Pythia, Llama-3.2, and Qwen-3.5. Our results demonstrate that QCHFT consistently enhances performance compared to both standard LoRA. On Mamba2-130M, the APC–PQC variant improves accuracy from 67.33% (LoRA) to 72.66%. These findings suggest that the high expressivity of quantum entanglement can be practically harnessed through shallow circuits, providing a superior, parameter-efficient pathway for adapting foundation models to complex, high-feature tasks.
Quantum-centric supercomputing presents a compelling framework for large-scale hybrid quantum-classical tasks. Although quantum machine learning (QML) offers theoretical benefits in various applications, challenges such as large-size data encoding in the input stage and the reliance on quantum resources in the inference stage limit its practicality for tasks like fine-tuning large language models (LLMs). Quantum parameter generation, a novel approach of QML, addresses these limitations by using quantum neural networks (QNNs) to generate classical model weights (parameters) exclusively during training, thereby decoupling inference from quantum hardware. In this work, we introduce Quantum Parameter Adaptation (QPA) in the framework of quantum parameter generation, which integrates QNNs with a classical multi-layer perceptron mapping model to generate parameters for fine-tuning methods. Using Gemma-2 and GPT-2 as case studies, QPA demonstrates significant parameter reduction for parameter-efficient fine-tuning methods, such as Low-Rank Adaptation (LoRA), while maintaining comparable or improved performance in text generation tasks. Specifically, QPA reduces the number of parameters to 52.06% of the original LoRA for GPT-2 with a slight performance gain of 0.75%, and to 16.84% for Gemma-2, with a marginal performance improvement of 0.07%. These results highlight QPA's ability to achieve efficient parameter reduction without sacrificing performance in the quantum parameter generation framework. This work showcases the potential of quantum-enhanced parameter reduction, offering a scalable quantum-classical solution for fine-tuning LLMs while preserving the feasibility of inference on classical hardware.
Large language models (LLMs) have transformed artificial intelligence, yet classical architectures impose a fundamental constraint: every trainable parameter demands classical memory that scales unfavourably with model size. Quantum computing offers a qualitatively different pathway, but practical demonstrations on real hardware have remained elusive for models of practical relevance. Here we show that Cayley-parameterised unitary adapters -- quantum circuit blocks inserted into the frozen projection layers of pre-trained LLMs and executed on a 156-qubit IBM Quantum System Two superconducting processor -- improve the perplexity of Llama 3.1 8B, an 8-billion-parameter model in widespread use, by 1.4% with only 6,000 additional parameters and end-to-end inference validated on real Quantum Processing Unit (QPU). A systematic study on SmolLM2 (135M parameters), chosen for its tractability, reveals monotonically improving perplexity with unitary block dimension, 83% recovery of compression-induced degradation, and correct answers to questions that both classical baselines fail -- with a sharp noise-expressivity phase transition identifying the concrete path to quantum utility at larger qubit scales.
Large language models (LLMs) require substantial resources for task-specific adaptation, that motivates the development of parameter-efficient fine-tuning (PEFT) methods. This paper presents quantum-amplitude embedded adaptation (QAA), a novel PEFT framework that logarithmically compresses activation vectors using quantum-amplitude embedding and applies expressive non-linear transformations via parameterized quantum circuits (PQCs). By replacing linear adapters in attention modules with compact quantum modules, QAA achieves high expressivity while drastically reducing the number of trainable parameters. Empirical results demonstrate that QAA performs on par with or better than existing PEFT under constrained memory and compute budgets, highlighting its potential for efficient LLM fine-tuning.
Quantum computing represents one of the most technically dense frontiers in modern science, with specialized vocabulary spanning quantum gates, superposition, entanglement, variational quantum algorithms, and error-correction codes. Providing accurate, real-time research-grade assistance in this domain requires an LLM that is both highly capable and deployable on resource-constrained hardware. This paper presents the design and deployment of a Quantum Computing Research Agent Chatbot powered by PEFT fine-tuning of a 1.58-bit quantized LLM (BitNet b1.58). We extend the mathematical frameworks of Low-Rank Adaptation (LoRA), Adapter Layers, and Prefix Tuning to the quantum-domain fine-tuning context, incorporating a bespoke quantum-terminology corpus and a Retrieval-Augmented Generation (RAG) layer backed by a curated quantum literature index. We derive the rank–accuracy trade-off in the context of quantum NLP tasks, show that LoRA at rank r = 16 achieves an F1 score of 88.7 on quantum question-answering benchmarks while using only 0.13% of full model parameters, and demonstrate end-to-end inference at 4.2 tokens/second on a single NVIDIA T4 GPU. The proposed agent pipeline integrates with academic literature APIs (arXiv, Semantic Scholar) and supports multi-turn research dialogues, citation generation, and mathematical expression rendering. Keywords— Quantum Computing Chatbot, PEFT, BitNet 1.58-bit Quantization, Low-Rank Adaptation, Research Agent, Retrieval-Augmented Generation, Ternary Weight Matrices, Quantum NLP