Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2410.17397

Quantum Large Language Models via Tensor Network Disentanglers

Borja Aizpurua, Loren, Fernando, Saeed S. Jahromi, Sukhbinder Singh, Román Orús

Published Oct 22, 2024• 4 citations
arXivHTMLPDF

Abstract

We introduce a framework for seamlessly integrating quantum computing into pretrained large language models (LLMs). The key idea is to construct a hybrid quantum-classical representation that exactly reproduces the original model, providing a principled starting point from which quantum resources can only improve performance. Our approach replaces the weight matrices in self-attention and multilayer perceptron layers with two variational quantum circuits coupled to a matrix product operator (MPO). Tensor network disentanglers transfer much of each layer's information into the quantum circuits, enabling the remaining tensor network to be compressed to a bond-dimension-one MPO with over three orders of magnitude fewer classical parameters (in our experiments, from 110,592 to approximately 36 for the replaced layer) and less than a 0.3\% increase in perplexity. Training an added unitary adapter on top of this representation then surpasses the original model, reducing perplexity by up to 1.6\%. Finally, we validate the hybrid architecture on a real quantum processor, demonstrating a practical route towards quantum-enhanced language models.

DOI
10.48550/arxiv.2410.17397

Related papers

Classical neural networks on quantum devices via tensor network disentanglers: A case study in image classification

Borja Aizpurua, Sukhbinder Singh, Román Orús

We address the problem of implementing bottleneck layers from classical pretrained neural networks on a quantum computer, with the goal of exploring intrinsically quantum ansatz for representing large linear layers within hybrid classical-quantum models. Our approach begins with a compression step in which the target linear layer is represented as an effective matrix product operator (MPO) without degrading model performance. The MPO is then further disentangled into a more compact form. This enables a hybrid classical-quantum execution scheme, where the disentangling circuits are deployed on a quantum computer while the remainder of the network—including the disentangled MPO—runs on classical hardware. We introduce two complementary algorithms for MPO disentangling: (1) an disentangling variational method leveraging standard tensor network optimization techniques and (2) an disentangling gradient-descent-based approach. We validate these methods through a proof-of-concept translation of simple classical neural networks for MNIST and CIFAR-10 image classification into a hybrid classical-quantum form.

arXiv:2509.06653•Physical Review Research•Mar 12, 2026•0 citations

Hybrid Quantum-Classical Transfer Learning for Parameter-Efficient Code Generation

Su-Chang Lim, Jong-Chan Kim

The remarkable success of large language models (LLMs) in code generation has transformed software development, with systems like Codex, CodeGen, and StarCoder demonstrating strong capabilities in synthesizing functional code from natural language descriptions. However, this progress comes at a steep computational cost. Modern code generation models require hundreds of millions to billions of parameters—GPT-3 contains 175 billion parameters, CodeGen-16B uses 16 billion, and the trend continues toward ever-larger models. This scale creates fundamental challenges in training cost, inference latency, memory consumption, and environmental impact. We present a quantum transfer learning framework as an alternative approach to classical model compression for code generation. By replacing selected MLP layers with variational quantum circuits (VQCs), we exploit the exponential representational capacity of quantum systems to achieve $\Theta (D \log D)$ parameter scaling instead of classical $\Theta (D^{2})$ . Our hybrid classical-quantum architecture balances three competing objectives: leveraging existing pre-trained knowledge, achieving parameter compression through quantum representation, and maintaining compatibility with near-term quantum hardware constraints. We demonstrate our approach on CodeGen-350M, a production-scale code generation model, adopting a bottleneck architecture that substantially reduces parameters compared to the original MLP layer. Through a fair comparison against a classical bottleneck baseline with near-identical parameter count, we show that the VQC achieves superior representational quality—lower perplexity and higher code generation accuracy—validating the quantum layer’s expressivity advantage in the compressed latent space. Our design employs data re-uploading, where classical input features are re-encoded at each variational layer via trainable RY gates, enhancing the circuit’s expressive power on near-term devices.

IEEE Access•Jan 1, 2026•0 citations

Hybrid Quantum Transformer for Language Generation

Kong, Desheng, Xiaohai Cui, Jin, Jiaying +2 more

Although quantum computing has been increasingly applied to replace classical computation, most existing quantum or hybrid models remain confined to simple tasks, with no successful application to large-scale natural language generation to date. In this work, we present the first hybrid quantum-classical large language model (LLM) for natural language generation, HyQuT, capable of performing coherent and context-aware dialogue. The proposed architecture integrates variational quantum circuits (VQCs) into the Transformer framework at both 8M and 150M parameter scales. Experimental results show that a minimal number of qubits (10 qubits with 80 quantum gates) can replace about 10% of the classical parameters in the 150M-parameter model, while achieving comparable convergence stability and generation quality. This study provides an early demonstration of the feasibility of integrating quantum computing to large-scale generative language models.

arXiv:2511.10653•Nov 2, 2025•0 citations

Elevating LLMs Via Synergy of Quantum Computing and Classical Transformer Models

Shashanka Shekhar Sharma, Ritisha Katiyar, Hrishikesh Sarma +1 more

Quantum computation offers new directions in computer science that go beyond classical systems' state-of-the-art abilities. The combination of quantum technologies with Large Language Models (LLMs) provides promising avenues for the creation of more efficient and powerful NLP architectures. In this research, we examine quantum-enhanced transformers and further develop the Self-Attention Sequential Quantum Transformer Channel (SASQuatch) framework to enhance inference time and perplexity. The models proposed are tested on sentence completion tasks and compared against a baseline transformer, showing significant gains in efficiency and predictive power. We further compare the method with other optimization methods, such as the Quantum Approximate Optimization Algorithm (QAOA), thus highlighting the necessity of further investigation of hybrid quantum architectures in improving the performance of large-scale language models.

International Conference on AI-ML Systems (AIMLSystems)•Oct 8, 2025•1 citation

HVIA: A Hybrid Variational Ising Architecture for Quantum Language Modeling

Chengang Mao, Haisheng Li

Abstract Language modeling in natural language processing has predominantly relied on large-scale classical neural architectures, yet constructing expressive and resource-efficient models compatible with Noisy Intermediate-Scale Quantum (NISQ) devices remains a fundamental open challenge. In this work, two Ising-inspired variational quantum-classical architectures are proposed to enhance representational capacity by leveraging spin-coupling mechanisms and Ising-type Hamiltonian evolution within hybrid classical-quantum layers. Specifically, a quantum Ising Ansatz circuit parameterized by an Ising Hamiltonian is constructed, and a quantum approximate optimization algorithm (QAOA) is incorporated to reduce the number of trainable circuit parameters while enabling learnable quantum evolution of hidden states. To achieve stable and efficient hybrid training under NISQ constraints, a joint optimization strategy is introduced that combines the Adam optimizer with quantum simultaneous perturbation stochastic approximation (SPSA), yielding convergence acceleration of up to $64\%$ relative to parameter-shift rule and Adam-only baselines. Empirical evaluations on the WikiText-2 benchmark demonstrate that the proposed architectures effectively capture long-range semantic dependencies, achieving a competitive perplexity of $\text{PPL} = 39.8$ (lower values indicate stronger language modeling performance). This result surpasses existing quantum language models, including Quixer, Quantum LSTM, and Quantum GRU, and approaches the performance of the classical NanoGPT baseline ($\text{PPL} = 40.5$). Collectively, these findings establish the viability of Ising-based hybrid quantum-classical networks as a promising direction for practical quantum natural language processing.

•Jun 12, 2026•0 citations

Quantum-enhanced Large Language Models on Quantum Hardware via Cayley Unitary Adapters

Borja Aizpurua, Sukhbinder Singh, Augustine Kshetrimayum +2 more

Large language models (LLMs) have transformed artificial intelligence, yet classical architectures impose a fundamental constraint: every trainable parameter demands classical memory that scales unfavourably with model size. Quantum computing offers a qualitatively different pathway, but practical demonstrations on real hardware have remained elusive for models of practical relevance. Here we show that Cayley-parameterised unitary adapters -- quantum circuit blocks inserted into the frozen projection layers of pre-trained LLMs and executed on a 156-qubit IBM Quantum System Two superconducting processor -- improve the perplexity of Llama 3.1 8B, an 8-billion-parameter model in widespread use, by 1.4% with only 6,000 additional parameters and end-to-end inference validated on real Quantum Processing Unit (QPU). A systematic study on SmolLM2 (135M parameters), chosen for its tractability, reveals monotonically improving perplexity with unitary block dimension, 83% recovery of compression-induced degradation, and correct answers to questions that both classical baselines fail -- with a sharp noise-expressivity phase transition identifying the concrete path to quantum utility at larger qubit scales.

quant-pharXiv:2605.05914•May 8, 2026•0 citations
We introduce a framework for seamlessly integrating quantum computing into pretrained large language models (LLMs). The key idea is to construct a hybrid quantum-classical representation that exactly reproduces the original model, providing a principled starting point from which quantum resources can only improve performance. Our approach replaces the weight matrices in self-attention and multilayer perceptron layers with two variational quantum circuits coupled to a matrix product operator (MPO). Tensor network disentanglers transfer much of each layer's information into the quantum circuits, enabling the remaining tensor network to be compressed to a bond-dimension-one MPO with over three orders of magnitude fewer classical parameters (in our experiments, from 110,592 to approximately 36 for the replaced layer) and less than a 0.3% increase in perplexity. Training an added unitary adapter on top of this representation then surpasses the original model, reducing perplexity by up to 1.6%. Finally, we validate the hybrid architecture on a real quantum processor, demonstrating a practical route towards quantum-enhanced language models.
We address the problem of implementing bottleneck layers from classical pretrained neural networks on a quantum computer, with the goal of exploring intrinsically quantum ansatz for representing large linear layers within hybrid classical-quantum models. Our approach begins with a compression step in which the target linear layer is represented as an effective matrix product operator (MPO) without degrading model performance. The MPO is then further disentangled into a more compact form. This enables a hybrid classical-quantum execution scheme, where the disentangling circuits are deployed on a quantum computer while the remainder of the network—including the disentangled MPO—runs on classical hardware. We introduce two complementary algorithms for MPO disentangling: (1) an disentangling variational method leveraging standard tensor network optimization techniques and (2) an disentangling gradient-descent-based approach. We validate these methods through a proof-of-concept translation of simple classical neural networks for MNIST and CIFAR-10 image classification into a hybrid classical-quantum form.
The remarkable success of large language models (LLMs) in code generation has transformed software development, with systems like Codex, CodeGen, and StarCoder demonstrating strong capabilities in synthesizing functional code from natural language descriptions. However, this progress comes at a steep computational cost. Modern code generation models require hundreds of millions to billions of parameters—GPT-3 contains 175 billion parameters, CodeGen-16B uses 16 billion, and the trend continues toward ever-larger models. This scale creates fundamental challenges in training cost, inference latency, memory consumption, and environmental impact. We present a quantum transfer learning framework as an alternative approach to classical model compression for code generation. By replacing selected MLP layers with variational quantum circuits (VQCs), we exploit the exponential representational capacity of quantum systems to achieve Θ(Dlog⁡D)\Theta (D \log D)Θ(DlogD) parameter scaling instead of classical Θ(D2)\Theta (D^{2})Θ(D2) . Our hybrid classical-quantum architecture balances three competing objectives: leveraging existing pre-trained knowledge, achieving parameter compression through quantum representation, and maintaining compatibility with near-term quantum hardware constraints. We demonstrate our approach on CodeGen-350M, a production-scale code generation model, adopting a bottleneck architecture that substantially reduces parameters compared to the original MLP layer. Through a fair comparison against a classical bottleneck baseline with near-identical parameter count, we show that the VQC achieves superior representational quality—lower perplexity and higher code generation accuracy—validating the quantum layer’s expressivity advantage in the compressed latent space. Our design employs data re-uploading, where classical input features are re-encoded at each variational layer via trainable RY gates, enhancing the circuit’s expressive power on near-term devices.
Although quantum computing has been increasingly applied to replace classical computation, most existing quantum or hybrid models remain confined to simple tasks, with no successful application to large-scale natural language generation to date. In this work, we present the first hybrid quantum-classical large language model (LLM) for natural language generation, HyQuT, capable of performing coherent and context-aware dialogue. The proposed architecture integrates variational quantum circuits (VQCs) into the Transformer framework at both 8M and 150M parameter scales. Experimental results show that a minimal number of qubits (10 qubits with 80 quantum gates) can replace about 10% of the classical parameters in the 150M-parameter model, while achieving comparable convergence stability and generation quality. This study provides an early demonstration of the feasibility of integrating quantum computing to large-scale generative language models.
Quantum computation offers new directions in computer science that go beyond classical systems' state-of-the-art abilities. The combination of quantum technologies with Large Language Models (LLMs) provides promising avenues for the creation of more efficient and powerful NLP architectures. In this research, we examine quantum-enhanced transformers and further develop the Self-Attention Sequential Quantum Transformer Channel (SASQuatch) framework to enhance inference time and perplexity. The models proposed are tested on sentence completion tasks and compared against a baseline transformer, showing significant gains in efficiency and predictive power. We further compare the method with other optimization methods, such as the Quantum Approximate Optimization Algorithm (QAOA), thus highlighting the necessity of further investigation of hybrid quantum architectures in improving the performance of large-scale language models.
Abstract Language modeling in natural language processing has predominantly relied on large-scale classical neural architectures, yet constructing expressive and resource-efficient models compatible with Noisy Intermediate-Scale Quantum (NISQ) devices remains a fundamental open challenge. In this work, two Ising-inspired variational quantum-classical architectures are proposed to enhance representational capacity by leveraging spin-coupling mechanisms and Ising-type Hamiltonian evolution within hybrid classical-quantum layers. Specifically, a quantum Ising Ansatz circuit parameterized by an Ising Hamiltonian is constructed, and a quantum approximate optimization algorithm (QAOA) is incorporated to reduce the number of trainable circuit parameters while enabling learnable quantum evolution of hidden states. To achieve stable and efficient hybrid training under NISQ constraints, a joint optimization strategy is introduced that combines the Adam optimizer with quantum simultaneous perturbation stochastic approximation (SPSA), yielding convergence acceleration of up to 64%64\%64% relative to parameter-shift rule and Adam-only baselines. Empirical evaluations on the WikiText-2 benchmark demonstrate that the proposed architectures effectively capture long-range semantic dependencies, achieving a competitive perplexity of PPL=39.8\text{PPL} = 39.8PPL=39.8 (lower values indicate stronger language modeling performance). This result surpasses existing quantum language models, including Quixer, Quantum LSTM, and Quantum GRU, and approaches the performance of the classical NanoGPT baseline (PPL=40.5\text{PPL} = 40.5PPL=40.5). Collectively, these findings establish the viability of Ising-based hybrid quantum-classical networks as a promising direction for practical quantum natural language processing.
Large language models (LLMs) have transformed artificial intelligence, yet classical architectures impose a fundamental constraint: every trainable parameter demands classical memory that scales unfavourably with model size. Quantum computing offers a qualitatively different pathway, but practical demonstrations on real hardware have remained elusive for models of practical relevance. Here we show that Cayley-parameterised unitary adapters -- quantum circuit blocks inserted into the frozen projection layers of pre-trained LLMs and executed on a 156-qubit IBM Quantum System Two superconducting processor -- improve the perplexity of Llama 3.1 8B, an 8-billion-parameter model in widespread use, by 1.4% with only 6,000 additional parameters and end-to-end inference validated on real Quantum Processing Unit (QPU). A systematic study on SmolLM2 (135M parameters), chosen for its tractability, reveals monotonically improving perplexity with unitary block dimension, 83% recovery of compression-induced degradation, and correct answers to questions that both classical baselines fail -- with a sharp noise-expressivity phase transition identifying the concrete path to quantum utility at larger qubit scales.