Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

International Conference on Software, Telecommunications and Computer Networks (SoftCOM)

QuantLLM: A Hybrid Classical-Quantum LLM Transformer with Adaptive Routing Framework for Inference Latency Minimization

Nishat Mahdiya Khan, Pronaya Bhattacharya, Sandip Roy, Sachin S. Shetty

Published Sep 18, 2025• 3 citations
Publisher

Abstract

Token generation in Large Language Models (LLMs) are designed on multi-head attention transformers. They capture long-range sequences inherently, which might span transformer full attention pass, and results in escalation of the model cost quadratically. This cost inflation trigger higher inference latency, limiting its deployment in resource-constrained scenarios. Quantum computing (QC) can address these limitations, owing to its advantages of entanglement in high-dimensional Hilbert spaces, and allow parallel sequence generation by LLMs. However, pure quantum transformers design are challenged by high noise fidelity, coherence, and lack of sufficient qubits for operation. Motivated by this, in this paper, we design a framework, QuantLLM, a hybrid classical-quantum transformer architecture that reduces latency via a token-wise, and semantically-aware adaptive routing mechanism. In the routing design, tokens are assigned to either classical or quantum attention paths computed by a differentiable complexity score, that minimizes the LLM inference latency. Results are evaluated on the benchmark WikiText-2 dataset. The framework reduces inference latency by a maximum of 30.06% compared to quantum-enhanced attention baselines. At a balanced setting, it achieves 84.1% accuracy with 290 ms latency, versus 85.7% with 418 ms against baseline scheme. For long sequences, up to 48.7% of tokens are routed through the quantum path, with token-level fidelity scores (0.872 for complex nouns and 0.122 for punctuation), which asserts optimized semantic-aware context routing. These results indicate the potential viability of the framework design in practical model settings.

DOI
10.23919/softcom66362.2025.11197342

Related papers

Quantum-Enhanced Attention Mechanism in NLP: A Hybrid Classical-Quantum Approach

S. M. Yousuf Iqbal Tomal, Abdullah Al Shafin, Bhattacharjee, Debojit +2 more

Recent advances in quantum computing have opened new pathways for enhancing deep learning architectures, particularly in domains characterized by high-dimensional and context-rich data such as natural language processing (NLP). In this work, we present a hybrid classical-quantum Transformer model that integrates a quantum-enhanced attention mechanism into the standard classical architecture. By embedding token representations into a quantum Hilbert space via parameterized variational circuits and exploiting entanglement-aware kernel similarities, the model captures complex semantic relationships beyond the reach of conventional dot-product attention. We demonstrate the effectiveness of this approach across diverse NLP benchmarks, showing improvements in both efficiency and representational capacity. The results section reveal that the quantum attention layer yields globally coherent attention maps and more separable latent features, while requiring comparatively fewer parameters than classical counterparts. These findings highlight the potential of quantum-classical hybrid models to serve as a powerful and resource-efficient alternative to existing attention mechanisms in NLP.

arXiv:2501.15630•Jan 26, 2025•3 citations

A Hybrid Transformer Architecture with a Quantized Self-Attention Mechanism Applied to Molecular Generation

Anthony M. Smaldone, Yu Shee, Gregory W. Kyro +4 more

The success of the self-attention mechanism in classical machine learning models has inspired the development of quantum analogs aimed at reducing the computational overhead. Self-attention integrates learnable query and key matrices to calculate attention scores between all pairs of tokens in a sequence. These scores are then multiplied by a learnable value matrix to obtain the output self-attention matrix, enabling the model to effectively capture long-range dependencies within the input sequence. Here, we propose a hybrid quantum-classical self-attention mechanism as part of a transformer decoder, the architecture underlying large language models (LLMs). To demonstrate its utility in chemistry, we train this model on the QM9 dataset for conditional generation, using SMILES strings as input, each labeled with a set of physicochemical properties that serve as conditions during inference. Our theoretical analysis shows that the time complexity of the query-key dot product is reduced from O ( n 2 d ) in a classical model to O ( n 2 log ⁡ d ) in our quantum model, where n and d represent the sequence length and the embedding dimension, respectively. We perform simulations using NVIDIA’s CUDA-Q platform, which is designed for efficient GPU scalability. This work provides a promising avenue for quantum-enhanced natural language processing (NLP).

arXiv:2502.19214•Journal of Chemical Theory and Computation•May 7, 2025•16 citations

GroverAttention: Quantum-Enhanced Attention Mechanism for Effcient Transformers

Belkacem Chikhaoui

Abstract Transformer-based models have revolutionized natural language processing (NLP), but their computational complexity remains a bottleneck. This paper proposes a novel approach to integrating Quantum Attention into Transformers, leveraging Grover’s algorithm to optimize query-key matching. By replacing classical attention mechanisms with quantum-enhanced token selection, we achieve reduced computational cost and improved inference speed. Experimental results on real-world datasets demonstrate the effectiveness of our hybrid Quantum-Classical Transformer, showing an improvement in accuracy with reduced training overhead. Our findings suggest that quantum-assisted attention has the potential to significantly optimize deep learning architectures for NLP applications.

•Oct 31, 2025•0 citations

The Quantum Sieve Tracer: A Hybrid Framework for Layer-Wise Activation Tracing in Large Language Models

Jonathan Pan

Mechanistic interpretability aims to reverse-engineer the internal computations of Large Language Models (LLMs), yet separating sparse semantic signals from high-dimensional polysemantic noise remains a significant challenge. This paper introduces the Quantum Sieve Tracer, a hybrid quantum-classical framework designed to characterize factual recall circuits. We implement a modular pipeline that first localizes critical layers using classical causal tracing, then maps specific attention head activations into an exponentially large quantum Hilbert space. Using open-weight models (Meta Llama-3.2-1B and Alibaba Qwen2.5-1.5B-Instruct), we perform a two-stage analysis that reveals a fundamental architectural divergence. While Qwen's layer 7 circuit functions as a classic Recall Hub, we discover that Llama's layer 9 acts as an Interference Suppression circuit, where ablating the identified heads paradoxically improves factual recall. Our results demonstrate that quantum kernels can distinguish between these constructive (recall) and reductive (suppression) mechanisms, offering a high-resolution tool for analyzing the fine-grained topology of attention.

quant-pharXiv:2602.06852•Feb 9, 2026•0 citations

Quantum-Inspired Transformers for Scalable Natural Language Processing Applications in Future AI

Priya Vij, Parmanand Yadav

Quantum-Inspired Transformers are a new class of approaches for natural language processing (NLP) that use principles inspired by physical quantum mechanics to encode and process linguistic information. Paper aims to improve both scalability and efficiency for large-scale NLP tasks in the next generation of artificial intelligence (AI). However, traditional transformers can struggle to encode complex semantic relationships across long sequences and can require high levels of compute resources, hence limiting scalability for large datasets and multilingual tasks. In addition, traditional embeddings do not represent multiple contextual meanings simultaneously, which decreases model expressiveness. To address these challenges and maximize the exploration of potential quantum effects in AI, this paper presents a Quantum State Embedding Attention (QSEA) framework that encodes tokens as quantum inspired states with complex-valued amplitudes. Attention, which is based on entangled states, is meant to efficiently capture long-range dependencies while reducing compute and increasing scalability, all without losing a rich understanding of context. Whereas multilingual translation needs to encode both the source similarity and the target language within a smaller semantic representation space. Our experimentation shows that QSEA consistently provides increased accuracy in decoding, measures attention across long sequences, and yields enduring scalability over large multilingual datasets. The evidence demonstrates the potential for QSEA to be the new layer in the future of AI systems, a layer that is scalable and a highperforming NLP application.

Procedia Computer Science•Jan 1, 2026•1 citation

QLAM: A Quantum Long-Attention Memory Approach to Long-Sequence Token Modeling

Hoang-Quan Nguyen, S. D. Pandey, Ky Luu

Modeling long-range dependencies in sequential data remains a central challenge in machine learning. Transformers address this challenge through attention mechanisms, but their quadratic complexity with respect to sequence length limits scalability to long contexts. State-space models (SSMs) provide an efficient alternative with linear-time computation by evolving a latent state through recurrent updates, but their memory is typically formed via additive or linear transitions, which can limit their ability to capture complex global interactions across tokens. In this work, we introduce one of the first studies to leverage the superposition property of quantum systems to enhance state-based sequence modeling. In particular, we propose Quantum Long-Attention Memory (QLAM), a hybrid quantum-classical memory mechanism that can be viewed as a quantum extension of state-space models. Instead of maintaining a classical latent state updated through additive dynamics, QLAM represents the hidden state as a quantum state whose amplitudes encode a superposition of historical information. The state evolves through parameterized quantum circuits conditioned on the input, enabling a non-classical, globally update mechanism. In this way, QLAM preserves the recurrent and linear-time structure of SSMs while fundamentally enriching the memory representation through quantum superposition. Unlike attention mechanisms that explicitly compute pairwise interactions, QLAM implicitly captures global dependencies through the evolution of the quantum state, and retrieves task-relevant information via query-dependent measurements. We evaluate QLAM on sequential variants of standard image classification benchmarks, including sMNIST, sFashion-MNIST, and sCIFAR-10, where images are flattened into token sequences. Across all tasks, QLAM consistently improves over recurrent baselines and transformer-based models.

arXiv:2605.13833•May 13, 2026•0 citations
Token generation in Large Language Models (LLMs) are designed on multi-head attention transformers. They capture long-range sequences inherently, which might span transformer full attention pass, and results in escalation of the model cost quadratically. This cost inflation trigger higher inference latency, limiting its deployment in resource-constrained scenarios. Quantum computing (QC) can address these limitations, owing to its advantages of entanglement in high-dimensional Hilbert spaces, and allow parallel sequence generation by LLMs. However, pure quantum transformers design are challenged by high noise fidelity, coherence, and lack of sufficient qubits for operation. Motivated by this, in this paper, we design a framework, QuantLLM, a hybrid classical-quantum transformer architecture that reduces latency via a token-wise, and semantically-aware adaptive routing mechanism. In the routing design, tokens are assigned to either classical or quantum attention paths computed by a differentiable complexity score, that minimizes the LLM inference latency. Results are evaluated on the benchmark WikiText-2 dataset. The framework reduces inference latency by a maximum of 30.06% compared to quantum-enhanced attention baselines. At a balanced setting, it achieves 84.1% accuracy with 290 ms latency, versus 85.7% with 418 ms against baseline scheme. For long sequences, up to 48.7% of tokens are routed through the quantum path, with token-level fidelity scores (0.872 for complex nouns and 0.122 for punctuation), which asserts optimized semantic-aware context routing. These results indicate the potential viability of the framework design in practical model settings.
Recent advances in quantum computing have opened new pathways for enhancing deep learning architectures, particularly in domains characterized by high-dimensional and context-rich data such as natural language processing (NLP). In this work, we present a hybrid classical-quantum Transformer model that integrates a quantum-enhanced attention mechanism into the standard classical architecture. By embedding token representations into a quantum Hilbert space via parameterized variational circuits and exploiting entanglement-aware kernel similarities, the model captures complex semantic relationships beyond the reach of conventional dot-product attention. We demonstrate the effectiveness of this approach across diverse NLP benchmarks, showing improvements in both efficiency and representational capacity. The results section reveal that the quantum attention layer yields globally coherent attention maps and more separable latent features, while requiring comparatively fewer parameters than classical counterparts. These findings highlight the potential of quantum-classical hybrid models to serve as a powerful and resource-efficient alternative to existing attention mechanisms in NLP.
The success of the self-attention mechanism in classical machine learning models has inspired the development of quantum analogs aimed at reducing the computational overhead. Self-attention integrates learnable query and key matrices to calculate attention scores between all pairs of tokens in a sequence. These scores are then multiplied by a learnable value matrix to obtain the output self-attention matrix, enabling the model to effectively capture long-range dependencies within the input sequence. Here, we propose a hybrid quantum-classical self-attention mechanism as part of a transformer decoder, the architecture underlying large language models (LLMs). To demonstrate its utility in chemistry, we train this model on the QM9 dataset for conditional generation, using SMILES strings as input, each labeled with a set of physicochemical properties that serve as conditions during inference. Our theoretical analysis shows that the time complexity of the query-key dot product is reduced from O ( n 2 d ) in a classical model to O ( n 2 log ⁡ d ) in our quantum model, where n and d represent the sequence length and the embedding dimension, respectively. We perform simulations using NVIDIA’s CUDA-Q platform, which is designed for efficient GPU scalability. This work provides a promising avenue for quantum-enhanced natural language processing (NLP).
Abstract Transformer-based models have revolutionized natural language processing (NLP), but their computational complexity remains a bottleneck. This paper proposes a novel approach to integrating Quantum Attention into Transformers, leveraging Grover’s algorithm to optimize query-key matching. By replacing classical attention mechanisms with quantum-enhanced token selection, we achieve reduced computational cost and improved inference speed. Experimental results on real-world datasets demonstrate the effectiveness of our hybrid Quantum-Classical Transformer, showing an improvement in accuracy with reduced training overhead. Our findings suggest that quantum-assisted attention has the potential to significantly optimize deep learning architectures for NLP applications.
Mechanistic interpretability aims to reverse-engineer the internal computations of Large Language Models (LLMs), yet separating sparse semantic signals from high-dimensional polysemantic noise remains a significant challenge. This paper introduces the Quantum Sieve Tracer, a hybrid quantum-classical framework designed to characterize factual recall circuits. We implement a modular pipeline that first localizes critical layers using classical causal tracing, then maps specific attention head activations into an exponentially large quantum Hilbert space. Using open-weight models (Meta Llama-3.2-1B and Alibaba Qwen2.5-1.5B-Instruct), we perform a two-stage analysis that reveals a fundamental architectural divergence. While Qwen's layer 7 circuit functions as a classic Recall Hub, we discover that Llama's layer 9 acts as an Interference Suppression circuit, where ablating the identified heads paradoxically improves factual recall. Our results demonstrate that quantum kernels can distinguish between these constructive (recall) and reductive (suppression) mechanisms, offering a high-resolution tool for analyzing the fine-grained topology of attention.
Quantum-Inspired Transformers are a new class of approaches for natural language processing (NLP) that use principles inspired by physical quantum mechanics to encode and process linguistic information. Paper aims to improve both scalability and efficiency for large-scale NLP tasks in the next generation of artificial intelligence (AI). However, traditional transformers can struggle to encode complex semantic relationships across long sequences and can require high levels of compute resources, hence limiting scalability for large datasets and multilingual tasks. In addition, traditional embeddings do not represent multiple contextual meanings simultaneously, which decreases model expressiveness. To address these challenges and maximize the exploration of potential quantum effects in AI, this paper presents a Quantum State Embedding Attention (QSEA) framework that encodes tokens as quantum inspired states with complex-valued amplitudes. Attention, which is based on entangled states, is meant to efficiently capture long-range dependencies while reducing compute and increasing scalability, all without losing a rich understanding of context. Whereas multilingual translation needs to encode both the source similarity and the target language within a smaller semantic representation space. Our experimentation shows that QSEA consistently provides increased accuracy in decoding, measures attention across long sequences, and yields enduring scalability over large multilingual datasets. The evidence demonstrates the potential for QSEA to be the new layer in the future of AI systems, a layer that is scalable and a highperforming NLP application.
Modeling long-range dependencies in sequential data remains a central challenge in machine learning. Transformers address this challenge through attention mechanisms, but their quadratic complexity with respect to sequence length limits scalability to long contexts. State-space models (SSMs) provide an efficient alternative with linear-time computation by evolving a latent state through recurrent updates, but their memory is typically formed via additive or linear transitions, which can limit their ability to capture complex global interactions across tokens. In this work, we introduce one of the first studies to leverage the superposition property of quantum systems to enhance state-based sequence modeling. In particular, we propose Quantum Long-Attention Memory (QLAM), a hybrid quantum-classical memory mechanism that can be viewed as a quantum extension of state-space models. Instead of maintaining a classical latent state updated through additive dynamics, QLAM represents the hidden state as a quantum state whose amplitudes encode a superposition of historical information. The state evolves through parameterized quantum circuits conditioned on the input, enabling a non-classical, globally update mechanism. In this way, QLAM preserves the recurrent and linear-time structure of SSMs while fundamentally enriching the memory representation through quantum superposition. Unlike attention mechanisms that explicitly compute pairwise interactions, QLAM implicitly captures global dependencies through the evolution of the quantum state, and retrieves task-relevant information via query-dependent measurements. We evaluate QLAM on sequential variants of standard image classification benchmarks, including sMNIST, sFashion-MNIST, and sCIFAR-10, where images are flattened into token sequences. Across all tasks, QLAM consistently improves over recurrent baselines and transformer-based models.