Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

QuantumGPTMini: A Hybrid Quantum-Classical Transformer for Enhanced NLP

Ramazan Amire, Amire Ramazan

Published Apr 28, 2025• 0 citations
Publisher

Abstract

Transformer architectures have driven advances in natural language processing, powering systems from question answering to creative text generation. Yet as model sizes have grown from hundreds of millions to hundreds of billions of parameters, the computational cost of these systems has become prohibitive, placing advanced language technology out of reach for many researchers. In this work, we introduce QuantumGPTMini, a hybrid quantum-classical transformer with just over one million classical parameters that leverages lightweight quantum circuits to perform its core attention and feed-forward computations. By encoding token embeddings both in conventional vectors and in the amplitudes of small variational quantum circuits, our model replaces the conventional softmax attention mechanism with an entanglement-driven alternative. When trained on the Wikipedia-2 corpus using PennyLane's lightning.qubit simulator on NVIDIA A100 GPUs, QuantumGPTMini matched the perplexity of a GPT-2 Small baseline while demonstrating superior pattern recognition and symbolic reasoning behaviors. We have made this system publicly accessible through a FastAPI endpoint and a Next.js web interface. While our initial results underscore the potential of quantum-augmented language models, they also highlight challenges in moving beyond simulation, including circuit depth limitations, decoherence, and the high overhead of parameter-shift gradient estimation.

DOI
10.22541/au.174587094.49802997/v1

Related papers

Quantum-Enhanced Attention Mechanism in NLP: A Hybrid Classical-Quantum Approach

S. M. Yousuf Iqbal Tomal, Abdullah Al Shafin, Bhattacharjee, Debojit +2 more

Recent advances in quantum computing have opened new pathways for enhancing deep learning architectures, particularly in domains characterized by high-dimensional and context-rich data such as natural language processing (NLP). In this work, we present a hybrid classical-quantum Transformer model that integrates a quantum-enhanced attention mechanism into the standard classical architecture. By embedding token representations into a quantum Hilbert space via parameterized variational circuits and exploiting entanglement-aware kernel similarities, the model captures complex semantic relationships beyond the reach of conventional dot-product attention. We demonstrate the effectiveness of this approach across diverse NLP benchmarks, showing improvements in both efficiency and representational capacity. The results section reveal that the quantum attention layer yields globally coherent attention maps and more separable latent features, while requiring comparatively fewer parameters than classical counterparts. These findings highlight the potential of quantum-classical hybrid models to serve as a powerful and resource-efficient alternative to existing attention mechanisms in NLP.

arXiv:2501.15630•Jan 26, 2025•3 citations

GroverAttention: Quantum-Enhanced Attention Mechanism for Effcient Transformers

Belkacem Chikhaoui

Abstract Transformer-based models have revolutionized natural language processing (NLP), but their computational complexity remains a bottleneck. This paper proposes a novel approach to integrating Quantum Attention into Transformers, leveraging Grover’s algorithm to optimize query-key matching. By replacing classical attention mechanisms with quantum-enhanced token selection, we achieve reduced computational cost and improved inference speed. Experimental results on real-world datasets demonstrate the effectiveness of our hybrid Quantum-Classical Transformer, showing an improvement in accuracy with reduced training overhead. Our findings suggest that quantum-assisted attention has the potential to significantly optimize deep learning architectures for NLP applications.

•Oct 31, 2025•0 citations

Quantum Natural Language Processing Using Hybrid AI Algorithms: A Variational Quantum–Classical Framework for Efficient Text Classification

P. Anitha

Quantum Natural Language Processing (QNLP) represents a novel interdisciplinaryparadigm that integrates Quantum Computing with Natural Language Processing toenhance semantic modeling and computational efficiency. Despite the remarkable successof transformer-based architectures such as BERT and GPT, classical NLP systems requirelarge-scale computational resources and extensive parameter tuning. This paper proposesa hybrid quantum–classical architecture employing parameterized quantum circuits (PQCs)integrated with classical embedding layers for text classification tasks. The frameworkutilizes classical preprocessing, quantum state encoding, variational optimization, andclassical post-measurement classification. Experimental results on benchmark sentimentanalysis datasets demonstrate competitive accuracy with significantly reduced parametercomplexity. The findings indicate that hybrid QNLP models provide enhanced semanticcompositionality and efficient representation in Hilbert space while remaining feasiblewithin Noisy Intermediate-Scale Quantum (NISQ) constraints

International Journal For Multidisciplinary Research•Mar 4, 2026•0 citations

A Hybrid Quantum-Classical Language Model for Text Generation: A Case Study on The Gift of the Magi

Aadrian Routh, Juhi M Singh

This paper presents a hybrid quantum-classical language model for text generation, evaluated through a detailed case study of O. Henry's classic short story, The Gift of the Magi. The model integrates a classical transformer with a quantum generator to enhance the extraction of abstract narrative features. Our system architecture includes stages for input tokenization, contextual embedding through multi-head attention, quantumenhanced feature extraction using a parameterized quantum circuit, and output refinement through a classical discriminator. Quantitative evaluations using the BLEU score, besides qualitative analysis, reveal that while the hybrid approach generates syntactically coherent text, it currently falls short in capturing the nuanced symbolic and emotional depth of the original narrative. The findings show the present limitations of quantum-enhanced language models and point towards future research directions in refining quantum circuit designs, reducing noise, and improving integration strategies to fully exploit the potential of quantum computing in natural language processing.

International Conference on Multidisciplinary Research and Innovations in Engineering (MRIE)•Jul 30, 2025•1 citation

A Hybrid Transformer Architecture with a Quantized Self-Attention Mechanism Applied to Molecular Generation

Anthony M. Smaldone, Yu Shee, Gregory W. Kyro +4 more

The success of the self-attention mechanism in classical machine learning models has inspired the development of quantum analogs aimed at reducing the computational overhead. Self-attention integrates learnable query and key matrices to calculate attention scores between all pairs of tokens in a sequence. These scores are then multiplied by a learnable value matrix to obtain the output self-attention matrix, enabling the model to effectively capture long-range dependencies within the input sequence. Here, we propose a hybrid quantum-classical self-attention mechanism as part of a transformer decoder, the architecture underlying large language models (LLMs). To demonstrate its utility in chemistry, we train this model on the QM9 dataset for conditional generation, using SMILES strings as input, each labeled with a set of physicochemical properties that serve as conditions during inference. Our theoretical analysis shows that the time complexity of the query-key dot product is reduced from O ( n 2 d ) in a classical model to O ( n 2 log ⁡ d ) in our quantum model, where n and d represent the sequence length and the embedding dimension, respectively. We perform simulations using NVIDIA’s CUDA-Q platform, which is designed for efficient GPU scalability. This work provides a promising avenue for quantum-enhanced natural language processing (NLP).

arXiv:2502.19214•Journal of Chemical Theory and Computation•May 7, 2025•16 citations

FitQT: a fully integrated and trainable quantum transformer for scalable natural language processing

Lisa Pham, Anh The Le

Abstract The combination of Transformer architectures with Quantum Computing has opened new directions in Quantum Natural Language Processing (QNLP) by leveraging the Transformer’s representational capacity together with the computational parallelism and high-dimensional feature space of quantum systems. However, existing approaches remain constrained by partial designs and limited integration, hindering scalability and trainability. This study introduces FitQT—a novel, fully integrated, and trainable quantum transformer that unifies three essential components: (1) a quantum multi-head attention mechanism employing a scaled Gaussian kernel, (2) a quantum feed-forward network, and (3) quantum residual connections. The residual connections stabilize gradient flow and facilitate the training of deeper quantum circuits, while the attention and feed-forward components jointly enhance representational capacity. Evaluations on sentiment analysis datasets demonstrate FitQT’s effectiveness. On the IMDb dataset, FitQT improves accuracy by 14.89% over the baseline Quantum Self-Attention Network and achieves competitive task-level performance relative to BERT-Base, despite being trained from scratch without large-scale pretraining, while requiring only 2,595 parameters and substantially reduced training time. Moreover, FitQT achieves state-of-the-art performance compared to prior quantum-based transformer architectures such as QSANN and QMSAN. These results highlight the importance of incorporating classical deep learning principles into quantum architectures and provide a practical foundation for scalable, resource-efficient QNLP systems suitable for the noisy intermediate-scale quantum era.

Engineering Research Express•Mar 19, 2026•0 citations
Transformer architectures have driven advances in natural language processing, powering systems from question answering to creative text generation. Yet as model sizes have grown from hundreds of millions to hundreds of billions of parameters, the computational cost of these systems has become prohibitive, placing advanced language technology out of reach for many researchers. In this work, we introduce QuantumGPTMini, a hybrid quantum-classical transformer with just over one million classical parameters that leverages lightweight quantum circuits to perform its core attention and feed-forward computations. By encoding token embeddings both in conventional vectors and in the amplitudes of small variational quantum circuits, our model replaces the conventional softmax attention mechanism with an entanglement-driven alternative. When trained on the Wikipedia-2 corpus using PennyLane's lightning.qubit simulator on NVIDIA A100 GPUs, QuantumGPTMini matched the perplexity of a GPT-2 Small baseline while demonstrating superior pattern recognition and symbolic reasoning behaviors. We have made this system publicly accessible through a FastAPI endpoint and a Next.js web interface. While our initial results underscore the potential of quantum-augmented language models, they also highlight challenges in moving beyond simulation, including circuit depth limitations, decoherence, and the high overhead of parameter-shift gradient estimation.
Recent advances in quantum computing have opened new pathways for enhancing deep learning architectures, particularly in domains characterized by high-dimensional and context-rich data such as natural language processing (NLP). In this work, we present a hybrid classical-quantum Transformer model that integrates a quantum-enhanced attention mechanism into the standard classical architecture. By embedding token representations into a quantum Hilbert space via parameterized variational circuits and exploiting entanglement-aware kernel similarities, the model captures complex semantic relationships beyond the reach of conventional dot-product attention. We demonstrate the effectiveness of this approach across diverse NLP benchmarks, showing improvements in both efficiency and representational capacity. The results section reveal that the quantum attention layer yields globally coherent attention maps and more separable latent features, while requiring comparatively fewer parameters than classical counterparts. These findings highlight the potential of quantum-classical hybrid models to serve as a powerful and resource-efficient alternative to existing attention mechanisms in NLP.
Abstract Transformer-based models have revolutionized natural language processing (NLP), but their computational complexity remains a bottleneck. This paper proposes a novel approach to integrating Quantum Attention into Transformers, leveraging Grover’s algorithm to optimize query-key matching. By replacing classical attention mechanisms with quantum-enhanced token selection, we achieve reduced computational cost and improved inference speed. Experimental results on real-world datasets demonstrate the effectiveness of our hybrid Quantum-Classical Transformer, showing an improvement in accuracy with reduced training overhead. Our findings suggest that quantum-assisted attention has the potential to significantly optimize deep learning architectures for NLP applications.
Quantum Natural Language Processing (QNLP) represents a novel interdisciplinaryparadigm that integrates Quantum Computing with Natural Language Processing toenhance semantic modeling and computational efficiency. Despite the remarkable successof transformer-based architectures such as BERT and GPT, classical NLP systems requirelarge-scale computational resources and extensive parameter tuning. This paper proposesa hybrid quantum–classical architecture employing parameterized quantum circuits (PQCs)integrated with classical embedding layers for text classification tasks. The frameworkutilizes classical preprocessing, quantum state encoding, variational optimization, andclassical post-measurement classification. Experimental results on benchmark sentimentanalysis datasets demonstrate competitive accuracy with significantly reduced parametercomplexity. The findings indicate that hybrid QNLP models provide enhanced semanticcompositionality and efficient representation in Hilbert space while remaining feasiblewithin Noisy Intermediate-Scale Quantum (NISQ) constraints
This paper presents a hybrid quantum-classical language model for text generation, evaluated through a detailed case study of O. Henry's classic short story, The Gift of the Magi. The model integrates a classical transformer with a quantum generator to enhance the extraction of abstract narrative features. Our system architecture includes stages for input tokenization, contextual embedding through multi-head attention, quantumenhanced feature extraction using a parameterized quantum circuit, and output refinement through a classical discriminator. Quantitative evaluations using the BLEU score, besides qualitative analysis, reveal that while the hybrid approach generates syntactically coherent text, it currently falls short in capturing the nuanced symbolic and emotional depth of the original narrative. The findings show the present limitations of quantum-enhanced language models and point towards future research directions in refining quantum circuit designs, reducing noise, and improving integration strategies to fully exploit the potential of quantum computing in natural language processing.
The success of the self-attention mechanism in classical machine learning models has inspired the development of quantum analogs aimed at reducing the computational overhead. Self-attention integrates learnable query and key matrices to calculate attention scores between all pairs of tokens in a sequence. These scores are then multiplied by a learnable value matrix to obtain the output self-attention matrix, enabling the model to effectively capture long-range dependencies within the input sequence. Here, we propose a hybrid quantum-classical self-attention mechanism as part of a transformer decoder, the architecture underlying large language models (LLMs). To demonstrate its utility in chemistry, we train this model on the QM9 dataset for conditional generation, using SMILES strings as input, each labeled with a set of physicochemical properties that serve as conditions during inference. Our theoretical analysis shows that the time complexity of the query-key dot product is reduced from O ( n 2 d ) in a classical model to O ( n 2 log ⁡ d ) in our quantum model, where n and d represent the sequence length and the embedding dimension, respectively. We perform simulations using NVIDIA’s CUDA-Q platform, which is designed for efficient GPU scalability. This work provides a promising avenue for quantum-enhanced natural language processing (NLP).
Abstract The combination of Transformer architectures with Quantum Computing has opened new directions in Quantum Natural Language Processing (QNLP) by leveraging the Transformer’s representational capacity together with the computational parallelism and high-dimensional feature space of quantum systems. However, existing approaches remain constrained by partial designs and limited integration, hindering scalability and trainability. This study introduces FitQT—a novel, fully integrated, and trainable quantum transformer that unifies three essential components: (1) a quantum multi-head attention mechanism employing a scaled Gaussian kernel, (2) a quantum feed-forward network, and (3) quantum residual connections. The residual connections stabilize gradient flow and facilitate the training of deeper quantum circuits, while the attention and feed-forward components jointly enhance representational capacity. Evaluations on sentiment analysis datasets demonstrate FitQT’s effectiveness. On the IMDb dataset, FitQT improves accuracy by 14.89% over the baseline Quantum Self-Attention Network and achieves competitive task-level performance relative to BERT-Base, despite being trained from scratch without large-scale pretraining, while requiring only 2,595 parameters and substantially reduced training time. Moreover, FitQT achieves state-of-the-art performance compared to prior quantum-based transformer architectures such as QSANN and QMSAN. These results highlight the importance of incorporating classical deep learning principles into quantum architectures and provide a practical foundation for scalable, resource-efficient QNLP systems suitable for the noisy intermediate-scale quantum era.