Transformers have become the dominant architecture for sequence modeling in natural language processing; however, their effectiveness critically depends on how positional information is encoded. Conventional positional encodings, while effective, may have limited structural flexibility for capturing complex global sequence relationships. Recent quantum-inspired approaches have sought to address this limitation, yet many either oversimplify quantum principles or introduce substantial computational or hardware overhead. We introduce a novel Quantum Fourier Transform (QFT)-inspired positional encoding scheme for transformers, motivated by the structured frequency representation of the QFT. Unlike prior approaches that either emulate quantum operations superficially or require complex circuit constructions, the proposed method provides a learnable hybrid encoding that preserves quantum-inspired structure while remaining aligned with hardware-efficient circuit primitives and structurally compatible with future near-term quantum implementations. Experiments on WikiText-103 indicate that the proposed encoding achieves competitive perplexity, improved robustness to input scrambling, and stable training behavior relative to alternative quantum-inspired baselines under the evaluated settings. Preliminary circuit-level simulations further suggest favorable noise resilience of the associated encoding primitives. These findings support the potential utility of incorporating quantum-inspired design principles into deep learning architectures and provide a foundation for future exploration at the interface of quantum computing and transformer-based natural language processing (NLP).
Transformers have become the dominant architecture for sequence modeling in natural language processing; however, their effectiveness critically depends on how positional information is encoded. Conventional positional encodings, while effective, may have limited structural flexibility for capturing complex global sequence relationships. Recent quantum-inspired approaches have sought to address this limitation, yet many either oversimplify quantum principles or introduce substantial computational or hardware overhead. We introduce a novel Quantum Fourier Transform (QFT)-inspired positional encoding scheme for transformers, motivated by the structured frequency representation of the QFT. Unlike prior approaches that either emulate quantum operations superficially or require complex circuit constructions, the proposed method provides a learnable hybrid encoding that preserves quantum-inspired structure while remaining aligned with hardware-efficient circuit primitives and structurally compatible with future near-term quantum implementations. Experiments on WikiText-103 indicate that the proposed encoding achieves competitive perplexity, improved robustness to input scrambling, and stable training behavior relative to alternative quantum-inspired baselines under the evaluated settings. Preliminary circuit-level simulations further suggest favorable noise resilience of the associated encoding primitives. These findings support the potential utility of incorporating quantum-inspired design principles into deep learning architectures and provide a foundation for future exploration at the interface of quantum computing and transformer-based natural language processing (NLP).
Quantum-Inspired Transformers are a new class of approaches for natural language processing (NLP) that use principles inspired by physical quantum mechanics to encode and process linguistic information. Paper aims to improve both scalability and efficiency for large-scale NLP tasks in the next generation of artificial intelligence (AI). However, traditional transformers can struggle to encode complex semantic relationships across long sequences and can require high levels of compute resources, hence limiting scalability for large datasets and multilingual tasks. In addition, traditional embeddings do not represent multiple contextual meanings simultaneously, which decreases model expressiveness. To address these challenges and maximize the exploration of potential quantum effects in AI, this paper presents a Quantum State Embedding Attention (QSEA) framework that encodes tokens as quantum inspired states with complex-valued amplitudes. Attention, which is based on entangled states, is meant to efficiently capture long-range dependencies while reducing compute and increasing scalability, all without losing a rich understanding of context. Whereas multilingual translation needs to encode both the source similarity and the target language within a smaller semantic representation space. Our experimentation shows that QSEA consistently provides increased accuracy in decoding, measures attention across long sequences, and yields enduring scalability over large multilingual datasets. The evidence demonstrates the potential for QSEA to be the new layer in the future of AI systems, a layer that is scalable and a highperforming NLP application.
Transformer architectures have driven advances in natural language processing, powering systems from question answering to creative text generation. Yet as model sizes have grown from hundreds of millions to hundreds of billions of parameters, the computational cost of these systems has become prohibitive, placing advanced language technology out of reach for many researchers. In this work, we introduce QuantumGPTMini, a hybrid quantum-classical transformer with just over one million classical parameters that leverages lightweight quantum circuits to perform its core attention and feed-forward computations. By encoding token embeddings both in conventional vectors and in the amplitudes of small variational quantum circuits, our model replaces the conventional softmax attention mechanism with an entanglement-driven alternative. When trained on the Wikipedia-2 corpus using PennyLane's lightning.qubit simulator on NVIDIA A100 GPUs, QuantumGPTMini matched the perplexity of a GPT-2 Small baseline while demonstrating superior pattern recognition and symbolic reasoning behaviors. We have made this system publicly accessible through a FastAPI endpoint and a Next.js web interface. While our initial results underscore the potential of quantum-augmented language models, they also highlight challenges in moving beyond simulation, including circuit depth limitations, decoherence, and the high overhead of parameter-shift gradient estimation.
Recent advances in quantum computing have opened new pathways for enhancing deep learning architectures, particularly in domains characterized by high-dimensional and context-rich data such as natural language processing (NLP). In this work, we present a hybrid classical-quantum Transformer model that integrates a quantum-enhanced attention mechanism into the standard classical architecture. By embedding token representations into a quantum Hilbert space via parameterized variational circuits and exploiting entanglement-aware kernel similarities, the model captures complex semantic relationships beyond the reach of conventional dot-product attention. We demonstrate the effectiveness of this approach across diverse NLP benchmarks, showing improvements in both efficiency and representational capacity. The results section reveal that the quantum attention layer yields globally coherent attention maps and more separable latent features, while requiring comparatively fewer parameters than classical counterparts. These findings highlight the potential of quantum-classical hybrid models to serve as a powerful and resource-efficient alternative to existing attention mechanisms in NLP.
Abstract Language modeling in natural language processing has predominantly relied on large-scale classical neural architectures, yet constructing expressive and resource-efficient models compatible with Noisy Intermediate-Scale Quantum (NISQ) devices remains a fundamental open challenge. In this work, two Ising-inspired variational quantum-classical architectures are proposed to enhance representational capacity by leveraging spin-coupling mechanisms and Ising-type Hamiltonian evolution within hybrid classical-quantum layers. Specifically, a quantum Ising Ansatz circuit parameterized by an Ising Hamiltonian is constructed, and a quantum approximate optimization algorithm (QAOA) is incorporated to reduce the number of trainable circuit parameters while enabling learnable quantum evolution of hidden states. To achieve stable and efficient hybrid training under NISQ constraints, a joint optimization strategy is introduced that combines the Adam optimizer with quantum simultaneous perturbation stochastic approximation (SPSA), yielding convergence acceleration of up to 64% relative to parameter-shift rule and Adam-only baselines. Empirical evaluations on the WikiText-2 benchmark demonstrate that the proposed architectures effectively capture long-range semantic dependencies, achieving a competitive perplexity of PPL=39.8 (lower values indicate stronger language modeling performance). This result surpasses existing quantum language models, including Quixer, Quantum LSTM, and Quantum GRU, and approaches the performance of the classical NanoGPT baseline (PPL=40.5). Collectively, these findings establish the viability of Ising-based hybrid quantum-classical networks as a promising direction for practical quantum natural language processing.
We propose a compact NISQ-compatible quantum transformer architecture for synthetic QNLP sequence modelling. The model preserves the autoregressive next-token interface of a classical transformer, but replaces attention and feed-forward sublayers with variational quantum encoder blocks, connector circuits, decoder blocks and a direct two-qubit measurement readout. Token contexts are angle-encoded into small quantum registers, processed by parallel variational heads and encoder integration circuits and conditioned through decoder ancillae to produce a distribution over a four-token vocabulary. We evaluate several architecture variants on deterministic and lexicographic grammar-generation tasks against a compact classical transformer baseline. The quantum models are trainable end-to-end and learn nontrivial grammar structure, including perfect deterministic generation in individual runs and high lexicographic validity in the strongest variant. The classical baseline remains more accurate and stable and the quantum models are sensitive to initialization. The contribution is therefore not a claim of quantum advantage, but a concrete architecture and evaluation of transformer-inspired QNLP sequence modelling under near-term quantum constraints.
This paper introduces an innovative theoretical framework for quantum-inspired data embeddings, grounded in foundational concepts of quantum mechanics such as superposition and entanglement. This approach aims to advance semi-supervised learning in contexts characterized by limited labeled data by enabling more intricate and expressive embeddings that capture the underlying structure of the data effectively. Grounded in foundational quantum mechanics concepts such as superposition and entanglement, this approach redefines data representation by enabling more intricate and expressive embeddings. Emulating quantum superposition encodes each data point as a probabilistic amalgamation of multiple feature states, facilitating a richer, multidimensional representation of underlying structures and patterns. Additionally, quantum-inspired entanglement mechanisms are harnessed to model intricate dependencies between labeled and unlabeled data, promoting enhanced knowledge transfer and structural inference within the learning paradigm. In contrast to conventional quantum machine learning methodologies that often rely on quantum hardware, this framework is fully realizable within classical computational architectures, thus bypassing the practical limitations of quantum hardware. The versatility of this model is illustrated through its application to critical domains such as medical diagnosis, resource-constrained natural language processing, and financial forecasting—areas where data scarcity impedes the efficacy of traditional models. Experimental evaluations reveal that quantum-inspired embeddings substantially outperform standard approaches, enhancing model resilience and generalization in high-dimensional, low-sample scenarios. This research marks a significant stride in integrating quantum theoretical principles with classical machine learning, broadening the scope of data representation and semi-supervised learning while circumventing the technological barriers of quantum computing infrastructure.