Abstract In the noisy intermediate-scale quantum (NISQ) era, developing attention mechanisms suitable for quantum hardware is essential for advancing quantum-enhanced Transformers applications. This work introduces a novel quantum Transformer model that integrates a low-depth, ancilla-free attention mechanism based on the Destructive swap-test variant, enabling efficient computation of attention scores without auxiliary qubits. Designed specifically for NISQ constraints, the model emphasizes noise resilience and reduced circuit depth, making it practical for current quantum devices. We evaluate the model using transparent and well-documented Natural Language Processing datasets designed for Quantum Computing (QC). Our results demonstrate competitive performance on language tasks, supported by both quantitative metrics that offer interpretable insights into the model's behavior. This work provides a viable pathway toward practical, interpretable quantum language models in the near term.
Abstract In the noisy intermediate-scale quantum (NISQ) era, developing attention mechanisms suitable for quantum hardware is essential for advancing quantum-enhanced Transformers applications. This work introduces a novel quantum Transformer model that integrates a low-depth, ancilla-free attention mechanism based on the Destructive swap-test variant, enabling efficient computation of attention scores without auxiliary qubits. Designed specifically for NISQ constraints, the model emphasizes noise resilience and reduced circuit depth, making it practical for current quantum devices. We evaluate the model using transparent and well-documented Natural Language Processing datasets designed for Quantum Computing (QC). Our results demonstrate competitive performance on language tasks, supported by both quantitative metrics that offer interpretable insights into the model's behavior. This work provides a viable pathway toward practical, interpretable quantum language models in the near term.
Quantum computing is rapidly emerging as a revolutionary computing paradigm with significant development potential. Variational quantum algorithms, especially those employing variational quantum circuits, have demonstrated notable advantages in domains including finance, chemistry, and machine learning. However, the practical deployment of these algorithms on Noisy Intermediate-Scale Quantum (NISQ) devices faces challenges due to quantum noise, state collapse, and limited qubits numbers. The real-world performance of these algorithms remains largely unexplored. This paper addresses this gap by proposing and implementing a novel method for training quantum self-attention model for text classification directly on quantum computing chips. This initiative marks the first instance of employing such models on actual quantum hardware. Our approach optimizes quantum circuit designs to mitigate the impact of quantum noise and introduces parallel training strategies to enhance efficiency. Our experiments, conducted on the “Wukong” superconducting 72-qubit quantum computer, demonstrate that our models can be effectively trained on real quantum hardware. The results from the actual quantum chips slightly outperformed those from simulators, confirming the practical feasibility of applying quantum self-attention model to natural language processing tasks. These results validate the effectiveness of our approach and provide valuable insights for the future deployment of quantum machine learning models.
Abstract The combination of Transformer architectures with Quantum Computing has opened new directions in Quantum Natural Language Processing (QNLP) by leveraging the Transformer’s representational capacity together with the computational parallelism and high-dimensional feature space of quantum systems. However, existing approaches remain constrained by partial designs and limited integration, hindering scalability and trainability. This study introduces FitQT—a novel, fully integrated, and trainable quantum transformer that unifies three essential components: (1) a quantum multi-head attention mechanism employing a scaled Gaussian kernel, (2) a quantum feed-forward network, and (3) quantum residual connections. The residual connections stabilize gradient flow and facilitate the training of deeper quantum circuits, while the attention and feed-forward components jointly enhance representational capacity. Evaluations on sentiment analysis datasets demonstrate FitQT’s effectiveness. On the IMDb dataset, FitQT improves accuracy by 14.89% over the baseline Quantum Self-Attention Network and achieves competitive task-level performance relative to BERT-Base, despite being trained from scratch without large-scale pretraining, while requiring only 2,595 parameters and substantially reduced training time. Moreover, FitQT achieves state-of-the-art performance compared to prior quantum-based transformer architectures such as QSANN and QMSAN. These results highlight the importance of incorporating classical deep learning principles into quantum architectures and provide a practical foundation for scalable, resource-efficient QNLP systems suitable for the noisy intermediate-scale quantum era.
Self-attention mechanisms, central to Transformer architectures, have become foundational in the development of large language models (LLMs) and various advanced machine learning systems. These mechanisms are pivotal for effectively capturing complex data dependencies and relationships, which is crucial for tasks such as natural language understanding and generation. However, as dimensionality increases, self-attention mechanisms face significant challenges related to computational efficiency and scalability. This paper introduces an encoding-based quantum self-attention mechanism designed to address these issues by embedding classical data into quantum states. This approach facilitates more efficient computation and reduces model complexity. To further enhance performance for real-world applications, particularly in financial time series forecasting, we propose a hybrid model that integrates Long Short-Term Memory (LSTM) networks with quantum self-attention mechanism. Given the constraints of the Noisy Intermediate-Scale Quantum (NISQ) era, this hybrid approach represents a plausible solution, leveraging the strengths of both classical and quantum methods and focusing on achieving faster convergence. Experimental results demonstrate that our approach converges significantly quicker than traditional models, showcasing the advantages of integrating quantum techniques with classical machine learning for practical applications.
Noisy Intermediate-Scale Quantum (NISQ) devices are promising platforms for practical quantum computing, but their performance is significantly affected by limited qubit availability, gate errors, decoherence, circuit depth, and hardware-dependent noise. Conventional quantum circuits are generally designed and optimized before execution and may not respond effectively to changing hardware conditions during computation. This research proposes a Self-Adaptive Quantum Circuit framework that dynamically modifies quantum circuits according to the current computational resources and noise characteristics of NISQ devices. The proposed framework monitors parameters such as qubit fidelity, gate error rates, circuit depth, execution time, and resource availability, and automatically selects suitable circuit configurations to improve computational performance. Adaptive mechanisms can be employed to reduce unnecessary gates, optimize qubit utilization, and mitigate the impact of hardware noise while maintaining the required computational accuracy. The framework will be evaluated using simulation platforms and available NISQ quantum hardware, comparing adaptive circuits with conventional static circuits using metrics such as accuracy, fidelity, circuit depth, gate count, execution time, and error rate. The expected outcome is a resource-efficient and noise-aware quantum circuit optimization approach capable of improving the reliability and performance of quantum applications on near-term quantum hardware. The proposed research aims to contribute toward more practical, flexible, and hardware-aware quantum computing systems for NISQ-era applications. Keywords: NISQ Computing, Self-Adaptive Quantum Circuits, Quantum Circuit Optimization, Quantum Error Mitigation, Quantum Noise, Quantum Computing.
Vectorized quantum block encoding provides a way to embed classical data into Hilbert space, offering a pathway for quantum models, such as Quantum Transformers (QT), that replace classical self-attention with quantum circuit simulations to operate more efficiently. Current QTs rely on deep-parameterized quantum circuits (PQCs), rendering them vulnerable to QPU noise, and thus hindering their practical performance. In this paper, we propose the Vectorized Quantum Transformer (VQT), a model that supports ideal masked-attention matrix computation through quantum approximation simulation and efficient training via vectorized nonlinear quantum encoder, yielding shot-efficient and gradient-free quantum circuit simulation (QCS) and reduced classical sampling overhead. In addition, we demonstrate an accuracy comparison for IBM and IonQ in quantum circuit simulation and competitive results in benchmarking natural language processing tasks on IBM’s state-of-the-art, high-fidelity Kingston QPU. Our noise intermediate-scale quantum (NISQ)-friendly VQT approach unlocks a novel architecture for end-to-end machine learning in quantum computing.
The remarkable success of Transformer architectures in Large Language Models (LLMs) has revolutionized natural language processing, yet the transition to quantum computing for next-generation language models remains an open challenge. While quantum computing promises exponential advantages, a fundamental gap exists between classical deep learning and quantum computing paradigms, particularly given the severe constraints of Noisy Intermediate-Scale Quantum (NISQ) devices, including barren plateaus, limited qubit coherence, and circuit depth restrictions. We present Adaptive Quantum-Classical Fusion (AQCF), the first framework to bridge classical and quantum computing for language models by reimagining Transformer architectures through quantum-classical co-design. Our key insight is that effective bridging requires dynamic adaptation rather than static translation—the framework analyzes input complexity in real-time to orchestrate seamless transitions between classical and quantum processing. AQCF introduces entropy-driven adaptive circuits that circumvent barren plateaus, quantum memory banks that unify classical attention with quantum state-based similarity retrieval, and intelligent fusion controllers that ensure each computational paradigm handles tasks where it naturally excels. This bridging architecture maintains full compatibility with existing classical Transformers while progressively incorporating quantum advantages as they become accessible. Experiments on sentiment analysis demonstrate that AQCF achieves competitive performance while significantly improving quantum resource efficiency, operating successfully within typical NISQ constraints. By establishing a seamless integration pathway from today's classical LLMs to tomorrow's quantum-enhanced models, our framework provides both immediate practical value on current quantum hardware and a clear evolution path toward full Quantum LLMs as technology matures.