Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2508.18464 · Proceedings of the AAAI Symposium Series

Vectorized Attention with Learnable Encoding for Quantum Transformer

Ziqing Guo, Ziwen Pan, Alex Khan, J. Balewski

Published Nov 23, 2025• 2 citations
arXivHTMLPDF

Abstract

Vectorized quantum block encoding provides a way to embed classical data into Hilbert space, offering a pathway for quantum models, such as Quantum Transformers (QT), that replace classical self-attention with quantum circuit simulations to operate more efficiently. Current QTs rely on deep-parameterized quantum circuits (PQCs), rendering them vulnerable to QPU noise, and thus hindering their practical performance. In this paper, we propose the Vectorized Quantum Transformer (VQT), a model that supports ideal masked-attention matrix computation through quantum approximation simulation and efficient training via vectorized nonlinear quantum encoder, yielding shot-efficient and gradient-free quantum circuit simulation (QCS) and reduced classical sampling overhead. In addition, we demonstrate an accuracy comparison for IBM and IonQ in quantum circuit simulation and competitive results in benchmarking natural language processing tasks on IBM’s state-of-the-art, high-fidelity Kingston QPU. Our noise intermediate-scale quantum (NISQ)-friendly VQT approach unlocks a novel architecture for end-to-end machine learning in quantum computing.

DOI
10.1609/aaaiss.v7i1.36905

Related papers

FitQT: a fully integrated and trainable quantum transformer for scalable natural language processing

Lisa Pham, Anh The Le

Abstract The combination of Transformer architectures with Quantum Computing has opened new directions in Quantum Natural Language Processing (QNLP) by leveraging the Transformer’s representational capacity together with the computational parallelism and high-dimensional feature space of quantum systems. However, existing approaches remain constrained by partial designs and limited integration, hindering scalability and trainability. This study introduces FitQT—a novel, fully integrated, and trainable quantum transformer that unifies three essential components: (1) a quantum multi-head attention mechanism employing a scaled Gaussian kernel, (2) a quantum feed-forward network, and (3) quantum residual connections. The residual connections stabilize gradient flow and facilitate the training of deeper quantum circuits, while the attention and feed-forward components jointly enhance representational capacity. Evaluations on sentiment analysis datasets demonstrate FitQT’s effectiveness. On the IMDb dataset, FitQT improves accuracy by 14.89% over the baseline Quantum Self-Attention Network and achieves competitive task-level performance relative to BERT-Base, despite being trained from scratch without large-scale pretraining, while requiring only 2,595 parameters and substantially reduced training time. Moreover, FitQT achieves state-of-the-art performance compared to prior quantum-based transformer architectures such as QSANN and QMSAN. These results highlight the importance of incorporating classical deep learning principles into quantum architectures and provide a practical foundation for scalable, resource-efficient QNLP systems suitable for the noisy intermediate-scale quantum era.

Engineering Research Express•Mar 19, 2026•0 citations

A Quantum Self-Attention Neural Network Model on Quantum Circuits

Jin Yan Zheng, Qing Zhong Gao, Daoyi Dong +2 more

This article proposes a quantum self-attention neural network (QSAN) model that is implementable on parameterized quantum circuits (PQCs), providing a novel avenue for addressing natural language processing (NLP) tasks via quantum computing. The QSAN architecture comprises four blocks: the data preprocessing block, the quantum encoding block, the model design block, and the network optimization block. Through these blocks, classical text data are initially preprocessed and encoded into quantum states. The rich semantic features and intricate relationships within these quantum states are then captured and learned by a quantum self-attention layer and a quantum fully connected layer. Finally, the model performance is enhanced by optimizing the network parameters. Simulation results on publicly available topic classification and sentiment analysis datasets demonstrate that the QSAN model outperforms state-of-the-art baselines. As a limitation of the current study, these evaluations are restricted to a maximum sentence length of eight words due to simulation and hardware constraints. Moreover, the effectiveness, scalability, trainability, and robustness of the proposed model are comprehensively evaluated, highlighting its superior performance and potential for advanced NLP tasks.

IEEE Transactions on Cybernetics•Jun 29, 2026•0 citations

QiT: Quantum-Inspired Transformer for Visual Recognition Task

Badri N. Patro, Vijay Agneeswaran

Quantum machine learning offers a compelling representational perspective: angle-encoded states inhabit Hilbert spaces in which periodic similarities and interactions can be expressed naturally. Realizing this perspective for visual recognition remains difficult, however, because present quantum neural networks are constrained by limited qubit counts, costly circuit simulation and measurement, noise, and unstable optimization on noisy intermediate-scale quantum devices. We investigate whether useful structural ideas from quantum models can instead be realized as scalable classical Transformer operations. We introduce QiT, a Quantum-inspired Transformer for vision tasks with three components: (i) angle-inspired encoding that maps image tokens to learned trigonometric Hilbert-space features analogous to quantum rotation-based state encoding; (ii) self-attention over these periodic features, inducing a classical cosine kernel approximated to quantum fidelity kernels; and (iii) gated multiplicative emulation, a trainable classical surrogate for interaction terms found in variational circuits. All components are differentiable tensor operations, so QiT claims neither quantum computation nor quantum speedup and retains the $\mathcal{O}(N^2D)$ attention complexity of a standard Vision Transformer. Across image-classification benchmarks, QiT is competitive with a matched classical Transformer while avoiding the severe runtime cost observed for a small simulated quantum Transformer. QiT-B reaches 78.3\% ImageNet-1K top-1 accuracy with 45.7M parameters and 11.5 GFLOPs. These results position QiT as a scalable baseline for isolating and evaluating quantum-motivated inductive biases in visual recognition.

quant-pharXiv:2609.17789•Sep 17, 2026•0 citations

Quantum Machine Learning: A Comparative Study of Variational Quantum Classifier with Classical Classifier Algorithms

Madona Alhalabi, Hafiz M. Asif, Hesham Alhalabi

With the advancement of quantum computing in the Noisy Intermediate-Scale Quantum (NISQ) era, hybrid quantum-classical algorithms have emerged as promising approaches for addressing certain machine learning tasks. This study investigates the Variational Quantum Classifier (VQC), a hybrid model that integrates parameterized quantum circuits with classical optimization routines, implemented via IBM’s Qiskit framework. We provide a detailed account of the VQC architecture, including feature encoding, ansatz construction, and optimization methodology. To assess its performance, we conduct a comparative analysis between VQC and classical Support Vector Classifiers (SVC) across synthetic and real-world datasets, including Iris and a binary-filtered version of MNIST. Experimental results reveal that classical models consistently achieve higher accuracy and faster training times. However, the VQC demonstrates functional viability, particularly under noise-free conditions and reduced-dimensionality settings. These findings underscore the current limitations of quantum models while highlighting the potential of VQC as a proof-of-concept for quantum-enhanced learning as quantum hardware matures.

International Symposium on Networks, Computers and Communications (ISNCC)•Oct 27, 2025•2 citations

Quantum Destructive Self-Attention for NISQ-EraTransformers

Yousra Bouakba, Hacene Belhadef

Abstract In the noisy intermediate-scale quantum (NISQ) era, developing attention mechanisms suitable for quantum hardware is essential for advancing quantum-enhanced Transformers applications. This work introduces a novel quantum Transformer model that integrates a low-depth, ancilla-free attention mechanism based on the Destructive swap-test variant, enabling efficient computation of attention scores without auxiliary qubits. Designed specifically for NISQ constraints, the model emphasizes noise resilience and reduced circuit depth, making it practical for current quantum devices. We evaluate the model using transparent and well-documented Natural Language Processing datasets designed for Quantum Computing (QC). Our results demonstrate competitive performance on language tasks, supported by both quantitative metrics that offer interpretable insights into the model's behavior. This work provides a viable pathway toward practical, interpretable quantum language models in the near term.

•Feb 25, 2026•0 citations

Quantum Parameterized Self-Attention Network for Image Classification

Wenwei Zhang, Jintao Wang, Tianyu Ye +1 more

Transformer now underpins modern AI as its core infrastructure. Its defining capability-dynamically focusing on the most relevant information in complex inputs-is bounded above by the self-attention scoring function. Quantum computing, with its superposition, entanglement, and probabilistic outputs, offers a fundamentally distinct computational framework for exploring beyond the design constraints of classical scoring functions. While quantum attention mechanisms have shown initial promise, existing works remain largely confined to redefining feature similarity measures, leaving the systematic use of parameterized quantum circuits (PQCs) as scoring functions largely unexplored; a substantial portion of existing schemes further rely on purely quantum architectures, precluding effective encoding of high-dimensional image inputs in the Noisy Intermediate-Scale Quantum era. We propose the Quantum Parameterized Self-Attention Network (QPSAN), implementing the self-attention scoring function via PQCs with only 5 trainable quantum parameters per layer. QPSAN computes query-key attention scores through quantum state encoding and joint measurement, yielding naturally bounded outputs without the explicit scaling of classical dot-product attention. We further establish a theoretical framework of the mathematical properties of this scoring function, demonstrating its potential to capture complex nonlinear query-key interactions, and quantifying the structural constraints of the encoding layer via effective degrees of freedom analysis. Experiments on four vision datasets show that QPSAN significantly outperforms the Vision Transformer (ViT) baseline, with the quantum representational advantage amplifying as data complexity increases. Ablation studies indicate that the performance gains may stem from the structural inductive bias of the quantum circuit rather than from parameter scale.

quant-pharXiv:2605.25365•May 26, 2026•1 citation
Vectorized quantum block encoding provides a way to embed classical data into Hilbert space, offering a pathway for quantum models, such as Quantum Transformers (QT), that replace classical self-attention with quantum circuit simulations to operate more efficiently. Current QTs rely on deep-parameterized quantum circuits (PQCs), rendering them vulnerable to QPU noise, and thus hindering their practical performance. In this paper, we propose the Vectorized Quantum Transformer (VQT), a model that supports ideal masked-attention matrix computation through quantum approximation simulation and efficient training via vectorized nonlinear quantum encoder, yielding shot-efficient and gradient-free quantum circuit simulation (QCS) and reduced classical sampling overhead. In addition, we demonstrate an accuracy comparison for IBM and IonQ in quantum circuit simulation and competitive results in benchmarking natural language processing tasks on IBM’s state-of-the-art, high-fidelity Kingston QPU. Our noise intermediate-scale quantum (NISQ)-friendly VQT approach unlocks a novel architecture for end-to-end machine learning in quantum computing.
Abstract The combination of Transformer architectures with Quantum Computing has opened new directions in Quantum Natural Language Processing (QNLP) by leveraging the Transformer’s representational capacity together with the computational parallelism and high-dimensional feature space of quantum systems. However, existing approaches remain constrained by partial designs and limited integration, hindering scalability and trainability. This study introduces FitQT—a novel, fully integrated, and trainable quantum transformer that unifies three essential components: (1) a quantum multi-head attention mechanism employing a scaled Gaussian kernel, (2) a quantum feed-forward network, and (3) quantum residual connections. The residual connections stabilize gradient flow and facilitate the training of deeper quantum circuits, while the attention and feed-forward components jointly enhance representational capacity. Evaluations on sentiment analysis datasets demonstrate FitQT’s effectiveness. On the IMDb dataset, FitQT improves accuracy by 14.89% over the baseline Quantum Self-Attention Network and achieves competitive task-level performance relative to BERT-Base, despite being trained from scratch without large-scale pretraining, while requiring only 2,595 parameters and substantially reduced training time. Moreover, FitQT achieves state-of-the-art performance compared to prior quantum-based transformer architectures such as QSANN and QMSAN. These results highlight the importance of incorporating classical deep learning principles into quantum architectures and provide a practical foundation for scalable, resource-efficient QNLP systems suitable for the noisy intermediate-scale quantum era.
This article proposes a quantum self-attention neural network (QSAN) model that is implementable on parameterized quantum circuits (PQCs), providing a novel avenue for addressing natural language processing (NLP) tasks via quantum computing. The QSAN architecture comprises four blocks: the data preprocessing block, the quantum encoding block, the model design block, and the network optimization block. Through these blocks, classical text data are initially preprocessed and encoded into quantum states. The rich semantic features and intricate relationships within these quantum states are then captured and learned by a quantum self-attention layer and a quantum fully connected layer. Finally, the model performance is enhanced by optimizing the network parameters. Simulation results on publicly available topic classification and sentiment analysis datasets demonstrate that the QSAN model outperforms state-of-the-art baselines. As a limitation of the current study, these evaluations are restricted to a maximum sentence length of eight words due to simulation and hardware constraints. Moreover, the effectiveness, scalability, trainability, and robustness of the proposed model are comprehensively evaluated, highlighting its superior performance and potential for advanced NLP tasks.
Quantum machine learning offers a compelling representational perspective: angle-encoded states inhabit Hilbert spaces in which periodic similarities and interactions can be expressed naturally. Realizing this perspective for visual recognition remains difficult, however, because present quantum neural networks are constrained by limited qubit counts, costly circuit simulation and measurement, noise, and unstable optimization on noisy intermediate-scale quantum devices. We investigate whether useful structural ideas from quantum models can instead be realized as scalable classical Transformer operations. We introduce QiT, a Quantum-inspired Transformer for vision tasks with three components: (i) angle-inspired encoding that maps image tokens to learned trigonometric Hilbert-space features analogous to quantum rotation-based state encoding; (ii) self-attention over these periodic features, inducing a classical cosine kernel approximated to quantum fidelity kernels; and (iii) gated multiplicative emulation, a trainable classical surrogate for interaction terms found in variational circuits. All components are differentiable tensor operations, so QiT claims neither quantum computation nor quantum speedup and retains the O(N2D)\mathcal{O}(N^2D)O(N2D) attention complexity of a standard Vision Transformer. Across image-classification benchmarks, QiT is competitive with a matched classical Transformer while avoiding the severe runtime cost observed for a small simulated quantum Transformer. QiT-B reaches 78.3\% ImageNet-1K top-1 accuracy with 45.7M parameters and 11.5 GFLOPs. These results position QiT as a scalable baseline for isolating and evaluating quantum-motivated inductive biases in visual recognition.
With the advancement of quantum computing in the Noisy Intermediate-Scale Quantum (NISQ) era, hybrid quantum-classical algorithms have emerged as promising approaches for addressing certain machine learning tasks. This study investigates the Variational Quantum Classifier (VQC), a hybrid model that integrates parameterized quantum circuits with classical optimization routines, implemented via IBM’s Qiskit framework. We provide a detailed account of the VQC architecture, including feature encoding, ansatz construction, and optimization methodology. To assess its performance, we conduct a comparative analysis between VQC and classical Support Vector Classifiers (SVC) across synthetic and real-world datasets, including Iris and a binary-filtered version of MNIST. Experimental results reveal that classical models consistently achieve higher accuracy and faster training times. However, the VQC demonstrates functional viability, particularly under noise-free conditions and reduced-dimensionality settings. These findings underscore the current limitations of quantum models while highlighting the potential of VQC as a proof-of-concept for quantum-enhanced learning as quantum hardware matures.
Abstract In the noisy intermediate-scale quantum (NISQ) era, developing attention mechanisms suitable for quantum hardware is essential for advancing quantum-enhanced Transformers applications. This work introduces a novel quantum Transformer model that integrates a low-depth, ancilla-free attention mechanism based on the Destructive swap-test variant, enabling efficient computation of attention scores without auxiliary qubits. Designed specifically for NISQ constraints, the model emphasizes noise resilience and reduced circuit depth, making it practical for current quantum devices. We evaluate the model using transparent and well-documented Natural Language Processing datasets designed for Quantum Computing (QC). Our results demonstrate competitive performance on language tasks, supported by both quantitative metrics that offer interpretable insights into the model's behavior. This work provides a viable pathway toward practical, interpretable quantum language models in the near term.
Transformer now underpins modern AI as its core infrastructure. Its defining capability-dynamically focusing on the most relevant information in complex inputs-is bounded above by the self-attention scoring function. Quantum computing, with its superposition, entanglement, and probabilistic outputs, offers a fundamentally distinct computational framework for exploring beyond the design constraints of classical scoring functions. While quantum attention mechanisms have shown initial promise, existing works remain largely confined to redefining feature similarity measures, leaving the systematic use of parameterized quantum circuits (PQCs) as scoring functions largely unexplored; a substantial portion of existing schemes further rely on purely quantum architectures, precluding effective encoding of high-dimensional image inputs in the Noisy Intermediate-Scale Quantum era. We propose the Quantum Parameterized Self-Attention Network (QPSAN), implementing the self-attention scoring function via PQCs with only 5 trainable quantum parameters per layer. QPSAN computes query-key attention scores through quantum state encoding and joint measurement, yielding naturally bounded outputs without the explicit scaling of classical dot-product attention. We further establish a theoretical framework of the mathematical properties of this scoring function, demonstrating its potential to capture complex nonlinear query-key interactions, and quantifying the structural constraints of the encoding layer via effective degrees of freedom analysis. Experiments on four vision datasets show that QPSAN significantly outperforms the Vision Transformer (ViT) baseline, with the quantum representational advantage amplifying as data complexity increases. Ablation studies indicate that the performance gains may stem from the structural inductive bias of the quantum circuit rather than from parameter scale.