Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics

Design of a Quantum Self-Attention Neural Network on Quantum Circuits

Jin Yan Zheng, Qing Zhong Gao, Zibo Miao

Published Oct 1, 2023• 7 citations
Publisher

Abstract

This paper proposes a quantum self-attention neural network (QSAN) model that can be deployed on quantum circuits, providing a novel avenue to processing text classification tasks in natural language processing (NLP). The QSAN framework is established by integrating four basic blocks: the data preprocessing block, the quantum encoding block, the model design block, and the network optimization block. Simulation results demonstrate remarkable convergence and accuracy on various text classification datasets. In particular, the proposed QSAN surpasses the existing state-of-the-art quantum NLP (QNLP) model in terms of test accuracy.

DOI
10.1109/smc53992.2023.10393989

Related papers

A Quantum Self-Attention Neural Network Model on Quantum Circuits

Jin Yan Zheng, Qing Zhong Gao, Daoyi Dong +2 more

This article proposes a quantum self-attention neural network (QSAN) model that is implementable on parameterized quantum circuits (PQCs), providing a novel avenue for addressing natural language processing (NLP) tasks via quantum computing. The QSAN architecture comprises four blocks: the data preprocessing block, the quantum encoding block, the model design block, and the network optimization block. Through these blocks, classical text data are initially preprocessed and encoded into quantum states. The rich semantic features and intricate relationships within these quantum states are then captured and learned by a quantum self-attention layer and a quantum fully connected layer. Finally, the model performance is enhanced by optimizing the network parameters. Simulation results on publicly available topic classification and sentiment analysis datasets demonstrate that the QSAN model outperforms state-of-the-art baselines. As a limitation of the current study, these evaluations are restricted to a maximum sentence length of eight words due to simulation and hardware constraints. Moreover, the effectiveness, scalability, trainability, and robustness of the proposed model are comprehensively evaluated, highlighting its superior performance and potential for advanced NLP tasks.

IEEE Transactions on Cybernetics•Jun 29, 2026•1 citation

Quantum Mixed-State Self-Attention Network

Fu Chen, Qinglin Zhao, Feng Li +3 more

Attention mechanisms have revolutionized natural language processing. Combining them with quantum computing aims to further advance this technology. This paper introduces a novel Quantum Mixed-State Self-Attention Network (QMSAN) for natural language processing tasks. Our model leverages quantum computing principles to enhance the effectiveness of self-attention mechanisms. QMSAN uses a quantum attention mechanism based on mixed state, allowing for direct similarity estimation between queries and keys in the quantum domain. This approach leads to more effective attention coefficient calculations. We also propose an innovative quantum positional encoding scheme, implemented through fixed quantum gates within the circuit, improving the model's ability to capture sequence information without additional qubit resources. In numerical experiments of text classification tasks on public datasets, QMSAN outperforms Quantum Self-Attention Neural Network (QSANN). Furthermore, we demonstrate QMSAN's robustness in different quantum noise environments, highlighting its potential for near-term quantum devices.

arXiv:2403.02871•Mar 5, 2024•1 citation

Self-supervised pre-trained neural network for quantum natural language processing

Ben Yao, Prayag Tiwari, Qiuchi Li

Quantum computing models have propelled advances in many application domains. However, in the field of natural language processing (NLP), quantum computing models are limited in representation capacity due to the high linearity of the underlying quantum computing architecture. This work attempts to address this limitation by leveraging the concept of self-supervised pre-training, a paradigm that has been propelling the rocketing development of NLP, to increase the power of quantum NLP models on the representation level. Specifically, we present a self-supervised pre-training approach to train quantum encodings of sentences, and fine-tune quantum circuits for downstream tasks on its basis. Experiments show that pre-trained mechanism brings remarkable improvement over end-to-end pure quantum models, yielding meaningful prediction results on a variety of downstream text classification datasets.

Neural Networks•Dec 6, 2024•17 citations

A Survey of Classical and Quantum Sequence Models

I-Chi Chen, Harshdeep Singh, V L Anukruti +2 more

Our primary objective is to conduct a brief survey of various classical and quantum neural net sequence models, which includes self-attention and recurrent neural networks, with a focus on recent quantum approaches proposed to work with near-term quantum devices, while exploring some basic enhancements for these quantum models. We re-implement a key representative set of these existing methods, adapting an image classification approach using quantum self-attention to create a quantum hybrid transformer that works for text and image classification, and applying quantum self-attention and quantum recurrent neural networks to natural language processing tasks. We also explore different encoding techniques and introduce positional encoding into quantum self-attention neural networks leading to improved accuracy and faster convergence in text and image classification experiments. This paper also performs a comparative analysis of classical self-attention models and their quantum counterparts, helping shed light on the differences in these models and their performance.

arXiv:2312.10242•International Conference on Communication Systems & Networks (COMSNETS)•Jan 3, 2024•5 citations

QSAN: A Near-Term Achievable Quantum Self-Attention Network

Jinjing Shi, Ren-Xin Zhao, Wenxuan Wang +2 more

Self-attention mechanism (SAM) is good at capturing the intrinsic connection between features to dramatically boost the performance of machine learning models. Nevertheless, the capability of SAM is not equipped with many current quantum machine learning (QML) models, thus confining their expansion on massive high-dimensional quantum data. To address the above problems, a quantum SAM (QSAM) consisting of a quantum logic similarity (QLS)-based quantum bit self-attention score matrix (QBSASM) is introduced to augment the data representation of SAM exponentially. According to QSAM, the framework and quantum circuits of a one-step achievable quantum self-attention network (QSAN) are designed to consider measurement times compression fully. Moreover, a prototype of quantum coordinates is presented during the design process to describe the mathematical relationship between the output bits and the control bits to facilitate the programming. Ultimately, MNIST binary classification experiments on the PennyLane platform and comparisons with cutting-edge QML models demonstrate QSAN converges about $1.7\times $ and $2.3\times $ faster than hardware-efficient ansatz and quantum approximate optimization algorithm (QAOA) ansatz, respectively, with similar parameter configurations and 100% prediction accuracy, which indicates that it has a better learning capability. In the CIFAR-10 classification experiments, QSAN achieves high prediction accuracy at a small scale relative to classical machine learning models. Predictably, QSAN elevates the efficiency of QML models and lays the foundation for future quantum computers to perform machine learning on massive amounts of data while promoting the advancement of quantum computer vision and other fields.

arXiv:2207.07563•IEEE Transactions on Neural Networks and Learning Systems•Dec 16, 2024•41 citations

Quantum-Inspired Self-Attention in a Large Language Model

Nikita Kuznetsov, Niyaz Ismagilov, Ernesto Campos

Recent advances in Natural Language Processing have been predominantly driven by transformer-based architectures, which rely heavily on self-attention mechanisms to model relationships between tokens in a sequence. Similarly, the field of Quantum Natural Language Processing, which seeks to leverage quantum principles to address challenges in language understanding and generation tasks, has seen the recent development of quantum self-attention mechanisms. We propose a classical quantum-inspired self-attention (QISA) mechanism and integrate it into the full autoregressive language modeling pipeline of GPT-1. To the best of our knowledge, this is the first integration of this kind, as previous quantum self-attention mechanisms have been primarily tested on text classification. In our experiments, QISA achieves better performance when compared to standard self-attention on the metrics character error rate ($15.5\times$ better), word error rate ($4.7 \times $) and cross-entropy loss ($13 \times$). This is achieved while only requiring a $ 2.6\times$ longer inference time.

cs.CLCross-listedarXiv:2603.03318•Feb 10, 2026•0 citations
This paper proposes a quantum self-attention neural network (QSAN) model that can be deployed on quantum circuits, providing a novel avenue to processing text classification tasks in natural language processing (NLP). The QSAN framework is established by integrating four basic blocks: the data preprocessing block, the quantum encoding block, the model design block, and the network optimization block. Simulation results demonstrate remarkable convergence and accuracy on various text classification datasets. In particular, the proposed QSAN surpasses the existing state-of-the-art quantum NLP (QNLP) model in terms of test accuracy.
This article proposes a quantum self-attention neural network (QSAN) model that is implementable on parameterized quantum circuits (PQCs), providing a novel avenue for addressing natural language processing (NLP) tasks via quantum computing. The QSAN architecture comprises four blocks: the data preprocessing block, the quantum encoding block, the model design block, and the network optimization block. Through these blocks, classical text data are initially preprocessed and encoded into quantum states. The rich semantic features and intricate relationships within these quantum states are then captured and learned by a quantum self-attention layer and a quantum fully connected layer. Finally, the model performance is enhanced by optimizing the network parameters. Simulation results on publicly available topic classification and sentiment analysis datasets demonstrate that the QSAN model outperforms state-of-the-art baselines. As a limitation of the current study, these evaluations are restricted to a maximum sentence length of eight words due to simulation and hardware constraints. Moreover, the effectiveness, scalability, trainability, and robustness of the proposed model are comprehensively evaluated, highlighting its superior performance and potential for advanced NLP tasks.
Attention mechanisms have revolutionized natural language processing. Combining them with quantum computing aims to further advance this technology. This paper introduces a novel Quantum Mixed-State Self-Attention Network (QMSAN) for natural language processing tasks. Our model leverages quantum computing principles to enhance the effectiveness of self-attention mechanisms. QMSAN uses a quantum attention mechanism based on mixed state, allowing for direct similarity estimation between queries and keys in the quantum domain. This approach leads to more effective attention coefficient calculations. We also propose an innovative quantum positional encoding scheme, implemented through fixed quantum gates within the circuit, improving the model's ability to capture sequence information without additional qubit resources. In numerical experiments of text classification tasks on public datasets, QMSAN outperforms Quantum Self-Attention Neural Network (QSANN). Furthermore, we demonstrate QMSAN's robustness in different quantum noise environments, highlighting its potential for near-term quantum devices.
Quantum computing models have propelled advances in many application domains. However, in the field of natural language processing (NLP), quantum computing models are limited in representation capacity due to the high linearity of the underlying quantum computing architecture. This work attempts to address this limitation by leveraging the concept of self-supervised pre-training, a paradigm that has been propelling the rocketing development of NLP, to increase the power of quantum NLP models on the representation level. Specifically, we present a self-supervised pre-training approach to train quantum encodings of sentences, and fine-tune quantum circuits for downstream tasks on its basis. Experiments show that pre-trained mechanism brings remarkable improvement over end-to-end pure quantum models, yielding meaningful prediction results on a variety of downstream text classification datasets.
Our primary objective is to conduct a brief survey of various classical and quantum neural net sequence models, which includes self-attention and recurrent neural networks, with a focus on recent quantum approaches proposed to work with near-term quantum devices, while exploring some basic enhancements for these quantum models. We re-implement a key representative set of these existing methods, adapting an image classification approach using quantum self-attention to create a quantum hybrid transformer that works for text and image classification, and applying quantum self-attention and quantum recurrent neural networks to natural language processing tasks. We also explore different encoding techniques and introduce positional encoding into quantum self-attention neural networks leading to improved accuracy and faster convergence in text and image classification experiments. This paper also performs a comparative analysis of classical self-attention models and their quantum counterparts, helping shed light on the differences in these models and their performance.
Self-attention mechanism (SAM) is good at capturing the intrinsic connection between features to dramatically boost the performance of machine learning models. Nevertheless, the capability of SAM is not equipped with many current quantum machine learning (QML) models, thus confining their expansion on massive high-dimensional quantum data. To address the above problems, a quantum SAM (QSAM) consisting of a quantum logic similarity (QLS)-based quantum bit self-attention score matrix (QBSASM) is introduced to augment the data representation of SAM exponentially. According to QSAM, the framework and quantum circuits of a one-step achievable quantum self-attention network (QSAN) are designed to consider measurement times compression fully. Moreover, a prototype of quantum coordinates is presented during the design process to describe the mathematical relationship between the output bits and the control bits to facilitate the programming. Ultimately, MNIST binary classification experiments on the PennyLane platform and comparisons with cutting-edge QML models demonstrate QSAN converges about $1.7\times $ and $2.3\times $ faster than hardware-efficient ansatz and quantum approximate optimization algorithm (QAOA) ansatz, respectively, with similar parameter configurations and 100% prediction accuracy, which indicates that it has a better learning capability. In the CIFAR-10 classification experiments, QSAN achieves high prediction accuracy at a small scale relative to classical machine learning models. Predictably, QSAN elevates the efficiency of QML models and lays the foundation for future quantum computers to perform machine learning on massive amounts of data while promoting the advancement of quantum computer vision and other fields.
Recent advances in Natural Language Processing have been predominantly driven by transformer-based architectures, which rely heavily on self-attention mechanisms to model relationships between tokens in a sequence. Similarly, the field of Quantum Natural Language Processing, which seeks to leverage quantum principles to address challenges in language understanding and generation tasks, has seen the recent development of quantum self-attention mechanisms. We propose a classical quantum-inspired self-attention (QISA) mechanism and integrate it into the full autoregressive language modeling pipeline of GPT-1. To the best of our knowledge, this is the first integration of this kind, as previous quantum self-attention mechanisms have been primarily tested on text classification. In our experiments, QISA achieves better performance when compared to standard self-attention on the metrics character error rate (15.5×15.5\times15.5× better), word error rate ($4.7 \times $) and cross-entropy loss (13×13 \times13×). This is achieved while only requiring a $ 2.6\times$ longer inference time.