Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2403.02871

Quantum Mixed-State Self-Attention Network

Fu Chen, Qinglin Zhao, Feng Li, Chuangtao Chen, Lin, Yangbin, Jianhong Lin

Published Mar 5, 2024• 1 citation
arXivHTMLPDF

Abstract

Attention mechanisms have revolutionized natural language processing. Combining them with quantum computing aims to further advance this technology. This paper introduces a novel Quantum Mixed-State Self-Attention Network (QMSAN) for natural language processing tasks. Our model leverages quantum computing principles to enhance the effectiveness of self-attention mechanisms. QMSAN uses a quantum attention mechanism based on mixed state, allowing for direct similarity estimation between queries and keys in the quantum domain. This approach leads to more effective attention coefficient calculations. We also propose an innovative quantum positional encoding scheme, implemented through fixed quantum gates within the circuit, improving the model's ability to capture sequence information without additional qubit resources. In numerical experiments of text classification tasks on public datasets, QMSAN outperforms Quantum Self-Attention Neural Network (QSANN). Furthermore, we demonstrate QMSAN's robustness in different quantum noise environments, highlighting its potential for near-term quantum devices.

DOI
10.48550/arxiv.2403.02871

Related papers

A Quantum Self-Attention Neural Network Model on Quantum Circuits

Jin Yan Zheng, Qing Zhong Gao, Daoyi Dong +2 more

This article proposes a quantum self-attention neural network (QSAN) model that is implementable on parameterized quantum circuits (PQCs), providing a novel avenue for addressing natural language processing (NLP) tasks via quantum computing. The QSAN architecture comprises four blocks: the data preprocessing block, the quantum encoding block, the model design block, and the network optimization block. Through these blocks, classical text data are initially preprocessed and encoded into quantum states. The rich semantic features and intricate relationships within these quantum states are then captured and learned by a quantum self-attention layer and a quantum fully connected layer. Finally, the model performance is enhanced by optimizing the network parameters. Simulation results on publicly available topic classification and sentiment analysis datasets demonstrate that the QSAN model outperforms state-of-the-art baselines. As a limitation of the current study, these evaluations are restricted to a maximum sentence length of eight words due to simulation and hardware constraints. Moreover, the effectiveness, scalability, trainability, and robustness of the proposed model are comprehensively evaluated, highlighting its superior performance and potential for advanced NLP tasks.

IEEE Transactions on Cybernetics•Jun 29, 2026•1 citation

Design of a Quantum Self-Attention Neural Network on Quantum Circuits

Jin Yan Zheng, Qing Zhong Gao, Zibo Miao

This paper proposes a quantum self-attention neural network (QSAN) model that can be deployed on quantum circuits, providing a novel avenue to processing text classification tasks in natural language processing (NLP). The QSAN framework is established by integrating four basic blocks: the data preprocessing block, the quantum encoding block, the model design block, and the network optimization block. Simulation results demonstrate remarkable convergence and accuracy on various text classification datasets. In particular, the proposed QSAN surpasses the existing state-of-the-art quantum NLP (QNLP) model in terms of test accuracy.

Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics•Oct 1, 2023•7 citations

A Survey of Classical and Quantum Sequence Models

I-Chi Chen, Harshdeep Singh, V L Anukruti +2 more

Our primary objective is to conduct a brief survey of various classical and quantum neural net sequence models, which includes self-attention and recurrent neural networks, with a focus on recent quantum approaches proposed to work with near-term quantum devices, while exploring some basic enhancements for these quantum models. We re-implement a key representative set of these existing methods, adapting an image classification approach using quantum self-attention to create a quantum hybrid transformer that works for text and image classification, and applying quantum self-attention and quantum recurrent neural networks to natural language processing tasks. We also explore different encoding techniques and introduce positional encoding into quantum self-attention neural networks leading to improved accuracy and faster convergence in text and image classification experiments. This paper also performs a comparative analysis of classical self-attention models and their quantum counterparts, helping shed light on the differences in these models and their performance.

arXiv:2312.10242•International Conference on Communication Systems & Networks (COMSNETS)•Jan 3, 2024•5 citations

Quantum-Inspired Self-Attention in a Large Language Model

Nikita Kuznetsov, Niyaz Ismagilov, Ernesto Campos

Recent advances in Natural Language Processing have been predominantly driven by transformer-based architectures, which rely heavily on self-attention mechanisms to model relationships between tokens in a sequence. Similarly, the field of Quantum Natural Language Processing, which seeks to leverage quantum principles to address challenges in language understanding and generation tasks, has seen the recent development of quantum self-attention mechanisms. We propose a classical quantum-inspired self-attention (QISA) mechanism and integrate it into the full autoregressive language modeling pipeline of GPT-1. To the best of our knowledge, this is the first integration of this kind, as previous quantum self-attention mechanisms have been primarily tested on text classification. In our experiments, QISA achieves better performance when compared to standard self-attention on the metrics character error rate ($15.5\times$ better), word error rate ($4.7 \times $) and cross-entropy loss ($13 \times$). This is achieved while only requiring a $ 2.6\times$ longer inference time.

cs.CLCross-listedarXiv:2603.03318•Feb 10, 2026•0 citations

Training Quantum Self-Attention Model in Near-Term Quantum Computer

Jieguang He, Yin Kan, Cheng Xue

Quantum computing is rapidly emerging as a revolutionary computing paradigm with significant development potential. Variational quantum algorithms, especially those employing variational quantum circuits, have demonstrated notable advantages in domains including finance, chemistry, and machine learning. However, the practical deployment of these algorithms on Noisy Intermediate-Scale Quantum (NISQ) devices faces challenges due to quantum noise, state collapse, and limited qubits numbers. The real-world performance of these algorithms remains largely unexplored. This paper addresses this gap by proposing and implementing a novel method for training quantum self-attention model for text classification directly on quantum computing chips. This initiative marks the first instance of employing such models on actual quantum hardware. Our approach optimizes quantum circuit designs to mitigate the impact of quantum noise and introduces parallel training strategies to enhance efficiency. Our experiments, conducted on the “Wukong” superconducting 72-qubit quantum computer, demonstrate that our models can be effectively trained on real quantum hardware. The results from the actual quantum chips slightly outperformed those from simulators, confirming the practical feasibility of applying quantum self-attention model to natural language processing tasks. These results validate the effectiveness of our approach and provide valuable insights for the future deployment of quantum machine learning models.

International Conference on Wireless Communications and Signal Processing (WCSP)•Oct 24, 2024•4 citations

Povm-Based Quantum Self-Attention Neural Network

Jiachun Wei, Zhimin He, Chuangtao Chen +2 more

Quantum Self-Attention Neural Network (QSANN) has demonstrated remarkable potential. However, it is limited to measuring only partial information from qubits, thereby restricting its information extraction capability. Moreover, the measurement process requires multiple measurements for each qubit, leading to inefficiency in utilizing the feature space. To address these challenges, this paper introduces a novel approach called Positive Operator-Valued Measure based Quantum Self-Attention Neural Network (POVM-QSANN). It leverages POVM operators as observables, enabling the extraction of comprehensive QKV (Query-Key-Value) feature vectors from each qubit. This innovative mechanism significantly enhances information extraction capability and efficiently utilizes the feature space. The experimental results underscore the considerable advancements of POVM-QSANN on MC and RP datasets. Particularly noteworthy is its remarkable accuracy increase of 9.68% compared to QSANN on the RP dataset.

International Conference on Wavelet Analysis and Pattern Recognition•Jul 9, 2023•6 citations
Attention mechanisms have revolutionized natural language processing. Combining them with quantum computing aims to further advance this technology. This paper introduces a novel Quantum Mixed-State Self-Attention Network (QMSAN) for natural language processing tasks. Our model leverages quantum computing principles to enhance the effectiveness of self-attention mechanisms. QMSAN uses a quantum attention mechanism based on mixed state, allowing for direct similarity estimation between queries and keys in the quantum domain. This approach leads to more effective attention coefficient calculations. We also propose an innovative quantum positional encoding scheme, implemented through fixed quantum gates within the circuit, improving the model's ability to capture sequence information without additional qubit resources. In numerical experiments of text classification tasks on public datasets, QMSAN outperforms Quantum Self-Attention Neural Network (QSANN). Furthermore, we demonstrate QMSAN's robustness in different quantum noise environments, highlighting its potential for near-term quantum devices.
This article proposes a quantum self-attention neural network (QSAN) model that is implementable on parameterized quantum circuits (PQCs), providing a novel avenue for addressing natural language processing (NLP) tasks via quantum computing. The QSAN architecture comprises four blocks: the data preprocessing block, the quantum encoding block, the model design block, and the network optimization block. Through these blocks, classical text data are initially preprocessed and encoded into quantum states. The rich semantic features and intricate relationships within these quantum states are then captured and learned by a quantum self-attention layer and a quantum fully connected layer. Finally, the model performance is enhanced by optimizing the network parameters. Simulation results on publicly available topic classification and sentiment analysis datasets demonstrate that the QSAN model outperforms state-of-the-art baselines. As a limitation of the current study, these evaluations are restricted to a maximum sentence length of eight words due to simulation and hardware constraints. Moreover, the effectiveness, scalability, trainability, and robustness of the proposed model are comprehensively evaluated, highlighting its superior performance and potential for advanced NLP tasks.
This paper proposes a quantum self-attention neural network (QSAN) model that can be deployed on quantum circuits, providing a novel avenue to processing text classification tasks in natural language processing (NLP). The QSAN framework is established by integrating four basic blocks: the data preprocessing block, the quantum encoding block, the model design block, and the network optimization block. Simulation results demonstrate remarkable convergence and accuracy on various text classification datasets. In particular, the proposed QSAN surpasses the existing state-of-the-art quantum NLP (QNLP) model in terms of test accuracy.
Our primary objective is to conduct a brief survey of various classical and quantum neural net sequence models, which includes self-attention and recurrent neural networks, with a focus on recent quantum approaches proposed to work with near-term quantum devices, while exploring some basic enhancements for these quantum models. We re-implement a key representative set of these existing methods, adapting an image classification approach using quantum self-attention to create a quantum hybrid transformer that works for text and image classification, and applying quantum self-attention and quantum recurrent neural networks to natural language processing tasks. We also explore different encoding techniques and introduce positional encoding into quantum self-attention neural networks leading to improved accuracy and faster convergence in text and image classification experiments. This paper also performs a comparative analysis of classical self-attention models and their quantum counterparts, helping shed light on the differences in these models and their performance.
Recent advances in Natural Language Processing have been predominantly driven by transformer-based architectures, which rely heavily on self-attention mechanisms to model relationships between tokens in a sequence. Similarly, the field of Quantum Natural Language Processing, which seeks to leverage quantum principles to address challenges in language understanding and generation tasks, has seen the recent development of quantum self-attention mechanisms. We propose a classical quantum-inspired self-attention (QISA) mechanism and integrate it into the full autoregressive language modeling pipeline of GPT-1. To the best of our knowledge, this is the first integration of this kind, as previous quantum self-attention mechanisms have been primarily tested on text classification. In our experiments, QISA achieves better performance when compared to standard self-attention on the metrics character error rate (15.5×15.5\times15.5× better), word error rate ($4.7 \times $) and cross-entropy loss (13×13 \times13×). This is achieved while only requiring a $ 2.6\times$ longer inference time.
Quantum computing is rapidly emerging as a revolutionary computing paradigm with significant development potential. Variational quantum algorithms, especially those employing variational quantum circuits, have demonstrated notable advantages in domains including finance, chemistry, and machine learning. However, the practical deployment of these algorithms on Noisy Intermediate-Scale Quantum (NISQ) devices faces challenges due to quantum noise, state collapse, and limited qubits numbers. The real-world performance of these algorithms remains largely unexplored. This paper addresses this gap by proposing and implementing a novel method for training quantum self-attention model for text classification directly on quantum computing chips. This initiative marks the first instance of employing such models on actual quantum hardware. Our approach optimizes quantum circuit designs to mitigate the impact of quantum noise and introduces parallel training strategies to enhance efficiency. Our experiments, conducted on the “Wukong” superconducting 72-qubit quantum computer, demonstrate that our models can be effectively trained on real quantum hardware. The results from the actual quantum chips slightly outperformed those from simulators, confirming the practical feasibility of applying quantum self-attention model to natural language processing tasks. These results validate the effectiveness of our approach and provide valuable insights for the future deployment of quantum machine learning models.
Quantum Self-Attention Neural Network (QSANN) has demonstrated remarkable potential. However, it is limited to measuring only partial information from qubits, thereby restricting its information extraction capability. Moreover, the measurement process requires multiple measurements for each qubit, leading to inefficiency in utilizing the feature space. To address these challenges, this paper introduces a novel approach called Positive Operator-Valued Measure based Quantum Self-Attention Neural Network (POVM-QSANN). It leverages POVM operators as observables, enabling the extraction of comprehensive QKV (Query-Key-Value) feature vectors from each qubit. This innovative mechanism significantly enhances information extraction capability and efficiently utilizes the feature space. The experimental results underscore the considerable advancements of POVM-QSANN on MC and RP datasets. Particularly noteworthy is its remarkable accuracy increase of 9.68% compared to QSANN on the RP dataset.