Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2207.07563 · IEEE Transactions on Neural Networks and Learning Systems

QSAN: A Near-Term Achievable Quantum Self-Attention Network

Jinjing Shi, Ren-Xin Zhao, Wenxuan Wang, Shichao Zhang, Xuelong Li

Published Dec 16, 2024• 41 citations
arXivHTMLPDF

Abstract

Self-attention mechanism (SAM) is good at capturing the intrinsic connection between features to dramatically boost the performance of machine learning models. Nevertheless, the capability of SAM is not equipped with many current quantum machine learning (QML) models, thus confining their expansion on massive high-dimensional quantum data. To address the above problems, a quantum SAM (QSAM) consisting of a quantum logic similarity (QLS)-based quantum bit self-attention score matrix (QBSASM) is introduced to augment the data representation of SAM exponentially. According to QSAM, the framework and quantum circuits of a one-step achievable quantum self-attention network (QSAN) are designed to consider measurement times compression fully. Moreover, a prototype of quantum coordinates is presented during the design process to describe the mathematical relationship between the output bits and the control bits to facilitate the programming. Ultimately, MNIST binary classification experiments on the PennyLane platform and comparisons with cutting-edge QML models demonstrate QSAN converges about $1.7\times $ and $2.3\times $ faster than hardware-efficient ansatz and quantum approximate optimization algorithm (QAOA) ansatz, respectively, with similar parameter configurations and 100% prediction accuracy, which indicates that it has a better learning capability. In the CIFAR-10 classification experiments, QSAN achieves high prediction accuracy at a small scale relative to classical machine learning models. Predictably, QSAN elevates the efficiency of QML models and lays the foundation for future quantum computers to perform machine learning on massive amounts of data while promoting the advancement of quantum computer vision and other fields.

DOI
10.1109/tnnls.2024.3504828

Related papers

QKSAN: A Quantum Kernel Self-Attention Network

Ren-Xin Zhao, Jinjing Shi, Xuelong Li

The Self-Attention Mechanism (SAM) excels at distilling important information from the interior of data to improve the computational efficiency of models. Nevertheless, many Quantum Machine Learning (QML) models lack the ability to distinguish the intrinsic connections of information like SAM, which limits their effectiveness on massive high-dimensional quantum data. To tackle the above issue, a Quantum Kernel Self-Attention Mechanism (QKSAM) is introduced to combine the data representation merit of Quantum Kernel Methods (QKM) with the efficient information extraction capability of SAM. Further, a Quantum Kernel Self-Attention Network (QKSAN) framework is proposed based on QKSAM, which ingeniously incorporates the Deferred Measurement Principle (DMP) and conditional measurement techniques to release half of quantum resources by mid-circuit measurement, thereby bolstering both feasibility and adaptability. Simultaneously, the Quantum Kernel Self-Attention Score (QKSAS) with an exponentially large characterization space is spawned to accommodate more information and determine the measurement conditions. Eventually, four QKSAN sub-models are deployed on PennyLane and IBM Qiskit platforms to perform binary classification on MNIST and Fashion MNIST, where the QKSAS tests and correlation assessments between noise immunity and learning ability are executed on the best-performing sub-model. The paramount experimental finding is that the QKSAN subclasses possess the potential learning advantage of acquiring impressive accuracies exceeding 98.05% with far fewer parameters than classical machine learning models. Predictably, QKSAN lays the foundation for future quantum computers to perform machine learning on massive amounts of data while driving advances in areas such as quantum computer vision.

arXiv:2308.13422•IEEE Transactions on Pattern Analysis and Machine Intelligence•Jul 29, 2024•80 citations

A Quantum Self-Attention Neural Network Model on Quantum Circuits

Jin Yan Zheng, Qing Zhong Gao, Daoyi Dong +2 more

This article proposes a quantum self-attention neural network (QSAN) model that is implementable on parameterized quantum circuits (PQCs), providing a novel avenue for addressing natural language processing (NLP) tasks via quantum computing. The QSAN architecture comprises four blocks: the data preprocessing block, the quantum encoding block, the model design block, and the network optimization block. Through these blocks, classical text data are initially preprocessed and encoded into quantum states. The rich semantic features and intricate relationships within these quantum states are then captured and learned by a quantum self-attention layer and a quantum fully connected layer. Finally, the model performance is enhanced by optimizing the network parameters. Simulation results on publicly available topic classification and sentiment analysis datasets demonstrate that the QSAN model outperforms state-of-the-art baselines. As a limitation of the current study, these evaluations are restricted to a maximum sentence length of eight words due to simulation and hardware constraints. Moreover, the effectiveness, scalability, trainability, and robustness of the proposed model are comprehensively evaluated, highlighting its superior performance and potential for advanced NLP tasks.

IEEE Transactions on Cybernetics•Jun 29, 2026•1 citation

Design of a Quantum Self-Attention Neural Network on Quantum Circuits

Jin Yan Zheng, Qing Zhong Gao, Zibo Miao

This paper proposes a quantum self-attention neural network (QSAN) model that can be deployed on quantum circuits, providing a novel avenue to processing text classification tasks in natural language processing (NLP). The QSAN framework is established by integrating four basic blocks: the data preprocessing block, the quantum encoding block, the model design block, and the network optimization block. Simulation results demonstrate remarkable convergence and accuracy on various text classification datasets. In particular, the proposed QSAN surpasses the existing state-of-the-art quantum NLP (QNLP) model in terms of test accuracy.

Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics•Oct 1, 2023•7 citations

Quantum Parameterized Self-Attention Network for Image Classification

Wenwei Zhang, Jintao Wang, Tianyu Ye +1 more

Transformer now underpins modern AI as its core infrastructure. Its defining capability-dynamically focusing on the most relevant information in complex inputs-is bounded above by the self-attention scoring function. Quantum computing, with its superposition, entanglement, and probabilistic outputs, offers a fundamentally distinct computational framework for exploring beyond the design constraints of classical scoring functions. While quantum attention mechanisms have shown initial promise, existing works remain largely confined to redefining feature similarity measures, leaving the systematic use of parameterized quantum circuits (PQCs) as scoring functions largely unexplored; a substantial portion of existing schemes further rely on purely quantum architectures, precluding effective encoding of high-dimensional image inputs in the Noisy Intermediate-Scale Quantum era. We propose the Quantum Parameterized Self-Attention Network (QPSAN), implementing the self-attention scoring function via PQCs with only 5 trainable quantum parameters per layer. QPSAN computes query-key attention scores through quantum state encoding and joint measurement, yielding naturally bounded outputs without the explicit scaling of classical dot-product attention. We further establish a theoretical framework of the mathematical properties of this scoring function, demonstrating its potential to capture complex nonlinear query-key interactions, and quantifying the structural constraints of the encoding layer via effective degrees of freedom analysis. Experiments on four vision datasets show that QPSAN significantly outperforms the Vision Transformer (ViT) baseline, with the quantum representational advantage amplifying as data complexity increases. Ablation studies indicate that the performance gains may stem from the structural inductive bias of the quantum circuit rather than from parameter scale.

quant-pharXiv:2605.25365•May 26, 2026•0 citations

GQHAN: A Grover-inspired Quantum Hard Attention Network

Ren-Xin Zhao, Jinjing Shi, Xuelong Li

Numerous current Quantum Machine Learning (QML) models exhibit an inadequacy in discerning the significance of quantum data, resulting in diminished efficacy when handling extensive quantum datasets. Hard Attention Mechanism (HAM), anticipated to efficiently tackle the above QML bottlenecks, encounters the substantial challenge of non-differentiability, consequently constraining its extensive applicability. In response to the dilemma of HAM and QML, a Grover-inspired Quantum Hard Attention Mechanism (GQHAM) consisting of a Flexible Oracle (FO) and an Adaptive Diffusion Operator (ADO) is proposed. Notably, the FO is designed to surmount the non-differentiable issue by executing the activation or masking of Discrete Primitives (DPs) with Flexible Control (FC) to weave various discrete destinies. Based on this, such discrete choice can be visualized with a specially defined Quantum Hard Attention Score (QHAS). Furthermore, a trainable ADO is devised to boost the generality and flexibility of GQHAM. At last, a Grover-inspired Quantum Hard Attention Network (GQHAN) based on QGHAM is constructed on PennyLane platform for Fashion MNIST binary classification. Experimental findings demonstrate that GQHAN adeptly surmounts the non-differentiability hurdle, surpassing the efficacy of extant quantum soft self-attention mechanisms in accuracies and learning ability. In noise experiments, GQHAN is robuster to bit-flip noise in accuracy and amplitude damping noise in learning performance. Predictably, the proposal of GQHAN enriches the Quantum Attention Mechanism (QAM), lays the foundation for future quantum computers to process large-scale data, and promotes the development of quantum computer vision.

arXiv:2401.14089•Jan 25, 2024•6 citations

QAHAN: A Quantum Annealing Hard Attention Network

Ren-Xin Zhao

Hard Attention Mechanisms (HAMs) effectively filter essential information discretely and significantly boost the performance of machine learning models on large datasets. Nevertheless, they confront the challenge of non-differentiability, which raises the risk of convergence to a local optimum. Quantum Annealing (QA) is expected to solve the above dilemma. We propose a Quantum Annealing Hard Attention Mechanism (QAHAM) for faster convergence to the global optimum without the need to compute gradients by exploiting the quantum tunneling effect. Based on the above theory, we construct a Quantum Annealing Hard Attention Network (QAHAN) on D-Wave and Pytorch platforms for MNIST and CIFAR-10 multi-classification. Experimental results indicate that the QAHAN converges faster, exhibits smoother accuracy and loss curves, and demonstrates superior noise robustness compared to two traditional HAMs. Predictably, our scheme accelerates the convergence between the fields of quantum algorithms and machine learning, while advancing the field of quantum machine vision.

arXiv:2412.20930•Dec 30, 2024•3 citations
Self-attention mechanism (SAM) is good at capturing the intrinsic connection between features to dramatically boost the performance of machine learning models. Nevertheless, the capability of SAM is not equipped with many current quantum machine learning (QML) models, thus confining their expansion on massive high-dimensional quantum data. To address the above problems, a quantum SAM (QSAM) consisting of a quantum logic similarity (QLS)-based quantum bit self-attention score matrix (QBSASM) is introduced to augment the data representation of SAM exponentially. According to QSAM, the framework and quantum circuits of a one-step achievable quantum self-attention network (QSAN) are designed to consider measurement times compression fully. Moreover, a prototype of quantum coordinates is presented during the design process to describe the mathematical relationship between the output bits and the control bits to facilitate the programming. Ultimately, MNIST binary classification experiments on the PennyLane platform and comparisons with cutting-edge QML models demonstrate QSAN converges about 1.7×1.7\times 1.7× and 2.3×2.3\times 2.3× faster than hardware-efficient ansatz and quantum approximate optimization algorithm (QAOA) ansatz, respectively, with similar parameter configurations and 100% prediction accuracy, which indicates that it has a better learning capability. In the CIFAR-10 classification experiments, QSAN achieves high prediction accuracy at a small scale relative to classical machine learning models. Predictably, QSAN elevates the efficiency of QML models and lays the foundation for future quantum computers to perform machine learning on massive amounts of data while promoting the advancement of quantum computer vision and other fields.
The Self-Attention Mechanism (SAM) excels at distilling important information from the interior of data to improve the computational efficiency of models. Nevertheless, many Quantum Machine Learning (QML) models lack the ability to distinguish the intrinsic connections of information like SAM, which limits their effectiveness on massive high-dimensional quantum data. To tackle the above issue, a Quantum Kernel Self-Attention Mechanism (QKSAM) is introduced to combine the data representation merit of Quantum Kernel Methods (QKM) with the efficient information extraction capability of SAM. Further, a Quantum Kernel Self-Attention Network (QKSAN) framework is proposed based on QKSAM, which ingeniously incorporates the Deferred Measurement Principle (DMP) and conditional measurement techniques to release half of quantum resources by mid-circuit measurement, thereby bolstering both feasibility and adaptability. Simultaneously, the Quantum Kernel Self-Attention Score (QKSAS) with an exponentially large characterization space is spawned to accommodate more information and determine the measurement conditions. Eventually, four QKSAN sub-models are deployed on PennyLane and IBM Qiskit platforms to perform binary classification on MNIST and Fashion MNIST, where the QKSAS tests and correlation assessments between noise immunity and learning ability are executed on the best-performing sub-model. The paramount experimental finding is that the QKSAN subclasses possess the potential learning advantage of acquiring impressive accuracies exceeding 98.05% with far fewer parameters than classical machine learning models. Predictably, QKSAN lays the foundation for future quantum computers to perform machine learning on massive amounts of data while driving advances in areas such as quantum computer vision.
This article proposes a quantum self-attention neural network (QSAN) model that is implementable on parameterized quantum circuits (PQCs), providing a novel avenue for addressing natural language processing (NLP) tasks via quantum computing. The QSAN architecture comprises four blocks: the data preprocessing block, the quantum encoding block, the model design block, and the network optimization block. Through these blocks, classical text data are initially preprocessed and encoded into quantum states. The rich semantic features and intricate relationships within these quantum states are then captured and learned by a quantum self-attention layer and a quantum fully connected layer. Finally, the model performance is enhanced by optimizing the network parameters. Simulation results on publicly available topic classification and sentiment analysis datasets demonstrate that the QSAN model outperforms state-of-the-art baselines. As a limitation of the current study, these evaluations are restricted to a maximum sentence length of eight words due to simulation and hardware constraints. Moreover, the effectiveness, scalability, trainability, and robustness of the proposed model are comprehensively evaluated, highlighting its superior performance and potential for advanced NLP tasks.
This paper proposes a quantum self-attention neural network (QSAN) model that can be deployed on quantum circuits, providing a novel avenue to processing text classification tasks in natural language processing (NLP). The QSAN framework is established by integrating four basic blocks: the data preprocessing block, the quantum encoding block, the model design block, and the network optimization block. Simulation results demonstrate remarkable convergence and accuracy on various text classification datasets. In particular, the proposed QSAN surpasses the existing state-of-the-art quantum NLP (QNLP) model in terms of test accuracy.
Transformer now underpins modern AI as its core infrastructure. Its defining capability-dynamically focusing on the most relevant information in complex inputs-is bounded above by the self-attention scoring function. Quantum computing, with its superposition, entanglement, and probabilistic outputs, offers a fundamentally distinct computational framework for exploring beyond the design constraints of classical scoring functions. While quantum attention mechanisms have shown initial promise, existing works remain largely confined to redefining feature similarity measures, leaving the systematic use of parameterized quantum circuits (PQCs) as scoring functions largely unexplored; a substantial portion of existing schemes further rely on purely quantum architectures, precluding effective encoding of high-dimensional image inputs in the Noisy Intermediate-Scale Quantum era. We propose the Quantum Parameterized Self-Attention Network (QPSAN), implementing the self-attention scoring function via PQCs with only 5 trainable quantum parameters per layer. QPSAN computes query-key attention scores through quantum state encoding and joint measurement, yielding naturally bounded outputs without the explicit scaling of classical dot-product attention. We further establish a theoretical framework of the mathematical properties of this scoring function, demonstrating its potential to capture complex nonlinear query-key interactions, and quantifying the structural constraints of the encoding layer via effective degrees of freedom analysis. Experiments on four vision datasets show that QPSAN significantly outperforms the Vision Transformer (ViT) baseline, with the quantum representational advantage amplifying as data complexity increases. Ablation studies indicate that the performance gains may stem from the structural inductive bias of the quantum circuit rather than from parameter scale.
Numerous current Quantum Machine Learning (QML) models exhibit an inadequacy in discerning the significance of quantum data, resulting in diminished efficacy when handling extensive quantum datasets. Hard Attention Mechanism (HAM), anticipated to efficiently tackle the above QML bottlenecks, encounters the substantial challenge of non-differentiability, consequently constraining its extensive applicability. In response to the dilemma of HAM and QML, a Grover-inspired Quantum Hard Attention Mechanism (GQHAM) consisting of a Flexible Oracle (FO) and an Adaptive Diffusion Operator (ADO) is proposed. Notably, the FO is designed to surmount the non-differentiable issue by executing the activation or masking of Discrete Primitives (DPs) with Flexible Control (FC) to weave various discrete destinies. Based on this, such discrete choice can be visualized with a specially defined Quantum Hard Attention Score (QHAS). Furthermore, a trainable ADO is devised to boost the generality and flexibility of GQHAM. At last, a Grover-inspired Quantum Hard Attention Network (GQHAN) based on QGHAM is constructed on PennyLane platform for Fashion MNIST binary classification. Experimental findings demonstrate that GQHAN adeptly surmounts the non-differentiability hurdle, surpassing the efficacy of extant quantum soft self-attention mechanisms in accuracies and learning ability. In noise experiments, GQHAN is robuster to bit-flip noise in accuracy and amplitude damping noise in learning performance. Predictably, the proposal of GQHAN enriches the Quantum Attention Mechanism (QAM), lays the foundation for future quantum computers to process large-scale data, and promotes the development of quantum computer vision.
Hard Attention Mechanisms (HAMs) effectively filter essential information discretely and significantly boost the performance of machine learning models on large datasets. Nevertheless, they confront the challenge of non-differentiability, which raises the risk of convergence to a local optimum. Quantum Annealing (QA) is expected to solve the above dilemma. We propose a Quantum Annealing Hard Attention Mechanism (QAHAM) for faster convergence to the global optimum without the need to compute gradients by exploiting the quantum tunneling effect. Based on the above theory, we construct a Quantum Annealing Hard Attention Network (QAHAN) on D-Wave and Pytorch platforms for MNIST and CIFAR-10 multi-classification. Experimental results indicate that the QAHAN converges faster, exhibits smoother accuracy and loss curves, and demonstrates superior noise robustness compared to two traditional HAMs. Predictably, our scheme accelerates the convergence between the fields of quantum algorithms and machine learning, while advancing the field of quantum machine vision.