Our primary objective is to conduct a brief survey of various classical and quantum neural net sequence models, which includes self-attention and recurrent neural networks, with a focus on recent quantum approaches proposed to work with near-term quantum devices, while exploring some basic enhancements for these quantum models. We re-implement a key representative set of these existing methods, adapting an image classification approach using quantum self-attention to create a quantum hybrid transformer that works for text and image classification, and applying quantum self-attention and quantum recurrent neural networks to natural language processing tasks. We also explore different encoding techniques and introduce positional encoding into quantum self-attention neural networks leading to improved accuracy and faster convergence in text and image classification experiments. This paper also performs a comparative analysis of classical self-attention models and their quantum counterparts, helping shed light on the differences in these models and their performance.
Self-attention mechanism (SAM) is good at capturing the intrinsic connection between features to dramatically boost the performance of machine learning models. Nevertheless, the capability of SAM is not equipped with many current quantum machine learning (QML) models, thus confining their expansion on massive high-dimensional quantum data. To address the above problems, a quantum SAM (QSAM) consisting of a quantum logic similarity (QLS)-based quantum bit self-attention score matrix (QBSASM) is introduced to augment the data representation of SAM exponentially. According to QSAM, the framework and quantum circuits of a one-step achievable quantum self-attention network (QSAN) are designed to consider measurement times compression fully. Moreover, a prototype of quantum coordinates is presented during the design process to describe the mathematical relationship between the output bits and the control bits to facilitate the programming. Ultimately, MNIST binary classification experiments on the PennyLane platform and comparisons with cutting-edge QML models demonstrate QSAN converges about $1.7\times $ and $2.3\times $ faster than hardware-efficient ansatz and quantum approximate optimization algorithm (QAOA) ansatz, respectively, with similar parameter configurations and 100% prediction accuracy, which indicates that it has a better learning capability. In the CIFAR-10 classification experiments, QSAN achieves high prediction accuracy at a small scale relative to classical machine learning models. Predictably, QSAN elevates the efficiency of QML models and lays the foundation for future quantum computers to perform machine learning on massive amounts of data while promoting the advancement of quantum computer vision and other fields.
We propose a variational quantum implementation of self-attention (QSA)-the core operation in transformers and large language models-which predicts future elements of a sequence by forming overlap-weighted combinations of past data. At variance with previous approaches, our QSA realizes the required nonlinearity through interference of state overlaps and a degree-
k polynomial kernel, and estimates a loss based on Rényi-
1/2 entropic functionals via two observables' expectation values, avoiding the decoding of amplitude-encoded predictions into classical probabilities. QSA also accommodates a constrained, trainable data-embedding tying state overlaps to data-level similarities. Its dominant end-to-end training complexity scales as
O(μ−1k2Td), versus
O(Tdk+1) of the fairest classical comparison, with
μ a training signal; we show numerically that this allows a complexity advantage in the regime where sequence length
T dominates the embedding size
d. In simulations, our QSA-based quantum transformer learns sequence prediction on classical data and on many-body transverse-field Ising trajectories-establishing trainable attention as a practical primitive for quantum dynamical modeling.