Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2602.06699

Quantum Attention by Overlap Interference: Predicting Sequences from Classical and Many-Body Quantum Data

Alessio Pecilli, Matteo Rosati

quant-phcs.CLcs.LGSubmitted Feb 6, 2026• Announced Feb 9, 2026• 0 citations
v1 · Feb 6, 2026v2 · Sep 24, 2026
arXivHTMLPDF

Abstract

We propose a variational quantum implementation of self-attention (QSA), the core operation in transformers and large language models, which predicts future elements of a sequence by forming overlap-weighted combinations of past data. At variance with previous approaches, our QSA realizes the required nonlinearity through interference of state overlaps and returns a Renyi-1/2 cross-entropy loss directly as the expectation value of an observable, avoiding the need to decode amplitude-encoded predictions into classical logits. Furthermore, QSA naturally accommodates a constrained, trainable data-embedding that ties quantum state overlaps to data-level similarities. We find a gate complexity dominant scaling O(T d^2) for QSA, versus O(T^2 d) classically, suggesting an advantage in the practical regime where the sequence length T dominates the embedding size d. In simulations, we show that our QSA-based quantum transformer learns sequence prediction on classical data and on many-body transverse-field Ising quantum trajectories, establishing trainable attention as a practical primitive for quantum dynamical modeling.

Comments
4 + 1 pages, 2 figures

Related papers

Hybrid Encoding-based Quantum Self Attention for Financial Time Series Analysis

S Lokes, Suresh Reddy, Samuel Yen Chi Chen

Self-attention mechanisms, central to Transformer architectures, have become foundational in the development of large language models (LLMs) and various advanced machine learning systems. These mechanisms are pivotal for effectively capturing complex data dependencies and relationships, which is crucial for tasks such as natural language understanding and generation. However, as dimensionality increases, self-attention mechanisms face significant challenges related to computational efficiency and scalability. This paper introduces an encoding-based quantum self-attention mechanism designed to address these issues by embedding classical data into quantum states. This approach facilitates more efficient computation and reduces model complexity. To further enhance performance for real-world applications, particularly in financial time series forecasting, we propose a hybrid model that integrates Long Short-Term Memory (LSTM) networks with quantum self-attention mechanism. Given the constraints of the Noisy Intermediate-Scale Quantum (NISQ) era, this hybrid approach represents a plausible solution, leveraging the strengths of both classical and quantum methods and focusing on achieving faster convergence. Experimental results demonstrate that our approach converges significantly quicker than traditional models, showcasing the advantages of integrating quantum techniques with classical machine learning for practical applications.

IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW)•Apr 6, 2025•1 citation

A Hybrid Transformer Architecture with a Quantized Self-Attention Mechanism Applied to Molecular Generation

Anthony M. Smaldone, Yu Shee, Gregory W. Kyro +4 more

The success of the self-attention mechanism in classical machine learning models has inspired the development of quantum analogs aimed at reducing the computational overhead. Self-attention integrates learnable query and key matrices to calculate attention scores between all pairs of tokens in a sequence. These scores are then multiplied by a learnable value matrix to obtain the output self-attention matrix, enabling the model to effectively capture long-range dependencies within the input sequence. Here, we propose a hybrid quantum-classical self-attention mechanism as part of a transformer decoder, the architecture underlying large language models (LLMs). To demonstrate its utility in chemistry, we train this model on the QM9 dataset for conditional generation, using SMILES strings as input, each labeled with a set of physicochemical properties that serve as conditions during inference. Our theoretical analysis shows that the time complexity of the query-key dot product is reduced from O ( n 2 d ) in a classical model to O ( n 2 log ⁡ d ) in our quantum model, where n and d represent the sequence length and the embedding dimension, respectively. We perform simulations using NVIDIA’s CUDA-Q platform, which is designed for efficient GPU scalability. This work provides a promising avenue for quantum-enhanced natural language processing (NLP).

arXiv:2502.19214•Journal of Chemical Theory and Computation•May 7, 2025•16 citations

Quantum Adaptive Self-Attention for quantum Transformer models

Chi‐Sheng Chen, En-Jui Kuo

Abstract A recurring weakness in quantum machine learning (QML) is that reported ‘quantum advantages’ are seldom tested against a capacity-matched classical control, leaving it unclear whether a gain comes from the quantum substrate or from the architectural change that accompanies it. Our primary contribution is methodological: a protocol for attributing such gains honestly—a capacity-matched classical bottleneck of identical parameter budget, transparent reporting of where quantum does not help, and validation on real quantum hardware—which we develop and apply through a concrete case study. That case study is Quantum Adaptive Self-Attention (QASA), a hybrid Transformer that replaces the value projection of a single encoder layer with a 36-parameter parameterized quantum circuit (PQC), keeping all other layers classical. Across nine synthetic benchmarks and the real-world ETTh1 dataset, QASA improves on a full-capacity classical Transformer for chaotic and trend-dominated signals. To ask whether this is a genuinely quantum effect, we introduce a control rarely applied in QML—a capacity-matched classical bottleneck with the same parameter budget—and find that it matches the PQC on the error metrics. The gain is therefore attributable to the low-rank value-projection bottleneck (an architectural parsimony principle), not to quantumness; adding further quantum layers only degrades performance and trainability. We accordingly position the quantum layer not as a source of accuracy advantage but as a competitive instantiation of this principle: its low-rank compression onto the signal’s intrinsic dimensionality is matched by a classical bottleneck, so the gain is architectural rather than quantum. The quantum layer’s distinguishing features are physical—a high circuit entanglement (Meyer–Wallach Q = 0.981 with only 27 CNOTs, which we report as a circuit-level property) and, most concretely, noisy intermediate-scale quantum deployability, which we verify by executing the trained model on a real IBM Quantum processor (one-step prediction within 7.5 % of noiseless simulation on the quantum-favored task and 25 % , with overlapping error bars, on a classical-favored control; no error mitigation). Relative to other quantum sequence models (quantum long short-term memory (QLSTM), QnnFormer), QASA is moreover the most resource-efficient of the three: it reaches competitive accuracy with the fewest quantum parameters (36 vs 90–128) and the strongest entanglement-per-gate ( Q = 0.981 with 27 CNOTs vs QLSTM’s 56), and retains trainable gradients that deeper circuits lose. We argue that capacity-matched baselines and honest reporting of where quantum does not help are prerequisites for credible quantum-machine-learning claims.

arXiv:2504.05336•Quantum Science and Technology•Jul 15, 2026•2 citations

QNet: A Quantum-Native Sequence Encoder Architecture

Wei Day, Haosheng Chen, Min-Te Sun

This work proposes QNet, a novel sequence encoder model that entirely inferences on the quantum computer using a minimum number of qubits. Let n and$d$represent the length of the sequence and the embedding size, respectively. The dot-product attention mechanism requires a time complexity of$O(n^{2}\cdot d)$, while QNet has merely$O(n+d)$quantum circuit depth. In addition, we introduce ResQNet, a quantum-classical hybrid model composed of several QNet blocks linked by residual connections, as an isomorph Transformer Encoder. We evaluated our work on various natural language processing tasks, including text classification, rating score prediction, and named entity recognition. Our models exhibit compelling performance over classical state-of-the-art models with a thousand times fewer parameters. In summary, this work investigates the advantage of machine learning on near-term quantum computers in sequential data by experimenting with natural language processing tasks.

arXiv:2210.17262•IEEE International Conference on Quantum Computing and Engineering (QCE)•Sep 17, 2023•3 citations

Quantum Self-Attention (QSA): Complex Projective Attention with Constructive and Destructive Interference

Heylel Yaka

Quantum Self-Attention (QSA) is a drop-in replacement for softmax attention that lifts the real probability simplex into complex projective space CP^{n-1}. By preserving phase information, QSA enables constructive and destructive interference, native subtractive logic, and antisymmetric relational encoding—capabilities impossible for softmax. This paper presents the full mathematical derivation, production-ready PyTorch implementations, and empirical benchmarks including:- 99.83% accuracy on harmonic interference (vs 97.80% softmax)- Stable training on causal arithmetic where softmax crashes- First large-scale graft of QSA into a frozen 1.3B model (DeepSeek-Coder) using NecroGraft, reducing training loss from 70.4 to 2.06 The architecture is strictly more expressive than affine-softmax models (Ω(N²) separation).

•Sep 2, 2026•0 citations

Quantum-Inspired Self-Attention in a Large Language Model

Nikita Kuznetsov, Niyaz Ismagilov, Ernesto Campos

Recent advances in Natural Language Processing have been predominantly driven by transformer-based architectures, which rely heavily on self-attention mechanisms to model relationships between tokens in a sequence. Similarly, the field of Quantum Natural Language Processing, which seeks to leverage quantum principles to address challenges in language understanding and generation tasks, has seen the recent development of quantum self-attention mechanisms. We propose a classical quantum-inspired self-attention (QISA) mechanism and integrate it into the full autoregressive language modeling pipeline of GPT-1. To the best of our knowledge, this is the first integration of this kind, as previous quantum self-attention mechanisms have been primarily tested on text classification. In our experiments, QISA achieves better performance when compared to standard self-attention on the metrics character error rate ($15.5\times$ better), word error rate ($4.7 \times $) and cross-entropy loss ($13 \times$). This is achieved while only requiring a $ 2.6\times$ longer inference time.

cs.CLCross-listedarXiv:2603.03318•Feb 10, 2026•0 citations
We propose a variational quantum implementation of self-attention (QSA), the core operation in transformers and large language models, which predicts future elements of a sequence by forming overlap-weighted combinations of past data. At variance with previous approaches, our QSA realizes the required nonlinearity through interference of state overlaps and returns a Renyi-1/2 cross-entropy loss directly as the expectation value of an observable, avoiding the need to decode amplitude-encoded predictions into classical logits. Furthermore, QSA naturally accommodates a constrained, trainable data-embedding that ties quantum state overlaps to data-level similarities. We find a gate complexity dominant scaling O(T d^2) for QSA, versus O(T^2 d) classically, suggesting an advantage in the practical regime where the sequence length T dominates the embedding size d. In simulations, we show that our QSA-based quantum transformer learns sequence prediction on classical data and on many-body transverse-field Ising quantum trajectories, establishing trainable attention as a practical primitive for quantum dynamical modeling.
Self-attention mechanisms, central to Transformer architectures, have become foundational in the development of large language models (LLMs) and various advanced machine learning systems. These mechanisms are pivotal for effectively capturing complex data dependencies and relationships, which is crucial for tasks such as natural language understanding and generation. However, as dimensionality increases, self-attention mechanisms face significant challenges related to computational efficiency and scalability. This paper introduces an encoding-based quantum self-attention mechanism designed to address these issues by embedding classical data into quantum states. This approach facilitates more efficient computation and reduces model complexity. To further enhance performance for real-world applications, particularly in financial time series forecasting, we propose a hybrid model that integrates Long Short-Term Memory (LSTM) networks with quantum self-attention mechanism. Given the constraints of the Noisy Intermediate-Scale Quantum (NISQ) era, this hybrid approach represents a plausible solution, leveraging the strengths of both classical and quantum methods and focusing on achieving faster convergence. Experimental results demonstrate that our approach converges significantly quicker than traditional models, showcasing the advantages of integrating quantum techniques with classical machine learning for practical applications.
The success of the self-attention mechanism in classical machine learning models has inspired the development of quantum analogs aimed at reducing the computational overhead. Self-attention integrates learnable query and key matrices to calculate attention scores between all pairs of tokens in a sequence. These scores are then multiplied by a learnable value matrix to obtain the output self-attention matrix, enabling the model to effectively capture long-range dependencies within the input sequence. Here, we propose a hybrid quantum-classical self-attention mechanism as part of a transformer decoder, the architecture underlying large language models (LLMs). To demonstrate its utility in chemistry, we train this model on the QM9 dataset for conditional generation, using SMILES strings as input, each labeled with a set of physicochemical properties that serve as conditions during inference. Our theoretical analysis shows that the time complexity of the query-key dot product is reduced from O ( n 2 d ) in a classical model to O ( n 2 log ⁡ d ) in our quantum model, where n and d represent the sequence length and the embedding dimension, respectively. We perform simulations using NVIDIA’s CUDA-Q platform, which is designed for efficient GPU scalability. This work provides a promising avenue for quantum-enhanced natural language processing (NLP).
Abstract A recurring weakness in quantum machine learning (QML) is that reported ‘quantum advantages’ are seldom tested against a capacity-matched classical control, leaving it unclear whether a gain comes from the quantum substrate or from the architectural change that accompanies it. Our primary contribution is methodological: a protocol for attributing such gains honestly—a capacity-matched classical bottleneck of identical parameter budget, transparent reporting of where quantum does not help, and validation on real quantum hardware—which we develop and apply through a concrete case study. That case study is Quantum Adaptive Self-Attention (QASA), a hybrid Transformer that replaces the value projection of a single encoder layer with a 36-parameter parameterized quantum circuit (PQC), keeping all other layers classical. Across nine synthetic benchmarks and the real-world ETTh1 dataset, QASA improves on a full-capacity classical Transformer for chaotic and trend-dominated signals. To ask whether this is a genuinely quantum effect, we introduce a control rarely applied in QML—a capacity-matched classical bottleneck with the same parameter budget—and find that it matches the PQC on the error metrics. The gain is therefore attributable to the low-rank value-projection bottleneck (an architectural parsimony principle), not to quantumness; adding further quantum layers only degrades performance and trainability. We accordingly position the quantum layer not as a source of accuracy advantage but as a competitive instantiation of this principle: its low-rank compression onto the signal’s intrinsic dimensionality is matched by a classical bottleneck, so the gain is architectural rather than quantum. The quantum layer’s distinguishing features are physical—a high circuit entanglement (Meyer–Wallach Q = 0.981 with only 27 CNOTs, which we report as a circuit-level property) and, most concretely, noisy intermediate-scale quantum deployability, which we verify by executing the trained model on a real IBM Quantum processor (one-step prediction within 7.5 % of noiseless simulation on the quantum-favored task and 25 % , with overlapping error bars, on a classical-favored control; no error mitigation). Relative to other quantum sequence models (quantum long short-term memory (QLSTM), QnnFormer), QASA is moreover the most resource-efficient of the three: it reaches competitive accuracy with the fewest quantum parameters (36 vs 90–128) and the strongest entanglement-per-gate ( Q = 0.981 with 27 CNOTs vs QLSTM’s 56), and retains trainable gradients that deeper circuits lose. We argue that capacity-matched baselines and honest reporting of where quantum does not help are prerequisites for credible quantum-machine-learning claims.
This work proposes QNet, a novel sequence encoder model that entirely inferences on the quantum computer using a minimum number of qubits. Let n anddddrepresent the length of the sequence and the embedding size, respectively. The dot-product attention mechanism requires a time complexity ofO(n2⋅d)O(n^{2}\cdot d)O(n2⋅d), while QNet has merelyO(n+d)O(n+d)O(n+d)quantum circuit depth. In addition, we introduce ResQNet, a quantum-classical hybrid model composed of several QNet blocks linked by residual connections, as an isomorph Transformer Encoder. We evaluated our work on various natural language processing tasks, including text classification, rating score prediction, and named entity recognition. Our models exhibit compelling performance over classical state-of-the-art models with a thousand times fewer parameters. In summary, this work investigates the advantage of machine learning on near-term quantum computers in sequential data by experimenting with natural language processing tasks.
Quantum Self-Attention (QSA) is a drop-in replacement for softmax attention that lifts the real probability simplex into complex projective space CP^{n-1}. By preserving phase information, QSA enables constructive and destructive interference, native subtractive logic, and antisymmetric relational encoding—capabilities impossible for softmax. This paper presents the full mathematical derivation, production-ready PyTorch implementations, and empirical benchmarks including:- 99.83% accuracy on harmonic interference (vs 97.80% softmax)- Stable training on causal arithmetic where softmax crashes- First large-scale graft of QSA into a frozen 1.3B model (DeepSeek-Coder) using NecroGraft, reducing training loss from 70.4 to 2.06 The architecture is strictly more expressive than affine-softmax models (Ω(N²) separation).
Recent advances in Natural Language Processing have been predominantly driven by transformer-based architectures, which rely heavily on self-attention mechanisms to model relationships between tokens in a sequence. Similarly, the field of Quantum Natural Language Processing, which seeks to leverage quantum principles to address challenges in language understanding and generation tasks, has seen the recent development of quantum self-attention mechanisms. We propose a classical quantum-inspired self-attention (QISA) mechanism and integrate it into the full autoregressive language modeling pipeline of GPT-1. To the best of our knowledge, this is the first integration of this kind, as previous quantum self-attention mechanisms have been primarily tested on text classification. In our experiments, QISA achieves better performance when compared to standard self-attention on the metrics character error rate (15.5×15.5\times15.5× better), word error rate ($4.7 \times $) and cross-entropy loss (13×13 \times13×). This is achieved while only requiring a $ 2.6\times$ longer inference time.