Quantum Self-Attention (QSA) is a drop-in replacement for softmax attention that lifts the real probability simplex into complex projective space CP^{n-1}. By preserving phase information, QSA enables constructive and destructive interference, native subtractive logic, and antisymmetric relational encoding—capabilities impossible for softmax. This paper presents the full mathematical derivation, production-ready PyTorch implementations, and empirical benchmarks including:- 99.83% accuracy on harmonic interference (vs 97.80% softmax)- Stable training on causal arithmetic where softmax crashes- First large-scale graft of QSA into a frozen 1.3B model (DeepSeek-Coder) using NecroGraft, reducing training loss from 70.4 to 2.06 The architecture is strictly more expressive than affine-softmax models (Ω(N²) separation).
Quantum Self-Attention (QSA) is a drop-in replacement for softmax attention that lifts the real probability simplex into complex projective space CP^{n-1}. By preserving phase information, QSA enables constructive and destructive interference, native subtractive logic, and antisymmetric relational encoding—capabilities impossible for softmax. This paper presents the full mathematical derivation, production-ready PyTorch implementations, and empirical benchmarks including:- 99.83% accuracy on harmonic interference (vs 97.80% softmax)- Stable training on causal arithmetic where softmax crashes- First large-scale graft of QSA into a frozen 1.3B model (DeepSeek-Coder) using NecroGraft, reducing training loss from 70.4 to 2.06 The architecture is strictly more expressive than affine-softmax models (Ω(N²) separation).
We propose a variational quantum implementation of self-attention (QSA)-the core operation in transformers and large language models-which predicts future elements of a sequence by forming overlap-weighted combinations of past data. At variance with previous approaches, our QSA realizes the required nonlinearity through interference of state overlaps and a degree-k polynomial kernel, and estimates a loss based on Rényi-1/2 entropic functionals via two observables' expectation values, avoiding the decoding of amplitude-encoded predictions into classical probabilities. QSA also accommodates a constrained, trainable data-embedding tying state overlaps to data-level similarities. Its dominant end-to-end training complexity scales as O(μ−1k2Td), versus O(Tdk+1) of the fairest classical comparison, with μ a training signal; we show numerically that this allows a complexity advantage in the regime where sequence length T dominates the embedding size d. In simulations, our QSA-based quantum transformer learns sequence prediction on classical data and on many-body transverse-field Ising trajectories-establishing trainable attention as a practical primitive for quantum dynamical modeling.
The automated design of parameterized quantum circuits for variational algorithms in the NISQ era faces a fundamental limitation, as conventional differentiable architecture search relies on classical models that fail to adequately represent quantum gate interactions under hardware noise. We introduce the Quantum-Based Self-Attention for Differentiable Quantum Architecture Search (QBSA-DQAS), a meta-learning framework featuring quantum-based self-attention and hardware-aware multi-objective search. The framework employs a two-stage quantum self-attention module that computes contextual dependencies by mapping architectural parameters through parameterized quantum circuits, replacing classical similarity metrics with quantum-derived attention scores, then applies position-wise quantum transformations for feature enrichment. Architecture search is guided by a task-agnostic multi-objective function jointly optimizing noisy expressibility and Probability of Successful Trials (PST). A post-search optimization stage applies gate commutation, fusion, and elimination to reduce circuit complexity. Experimental validation demonstrates superior performance on VQE tasks and large-scale Wireless Sensor Networks. For VQE on H2, QBSA-DQAS achieves 0.9 accuracy compared to 0.89 for standard DQAS. Post-search optimization reduces discovered circuit complexity by up to 44% in gate count and 47% in depth without accuracy degradation. The framework maintains robust performance across three molecules and five IBM quantum hardware noise models. For WSN routing, discovered circuits achieve 8.6% energy reduction versus QAOA and 40.7% versus classical greedy methods, establishing the effectiveness of quantum-native architecture search for NISQ applications.
Abstract A recurring weakness in quantum machine learning (QML) is that reported ‘quantum advantages’ are seldom tested against a capacity-matched classical control, leaving it unclear whether a gain comes from the quantum substrate or from the architectural change that accompanies it. Our primary contribution is methodological: a protocol for attributing such gains honestly—a capacity-matched classical bottleneck of identical parameter budget, transparent reporting of where quantum does not help, and validation on real quantum hardware—which we develop and apply through a concrete case study. That case study is Quantum Adaptive Self-Attention (QASA), a hybrid Transformer that replaces the value projection of a single encoder layer with a 36-parameter parameterized quantum circuit (PQC), keeping all other layers classical. Across nine synthetic benchmarks and the real-world ETTh1 dataset, QASA improves on a full-capacity classical Transformer for chaotic and trend-dominated signals. To ask whether this is a genuinely quantum effect, we introduce a control rarely applied in QML—a capacity-matched classical bottleneck with the same parameter budget—and find that it matches the PQC on the error metrics. The gain is therefore attributable to the low-rank value-projection bottleneck (an architectural parsimony principle), not to quantumness; adding further quantum layers only degrades performance and trainability. We accordingly position the quantum layer not as a source of accuracy advantage but as a competitive instantiation of this principle: its low-rank compression onto the signal’s intrinsic dimensionality is matched by a classical bottleneck, so the gain is architectural rather than quantum. The quantum layer’s distinguishing features are physical—a high circuit entanglement (Meyer–Wallach Q = 0.981 with only 27 CNOTs, which we report as a circuit-level property) and, most concretely, noisy intermediate-scale quantum deployability, which we verify by executing the trained model on a real IBM Quantum processor (one-step prediction within 7.5 % of noiseless simulation on the quantum-favored task and 25 % , with overlapping error bars, on a classical-favored control; no error mitigation). Relative to other quantum sequence models (quantum long short-term memory (QLSTM), QnnFormer), QASA is moreover the most resource-efficient of the three: it reaches competitive accuracy with the fewest quantum parameters (36 vs 90–128) and the strongest entanglement-per-gate ( Q = 0.981 with 27 CNOTs vs QLSTM’s 56), and retains trainable gradients that deeper circuits lose. We argue that capacity-matched baselines and honest reporting of where quantum does not help are prerequisites for credible quantum-machine-learning claims.
Transformer now underpins modern AI as its core infrastructure. Its defining capability-dynamically focusing on the most relevant information in complex inputs-is bounded above by the self-attention scoring function. Quantum computing, with its superposition, entanglement, and probabilistic outputs, offers a fundamentally distinct computational framework for exploring beyond the design constraints of classical scoring functions. While quantum attention mechanisms have shown initial promise, existing works remain largely confined to redefining feature similarity measures, leaving the systematic use of parameterized quantum circuits (PQCs) as scoring functions largely unexplored; a substantial portion of existing schemes further rely on purely quantum architectures, precluding effective encoding of high-dimensional image inputs in the Noisy Intermediate-Scale Quantum era. We propose the Quantum Parameterized Self-Attention Network (QPSAN), implementing the self-attention scoring function via PQCs with only 5 trainable quantum parameters per layer. QPSAN computes query-key attention scores through quantum state encoding and joint measurement, yielding naturally bounded outputs without the explicit scaling of classical dot-product attention. We further establish a theoretical framework of the mathematical properties of this scoring function, demonstrating its potential to capture complex nonlinear query-key interactions, and quantifying the structural constraints of the encoding layer via effective degrees of freedom analysis. Experiments on four vision datasets show that QPSAN significantly outperforms the Vision Transformer (ViT) baseline, with the quantum representational advantage amplifying as data complexity increases. Ablation studies indicate that the performance gains may stem from the structural inductive bias of the quantum circuit rather than from parameter scale.
We formulate automated market maker (AMM) rebalancing as a binary detection problem and study a hybrid quantum–classical self-attention block, Quantum Adaptive Self-Attention (QASA). QASA constructs quantum queries/keys/values via variational quantum circuits (VQCs) and applies standard softmax attention over Pauli-Z expectation vectors, yielding a drop-in attention module for financial time-series decision making. Using daily data for BTCUSDC over Jan-2024–Jan-2025 with a 70/15/15 time-series split, we compare QASA against classical ensembles, a transformer, and pure quantum baselines under Return, Sharpe, and Max Drawdown. The QASA-Sequence variant attains the best single-model risk-adjusted performance (13.99% return; Sharpe 1.76), while hybrid models average 11.2% return (vs. 9.8% classical; 4.4% pure quantum), indicating a favorable performance–stability–cost trade-off.
Abstract Hybrid quantum-classical neural networks commonly convert a parameterized quantum circuit state into classical features through expectation-value measurements. Although compact, this readout may discard useful information contained in the circuit state. This study investigates whether full computational-basis probability readout improves a compact hybrid quantum self-attention model for lightweight medical image classification. A token-embedded hybrid quantum vision transformer, TE-HQViT-Probs, was evaluated on PneumoniaMNIST, BreastMNIST, and OrganAMNIST. Query and key projections were computed classically, whereas the value path used amplitude embedding, a four-qubit, two-layer parameterized quantum circuit, and full-probability readout. Experiments used ten random seeds and compared full-probability readout with expectation-value readout, classical attention controls, a frozen-random probability-readout control, and standard classical vision baselines. Additional analyses included a controlled readout ladder, paired Wilcoxon tests, two one-sided tests (TOST) for practical equivalence, readout compression, frozen readout-only probes, and a mixed-state depolarizing-channel stress test. In the main benchmark, TE-HQViT-Probs improved mean area under the receiver operating characteristic curve (AUC) over the expectation-value variant by 0.0062, 0.0497, and 0.0236 on PneumoniaMNIST, BreastMNIST, and OrganAMNIST, respectively. In the controlled readout-ladder experiment, Full-Probs improved over Expval-Z by 0.0052, 0.0847, and 0.0266. The effects were nevertheless dataset-dependent: PneumoniaMNIST showed only a small improvement, BreastMNIST remained below a classical softmax-attention control, and frozen-random probability readout performed similarly to trainable probability readout in several settings. These results indicate that full-probability readout can preserve class-relevant information lost by simple expectation-value readout in small-qubit simulation experiments. The conclusion is methodological rather than computational: readout design is an important architectural factor in hybrid quantum attention models. The results do not establish quantum advantage, hardware speedup, or clinical deployment readiness.