The attention mechanism forms the foundation of many modern AI models such as the Transformer. In one subclass of problems where attention is used, inputs and outputs are bound to the probability simplex so that all outputs sum to one. In this setting, softmax attention admits an exact, component-by-component quantum realization. Attention scores are Hadamard-test statistics on block-encoded projections of amplitude-encoded inputs. The exponential softmax is the interior of a cosine-squared family generated by Born-rule measurement under an exact bijection, whose boundary expresses sparse attention with exact zeros at finite parameter values. The softmax temperature is a repetition count where post-selected measurement rounds realize discretized inverse temperature exactly. Value aggregation is a deterministic column-loading channel that dilates the column-stochastic value matrix. The gated residual is the preparation angle of a single ancilla, with the additive identity at a mixing angle of $π$/2. Every learnable parameter is a rotation-gate angle. The composed layer is exact in the infinite-shot limit with one measure-and-reload step per attention score; a fully-coherent variant is $ε$-approximate via quantum singular value transformation in the infinite depth limit. The algebraic core is machine-checked in Lean 4.
The attention mechanism forms the foundation of many modern AI models such as the Transformer. In one subclass of problems where attention is used, inputs and outputs are bound to the probability simplex so that all outputs sum to one. In this setting, softmax attention admits an exact, component-by-component quantum realization. Attention scores are Hadamard-test statistics on block-encoded projections of amplitude-encoded inputs. The exponential softmax is the interior of a cosine-squared family generated by Born-rule measurement under an exact bijection, whose boundary expresses sparse attention with exact zeros at finite parameter values. The softmax temperature is a repetition count where post-selected measurement rounds realize discretized inverse temperature exactly. Value aggregation is a deterministic column-loading channel that dilates the column-stochastic value matrix. The gated residual is the preparation angle of a single ancilla, with the additive identity at a mixing angle of π/2. Every learnable parameter is a rotation-gate angle. The composed layer is exact in the infinite-shot limit with one measure-and-reload step per attention score; a fully-coherent variant is ε-approximate via quantum singular value transformation in the infinite depth limit. The algebraic core is machine-checked in Lean 4.
This publication introduces the Jordan Spectral Transformer (JST), a quantum-inspired neural architecture in which conventional softmax attention is replaced by density-matrix evolution followed by Born-rule measurement. The central Jordan step is ρ′=φ−1UρU†+φ−2ρ,\rho'=\varphi^{-1}U\rho U^\dagger+\varphi^{-2}\rho,ρ′=φ−1UρU†+φ−2ρ, where ρ\rhoρ is a density matrix, U=exp(−i dt H)U=\exp(-i\,dt\,H)U=exp(−idtH) is the unitary evolution generated by a Hermitian Hamiltonian HHH, and φ=(1+5)/2\varphi=(1+\sqrt{5})/2φ=(1+5)/2 is the golden ratio. The coefficients satisfy φ−1+φ−2=1,\varphi^{-1}+\varphi^{-2}=1,φ−1+φ−2=1, making the update a convex combination. The paper identifies this as the unique positive weighting pair (a,b)(a,b)(a,b) satisfying a+b=1a+b=1a+b=1 and b=a2b=a^2b=a2. The proposed JST pipeline consists of: Sovereign Piper Encoder conversion of an input signal into a density matrix Repeated Jordan evolution Born-rule probability measurement Signal reconstruction Append-only WORM receipt generation The full forward path is expressed as: x→ρ0→ρN→{pj}→x^→(x^,receipt).x \rightarrow \rho_0 \rightarrow \rho_N \rightarrow \{p_j\} \rightarrow \hat{x} \rightarrow (\hat{x},\text{receipt}).x→ρ0→ρN→{pj}→x^→(x^,receipt). The publication derives an adjoint-gradient expression for learning and presents a tight-frame encoding model. It distinguishes between a linear encoder, for which an exact Parseval-style round trip is claimed, and a normalized softmax variant used to produce probability-simplex outputs but which does not generally preserve exact reconstruction. A central algebraic result states that every fixed point of the Jordan operator commutes with the unitary evolution: T(ρ∗)=ρ∗⟹[U,ρ∗]=0.T(\rho^*)=\rho^* \quad\Longrightarrow\quad [U,\rho^*]=0.T(ρ∗)=ρ∗⟹[U,ρ∗]=0. This follows from the golden-ratio identity 1−φ−2=φ−11-\varphi^{-2}=\varphi^{-1}1−φ−2=φ−1. The supplied Lean 4 artifact verifies the scalar cancellation model; the paper explicitly notes that a complete matrix-level Mathlib formalization remains separate work. The paper also studies convergence. For a fixed unitary UUU, the displayed norm estimate establishes that the Jordan operator is non-expansive with Lipschitz constant 1, not automatically a strict Banach contraction. Strict contraction is instead discussed under averaging or signal-dependent unitary evolution. The included Lean code checks the decreasing scalar sequence (φ−1)N(\varphi^{-1})^N(φ−1)N using IEEE-754 floating-point arithmetic and does not by itself prove density-matrix operator contraction. PAR-011 is presented as part of The Parr Papers: Sovereign Convergence, with prior-art attribution to Ahmad Ali Parr, SnapKitty Collective, and the Bel Esprit D’Accord Irrevocable Trust.
We derive two attention operators from generalized statistical entropies. Kaniadakis entropy yields an exact full-support normalization whose weights and low-score sensitivities decay algebraically, rather than exponentially as in Softmax or by exact truncation as in entmax. Classical Abe entropy yields an implicit reciprocal-symmetric operator. With q=eε, the involution q↔q−1 removes every odd correction about Softmax; we obtain the normalized second- and fourth-order terms, including the deformation of the normalization multiplier. These stationary laws follow from a Fisher-metric Lagrangian on the probability simplex, whose Shannon sector recovers scaled dot-product Softmax. We also give a tangent-gradient test for deciding whether changing the entropy changes the attention profile or only its scale. Rényi and two-parameter Sharma--Mittal entropies retain the Tsallis--entmax inverse-gradient shape, but their global moments make the effective temperature input dependent when the external temperature is fixed. Distinguishing profile-shape equivalence from fixed-parameter operator equivalence separates new normalization shapes from adaptive rescalings and organizes the operators by support, tail behavior, and realization complexity.
Given the query, key and value matrices Q,K,V∈Rn×d, the attention module is defined as Att(Q,K,V)=D−1AV where A=exp(QK⊤/d) with exp(⋅) applied entrywise, D=diag(A1n). The attention module is the backbone of modern transformers and large language models, but explicitly forming the softmax matrix D−1A incurs Ω(n2) time, motivating numerous approximation schemes that reduce runtime to O(nd) via sparsity or low-rank factorization.
We propose a quantum data structure that approximates any row of Att(Q,K,V) using only row queries to Q,K,V. Our algorithm preprocesses these matrices in O(ε−1n0.5(sλ2.5+sλ1.5d+α0.5d)) time, where ε is the target accuracy, sλ is the λ-statistical dimension of the exponential kernel defined by Q and K, and α measures the row distortion of V that is at most d/srank(V), the stable rank of V. Each row query can be answered in O(sλ2+sλd) time.
To our knowledge, this is the first quantum data structure that approximates rows of the attention matrix in sublinear time with respect to n. Our approach relies on a quantum Nyström approximation of the exponential kernel, quantum multivariate mean estimation for computing D, and quantum leverage score sampling for the multiplication with V.
Abstract We formalize Quantum Hybrid Modules (QHM) to evaluate the insertion of quantum subroutines into classical AI pipelines with explicit assumptions about data loading, noise, and measurement overhead. The framework comprises three components: (1) a Hadamard Quantum Self-Attention (HQSA) module computing sparse attention via Hadamard-test inner products in O(nn/k⋅logd/ϵ) time assuming quantum random access memory (qRAM); (2) a Quantum Approximate Optimization Algorithm (QAOA) module for combinatorial optimization sub-problems; and (3) a Grover-accelerated neurosymbolic verification loop for quadratically faster constraint checking. We evaluate QHM through 22 experiments spanning statevector and GPU simulation up to 30 qubits, cloud simulators, and five quantum processing unit (QPU) experiments on Rigetti Cepheus and IonQ Forte, including 8--100-qubit superconducting and 8--30-qubit trapped-ion QAOA. Simulations confirm the intended mechanisms: QAOA exceeds the Goemans--Williamson bound on selected 3-regular instances, Hadamard-test attention reproduces classical softmax, and Grover verification reaches a 20,862× query reduction at 30 qubits. Classical, dequantized, and qRAM-free baselines expose current limits: small quantum kernels lag tuned classical baselines, hybrid variational quantum circuit layers add training cost without consistent accuracy gains, and shot-based noisy intermediate-scale quantum (NISQ) attention is limited by systematic bias. Hardware results are depth- and platform-dependent: shallow circuits retain measurable signal, Rigetti QAOA depolarizes to the random baseline and is not recovered by post-hoc mitigation, and IonQ Forte preserves QAOA signal but remains below noiseless performance. We do not claim near-term quantum advantage; we characterize the gap between asymptotic theory and present-day hardware.
Transformer now underpins modern AI as its core infrastructure. Its defining capability-dynamically focusing on the most relevant information in complex inputs-is bounded above by the self-attention scoring function. Quantum computing, with its superposition, entanglement, and probabilistic outputs, offers a fundamentally distinct computational framework for exploring beyond the design constraints of classical scoring functions. While quantum attention mechanisms have shown initial promise, existing works remain largely confined to redefining feature similarity measures, leaving the systematic use of parameterized quantum circuits (PQCs) as scoring functions largely unexplored; a substantial portion of existing schemes further rely on purely quantum architectures, precluding effective encoding of high-dimensional image inputs in the Noisy Intermediate-Scale Quantum era. We propose the Quantum Parameterized Self-Attention Network (QPSAN), implementing the self-attention scoring function via PQCs with only 5 trainable quantum parameters per layer. QPSAN computes query-key attention scores through quantum state encoding and joint measurement, yielding naturally bounded outputs without the explicit scaling of classical dot-product attention. We further establish a theoretical framework of the mathematical properties of this scoring function, demonstrating its potential to capture complex nonlinear query-key interactions, and quantifying the structural constraints of the encoding layer via effective degrees of freedom analysis. Experiments on four vision datasets show that QPSAN significantly outperforms the Vision Transformer (ViT) baseline, with the quantum representational advantage amplifying as data complexity increases. Ablation studies indicate that the performance gains may stem from the structural inductive bias of the quantum circuit rather than from parameter scale.
Powerful generative artificial intelligence from large language models (LLMs) harnesses extensive computational resources for inference. In this work, we investigate the transformer architecture, a key component of these models, under the lens of fault-tolerant quantum computing. We develop quantum subroutines to construct the building blocks in the transformer, including the self-attention, residual connection with layer normalization, and feed-forward network. As an important subroutine, we show how to efficiently implement the Hadamard product and element-wise functions of matrices on quantum computers. Our algorithm prepares an amplitude encoding of the transformer output, which can be measured for prediction or use in the next layer. We find that the matrix norm of the input sequence plays a dominant role in the quantum complexity. With numerical experiments on open-source LLMs, including for bio-informatics applications, we demonstrate the potential of a quantum speedup for transformer inference in practical regimes.