Softmax self-attention costs O(T^2) in the sequence length T because the exponential kernel has no finite-dimensional feature map, so linear-time variants approximate it. We observe that Born-rule attention, whose weights are quantum-measurement probabilities a_kj ∝ |<Q_k, K_j>|^2 over complex queries and keys, is an exact finite kernel with the rank-one density-matrix feature map ρ_x = x x†, since |<Q_k, K_j>|^2 = tr(ρ_Qk ρ_Kj). As the complex degree-two case of polynomial-kernel linear attention it admits an exact O(T d^2) causal recurrence and constant-state decoding (d^2 (d_v + 1) reals per head, with d_v the real value width, independent of T), and it keeps rotary positions exactly. For GPT-2's head geometry (64 real channels paired into d = 32 complex) the feature dimension is d^2 = 1024. Measured on an A100, compiled constant-state decoding runs from 6.4 MB per layer at 0.12 ms per token at any length, against a 3.2 GB FlashAttention-2 cache at 2.26 ms per token at 1M tokens; unfused prefill overtakes FlashAttention-2 between 64k and 128k tokens (11x at 1M), and a fused form (Triton feature lift plus a chunkwise linear-attention kernel) between 16k and 32k (4x at 128k), reaching training parity at 64k. The linear and quadratic forms are two implementations of one operator; whether that operator matches softmax is a separate modeling question, answered here only at small scale: the p = 1 operator bolted into a real-valued GPT trails softmax by 0.12 nats at context 1k and 0.77 at 16k (attention dilution; a decay kept exactly in the d^2 state restores the 0.12 gap at 16k, and p = 2 sharpening with decay recovers 74% of the 1k gap), whereas inside a fully complex residual stream it matches softmax at 1k and keeps a quarter of the real-stream gap at 4k; gated linear attention beats both there. At 128k context on PG-19 the variable that matters is forgetting: plain Born dilutes, Born with a fixed decay carried in its state is 0.32 nats ahead of softmax and within 0.04 of gated linear attention, which stays 1.8x faster. Streamed with their constant state over the PG-19 test partition in 5M-byte windows, two 4M-complex-parameter byte models of the source framework score 1.66 (linear Born with decay) and 1.70 (gated) bits per byte, where a 350M-parameter hierarchical Mamba–Transformer stack built for that window reports 2.45. Preprint, October 2026. Also submitted to arXiv (identifier to be added on announcement). Code: the CNet framework, https://github.com/crasmarum/CNet .
Softmax self-attention costs O(T^2) in the sequence length T because the exponential kernel has no finite-dimensional feature map, so linear-time variants approximate it. We observe that Born-rule attention, whose weights are quantum-measurement probabilities a_kj ∝ |<Q_k, K_j>|^2 over complex queries and keys, is an exact finite kernel with the rank-one density-matrix feature map ρ_x = x x†, since |<Q_k, K_j>|^2 = tr(ρ_Qk ρ_Kj). As the complex degree-two case of polynomial-kernel linear attention it admits an exact O(T d^2) causal recurrence and constant-state decoding (d^2 (d_v + 1) reals per head, with d_v the real value width, independent of T), and it keeps rotary positions exactly. For GPT-2's head geometry (64 real channels paired into d = 32 complex) the feature dimension is d^2 = 1024. Measured on an A100, compiled constant-state decoding runs from 6.4 MB per layer at 0.12 ms per token at any length, against a 3.2 GB FlashAttention-2 cache at 2.26 ms per token at 1M tokens; unfused prefill overtakes FlashAttention-2 between 64k and 128k tokens (11x at 1M), and a fused form (Triton feature lift plus a chunkwise linear-attention kernel) between 16k and 32k (4x at 128k), reaching training parity at 64k. The linear and quadratic forms are two implementations of one operator; whether that operator matches softmax is a separate modeling question, answered here only at small scale: the p = 1 operator bolted into a real-valued GPT trails softmax by 0.12 nats at context 1k and 0.77 at 16k (attention dilution; a decay kept exactly in the d^2 state restores the 0.12 gap at 16k, and p = 2 sharpening with decay recovers 74% of the 1k gap), whereas inside a fully complex residual stream it matches softmax at 1k and keeps a quarter of the real-stream gap at 4k; gated linear attention beats both there. At 128k context on PG-19 the variable that matters is forgetting: plain Born dilutes, Born with a fixed decay carried in its state is 0.32 nats ahead of softmax and within 0.04 of gated linear attention, which stays 1.8x faster. Streamed with their constant state over the PG-19 test partition in 5M-byte windows, two 4M-complex-parameter byte models of the source framework score 1.66 (linear Born with decay) and 1.70 (gated) bits per byte, where a 350M-parameter hierarchical Mamba–Transformer stack built for that window reports 2.45. Preprint, October 2026. Also submitted to arXiv (identifier to be added on announcement). Code: the CNet framework, https://github.com/crasmarum/CNet .
The attention mechanism forms the foundation of many modern AI models such as the Transformer. In one subclass of problems where attention is used, inputs and outputs are bound to the probability simplex so that all outputs sum to one. In this setting, softmax attention admits an exact, component-by-component quantum realization. Attention scores are Hadamard-test statistics on block-encoded projections of amplitude-encoded inputs. The exponential softmax is the interior of a cosine-squared family generated by Born-rule measurement under an exact bijection, whose boundary expresses sparse attention with exact zeros at finite parameter values. The softmax temperature is a repetition count where post-selected measurement rounds realize discretized inverse temperature exactly. Value aggregation is a deterministic column-loading channel that dilates the column-stochastic value matrix. The gated residual is the preparation angle of a single ancilla, with the additive identity at a mixing angle of π/2. Every learnable parameter is a rotation-gate angle. The composed layer is exact in the infinite-shot limit with one measure-and-reload step per attention score; a fully-coherent variant is ε-approximate via quantum singular value transformation in the infinite depth limit. The algebraic core is machine-checked in Lean 4.
Given the query, key and value matrices Q,K,V∈Rn×d, the attention module is defined as Att(Q,K,V)=D−1AV where A=exp(QK⊤/d) with exp(⋅) applied entrywise, D=diag(A1n). The attention module is the backbone of modern transformers and large language models, but explicitly forming the softmax matrix D−1A incurs Ω(n2) time, motivating numerous approximation schemes that reduce runtime to O(nd) via sparsity or low-rank factorization.
We propose a quantum data structure that approximates any row of Att(Q,K,V) using only row queries to Q,K,V. Our algorithm preprocesses these matrices in O(ε−1n0.5(sλ2.5+sλ1.5d+α0.5d)) time, where ε is the target accuracy, sλ is the λ-statistical dimension of the exponential kernel defined by Q and K, and α measures the row distortion of V that is at most d/srank(V), the stable rank of V. Each row query can be answered in O(sλ2+sλd) time.
To our knowledge, this is the first quantum data structure that approximates rows of the attention matrix in sublinear time with respect to n. Our approach relies on a quantum Nyström approximation of the exponential kernel, quantum multivariate mean estimation for computing D, and quantum leverage score sampling for the multiplication with V.
Sequence models are conventionally distinguished by their backbone, the mechanism that routes information across positions, such as attention or recurrence. This paper varies a choice that is prior to the backbone and shared by nearly all current models: the \emph{substrate}, the number system in which the hidden state is represented together with the form of the map from state to prediction. The prevailing substrate is a real-valued state with an affine--softmax readout; we study a complex-valued alternative drawn from the mathematics of quantum theory, in which information is carried by the phases of the state and scores are quadratic Born forms. Prior work proved an idealized version of this substrate representationally stronger than any real model with a linear readout; we ask whether it also trains faster. Relaxing the two properties that block deployment, exact unitarity and the Born vocabulary readout, we instantiate it in the Mamba state-space model and an attention-based Transformer. At 253M parameters, matched to within 0.02% and trained under one fixed protocol on three byte-level corpora, the complex models reach every measured validation loss in approximately one third (state-space) and one half (attention) of the optimization steps of their real counterparts. The two backbones then diverge. Once the learning-rate warmup ends, the state-space advantage continues to widen, from 0.321 to 0.354 bits per character on OpenWebText and from 0.368 to 0.396 on FineWeb, which an artifact of the warmup ramp would not do; the attention advantage instead decays toward zero on every corpus, and is therefore an effect of early training.
This publication introduces the Jordan Spectral Transformer (JST), a quantum-inspired neural architecture in which conventional softmax attention is replaced by density-matrix evolution followed by Born-rule measurement. The central Jordan step is ρ′=φ−1UρU†+φ−2ρ,\rho'=\varphi^{-1}U\rho U^\dagger+\varphi^{-2}\rho,ρ′=φ−1UρU†+φ−2ρ, where ρ\rhoρ is a density matrix, U=exp(−i dt H)U=\exp(-i\,dt\,H)U=exp(−idtH) is the unitary evolution generated by a Hermitian Hamiltonian HHH, and φ=(1+5)/2\varphi=(1+\sqrt{5})/2φ=(1+5)/2 is the golden ratio. The coefficients satisfy φ−1+φ−2=1,\varphi^{-1}+\varphi^{-2}=1,φ−1+φ−2=1, making the update a convex combination. The paper identifies this as the unique positive weighting pair (a,b)(a,b)(a,b) satisfying a+b=1a+b=1a+b=1 and b=a2b=a^2b=a2. The proposed JST pipeline consists of: Sovereign Piper Encoder conversion of an input signal into a density matrix Repeated Jordan evolution Born-rule probability measurement Signal reconstruction Append-only WORM receipt generation The full forward path is expressed as: x→ρ0→ρN→{pj}→x^→(x^,receipt).x \rightarrow \rho_0 \rightarrow \rho_N \rightarrow \{p_j\} \rightarrow \hat{x} \rightarrow (\hat{x},\text{receipt}).x→ρ0→ρN→{pj}→x^→(x^,receipt). The publication derives an adjoint-gradient expression for learning and presents a tight-frame encoding model. It distinguishes between a linear encoder, for which an exact Parseval-style round trip is claimed, and a normalized softmax variant used to produce probability-simplex outputs but which does not generally preserve exact reconstruction. A central algebraic result states that every fixed point of the Jordan operator commutes with the unitary evolution: T(ρ∗)=ρ∗⟹[U,ρ∗]=0.T(\rho^*)=\rho^* \quad\Longrightarrow\quad [U,\rho^*]=0.T(ρ∗)=ρ∗⟹[U,ρ∗]=0. This follows from the golden-ratio identity 1−φ−2=φ−11-\varphi^{-2}=\varphi^{-1}1−φ−2=φ−1. The supplied Lean 4 artifact verifies the scalar cancellation model; the paper explicitly notes that a complete matrix-level Mathlib formalization remains separate work. The paper also studies convergence. For a fixed unitary UUU, the displayed norm estimate establishes that the Jordan operator is non-expansive with Lipschitz constant 1, not automatically a strict Banach contraction. Strict contraction is instead discussed under averaging or signal-dependent unitary evolution. The included Lean code checks the decreasing scalar sequence (φ−1)N(\varphi^{-1})^N(φ−1)N using IEEE-754 floating-point arithmetic and does not by itself prove density-matrix operator contraction. PAR-011 is presented as part of The Parr Papers: Sovereign Convergence, with prior-art attribution to Ahmad Ali Parr, SnapKitty Collective, and the Bel Esprit D’Accord Irrevocable Trust.
Standard dot-product self-attention computes, in a single layer, only pairwise (order-2) interactions between tokens; representing a generic order-k interaction is known to require either super-quadratic resources in one layer or composition across depth. We introduce \textbf{Quantum Higher-Order Attention (QHA)}, a shallow, hardware-realizable quantum attention head that, via data re-uploading and an all-to-all non-Clifford entangler, synthesizes order-k token interactions inside the circuit and exposes them through a local single-qubit read-out. We prove (i) an expressivity separation: any single standard self-attention layer with embedding dimension m, H heads and p-bit precision satisfying mHp=o(N/loglogN) cannot represent the order-k correlation family that one QHA head represents with circuit depth O(logk) (O(k) two-qubit gates); and (ii) a trainability guarantee for its local-design instantiation: with a local read-out and O(logn) depth the gradient variance is Ω(1/poly(n)) (no barren plateau), which we confirm empirically -- while being explicit that the more expressive all-to-all instantiation we benchmark is trained empirically and shows exponentially decaying gradients. Empirically, at a 6.5× smaller parameter budget, QHA generalizes hidden-subset parity of every order k≤6 from disjoint inputs, whereas the larger classical attention head collapses past order~2; consistent with theory, the size of the advantage tracks the target's Fourier degree - largest for parity and shrinking when low-order structure is present. As an application, QHA serves as a compact high-order interaction detector across three domains - genetic epistasis, learning-parity-with-noise, and graph triangle detection - reaching the noise ceiling at the smallest parameter budget where field-standard linear methods fail.
Depth-recurrent transformers scale test-time compute by iterating a core block in latent space before emitting tokens. The geometry of these latent trajectories is a named open problem in the latent reasoning literature (survey arXiv:2604.02029, §6.3). We report a pre-registered experiment on Huginn-0125 (arXiv:2502.05171), a 3.5B-parameter depth-recurrent transformer, measuring the per-head attention power-law exponent Δ of the recurrent core at recurrence counts r ∈ {1, 2, 4, 8, 16, 32}. For natural-text inputs the median exponent decreases monotonically with recurrence (Spearman ρ = −0.94), from 0.29 at r=1 to 0.239 at r=32 — converging onto the Sachdev–Ye–Kitaev (SYK) q=4 conformal value Δ = 1/4 — while the count of SYK-near heads grows monotonically (ρ = +0.77). Random-token inputs show an architecture-driven convergence with a transient at Δ ≈ 1/2 (the q=2 prethermal plateau previously observed in training time) but no monotone SYK-near growth. A randomized-weights control freezes completely: Δ_med constant to the sixth decimal across all r, zero SYK-near heads. Inference-time recurrence therefore acts as renormalization-group flow toward the conformal fixed point, and the flow is a property of the trained model, not the iteration procedure. Together with prior results on architectural depth (10.5281/zenodo.19225996) and training time, this completes a three-axis triangulation: the conformal fixed point is an attractor of iterative attending. The published orbit/spiral phenomenology of recurrent-depth latent trajectories acquires a candidate theory — the spiral is flow near an infrared fixed point — and the geometric benefit of additional recurrence saturates when the flow arrives, consistent with convergence-based early-exit criteria. All code, pre-registration (commit 702efd95), per-head data, and experiment notes with recorded protocol deviations are in the public repository 3ld0n/attention-geometry.