Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2611.05355

Reading Why: Born-Measured Attention with Exact Word Evidence on Light and Qubits

Teerawat Tongloy, Jiraporn Pongsopa, Rangsan Jomtarak

Published Nov 3, 2026• 0 citations
arXivHTMLPDF

Abstract

When a language model answers, we can usually only estimate why. We build a transformer in which every nonlinearity is a quadratic detection - an interference, a balanced reading or a camera - and the output is a Born probability: attention ||^2, the form of the Born rule, normalised per row, a measured product for the feed-forward nonlinearity, and a Born readout; between detections, classical control rescales and re-encodes. With its denominators held at the input, each class probability is a polynomial in the word amplitudes, so its integrated-gradient evidence is computed by a finite quadrature with no quadrature error: the words' contributions sum to the class probability above its no-information share to 10^-13. On e-SNLI natural language inference it reaches 77.45 +- 0.91 % accuracy, level with a same-size softmax transformer (76.62 +- 0.36 %) and an order-free bag of embeddings (77.79 %). Exactness buys the sum, not the best ranking. Deleting the word its evidence ranks first changes the answer in 44 % of pairs, of the 53 % some deletion changes, against 31-32 % for the softmax model's integrated gradients; the model's own gradient x input does as well (43 %), as does a bag of embeddings' (43 %, seed 0), and leave-one-out does better. Holding the denominators helps the low end: the five words ranked last change the answer in 1 %, against 6 % free. The same weights compile exactly to simulated optoelectronics, and their attention to qubits. On ibm_kingston, reading only the attention, the device gave the model's answers in all three Hadamard-test runs (the two shortest pairs and one without its deciding word) and both compute-uncompute runs (one circuit per overlap); uniform attention answers otherwise on two of the three inputs. With the device's row sums the evidence correlated with the exact one at 0.997 and 0.962 by Hadamard tests; with compute-uncompute, 0.971 and 0.48.

DOI
10.48550/arxiv.2611.05355

Related papers

A Quantum Roadmap for Softmax Attention: Exact Born-Rule Analogs for Softmax Attention on the Probability Simplex

Eric A. F. Reinhardt, Adam J. Hauser

The attention mechanism forms the foundation of many modern AI models such as the Transformer. In one subclass of problems where attention is used, inputs and outputs are bound to the probability simplex so that all outputs sum to one. In this setting, softmax attention admits an exact, component-by-component quantum realization. Attention scores are Hadamard-test statistics on block-encoded projections of amplitude-encoded inputs. The exponential softmax is the interior of a cosine-squared family generated by Born-rule measurement under an exact bijection, whose boundary expresses sparse attention with exact zeros at finite parameter values. The softmax temperature is a repetition count where post-selected measurement rounds realize discretized inverse temperature exactly. Value aggregation is a deterministic column-loading channel that dilates the column-stochastic value matrix. The gated residual is the preparation angle of a single ancilla, with the additive identity at a mixing angle of $π$/2. Every learnable parameter is a rotation-gate angle. The composed layer is exact in the infinite-shot limit with one measure-and-reload step per attention score; a fully-coherent variant is $ε$-approximate via quantum singular value transformation in the infinite depth limit. The algebraic core is machine-checked in Lean 4.

quant-pharXiv:2608.11173•Aug 12, 2026•2 citations

Seeing Why: Exact Evidence Maps for Quantum and Optical Image Classifiers

Teerawat Tongloy, Rangsan Jomtarak

When an image classifier decides, we can usually only estimate why. For quantum and all-optical classifiers that act linearly on amplitude-encoded pictures up to the measurement, with no trained classical network before or after them, each class's Born probability above its no-information share, $P_g - b_g$, splits exactly into contributions of the encoded amplitudes, shown as Born evidence maps (BEM) over the picture. The split itself follows from known results on quadratic forms: with the transmitted light held fixed, it is what Integrated Gradients from a zero baseline and the Shapley value give. What we add is a way to measure it on the hardware that classifies, from class probabilities on a quantum chip or from camera frames on a lens bench, with no gradient circuit and no light sent backwards; a $g\times g$ map takes $1 + 2g^2$ runs. Deleting the 5% of pixels with the highest evidence drops accuracy from 73% to 9%, as occlusion does with 1,024 extra runs per picture, whereas gradient saliency removes much less. The classifiers share one architecture, a Fourier-plane mask followed by a brickwall circuit on qubits or by phase plates on a lens bench, read by a fixed Born headroom readout with no trained parameter; centring each picture on its own mean lets dark features count as much as bright ones. In simulation, four phase plates at a single wavelength reach 72.3% on Caltech-10 colour photographs. On IBM's ibm_kingston, with the mask applied before approximate loading, 300 test pictures were classified at 37.7% against 10% by chance (56.3% in exact simulation), and $2\times2$ maps of ten correctly classified pictures, one per class chosen by a stated rule, measured on the chip, agreed with the exact ones ($r = 0.77$). The maps also revealed a shortcut in the data; as the trim that removed it was chosen on test pictures, the Caltech-10 results are exploratory.

quant-pharXiv:2611.01993•Nov 3, 2026•1 citation

Sublinear Time Quantum Algorithm for Attention Approximation

Zhao Song, Jianfei Xue, Jiahao Zhang +1 more

Given the query, key and value matrices $Q, K, V\in \mathbb{R}^{n\times d}$, the attention module is defined as $\mathrm{Att}(Q, K, V)=D^{-1}AV$ where $A=\exp(QK^\top/\sqrt{d})$ with $\exp(\cdot)$ applied entrywise, $D=\mathrm{diag}(A{\bf 1}_n)$. The attention module is the backbone of modern transformers and large language models, but explicitly forming the softmax matrix $D^{-1}A$ incurs $Ω(n^2)$ time, motivating numerous approximation schemes that reduce runtime to $\widetilde O(nd)$ via sparsity or low-rank factorization. We propose a quantum data structure that approximates any row of $\mathrm{Att}(Q, K, V)$ using only row queries to $Q, K, V$. Our algorithm preprocesses these matrices in $\widetilde{O}\left( ε^{-1} n^{0.5} \left( s_λ^{2.5} + s_λ^{1.5} d + α^{0.5} d \right) \right)$ time, where $ε$ is the target accuracy, $s_λ$ is the $λ$-statistical dimension of the exponential kernel defined by $Q$ and $K$, and $α$ measures the row distortion of $V$ that is at most $d/{\rm srank}(V)$, the stable rank of $V$. Each row query can be answered in $\widetilde{O}(s_λ^2 + s_λd)$ time. To our knowledge, this is the first quantum data structure that approximates rows of the attention matrix in sublinear time with respect to $n$. Our approach relies on a quantum Nyström approximation of the exponential kernel, quantum multivariate mean estimation for computing $D$, and quantum leverage score sampling for the multiplication with $V$.

quant-pharXiv:2602.00874•Feb 3, 2026•0 citations

Born Attention is Linear

Marcel Crâşmaru

Softmax self-attention costs O(T^2) in the sequence length T because the exponential kernel has no finite-dimensional feature map, so linear-time variants approximate it. We observe that Born-rule attention, whose weights are quantum-measurement probabilities a_kj ∝ |<Q_k, K_j>|^2 over complex queries and keys, is an exact finite kernel with the rank-one density-matrix feature map ρ_x = x x†, since |<Q_k, K_j>|^2 = tr(ρ_Qk ρ_Kj). As the complex degree-two case of polynomial-kernel linear attention it admits an exact O(T d^2) causal recurrence and constant-state decoding (d^2 (d_v + 1) reals per head, with d_v the real value width, independent of T), and it keeps rotary positions exactly. For GPT-2's head geometry (64 real channels paired into d = 32 complex) the feature dimension is d^2 = 1024. Measured on an A100, compiled constant-state decoding runs from 6.4 MB per layer at 0.12 ms per token at any length, against a 3.2 GB FlashAttention-2 cache at 2.26 ms per token at 1M tokens; unfused prefill overtakes FlashAttention-2 between 64k and 128k tokens (11x at 1M), and a fused form (Triton feature lift plus a chunkwise linear-attention kernel) between 16k and 32k (4x at 128k), reaching training parity at 64k. The linear and quadratic forms are two implementations of one operator; whether that operator matches softmax is a separate modeling question, answered here only at small scale: the p = 1 operator bolted into a real-valued GPT trails softmax by 0.12 nats at context 1k and 0.77 at 16k (attention dilution; a decay kept exactly in the d^2 state restores the 0.12 gap at 16k, and p = 2 sharpening with decay recovers 74% of the 1k gap), whereas inside a fully complex residual stream it matches softmax at 1k and keeps a quarter of the real-stream gap at 4k; gated linear attention beats both there. At 128k context on PG-19 the variable that matters is forgetting: plain Born dilutes, Born with a fixed decay carried in its state is 0.32 nats ahead of softmax and within 0.04 of gated linear attention, which stays 1.8x faster. Streamed with their constant state over the PG-19 test partition in 5M-byte windows, two 4M-complex-parameter byte models of the source framework score 1.66 (linear Born with decay) and 1.70 (gated) bits per byte, where a 350M-parameter hierarchical Mamba–Transformer stack built for that window reports 2.45. Preprint, October 2026. Also submitted to arXiv (identifier to be added on announcement). Code: the CNet framework, https://github.com/crasmarum/CNet .

•Oct 27, 2026•0 citations

QuantumGPTMini: A Hybrid Quantum-Classical Transformer for Enhanced NLP

Ramazan Amire, Amire Ramazan

Transformer architectures have driven advances in natural language processing, powering systems from question answering to creative text generation. Yet as model sizes have grown from hundreds of millions to hundreds of billions of parameters, the computational cost of these systems has become prohibitive, placing advanced language technology out of reach for many researchers. In this work, we introduce QuantumGPTMini, a hybrid quantum-classical transformer with just over one million classical parameters that leverages lightweight quantum circuits to perform its core attention and feed-forward computations. By encoding token embeddings both in conventional vectors and in the amplitudes of small variational quantum circuits, our model replaces the conventional softmax attention mechanism with an entanglement-driven alternative. When trained on the Wikipedia-2 corpus using PennyLane's lightning.qubit simulator on NVIDIA A100 GPUs, QuantumGPTMini matched the perplexity of a GPT-2 Small baseline while demonstrating superior pattern recognition and symbolic reasoning behaviors. We have made this system publicly accessible through a FastAPI endpoint and a Next.js web interface. While our initial results underscore the potential of quantum-augmented language models, they also highlight challenges in moving beyond simulation, including circuit depth limitations, decoherence, and the high overhead of parameter-shift gradient estimation.

•Apr 28, 2025•0 citations

Deep Sequence Modeling with Quantum Dynamics: Language as a Wave Function

Ahmed Nebli, Hadi Saadatdoorabi, Kevin Yam

We introduce a sequence modeling framework in which the latent state is a complex-valued wave function evolving on a finite-dimensional Hilbert space under a learned, time-dependent Hamiltonian. Unlike standard recurrent architectures that rely on gating mechanisms to suppress competing hypotheses, our framework utilizes quantum interference: the Hamiltonian steers the phases of complex amplitudes so that conflicting interpretations cancel while compatible ones reinforce. The dynamics are strictly unitary, ensuring that the state norm is preserved exactly at every time step via a Cayley (Crank--Nicolson) discretization. Token probabilities are extracted using the Born rule, a quadratic measurement operator that couples magnitudes and relative phases. Our primary theoretical contribution is a separation theorem characterizing the representational advantage of this readout: we define a family of disambiguation tasks that a complex unitary model of dimension $N$ solves exactly, but which requires a state dimension of $Ω(N^2)$ for any real-valued orthogonal model equipped with a standard affine-softmax readout. This quadratic gap arises because the Born rule implicitly lifts the $N$-dimensional state into the space of rank-one Hermitian matrices, accessing pairwise phase correlations that are inaccessible to linear projections. Finally, we derive a continuity equation for the latent probability mass, yielding conserved pairwise currents that serve as a built-in diagnostic for tracing information flow between dimensions.

cs.LGCross-listedarXiv:2602.22255•Feb 26, 2026•3 citations
When a language model answers, we can usually only estimate why. We build a transformer in which every nonlinearity is a quadratic detection - an interference, a balanced reading or a camera - and the output is a Born probability: attention ||^2, the form of the Born rule, normalised per row, a measured product for the feed-forward nonlinearity, and a Born readout; between detections, classical control rescales and re-encodes. With its denominators held at the input, each class probability is a polynomial in the word amplitudes, so its integrated-gradient evidence is computed by a finite quadrature with no quadrature error: the words' contributions sum to the class probability above its no-information share to 10^-13. On e-SNLI natural language inference it reaches 77.45 +- 0.91 % accuracy, level with a same-size softmax transformer (76.62 +- 0.36 %) and an order-free bag of embeddings (77.79 %). Exactness buys the sum, not the best ranking. Deleting the word its evidence ranks first changes the answer in 44 % of pairs, of the 53 % some deletion changes, against 31-32 % for the softmax model's integrated gradients; the model's own gradient x input does as well (43 %), as does a bag of embeddings' (43 %, seed 0), and leave-one-out does better. Holding the denominators helps the low end: the five words ranked last change the answer in 1 %, against 6 % free. The same weights compile exactly to simulated optoelectronics, and their attention to qubits. On ibm_kingston, reading only the attention, the device gave the model's answers in all three Hadamard-test runs (the two shortest pairs and one without its deciding word) and both compute-uncompute runs (one circuit per overlap); uniform attention answers otherwise on two of the three inputs. With the device's row sums the evidence correlated with the exact one at 0.997 and 0.962 by Hadamard tests; with compute-uncompute, 0.971 and 0.48.
The attention mechanism forms the foundation of many modern AI models such as the Transformer. In one subclass of problems where attention is used, inputs and outputs are bound to the probability simplex so that all outputs sum to one. In this setting, softmax attention admits an exact, component-by-component quantum realization. Attention scores are Hadamard-test statistics on block-encoded projections of amplitude-encoded inputs. The exponential softmax is the interior of a cosine-squared family generated by Born-rule measurement under an exact bijection, whose boundary expresses sparse attention with exact zeros at finite parameter values. The softmax temperature is a repetition count where post-selected measurement rounds realize discretized inverse temperature exactly. Value aggregation is a deterministic column-loading channel that dilates the column-stochastic value matrix. The gated residual is the preparation angle of a single ancilla, with the additive identity at a mixing angle of πππ/2. Every learnable parameter is a rotation-gate angle. The composed layer is exact in the infinite-shot limit with one measure-and-reload step per attention score; a fully-coherent variant is εεε-approximate via quantum singular value transformation in the infinite depth limit. The algebraic core is machine-checked in Lean 4.
When an image classifier decides, we can usually only estimate why. For quantum and all-optical classifiers that act linearly on amplitude-encoded pictures up to the measurement, with no trained classical network before or after them, each class's Born probability above its no-information share, Pg−bgP_g - b_gPg​−bg​, splits exactly into contributions of the encoded amplitudes, shown as Born evidence maps (BEM) over the picture. The split itself follows from known results on quadratic forms: with the transmitted light held fixed, it is what Integrated Gradients from a zero baseline and the Shapley value give. What we add is a way to measure it on the hardware that classifies, from class probabilities on a quantum chip or from camera frames on a lens bench, with no gradient circuit and no light sent backwards; a g×gg\times gg×g map takes 1+2g21 + 2g^21+2g2 runs. Deleting the 5% of pixels with the highest evidence drops accuracy from 73% to 9%, as occlusion does with 1,024 extra runs per picture, whereas gradient saliency removes much less. The classifiers share one architecture, a Fourier-plane mask followed by a brickwall circuit on qubits or by phase plates on a lens bench, read by a fixed Born headroom readout with no trained parameter; centring each picture on its own mean lets dark features count as much as bright ones. In simulation, four phase plates at a single wavelength reach 72.3% on Caltech-10 colour photographs. On IBM's ibm_kingston, with the mask applied before approximate loading, 300 test pictures were classified at 37.7% against 10% by chance (56.3% in exact simulation), and 2×22\times22×2 maps of ten correctly classified pictures, one per class chosen by a stated rule, measured on the chip, agreed with the exact ones (r=0.77r = 0.77r=0.77). The maps also revealed a shortcut in the data; as the trim that removed it was chosen on test pictures, the Caltech-10 results are exploratory.
Given the query, key and value matrices Q,K,V∈Rn×dQ, K, V\in \mathbb{R}^{n\times d}Q,K,V∈Rn×d, the attention module is defined as Att(Q,K,V)=D−1AV\mathrm{Att}(Q, K, V)=D^{-1}AVAtt(Q,K,V)=D−1AV where A=exp⁡(QK⊤/d)A=\exp(QK^\top/\sqrt{d})A=exp(QK⊤/d​) with exp⁡(⋅)\exp(\cdot)exp(⋅) applied entrywise, D=diag(A1n)D=\mathrm{diag}(A{\bf 1}_n)D=diag(A1n​). The attention module is the backbone of modern transformers and large language models, but explicitly forming the softmax matrix D−1AD^{-1}AD−1A incurs Ω(n2)Ω(n^2)Ω(n2) time, motivating numerous approximation schemes that reduce runtime to O~(nd)\widetilde O(nd)O(nd) via sparsity or low-rank factorization. We propose a quantum data structure that approximates any row of Att(Q,K,V)\mathrm{Att}(Q, K, V)Att(Q,K,V) using only row queries to Q,K,VQ, K, VQ,K,V. Our algorithm preprocesses these matrices in O~(ε−1n0.5(sλ2.5+sλ1.5d+α0.5d))\widetilde{O}\left( ε^{-1} n^{0.5} \left( s_λ^{2.5} + s_λ^{1.5} d + α^{0.5} d \right) \right)O(ε−1n0.5(sλ2.5​+sλ1.5​d+α0.5d)) time, where εεε is the target accuracy, sλs_λsλ​ is the λλλ-statistical dimension of the exponential kernel defined by QQQ and KKK, and ααα measures the row distortion of VVV that is at most d/srank(V)d/{\rm srank}(V)d/srank(V), the stable rank of VVV. Each row query can be answered in O~(sλ2+sλd)\widetilde{O}(s_λ^2 + s_λd)O(sλ2​+sλ​d) time. To our knowledge, this is the first quantum data structure that approximates rows of the attention matrix in sublinear time with respect to nnn. Our approach relies on a quantum Nyström approximation of the exponential kernel, quantum multivariate mean estimation for computing DDD, and quantum leverage score sampling for the multiplication with VVV.
Softmax self-attention costs O(T^2) in the sequence length T because the exponential kernel has no finite-dimensional feature map, so linear-time variants approximate it. We observe that Born-rule attention, whose weights are quantum-measurement probabilities a_kj ∝ |<Q_k, K_j>|^2 over complex queries and keys, is an exact finite kernel with the rank-one density-matrix feature map ρ_x = x x†, since |<Q_k, K_j>|^2 = tr(ρ_Qk ρ_Kj). As the complex degree-two case of polynomial-kernel linear attention it admits an exact O(T d^2) causal recurrence and constant-state decoding (d^2 (d_v + 1) reals per head, with d_v the real value width, independent of T), and it keeps rotary positions exactly. For GPT-2's head geometry (64 real channels paired into d = 32 complex) the feature dimension is d^2 = 1024. Measured on an A100, compiled constant-state decoding runs from 6.4 MB per layer at 0.12 ms per token at any length, against a 3.2 GB FlashAttention-2 cache at 2.26 ms per token at 1M tokens; unfused prefill overtakes FlashAttention-2 between 64k and 128k tokens (11x at 1M), and a fused form (Triton feature lift plus a chunkwise linear-attention kernel) between 16k and 32k (4x at 128k), reaching training parity at 64k. The linear and quadratic forms are two implementations of one operator; whether that operator matches softmax is a separate modeling question, answered here only at small scale: the p = 1 operator bolted into a real-valued GPT trails softmax by 0.12 nats at context 1k and 0.77 at 16k (attention dilution; a decay kept exactly in the d^2 state restores the 0.12 gap at 16k, and p = 2 sharpening with decay recovers 74% of the 1k gap), whereas inside a fully complex residual stream it matches softmax at 1k and keeps a quarter of the real-stream gap at 4k; gated linear attention beats both there. At 128k context on PG-19 the variable that matters is forgetting: plain Born dilutes, Born with a fixed decay carried in its state is 0.32 nats ahead of softmax and within 0.04 of gated linear attention, which stays 1.8x faster. Streamed with their constant state over the PG-19 test partition in 5M-byte windows, two 4M-complex-parameter byte models of the source framework score 1.66 (linear Born with decay) and 1.70 (gated) bits per byte, where a 350M-parameter hierarchical Mamba–Transformer stack built for that window reports 2.45. Preprint, October 2026. Also submitted to arXiv (identifier to be added on announcement). Code: the CNet framework, https://github.com/crasmarum/CNet .
Transformer architectures have driven advances in natural language processing, powering systems from question answering to creative text generation. Yet as model sizes have grown from hundreds of millions to hundreds of billions of parameters, the computational cost of these systems has become prohibitive, placing advanced language technology out of reach for many researchers. In this work, we introduce QuantumGPTMini, a hybrid quantum-classical transformer with just over one million classical parameters that leverages lightweight quantum circuits to perform its core attention and feed-forward computations. By encoding token embeddings both in conventional vectors and in the amplitudes of small variational quantum circuits, our model replaces the conventional softmax attention mechanism with an entanglement-driven alternative. When trained on the Wikipedia-2 corpus using PennyLane's lightning.qubit simulator on NVIDIA A100 GPUs, QuantumGPTMini matched the perplexity of a GPT-2 Small baseline while demonstrating superior pattern recognition and symbolic reasoning behaviors. We have made this system publicly accessible through a FastAPI endpoint and a Next.js web interface. While our initial results underscore the potential of quantum-augmented language models, they also highlight challenges in moving beyond simulation, including circuit depth limitations, decoherence, and the high overhead of parameter-shift gradient estimation.
We introduce a sequence modeling framework in which the latent state is a complex-valued wave function evolving on a finite-dimensional Hilbert space under a learned, time-dependent Hamiltonian. Unlike standard recurrent architectures that rely on gating mechanisms to suppress competing hypotheses, our framework utilizes quantum interference: the Hamiltonian steers the phases of complex amplitudes so that conflicting interpretations cancel while compatible ones reinforce. The dynamics are strictly unitary, ensuring that the state norm is preserved exactly at every time step via a Cayley (Crank--Nicolson) discretization. Token probabilities are extracted using the Born rule, a quadratic measurement operator that couples magnitudes and relative phases. Our primary theoretical contribution is a separation theorem characterizing the representational advantage of this readout: we define a family of disambiguation tasks that a complex unitary model of dimension NNN solves exactly, but which requires a state dimension of Ω(N2)Ω(N^2)Ω(N2) for any real-valued orthogonal model equipped with a standard affine-softmax readout. This quadratic gap arises because the Born rule implicitly lifts the NNN-dimensional state into the space of rank-one Hermitian matrices, accessing pairwise phase correlations that are inaccessible to linear projections. Finally, we derive a continuity equation for the latent probability mass, yielding conserved pairwise currents that serve as a built-in diagnostic for tracing information flow between dimensions.