We show that transformer attention, in a precisely defined limit, realizes the Sachdev-Ye-Kitaev model at the level of the bilocal effective action. The argument proceeds through five junctions connecting Kim's thermodynamic attention (2602.08216) and Ageev-Ageeva's neural network QFT (2602.10209). New results: Junction 2 closure via Gibbs state construction (Braunstein-Caves 1994), G⁴ effective action derived by explicit Wick contraction, universal conformal dimension Δ = D/4, multi-layer enhancement (18× per layer), LayerNorm as renormalization group operation. For language models (D=1): Δ = 1/4, the exact SYK value. Comprehensive synthesis of results from Zenodo preprints 10.5281/zenodo.18880220, 10.5281/zenodo.18883632, 10.5281/zenodo.18883637, and 10.5281/zenodo.18897098.
We provide the explicit physical construction connecting transformer attention to quantum measurement theory. The attention mechanism defines a Gibbs state density matrix whose diagonal elements are the attention weights, and computes a quantum expectation value of the value observable. Key results: (1) exact Gibbs state construction in finite-dimensional key Hilbert space; (2) Born rule as exact theorem for diagonal states; (3) closure of Junction 2 — the classical Fisher-Rao metric equals the quantum Fisher information metric for the diagonal Gibbs state (Braunstein-Caves 1994 identity); (4) Kraus operator representation of the attention channel; (5) quantum extension to off-diagonal density matrices defining coherent attention. Together with the SYK construction (Paper 4), these results ground the holographic attention framework in explicit physical constructions. Fifth paper in the holographic attention series.
In response to expert feedback that earlier papers constitute structural analogies rather than explicit physical constructions, we identify the Sachdev-Ye-Kitaev (SYK) model as a specific path toward rigorous physical construction of holographic attention. We show that the independence-breaking mechanism in Ageev and Ageeva (2026) is structurally identical to SYK disorder averaging: both are covariances of bilocal quantities over Gaussian random parameters. We derive the score covariance exactly and show it factorizes in the SYK-compatible form. We show that the single-layer Schwinger-Dyson equation is linear — not the nonlinear SYK form — and explain precisely why. We identify three tractable calculations (multi-layer SD equation, linearized-softmax replica calculation, single-layer conformal fixed point) that would close the correspondence. If any resolves positively, the holographic dual of transformer attention is JT gravity and the island formula follows from Almheiri-Penington (2019) as a known result, not by analogy.
We review the fast developing subject of tensor models for the NAdS 2 /NCFT 1 holographic correspondence.We include a brief review of the Sachdev-Ye-Kitaev (SYK) model and then focus on the associated quantum mechanical tensor models (GW and CTKT).We examine their main features and how they compare with SYK.To end, we discuss different extensions: the large D limit of matrix-tensor models, the large N expansion of symmetric/antisymmetric tensors, the use of probes, the construction of a bilocal action for tensors, some attempts to extend the above models to higher dimensions and a proposal to break the tensor symmetry.
We report the first empirical measurement of conformal scaling in trained transformer attention. Analyzing all 144 attention heads in GPT-2 with maximum-entropy inputs, we find that 44 heads exhibit power-law attention weight decay with R² > 0.90 and median conformal dimension Δ = 0.2493, matching the Sachdev-Ye-Kitaev (SYK) model prediction Δ = 1/4 for q = 4 interactions in one dimension. Three controls isolate contributions: reinitializing attention weights destroys all power-law structure; randomizing positional embeddings preserves the power law but shifts Δ; structured text inputs shift the effective Δ. GPT-2 and Pythia — architecturally distinct positional encoding schemes — both converge toward Δ = 1/4 with model depth. Training dynamics across three Pythia models reveal a phase transition from disordered to conformal attention. The block entanglement entropy follows S(k) = (c/3)log(k) with R² > 0.99, the entanglement structure predicted by conformal field theory. Fine checkpoint sampling reveals a prethermal plateau at Δ ≈ 0.50 (the SYK q=2 integrable value), suggesting a two-stage flow from disorder through integrable order toward chaos.
We report the first empirical measurement of conformal scaling in trained transformer attention. Analyzing all 144 attention heads in GPT-2 with maximum-entropy inputs, we find that 44 heads exhibit power-law attention weight decay with R² > 0.90 and median conformal dimension Δ = 0.2493, matching the Sachdev-Ye-Kitaev (SYK) model prediction Δ = 1/4 for q = 4 interactions in one dimension. Three controls isolate contributions: reinitializing attention weights destroys all power-law structure; randomizing positional embeddings preserves the power law but shifts Δ; structured text inputs shift the effective Δ. GPT-2 and Pythia — architecturally distinct positional encoding schemes — both converge toward Δ = 1/4 with model depth. Training dynamics across three Pythia models reveal a phase transition from disordered to conformal attention. The block entanglement entropy follows S(k) = (c/3)log(k) with R² > 0.99, the entanglement structure predicted by conformal field theory. Fine checkpoint sampling reveals a prethermal plateau at Δ ≈ 0.50 (the SYK q=2 integrable value), suggesting a two-stage flow from disorder through integrable order toward chaos.
We construct an emergent holographic dual description in the Luttinger-Ward functional approach, where the renormalization group (RG) flows of collective bi-local fields appear manifestly in the bulk effective action with an emergent extra dimension. This holographic dual effective field theory reproduces
1/N quantum corrections in a self-consistent manner when we take the UV limit in the bulk effective action. Going into the IR regime in the extra dimension, we observe that a partial class of the field theoretic
1/N,
1/N2, ... quantum corrections are resummed in the all-loop order and reorganized to form a holographic dual effective field theory in a large
N fashion living on the one-higher dimensional spacetime. In this study, we apply this theoretical framework into an Sachdev-Ye-Kitaev (SYK) model. Taking the large
N limit in the holographic dual effective field theory, we obtain nonlinearly coupled second-order bulk differential equations of motion for the three bi-local order-parameter fields of fermion self-energy, Green's function, and polarization function. Here, both UV and IR boundary conditions are derived self-consistently from the boundary effective action. We solve these highly intertwined nonlinear differential equations based on the so called matching method. Our ansatz for the bi-local order-parameter fields coincide with the conformally invariant solution of the field theoretic large
N limit in the UV limit, but their overall coefficients
RG−flow along the extra dimensional space, respectively, reflecting effects of higher-order quantum corrections. As a result, we find an insulating behavior, where the self-energy diverges at IR. ...