Fine-tuning a language model often fails silently when its correct completion must outrank a near-synonym competitor. Cross-entropy loss falls monotonically while the correct token never overtakes the competitor. We study five transformer architectures of two families over a fivefold parameter range, on ten contexts where the correct completion and nearest competitor overlap substantially. We construct an order parameter combining the predicted distribution with the geometric overlap between token embeddings. The density-matrix formulation is natural because the prediction distribution lives over a non-orthogonal embedding basis. It decomposes additively into a signal, tracking commitment to the correct token over its nearest competitor, and a background drag, set by how the embedding bulk leaks probability into the score. This isolates two failure modes. In kinematic failure the signal stays small and the model never commits. In structural failure the drag actively worsens as fine-tuning proceeds and the model degrades geometrically even as its loss decreases. Sharp jumps resemble phase transitions but are not. Tracking it after every gradient step rules out spontaneous symmetry breaking. Under LoRA fine-tuning the embedding matrix is frozen, so no geometric phase transition is possible and the discontinuity lies in the softmax readout. One dimensionless quantity is consistent across all five models under full fine-tuning, while a second sorts them into two classes by bulk embedding distribution and predicts whether LoRA alone can drive a sentence to commit. As a blind test, the framework predicts the critical learning rate of a held-out architecture, used to fit no parameter, to within 2.1% of a subsequent sweep. All findings concern the near-synonym geometric mechanism studied here and should not be extrapolated without re-calibration to instruction tuning or broader tasks.
Fine-tuning a language model often fails silently when its correct completion must outrank a near-synonym competitor. Cross-entropy loss falls monotonically while the correct token never overtakes the competitor. We study five transformer architectures of two families over a fivefold parameter range, on ten contexts where the correct completion and nearest competitor overlap substantially. We construct an order parameter combining the predicted distribution with the geometric overlap between token embeddings. The density-matrix formulation is natural because the prediction distribution lives over a non-orthogonal embedding basis. It decomposes additively into a signal, tracking commitment to the correct token over its nearest competitor, and a background drag, set by how the embedding bulk leaks probability into the score. This isolates two failure modes. In kinematic failure the signal stays small and the model never commits. In structural failure the drag actively worsens as fine-tuning proceeds and the model degrades geometrically even as its loss decreases. Sharp jumps resemble phase transitions but are not. Tracking it after every gradient step rules out spontaneous symmetry breaking. Under LoRA fine-tuning the embedding matrix is frozen, so no geometric phase transition is possible and the discontinuity lies in the softmax readout. One dimensionless quantity is consistent across all five models under full fine-tuning, while a second sorts them into two classes by bulk embedding distribution and predicts whether LoRA alone can drive a sentence to commit. As a blind test, the framework predicts the critical learning rate of a held-out architecture, used to fit no parameter, to within 2.1% of a subsequent sweep. All findings concern the near-synonym geometric mechanism studied here and should not be extrapolated without re-calibration to instruction tuning or broader tasks.
Large language models (LLMs) can be adapted to new tasks using parameter-efficient fine-tuning (PEFT) methods that modify only a small number of trainable parameters, often through low-rank updates. In this work, we adopt a quantum-information-inspired perspective to understand their effectiveness. From this perspective, low-rank parameterizations naturally correspond to low-dimensional Matrix Product States (MPS) representations, which enable entanglement-based characterizations of parameter structure. Thereby, we term and measure "Artificial Entanglement", defined as the entanglement entropy of the parameters in artificial neural networks (in particular the LLMs). We first study the representative low-rank adaptation (LoRA) PEFT method, alongside full fine-tuning (FFT), using LLaMA models at the 1B and 8B scales trained on the Tulu3 and OpenThoughts3 datasets, and uncover: (i) Internal artificial entanglement in the updates of query and value projection matrices in LoRA follows a volume law with a central suppression (termed as the "Entanglement Valley"), which is sensitive to hyper-parameters and is distinct from that in FFT; (ii) External artificial entanglement in attention matrices, corresponding to token-token correlations in representation space, follows an area law with logarithmic corrections and remains robust to LoRA hyper-parameters and training steps. Drawing a parallel to the No-Hair Theorem in black hole physics, we propose that although LoRA and FFT induce distinct internal entanglement signatures, such differences do not manifest in the attention outputs, suggesting a "no-hair" property that results in the effectiveness of low rank updates. We further provide theoretical support based on random matrix theory, and extend our analysis to an MPS Adaptation PEFT method, which exhibits qualitatively similar behaviors.
The rapid progress of large language models (LLMs) has transformed natural language processing, yet the challenge of efficient adaptation remains unresolved. Full fine-tuning achieves strong performance but imposes prohibitive computational and memory costs. Parameter-efficient fine-tuning (PEFT) strategies, such as low-rank adaptation (LoRA), Prefix tuning, and sparse low-rank adaptation (SoRA), address this issue by reducing trainable parameters while maintaining competitive accuracy. However, these methods often encounter limitations in scalability, stability, and generalization across diverse tasks. Recent advances in quantum deep learning introduce novel opportunities through quantum-inspired encoding and parameterized quantum circuits (PQCs). In particular, the quantum-amplitude embedded adaptation (QAA) framework demonstrates expressive model updates with minimal overhead. This paper presents a systematic survey and comparative analysis of conventional PEFT methods and QAA. The analysis demonstrates trade-offs in convergence, efficiency, and representational capacity, while providing insight into the potential of quantum approaches for future LLM adaptation.
We report a three-month autonomous research-agent program testing five solid-state-physics-inspired compression mappings on pretrained language models, with predictions committed to git before any pilot data and a 3-sigma gate deciding PASS or SHELVE. The common anchor -- area-law / Kohn-nearsighted decay of the one-particle density matrix -- has a distance face (P001 Wannier, P002 tight-binding) and a rank face (P003 DMRG-truncated MLPs, P005 Wilson-RG, P011 tensor-train embeddings). P005 was pre-empted at Phase 1; three of four Phase-3 pilots were falsified. On the attention face, GPT-2-medium attention-versus-distance is best fit by a stretched exponential in 12 of 16 median-layer heads once probe padding is excluded, and a tight-binding cutoff costs +96% perplexity (P002); on Pythia-160M the Wannier sparsity 0.054 +/- 0.004 is indistinguishable from PCA, random-Haar and identity baselines (P001). On the rank face, per-token tensor-train bond dimension does not track surprisal (r = 0.016 vs a pre-registered 0.65) and the format inflates rather than compresses (P011). P003 is mixed: its scaling claim shelved (r = -0.434), its MPO premise died at stage-0, and its cross-paper check, r = 0.523 as first written, collapses to 0.047 under the same correction, leaving both cross-paper checks null. The results invert the pre-registered prediction that most attention heads behave like Kohn-nearsighted insulators, pointing instead to critical, glassy or heavy-tailed regimes; the inversion is specific to the <= 350M scale tested, while the rank-face no-gain result held to 7-8B. We contribute the pre-registration + 3-sigma + cluster-framing + append-only-catalogue discipline -- including why our own enforcement gate was designed but not deployed -- four pre-registered negative results with full data release, and the inversion. The catalogue holds eighteen concluded studies, seventeen negative.
When full-parameter updates are impractical for large language models (LLMs), parameter-efficient fine-tuning (PEFT) is commonly employed to reduce the number of trainable parameters. Classical PEFT methods, such as low-rank adaptation (LoRA), are limited to linear transformations and may not capture complex, high-order feature interactions under stringent parameter constraints. In contrast, parameterized quantum circuits (PQCs) provide high-order expressivity per qubit rotation degree of freedom. Inspired by PQCs, in this paper, we propose quantum cross-hybrid fine-tuning (QCHFT), a suite of four PQC-augmented adapters: Matrix Product State PQC, Alternating and Hierarchical Pairwise Contraction PQC, and LinearA–Quantum–LinearB PQC. To evaluate our QCHFT adapters, we conduct experiments on a public multi-tier IoT resource-allocation task that serves as a challenging benchmark with intricate cross-tier interactions data and limited parameter budgets. We conduct experiments on a multi-tier IoT resource-allocation task across five causal LLM backbones scaling from 125M to 2B parameters, including Mamba2, GPT-Neo, Pythia, Llama-3.2, and Qwen-3.5. Our results demonstrate that QCHFT consistently enhances performance compared to both standard LoRA. On Mamba2-130M, the APC–PQC variant improves accuracy from 67.33% (LoRA) to 72.66%. These findings suggest that the high expressivity of quantum entanglement can be practically harnessed through shallow circuits, providing a superior, parameter-efficient pathway for adapting foundation models to complex, high-feature tasks.
An autoregressive language model, read at the token-commit step, is a near-fully-inspectable classical bounded-inference substrate: the competing-route distribution over candidate tokens — the live propensity before it resolves into one committed answer — is read directly from the output logits. Used as a measurement model, it lets each structural signature of quantum measurement be tested one by one, and the genuinely-quantum residue isolated by its absence. The substrate reproduces the phenomenology of quantum measurement from classical self-conditioning — back-action, non-commutativity (order effects), entropy-collapse on commitment, a substantial Busemeyer–Wang QQ-structure (order-effect difference q not distinguishable from the parameter-free quantum value 0; pooled across eight de-saturated base models over five vendors and four architectures, N=96, q=-0.003, 95% CI [-0.025, +0.019]), interference, einselection, an Aharonov-style weak-versus-strong asymmetry, and measurement-mode dependence — but not the quantum core: genuine contextuality is robustly absent (an analog Contextuality-by-Default cyclic-4 analysis gives CNT < 0 in every model and scenario), and no phase/coherence is demonstrated. A controllable analog Bell-loophole testbed makes each premise's price visible within one substrate (S_analog ~ 2.00 with all loopholes closed; ~ 2.8 on opening locality or measurement-independence), and a within-substrate timeline reads the propensity resolving across network depth, including a maximally-contested case whose outcome is never created. The order-effect incompatibility is shown to be a measured, correctly-signed classical-complementarity dial that falls as the readout partition refines. Positioning. The contribution is the demarcation itself, with statistics, made possible by an inspectability no veiled (physical or psychological) substrate allows. It instantiates beim Graben & Atmanspacher's classical complementarity and the Busemeyer–Wang / Pothos quantum-cognition program empirically; it makes no claim about physical reality. Companion papers (Friction Theory series): Paper 1 — Friction as the cost of probabilistic computation (10.5281/zenodo.20012654); Paper 10 — Race all the way down/up (10.5281/zenodo.20014567); Paper 21 — LLMs as a measurement model for bounded-decision cognition (10.5281/zenodo.20562416); Paper 28 — Dread without a dreader (10.5281/zenodo.20690747).Paper 18 in the Friction Theory paper-series. Submitted as a careful, honestly-scoped empirical demarcation (target venue: Entropy, Quantum Information / quantum-cognition & foundations-of-measurement). v2 (August 2026) — prior-art revision. The paper describes competition resolving to a commitment and cited none of the tradition that formalised it. Luce (1959) for the choice axiom the softmax implements, Ratcliff (1978) for evidence accumulated to a bound, and Usher and McClelland (2001) for the leaky competing accumulator are now credited at the point where the reading is made, with the limit stated: the estimators here are computed on a language model's output distribution, and the parallel is why the measurement was worth making rather than evidence of a shared mechanism. Editorial pass. Earlier versions remain in the version history.
We introduce a hybrid quantum-classical deep learning architecture for large language model fine-tuning. The classical portion of the architecture is a sentence transformer that is powerful enough to display significant accuracy for complex tasks such as sentiment prediction. The quantum portion of the architecture consists of parameterized quantum circuits that utilize long-range connections between qubits. We analyze the performance of the hybrid models for various settings of hyperparameters, including the number of qubits, the depth of the quantum circuits, learning rate, number of reuploading steps, etc. Based on a screening study of main effects, we show an overall improvement in prediction accuracy over a comparable classical baseline, with a trend of increasing accuracy with number of qubits. We observe up to 3.14% improvements in accuracy over classical architectures of comparable model size, within the set of hyperparameters probed in this study. We demonstrate the contribution of each module in our architecture through ablation studies. Our studies are based on finite shot-counts and include simulations based on noisy quantum gates.