Contextuality is a distinguishing feature of quantum mechanics, and there is evidence that it is a necessary condition for quantum advantage in the context of magic state distillation. To harness it, researchers have been asking whether similar phenomena arise in other domains. The answer has been affirmative, e.g. in behavioural sciences. However, one has to adopt frameworks that take some degree of signalling into account. Two such frameworks exist: (i) a signalling-corrected sheaf-theoretic model and (ii) the Contextuality-by-Default (CbD) framework. This paper provides the first large-scale experimental evidence for contextuality in the large language model BERT. We constructed a linguistic schema modelled over a contextual quantum scenario, instantiated it in the Simple English Wikipedia, and extracted probability distributions for the instances. This led to the discovery of 77 118 sheaf-contextual and 36 938 948 CbD contextual instances. We prove that these contextual instances arise from semantically similar words by deriving an equation that relates degrees of contextuality to the Euclidean distance of BERT’s embedding vectors. Our linguistic schema is a variant of the coreference resolution challenge. These results suggest that quantum methods may be advantageous in the field of natural language processing.
Contextuality is a distinguishing feature of quantum mechanics, and there is evidence that it is a necessary condition for quantum advantage in the context of magic state distillation. To harness it, researchers have been asking whether similar phenomena arise in other domains. The answer has been affirmative, e.g. in behavioural sciences. However, one has to adopt frameworks that take some degree of signalling into account. Two such frameworks exist: (i) a signalling-corrected sheaf-theoretic model and (ii) the Contextuality-by-Default (CbD) framework. This paper provides the first large-scale experimental evidence for contextuality in the large language model BERT. We constructed a linguistic schema modelled over a contextual quantum scenario, instantiated it in the Simple English Wikipedia, and extracted probability distributions for the instances. This led to the discovery of 77 118 sheaf-contextual and 36 938 948 CbD contextual instances. We prove that these contextual instances arise from semantically similar words by deriving an equation that relates degrees of contextuality to the Euclidean distance of BERT’s embedding vectors. Our linguistic schema is a variant of the coreference resolution challenge. These results suggest that quantum methods may be advantageous in the field of natural language processing.
Ambiguities of natural language do not preclude us from using it and context helps in getting ideas across. They, nonetheless, pose a key challenge to the development of competent machines to understand natural language and use it as humans do. Contextuality is an unparalleled phenomenon in quantum mechanics, where different mathematical formalisms have been put forwards to understand and reason about it. In this paper, we construct a schema for anaphoric ambiguities that exhibits quantum-like contextuality. We use a recently developed criterion of sheaf-theoretic contextuality that is applicable to signalling models. We then take advantage of the neural word embedding engine BERT to instantiate the schema to natural language examples and extract probability distributions for the instances. As a result, plenty of sheaf-contextual examples were discovered in the natural language corpora BERT utilises. Our hope is that these examples will pave the way for future research and for finding ways to extend applications of quantum computing to natural language processing.
Language is contextual as meanings of words are dependent on their contexts. Contextuality is, concomitantly, a well-defined concept in quantum mechanics where it is considered a major resource for quantum computations. We investigate whether natural language exhibits any of the quantum mechanics' contextual features. We show that meaning combinations in ambiguous phrases can be modelled in the sheaf-theoretic framework for quantum contextuality, where they can become possibilistically contextual. Using the framework of Contextuality-by-Default (CbD), we explore the probabilistic variants of these and show that CbD-contextuality is also possible.
Quantum computing has demonstrated computational advantages over classical computing, yet its applications to natural language processing (NLP) remain in their early stages. Quantum contextuality, a fundamental feature of quantum mechanics, has been identified as a resource for achieving these advantages. Meanwhile, ambiguities in human languages present challenges in building systems capable of understanding and generating natural language. Many of these ambiguities can be addressed by considering the context in which they arise. This thesis investigates the connection between quantum contextuality and ambiguities in natural language. It introduces a framework that models ambiguities in language as a form of measurement, analogous to physical experiments. The findings reveal that contextuality is present in natural language and that its degree is linked to the reasoning capabilities of large language models (LLMs).
Life of Research AbstractThis paper introduces a novel theoretical framework for understanding semantic processing in Large LanguageModels through the lens of sheaf theory and quantum contextuality. We propose that the semantic structure ofLLM-generated text can be rigorously modeled as a sheaf over the topological space of prompt contexts, wherelocal semantic sections may fail to glue into globally consistent interpretations. This mathematical formalizationreveals a deep structural parallel between LLM behavior and quantum mechanical systems exhibitingcontextuality, as characterized by the Kochen-Specker theorem. Our central thesis posits that hallucinationphenomena in LLMs arise fundamentally from cohomological obstructions to the existence of global semanticsections, analogous to how quantum systems exhibit measurement outcomes that cannot be explained by preexisting hidden variables. We develop a complete categorical framework using topos-theoretic methods, introduceformal definitions of semantic presheaves and their associated Grothendieck topologies, and demonstrate howCech cohomology groups can quantify the degree of semantic inconsistency in model outputs. This work bridgesabstract mathematics with practical AI interpretability, offering new diagnostic tools for understanding when andwhy language models produce incoherent or factually inconsistent outputs.
This preprint defines the classical candidate-geometry baselines that should be compared before any claim of non-classicality in the quantum-like contextual non-additivity, order effects, and Bell-type indicators reported for large language models (LLMs) and masked language models (MLMs), and quantifies their explanatory power and limits. It does not claim physical quantum coherence in AI, nor a full explanation of AI quantum-like behavior. The revised edition adds four components. (i) Detection-power certification: the detector's condition is ICC < S_odd - 2, and Tsirelson-level injection tests certify that genuine contextuality of that magnitude would be detectable at the empirical signaling levels; the 312 Experiment-4 tables all satisfy S_odd < 2 with zero corrected excess, so the negative result is not attributable to ICC-induced blindness. (ii) A product-distribution theorem establishing that the zero excess of independent scoring is a mathematical necessity, not an empirical finding. (iii) A structural reproduction of the Lo et al. construction with a no-model null: uniformly random marginals without any language model already reach a CbD-contextual rate of 0.667 (vs. 0.711 reported), and the real BERT/RoBERTa rates are largely indistinguishable from a marginal-matched sign-randomized null at substantive effect sizes. (iv) A multi-model audit across six models from four families: raw S_odd > 2 appears (max 2.773) but is entirely explained by direct influence, with zero corrected excess across all 144 tables and on the powered subset. The package includes the manuscript (LaTeX source and PDF), figures, reproduction code, and data (including the raw joint probabilities from which the per-table metrics are regenerated and cross-checked). Central thesis: reading AI quantum-like behavior as non-classicality requires exceeding not the raw score but the combination of classical candidate-geometry baselines, direct-influence correction, detector-power certification, and a no-model null. This work is neither a full reproduction nor a refutation of Lo et al.; it audits the interpretation of raw contextual-instance counts. Manuscript language: English. Code is released under the MIT License; text and figures under CC BY 4.0. Note: Parts of the manuscript were linguistically and structurally refined with the assistance of AI-based tools.All scientific content, analysis, and conclusions are the author's own.
It is well known that in quantum mechanics we cannot always define consistently properties that are context independent. Many approaches exist to describe contextual properties, such as Contextuality by Default (CbD), sheaf theory, topos theory, and non-standard or signed probabilities. In this paper, we propose a treatment of contextual properties that is specific to quantum mechanics, as it relies on the relationship between contextuality and indistinguishability. In particular, we propose that if we assume the ontological thesis that quantum particles or properties can be indistinguishable yet different, no contradiction arising from a Kochen–Specker-type argument appears: when we repeat an experiment, we are in reality performing an experiment measuring a property that is indistinguishable from the first, but not the same. We will discuss how the consequences of this move may help us understand quantum contextuality.