Sheaf-Theoretic Semantics and Quantum Contextuality in Large Language Models: A Unified Categorical Framework for Understanding Semantic Coherence and Hallucination Phenomena
Life of Research AbstractThis paper introduces a novel theoretical framework for understanding semantic processing in Large LanguageModels through the lens of sheaf theory and quantum contextuality. We propose that the semantic structure ofLLM-generated text can be rigorously modeled as a sheaf over the topological space of prompt contexts, wherelocal semantic sections may fail to glue into globally consistent interpretations. This mathematical formalizationreveals a deep structural parallel between LLM behavior and quantum mechanical systems exhibitingcontextuality, as characterized by the Kochen-Specker theorem. Our central thesis posits that hallucinationphenomena in LLMs arise fundamentally from cohomological obstructions to the existence of global semanticsections, analogous to how quantum systems exhibit measurement outcomes that cannot be explained by preexisting hidden variables. We develop a complete categorical framework using topos-theoretic methods, introduceformal definitions of semantic presheaves and their associated Grothendieck topologies, and demonstrate howCech cohomology groups can quantify the degree of semantic inconsistency in model outputs. This work bridgesabstract mathematics with practical AI interpretability, offering new diagnostic tools for understanding when andwhy language models produce incoherent or factually inconsistent outputs.
Life of Research AbstractThis paper introduces a novel theoretical framework for understanding semantic processing in Large LanguageModels through the lens of sheaf theory and quantum contextuality. We propose that the semantic structure ofLLM-generated text can be rigorously modeled as a sheaf over the topological space of prompt contexts, wherelocal semantic sections may fail to glue into globally consistent interpretations. This mathematical formalizationreveals a deep structural parallel between LLM behavior and quantum mechanical systems exhibitingcontextuality, as characterized by the Kochen-Specker theorem. Our central thesis posits that hallucinationphenomena in LLMs arise fundamentally from cohomological obstructions to the existence of global semanticsections, analogous to how quantum systems exhibit measurement outcomes that cannot be explained by preexisting hidden variables. We develop a complete categorical framework using topos-theoretic methods, introduceformal definitions of semantic presheaves and their associated Grothendieck topologies, and demonstrate howCech cohomology groups can quantify the degree of semantic inconsistency in model outputs. This work bridgesabstract mathematics with practical AI interpretability, offering new diagnostic tools for understanding when andwhy language models produce incoherent or factually inconsistent outputs.
The main subjects of this thesis are non-locality and contextuality, two fundamental features of quantum mechanics that constitute valuable resources for quantum computation. Our analysis is based on Abramsky & Brandenburger's sheaf theoretic framework, which captures both these phenomena in a unified treatment and in a very general setting. This high-level description transcends quantum physics and allows to precisely characterise the notion of contextuality as the apparent paradox realised by data being locally consistent but globally inconsistent. More specifically, we aim to develop a deeper understanding of so-called logical forms of contextuality, i.e. situations where the phenomenon can be witnessed using purely logical arguments, disregarding probabilities. The sheaf theoretic description of logical contextuality has recently inspired the development of a topological treatment of the phenomenon based on sheaf cohomology. In this thesis, we embark on a detailed analysis of the cohomology of contextuality, exposing key shortcomings in the current methods, and introducing an (almost) complete cohomological characterisation of logical forms of contextuality. More specifically, we show that, in its current formulation, sheaf cohomology does not constitute a complete invariant for contextuality, not even in its strongest forms, and that higher cohomology groups cannot be used to study the phenomenon. Then, we solve these issues by introducing a novel construction, which derives refined versions of the presheaves describing empirical models to expose their deeper extendability properties, resulting in a sheaf cohomological invariant which is applicable to the vast majority of empirical models, and conjectured to work universally. We propose a general theory of contextual semantics using the language of valuation algebras. In particular, we give a general definition of contextual behaviour as a fundamental gap between local agreement and global disagreement of information sources. Not only does this formalism aptly capture and generalise the known instances of contextuality beyond quantum theory, but it also provides inspiration for further applications of the phenomenon, and paves the way for the transfer of results and techniques between different fields. We give a prime example of this potential by developing faster algorithms to detect contextuality based on mainstream methods of generic inference. Finally, we turn our attention back to instances of contextuality in quantum physics, and study strong contextuality in multi-qubit states. We give a complete combinatorial characterisation of All-vs-Nothing proofs of strong contextuality in stabiliser quantum mechanics. This allows to produce the complete list of all stabiliser states exhibiting this kind of contextuality, which consitututes an important resource in certain models of quantum computation. Then, we extend our search for strongly contextual behaviour beyond stabiliser states, and identify the minimum quantum resources needed to realise strong non-locality. Additional results include a partial classification of strongly non-local models comprised of three-qubit states and local projective measurements, and the introduction of a new infinite family of strongly non-local three-qubit states.
Semantic communication (SC) enables bandwidth-efficient coordination in multi-agent systems by transmitting meaning rather than raw bits. However, when agents employ heterogeneous sensing modalities and AI architectures, perfect bit-level transmission no longer guarantees mutual understanding. Although deep learning methods for semantic compression have advanced, the information-theoretic limits of semantic alignment under heterogeneity remain poorly understood. Notably, semantic ambiguity shares the same mathematical structure as quantum contextuality, as both arise from cohomological obstructions, motivating a quantum formulation of SC. In this paper, an information-theoretic framework for quantum semantic communication is proposed using sheaf cohomology. Multi-agent semantic networks are modeled as quantum sheaves, where agents meaning spaces are Hilbert spaces connected by quantum channels. The first sheaf cohomology group is shown to characterize irreducible semantic ambiguity, representing a fundamental obstruction to alignment that no local processing can resolve. The minimum communication rate required for semantic alignment is proven to scale with the logarithm of the dimension of the cohomological space, establishing a semantic analog of Shannon limits. For entanglement-assisted channels, the achievable capacity is shown to strictly exceed classical bounds, with each shared ebit reducing the required classical communication by one bit, providing a rigorous interpretation of shared context. Additionally, quantum contextuality is shown to reduce cohomological obstructions, and a duality between quantum discord and integrated semantic information is established, linking quantum correlations to irreducible semantic content. This framework provides rigorous foundations for quantum-enhanced semantic communication in autonomous multi-agent systems.
Contextuality has become a cornerstone of modern foundational approaches to quantum physics. This paper introduces contextual mathematics (CM), a categorical framework for context-indexed observations and agent-dependent evaluation. Contexts are organized into a category Ctx, and observation is modeled as a contravariant bifunctor into an epistemic category E; context refinements induce restriction maps, and a Grothendieck construction organizes evaluations over agent–context pairs. We show how the established sheaf-theoretic account of Kochen–Specker contextuality (Abramsky–Brandenburger) can be represented inside this framework, recovering non-existence of a global section as the relevant obstruction. We then formulate, as comparison problems rather than established equivalences, several further candidate obstruction theories that a fully developed CM might connect to this sheaf-theoretic core: homological extension classes (Ext1 ), Kan-extension reconstruction, deformation-quantization cocycles, and device-dependent restriction in algebraic quantum field theory (AQFT) over globally hyperbolic spacetimes. We are explicit about which of these connections are proved, which are illustrative analogies, and which remain open.
Contextuality is a distinguishing feature of quantum mechanics, and there is evidence that it is a necessary condition for quantum advantage in the context of magic state distillation. To harness it, researchers have been asking whether similar phenomena arise in other domains. The answer has been affirmative, e.g. in behavioural sciences. However, one has to adopt frameworks that take some degree of signalling into account. Two such frameworks exist: (i) a signalling-corrected sheaf-theoretic model and (ii) the Contextuality-by-Default (CbD) framework. This paper provides the first large-scale experimental evidence for contextuality in the large language model BERT. We constructed a linguistic schema modelled over a contextual quantum scenario, instantiated it in the Simple English Wikipedia, and extracted probability distributions for the instances. This led to the discovery of 77 118 sheaf-contextual and 36 938 948 CbD contextual instances. We prove that these contextual instances arise from semantically similar words by deriving an equation that relates degrees of contextuality to the Euclidean distance of BERT’s embedding vectors. Our linguistic schema is a variant of the coreference resolution challenge. These results suggest that quantum methods may be advantageous in the field of natural language processing.
We use the mathematical language of sheaf theory to give a unified treatment of non-locality and contextuality, in a setting that generalizes the familiar probability tables used in non-locality theory to arbitrary measurement covers ; this includes Kochen–Specker configurations and more. We show that contextuality, and non-locality as a special case, correspond exactly to obstructions to the existence of global sections . We describe a linear algebraic approach to computing these obstructions, which allows a systematic treatment of arguments for non-locality and contextuality. We distinguish a proper hierarchy of strengths of no-go theorems, and show that three leading examples—due to Bell, Hardy and Greenberger, Horne and Zeilinger, respectively—occupy successively higher levels of this hierarchy. A general correspondence is shown between the existence of local hidden-variable realizations using negative probabilities, and no-signalling; this is based on a result showing that the linear subspaces generated by the non-contextual and no-signalling models, over an arbitrary measurement cover, coincide. Maximal non-locality is generalized to maximal contextuality, and characterized in purely qualitative terms, as the non-existence of global sections in the support. A general setting is developed for the Kochen–Specker-type results, as generic, model-independent proofs of maximal contextuality, and a new combinatorial condition is given, which generalizes the 'parity proofs' commonly found in the literature. We also show how our abstract setting can be represented in quantum mechanics. This leads to a strengthening of the usual no-signalling theorem, which shows that quantum mechanics obeys no-signalling for arbitrary families of commuting observables, not just those represented on different factors of a tensor product.
Large Language Models (LLMs) excel in natural language tasks, but their high-dimensional embedding spaces pose significant interpretability challenges. Current approaches often linearize these spaces, overlooking the complex dynamics inherent in Transformer architectures. This article proposes a quantum framework to analyze LLM representations, leveraging quantum mechanical tools to explore semantic relationships and contextual influences. We introduce a layered hierarchy of semantic spaces and demonstrate that a classical LLM embedding system has an exact quantum mechanical analogue. Using this analogue, we model phenomena such as the modulation of Semantic Noise, the emergence of hallucinations via quantum tunneling, and the formation of stable semantic representations as soliton solutions. Furthermore, we present a simple quantum circuit design, demonstrating the possibility of using quantum computers to probe, analyze and go beyond real-valued LLM embedding spaces, potentially revealing structural information and relationships not readily accessible through classical techniques. This perspective enhances our understanding of LLM representations, leading to improved methods for analyzing and controlling LLM behavior, and supporting research into more efficient, reliable, and trustworthy AI systems.