Ambiguities of natural language do not preclude us from using it and context helps in getting ideas across. They, nonetheless, pose a key challenge to the development of competent machines to understand natural language and use it as humans do. Contextuality is an unparalleled phenomenon in quantum mechanics, where different mathematical formalisms have been put forwards to understand and reason about it. In this paper, we construct a schema for anaphoric ambiguities that exhibits quantum-like contextuality. We use a recently developed criterion of sheaf-theoretic contextuality that is applicable to signalling models. We then take advantage of the neural word embedding engine BERT to instantiate the schema to natural language examples and extract probability distributions for the instances. As a result, plenty of sheaf-contextual examples were discovered in the natural language corpora BERT utilises. Our hope is that these examples will pave the way for future research and for finding ways to extend applications of quantum computing to natural language processing.
Ambiguities of natural language do not preclude us from using it and context helps in getting ideas across. They, nonetheless, pose a key challenge to the development of competent machines to understand natural language and use it as humans do. Contextuality is an unparalleled phenomenon in quantum mechanics, where different mathematical formalisms have been put forwards to understand and reason about it. In this paper, we construct a schema for anaphoric ambiguities that exhibits quantum-like contextuality. We use a recently developed criterion of sheaf-theoretic contextuality that is applicable to signalling models. We then take advantage of the neural word embedding engine BERT to instantiate the schema to natural language examples and extract probability distributions for the instances. As a result, plenty of sheaf-contextual examples were discovered in the natural language corpora BERT utilises. Our hope is that these examples will pave the way for future research and for finding ways to extend applications of quantum computing to natural language processing.
Contextuality is a distinguishing feature of quantum mechanics, and there is evidence that it is a necessary condition for quantum advantage in the context of magic state distillation. To harness it, researchers have been asking whether similar phenomena arise in other domains. The answer has been affirmative, e.g. in behavioural sciences. However, one has to adopt frameworks that take some degree of signalling into account. Two such frameworks exist: (i) a signalling-corrected sheaf-theoretic model and (ii) the Contextuality-by-Default (CbD) framework. This paper provides the first large-scale experimental evidence for contextuality in the large language model BERT. We constructed a linguistic schema modelled over a contextual quantum scenario, instantiated it in the Simple English Wikipedia, and extracted probability distributions for the instances. This led to the discovery of 77 118 sheaf-contextual and 36 938 948 CbD contextual instances. We prove that these contextual instances arise from semantically similar words by deriving an equation that relates degrees of contextuality to the Euclidean distance of BERT’s embedding vectors. Our linguistic schema is a variant of the coreference resolution challenge. These results suggest that quantum methods may be advantageous in the field of natural language processing.
Language is contextual as meanings of words are dependent on their contexts. Contextuality is, concomitantly, a well-defined concept in quantum mechanics where it is considered a major resource for quantum computations. We investigate whether natural language exhibits any of the quantum mechanics' contextual features. We show that meaning combinations in ambiguous phrases can be modelled in the sheaf-theoretic framework for quantum contextuality, where they can become possibilistically contextual. Using the framework of Contextuality-by-Default (CbD), we explore the probabilistic variants of these and show that CbD-contextuality is also possible.
Quantum computing has demonstrated computational advantages over classical computing, yet its applications to natural language processing (NLP) remain in their early stages. Quantum contextuality, a fundamental feature of quantum mechanics, has been identified as a resource for achieving these advantages. Meanwhile, ambiguities in human languages present challenges in building systems capable of understanding and generating natural language. Many of these ambiguities can be addressed by considering the context in which they arise. This thesis investigates the connection between quantum contextuality and ambiguities in natural language. It introduces a framework that models ambiguities in language as a form of measurement, analogous to physical experiments. The findings reveal that contextuality is present in natural language and that its degree is linked to the reasoning capabilities of large language models (LLMs).
Formal languages are essential for computer programming and are constructed to be easily processed by computers. In contrast, natural languages are much more challenging and instigated the field of Natural Language Processing (NLP). One major obstacle is the ubiquity of ambiguities. Recent advances in NLP have led to the development of large language models, which can resolve ambiguities with high accuracy. At the same time, quantum computers have gained much attention in recent years as they can solve some computational problems faster than classical computers. This new computing paradigm has reached the fields of machine learning and NLP, where hybrid classical-quantum learning algorithms have emerged. However, more research is needed to identify which NLP tasks could benefit from a genuine quantum advantage. In this thesis, we applied formalisms arising from foundational quantum mechanics, such as contextuality and causality, to study ambiguities arising from linguistics. By doing so, we also reproduced psycholinguistic results relating to the human disambiguation process. These results were subsequently used to predict human behaviour and outperformed current NLP methods.
The main subjects of this thesis are non-locality and contextuality, two fundamental features of quantum mechanics that constitute valuable resources for quantum computation. Our analysis is based on Abramsky & Brandenburger's sheaf theoretic framework, which captures both these phenomena in a unified treatment and in a very general setting. This high-level description transcends quantum physics and allows to precisely characterise the notion of contextuality as the apparent paradox realised by data being locally consistent but globally inconsistent. More specifically, we aim to develop a deeper understanding of so-called logical forms of contextuality, i.e. situations where the phenomenon can be witnessed using purely logical arguments, disregarding probabilities. The sheaf theoretic description of logical contextuality has recently inspired the development of a topological treatment of the phenomenon based on sheaf cohomology. In this thesis, we embark on a detailed analysis of the cohomology of contextuality, exposing key shortcomings in the current methods, and introducing an (almost) complete cohomological characterisation of logical forms of contextuality. More specifically, we show that, in its current formulation, sheaf cohomology does not constitute a complete invariant for contextuality, not even in its strongest forms, and that higher cohomology groups cannot be used to study the phenomenon. Then, we solve these issues by introducing a novel construction, which derives refined versions of the presheaves describing empirical models to expose their deeper extendability properties, resulting in a sheaf cohomological invariant which is applicable to the vast majority of empirical models, and conjectured to work universally. We propose a general theory of contextual semantics using the language of valuation algebras. In particular, we give a general definition of contextual behaviour as a fundamental gap between local agreement and global disagreement of information sources. Not only does this formalism aptly capture and generalise the known instances of contextuality beyond quantum theory, but it also provides inspiration for further applications of the phenomenon, and paves the way for the transfer of results and techniques between different fields. We give a prime example of this potential by developing faster algorithms to detect contextuality based on mainstream methods of generic inference. Finally, we turn our attention back to instances of contextuality in quantum physics, and study strong contextuality in multi-qubit states. We give a complete combinatorial characterisation of All-vs-Nothing proofs of strong contextuality in stabiliser quantum mechanics. This allows to produce the complete list of all stabiliser states exhibiting this kind of contextuality, which consitututes an important resource in certain models of quantum computation. Then, we extend our search for strongly contextual behaviour beyond stabiliser states, and identify the minimum quantum resources needed to realise strong non-locality. Additional results include a partial classification of strongly non-local models comprised of three-qubit states and local projective measurements, and the introduction of a new infinite family of strongly non-local three-qubit states.
Understanding the fundamental mechanisms governing the production of meaning in the processing of natural language is critical for designing safe, thoughtful, engaging, and empowering human-agent interactions. If meaning is constituted rather than retrieved, then the search for context-independent features or circuits in the pursuit of mechanistic interpretability may be fundamentally limited. Experiments in cognitive science and social psychology have demonstrated that human semantic processing exhibits contextuality more consistent with quantum logical mechanisms than classical Boolean theories, and recent works have found similar results in large language models---in particular, clear violations of the Bell inequality in experiments of contextuality during interpretation of ambiguous expressions. In this work, we explore the CHSH ∣S∣ parameter---the metric associated with the inequality---across the inference parameter space of models spanning four orders of magnitude in scale and cross-reference our findings with MMLU, hallucination rate, and nonsense detection benchmarks. We find that the interquartile range of the ∣S∣ distribution is completely orthogonal to all external benchmarks, while overall violation rate shows weak anticorrelation with all three benchmarks. We investigate how ∣S∣ varies with sampling parameters and word order, and discuss the information-theoretic constraints that genuine contextuality imposes on prompt injection defenses and its human analogue, whereby careful construction and maintenance of social contextuality can be carried out at scale, shaping the space of possible interpretations before any particular one is reached. We consider the implications for mechanistic interpretability and how genuine contextuality sets an information-theoretic bound on the decomposability of semantic processing.