Here we consider some well-known facts in syntax from a physics perspective, allowing us to establish equivalences between both fields with many consequences. Mainly, we observe that the operation MERGE, put forward by N. Chomsky in 1995, can be interpreted as a physical information coarse-graining. Thus, MERGE in linguistics entails information renormalization in physics, according to different time scales. We make this point mathematically formal in terms of language models. In this setting, MERGE amounts to a probability tensor implementing a coarse-graining, akin to a probabilistic context-free grammar. The probability vectors of meaningful sentences are given by stochastic tensor networks (TN) built from diagonal tensors and which are mostly loop-free, such as Tree Tensor Networks and Matrix Product States, thus being computationally very efficient to manipulate. We show that this implies the polynomially-decaying (long-range) correlations experimentally observed in language, and also provides arguments in favour of certain types of neural networks for language processing. Moreover, we show how to obtain such language models from quantum states that can be efficiently prepared on a quantum computer, and use this to find bounds on the perplexity of the probability distribution of words in a sentence. Implications of our results are discussed across several ambits.
Here we consider some well-known facts in syntax from a physics perspective, allowing us to establish equivalences between both fields with many consequences. Mainly, we observe that the operation MERGE, put forward by N. Chomsky in 1995, can be interpreted as a physical information coarse-graining. Thus, MERGE in linguistics entails information renormalization in physics, according to different time scales. We make this point mathematically formal in terms of language models. In this setting, MERGE amounts to a probability tensor implementing a coarse-graining, akin to a probabilistic context-free grammar. The probability vectors of meaningful sentences are given by stochastic tensor networks (TN) built from diagonal tensors and which are mostly loop-free, such as Tree Tensor Networks and Matrix Product States, thus being computationally very efficient to manipulate. We show that this implies the polynomially-decaying (long-range) correlations experimentally observed in language, and also provides arguments in favour of certain types of neural networks for language processing. Moreover, we show how to obtain such language models from quantum states that can be efficiently prepared on a quantum computer, and use this to find bounds on the perplexity of the probability distribution of words in a sentence. Implications of our results are discussed across several ambits.
Abstract Language is not fixed. It is entangled, probabilistic, and observer-dependent. This paper introduces a new paradigm in linguistic theory named "Quantum Linguistics," inspired by the principles of Quantum Physics. Quantum physics and Linguistics have long been regarded as unrelated disciplines,describing nature at the inanimate microlevel on the one hand and language and its scientific study on the other hand. Over the past decades the linguistics has succeeded in providing ever more and refined explanations of generative grammar. Since quantum physics and linguistics deal with the symbolic, rule-governed structure of human language and the non-classical, probabilistic behaviour of subatomic particles, respectively, they have historically been seen as two separate disciplines. This framework allows modeling of polysemy, ambiguity, free word order, and argument structure in a unified, mathematically rigorous manner.To operationalize the model, we implement a Quantum Syntax-Semantics Integration (QSSI) pipeline, which encodes sentences into vector states, applies quantum operations for compositional syntax and semantics, and produces contextually resolved meanings through measurement. Two variants of the pipeline are considered: a classical simulation for experimental validation and a quantum-simulated version for future hardware implementation. The linguistic theory with quantum principles such as superposition, entanglement, and measurement, our model provides a novel probabilistic and context-dependent approach to understand language, explore new insights into human cognitive-linguistic processes, semantic ambiguity observation, and advanced natural language processing applications.
Modeling joint probability distributions over sequences has been studied from many perspectives. The physics community developed matrix product states, a tensor-train decomposition for probabilistic modeling, motivated by the need to tractably model many-body systems. But similar models have also been studied in the stochastic processes and weighted automata literature, with little work on how these bodies of work relate to each other. We address this gap by showing how stationary or uniform versions of popular quantum tensor network models have equivalent representations in the stochastic processes and weighted automata literature, in the limit of infinitely long sequences. We demonstrate several equivalence results between models used in these three communities: (i) uniform variants of matrix product states, Born machines and locally purified states from the quantum tensor networks literature, (ii) predictive state representations, hidden Markov models, norm-observable operator models and hidden quantum Markov models from the stochastic process literature,and (iii) stochastic weighted automata, probabilistic automata and quadratic automata from the formal languages literature. Such connections may open the door for results and methods developed in one area to be applied in another.
We provide conceptual and mathematical foundations for near-term quantum natural language processing (QNLP), and do so in quantum computer scientist friendly terms. We opted for an expository presentation style, and provide references for supporting empirical evidence and formal statements concerning mathematical generality. We recall how the quantum model for natural language that we employ canonically combines linguistic meanings with rich linguistic structure, most notably grammar. In particular, the fact that it takes a quantum-like model to combine meaning and structure, establishes QNLP as quantum-native, on par with simulation of quantum systems. Moreover, the now leading Noisy Intermediate-Scale Quantum (NISQ) paradigm for encoding classical data on quantum hardware, variational quantum circuits, makes NISQ exceptionally QNLP-friendly: linguistic structure can be encoded as a free lunch, in contrast to the apparently exponentially expensive classical encoding of grammar. Quantum speed-up for QNLP tasks has already been established in previous work with Will Zeng. Here we provide a broader range of tasks which all enjoy the same advantage. Diagrammatic reasoning is at the heart of QNLP. Firstly, the quantum model interprets language as quantum processes via the diagrammatic formalism of categorical quantum mechanics. Secondly, these diagrams are via ZX-calculus translated into quantum circuits. Parameterisations of meanings then become the circuit variables to be learned. Our encoding of linguistic structure within quantum circuits also embodies a novel approach for establishing word-meanings that goes beyond the current standards in mainstream AI, by placing linguistic structure at the heart of Wittgenstein's meaning-is-context.
This is a book about operational probabilistic theories. The standard approach in such theories is from a time forward perspective. In this book we mostly take a time symmetric perspective. This presents a branding problem. Is this a niche book merely about time symmetry? No. This is a comprehensive book about operational probabilistic theories, but mostly from a time symmetric perspective. In fact, this book consists of (1) a simple book about simple operations having simple causal structure (where all the inputs are before all the outputs), and (2) a complex book about complex operations that can have complicated causal structure (a complex operation is equipped with a causal diagram). For the simple case we are able to show that the time symmetric perspective is equivalent to the time forward perspective. In each book we set up (A) operational probabilistic theories (OPTs) in terms of operations, (B) Operational Quantum Theory (OQT) in terms of operator tensors which correspond to operations, and (C) the theory of Hilbert objects which can be doubled up to give operator tensors. Operations are required to be physical which guarantees that circuits built out of operations have probabilities between 0 and 1 and that certain causality conditions are met. We prove that when we wire together operations the resulting networks are also physical. We model Sorkin's impossible measurements with complex operations and show that physicality prevents anomalous signalling. We develop diagrammatic notation for Hilbert objects. This includes mirrors for doubling up and mirror theorems. We use this framework to prove time symmetric causal dilation theorems for various causal diagrams.
We introduce a class of quantum non-Markovian processes—dubbed process trees—that exhibit polynomially decaying temporal correlations and memory distributed across timescales. This class of processes is described by a tensor network with treelike geometry whose component tensors are (1) causality-preserving maps (superprocesses) and (2) locality-preserving temporal change-of-scale transformations. We show that the long-range correlations in this class of processes tends to originate almost entirely from memory effects and can accommodate genuinely quantum power-law correlations in time. Importantly, this class allows efficient computation of multitime correlation functions. To showcase the potential utility of this model-agnostic class for numerical simulation of physical models, we show how it can efficiently approximate the strong memory dynamics of the paradigmatic spin-boson model, in terms of arbitrary multitime features. In contrast to an equivalently costly matrix-product-operator representation, the ansatz produces a fiducial characterization of the relevant physics. Finally, leveraging 2D tensor-network renormalization-group methods, we detail an algorithm for deriving a process tree from an underlying Hamiltonian via the Feynmann-Vernon influence functional. Our work lays the foundation for the development of more efficient numerical techniques in the field of strongly interacting open quantum systems, as well as the theoretical development of a temporal renormalization-group scheme. Published by the American Physical Society 2024
Natural language processing (NLP) is at the forefront of great advances in contemporary AI, and it is arguably one of the most challenging areas of the field. At the same time, with the steady growth of quantum hardware and notable improvements towards implementations of quantum algorithms, we are approaching an era when quantum computers perform tasks that cannot be done on classical computers with a reasonable amount of resources. This provides a new range of opportunities for AI, and for NLP specifically. Earlier work has already demonstrated a potential quantum advantage for NLP in a number of manners: (i) algorithmic speedups for search-related or classification tasks, which are the most dominant tasks within NLP, (ii) exponentially large quantum state spaces allow for accommodating complex linguistic structures, (iii) novel models of meaning employing density matrices naturally model linguistic phenomena such as hyponymy and linguistic ambiguity, among others. In this work, we perform the first implementation of an NLP task on noisy intermediate-scale quantum (NISQ) hardware. Sentences are instantiated as parameterised quantum circuits. We encode word-meanings in quantum states and we explicitly account for grammatical structure, which even in mainstream NLP is not commonplace, by faithfully hard-wiring it as entangling operations. This makes our approach to quantum natural language processing (QNLP) particularly NISQ-friendly. Our novel QNLP model shows concrete promise for scalability as the quality of the quantum hardware improves in the near future.