Recently, a Quantum Language Model (QLM) was proposed to model term dependencies upon Quantum Theory (QT) framework and successively applied in Information Retrieval (IR). Nevertheless, QLM's dependency is based on co-occurrences of terms and has not yet taken into account the Quantum Entanglement (QE), which is a key quantum concept and has a significant cognitive implication. In QT, an entangled state can provide a more complete description for the nature of realities, and determine intrinsic correlations of considered objects globally, rather than those co-occurrences on the surface. It is, however, a real challenge to decide and measure QE using the classical statistics of texts in a post-measurement configuration. In order to circumvent this problem, we theoretically prove the connection between QE and statistically Unconditional Pure Dependence (UPD). Since UPD has an implementable deciding algorithm, we can in turn characterize QE by extracting the UPD patterns from texts. This leads to a measurable QE, based on which we further advance the existing QLM framework. We empirically compare our model with related models, and the results demonstrate the effectiveness of our model.
Recently, a Quantum Language Model (QLM) was proposed to model term dependencies upon Quantum Theory (QT) framework and successively applied in Information Retrieval (IR). Nevertheless, QLM's dependency is based on co-occurrences of terms and has not yet taken into account the Quantum Entanglement (QE), which is a key quantum concept and has a significant cognitive implication. In QT, an entangled state can provide a more complete description for the nature of realities, and determine intrinsic correlations of considered objects globally, rather than those co-occurrences on the surface. It is, however, a real challenge to decide and measure QE using the classical statistics of texts in a post-measurement configuration. In order to circumvent this problem, we theoretically prove the connection between QE and statistically Unconditional Pure Dependence (UPD). Since UPD has an implementable deciding algorithm, we can in turn characterize QE by extracting the UPD patterns from texts. This leads to a measurable QE, based on which we further advance the existing QLM framework. We empirically compare our model with related models, and the results demonstrate the effectiveness of our model.
Quantum language models (QLMs) in which words are modeled as a quantum superposition of sememes have demonstrated a high level of model transparency and good post-hoc interpretability. Nevertheless, in the current literature, word sequences are basically modeled as a classical mixture of word states, which cannot fully exploit the potential of a quantum probabilistic description. A quantum-inspired neural network (NN) module is yet to be developed to explicitly capture the nonclassical correlations within the word sequences. We propose a NN model with a novel entanglement embedding (EE) module, whose function is to transform the word sequence into an entangled pure state representation. Strong quantum entanglement, which is the central concept of quantum information and an indication of parallelized correlations among the words, is observed within the word sequences. The proposed QLM with EE (QLM-EE) is proposed to implement on classical computing devices with a quantum-inspired NN structure, and numerical experiments show that QLM-EE achieves superior performance compared with the classical deep NN models and other QLMs on question answering (QA) datasets. In addition, the post-hoc interpretability of the model can be improved by quantifying the degree of entanglement among the word states.
Traditional information retrieval (IR) models use bag-of-words as the basic representation and assume that some form of independence holds between terms. Representing term dependencies and defining a scoring function capable of integrating such additional evidence is theoretically and practically challenging. Recently, Quantum Theory (QT) has been proposed as a possible, more general framework for IR. However, only a limited number of investigations have been made and the potential of QT has not been fully explored and tested. We develop a new, generalized Language Modeling approach for IR by adopting the probabilistic framework of QT. In particular, quantum probability could account for both single and compound terms at once without having to extend the term space artificially as in previous studies. This naturally allows us to avoid the weight-normalization problem, which arises in the current practice by mixing scores from matching compound terms and from matching single terms. Our model is the first practical application of quantum probability to show significant improvements over a robust bag-of-words baseline and achieves better performance on a stronger non bag-of-words baseline.
Purpose The paper aims to provide a semiotic interpretation of the role played by entanglement in quantum-based models aimed to information retrieval and suggests possible improvements. Actual models are capable of retrieving documents relevant to a query composed of a keyword and its acceptation expressed by a given context. The paper also considers some analogies between this technique and quantum-based approaches in other disciplines to discuss the consequence of this quantum turn, as epistemology and philosophy of language are concerned. Design/methodology/approach We use quantum geometry to design a formal model for textual semiotics. In particular, the authors refer to Greimas’s work on semantics and information theory, to Eco’s writings on semantic memory and to Lotman’s work on a cybernetic notion of culture. Findings Quantum approaches imply a particular point of view on meaning. Meaning is not a real, positive quality of a given word. It is a net of relations constructed in the text, whose value is progressively determined during the reading process. Furthermore, reading is not a neutral operation: to read is to determine meaning. If it is said that, from a general semiotic point of view, meaning is stored in quantum semantic memories and is read/written by semantic machines, then the operation of “reading/writing” is analogous to the operation of measuring in quantum theory: in other terms, meaning is a value, and this implies an instance (not necessarily human) according to which values are valuable. Research limitations/implications The authors are not proposing a complete quantum semantics. At the present, quantum information retrieval can detect the presence of semantic relations. The authors suggest a way to characterize them, leaving open the problem on how to formalize the document as a vector in four-state semantic space. Practical implications A quantum turn shows deep semiotic implications on the approach to language, which shows an immanent semantic organization not reducible to syntax and morphology. This organization is probabilistic and indeterministic and explains to what extent text fixes the meaning of its lexical units. Social implications In the authors’ perspective, signification is not the exclusivity of a human subject. Criticizing Turing test, the great semiotic and cybernetic scholar Jurij Lotman wrote that if we identify “intelligent” and “human”, we raise the failings of an actual form of intelligence to the rank of an essential characteristic. On this line, meaning is considered as a feature of social, artificial and biological systems. Originality/value The adoption of quantum formalism seems in line with cybernetic framework, involving a probabilistic, non-cartesian point of view on meaning aimed to critically discuss the human–machine relation. Furthermore, Quantum theory (QT) implies a phenomenological point of view on the conditions of possibility of meaning.
The mathematical formalism of quantum theory has been successfully used in human cognition to model decision processes and to deliver representations of human knowledge. As such, quantum cognition inspired tools have improved technologies for Natural Language Processing and Information Retrieval. In this paper, we overview the quantum cognition approach developed in our Brussels team during the last two decades, specifically our identification of quantum structures in human concepts and language, and the modeling of data from psychological and corpus-text-based experiments. We discuss our quantum-theoretic framework for concepts and their conjunctions/disjunctions in a Fock-Hilbert space structure, adequately modeling a large amount of data collected on concept combinations. Inspired by this modeling, we put forward elements for a quantum contextual and meaning-based approach to information technologies in which 'entities of meaning' are inversely reconstructed from texts, which are considered as traces of these entities' states.
The recently proposed quantum language model (QLM) aimed at a principled approach to modeling term dependency by applying the quantum probability theory. The latest development for a more effective QLM has adopted word embeddings as a kind of global dependency information and integrated the quantum-inspired idea in a neural network architecture. While these quantum-inspired LMs are theoretically more general and also practically effective, they have two major limitations. First, they have not taken into account the interaction among words with multiple meanings, which is common and important in understanding natural language text. Second, the integration of the quantum-inspired LM with the neural network was mainly for effective training of parameters, yet lacking a theoretical foundation accounting for such integration. To address these two issues, in this paper, we propose a Quantum Many-body Wave Function (QMWF) inspired language modeling approach. The QMWF inspired LM can adopt the tensor product to model the aforesaid interaction among words. It also enables us to reveal the inherent necessity of using Convolutional Neural Network (CNN) in QMWF language modeling. Furthermore, our approach delivers a simple algorithm to represent and match text/sentence pairs. Systematic evaluation shows the effectiveness of the proposed QMWF-LM algorithm, in comparison with the state of the art quantum-inspired LMs and a couple of CNN-based methods, on three typical Question Answering (QA) datasets.
A lexeme can be represented as a vector in a Hilbert space. Each element of the vector represents the weight of a semantic relation with a second vector: the context of a second lexeme. Two query operators are defined in a way that they attribute the value +1 to the component of the state that corresponds to the word meaning we are interested in, and -1 in the orthogonal direction. Entangled states have been used to detect semantic relationships in Information Retrieval (IR). In particular, since the first operator corresponds to negation and the conditional gate corresponds to an implication in classical logic, they can represent two basic semantic relationships : hyponymy (e.g. genus-species) and antonymy (e.g. masculine-feminine). In fact, according to Semantics, if A and B are antonymic terms, then A is an hyponym of (i.e. implies) not-B as in the Gremias semiotic square. We underline how the relations of a given semantic universe, and the corresponding result of our operators, depends only on the considered text. Among others, this analogy suggests the application of quantum-based models to language and IR.