This paper seeks to model human language by the mathematical framework of quantum physics. With the well-designed mathematical formulations in quantum physics, this framework unifies different linguistic units in a single complex-valued vector space, e.g. words as particles in quantum states and sentences as mixed systems. A complex-valued network is built to implement this framework for semantic matching. With well-constrained complex-valued components, the network admits interpretations to explicit physical meanings. The proposed complex-valued network for matching (CNM) achieves comparable performances to strong CNN and RNN baselines on two benchmarking question answering (QA) datasets.
This paper seeks to model human language by the mathematical framework of quantum physics. With the well-designed mathematical formulations in quantum physics, this framework unifies different linguistic units in a single complex-valued vector space, e.g. words as particles in quantum states and sentences as mixed systems. A complex-valued network is built to implement this framework for semantic matching. With well-constrained complex-valued components, the network admits interpretations to explicit physical meanings. The proposed complex-valued network for matching (CNM) achieves comparable performances to strong CNN and RNN baselines on two benchmarking question answering (QA) datasets.
Language modeling is essential in Natural Language Processing and Information Retrieval related tasks. After the statistical language models, Quantum Language Model (QLM) has been proposed to unify both single words and compound terms in the same probability space without extending term space exponentially. Although QLM achieved good performance in ad hoc retrieval, it still has two major limitations: (1) QLM cannot make use of supervised information, mainly due to the iterative and non-differentiable estimation of the density matrix, which represents both queries and documents in QLM. (2) QLM assumes the exchangeability of words or word dependencies, neglecting the order or position information of words. This article aims to generalize QLM and make it applicable to more complicated matching tasks (e.g., Question Answering) beyond ad hoc retrieval. We propose a complex-valued neural network-based QLM solution called C-NNQLM to employ an end-to-end approach to build and train density matrices in a light-weight and differentiable manner, and it can therefore make use of external well-trained word vectors and supervised labels. Furthermore, C-NNQLM adopts complex-valued word vectors whose phase vectors can directly encode the order (or position) information of words. Note that complex numbers are also essential in the quantum theory. We show that the real-valued NNQLM (R-NNQLM) is a special case of C-NNQLM. The experimental results on the QA task show that both R-NNQLM and C-NNQLM achieve much better performance than the vanilla QLM, and C-NNQLM’s performance is on par with state-of-the-art neural network models. We also evaluate the proposed C-NNQLM on text classification and document retrieval tasks. The results on most datasets show that the C-NNQLM can outperform R-NNQLM, which demonstrates the usefulness of the complex representation for words and sentences in C-NNQLM.
Multi-task learning has demonstrated remarkable success across a broad spectrum of natural language processing tasks, particularly with neural network-based methods. Despite these advances, a fundamental gap remains in explaining the relationship between task-relatedness and model effectiveness. To address this issue, we propose a novel approach for implicitly modeling task-relatedness by leveraging a quantum physical mathematical framework. In this paper, we introduce a complex-valued neural network designed to encapsulate and analyze task-relatedness. Within this framework, sentences originating from diverse tasks are encoded as mixed quantum systems, represented on a meticulously defined Semantic Hilbert Space. This allows the network to interpret inter-task relationships through the explicit physical semantics of well-constrained components grounded in quantum probability theory. By adhering to these rigorous principles, our model not only establishes a robust method for quantifying task-relatedness but also fosters a deeper, self-explanatory understanding of the underlying processes. To validate the efficacy of our approach, we conducted extensive experiments across five benchmark text classification tasks. The results demonstrate both the superior performance and the interpretability of the proposed model, highlighting its potential as a self-explanatory system for multi-task learning in NLP.
In linguistics, the uncertainty of context due to polysemy is widespread, which attracts much attention. Quantum-inspired complex word embedding based on Hilbert space plays an important role in natural language processing (NLP), which fully leverages the similarity between quantum states and word tokens. A word containing multiple meanings could correspond to a single quantum particle which may exist in several possible states, and a sentence could be analogous to the quantum system where particles interfere with each other. Motivated by quantum-inspired complex word embedding, interpretable complex-valued word embedding (ICWE) is proposed to design two end-to-end quantum-inspired deep neural networks (ICWE-QNN and CICWE-QNN representing convolutional complex-valued neural network based on ICWE) for binary text classification. They have the proven feasibility and effectiveness in the application of NLP and can solve the problem of text information loss in CE-Mix [1] model caused by neglecting the important linguistic features of text, since linguistic feature extraction is presented in our model with deep learning algorithms, in which gated recurrent unit (GRU) extracts the sequence information of sentences, attention mechanism makes the model focus on important words in sentences and convolutional layer captures the local features of projected matrix. The model ICWE-QNN can avoid random combination of word tokens and CICWE-QNN fully considers textual features of the projected matrix. Experiments conducted on five benchmarking classification datasets demonstrate our proposed models have higher accuracy than the compared traditional models including CaptionRep BOW, DictRep BOW and Paragram-Phrase, and they also have great performance on F1-score. Eespecially, CICWE-QNN model has higher accuracy than the quantum-inspired model CE-Mix as well for four datasets including SST, SUBJ, CR and MPQA. It is a meaningful and effictive exploration to design quantum-inspired deep neural networks to promote the performance of text classification.
Real-valued word vectors (RWV) often perform poorly when showing semantic relations such as the ambiguity of the word itself (“spring” can refer to a kind of season or fountain) and the polarity between words (words “love” and “hate” have a high degree of similarity due to their similar distribution). Although dynamic word vectors alleviate these problems, such methods are still lacking interpretability. Inspired by quantum theory, we believe that basic reasons are insufficient modeling ability of RWV and weak abstract ability of language models. To solve the above problems, we propose to use complex-valued word vectors (CWV) for the representation, which can naturally model semantic relations of words. We also design an explicable lexicon-extended quantum probability neural network (LQPNN) based on the theory of quantum measurement to training CWV. Experiments on three datasets of classification demonstrate the effectiveness of our model, which achieves superior performances.
Recently, the mathematical framework of quantum mechanics has boosted interpretability and performance in a variety of NLP tasks. Since different linguistic units are explicitly weighted by this framework in density matrix, it is also often believed that the models inspired by quantum mechanics afford transparency. However, the relationship between weights of linguistic units and model outputs is uncertain. In this work we test whether this assumption holds by performing extensive experiments in question answering (QA) model to manipulating semantic weights and assessing the resulting differences in its output. Although we observe in some ways that higher weights do correlate with greater effect on model, but most of the data does not support it. i.e., the weight is independent of the prediction, and different weight distributions can produce equivalent result. Our results illustrate that standard quantum-inspired model has interpretable structure but does not accurately model feature importance and requires improvement in semantic representation and model interpretability.
The performance of Feedforward Neural Networks (FNNs) in specific application areas is often higher than the upper limit allowed by (classical) theory, and therefore is called “Alchemy”, that is, the lack of interpretability of FNNs. In this paper, based on the Quantum Weak Measurement theory (QWM), we propose an interpretable network with complex-valued representation to deal with the insufficient interpretability of FNNs and the Semantic Interpretability of Complex Numbers (SICN) from a novel perspective. For the specific role of complex-valued representation in capturing semantic information, we give an in-depth analysis that the real and imaginary parts of complex numbers can simultaneously represent two different types of complementary information. Finally, we propose a QWM inspired interpretable Network for text representation learning (i.e., QWMN) and implement two frameworks for Question Answering (QWMN-QA) and Text Classification (QWMN-TC) respectively. Experimental results prove the feasibility, effectiveness of QWMN.