Real-valued word vectors (RWV) often perform poorly when showing semantic relations such as the ambiguity of the word itself (“spring” can refer to a kind of season or fountain) and the polarity between words (words “love” and “hate” have a high degree of similarity due to their similar distribution). Although dynamic word vectors alleviate these problems, such methods are still lacking interpretability. Inspired by quantum theory, we believe that basic reasons are insufficient modeling ability of RWV and weak abstract ability of language models. To solve the above problems, we propose to use complex-valued word vectors (CWV) for the representation, which can naturally model semantic relations of words. We also design an explicable lexicon-extended quantum probability neural network (LQPNN) based on the theory of quantum measurement to training CWV. Experiments on three datasets of classification demonstrate the effectiveness of our model, which achieves superior performances.
Real-valued word vectors (RWV) often perform poorly when showing semantic relations such as the ambiguity of the word itself (“spring” can refer to a kind of season or fountain) and the polarity between words (words “love” and “hate” have a high degree of similarity due to their similar distribution). Although dynamic word vectors alleviate these problems, such methods are still lacking interpretability. Inspired by quantum theory, we believe that basic reasons are insufficient modeling ability of RWV and weak abstract ability of language models. To solve the above problems, we propose to use complex-valued word vectors (CWV) for the representation, which can naturally model semantic relations of words. We also design an explicable lexicon-extended quantum probability neural network (LQPNN) based on the theory of quantum measurement to training CWV. Experiments on three datasets of classification demonstrate the effectiveness of our model, which achieves superior performances.
Recently, a Quantum Probability Drive Network (QPDN) is proposed to model different levels of semantic units by extending word embedding to complex-valued representation (CR). The extended complex-valued embeddings are still insensitive to polarity causing that they generalize badly in sentiment analysis (SA). To solve it, we propose a method of encoding sentiment information into sentiment words for SA. Attention mechanism and an auxiliary task are introduced to help learn the CR of sentiment words with the help of the sentiment lexicon. We use the amplitude part to represent the distributional information and the phase part to represent the sentiment information of the language. Experiments on three popular SA datasets show that our method is effective.
Language modeling is essential in Natural Language Processing and Information Retrieval related tasks. After the statistical language models, Quantum Language Model (QLM) has been proposed to unify both single words and compound terms in the same probability space without extending term space exponentially. Although QLM achieved good performance in ad hoc retrieval, it still has two major limitations: (1) QLM cannot make use of supervised information, mainly due to the iterative and non-differentiable estimation of the density matrix, which represents both queries and documents in QLM. (2) QLM assumes the exchangeability of words or word dependencies, neglecting the order or position information of words. This article aims to generalize QLM and make it applicable to more complicated matching tasks (e.g., Question Answering) beyond ad hoc retrieval. We propose a complex-valued neural network-based QLM solution called C-NNQLM to employ an end-to-end approach to build and train density matrices in a light-weight and differentiable manner, and it can therefore make use of external well-trained word vectors and supervised labels. Furthermore, C-NNQLM adopts complex-valued word vectors whose phase vectors can directly encode the order (or position) information of words. Note that complex numbers are also essential in the quantum theory. We show that the real-valued NNQLM (R-NNQLM) is a special case of C-NNQLM. The experimental results on the QA task show that both R-NNQLM and C-NNQLM achieve much better performance than the vanilla QLM, and C-NNQLM’s performance is on par with state-of-the-art neural network models. We also evaluate the proposed C-NNQLM on text classification and document retrieval tasks. The results on most datasets show that the C-NNQLM can outperform R-NNQLM, which demonstrates the usefulness of the complex representation for words and sentences in C-NNQLM.
Polysemy is a common phenomenon in linguistics. Quantum-inspired complex word embeddings based on Semantic Hilbert Space play an important role in natural language processing, which may accurately define a genuine probability distribution over the word space. The existing quantum-inspired works manipulate on the real-valued vectors to compose the complex-valued word embeddings, which lack direct complex-valued pre-trained word representations. Motivated by quantum-inspired complex word embeddings, we propose a complex-valued pre-trained word embedding based on density matrices, called Word2State. Unlike the existing static word embeddings, our proposed model can provide non-linear semantic composition in the form of amplitude and phase, which also defines an authentic probabilistic distribution. We evaluate this model on twelve datasets from the word similarity task and six datasets from the relevant downstream tasks. The experimental results on different tasks demonstrate that our proposed pre-trained word embedding can capture richer semantic information and exhibit greater flexibility in expressing uncertainty.
Sentiment analysis aims to study, analyse and identify the sentiment polarity contained in subjective documents. In the realm of natural language processing (NLP), the study of sentiment analysis and its subtask research is a hot topic, which has very important significance. The existing sentiment analysis methods based on sentiment lexicon and machine learning take into account contextual semantic information, but these methods still lack the ability to utilize context information, so they cannot effectively encode context information. Inspired by the concept of density matrix in quantum mechanics, we propose a sentiment analysis method, named Complex-valued Quantum-enhanced Long Short-term Memory Neural Network (CQLSTM). It leverages complex-valued embedding to incorporate more semantic information and utilizes the Complex-valued Quantum-enhanced Long Short-term Memory Neural Network for feature extraction. Specifically, a complex-valued neural network based on density matrix is used to capture interactions between words (i.e., the correlation between words). Additionally, the Complex-valued Quantum-enhanced Long Short-term Memory Neural Network, which is inspired by the quantum measurement theory and quantum long short-term memory neural network, is developed to learn interactions between sentences (i.e., contextual semantic information). This approach effectively encodes semantic dependencies, which reflects the dispersion of words in the embedded space of sentences and comprehensively captures interactive information and long-term dependencies among the emotional features between words. Comparative experiments were performed on four sentiment analysis datasets using five traditional models, showcasing the effectiveness of the CQLSTM model.
In linguistics, the uncertainty of context due to polysemy is widespread, which attracts much attention. Quantum-inspired complex word embedding based on Hilbert space plays an important role in natural language processing (NLP), which fully leverages the similarity between quantum states and word tokens. A word containing multiple meanings could correspond to a single quantum particle which may exist in several possible states, and a sentence could be analogous to the quantum system where particles interfere with each other. Motivated by quantum-inspired complex word embedding, interpretable complex-valued word embedding (ICWE) is proposed to design two end-to-end quantum-inspired deep neural networks (ICWE-QNN and CICWE-QNN representing convolutional complex-valued neural network based on ICWE) for binary text classification. They have the proven feasibility and effectiveness in the application of NLP and can solve the problem of text information loss in CE-Mix [1] model caused by neglecting the important linguistic features of text, since linguistic feature extraction is presented in our model with deep learning algorithms, in which gated recurrent unit (GRU) extracts the sequence information of sentences, attention mechanism makes the model focus on important words in sentences and convolutional layer captures the local features of projected matrix. The model ICWE-QNN can avoid random combination of word tokens and CICWE-QNN fully considers textual features of the projected matrix. Experiments conducted on five benchmarking classification datasets demonstrate our proposed models have higher accuracy than the compared traditional models including CaptionRep BOW, DictRep BOW and Paragram-Phrase, and they also have great performance on F1-score. Eespecially, CICWE-QNN model has higher accuracy than the quantum-inspired model CE-Mix as well for four datasets including SST, SUBJ, CR and MPQA. It is a meaningful and effictive exploration to design quantum-inspired deep neural networks to promote the performance of text classification.
The performance of Feedforward Neural Networks (FNNs) in specific application areas is often higher than the upper limit allowed by (classical) theory, and therefore is called “Alchemy”, that is, the lack of interpretability of FNNs. In this paper, based on the Quantum Weak Measurement theory (QWM), we propose an interpretable network with complex-valued representation to deal with the insufficient interpretability of FNNs and the Semantic Interpretability of Complex Numbers (SICN) from a novel perspective. For the specific role of complex-valued representation in capturing semantic information, we give an in-depth analysis that the real and imaginary parts of complex numbers can simultaneously represent two different types of complementary information. Finally, we propose a QWM inspired interpretable Network for text representation learning (i.e., QWMN) and implement two frameworks for Question Answering (QWMN-QA) and Text Classification (QWMN-TC) respectively. Experimental results prove the feasibility, effectiveness of QWMN.