In the field of Natural Language Processing (NLP), especially for many specific tasks, representing and modeling semantic of text plays essential role. Generally speaking, by extracting feature of words, we can find the internal relationship between words and model text semantics. Further more, there have been achievements in using quantum concepts to construct language models, which suggests the effectiveness of applying quantum theory to semantic learning and representation. However, in the current research of quantum-inspired language model, semantic of the text is often directly modeled by quantum concept (e.g., density matrix), which blurs the original meaning of the word itself. In this study, we propose the composite semantic system (CSS) hypothesis and construct a quantum-inspired semantic representation network (QSRN). We apply QSRN to four different text classification tasks, the experimental results show the effectiveness of the algorithm and verify the hypothesis we proposed.
In the field of Natural Language Processing (NLP), especially for many specific tasks, representing and modeling semantic of text plays essential role. Generally speaking, by extracting feature of words, we can find the internal relationship between words and model text semantics. Further more, there have been achievements in using quantum concepts to construct language models, which suggests the effectiveness of applying quantum theory to semantic learning and representation. However, in the current research of quantum-inspired language model, semantic of the text is often directly modeled by quantum concept (e.g., density matrix), which blurs the original meaning of the word itself. In this study, we propose the composite semantic system (CSS) hypothesis and construct a quantum-inspired semantic representation network (QSRN). We apply QSRN to four different text classification tasks, the experimental results show the effectiveness of the algorithm and verify the hypothesis we proposed.
Natural language processing (NLP) may face the inexplicable "black-box" problem of parameters and unreasonable modeling for lack of embedding of some characteristics of natural language, while the quantum-inspired models based on quantum theory may provide a potential solution. However, the essential prior knowledge and pretrained text features are often ignored at the early stage of the development of quantum-inspired models. To attacking the above challenges, a pretrained quantum-inspired deep neural network is proposed in this work, which is constructed based on quantum theory for carrying out strong performance and great interpretability in related NLP fields. Concretely, a quantum-inspired pretrained feature embedding (QPFE) method is first developed to model superposition states for words to embed more textual features. Then, a QPFE-ERNIE model is designed by merging the semantic features learned from the prevalent pretrained model ERNIE, which is verified with two NLP downstream tasks: 1) sentiment classification and 2) word sense disambiguation (WSD). In addition, schematic quantum circuit diagrams are provided, which has potential impetus for the future realization of quantum NLP with quantum device. Finally, the experiment results demonstrate QPFE-ERNIE is significantly better for sentiment classification than gated recurrent unit (GRU), BiLSTM, and TextCNN on five datasets in all metrics and achieves better results than ERNIE in accuracy, F1-score, and precision on two datasets (CR and SST), and it also has advantage for WSD over the classical models, including BERT (improves F1-score by 5.2 on average) and ERNIE (improves F1-score by 4.2 on average) and improves the F1-score by 8.7 on average compared with a previous quantum-inspired model QWSD. QPFE-ERNIE provides a novel pretrained quantum-inspired model for solving NLP problems, and it lays a foundation for exploring more quantum-inspired models in the future.
In linguistics, the uncertainty of context due to polysemy is widespread, which attracts much attention. Quantum-inspired complex word embedding based on Hilbert space plays an important role in natural language processing (NLP), which fully leverages the similarity between quantum states and word tokens. A word containing multiple meanings could correspond to a single quantum particle which may exist in several possible states, and a sentence could be analogous to the quantum system where particles interfere with each other. Motivated by quantum-inspired complex word embedding, interpretable complex-valued word embedding (ICWE) is proposed to design two end-to-end quantum-inspired deep neural networks (ICWE-QNN and CICWE-QNN representing convolutional complex-valued neural network based on ICWE) for binary text classification. They have the proven feasibility and effectiveness in the application of NLP and can solve the problem of text information loss in CE-Mix [1] model caused by neglecting the important linguistic features of text, since linguistic feature extraction is presented in our model with deep learning algorithms, in which gated recurrent unit (GRU) extracts the sequence information of sentences, attention mechanism makes the model focus on important words in sentences and convolutional layer captures the local features of projected matrix. The model ICWE-QNN can avoid random combination of word tokens and CICWE-QNN fully considers textual features of the projected matrix. Experiments conducted on five benchmarking classification datasets demonstrate our proposed models have higher accuracy than the compared traditional models including CaptionRep BOW, DictRep BOW and Paragram-Phrase, and they also have great performance on F1-score. Eespecially, CICWE-QNN model has higher accuracy than the quantum-inspired model CE-Mix as well for four datasets including SST, SUBJ, CR and MPQA. It is a meaningful and effictive exploration to design quantum-inspired deep neural networks to promote the performance of text classification.
The recent developments in the field of deep learning have made Natural Language Processing (NLP) quite useful, but there are still issues of the complexity of computation, energy consumption, and modeling of complex semantics. Representations and optimization techniques that are quantum enhanced are available in quantum computing and provide new possibilities. Hybrid Quantum Neural Networks (HQNNs) combine quantum computational concepts and classical neural networks, allowing the introduction of a new era in intelligent language processing. The chapter provides the basics of quantum computing and neural NLP models and looks at the hybrid methods of semantic representation and learning. The text classification, sentiment analysis, named entity recognition, and machine translation tasks that are fundamental to NLP are examined through a hybrid quantum-neural approach. Such methods as quantum data encoding, quantum feature extraction and hybrid training approaches are described and the issues associated with scalability, noise, and hardware limitations are discussed.
Text classification remains among the most significant tasks in natural language processing and the field has been used in healthcare, law, social media analysis, and economic literature. Classical attention networks existing machine learning approaches and deep learning approaches, including LSTM, BiLSTM, Transformers, and classical attention networks, are susceptible to such problems as long-range dependencies and semantic ambiguity, as well as sparse or underrepresented text. To deal with these deficiencies, this research presents a proposed a Quantum-Enhanced Semantic Attention Network (QESAN), a quantum computing concept integrated into semantic attention-based deep learning paradigm. The proposed QESAN coded textual features using amplitude encoding, quantum superposition, and entanglement-like transformations, such that the model is able to learn not only complex relations between interdependencies, but also fairly subtle semantic relations which are generally hard to learn on classical models. The result of these quantum-enhanced representations is then sent through an attention mechanism which acts on the most semantically significant aspects to produce results that are context-sensitive and can be interpreted. The Kaggle Quantum Programming experimental results indicate that QESAN is effective with accuracy of 98.7 which is far superior than the LSTM, BiLSTM, Transformer, and classical attention networks. The results indicate the ability of QESAN to solve the conventional issues of the text classification process particularly in the assignment of dealing with the long-range dependencies, semantic ambiguity, and sparse data. Overall, the integration of the concept of quantum computing with the attention-based deep learning provides the scalable and resilient solution to the issue of the modern text classification problems of the high-performance.
Multi-task learning has demonstrated remarkable success across a broad spectrum of natural language processing tasks, particularly with neural network-based methods. Despite these advances, a fundamental gap remains in explaining the relationship between task-relatedness and model effectiveness. To address this issue, we propose a novel approach for implicitly modeling task-relatedness by leveraging a quantum physical mathematical framework. In this paper, we introduce a complex-valued neural network designed to encapsulate and analyze task-relatedness. Within this framework, sentences originating from diverse tasks are encoded as mixed quantum systems, represented on a meticulously defined Semantic Hilbert Space. This allows the network to interpret inter-task relationships through the explicit physical semantics of well-constrained components grounded in quantum probability theory. By adhering to these rigorous principles, our model not only establishes a robust method for quantifying task-relatedness but also fosters a deeper, self-explanatory understanding of the underlying processes. To validate the efficacy of our approach, we conducted extensive experiments across five benchmark text classification tasks. The results demonstrate both the superior performance and the interpretability of the proposed model, highlighting its potential as a self-explanatory system for multi-task learning in NLP.
Language Modeling (LM) is a fundamental research topic in a range of areas. Recently, inspired by quantum theory, a novel Quantum Language Model (QLM) has been proposed for Information Retrieval (IR). In this paper, we aim to broaden the theoretical and practical basis of QLM. We develop a Neural Network based Quantum-like Language Model (NNQLM) and apply it to Question Answering. Specifically, based on word embeddings, we design a new density matrix, which represents a sentence (e.g., a question or an answer) and encodes a mixture of semantic subspaces. Such a density matrix, together with a joint representation of the question and the answer, can be integrated into neural network architectures (e.g., 2-dimensional convolutional neural networks). Experiments on the TREC-QA and WIKIQA datasets have verified the effectiveness of our proposed models.