Word meaning has different aspects, while the existing word representation "compresses" these aspects into a single vector, and it needs further analysis to recover the information in different dimensions. Inspired by quantum probability, we represent words as density matrices, which are inherently capable of representing mixed states. The experiment shows that the density matrix representation can effectively capture different aspects of word meaning while maintaining comparable reliability with the vector representation. Furthermore, we propose a novel method to combine the coherent summation and incoherent summation in the computation of both vectors and density matrices. It achieves consistent improvement on word analogy task.
Word meaning has different aspects, while the existing word representation "compresses" these aspects into a single vector, and it needs further analysis to recover the information in different dimensions. Inspired by quantum probability, we represent words as density matrices, which are inherently capable of representing mixed states. The experiment shows that the density matrix representation can effectively capture different aspects of word meaning while maintaining comparable reliability with the vector representation. Furthermore, we propose a novel method to combine the coherent summation and incoherent summation in the computation of both vectors and density matrices. It achieves consistent improvement on word analogy task.
Polysemy is a common phenomenon in linguistics. Quantum-inspired complex word embeddings based on Semantic Hilbert Space play an important role in natural language processing, which may accurately define a genuine probability distribution over the word space. The existing quantum-inspired works manipulate on the real-valued vectors to compose the complex-valued word embeddings, which lack direct complex-valued pre-trained word representations. Motivated by quantum-inspired complex word embeddings, we propose a complex-valued pre-trained word embedding based on density matrices, called Word2State. Unlike the existing static word embeddings, our proposed model can provide non-linear semantic composition in the form of amplitude and phase, which also defines an authentic probabilistic distribution. We evaluate this model on twelve datasets from the word similarity task and six datasets from the relevant downstream tasks. The experimental results on different tasks demonstrate that our proposed pre-trained word embedding can capture richer semantic information and exhibit greater flexibility in expressing uncertainty.
A challenging task for word embeddings is to capture the emergent meaning or polarity of a combination of individual words.For example, existing approaches in word embeddings will assign high probabilities to the words "Penguin" and "Fly" if they frequently co-occur, but it fails to capture the fact that they occur in an opposite sense -Penguins do not fly.We hypothesize that humans do not associate a single polarity or sentiment to each word.The word contributes to the overall polarity of a combination of words depending upon which other words it is combined with.This is analogous to the behavior of microscopic particles which exist in all possible states at the same time and interfere with each other to give rise to new states depending upon their relative phases.We make use of the Hilbert Space representation of such particles in Quantum Mechanics where we subscribe a relative phase to each word, which is a complex number, and investigate two such quantum inspired models to derive the meaning of a combination of words.The proposed models 1 achieve better performances than state-ofthe-art non-quantum models on the binary sentence classification task.
In the field of Natural Language Processing (NLP), especially for many specific tasks, representing and modeling semantic of text plays essential role. Generally speaking, by extracting feature of words, we can find the internal relationship between words and model text semantics. Further more, there have been achievements in using quantum concepts to construct language models, which suggests the effectiveness of applying quantum theory to semantic learning and representation. However, in the current research of quantum-inspired language model, semantic of the text is often directly modeled by quantum concept (e.g., density matrix), which blurs the original meaning of the word itself. In this study, we propose the composite semantic system (CSS) hypothesis and construct a quantum-inspired semantic representation network (QSRN). We apply QSRN to four different text classification tasks, the experimental results show the effectiveness of the algorithm and verify the hypothesis we proposed.
Abstract So far, research on quantum image representation has gone through more than 20 years. During this time, the quantum image representation models used have almost all been based on state vectors. However, in practical problems, the environment and the principal quantum system cannot be separated, and isolated quantum systems do not exist in principle. This case is often referred to as an open quantum system. In open quantum systems, many problems involve density matrices, such as the calculation of Von Neumann entropy, the quantization of coherence, and the operator-sum representations of quantum operations. Therefore, the existing quantum image representation models are only suitable for closed quantum systems. To this end, the paper proposes three models that can not only represent quantum images in an open quantum system but also decompose the evolution process of quantum images utilizing operator-sum decomposition. These three models are the representation model of quantum gray-scale images, the tensor product representation model of quantum color images, and the representation model of quantum color images based on mixed states in the Bloch sphere, respectively. All these image representation models have strong correlations among them and are very different from their classical analogues. Between them, the biggest difference is that the paper employs density matrices, inspired by incoherent-coherent states, to represent quantum images rather than classical state vectors. By means of one of the representation models proposed in the paper, we finally demonstrate the evolution process of the quantum image going through the amplitude damping channel.
Abstract To emphasize the semantic impact of local semantic and grammatical information among adjacent words in the input text, we establish a constraint functions-based quantum-like tensor compression sentence representation model by integrating the concept of extending the pure state-based density matrix to the mixed-state projection operator in quantum mechanics. The provided model highlights the semantic significance of mixed word associations in the input text, simultaneously reducing the reliance on information derived solely from dictionary statistics. We combine the correlation coefficient with the attention mechanism to establish the correlation coefficient between words. The quantum-like sentence representation based on pure state density matrix is extended to the projection operator of mixed states. Combining the acquisition of maximum in convex optimization, a constraint functions-based quantum-like text representation pruning model is established to reduce redundant information caused by dimensional expansion of tensor operations. The experimental results on SICK-2014, STS-benchmark, and STS-companion show that the provided model is more effective than the mainstream models in mining semantic information, especially more sensitive to the negative semantics of sentences.
Word embeddings and vector representations for linguistic units are important in Natural Language Processing (NLP) and AI systems. Engineering of such models involves significant effort, large amounts of data, and costly computations. Many models and systems have been released to the general public, enabling significant research and development in the classical NLP domain. We demonstrate how word and text embeddings as n-dimensional dense vectors of n real numbers in classical computers can be mapped to highly compressed quantum states or quantum embedding using log(n) qubits. In similarity experiments performed on classical and quantum computers, we show no significant information loss that would affect vector distance scores representing words’ semantic similarity. The results show that existing NLP embedding models in classical computing environments can be used in quantum computing. We discuss the issues and limitations of this approach in the context of current quantum computing environments.