The word: Zitterbewebung

Saul T. Epstein

Qiuchi Li, Sagar Uprety, Benyou Wang +1 more

A challenging task for word embeddings is to capture the emergent meaning or polarity of a combination of individual words.For example, existing approaches in word embeddings will assign high probabilities to the words "Penguin" and "Fly" if they frequently co-occur, but it fails to capture the fact that they occur in an opposite sense -Penguins do not fly.We hypothesize that humans do not associate a single polarity or sentiment to each word.The word contributes to the overall polarity of a combination of words depending upon which other words it is combined with.This is analogous to the behavior of microscopic particles which exist in all possible states at the same time and interfere with each other to give rise to new states depending upon their relative phases.We make use of the Hilbert Space representation of such particles in Quantum Mechanics where we subscribe a relative phase to each word, which is a complex number, and investigate two such quantum inspired models to derive the meaning of a combination of words.The proposed models 1 achieve better performances than state-ofthe-art non-quantum models on the binary sentence classification task.

Guangcheng Liu, Yuexian Hou

Real-valued word vectors (RWV) often perform poorly when showing semantic relations such as the ambiguity of the word itself (“spring” can refer to a kind of season or fountain) and the polarity between words (words “love” and “hate” have a high degree of similarity due to their similar distribution). Although dynamic word vectors alleviate these problems, such methods are still lacking interpretability. Inspired by quantum theory, we believe that basic reasons are insufficient modeling ability of RWV and weak abstract ability of language models. To solve the above problems, we propose to use complex-valued word vectors (CWV) for the representation, which can naturally model semantic relations of words. We also design an explicable lexicon-extended quantum probability neural network (LQPNN) based on the theory of quantum measurement to training CWV. Experiments on three datasets of classification demonstrate the effectiveness of our model, which achieves superior performances.

John Maddox

Torsten Fließbach

Chenchen Zhang, Qiuchi Li, Zhan Su +1 more

Polysemy is a common phenomenon in linguistics. Quantum-inspired complex word embeddings based on Semantic Hilbert Space play an important role in natural language processing, which may accurately define a genuine probability distribution over the word space. The existing quantum-inspired works manipulate on the real-valued vectors to compose the complex-valued word embeddings, which lack direct complex-valued pre-trained word representations. Motivated by quantum-inspired complex word embeddings, we propose a complex-valued pre-trained word embedding based on density matrices, called Word2State. Unlike the existing static word embeddings, our proposed model can provide non-linear semantic composition in the form of amplitude and phase, which also defines an authentic probabilistic distribution. We evaluate this model on twelve datasets from the word similarity task and six datasets from the relevant downstream tasks. The experimental results on different tasks demonstrate that our proposed pre-trained word embedding can capture richer semantic information and exhibit greater flexibility in expressing uncertainty.