arXiv:2506.00321

QTP-Net: A Quantum Text Pre-training Network for Natural Language Processing

Ren-Xin Zhao

Published May 31, 2025• 0 citations

Abstract

Natural Language Processing (NLP) faces challenges in the ability to quickly model polysemous words. The Grover's Algorithm (GA) is expected to solve this problem but lacks adaptability. To address the above dilemma, a Quantum Text Pre-training Network (QTP-Net) is proposed to improve the performance of NLP tasks. First, a Quantum Enhanced Pre-training Feature Embedding (QEPFE) is developed to encode multiple meanings of words into quantum superposition states and exploit adaptive GA to fast capture rich text features. Subsequently, the QEPFE is combined with the Enhanced Representation through kNowledge IntEgration (ERNIE), a pre-trained language model proposed by Baidu, to construct QTP-Net, which is evaluated on Sentiment Classification (SC) and Word Sense Disambiguation (WSD) tasks. Experiments show that in SC, the QTP-Net improves the average accuracy by 0.024 and the F1 score by 0.029 on six benchmark datasets, comprehensively outperforming both classical and quantum-inspired models. In WSD, it reaches 0.784 average F1 score, which is 0.016 higher than the sub-optimal GlossBERT, and significantly leads on SE2, SE13, and SE15. QTP-Net provides a new solution for implicit semantic modeling in NLP and lays the foundation for future research on quantum-enhanced models.

DOI: 10.48550/arxiv.2506.00321

Related papers

Pretrained Quantum-Inspired Deep Neural Network for Natural Language Processing

Jinjing Shi, Tian Chen, Wei Lai +2 more

Natural language processing (NLP) may face the inexplicable "black-box" problem of parameters and unreasonable modeling for lack of embedding of some characteristics of natural language, while the quantum-inspired models based on quantum theory may provide a potential solution. However, the essential prior knowledge and pretrained text features are often ignored at the early stage of the development of quantum-inspired models. To attacking the above challenges, a pretrained quantum-inspired deep neural network is proposed in this work, which is constructed based on quantum theory for carrying out strong performance and great interpretability in related NLP fields. Concretely, a quantum-inspired pretrained feature embedding (QPFE) method is first developed to model superposition states for words to embed more textual features. Then, a QPFE-ERNIE model is designed by merging the semantic features learned from the prevalent pretrained model ERNIE, which is verified with two NLP downstream tasks: 1) sentiment classification and 2) word sense disambiguation (WSD). In addition, schematic quantum circuit diagrams are provided, which has potential impetus for the future realization of quantum NLP with quantum device. Finally, the experiment results demonstrate QPFE-ERNIE is significantly better for sentiment classification than gated recurrent unit (GRU), BiLSTM, and TextCNN on five datasets in all metrics and achieves better results than ERNIE in accuracy, F1-score, and precision on two datasets (CR and SST), and it also has advantage for WSD over the classical models, including BERT (improves F1-score by 5.2 on average) and ERNIE (improves F1-score by 4.2 on average) and improves the F1-score by 8.7 on average compared with a previous quantum-inspired model QWSD. QPFE-ERNIE provides a novel pretrained quantum-inspired model for solving NLP problems, and it lays a foundation for exploring more quantum-inspired models in the future.

IEEE Transactions on CyberneticsMay 29, 202438 citations

Characterizing Quantum Classifier Utility in Natural Language Processing Workflows

Kathleen E. Hamilton, Mayanka Chandrashekar, John P. Gounley +5 more

Quantum Natural Language Processing (QNLP) develops natural language processing (NLP) models for deployment on quantum computers. We explore feature and data prototype selection techniques to address challenges posed by encoding high dimensional features. Our study builds quantum circuit classifiers that includes classical feature pre-processing, quantum embedding and quantum model training. The quantum models are built on 4 or 6 qubits and the quantum neural network (QNN) uses the established bricklayer design. We compare the dependence of model performance (in terms of accuracy and F1 scores) on feature length, embedding gates and parameterized unitary design. We compare the performance of quantum machine learning models to classical convolution neural network model (CNN) on binary and multi-class classification tasks using two datasets of synthetic features and labels. The first is the ECP-CANDLE P3B3 dataset a corpus of synthetically generated cancer pathology reports. The second dataset is extracted from well-known benchmark dataset (MADELON) - features are generated with a combination of informative, repeated and uninformative features. Both datasets are used for binary classification and multi-class classification with 3 classes. We observe robust, accurate performance from all models on the binary classification tasks, but multiclass classification is a challenge for the quantum models-there is a notable decrease in accuracy when using 3 classes. Overall the performance is comparable in terms of recall and accuracy between QNNs and CNNs, even with large datasets. These results provide a point of comparison between quantum and classical models on real-world datasets.

IEEE International Conference on Quantum Computing and Engineering (QCE)Sep 17, 20231 citation

SentiQNN: Advancing Sentiment Analysis through Quantum-Inspired Neural Networks- A Comparative Analysis with Existing Methodologies

Lateshwari, Sushil Kumar Bansal

This paper presents SentiQNN (Sentiment Quantum Neural Network), a novel quantum-inspired neural network architecture specifically designed for sentiment analysis tasks in natural language processing. SentiQNN leverages quantum computing principles including superposition, entanglement, and quantum measurement to enhance traditional neural network capabilities for text classification. The model incorporates complex-valued embeddings, quantum mixture operations, and multi-task learning with sentiment lexicon integration to achieve superior performance in sentiment classification tasks. Through extensive experimentation across multiple benchmark datasets, SentiQNN demonstrates state-of-the-art performance with 88.11% accuracy, outperforming traditional classical approaches by 3-7%. The architecture introduces innovative quantum state preparation layers, quantum circuit operations, and quantum measurement mechanisms specifically optimized for natural language understanding. Our comprehensive evaluation reveals that quantum-inspired approaches can significantly enhance sentiment analysis performance while maintaining computational efficiency. The findings contribute to the emerging field of quantum machine learning and provide a foundation for future research in quantum-enhanced natural language processing systems.

Oct 27, 20250 citations

Two End-to-End Quantum-Inspired Deep Neural Networks for Text Classification

Jinjing Shi, Zhenhuan Li, Wei Lai +4 more

In linguistics, the uncertainty of context due to polysemy is widespread, which attracts much attention. Quantum-inspired complex word embedding based on Hilbert space plays an important role in natural language processing (NLP), which fully leverages the similarity between quantum states and word tokens. A word containing multiple meanings could correspond to a single quantum particle which may exist in several possible states, and a sentence could be analogous to the quantum system where particles interfere with each other. Motivated by quantum-inspired complex word embedding, interpretable complex-valued word embedding (ICWE) is proposed to design two end-to-end quantum-inspired deep neural networks (ICWE-QNN and CICWE-QNN representing convolutional complex-valued neural network based on ICWE) for binary text classification. They have the proven feasibility and effectiveness in the application of NLP and can solve the problem of text information loss in CE-Mix [1] model caused by neglecting the important linguistic features of text, since linguistic feature extraction is presented in our model with deep learning algorithms, in which gated recurrent unit (GRU) extracts the sequence information of sentences, attention mechanism makes the model focus on important words in sentences and convolutional layer captures the local features of projected matrix. The model ICWE-QNN can avoid random combination of word tokens and CICWE-QNN fully considers textual features of the projected matrix. Experiments conducted on five benchmarking classification datasets demonstrate our proposed models have higher accuracy than the compared traditional models including CaptionRep BOW, DictRep BOW and Paragram-Phrase, and they also have great performance on F1-score. Eespecially, CICWE-QNN model has higher accuracy than the quantum-inspired model CE-Mix as well for four datasets including SST, SUBJ, CR and MPQA. It is a meaningful and effictive exploration to design quantum-inspired deep neural networks to promote the performance of text classification.

IEEE Transactions on Knowledge and Data EngineeringNov 25, 202154 citations

Quantum-Enhanced Word Sense Disambiguation using Quantum Parallelism and Hybrid Quantum-Classical Models

Dalydas Karanath, Udai Shankar

The paper proposes a quantum-classical hybrid method of Word Sense Disambiguation (WSD) to combine quantum inspired computation on WSD with classical natural language processing methods. The quantum state encoding is proposed to make ambiguous words coded in the framework as quantum superposition of all the possible senses, and semantic interpretations can be explored by encoding quantum states. A simulation of quantum circuits based on simulated quantum interactions was used to create a hybrid model involving variational quantum circuits and classical neural layer using the PyTorch platform. The system was trained and tested on a manually annotated collection of 5,000 polysemous sentences spanning various domains and tested on WordNet synsets. The experimental findings reveal that the hybrid quantum-classical model of WSD has 94.7 per cent accuracy and weighted F1-score of 94.0 per cent, which is better than CNN-LSTM and Transformer (BERT-base) baselines. The confusion matrix affirms enhanced sense differentiation and less semantic duplicity. The suggested architecture illustrates the possibility of quantum-enhanced natural language processing to be used in order to reach a higher contextual accuracy without compromising the computational efficiency that would be suitable with near-term quantum hardware.

International Conference on IoT Based Control Networks and Intelligent Systems (ICICNIS)Dec 15, 20250 citations

Self-supervised pre-trained neural network for quantum natural language processing

Ben Yao, Prayag Tiwari, Qiuchi Li

Quantum computing models have propelled advances in many application domains. However, in the field of natural language processing (NLP), quantum computing models are limited in representation capacity due to the high linearity of the underlying quantum computing architecture. This work attempts to address this limitation by leveraging the concept of self-supervised pre-training, a paradigm that has been propelling the rocketing development of NLP, to increase the power of quantum NLP models on the representation level. Specifically, we present a self-supervised pre-training approach to train quantum encodings of sentences, and fine-tune quantum circuits for downstream tasks on its basis. Experiments show that pre-trained mechanism brings remarkable improvement over end-to-end pure quantum models, yielding meaningful prediction results on a variety of downstream text classification datasets.

Neural NetworksDec 6, 202417 citations