Quantum computing models have propelled advances in many application domains. However, in the field of natural language processing (NLP), quantum computing models are limited in representation capacity due to the high linearity of the underlying quantum computing architecture. This work attempts to address this limitation by leveraging the concept of self-supervised pre-training, a paradigm that has been propelling the rocketing development of NLP, to increase the power of quantum NLP models on the representation level. Specifically, we present a self-supervised pre-training approach to train quantum encodings of sentences, and fine-tune quantum circuits for downstream tasks on its basis. Experiments show that pre-trained mechanism brings remarkable improvement over end-to-end pure quantum models, yielding meaningful prediction results on a variety of downstream text classification datasets.
Quantum computing models have propelled advances in many application domains. However, in the field of natural language processing (NLP), quantum computing models are limited in representation capacity due to the high linearity of the underlying quantum computing architecture. This work attempts to address this limitation by leveraging the concept of self-supervised pre-training, a paradigm that has been propelling the rocketing development of NLP, to increase the power of quantum NLP models on the representation level. Specifically, we present a self-supervised pre-training approach to train quantum encodings of sentences, and fine-tune quantum circuits for downstream tasks on its basis. Experiments show that pre-trained mechanism brings remarkable improvement over end-to-end pure quantum models, yielding meaningful prediction results on a variety of downstream text classification datasets.
The emerging classical-quantum transfer learning paradigm has brought a decent performance to quantum computational models in many tasks, such as computer vision, by enabling a combination of quantum models and classical pre-trained neural networks. However, using quantum computing with pre-trained models has yet to be explored in natural language processing (NLP). Due to the high linearity constraints of the underlying quantum computing infrastructures, existing Quantum NLP models are limited in performance on real tasks. We fill this gap by pre-training a sentence state with complex-valued BERT-like architecture, and adapting it to the classical-quantum transfer learning scheme for sentence classification. On quantum simulation experiments, the pre-trained representation can bring 50\% to 60\% increases to the capacity of end-to-end quantum models.
This article proposes a quantum self-attention neural network (QSAN) model that is implementable on parameterized quantum circuits (PQCs), providing a novel avenue for addressing natural language processing (NLP) tasks via quantum computing. The QSAN architecture comprises four blocks: the data preprocessing block, the quantum encoding block, the model design block, and the network optimization block. Through these blocks, classical text data are initially preprocessed and encoded into quantum states. The rich semantic features and intricate relationships within these quantum states are then captured and learned by a quantum self-attention layer and a quantum fully connected layer. Finally, the model performance is enhanced by optimizing the network parameters. Simulation results on publicly available topic classification and sentiment analysis datasets demonstrate that the QSAN model outperforms state-of-the-art baselines. As a limitation of the current study, these evaluations are restricted to a maximum sentence length of eight words due to simulation and hardware constraints. Moreover, the effectiveness, scalability, trainability, and robustness of the proposed model are comprehensively evaluated, highlighting its superior performance and potential for advanced NLP tasks.
This paper proposes a quantum self-attention neural network (QSAN) model that can be deployed on quantum circuits, providing a novel avenue to processing text classification tasks in natural language processing (NLP). The QSAN framework is established by integrating four basic blocks: the data preprocessing block, the quantum encoding block, the model design block, and the network optimization block. Simulation results demonstrate remarkable convergence and accuracy on various text classification datasets. In particular, the proposed QSAN surpasses the existing state-of-the-art quantum NLP (QNLP) model in terms of test accuracy.
Quantum neural networks (QNNs) have recently attracted remarkable attention as a potential alternative to classical and deep neural network (NN) models particularly under constraints imposed by noisy intermediate-scale quantum (NISQ) devices, which support only shallow circuits with few quantum bits (qubits) and limited variational depth. However, applying QNNs to natural language processing (NLP) remains challenging because text representations are typically high-dimensional, while NISQ classifiers can only encode a small number of features at a time, requiring classical compression to enable stable training and flexible circuit configuration. In addition, many existing studies integrate QNNs into rich-text representation pipelines, where the quantum circuit is often implemented as a variational quantum circuit (VQC). Here, the VQC acts as an additional component within a larger model, and it remains unclear how the quantum component improves the classification task. Therefore, in this work, we propose a fully reproducible and compact QNN pipeline that provides a controlled setting for studying the contribution of the VQC to text classification. The model is trained end-to-end with a lightweight classical compression layer, and the overall study is supported by systematic ablation analyses of the compression layer, the classifier heads under a shared frozen low-dimensional representation, and quantum circuit structure. Our model uses the term frequency-inverse document frequency (TF-IDF) features with linear compression followed by a shallow four-qubit, two-layer VQC, implemented in PyTorch and PennyLane for end-to-end training. It is evaluated on five benchmark text classification datasets against classical baselines, including logistic regression and multilayer perceptron (MLP) models. Experimental results show that the proposed QNN achieves a competitive performance across all evaluated datasets, yielding the best results on three of the five benchmark datasets.
In recent developments, deep learning methodologies applied to Natural Language Processing (NLP) have revealed a paradox: They improve performance but demand considerable data and resources for their training. Alternatively, quantum computing exploits the principles of quantum mechanics to overcome the computational limitations of current methodologies, thereby establishing an emerging field known as quantum natural language processing (QNLP). This domain holds the potential to attain a quantum advantage in the processing of linguistic structures, surpassing classical models in both efficiency and accuracy. In this paper, it is proposed to categorise QNLP models based on quantum computing principles, architecture, and computational approaches. This paper attempts to provide a survey on how quantum meets language by mapping state-of-the-art in this area, embracing quantum encoding techniques for classical data, QNLP models for prevalent NLP tasks, and quantum optimisation techniques for hyper parameter tuning. The landscape of quantum computing approaches applied to various NLP tasks is summarised by showcasing the specific QNLP methods used, and the popularity of these methods is indicated by their count. From the findings, it is observed that QNLP approaches are still limited to small data sets, with only a few models explored extensively, and there is increasing interest in the application of quantum computing to natural language processing tasks.
Quantum Natural Language Processing (QNLP) develops natural language processing (NLP) models for deployment on quantum computers. We explore feature and data prototype selection techniques to address challenges posed by encoding high dimensional features. Our study builds quantum circuit classifiers that includes classical feature pre-processing, quantum embedding and quantum model training. The quantum models are built on 4 or 6 qubits and the quantum neural network (QNN) uses the established bricklayer design. We compare the dependence of model performance (in terms of accuracy and F1 scores) on feature length, embedding gates and parameterized unitary design. We compare the performance of quantum machine learning models to classical convolution neural network model (CNN) on binary and multi-class classification tasks using two datasets of synthetic features and labels. The first is the ECP-CANDLE P3B3 dataset a corpus of synthetically generated cancer pathology reports. The second dataset is extracted from well-known benchmark dataset (MADELON) - features are generated with a combination of informative, repeated and uninformative features. Both datasets are used for binary classification and multi-class classification with 3 classes. We observe robust, accurate performance from all models on the binary classification tasks, but multiclass classification is a challenge for the quantum models-there is a notable decrease in accuracy when using 3 classes. Overall the performance is comparable in terms of recall and accuracy between QNNs and CNNs, even with large datasets. These results provide a point of comparison between quantum and classical models on real-world datasets.