International Conference on Wavelet Analysis and Pattern Recognition

Povm-Based Quantum Self-Attention Neural Network

Jiachun Wei, Zhimin He, Chuangtao Chen, Maijie Deng, Haozhen Situ

Published Jul 9, 2023• 6 citations

Abstract

Quantum Self-Attention Neural Network (QSANN) has demonstrated remarkable potential. However, it is limited to measuring only partial information from qubits, thereby restricting its information extraction capability. Moreover, the measurement process requires multiple measurements for each qubit, leading to inefficiency in utilizing the feature space. To address these challenges, this paper introduces a novel approach called Positive Operator-Valued Measure based Quantum Self-Attention Neural Network (POVM-QSANN). It leverages POVM operators as observables, enabling the extraction of comprehensive QKV (Query-Key-Value) feature vectors from each qubit. This innovative mechanism significantly enhances information extraction capability and efficiently utilizes the feature space. The experimental results underscore the considerable advancements of POVM-QSANN on MC and RP datasets. Particularly noteworthy is its remarkable accuracy increase of 9.68% compared to QSANN on the RP dataset.

DOI: 10.1109/icwapr58546.2023.10337270

Related papers

Quantum Mixed-State Self-Attention Network

Fu Chen, Qinglin Zhao, Feng Li +3 more

Attention mechanisms have revolutionized natural language processing. Combining them with quantum computing aims to further advance this technology. This paper introduces a novel Quantum Mixed-State Self-Attention Network (QMSAN) for natural language processing tasks. Our model leverages quantum computing principles to enhance the effectiveness of self-attention mechanisms. QMSAN uses a quantum attention mechanism based on mixed state, allowing for direct similarity estimation between queries and keys in the quantum domain. This approach leads to more effective attention coefficient calculations. We also propose an innovative quantum positional encoding scheme, implemented through fixed quantum gates within the circuit, improving the model's ability to capture sequence information without additional qubit resources. In numerical experiments of text classification tasks on public datasets, QMSAN outperforms Quantum Self-Attention Neural Network (QSANN). Furthermore, we demonstrate QMSAN's robustness in different quantum noise environments, highlighting its potential for near-term quantum devices.

arXiv:2403.02871Mar 5, 20241 citation

A Quantum Self-Attention Neural Network Model on Quantum Circuits

Jin Yan Zheng, Qing Zhong Gao, Daoyi Dong +2 more

This article proposes a quantum self-attention neural network (QSAN) model that is implementable on parameterized quantum circuits (PQCs), providing a novel avenue for addressing natural language processing (NLP) tasks via quantum computing. The QSAN architecture comprises four blocks: the data preprocessing block, the quantum encoding block, the model design block, and the network optimization block. Through these blocks, classical text data are initially preprocessed and encoded into quantum states. The rich semantic features and intricate relationships within these quantum states are then captured and learned by a quantum self-attention layer and a quantum fully connected layer. Finally, the model performance is enhanced by optimizing the network parameters. Simulation results on publicly available topic classification and sentiment analysis datasets demonstrate that the QSAN model outperforms state-of-the-art baselines. As a limitation of the current study, these evaluations are restricted to a maximum sentence length of eight words due to simulation and hardware constraints. Moreover, the effectiveness, scalability, trainability, and robustness of the proposed model are comprehensively evaluated, highlighting its superior performance and potential for advanced NLP tasks.

IEEE Transactions on CyberneticsJun 29, 20261 citation

Enhancing Text Classification Through Quantum Transfer Learning: A Hybrid Quantum-Classical Approach With Complex Kernel Self-Attention Networks

Xiaoxiao Chen, Xiaoping Lou

Quantum computing, with its unique principles such as superposition and entanglement, promises to transform various domains, including artificial intelligence. This study investigates the fusion of quantum computing with contemporary AI technologies through quantum transfer learning. We introduce the Complex Quantum Kernel Self-Attention Network (CQKSAN), a novel model that leverages quantum circuits and self-attention mechanisms to boost text classification performance. By applying quantum transfer learning, CQKSAN harnesses the strengths of pre-trained classical models and enhances them with quantum capabilities. Experimental results on multiple binary classification datasets demonstrate that CQKSAN achieves a more efficient learning process. Specifically, it improves training accuracy and validation accuracy by up to 24.31% and 28.35%, respectively, and increases the Matthews Correlation Coefficient by 113.10%. Moreover, CQKSAN consistently outperforms the BERT baseline in both training and validation accuracy across various datasets, showcasing its strong learning capability and generalization performance in handling tasks with diverse linguistic structures and complex semantics. Our findings underscore the potential of hybrid quantum-classical approaches, while also identifying the current limitations and paving the way for future research in fully quantum-based learning systems.

IEEE AccessJan 1, 20255 citations

Design of a Quantum Self-Attention Neural Network on Quantum Circuits

Jin Yan Zheng, Qing Zhong Gao, Zibo Miao

This paper proposes a quantum self-attention neural network (QSAN) model that can be deployed on quantum circuits, providing a novel avenue to processing text classification tasks in natural language processing (NLP). The QSAN framework is established by integrating four basic blocks: the data preprocessing block, the quantum encoding block, the model design block, and the network optimization block. Simulation results demonstrate remarkable convergence and accuracy on various text classification datasets. In particular, the proposed QSAN surpasses the existing state-of-the-art quantum NLP (QNLP) model in terms of test accuracy.

Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and CyberneticsOct 1, 20237 citations

QKSAN: A Quantum Kernel Self-Attention Network

Ren-Xin Zhao, Jinjing Shi, Xuelong Li

The Self-Attention Mechanism (SAM) excels at distilling important information from the interior of data to improve the computational efficiency of models. Nevertheless, many Quantum Machine Learning (QML) models lack the ability to distinguish the intrinsic connections of information like SAM, which limits their effectiveness on massive high-dimensional quantum data. To tackle the above issue, a Quantum Kernel Self-Attention Mechanism (QKSAM) is introduced to combine the data representation merit of Quantum Kernel Methods (QKM) with the efficient information extraction capability of SAM. Further, a Quantum Kernel Self-Attention Network (QKSAN) framework is proposed based on QKSAM, which ingeniously incorporates the Deferred Measurement Principle (DMP) and conditional measurement techniques to release half of quantum resources by mid-circuit measurement, thereby bolstering both feasibility and adaptability. Simultaneously, the Quantum Kernel Self-Attention Score (QKSAS) with an exponentially large characterization space is spawned to accommodate more information and determine the measurement conditions. Eventually, four QKSAN sub-models are deployed on PennyLane and IBM Qiskit platforms to perform binary classification on MNIST and Fashion MNIST, where the QKSAS tests and correlation assessments between noise immunity and learning ability are executed on the best-performing sub-model. The paramount experimental finding is that the QKSAN subclasses possess the potential learning advantage of acquiring impressive accuracies exceeding 98.05% with far fewer parameters than classical machine learning models. Predictably, QKSAN lays the foundation for future quantum computers to perform machine learning on massive amounts of data while driving advances in areas such as quantum computer vision.

arXiv:2308.13422IEEE Transactions on Pattern Analysis and Machine IntelligenceJul 29, 202480 citations

Self-Organizing Neural Networks for Quantum Feature Learning

Jincheng Zhang

This paper investigates the application of Self-Organizing Neural Networks (SONNs) to quantum feature learning, leveraging the principles of quantum entanglement and dynamic quantum state evolution. The core claim is that this approach enables the creation of more nuanced and deeper feature representations, ultimately improving model accuracy within the context of quantum computation. We explore how quantum-enhanced feature learning can be achieved through a novel dynamic adjustment process based on the interplay between quantum states and neural network weights. The study focuses on the potential of this methodology to address challenges associated with traditional feature extraction methods in quantum algorithms.

Aug 31, 20260 citations