Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

International Journal of Artificial Intelligence Tools

QHAN: Quantum-inspired Hierarchical Attention Mechanism Network for Question Answering

Peng Guo, Panpan Wang

Published Jan 11, 2023• 5 citations
Publisher

Abstract

The approach to question answering is challenging because it usually requires finding useful information from within and between question and answer sentences for sentence semantic matching. The key information mined from existing question and answer sentences, as a supplement to semantic information, maybe helpful for this task. However, capturing the intra-sentence and inter-sentence semantic interactions is somewhat difficult given the implicit interrelationships between question and answer sentences. Although the learning effect of multi-layer neural network is good, it lacks explanatory and theoretical support. This paper mainly studies how to select the best answer sentence from a set of question and answer sentences, mine information more effectively, perform semantic matching, and have a certain interpretability. Therefore, we propose a hierarchical attention network (QHAN) based on the mathematical framework of quantum theory, which integrates the attention mechanism under quantum measurement and density matrix (DMATT) and the attention mechanism under quantum weak measurement and weak value (WMATT) in one in a unified model framework. QHAN can not only discover key sentences in a set of question and answer sentences and key words in a sentence, but also perform sentence semantic matching quickly and accurately. Furthermore, QHAN has the advantage of interpretability in terms of models due to the physical meaning and attentional full weight distribution implied by quantum theory. Extensive experiments on the question answering dataset show that our method is comparable to the baselines and can explain why the selected question and answer sentence is the best option in terms of intra-sentence and inter-sentence attention distributions.

DOI
10.1142/s0218213023600096

Related papers

End-to-End Quantum-like Language Models with Application to Question Answering

Peng Zhang, Jiabin Niu, Zhan Su +3 more

Language Modeling (LM) is a fundamental research topic in a range of areas. Recently, inspired by quantum theory, a novel Quantum Language Model (QLM) has been proposed for Information Retrieval (IR). In this paper, we aim to broaden the theoretical and practical basis of QLM. We develop a Neural Network based Quantum-like Language Model (NNQLM) and apply it to Question Answering. Specifically, based on word embeddings, we design a new density matrix, which represents a sentence (e.g., a question or an answer) and encodes a mixture of semantic subspaces. Such a density matrix, together with a joint representation of the question and the answer, can be integrated into neural network architectures (e.g., 2-dimensional convolutional neural networks). Experiments on the TREC-QA and WIKIQA datasets have verified the effectiveness of our proposed models.

Proceedings of the AAAI Conference on Artificial Intelligence•Apr 27, 2018•86 citations

The Interpretability of Quantum-Inspired Neural Network

Shikai Song, Yuexian Hou, Guangcheng Liu

Recently, the mathematical framework of quantum mechanics has boosted interpretability and performance in a variety of NLP tasks. Since different linguistic units are explicitly weighted by this framework in density matrix, it is also often believed that the models inspired by quantum mechanics afford transparency. However, the relationship between weights of linguistic units and model outputs is uncertain. In this work we test whether this assumption holds by performing extensive experiments in question answering (QA) model to manipulating semantic weights and assessing the resulting differences in its output. Although we observe in some ways that higher weights do correlate with greater effect on model, but most of the data does not support it. i.e., the weight is independent of the prediction, and different weight distributions can produce equivalent result. Our results illustrate that standard quantum-inspired model has interpretable structure but does not accurately model feature importance and requires improvement in semantic representation and model interpretability.

International Conference on Artificial Intelligence and Big Data (ICAIBD)•May 28, 2021•3 citations

A Quantum Expectation Value Based Language Model with Application to Question Answering

Qin Zhao, Chenguang Hou, Changjian Liu +2 more

Quantum-inspired language models have been introduced to Information Retrieval due to their transparency and interpretability. While exciting progresses have been made, current studies mainly investigate the relationship between density matrices of difference sentence subspaces of a semantic Hilbert space. The Hilbert space as a whole which has a unique density matrix is lack of exploration. In this paper, we propose a novel Quantum Expectation Value based Language Model (QEV-LM). A unique shared density matrix is constructed for the Semantic Hilbert Space. Words and sentences are viewed as different observables in this quantum model. Under this background, a matching score describing the similarity between a question-answer pair is naturally explained as the quantum expectation value of a joint question-answer observable. In addition to the theoretical soundness, experiment results on the TREC-QA and WIKIQA datasets demonstrate the computational efficiency of our proposed model with excellent performance and low time consumption.

Entropy•May 9, 2020•10 citations

A Quantum-inspired Language Model Based on Transformer and Unitary Evolution for Question-answering

Yuheng Zha, Lian-He Shao, Xihan Wang +1 more

Quantum-Inspired Language Models (QLMs) model sentences through the superposition of semantic features and have achieved certain progress in various tasks. The defining characteristic of this approach lies in its use of the mathematical framework of quantum mechanics to model textual data. However, existing QLMs lack mature iterative update mechanisms for sentence representations. This paper proposes a Quantum-Inspired Unitary Transformation and Transformer for Question-Answering (QUTT-QA) model, which enhances the semantic expressiveness of word vectors via pretraining and leverages quantum-inspired operations to strengthen the model's capability in processing complex sequential data. Experimental results on QA tasks show that the proposed QUTT-QA model outperforms most traditional QLMs in accuracy and recall. Additionally, ablation studies on key quantum modules validate the effectiveness of the QUTT-QA model.

•Feb 6, 2026•0 citations

CNM: An Interpretable Complex-valued Network for Matching

Qiuchi Li, Benyou Wang, Massimo Melucci

This paper seeks to model human language by the mathematical framework of quantum physics. With the well-designed mathematical formulations in quantum physics, this framework unifies different linguistic units in a single complex-valued vector space, e.g. words as particles in quantum states and sentences as mixed systems. A complex-valued network is built to implement this framework for semantic matching. With well-constrained complex-valued components, the network admits interpretations to explicit physical meanings. The proposed complex-valued network for matching (CNM) achieves comparable performances to strong CNN and RNN baselines on two benchmarking question answering (QA) datasets.

arXiv:1904.05298•Apr 10, 2019•29 citations

Attention-based quantum tomography

Peter Cha, Paul Ginsparg, Felix Wu +3 more

Abstract With rapid progress across platforms for quantum systems, the problem of many-body quantum state reconstruction for noisy quantum states becomes an important challenge. There has been a growing interest in approaching the problem of quantum state reconstruction using generative neural network models. Here we propose the ‘attention-based quantum tomography’ (AQT), a quantum state reconstruction using an attention mechanism-based generative network that learns the mixed state density matrix of a noisy quantum state. AQT is based on the model proposed in ‘Attention is all you need’ by Vaswani et al (2017 NIPS) that is designed to learn long-range correlations in natural language sentences and thereby outperform previous natural language processing (NLP) models. We demonstrate not only that AQT outperforms earlier neural-network-based quantum state reconstruction on identical tasks but that AQT can accurately reconstruct the density matrix associated with a noisy quantum state experimentally realized in an IBMQ quantum computer. We speculate the success of the AQT stems from its ability to model quantum entanglement across the entire quantum system much as the attention model for NLP captures the correlations among words in a sentence.

arXiv:2006.12469•Machine Learning Science and Technology•Nov 3, 2021•78 citations
The approach to question answering is challenging because it usually requires finding useful information from within and between question and answer sentences for sentence semantic matching. The key information mined from existing question and answer sentences, as a supplement to semantic information, maybe helpful for this task. However, capturing the intra-sentence and inter-sentence semantic interactions is somewhat difficult given the implicit interrelationships between question and answer sentences. Although the learning effect of multi-layer neural network is good, it lacks explanatory and theoretical support. This paper mainly studies how to select the best answer sentence from a set of question and answer sentences, mine information more effectively, perform semantic matching, and have a certain interpretability. Therefore, we propose a hierarchical attention network (QHAN) based on the mathematical framework of quantum theory, which integrates the attention mechanism under quantum measurement and density matrix (DMATT) and the attention mechanism under quantum weak measurement and weak value (WMATT) in one in a unified model framework. QHAN can not only discover key sentences in a set of question and answer sentences and key words in a sentence, but also perform sentence semantic matching quickly and accurately. Furthermore, QHAN has the advantage of interpretability in terms of models due to the physical meaning and attentional full weight distribution implied by quantum theory. Extensive experiments on the question answering dataset show that our method is comparable to the baselines and can explain why the selected question and answer sentence is the best option in terms of intra-sentence and inter-sentence attention distributions.
Language Modeling (LM) is a fundamental research topic in a range of areas. Recently, inspired by quantum theory, a novel Quantum Language Model (QLM) has been proposed for Information Retrieval (IR). In this paper, we aim to broaden the theoretical and practical basis of QLM. We develop a Neural Network based Quantum-like Language Model (NNQLM) and apply it to Question Answering. Specifically, based on word embeddings, we design a new density matrix, which represents a sentence (e.g., a question or an answer) and encodes a mixture of semantic subspaces. Such a density matrix, together with a joint representation of the question and the answer, can be integrated into neural network architectures (e.g., 2-dimensional convolutional neural networks). Experiments on the TREC-QA and WIKIQA datasets have verified the effectiveness of our proposed models.
Recently, the mathematical framework of quantum mechanics has boosted interpretability and performance in a variety of NLP tasks. Since different linguistic units are explicitly weighted by this framework in density matrix, it is also often believed that the models inspired by quantum mechanics afford transparency. However, the relationship between weights of linguistic units and model outputs is uncertain. In this work we test whether this assumption holds by performing extensive experiments in question answering (QA) model to manipulating semantic weights and assessing the resulting differences in its output. Although we observe in some ways that higher weights do correlate with greater effect on model, but most of the data does not support it. i.e., the weight is independent of the prediction, and different weight distributions can produce equivalent result. Our results illustrate that standard quantum-inspired model has interpretable structure but does not accurately model feature importance and requires improvement in semantic representation and model interpretability.
Quantum-inspired language models have been introduced to Information Retrieval due to their transparency and interpretability. While exciting progresses have been made, current studies mainly investigate the relationship between density matrices of difference sentence subspaces of a semantic Hilbert space. The Hilbert space as a whole which has a unique density matrix is lack of exploration. In this paper, we propose a novel Quantum Expectation Value based Language Model (QEV-LM). A unique shared density matrix is constructed for the Semantic Hilbert Space. Words and sentences are viewed as different observables in this quantum model. Under this background, a matching score describing the similarity between a question-answer pair is naturally explained as the quantum expectation value of a joint question-answer observable. In addition to the theoretical soundness, experiment results on the TREC-QA and WIKIQA datasets demonstrate the computational efficiency of our proposed model with excellent performance and low time consumption.
Quantum-Inspired Language Models (QLMs) model sentences through the superposition of semantic features and have achieved certain progress in various tasks. The defining characteristic of this approach lies in its use of the mathematical framework of quantum mechanics to model textual data. However, existing QLMs lack mature iterative update mechanisms for sentence representations. This paper proposes a Quantum-Inspired Unitary Transformation and Transformer for Question-Answering (QUTT-QA) model, which enhances the semantic expressiveness of word vectors via pretraining and leverages quantum-inspired operations to strengthen the model's capability in processing complex sequential data. Experimental results on QA tasks show that the proposed QUTT-QA model outperforms most traditional QLMs in accuracy and recall. Additionally, ablation studies on key quantum modules validate the effectiveness of the QUTT-QA model.
This paper seeks to model human language by the mathematical framework of quantum physics. With the well-designed mathematical formulations in quantum physics, this framework unifies different linguistic units in a single complex-valued vector space, e.g. words as particles in quantum states and sentences as mixed systems. A complex-valued network is built to implement this framework for semantic matching. With well-constrained complex-valued components, the network admits interpretations to explicit physical meanings. The proposed complex-valued network for matching (CNM) achieves comparable performances to strong CNN and RNN baselines on two benchmarking question answering (QA) datasets.
Abstract With rapid progress across platforms for quantum systems, the problem of many-body quantum state reconstruction for noisy quantum states becomes an important challenge. There has been a growing interest in approaching the problem of quantum state reconstruction using generative neural network models. Here we propose the ‘attention-based quantum tomography’ (AQT), a quantum state reconstruction using an attention mechanism-based generative network that learns the mixed state density matrix of a noisy quantum state. AQT is based on the model proposed in ‘Attention is all you need’ by Vaswani et al (2017 NIPS) that is designed to learn long-range correlations in natural language sentences and thereby outperform previous natural language processing (NLP) models. We demonstrate not only that AQT outperforms earlier neural-network-based quantum state reconstruction on identical tasks but that AQT can accurately reconstruct the density matrix associated with a noisy quantum state experimentally realized in an IBMQ quantum computer. We speculate the success of the AQT stems from its ability to model quantum entanglement across the entire quantum system much as the attention model for NLP captures the correlations among words in a sentence.