Document ranking based on probabilistic evaluations of relevance is known to exhibit non-classical correlations, which may be explained by admitting a complex structure of the event space, namely, by assuming the events to emerge from multiple sample spaces. The structure of event space formed by overlapping sample spaces is known in quantum mechanics, they may exhibit some counter-intuitive features, called quantum contextuality. In this Note I observe that from the structural point of view quantum contextuality looks similar to personalization of information retrieval scenarios. Along these lines, Knowledge Revision is treated as operationalistic measurement and a way to quantify the rate of personalization of Information Retrieval scenarios is suggested.
Document ranking based on probabilistic evaluations of relevance is known to exhibit non-classical correlations, which may be explained by admitting a complex structure of the event space, namely, by assuming the events to emerge from multiple sample spaces. The structure of event space formed by overlapping sample spaces is known in quantum mechanics, they may exhibit some counter-intuitive features, called quantum contextuality. In this Note I observe that from the structural point of view quantum contextuality looks similar to personalization of information retrieval scenarios. Along these lines, Knowledge Revision is treated as operationalistic measurement and a way to quantify the rate of personalization of Information Retrieval scenarios is suggested.
Probabilistic models require the notion of event space for defining a probability measure. An event space has a probability measure which ensues the Kolmogorov axioms. However, the probabilities observed from distinct sources, such as that of relevance of documents, may not admit a single event space thus causing some issues. In this article, some results are introduced for ensuring whether the observed prob- abilities of relevance of documents admit a single event space. More- over, an alternative framework of probability is introduced, thus chal- lenging the use of classical probability for ranking documents. Some reflections on the convenience of extending the classical probabilis- tic retrieval toward a more general framework which encompasses the issues are made.
In this thesis we investigate the use of quantum probability theory for ranking documents. Quantum probability theory is used to estimate the probability of relevance of a document given a user's query. We posit that quantum probability theory can lead to a better estimation of the probability of a document being relevant to a user's query than the common IR approach, i. e. the Probability Ranking Principle (PRP), which is based upon Kolmogorovian probability theory. Following our hypothesis, we formulate an analogy between the document retrieval scenario and a physical scenario, that of the double slit experiment. Through the analogy, we propose a novel ranking approach, the quantum probability ranking principle (qPRP). Key to our proposal is the presence of quantum interference. Mathematically, this is the statistical deviation between empirical observations and expected values predicted by the Kolmogorovian rule of additivity of probabilities of disjoint events in configurations such that of the double slit experiment. While PRP explicitly assumes that the relevancy of a document is independent of that of other documents, we suggest that qPRP implicitly models interdependent document relevance through quantum interference and thus is suited to those document ranking tasks where the independence assumption fails. Throughout the thesis, we also suggest how quantum interference can be estimated for effective document ranking. To validate our proposal and to gain more insights about approaches for document ranking, we (1) analyse PRP, qPRP and other ranking approaches, exposing the assumptions underlying their ranking criteria and formulating the conditions for the optimality of the two ranking principles, (2) empirically compare three ranking principles (i. e. PRP, interactive PRP, and qPRP) and two state-of-the-art ranking strategies in two retrieval scenarios, those of ad-hoc retrieval and diversity retrieval, (3) analytically contrast the ranking criteria of the examined approaches, exposing similarities and differences, (4) study the ranking behaviours of approaches alternative to PRP in terms of the kinematics they impose on relevant documents, i. e. by considering the extent and direction of the movements of relevant documents across the ranking recorded when comparing PRP against its alternatives. Our findings show that the effectiveness of the examined ranking approaches strongly depends upon the evaluation context. In the traditional evaluation context of ad-hoc retrieval, PRP is empirically shown to be better than or comparable to alternative ranking approaches. However, when evaluation contexts that account for interdependent document relevance are examined (i. e. when the relevance of a document is assessed also with respect to other retrieved documents, as it is the case in the diversity retrieval scenario), the use of quantum probability theory and thus of qPRP is shown to improve retrieval and ranking effectiveness over the traditional PRP and alternative ranking strategies, such as Maximal Marginal Relevance, Portfolio theory, and Interactive PRP. This work represents a significant step forward regarding the use of quantum theory in information retrieval. It demonstrates that the application of quantum theory to problems within information retrieval can lead to improvements both in modelling power and retrieval effectiveness, allowing the constructions of models that capture the complexity of information retrieval situations. Furthermore, the thesis opens up a number of lines of future research. These include investigating estimations and approximations of quantum interference in qPRP, exploiting complex numbers for the representation of documents and queries, and applying the concepts underlying qPRP to tasks other than document ranking. This dissertation was completed at School of Computing Science, University of Glasgow under the advise of Dr. Leif Azzopardi and Prof. Keith van Rijsbergen. Prof. Norbert Fuhr, Dr. Iadh Ounis, and Dr. John O'Donnell served as dissertation committee members. For the full dissertation, visit: http://theses.gla.ac.uk/3463.
Quantum-inspired algorithms represent an important direction in modern software information technologies that use heuristic methods and approaches of quantum science. This work presents a quantum approach for document search, retrieval, and ranking based on the Bell-like test, which is well-known in quantum physics. We propose quantum probability theory in the hyperspace analog to language (HAL) framework exploiting a Hilbert space for word and document vector specification. The quantum approach allows for accounting for specific user preferences in different contexts. To verify the algorithm proposed, we use a dataset of synthetic advertising text documents from travel agencies generated by the OpenAI GPT-4 model. We show that the "entanglement" in two-word document search and retrieval can be recognized as the frequent occurrence of two words in incompatible query contexts. We have found that the user preferences and word ordering in the query play a significant role in relatively small sizes of the HAL window. The comparison with the cosine similarity metrics demonstrates the key advantages of our approach based on the user-enforced contextual and semantic relationships between words and not just their superficial occurrence in texts. Our approach to retrieving and ranking documents allows for the creation of new information search engines that require no resource-intensive deep machine learning algorithms.
This thesis is fundamentally about using conceptual and mathematical constructs from the area of Quantum Theory in Information Retrieval (IR). The need and motivation for this is two-fold – firstly, it has been increasingly shown in decision sciences that human decision-making does not always conform to the norms of traditional probability and logic framework. The quantum framework offers a generalised probability and logic framework which can model decisions or judgements under dynamic context and ambiguity. Secondly, there is a need in IR for theories and models which improve our understanding of user behaviour. Hence it is worth exploring the combination of the quantum framework and IR, especially focused on the user aspects of IR. The overarching research question is whether there is evidence of user behaviour in IR scenarios which warrants the need for a quantum based approach by way of showing the limitations of the traditional (classical) approach. The methodology involves analysing data to detect quantum-like phenomena of interference, contextuality, incompatibility, etc. from two common data sources in IR–standard datasets like query log data, and through crowdsourced user studies designed similar to some quantum physics or cognitive science experiments. While the evidence of quantum-like phenomena from standard datasets is not convincing, we find that some of the user studies reveal the quantum-like structure of document judgements. One of the key findings which has implication for IR is the dynamic interactions between the different dimensions of relevance. For example, a user’s judgement of reliability of a document depends significantly on whether they found it understandable or not. Thus, the consideration of one relevance dimension or document feature can provide a context for another dimension, contrary to the current IR models which consider these features to be independent of each other and an objective property of the document. The quantum framework has been especially designed to deal with such scenarios where properties of systems or objects do not exist independent of measurement context. The thesis concludes with suggestions about incorporating quantum mathematical constructs into state-of-art IR algorithms.
The probabilistic formalism of quantum physics is said to provide a sound basis for building a principled information retrieval framework. Such a framework can be based on the notion of information need vector spaces where events, such as document relevance or observed user interactions, correspond to subspaces. As in quantum theory, a probability distribution over these subspaces is defined through weighted sets of state vectors (density operators), and used to represent the current view of the retrieval system on the user information need. Tensor spaces can be used to capture different aspects of information needs. Our evaluation shows that the framework can lead to acceptable performance in an ad-hoc retrieval task. Going beyond this, we discuss the potential of the framework for three active challenges in information retrieval, namely, interaction, novelty and diversity.
Recent research on quantum contextuality has been strongly centered on device-independent frameworks, such as the many graph approaches to contextuality and the celebrated sheaf-theoretical approach. Contextuality is described in these frameworks as a property of data only, making it possible to characterize and quantify the phenomena regardless of the reasons why it occurs. In this paper we look beyond the data and focus on possible explanations for this experimental fact. We show that a classical system generating contextual data can easily be found if the following conditions are satisfied (1) We only have access to a specific collection of epistemic measurements (which, all things considered, is basically Bohr's view on quantum measurements) and (2) There is a limitation on which of these measurements can be jointly performed. The way we see it, this example indicates that contextuality may be a consequence of the type of measurement taken into account, instead of an intrinsic feature of the system upon which these measurements are performed; if this is correct, the widespread idea that quantum contextuality is a non-classical feature can be avoided.