Maximum likelihood prediction (MLP) is a core task at the heart of modern large language models. Here, we study a quantum version of this task for a simplified data model consisting of independent and identically distributed samples, as a first step. The quantum maximum likelihood predictor (QMLP) is obtained by embedding of empirical probability distributions into quantum states and performing a minimization of quantum relative entropy over a given class of states. We derive non-asymptotic performance guarantees for QMLP in terms of convergence rates and concentration inequalities, both in trace norm and quantum relative entropy. Our approach provides a unified framework to handle MLP within both classical and quantum LLMs. We also consider the related problem of quantum information projection and generalize the quantum Pythagorean theorem to mixture families specified by possibly non-self-adjoint linear constraints. We further show that the Pythagorean inequality continues to hold in the infinite-dimensional setting whenever the convex information-projection problem attains a finite minimum.
Maximum likelihood prediction (MLP) is a core task at the heart of modern large language models. Here, we study a quantum version of this task for a simplified data model consisting of independent and identically distributed samples, as a first step. The quantum maximum likelihood predictor (QMLP) is obtained by embedding of empirical probability distributions into quantum states and performing a minimization of quantum relative entropy over a given class of states. We derive non-asymptotic performance guarantees for QMLP in terms of convergence rates and concentration inequalities, both in trace norm and quantum relative entropy. Our approach provides a unified framework to handle MLP within both classical and quantum LLMs. We also consider the related problem of quantum information projection and generalize the quantum Pythagorean theorem to mixture families specified by possibly non-self-adjoint linear constraints. We further show that the Pythagorean inequality continues to hold in the infinite-dimensional setting whenever the convex information-projection problem attains a finite minimum.
The Expectation-Maximization (EM) algorithm is a fundamental tool in unsupervised machine learning. It is often used as an efficient way to solve Maximum Likelihood (ML) estimation problems, especially for models with latent variables. It is also the algorithm of choice to fit mixture models: generative models that represent unlabelled points originating from k different processes, as samples from k multivariate distributions. In this work we define and use a quantum version of EM to fit a Gaussian Mixture Model. Given quantum access to a dataset of n vectors of dimension d, our algorithm has convergence and precision guarantees similar to the classical algorithm, but the runtime is only polylogarithmic in the number of elements in the training set, and is polynomial in other parameters - as the dimension of the feature space, and the number of components in the mixture. We generalize further the algorithm in two directions. First, we show how to fit any mixture model of probability distributions in the exponential family. Then, we show how to use this algorithm to compute the Maximum a Posteriori (MAP) estimate of a mixture model: the Bayesian approach to likelihood estimation problems. We discuss the performance of the algorithm on a dataset that is expected to be classified successfully by this algorithm, arguing that on those cases we can give strong guarantees on the runtime.
Conventional methods for computing maximum-likelihood estimators (MLE) often converge slowly in practical situations, leading to a search for simplifying methods that rely on additional assumptions for their validity. In this work, we provide a fast and reliable algorithm for maximum-likelihood reconstruction that avoids this slow convergence. Our method utilizes the state-of-the-art convex optimization scheme, an accelerated projected-gradient method, that allows one to accommodate the quantum nature of the problem in a different way than in the standard methods. We demonstrate the power of our approach by comparing its performance with other algorithms for n-qubit state tomography. In particular, an eight-qubit situation that purportedly took weeks of computation time in 2005 can now be completed in under a minute for a single set of data, with far higher accuracy than previously possible. This refutes the common claim that MLE reconstruction is slow and reduces the need for alternative methods that often come with difficult-to-verify assumptions. In fact, recent methods assuming Gaussian statistics or relying on compressed sensing ideas are demonstrably inapplicable for the situation under consideration here. Our algorithm can be applied to general optimization problems over the quantum state space; the philosophy of projected gradients can further be utilized for optimization contexts with general constraints.
Over the last decade, representation learning, which embeds complex information extracted from large amounts of data into dense vector spaces, has emerged as a key technique in machine learning. Among other applications, it has been a key building block for large language models and advanced computer vision systems based on contrastive learning. A core component of representation learning systems is the projection head, which maps the original embeddings into different, often compressed spaces, while preserving the similarity relationship between vectors. In this paper, we propose a quantum-inspired projection head that includes a corresponding quantum-inspired similarity metric. Specifically, we map classical embeddings onto quantum states in Hilbert space and introduce a quantum circuit-based projection head to reduce embedding dimensionality. To evaluate the effectiveness of this approach, we extended the BERT language model by integrating our projection head for embedding compression. We compared the performance of embeddings, which were compressed using our quantum-inspired projection head, with those compressed using a classical projection head on information retrieval tasks using the TREC 2019 and TREC 2020 Deep Learning benchmarks. The results demonstrate that our quantum-inspired method achieves competitive performance relative to the classical method while utilizing 32 times fewer parameters. Furthermore, when trained from scratch, it notably excels, particularly on smaller datasets. This work not only highlights the effectiveness of the quantum-inspired approach but also emphasizes the utility of efficient, ad hoc low-entanglement circuit simulations within neural networks as a powerful quantum-inspired technique.
We establish a sharp quantitative stability theorem for the von Neumann entropy under a natural class of convex constraints induced by orthogonal block decompositions of finite-dimensional Hilbert spaces. Given a block–convex set of block-diagonal quantum states, we prove that any state whose entropy is within ε of the minimal value must lie within O(ε)O(\sqrt{\varepsilon})O(ε) in trace norm of the manifold of entropy minimizers. The stability exponent is shown to be optimal, and purely quadratic stability is characterized precisely in the confining (fixed-support) regime. The proof relies on a canonical decomposition of entropy into classical (Shannon) and conditional components, combined with sharp relative entropy estimates and geometric properties of the marginal polytope. As an application, we obtain quantitative non-concentration bounds for spectral measures associated with finite spectral decompositions of self-adjoint operators. The framework is entirely finite-dimensional, independent of arithmetic or dynamical assumptions, and provides a general geometric stability principle for entropy minimization under structured convex constraints.
Classical probabilistic models of (noisy) quantum systems are not only relevant for understanding the non-classical features of quantum mechanics, but they are also useful for determining the possible advantage of using quantum resources for information processing tasks. A common feature of these models is the presence of inaccessible information, as captured by the concept of preparation contextuality: There are ensembles of quantum states described by the same density operator, and hence operationally indistinguishable, and yet in any probabilistic (ontological) model, they should be described by distinct probability distributions. In this work, we quantify the inaccessible information of a model in terms of the maximum distinguishability of probability distributions associated to any pair of ensembles with identical density operators, as quantified by the total variation distance of the distributions. We obtain a family of lower bounds on this maximum distinguishability in terms of experimentally measurable quantities. In the case of an ideal qubit this leads to a lower bound of, approximately, 0.07. These bounds can also be interpreted as a new class of robust preparation non-contextuality inequalities. Our non-contextuality inequalities are phrased in terms of generalizations of max-relative entropy and trace distance for general operational theories, which could be of independent interest. Under sufficiently strong noise any quantum system becomes preparation non-contextual, i.e., can be described by models with zero inaccessible information. Using our non-contextuality inequalities, we show that this can happen only if the noise channel has the average gate fidelity less than or equal to 1/D(1+1/2+...+1/D), where D is the dimension of the Hilbert space.
Classical probability theory is based on assumptions which are often violated in practice. Therefore quantum probability is a proposed alternative not only in quantum physics, but also in other sciences. However, so far it mostly criticizes the classical approach, but does not suggest a working alternative. Maximum likelihood estimators were given very low attention in this context. We show that they can be correctly defined and their computation in closed form is feasible at least in some cases.