arXiv:2401.18049 · Physical Review A

Enhanced observable estimation through classical optimization of informationally overcomplete measurement data: Beyond classical shadows

Joonas Malmi, Keijo Korhonen, Daniel Cavalcanti, Guillermo García-Pérez

Published Jun 10, 2024• 17 citations

Abstract

In recent years, informationally complete measurements have attracted considerable attention, especially in the context of classical shadows. In the particular case of informationally overcomplete measurements, for which the number of possible outcomes exceeds the dimension of the space of linear operators in Hilbert space, the dual positive operator-valued measure operators used to interpret the measurement outcomes are not uniquely defined. In this paper, we propose a method to optimize the dual operators after the measurements have been carried out in order to produce sharper, unbiased estimations of observables of interest. We discuss how this procedure can produce zero-variance estimations in cases where the classical shadows formalism, which relies on so-called canonical duals, incurs exponentially large measurement overheads. We also analyze the algorithm in the context of quantum simulation with randomized Pauli measurements, and show that it can significantly reduce statistical errors with respect to canonical duals on multiple observable estimations.

DOI: 10.1103/physreva.109.062412

Related papers

Improving shadow estimation with locally-optimal dual frames

Keijo Korhonen, Stefano Mangini, Joonas Malmi +2 more

Accurate estimation of observables in quantum systems is a central challenge in quantum information science, yet practical implementations are fundamentally constrained by the limited number of measurement shots. In this work we explore a variation of the classical shadows protocol in which the measurements are kept local while allowing the resulting classical shadows themselves to be correlated. By constructing locally optimal shadows, we obtain unbiased estimators that are competitive with state-of-the-art methods in terms of measurement overhead, while requiring only single-qubit measurements and allowing for the estimation of any observable in pure post-processing, reducing estimation errors by orders of magnitude over standard classical shadows. We validate our approach through numerical experiments on molecular Hamiltonians with up to 40 qubits consistently observing significant reductions in estimation errors, including for estimations of multiple chemically relevant observables simultaneously.

quant-pharXiv:2511.02555Physical Review ANov 5, 20250 citations

Dual-frame optimization for informationally complete quantum measurements

Laurin E. Fischer, T. K. L. Dao, Ivano Tavernelli +1 more

Randomized measurement protocols such as classical shadows represent powerful resources for quantum technologies, with applications ranging from quantum state characterization and process tomography to machine learning and error mitigation. Recently, the notion of measurement dual frames, in which classical shadows are generalized to dual operators of positive operator-valued measure (POVM) effects, resurfaced in the literature. This brought attention to additional degrees of freedom in the postprocessing stage of randomized measurements that are often neglected by established techniques. In this work, we leverage dual frames to construct improved observable estimators from informationally complete measurement samples. We introduce novel classes of parametrized frame superoperators and optimization-free dual frames based on empirical frequencies, which offer advantages over their canonical counterparts while retaining computational efficiency. Remarkably, this comes at almost no quantum or classical cost, thus rendering dual frame optimization a valuable addition to the randomized measurement toolbox. Published by the American Physical Society 2024

arXiv:2401.18071Physical Review AJun 11, 202419 citations

Variance reduction methods in the estimation of Pauli sums

Søren Fuglede Jørgensen, Rafael Emilio Barfknecht, Patrick Ettenhuber +1 more

Accurately estimating expectation values of quantum observables with as few measurements as possible is crucial to many quantum computing applications. We introduce a framework that covers many of existing measurement strategies and introduce heuristics that can be used to enhance randomized schemes, including those based on Pauli grouping with inverse probability weighting and variants of the classical shadow algorithm. We show how to maximize information gain from such schemes, while carefully optimizing the distribution of possible measurements, and show that simple grouping algorithms can get close to, and in some cases exceed, state-of-the-art accuracy for unbiased estimation of expectation values on a standard quantum chemistry benchmark. We show how these randomized methods may be compared to more recent measurement schemes, such as shadow grouping, derandomized shadow, and overlapped grouping measurement, we show how the same strategies can be used to augment these schemes, and we demonstrate that we can reduce measurement costs by up to a factor of two by allowing Clifford measurement circuits for otherwise Clifford-less methods.

quant-pharXiv:2603.20029Mar 23, 20260 citations

Post-processing optimization and optimal bounds for non-adaptive shadow tomography

Andrea Caprotti, Joshua Morris, Borivoje Dakić

Informationally overcomplete POVMs are known to outperform minimally complete measurements in many tomography and estimation tasks, and they also leave a purely classical freedom in shadow tomography: the same observable admits infinitely many unbiased linear reconstructions from identical measurement data. We formulate the choice of reconstruction coefficients as a convex minimax problem and give an algorithm with guaranteed convergence that returns the tightest state-independent variance bound achievable by post-processing for a fixed POVM and observable. Numerical examples show that the resulting estimators can dramatically reduce sampling complexity relative to standard (canonical) reconstructions, and can even improve the qualitative scaling with system size for structured noncommuting targets.

quant-pharXiv:2601.16266Jan 26, 20260 citations

Quantum-enhanced metrology with classical shadows

Luca Ion, Manuel Gessner

Quantum metrology relies on observables that maximize information about the parameter of interest, but are often hard to access. We develop a protocol that combines Pauli moments estimated from a fixed ensemble of randomized local measurements into an optimized method-of-moments estimator. For local Pauli classical shadows, we derive the shadow information which determines estimator variance and sample-complexity guarantees. Identifying the separable-state sensitivity limit, we show that entanglement can lead to a constant-factor sensitivity enhancement, as illustrated with one-axis-twisted probes. This protocol allows for efficient circuit-based metrology without choosing an observable.

quant-pharXiv:2610.35632Oct 27, 20260 citations

Minimax Optimal Estimation of Expectation Values

Seshadri, Akshay

There is growing interest in constructing quantum devices that control increasingly large numbers of quantum systems. There is a tremendous need for methods that can measure the quality and other properties of these devices. Such measurements often amount to learning the expectation value of a quantum observable with respect to the quantum state of the device. Learning expectation values is also a key component in many well-known applications such as quantum machine learning and quantum optimization algorithms. Because of the large system sizes involved, it is essential to find methods for learning expectation values that are efficient with respect to the system size. Existing methods that have rigorous guarantees and are practical to implement are often not optimal for observables of interest. Other methods are heuristic, or require the use measurement protocols that are challenging to implement experimentally with current technology. In this study, we propose an estimation procedure, The Optimal Observable expectation value Learner or TOOL, that can learn the expectation values of observables using the outcomes of any given measurement protocol. We show that there is a seminorm on the set of all observables, which we call the minimax norm, that characterizes the smallest possible estimation error for learning a given observable using the outcomes of a given non-adaptive measurement protocol to a specified confidence level in the worst case over all states. We prove that TOOL is minimax optimal for every observable by showing that it can achieve an estimation error to within a small constant factor of the minimax norm. For many applications, one wishes to learn the expectation value of more than one observable from the same experiment. A popular method for learning the expectation values of one or many observables with rigorous guarantees is classical shadows. Classical shadows has near-optimal performance in the worst case over all observables. We prove that TOOL always performs at least as well as classical shadows. Moreover, we show by example that TOOL dramatically outperforms classical shadows for many observables of interest. This highlights the need to characterize the optimal performance for the task of simultaneously learning the expectation values of many observables. Under a mild assumption, we give such a characterization using the minimax norm and prove that TOOL is nearly minimax optimal for this task. We also study the applications of TOOL to fidelity estimation. Using experimental data from a trapped-ion quantum computer, we show that TOOL performs well in practice and matches the estimates obtained from Maximum Likelihood Estimation (MLE), but with rigorous guarantees on the estimation error unlike MLE. We also compare TOOL with another popular method called direct fidelity estimation, which estimates the fidelity by judiciously sampling Pauli observables and measuring them. We show that there is a different importance sampling scheme for Pauli measurements for which TOOL performs as well as, or better than, direct fidelity estimation. Since TOOL constructs an estimator using only the observable, the measurement protocol, and the confidence level, it provides the flexibility to perform estimation for experiments that have already been performed and experiments that will be performed in the future. Similarly, since the minimax norm can be computed beforehand, it can be used to compare the performance of different measurement protocols and allow minimax optimal design of experiments.

Apr 15, 20250 citations