Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2607.17219

Auditing Question-Order Effects in Large Language Models with the QQ Equality: Mechanism Characterization and a Saturation Caveat

Pilsung Kang

cs.CLcs.AIquant-phstat.MESubmitted Jul 19, 2026• Announced Jul 21, 2026• 0 citations
v1 · Jul 19, 2026v2 · Jul 30, 2026
arXivHTMLPDF

Abstract

Human survey respondents exhibit question-order effects that satisfy the QQ (quantum question) equality, an a priori, parameter-free prediction of the projective quantum question-order model. We develop the QQ equality into an audit criterion for sequential judgments of autoregressive large language models (LLMs). Theoretically, we characterize which mechanism classes satisfy it robustly: marginal-independent kernels satisfy QQ iff all four mismatch transition rates coincide (a class containing the 2D rank-1 projective model with a fixed measurement pair under state variation); a polarity- and position-dependent repetition family is characterized by an exact cross-symmetry condition with closed-form violations; QQ-satisfying behaviors are closed under order-matched mixing; and the rank-2 Contextuality-by-Default criterion translates into audit coordinates as $|\qQQ|\le\OSS$, where $\OSS$ (the order-sensitivity score) totals the order sensitivity of the two marginals. Methodologically, we develop a pre-specified, audit-logged pipeline applicable to any model exposing next-token log-probabilities; it combines worst-case robustness envelopes, sampling-consistency spot checks, full label counterbalancing, and a saturation diagnostic. Empirically, in a first-signal pilot on an open-weight instruction-tuned model under two framings, all pre-specified health gates passed, yet 17/18 and 7/8 item pairs, respectively, were saturated (near-deterministic), and no item was certified residually contextual. Forced-binary next-token log-probabilities were thus inadequate for distribution-level QQ audits under the tested model and prompting conditions; we recommend pre-specified saturation diagnostics whenever next-token distributions are treated as survey-response distributions.

Related papers

A Quantum Question Order Model Supported by Empirical Tests of anA Prioriand Precise Prediction

Zheng Wang, Jerome R. Busemeyer

Question order effects are commonly observed in self-report measures of judgment and attitude. This article develops a quantum question order model (the QQ model) to account for four types of question order effects observed in literature. First, the postulates of the QQ model are presented. Second, an a priori, parameter-free, and precise prediction, called the QQ equality, is derived from these mathematical principles, and six empirical data sets are used to test the prediction. Third, a new index is derived from the model to measure similarity between questions. Fourth, we show that in contrast to the QQ model, Bayesian and Markov models do not generally satisfy the QQ equality and thus cannot account for the reported empirical data that support this equality. Finally, we describe the conditions under which order effects are predicted to occur, and we review a broader range of findings that are encompassed by these very same quantum principles. We conclude that quantum probability theory, initially invented to explain order effects on measurements in physics, appears to be a powerful natural explanation for order effects of self-report measures in social and behavioral sciences, too.

Topics in Cognitive Science•Sep 11, 2013•220 citations

Modeling psychological measurements with quantum instruments: Combination of question order effect, response replicability effect, and QQ-equality

Masanao Ozawa, Andrei Yu. Khrennikov

We continue to analyze matching of some basic constraints on human's decision making with quantum theory of measurement. As was found, quantum measurement theory based on the projection postulate does not match with combination of the question order effect (QOE) and the response replicability effect (RRE). This was the alarm signal for quantum-like modeling of decision making. Recently, it was shown that this objection to quantum-like modeling can be removed on the basis of the general measurement theory based on quantum instruments. In the present paper, the problem of combination QOE, RRE with the famous QQ-equality is analyzed. This equality was derived by Busemeyer and Wang and it was shown (in the joint paper with Solloway and Shiffrin) that statistical data from many social opinion polls satisfies it. Now, we construct quantum instruments satisfying QQE, RRE, and QQ-inequality. Moreover, we show that this model reproduces the statistics of the famous Clinton-Gore Poll data almost faithfully with a prior belief state independent of the question order. This model successfully removes the order effect from the data to determine the genuine distribution of the opinion to the Poll. Independently of modeling of the concrete psychological effects, the paper provides psychologist-friendly presentation of the theory of quantum instruments - the most general mathematical framework for quantum measurements. We hope that this theory will attract attention of psychologists and generate further psychological applications.

•Oct 11, 2020•0 citations

Modeling combination of question order effect, response replicability effect, and QQ-equality with quantum instruments

Masanao Ozawa, Andrei Yu. Khrennikov

We continue to analyze basic constraints on human's decision making from the viewpoint of quantum measurement theory (QMT). As has been found, the conventional QMT based on the projection postulate cannot account for combination of the question order effect (QOE) and the response replicability effect (RRE). This was an alarm signal for quantum-like modeling of decision making. Recently, it was shown that this objection to quantum-like modeling can be removed on the basis of the general QMT based on quantum instruments. In the present paper we analyse the problem of combination of QOE, RRE, and the famous QQ-equality (QQE). This equality was derived by Busemeyer and Wang and it was shown (in the joint paper with Solloway and Shiffrin) that statistical data from many social opinion polls satisfies it. Now, we construct quantum instruments satisfying QOE, RRE, and QQE. The general features of our approach are formalized with postulates which generalize {\it Wang-Busemeyer} postulates for quantum-like modeling of decision making. Moreover, we show that our model closely reproduces the statistics of the famous Clinton-Gore Poll data with a prior belief state independent of the question order. This model successfully removes the order effect from the data to determine the genuine distribution of the opinions in the Poll. The paper also provides a psychologist-friendly introduction to the theory of quantum instruments - the most general mathematical framework for quantum measurements. We hope that this theory will attract attention of psychologists and will stimulate further applications.

arXiv:2010.10444•Journal of Mathematical Psychology•Dec 26, 2020•55 citations

An LLM as a controllable, fully-inspectable model of measurement: which features of quantum measurement are generic to bounded inference, and which are not

Tomas Pødenphant Lund

An autoregressive language model, read at the token-commit step, is a near-fully-inspectable classical bounded-inference substrate: the competing-route distribution over candidate tokens — the live propensity before it resolves into one committed answer — is read directly from the output logits. Used as a measurement model, it lets each structural signature of quantum measurement be tested one by one, and the genuinely-quantum residue isolated by its absence. The substrate reproduces the phenomenology of quantum measurement from classical self-conditioning — back-action, non-commutativity (order effects), entropy-collapse on commitment, a substantial Busemeyer–Wang QQ-structure (order-effect difference q not distinguishable from the parameter-free quantum value 0; pooled across eight de-saturated base models over five vendors and four architectures, N=96, q=-0.003, 95% CI [-0.025, +0.019]), interference, einselection, an Aharonov-style weak-versus-strong asymmetry, and measurement-mode dependence — but not the quantum core: genuine contextuality is robustly absent (an analog Contextuality-by-Default cyclic-4 analysis gives CNT < 0 in every model and scenario), and no phase/coherence is demonstrated. A controllable analog Bell-loophole testbed makes each premise's price visible within one substrate (S_analog ~ 2.00 with all loopholes closed; ~ 2.8 on opening locality or measurement-independence), and a within-substrate timeline reads the propensity resolving across network depth, including a maximally-contested case whose outcome is never created. The order-effect incompatibility is shown to be a measured, correctly-signed classical-complementarity dial that falls as the readout partition refines. Positioning. The contribution is the demarcation itself, with statistics, made possible by an inspectability no veiled (physical or psychological) substrate allows. It instantiates beim Graben & Atmanspacher's classical complementarity and the Busemeyer–Wang / Pothos quantum-cognition program empirically; it makes no claim about physical reality. Companion papers (Friction Theory series): Paper 1 — Friction as the cost of probabilistic computation (10.5281/zenodo.20012654); Paper 10 — Race all the way down/up (10.5281/zenodo.20014567); Paper 21 — LLMs as a measurement model for bounded-decision cognition (10.5281/zenodo.20562416); Paper 28 — Dread without a dreader (10.5281/zenodo.20690747).Paper 18 in the Friction Theory paper-series. Submitted as a careful, honestly-scoped empirical demarcation (target venue: Entropy, Quantum Information / quantum-cognition & foundations-of-measurement). v2 (August 2026) — prior-art revision. The paper describes competition resolving to a commitment and cited none of the tradition that formalised it. Luce (1959) for the choice axiom the softmax implements, Ratcliff (1978) for evidence accumulated to a bound, and Usher and McClelland (2001) for the leaky competing accumulator are now credited at the point where the reading is made, with the limit stated: the estimators here are computed on a language model's output distribution, and the parallel is why the measurement was worth making rather than evidence of a shared mechanism. Editorial pass. Earlier versions remain in the version history.

•Jun 16, 2026•0 citations

A Classical Candidate-Geometry Baseline Audit Framework for Quantum-like Contextuality in AI

John Jude Hathway

This preprint defines the classical candidate-geometry baselines that should be compared before any claim of non-classicality in the quantum-like contextual non-additivity, order effects, and Bell-type indicators reported for large language models (LLMs) and masked language models (MLMs), and quantifies their explanatory power and limits. It does not claim physical quantum coherence in AI, nor a full explanation of AI quantum-like behavior. The revised edition adds four components. (i) Detection-power certification: the detector's condition is ICC < S_odd - 2, and Tsirelson-level injection tests certify that genuine contextuality of that magnitude would be detectable at the empirical signaling levels; the 312 Experiment-4 tables all satisfy S_odd < 2 with zero corrected excess, so the negative result is not attributable to ICC-induced blindness. (ii) A product-distribution theorem establishing that the zero excess of independent scoring is a mathematical necessity, not an empirical finding. (iii) A structural reproduction of the Lo et al. construction with a no-model null: uniformly random marginals without any language model already reach a CbD-contextual rate of 0.667 (vs. 0.711 reported), and the real BERT/RoBERTa rates are largely indistinguishable from a marginal-matched sign-randomized null at substantive effect sizes. (iv) A multi-model audit across six models from four families: raw S_odd > 2 appears (max 2.773) but is entirely explained by direct influence, with zero corrected excess across all 144 tables and on the powered subset. The package includes the manuscript (LaTeX source and PDF), figures, reproduction code, and data (including the raw joint probabilities from which the per-table metrics are regenerated and cross-checked). Central thesis: reading AI quantum-like behavior as non-classicality requires exceeding not the raw score but the combination of classical candidate-geometry baselines, direct-influence correction, detector-power certification, and a no-model null. This work is neither a full reproduction nor a refutation of Lo et al.; it audits the interpretation of raw contextual-instance counts. Manuscript language: English. Code is released under the MIT License; text and figures under CC BY 4.0. Note: Parts of the manuscript were linguistically and structurally refined with the assistance of AI-based tools.All scientific content, analysis, and conclusions are the author's own.

•Jun 13, 2026•0 citations

Benchmarking the Quantum Leap: Auditing LLM Evaluation for Quantum Code Generation

Ethan Seefried, Romit Ghosh, Sharmin Afrose +4 more

Large language models (LLMs) can generate quantum programs, but reliable evaluation remains difficult because benchmarks differ in task composition, scoring, and failure handling. We evaluate 15 hosted frontier, open-weight, quantum-specialized, and retrieval-augmented models on five quantum code-generation suites (530 tasks), plus 10 models on our new QuantumBenchEval (QBE) benchmark (96 tasks across six topics). We report bootstrap confidence intervals, multiplicity-adjusted paired comparisons, and audits of output validity and evaluator behavior. QiskitHumanEval-Standard approaches saturation (98.7% pass@1), while its Hard variant retains headroom (63.8%). None of the five top-two comparisons across primary suites remains significant after multiple-comparison correction. Evaluator design substantially changes scores: removing checks for unspecified quantities raises GPT-6 Astra's QBE pass@1 from 0% to 94.1% (T1) and from 1.2% to 92.9% (T6), without changing model outputs. We identify 17 QuanBench-117 tasks with shared failures linked to benchmark-owned imports. Controlled probes also reveal unexecuted unit tests, leakage of canonical functions into candidate execution, and reference imports that block otherwise valid candidates. These defects can produce both false failures and false passes, so historical QuanBench results require revalidation under an isolated evaluator. Overall, leaderboard scores alone do not adequately characterize progress in quantum code generation. We release our harness, per-sample results, and audit artifacts and recommend validating evaluators, isolating candidate code, and reporting uncertainty and output validity.

cs.SECross-listedarXiv:2610.28678•Oct 20, 2026•0 citations
Human survey respondents exhibit question-order effects that satisfy the QQ (quantum question) equality, an a priori, parameter-free prediction of the projective quantum question-order model. We develop the QQ equality into an audit criterion for sequential judgments of autoregressive large language models (LLMs). Theoretically, we characterize which mechanism classes satisfy it robustly: marginal-independent kernels satisfy QQ iff all four mismatch transition rates coincide (a class containing the 2D rank-1 projective model with a fixed measurement pair under state variation); a polarity- and position-dependent repetition family is characterized by an exact cross-symmetry condition with closed-form violations; QQ-satisfying behaviors are closed under order-matched mixing; and the rank-2 Contextuality-by-Default criterion translates into audit coordinates as ∣\qQQ∣≤\OSS|\qQQ|\le\OSS∣\qQQ∣≤\OSS, where \OSS\OSS\OSS (the order-sensitivity score) totals the order sensitivity of the two marginals. Methodologically, we develop a pre-specified, audit-logged pipeline applicable to any model exposing next-token log-probabilities; it combines worst-case robustness envelopes, sampling-consistency spot checks, full label counterbalancing, and a saturation diagnostic. Empirically, in a first-signal pilot on an open-weight instruction-tuned model under two framings, all pre-specified health gates passed, yet 17/18 and 7/8 item pairs, respectively, were saturated (near-deterministic), and no item was certified residually contextual. Forced-binary next-token log-probabilities were thus inadequate for distribution-level QQ audits under the tested model and prompting conditions; we recommend pre-specified saturation diagnostics whenever next-token distributions are treated as survey-response distributions.
Question order effects are commonly observed in self-report measures of judgment and attitude. This article develops a quantum question order model (the QQ model) to account for four types of question order effects observed in literature. First, the postulates of the QQ model are presented. Second, an a priori, parameter-free, and precise prediction, called the QQ equality, is derived from these mathematical principles, and six empirical data sets are used to test the prediction. Third, a new index is derived from the model to measure similarity between questions. Fourth, we show that in contrast to the QQ model, Bayesian and Markov models do not generally satisfy the QQ equality and thus cannot account for the reported empirical data that support this equality. Finally, we describe the conditions under which order effects are predicted to occur, and we review a broader range of findings that are encompassed by these very same quantum principles. We conclude that quantum probability theory, initially invented to explain order effects on measurements in physics, appears to be a powerful natural explanation for order effects of self-report measures in social and behavioral sciences, too.
We continue to analyze matching of some basic constraints on human's decision making with quantum theory of measurement. As was found, quantum measurement theory based on the projection postulate does not match with combination of the question order effect (QOE) and the response replicability effect (RRE). This was the alarm signal for quantum-like modeling of decision making. Recently, it was shown that this objection to quantum-like modeling can be removed on the basis of the general measurement theory based on quantum instruments. In the present paper, the problem of combination QOE, RRE with the famous QQ-equality is analyzed. This equality was derived by Busemeyer and Wang and it was shown (in the joint paper with Solloway and Shiffrin) that statistical data from many social opinion polls satisfies it. Now, we construct quantum instruments satisfying QQE, RRE, and QQ-inequality. Moreover, we show that this model reproduces the statistics of the famous Clinton-Gore Poll data almost faithfully with a prior belief state independent of the question order. This model successfully removes the order effect from the data to determine the genuine distribution of the opinion to the Poll. Independently of modeling of the concrete psychological effects, the paper provides psychologist-friendly presentation of the theory of quantum instruments - the most general mathematical framework for quantum measurements. We hope that this theory will attract attention of psychologists and generate further psychological applications.
We continue to analyze basic constraints on human's decision making from the viewpoint of quantum measurement theory (QMT). As has been found, the conventional QMT based on the projection postulate cannot account for combination of the question order effect (QOE) and the response replicability effect (RRE). This was an alarm signal for quantum-like modeling of decision making. Recently, it was shown that this objection to quantum-like modeling can be removed on the basis of the general QMT based on quantum instruments. In the present paper we analyse the problem of combination of QOE, RRE, and the famous QQ-equality (QQE). This equality was derived by Busemeyer and Wang and it was shown (in the joint paper with Solloway and Shiffrin) that statistical data from many social opinion polls satisfies it. Now, we construct quantum instruments satisfying QOE, RRE, and QQE. The general features of our approach are formalized with postulates which generalize {\it Wang-Busemeyer} postulates for quantum-like modeling of decision making. Moreover, we show that our model closely reproduces the statistics of the famous Clinton-Gore Poll data with a prior belief state independent of the question order. This model successfully removes the order effect from the data to determine the genuine distribution of the opinions in the Poll. The paper also provides a psychologist-friendly introduction to the theory of quantum instruments - the most general mathematical framework for quantum measurements. We hope that this theory will attract attention of psychologists and will stimulate further applications.
An autoregressive language model, read at the token-commit step, is a near-fully-inspectable classical bounded-inference substrate: the competing-route distribution over candidate tokens — the live propensity before it resolves into one committed answer — is read directly from the output logits. Used as a measurement model, it lets each structural signature of quantum measurement be tested one by one, and the genuinely-quantum residue isolated by its absence. The substrate reproduces the phenomenology of quantum measurement from classical self-conditioning — back-action, non-commutativity (order effects), entropy-collapse on commitment, a substantial Busemeyer–Wang QQ-structure (order-effect difference q not distinguishable from the parameter-free quantum value 0; pooled across eight de-saturated base models over five vendors and four architectures, N=96, q=-0.003, 95% CI [-0.025, +0.019]), interference, einselection, an Aharonov-style weak-versus-strong asymmetry, and measurement-mode dependence — but not the quantum core: genuine contextuality is robustly absent (an analog Contextuality-by-Default cyclic-4 analysis gives CNT < 0 in every model and scenario), and no phase/coherence is demonstrated. A controllable analog Bell-loophole testbed makes each premise's price visible within one substrate (S_analog ~ 2.00 with all loopholes closed; ~ 2.8 on opening locality or measurement-independence), and a within-substrate timeline reads the propensity resolving across network depth, including a maximally-contested case whose outcome is never created. The order-effect incompatibility is shown to be a measured, correctly-signed classical-complementarity dial that falls as the readout partition refines. Positioning. The contribution is the demarcation itself, with statistics, made possible by an inspectability no veiled (physical or psychological) substrate allows. It instantiates beim Graben & Atmanspacher's classical complementarity and the Busemeyer–Wang / Pothos quantum-cognition program empirically; it makes no claim about physical reality. Companion papers (Friction Theory series): Paper 1 — Friction as the cost of probabilistic computation (10.5281/zenodo.20012654); Paper 10 — Race all the way down/up (10.5281/zenodo.20014567); Paper 21 — LLMs as a measurement model for bounded-decision cognition (10.5281/zenodo.20562416); Paper 28 — Dread without a dreader (10.5281/zenodo.20690747).Paper 18 in the Friction Theory paper-series. Submitted as a careful, honestly-scoped empirical demarcation (target venue: Entropy, Quantum Information / quantum-cognition & foundations-of-measurement). v2 (August 2026) — prior-art revision. The paper describes competition resolving to a commitment and cited none of the tradition that formalised it. Luce (1959) for the choice axiom the softmax implements, Ratcliff (1978) for evidence accumulated to a bound, and Usher and McClelland (2001) for the leaky competing accumulator are now credited at the point where the reading is made, with the limit stated: the estimators here are computed on a language model's output distribution, and the parallel is why the measurement was worth making rather than evidence of a shared mechanism. Editorial pass. Earlier versions remain in the version history.
This preprint defines the classical candidate-geometry baselines that should be compared before any claim of non-classicality in the quantum-like contextual non-additivity, order effects, and Bell-type indicators reported for large language models (LLMs) and masked language models (MLMs), and quantifies their explanatory power and limits. It does not claim physical quantum coherence in AI, nor a full explanation of AI quantum-like behavior. The revised edition adds four components. (i) Detection-power certification: the detector's condition is ICC < S_odd - 2, and Tsirelson-level injection tests certify that genuine contextuality of that magnitude would be detectable at the empirical signaling levels; the 312 Experiment-4 tables all satisfy S_odd < 2 with zero corrected excess, so the negative result is not attributable to ICC-induced blindness. (ii) A product-distribution theorem establishing that the zero excess of independent scoring is a mathematical necessity, not an empirical finding. (iii) A structural reproduction of the Lo et al. construction with a no-model null: uniformly random marginals without any language model already reach a CbD-contextual rate of 0.667 (vs. 0.711 reported), and the real BERT/RoBERTa rates are largely indistinguishable from a marginal-matched sign-randomized null at substantive effect sizes. (iv) A multi-model audit across six models from four families: raw S_odd > 2 appears (max 2.773) but is entirely explained by direct influence, with zero corrected excess across all 144 tables and on the powered subset. The package includes the manuscript (LaTeX source and PDF), figures, reproduction code, and data (including the raw joint probabilities from which the per-table metrics are regenerated and cross-checked). Central thesis: reading AI quantum-like behavior as non-classicality requires exceeding not the raw score but the combination of classical candidate-geometry baselines, direct-influence correction, detector-power certification, and a no-model null. This work is neither a full reproduction nor a refutation of Lo et al.; it audits the interpretation of raw contextual-instance counts. Manuscript language: English. Code is released under the MIT License; text and figures under CC BY 4.0. Note: Parts of the manuscript were linguistically and structurally refined with the assistance of AI-based tools.All scientific content, analysis, and conclusions are the author's own.
Large language models (LLMs) can generate quantum programs, but reliable evaluation remains difficult because benchmarks differ in task composition, scoring, and failure handling. We evaluate 15 hosted frontier, open-weight, quantum-specialized, and retrieval-augmented models on five quantum code-generation suites (530 tasks), plus 10 models on our new QuantumBenchEval (QBE) benchmark (96 tasks across six topics). We report bootstrap confidence intervals, multiplicity-adjusted paired comparisons, and audits of output validity and evaluator behavior. QiskitHumanEval-Standard approaches saturation (98.7% pass@1), while its Hard variant retains headroom (63.8%). None of the five top-two comparisons across primary suites remains significant after multiple-comparison correction. Evaluator design substantially changes scores: removing checks for unspecified quantities raises GPT-6 Astra's QBE pass@1 from 0% to 94.1% (T1) and from 1.2% to 92.9% (T6), without changing model outputs. We identify 17 QuanBench-117 tasks with shared failures linked to benchmark-owned imports. Controlled probes also reveal unexecuted unit tests, leakage of canonical functions into candidate execution, and reference imports that block otherwise valid candidates. These defects can produce both false failures and false passes, so historical QuanBench results require revalidation under an isolated evaluator. Overall, leaderboard scores alone do not adequately characterize progress in quantum code generation. We release our harness, per-sample results, and audit artifacts and recommend validating evaluators, isolating candidate code, and reporting uncertainty and output validity.