Two Methodological Pitfalls in Matched-Parameter Quantum–Classical Benchmarking for Paraphrase Identification: Class-Imbalance Collapse and Readout-Offset Asymmetry
Quantum machine learning is commonly benchmarked by matching a quantum head’s parameter count to that of a classical baseline. We show that this control is insufficient. We compare a 32-parameter parameterized quantum circuit (PQC) head against a nominally matched 31-parameter multi-layer perceptron (MLP) on three paraphrase tasks—the Microsoft Research Paraphrase Corpus (MRPC), Paraphrase Adversaries from Word Scrambling (PAWS), and Quora Question Pairs (QQP)—and document two pitfalls that inflate an apparent quantum deficit. First, under the tested protocol, unweighted binary cross-entropy on imbalanced data leads to near-all-positive prediction, producing an illusory equivalence. Second, the surviving gap traces to a readout that sits offset from the decision threshold at initialization (⟨Z0⟩=+0.922), and no circuit parameter acts as a threshold shift that could recenter it. A pre-specified calibration experiment removes 61% to 76% of the gap, and a training-free probe detects the condition in one forward pass. Threshold-free metrics then changed our own conclusions. On PAWS, both heads rank barely above chance (area under the receiver operating characteristic curve, AUC, at most 0.508), so the difference there is threshold placement. Only QQP retains a residual, and a post hoc control doubling the qubit count narrows it by 42% without removing it (paired p=0.0009). This pattern is consistent with a capacity or optimization limitation, but does not distinguish between them. Four of five pre-specified predictions failed. Parameter roles and readout offsets must be audited, not only counts.
Quantum machine learning is commonly benchmarked by matching a quantum head’s parameter count to that of a classical baseline. We show that this control is insufficient. We compare a 32-parameter parameterized quantum circuit (PQC) head against a nominally matched 31-parameter multi-layer perceptron (MLP) on three paraphrase tasks—the Microsoft Research Paraphrase Corpus (MRPC), Paraphrase Adversaries from Word Scrambling (PAWS), and Quora Question Pairs (QQP)—and document two pitfalls that inflate an apparent quantum deficit. First, under the tested protocol, unweighted binary cross-entropy on imbalanced data leads to near-all-positive prediction, producing an illusory equivalence. Second, the surviving gap traces to a readout that sits offset from the decision threshold at initialization (⟨Z0⟩=+0.922), and no circuit parameter acts as a threshold shift that could recenter it. A pre-specified calibration experiment removes 61% to 76% of the gap, and a training-free probe detects the condition in one forward pass. Threshold-free metrics then changed our own conclusions. On PAWS, both heads rank barely above chance (area under the receiver operating characteristic curve, AUC, at most 0.508), so the difference there is threshold placement. Only QQP retains a residual, and a post hoc control doubling the qubit count narrows it by 42% without removing it (paired p=0.0009). This pattern is consistent with a capacity or optimization limitation, but does not distinguish between them. Four of five pre-specified predictions failed. Parameter roles and readout offsets must be audited, not only counts.
Rigorous empirical validation of quantum machine learning on natural language tasks remains scarce. We evaluate a 10-qubit hybrid quantum-classical variational circuit (2,148 parameters) for paraphrase detection across three benchmarks: MRPC, Quora Question Pairs (QQP), and adversarial PAWS. On QQP (n = 10 seeds), the circuit achieves 75.53% +/- 0.75%. accuracy, statistically outperforming parameter-matched classical baselines (DeepMLP: p = 0.015, Cohen's d = 1.20; F1: p < 0.001, d = 2.32) and surpassing DistilBERT-4bit with 31,191 fewer parameters. On MRPC the optimal 2-layer variant reaches 92% of BERT-base accuracy at 54,000 lower parameter cost. Circuit depth analysis reveals a dataset-depth scaling effect; entanglement analysis via the Meyer-Wallach measure identifies multi-qubit entanglement as the primary performance driver (r = 0.85 across four variants). Adversarial evaluation on PAWS reveals emergent robustness: 98.2% recall versus 81.6% classical (+16.6 pp, d = 1.24, p < 0.001), without adversarial training. These results constitute the first systematic parameter-matched multi-benchmark empirical validation of hybrid variational circuits for NLP. All results are from classical simulation; hardware validation is future work.
Quantum machine learning (QML) for network intrusion detection (NIDS) is routinely reported to reach near-perfect accuracy, yet the most rigorous studies find that well-tuned classical models remain competitive, and that apparent quantum gains may be artefacts of classical dimensionality reduction and implicit regularisation rather than genuine quantum effects. We ask not whether a quantum model can post a high accuracy, but how quantum the advantage really is. We present a unified, reproducible QML-IDS benchmark evaluating hybrid variational quantum circuits and quantum-kernel SVMs against five honestly-tuned classical baselines across four standard NIDS datasets (NSL-KDD, UNSW-NB15, CICIDS2017, NF-ToN-IoT-v2) under one leakage-controlled protocol, with an equal-budget feature view, imbalance- and calibration-aware metrics with significance testing, and a simulated NISQ noise sweep. We introduce a quantum-attribution audit (parameter-matched classical controls, a random-feature kernel, and a regularisation sweep) that quantifies how much of any gain is genuinely attributable to the quantum component. Tuned classical models (Random Forest, XGBoost) match or exceed the quantum models on aggregate detection on every dataset, and the audit attributes this to classical preprocessing and regularisation rather than quantum effects. Two advantages survive false-discovery-rate correction: the quantum-kernel SVM out-ranks its direct classical surrogate (a random-feature kernel) on AUPRC and ROC-AUC, and a small four-qubit hybrid out-detects the best classical baseline at the 1% false-positive operating point on the distribution-shifted NSL-KDD task (p = 0.005, BH q = 0.030). Code, seeds, and splits are released; our contribution stands whether quantum wins, ties, or loses.
Abstract A recurring weakness in quantum machine learning (QML) is that reported ‘quantum advantages’ are seldom tested against a capacity-matched classical control, leaving it unclear whether a gain comes from the quantum substrate or from the architectural change that accompanies it. Our primary contribution is methodological: a protocol for attributing such gains honestly—a capacity-matched classical bottleneck of identical parameter budget, transparent reporting of where quantum does not help, and validation on real quantum hardware—which we develop and apply through a concrete case study. That case study is Quantum Adaptive Self-Attention (QASA), a hybrid Transformer that replaces the value projection of a single encoder layer with a 36-parameter parameterized quantum circuit (PQC), keeping all other layers classical. Across nine synthetic benchmarks and the real-world ETTh1 dataset, QASA improves on a full-capacity classical Transformer for chaotic and trend-dominated signals. To ask whether this is a genuinely quantum effect, we introduce a control rarely applied in QML—a capacity-matched classical bottleneck with the same parameter budget—and find that it matches the PQC on the error metrics. The gain is therefore attributable to the low-rank value-projection bottleneck (an architectural parsimony principle), not to quantumness; adding further quantum layers only degrades performance and trainability. We accordingly position the quantum layer not as a source of accuracy advantage but as a competitive instantiation of this principle: its low-rank compression onto the signal’s intrinsic dimensionality is matched by a classical bottleneck, so the gain is architectural rather than quantum. The quantum layer’s distinguishing features are physical—a high circuit entanglement (Meyer–Wallach Q = 0.981 with only 27 CNOTs, which we report as a circuit-level property) and, most concretely, noisy intermediate-scale quantum deployability, which we verify by executing the trained model on a real IBM Quantum processor (one-step prediction within 7.5 % of noiseless simulation on the quantum-favored task and 25 % , with overlapping error bars, on a classical-favored control; no error mitigation). Relative to other quantum sequence models (quantum long short-term memory (QLSTM), QnnFormer), QASA is moreover the most resource-efficient of the three: it reaches competitive accuracy with the fewest quantum parameters (36 vs 90–128) and the strongest entanglement-per-gate ( Q = 0.981 with 27 CNOTs vs QLSTM’s 56), and retains trainable gradients that deeper circuits lose. We argue that capacity-matched baselines and honest reporting of where quantum does not help are prerequisites for credible quantum-machine-learning claims.
We test whether a parameterized quantum circuit (PQC) improves a hybrid quantum-classical model's performance on classical datasets, using an interface-matched classical map as the control while holding all other components fixed. Our architecture, Quantum-Embedded Attention (QEA), uses a learnable projector to compress backbone features into an nq-dimensional angle vector, a shallow PQC to map those angles to one- and two-qubit Pauli expectations, and a classical attention decoder to produce class logits. We hypothesized the PQC would improve accuracy or seed-to-seed stability over a classical map with matched input/output dimensions. We test this with an interface-matched 2×2 factorial on Breast Cancer Wisconsin at nq∈{4,8}, independently swapping the PQC for a classical map and the attention decoder for a linear head, across five paired seeds per cell. Three of four paired quantum-minus-classical 95% confidence intervals include zero; the fourth, a +1.63 percentage-point contrast for the attention decoder at nq=4, reverses sign at nq=8 and does not survive correction across the four contrasts. The experiment thus shows no consistent PQC contribution and cannot establish equivalence. A five-dataset cross-modality grid shows comparable accuracy on AG~News, Breast Cancer Wisconsin, and BirdCLEF but a large deficit on CIFAR-10; these cells are not interface-matched and are interpreted descriptively. We report all planned canonical runs, distinguish current Pauli-readout results from legacy probability-readout experiments, and analyze bottleneck, simulation, finite-shot, and noise limitations. The results do not establish a quantum advantage; they demonstrate why controlled component attribution is necessary before crediting a hybrid model's performance to its quantum layer.
As public-key cryptography is migrated to post-quantum standards, it creates an urgent need to triage which of the tens of thousands of publicly disclosed software vulnerabilities are cryptographically critical and among those, so vulnerable to fault-tolerant quantum adversaries. This triage is framed as a supervised text-classification problem on the CVE/CWE corpus (1999-2025, ~2.8×10^5 data points), and we propose a two-stage pipeline—(i) a transparent-labels rule-based first quote to isolate crypto-only records & assign Shor/Grover-driven levels of quantum Ness risk using an unsupervised clustering approach, followed by (ii) a hybrid quantum/classical classifier that combines addition-detection multivariate TF–IDF + CVSS features with four-qubit variational quantum circuit (VQC/Ansatz)-based VQCs trained via the parameter-shift rule in PennyLane & PyTorch.The hybrid model reaches 87.86% accuracy and a macro-F1 of 0.479 (binary task) and 49.13% accuracy (four-class task) against classical baselines (logistic regression, RBF-SVM, random forest, and a matched multilayer perceptron), is comparable to the classical neural baseline, but remains below the tree-ensemble upper bound. A qubit-count ablation indicates a small but consistent macro-F1 degradation from four to six qubits, empirically consistent with the barren-plateau phenomenon. We take the means over random seeds to report all results, analyze the contribution of extreme class imbalance dominating performance, and discuss ramifications for operations through the lens of prioritizing cryptographic remediation in U.S. defense networks. We present a feasibility study predictive of performance, with honest benchmarks based on experimental data, not a claim of quantum advantage.
Abstract Quantum-inspired (QI) optimizers are increasingly proposed as alternatives to classical gradient-based training, yet are frequently benchmarked through classical stand-ins rather than themselves. Across tabular, vision, and language tasks we evaluate three classes: direct implementations of published update rules (Spall’s SPSA; quantum-behaved particle swarm optimization, QPSO, adapted to stochastic minibatches), a documented classical analog of quantum natural gradient (an NGD-style diagonal second-moment preconditioner), and the hyperparameter-control configurations under which such methods are often reported. The gap between these classes is our central result: on the tabular benchmark the hyperparameter control trains with the classical baselines while the direct algorithms fall ≈ 10–24 points below it, and on CIFAR-10/ResNet18, where v1’s “SPSA” arm was a learning-rate-scaled Adam, Adam itself stands ≈ 59 points above genuine SPSA. The QPSO adaptation stays at chance on LSTM tasks a legacy “QPSO” proxy solves at 79–81%, at 18–38 × Adam’s cost there and up to 74 × on the small CV model. A seed-paired 2×2 factorial on RoBERTa identifies the smaller learning rate as the dominant observed component of the earlier “QNG advantage” ( +0.50 points at the dataset level); with only three datasets, modified Hartung–Knapp uncertainty spans zero. The configured NGD arm shows no detectable advantage over a learning-rate-matched AdamW (descriptive pooled p=0.23 ) at 6– 7% extra cost, and on ResNet18 its trained endpoints have larger dominant-curvature estimates than the Adam family’s. Headline comparisons use five seeds, paired tests, Holm correction, and stated detection limits. We conclude with an auditing taxonomy (direct algorithm / analog / hyperparameter control): within the arms, protocols, and scales evaluated, no advantage attributable to a QI mechanism is detected after these controls.