Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2611.14210

Tree-Ising: Repairing Language Models with Bit-Level Ising Edits

Hadi Salloum, Ammar Ali

Published Nov 10, 2026• 0 citations
arXivHTMLPDF

Abstract

Weight-space programming, introduced in recent MIT work by Takehi et al., enables a frontier LLM agent to read a network's weights and write a repair. Their GPT-6 Astra agent corrected the reasoning loops of Qwen3.5-0.8B without gradients, using 158 agent turns, 89.7 million tokens, and 50 forward-pass requests. Our method, Tree-Ising, casts this repair as a sparse discrete optimization problem over the model's own BF16 weights. Forward statistics define an objective that weakens a loop direction, strengthens a concluding direction, and preserves the layer's output on productive tokens. Within a state-tree representation, where lossless BF16 storage acts as a special case, the admissible edits are sign flips and exponent shifts that preserve all shared group exponents and mantissas. We decompose this objective exactly into independent QUBO (Ising) problems, one per output row, which we solve on a GPU using simulated annealing or quantum-inspired simulated bifurcation. On the agent's two layers, Tree-Ising achieves 54.6% on MATH-500, outperforming the agent's edit (51.7%) under the same evaluation protocol, while the base model scores 7.9% as reported. The simulated bifurcation solver yields a statistically indistinguishable repair at 55.1%. Both solvers outperform the agent edit across all five math benchmarks, with statistically significant improvements on MATH-500 and GSM8K after correcting for multiple comparisons. Our approach modifies 13% to 17% fewer weights and reduces BF16 bit flips by a factor of 2.2 to 2.3. It requires only four forward-pass requests and zero LLM tokens, with the annealing recipe costing about $0.90 in GPU time. On Qwen3.5-2B, with layers and dose selected using only the base model, the same recipe increases greedy GSM8K accuracy from 21.6% to 69.4%. As a proof of concept, we execute an eight-variable repair subproblem using QAOA on an IBM Heron processor.

DOI
10.48550/arxiv.2611.14210

Related papers

Weight-Space Programming of Variational Quantum Circuits: Large Language Models Write Quantum Circuit Parameters Without an Optimizer

Debarshi Kundu, Swaroop Ghosh, Vasant Honavar

Variational quantum algorithms (VQAs) optimize circuit angles through repeated quantum-device evaluations, often facing vanishing gradients and high optimization costs. We ask whether a large language model (LLM) can write these angles directly. Extending *weight-space programming* from classical networks to quantum circuits, we provide a frontier LLM (\gpt) with the problem, circuit, and gate conventions. Its angles are executed exactly as written, without returning losses, gradients, or accuracy feedback. Across 139 requests, we test handwritten-digit classification, VQE ground-state estimation for spin chains and H$_2$, and MaxCut QAOA, comparing against random weights, classical optimization, and optimization initialized with LLM weights on the same circuit. When parameters have interpretable structure, the LLM matches or approaches the optimizer: it achieves 95--100\% of the energy spectrum on every VQE problem, derives the exact H$_2$ ground state in closed form, finds optimal depth-one QAOA angles for every graph, and reaches 93--96\% accuracy on digits 3 versus 8 with 12--18 hand-written weights, outperforming an optimizer trained on the same ten examples. Its weights are also strong warm starts: optimization reaches its best value after a median of one evaluation rather than hundreds. Allowed to design circuits, the LLM chooses data-dependent rotation chains encoding pixel importance in all 30 cases (one qubit for binary tasks, one chain per qubit for ten classes), reaching 65\% accuracy across ten digits. Yet the approach fails in opaque weight spaces: accuracy drops to chance when fixed circuits exceed 30 weights, while optimization improves. It is also sensitive to gate conventions: 6 of 42 QAOA replies reverse mixing-angle signs and fail. We release all prompts, replies, and scores.

arXiv:2611.11537•Nov 8, 2026•1 citation

LLM Compression by Block Removal with Constrained Binary Optimization

David Jansen, Roman Rausch, Ali Hashemi +2 more

In this paper, we formulate the compression of large language models (LLMs) by optimally deleting transformer blocks (``block removal'') as a constrained binary optimization (CBO) problem that can be mapped to a physical system (Ising glass), whose energies are a strong proxy for downstream model performance. This formulation enables an efficient ranking of a large number of candidate block-removal configurations yielding many high-quality, non-trivial solutions beyond those only removing consecutive regions. Our method performs strongly in the deep compression regime, such as for 50% compression of Llama-3.3-70B-Instruct, where we achieve an almost 23 percentage point increase on the MMLU benchmark compared to other state-of-the-art (SOTA) block-removal methods. For lighter compression, it performs on par with those methods across several benchmarks for Llama-3.1-8B-Instruct, Qwen3-14B (both before and after retraining), as well as Llama-3.3-70B-Instruct. The approach is computationally efficient and requires only forward and backward passes on a calibration dataset for a few active parameters. Additionally, we demonstrate that using good heuristic solvers for the CBO problem provides solutions that perform well on downstream tasks in negligible runtime when it is unfeasible to solve the problem exactly. The method can be readily applied to any architecture. We illustrate this generality on the recent NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 model, which exhibits a highly inhomogeneous and challenging block structure, and where we outperform SOTA for AIME25 and GPQA when removing either 2 attention layers or 3 mixture-of-experts layers.

cs.LGCross-listedarXiv:2602.00161•Feb 2, 2026

Semantic-Preserving QUBO Encodings for Fault Tree Analysis: Hierarchical Repair and Constraint-Level Formulations

Devin Peters

Quadratic Unconstrained Binary Optimization formulations for fault tree analysis encode gate logic through penalty terms, but prior work has demonstrated that these encodings fail to preserve fault tree semantics at scale: across 80 extended fault tree instances, only 7.5% of Rosenberg-based soft QUBO solutions match the constrained exact baseline, while 27.5% produce outright semantic violations. This paper develops two complementary methods to restore semantic correctness. A Hierarchical Semantic Validator and repair framework achieves 100% semantic validity across all 80 cohort instances through structure-aware bit-flip correction, improving the exact match rate from 7.5% to 28.75%. A constraint-level QUBO encoding eliminates penalty-based failures entirely by restricting optimization to the feasible subspace, reducing qubit requirements by a factor of 1.85x relative to Rosenberg encodings. Validation through Qiskit mixer circuits implementing a feasibility-preserving Quantum Alternating Operator Ansatz confirms zero probability leakage onto infeasible states for representative instances. These methods establish that semantic-preserving quantum optimization for fault tree minimal cut set identification is achievable with current algorithmic tools. Complete experimental artifacts with SHA-256 verification are provided under the DOI above.

•Feb 21, 2026•0 citations

E8-QAT: Efficient 2-Bit Quantization-Aware Training with Implicit Lattice Codebooks

Xinyu Lei, Hong Huang, Jiebin Zhang +6 more

Low-bit quantization reduces the memory footprint of large language models, but preserving accuracy with efficient training remains challenging. Scalar quantization encodes weights independently, while explicit-codebook vector quantization introduces costly search during training. We propose E8-QAT, a 2-bit quantization-aware training framework that uses the $E_8$ lattice as an implicit eight-dimensional codebook. It jointly quantizes weight groups through arithmetic projection without storing or searching an explicit codebook during the training forward pass. A 2-bit core encoding and sparse extension preserve the selected lattice points exactly, adding less than 0.1\% to core index storage. On Qwen3.6-35B-A3B, E8-QAT matches the performance of the full-precision model in terms of the average score across the four tasks. E8-QAT improves over the state-of-the-art baseline by 2.03\% on IFEval and by 1.51\% on LiveCodeBench, under the same data, schedule, and bit width. The 2-bit representation reduces peak memory by 80\% and supports a $4\times$ larger batch. At each representation's largest feasible batch, E8-QAT reaches $1.97\times$ the decoding throughput of BF16.

arXiv:2610.27699•Oct 20, 2026•1 citation

Benchmarking Large Language Models on Repairing Qiskit Programs using Bugs4Q

Saumya Brahmbhatt, Mitali Hukkeri, Dongchan Kim +2 more

In quantum programs, Bugs4Q is a widely used benchmark containing real quantum defects. However, its evaluation assumes that benchmark labels remain valid and that generated fixes execute in the target environment. We evaluate two Bugs4Q versions containing 67 unique real Qiskit defects, adding executable tests where missing, and re-run all entries across six pinned Qiskit releases (0.25.0, 0.45.0, 1.0.0, 1.1.1, 2.0.0, and 2.3.1). We find that quantum benchmarks can suffer from silent label inversion: entries become invalid without errors when reference fixes stop executing or buggy programs no longer reproduce failures. Thus, correctness depends on the (benchmark, version) pair rather than the benchmark alone. We evaluate four LLMs (GPT-4o-mini, GPT-5o-mini, GPT-5.4, and GPT-5.4-mini), generating up to 10 repair candidates per defect and testing them across all versions. GPT-5.4 achieves the highest pass@10 (48.8%), followed by GPT-5.4-mini (47.3%), GPT-5o-mini (30.3%), and GPT-4o-mini (22.6%). All models perform best on Qiskit 0.45.0 and decline after the Qiskit 1.0 transition. Many failures arise from deprecated or incompatible APIs rather than incorrect repairs, and 64\% of successful repairs occur on entries invalid under the target version. We release a re-validated, version-pinned Bugs4Q benchmark and show that benchmark validation must precede repair evaluation.

cs.SECross-listedarXiv:2607.09007•Jul 13, 2026•0 citations

Quantum-Enhanced LLM Cascade Routing: A QAOA Approach to Cost-Optimal Model Selection in Multi-Agent Systems

AMIT SAMSON PATOLE

The rapid proliferation of large language model(LLM) powered multi-agent systems creates a non-trivial combinatorialoptimization problem: routing heterogeneous tasksto the most cost-effective model tier while maintaining qualityguarantees. Current production systems rely on static lookuptables, which over-provision expensive models and waste computationalbudget. We formalize the LLM Cascade Routing Problem(LCRP) as a Quadratic Unconstrained Binary Optimization(QUBO) problem and solve it using the Quantum ApproximateOptimization Algorithm (QAOA). We benchmark QAOA againstgreedy heuristics and simulated annealing using both Google Cirqsimulation and real IBM Quantum hardware (156-qubit Heronprocessors). Experiments across three IBM backends (ibm fez,ibm kingston, ibm marrakesh) on problem instances from 6 to18 qubits reveal three key findings: (i) shallow QAOA circuits(p=1, depth 52) achieve 24–25% valid assignment rate on realhardware versus <1% for deeper circuits (p = 2, depth 101),a 25× improvement reproduced across all three backends; (ii)hardware constraint satisfaction degrades steeply with problemsize, dropping from 24–26% at 6 qubits to 0.9–1.4% at 18qubits for p = 1; and (iii) results are reproducible across allthree backends with consistent valid rates within ±0.5% at12 qubits. We further evaluate warm-start initialization fromclassical solutions and feasibility-first decoding, demonstratingthat post-processing can recover 100% valid assignments fromnoisy hardware output. To our knowledge, this is the firstquantum computing formulation of the LLM model routingproblem. We provide an open-source implementation with 28verifiable IBM Quantum job IDs.Index Terms—Quantum Computing, QAOA, LLM Routing,Multi-Agent Systems, Combinatorial Optimization, QUBO, NISQ

•Apr 7, 2026•0 citations
Weight-space programming, introduced in recent MIT work by Takehi et al., enables a frontier LLM agent to read a network's weights and write a repair. Their GPT-6 Astra agent corrected the reasoning loops of Qwen3.5-0.8B without gradients, using 158 agent turns, 89.7 million tokens, and 50 forward-pass requests. Our method, Tree-Ising, casts this repair as a sparse discrete optimization problem over the model's own BF16 weights. Forward statistics define an objective that weakens a loop direction, strengthens a concluding direction, and preserves the layer's output on productive tokens. Within a state-tree representation, where lossless BF16 storage acts as a special case, the admissible edits are sign flips and exponent shifts that preserve all shared group exponents and mantissas. We decompose this objective exactly into independent QUBO (Ising) problems, one per output row, which we solve on a GPU using simulated annealing or quantum-inspired simulated bifurcation. On the agent's two layers, Tree-Ising achieves 54.6% on MATH-500, outperforming the agent's edit (51.7%) under the same evaluation protocol, while the base model scores 7.9% as reported. The simulated bifurcation solver yields a statistically indistinguishable repair at 55.1%. Both solvers outperform the agent edit across all five math benchmarks, with statistically significant improvements on MATH-500 and GSM8K after correcting for multiple comparisons. Our approach modifies 13% to 17% fewer weights and reduces BF16 bit flips by a factor of 2.2 to 2.3. It requires only four forward-pass requests and zero LLM tokens, with the annealing recipe costing about $0.90 in GPU time. On Qwen3.5-2B, with layers and dose selected using only the base model, the same recipe increases greedy GSM8K accuracy from 21.6% to 69.4%. As a proof of concept, we execute an eight-variable repair subproblem using QAOA on an IBM Heron processor.
Variational quantum algorithms (VQAs) optimize circuit angles through repeated quantum-device evaluations, often facing vanishing gradients and high optimization costs. We ask whether a large language model (LLM) can write these angles directly. Extending *weight-space programming* from classical networks to quantum circuits, we provide a frontier LLM (\gpt) with the problem, circuit, and gate conventions. Its angles are executed exactly as written, without returning losses, gradients, or accuracy feedback. Across 139 requests, we test handwritten-digit classification, VQE ground-state estimation for spin chains and H2_22​, and MaxCut QAOA, comparing against random weights, classical optimization, and optimization initialized with LLM weights on the same circuit. When parameters have interpretable structure, the LLM matches or approaches the optimizer: it achieves 95--100\% of the energy spectrum on every VQE problem, derives the exact H2_22​ ground state in closed form, finds optimal depth-one QAOA angles for every graph, and reaches 93--96\% accuracy on digits 3 versus 8 with 12--18 hand-written weights, outperforming an optimizer trained on the same ten examples. Its weights are also strong warm starts: optimization reaches its best value after a median of one evaluation rather than hundreds. Allowed to design circuits, the LLM chooses data-dependent rotation chains encoding pixel importance in all 30 cases (one qubit for binary tasks, one chain per qubit for ten classes), reaching 65\% accuracy across ten digits. Yet the approach fails in opaque weight spaces: accuracy drops to chance when fixed circuits exceed 30 weights, while optimization improves. It is also sensitive to gate conventions: 6 of 42 QAOA replies reverse mixing-angle signs and fail. We release all prompts, replies, and scores.
In this paper, we formulate the compression of large language models (LLMs) by optimally deleting transformer blocks (``block removal'') as a constrained binary optimization (CBO) problem that can be mapped to a physical system (Ising glass), whose energies are a strong proxy for downstream model performance. This formulation enables an efficient ranking of a large number of candidate block-removal configurations yielding many high-quality, non-trivial solutions beyond those only removing consecutive regions. Our method performs strongly in the deep compression regime, such as for 50% compression of Llama-3.3-70B-Instruct, where we achieve an almost 23 percentage point increase on the MMLU benchmark compared to other state-of-the-art (SOTA) block-removal methods. For lighter compression, it performs on par with those methods across several benchmarks for Llama-3.1-8B-Instruct, Qwen3-14B (both before and after retraining), as well as Llama-3.3-70B-Instruct. The approach is computationally efficient and requires only forward and backward passes on a calibration dataset for a few active parameters. Additionally, we demonstrate that using good heuristic solvers for the CBO problem provides solutions that perform well on downstream tasks in negligible runtime when it is unfeasible to solve the problem exactly. The method can be readily applied to any architecture. We illustrate this generality on the recent NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 model, which exhibits a highly inhomogeneous and challenging block structure, and where we outperform SOTA for AIME25 and GPQA when removing either 2 attention layers or 3 mixture-of-experts layers.
Quadratic Unconstrained Binary Optimization formulations for fault tree analysis encode gate logic through penalty terms, but prior work has demonstrated that these encodings fail to preserve fault tree semantics at scale: across 80 extended fault tree instances, only 7.5% of Rosenberg-based soft QUBO solutions match the constrained exact baseline, while 27.5% produce outright semantic violations. This paper develops two complementary methods to restore semantic correctness. A Hierarchical Semantic Validator and repair framework achieves 100% semantic validity across all 80 cohort instances through structure-aware bit-flip correction, improving the exact match rate from 7.5% to 28.75%. A constraint-level QUBO encoding eliminates penalty-based failures entirely by restricting optimization to the feasible subspace, reducing qubit requirements by a factor of 1.85x relative to Rosenberg encodings. Validation through Qiskit mixer circuits implementing a feasibility-preserving Quantum Alternating Operator Ansatz confirms zero probability leakage onto infeasible states for representative instances. These methods establish that semantic-preserving quantum optimization for fault tree minimal cut set identification is achievable with current algorithmic tools. Complete experimental artifacts with SHA-256 verification are provided under the DOI above.
Low-bit quantization reduces the memory footprint of large language models, but preserving accuracy with efficient training remains challenging. Scalar quantization encodes weights independently, while explicit-codebook vector quantization introduces costly search during training. We propose E8-QAT, a 2-bit quantization-aware training framework that uses the E8E_8E8​ lattice as an implicit eight-dimensional codebook. It jointly quantizes weight groups through arithmetic projection without storing or searching an explicit codebook during the training forward pass. A 2-bit core encoding and sparse extension preserve the selected lattice points exactly, adding less than 0.1\% to core index storage. On Qwen3.6-35B-A3B, E8-QAT matches the performance of the full-precision model in terms of the average score across the four tasks. E8-QAT improves over the state-of-the-art baseline by 2.03\% on IFEval and by 1.51\% on LiveCodeBench, under the same data, schedule, and bit width. The 2-bit representation reduces peak memory by 80\% and supports a 4×4\times4× larger batch. At each representation's largest feasible batch, E8-QAT reaches 1.97×1.97\times1.97× the decoding throughput of BF16.
In quantum programs, Bugs4Q is a widely used benchmark containing real quantum defects. However, its evaluation assumes that benchmark labels remain valid and that generated fixes execute in the target environment. We evaluate two Bugs4Q versions containing 67 unique real Qiskit defects, adding executable tests where missing, and re-run all entries across six pinned Qiskit releases (0.25.0, 0.45.0, 1.0.0, 1.1.1, 2.0.0, and 2.3.1). We find that quantum benchmarks can suffer from silent label inversion: entries become invalid without errors when reference fixes stop executing or buggy programs no longer reproduce failures. Thus, correctness depends on the (benchmark, version) pair rather than the benchmark alone. We evaluate four LLMs (GPT-4o-mini, GPT-5o-mini, GPT-5.4, and GPT-5.4-mini), generating up to 10 repair candidates per defect and testing them across all versions. GPT-5.4 achieves the highest pass@10 (48.8%), followed by GPT-5.4-mini (47.3%), GPT-5o-mini (30.3%), and GPT-4o-mini (22.6%). All models perform best on Qiskit 0.45.0 and decline after the Qiskit 1.0 transition. Many failures arise from deprecated or incompatible APIs rather than incorrect repairs, and 64\% of successful repairs occur on entries invalid under the target version. We release a re-validated, version-pinned Bugs4Q benchmark and show that benchmark validation must precede repair evaluation.
The rapid proliferation of large language model(LLM) powered multi-agent systems creates a non-trivial combinatorialoptimization problem: routing heterogeneous tasksto the most cost-effective model tier while maintaining qualityguarantees. Current production systems rely on static lookuptables, which over-provision expensive models and waste computationalbudget. We formalize the LLM Cascade Routing Problem(LCRP) as a Quadratic Unconstrained Binary Optimization(QUBO) problem and solve it using the Quantum ApproximateOptimization Algorithm (QAOA). We benchmark QAOA againstgreedy heuristics and simulated annealing using both Google Cirqsimulation and real IBM Quantum hardware (156-qubit Heronprocessors). Experiments across three IBM backends (ibm fez,ibm kingston, ibm marrakesh) on problem instances from 6 to18 qubits reveal three key findings: (i) shallow QAOA circuits(p=1, depth 52) achieve 24–25% valid assignment rate on realhardware versus <1% for deeper circuits (p = 2, depth 101),a 25× improvement reproduced across all three backends; (ii)hardware constraint satisfaction degrades steeply with problemsize, dropping from 24–26% at 6 qubits to 0.9–1.4% at 18qubits for p = 1; and (iii) results are reproducible across allthree backends with consistent valid rates within ±0.5% at12 qubits. We further evaluate warm-start initialization fromclassical solutions and feasibility-first decoding, demonstratingthat post-processing can recover 100% valid assignments fromnoisy hardware output. To our knowledge, this is the firstquantum computing formulation of the LLM model routingproblem. We provide an open-source implementation with 28verifiable IBM Quantum job IDs.Index Terms—Quantum Computing, QAOA, LLM Routing,Multi-Agent Systems, Combinatorial Optimization, QUBO, NISQ