What is the fastest Artificial Intelligence Large Language Model (AI LLM) for generating quantum operations? To answer this, we present the first benchmarking study comparing popular and publicly available AI models tasked with creating quantum gate designs. The Wolfram Mathematica framework was used to interface with the six AI LLMs, including Google Gemini 2.0 Flash, Anthropic Claude 3 Haiku, WolframLLM Notebook Assistant For Mathematica V14.3.0.0, OpenAI ChatGPT Omni 4 Mini, Google Gemma 3 4b 1t, and DeepSeek Chat V3. Our novel study found the following: (1) Gemini 2.0 Flash is overall the fastest AI LLM of the models tested in producing average quantum gate designs at 2.66101 s, factoring in the “thinking” execution time and ServiceConnect network latencies. (2) On average, four out of the ten quantum operations that the six LLMs produced compiled in Python version 3.13.5 (40.8% success rate). (3) Quantum operations averaged approximately 21–45 Lines of Code (omitting nonsensical outliers). (4) DeepSeek Chat V3 produced the shortest code with an average of 21.6 lines. This comparison evaluates the time taken by each AI LLM platform to generate quantum operations (including ServiceConnect networking times). These findings highlight a promising horizon where publicly available Large Language Models can become fast collaborators with quantum computers, enabling rapid quantum gate synthesis and paving the way for greater interoperability between two remarkable and cutting-edge technologies.
What is the fastest Artificial Intelligence Large Language Model (AI LLM) for generating quantum operations? To answer this, we present the first benchmarking study comparing popular and publicly available AI models tasked with creating quantum gate designs. The Wolfram Mathematica framework was used to interface with the six AI LLMs, including Google Gemini 2.0 Flash, Anthropic Claude 3 Haiku, WolframLLM Notebook Assistant For Mathematica V14.3.0.0, OpenAI ChatGPT Omni 4 Mini, Google Gemma 3 4b 1t, and DeepSeek Chat V3. Our novel study found the following: (1) Gemini 2.0 Flash is overall the fastest AI LLM of the models tested in producing average quantum gate designs at 2.66101 s, factoring in the “thinking” execution time and ServiceConnect network latencies. (2) On average, four out of the ten quantum operations that the six LLMs produced compiled in Python version 3.13.5 (40.8% success rate). (3) Quantum operations averaged approximately 21–45 Lines of Code (omitting nonsensical outliers). (4) DeepSeek Chat V3 produced the shortest code with an average of 21.6 lines. This comparison evaluates the time taken by each AI LLM platform to generate quantum operations (including ServiceConnect networking times). These findings highlight a promising horizon where publicly available Large Language Models can become fast collaborators with quantum computers, enabling rapid quantum gate synthesis and paving the way for greater interoperability between two remarkable and cutting-edge technologies.
Large language models (LLMs) promise transformative change to fields as diverse as medical diagnosis, legal services, and software development. One reason for such an impact is LLMs’ ability to make highly technical endeavors more accessible to a broader audience. Accessibility has long been a goal for the growing fields of quantum computing, informatics, and engineering, especially as more quantum systems become publicly available via cloud interfaces. Between programming quantum computers and using LLMs, the latter seems the more accessible task: while leveraging an LLM’s fullest potential requires experience with prompt engineering, any literate person can provide queries and read responses. By contrast, designing and executing quantum programs—outside of those available online—requires significant background knowledge, from selection of operations for algorithm implementation to configuration choices for particular hardware specifications and providers. Current research is exploring LLM utility for classical software development, but there has been relatively little investigation into the same for quantum programming. Consequently, this work is a first look at how well an uncustomized, publicly available LLM can write straightforward quantum circuits. We examine how well OpenAI’s ChatGPT (GPT-4) can write quantum circuits for two hardware providers: the superconducting qubit machines of IBM and the photonic devices of Xanadu. We find that ChatGPT currently fares substantially better with the former.
We demonstrate that large language model (LLM) agents can autonomously perform tensor network simulations of quantum many-body systems, achieving approximately 90% success rate across representative benchmark tasks. Tensor network methods are powerful tools for quantum simulation, but their effective use requires expertise typically acquired through years of graduate training. By combining in-context learning with curated documentation and multi-agent decomposition, we create autonomous AI agents that can be trained in specialized computational domains within minutes. We benchmark three configurations (baseline, single-agent with in-context learning, and multi-agent with in-context learning) on problems spanning quantum phase transitions, open quantum system dynamics, and photochemical reactions. Systematic evaluation using DeepSeek-V3.2, Gemini 2.5 Pro, and Claude Opus 4.5 demonstrates that both in-context learning and multi-agent architecture are essential. Analysis of failure modes reveals characteristic patterns across models, with the multi-agent configuration substantially reducing implementation errors and hallucinations compared to simpler architectures.
We present a systematic evaluation of large language models on quantum mechanics problem-solving. Our study evaluates 15 models from five providers (OpenAI, Anthropic, Google, Alibaba, DeepSeek) spanning three capability tiers on 20 tasks covering derivations, creative problems, non-standard concepts, and numerical computation, comprising 900 baseline and 75 tool-augmented assessments. Results reveal clear tier stratification: flagship models achieve 81\% average accuracy, outperforming mid-tier (77\%) and fast models (67\%) by 4pp and 14pp respectively. Task difficulty patterns emerge distinctly: derivations show highest performance (92\% average, 100\% for flagship models), while numerical computation remains most challenging (42\%). Tool augmentation on numerical tasks yields task-dependent effects: modest overall improvement (+4.4pp) at 3x token cost masks dramatic heterogeneity ranging from +29pp gains to -16pp degradation. Reproducibility analysis across three runs quantifies 6.3pp average variance, with flagship models demonstrating exceptional stability (GPT-5 achieves zero variance) while specialized models require multi-run evaluation. This work contributes: (i) a benchmark for quantum mechanics with automatic verification, (ii) systematic evaluation quantifying tier-based performance hierarchies, (iii) empirical analysis of tool augmentation trade-offs, and (iv) reproducibility characterization. All tasks, verifiers, and results are publicly released.
We present a systematic evaluation of large language models on quantum mechanics problem-solving. Our study evaluates 15 models from five providers (OpenAI, Anthropic, Google, Alibaba, DeepSeek) spanning three capability tiers on 20 tasks covering derivations, creative problems, non-standard concepts, and numerical computation, comprising 900 baseline and 75 tool-augmented assessments. Results reveal clear tier stratification: flagship models achieve 81% average accuracy, outperforming mid-tier (77%) and fast models (67%) by 4pp and 14pp respectively. Task difficulty patterns emerge distinctly: derivations show highest performance (92% average, 100% for flagship models), while numerical computation remains most challenging (42%). Tool augmentation on numerical tasks yields task-dependent effects: modest overall improvement (+4.4pp) at 3x token cost masks dramatic heterogeneity ranging from +29pp gains to -16pp degradation. Reproducibility analysis across three runs quantifies 6.3pp average variance, with flagshipmodels demonstrating exceptional stability (GPT-5 achieves zero variance) while specialized models require multi-run evaluation. This work contributes: (i) a benchmark for quantum mechanics with automatic verification, (ii) systematic evaluation quantifying tier-based performance hierarchies, (iii) empirical analysis of tool augmentation trade-offs, and (iv) reproducibility characterization. All tasks, verifiers, and results are publicly released.
Large Language Models (LLMs) contribute significantly to the development of conversational AI and has great potentials to assist the scientific research in various areas. This paper attempts to address the following questions: What opportunities do the current generation of generative pre-trained transformers (GPTs) offer for the developments of noisy intermediate-scale quantum (NISQ) technologies? Additionally, what potentials does the forthcoming generation of GPTs possess to push the frontier of research in fault-tolerant quantum computing (FTQC)? In this paper, we implement a QGAS model, which can rapidly propose promising ansatz architectures and evaluate them with application benchmarks including quantum chemistry and quantum finance tasks. Our results demonstrate that after a limited number of prompt guidelines and iterations, we can obtain a high-performance ansatz which is able to produce comparable results that are achieved by state-of-the-art quantum architecture search methods. This study provides a simple overview of GPT's capabilities in supporting quantum computing research while highlighting the limitations of the current GPT at the same time. Additionally, we discuss futuristic applications for LLM in quantum research.
Recent advances in Large Language Models (LLMs) have demonstrated strong potential in code generation, yet their effectiveness in quantum computing remains underexplored. This paper benchmarks LLMs for PennyLane-based quantum code generation using real-world challenges from the Quantum Hackathon (QHack). We introduce QHackBench, a novel benchmark dataset derived from QHack competitions, and evaluate model performance under vanilla prompting and Retrieval-Augmented Generation (RAG). Our structured evaluation framework assesses functional correctness, syntactic validity, and execution success across varying challenge difficulties. Results indicate that RAG-enhanced models, supplemented with an augmented PennyLane dataset, approximately generate similar results as the standard prompting, particularly in complex quantum algorithms. Additionally, we introduce a multi-agent evaluation pipeline that iteratively refines incorrect solutions, further enhancing execution success rates. To foster further research, we commit to publicly releasing QHackBench, along with our evaluation framework and experimental results, enabling continued advancements in AI-assisted quantum programming.