Commercially available Noisy Intermediate-Scale Quantum (NISQ) devices now make small hybrid quantum-classical experiments practical, but many tools hide configuration or demand ad-hoc scripting.
We introduce the Quantum Experiment Framework (QEF): A lightweight framework designed to support the systematic, hypothesis-driven study of quantum algorithms. Unlike many existing approaches, QEF emphasises iterative, exploratory analysis of evolving experimental strategies rather than exhaustive empirical evaluation of fixed algorithms using predefined quality metrics. The framework's design is informed by a comprehensive review of the literature, identifying principal parameters and measurement practices currently reported in the field.
QEF captures all key aspects of quantum software and algorithm experiments through a concise specification that expands into a Cartesian product of variants for controlled large-scale parameter sweeps. This design enables rigorous and systematic evaluation, as well as precise reproducibility. Large sweeps are automatically partitioned into asynchronous jobs across simulators or cloud hardware, and ascertain full hyper-parameter traceability. QEF supports parameter reuse to improve overall experiment runtimes, and collects all metrics and metadata into a form that can be conveniently explored with standard statistical and visualisation software.
By combining reproducibility and scalability while avoiding the complexities of full workflow engines, QEF seeks to lower the practical barriers to empirical research on quantum algorithms, whether these are designed for current NISQ devices or future error-corrected quantum systems.
Comments
to appear as a workshop paper at ICSOC 2025 (SQS 2025)
Commercially available Noisy Intermediate-Scale Quantum (NISQ) devices now make small hybrid quantum-classical experiments practical, but many tools hide configuration or demand ad-hoc scripting.
We introduce the Quantum Experiment Framework (QEF): A lightweight framework designed to support the systematic, hypothesis-driven study of quantum algorithms. Unlike many existing approaches, QEF emphasises iterative, exploratory analysis of evolving experimental strategies rather than exhaustive empirical evaluation of fixed algorithms using predefined quality metrics. The framework's design is informed by a comprehensive review of the literature, identifying principal parameters and measurement practices currently reported in the field.
QEF captures all key aspects of quantum software and algorithm experiments through a concise specification that expands into a Cartesian product of variants for controlled large-scale parameter sweeps. This design enables rigorous and systematic evaluation, as well as precise reproducibility. Large sweeps are automatically partitioned into asynchronous jobs across simulators or cloud hardware, and ascertain full hyper-parameter traceability. QEF supports parameter reuse to improve overall experiment runtimes, and collects all metrics and metadata into a form that can be conveniently explored with standard statistical and visualisation software.
By combining reproducibility and scalability while avoiding the complexities of full workflow engines, QEF seeks to lower the practical barriers to empirical research on quantum algorithms, whether these are designed for current NISQ devices or future error-corrected quantum systems.
Reproducibility and comparability remain fundamental challenges in the empirical evaluation of Quantum Software Engineering (QSE) tools. These challenges arise from the intrinsic probabilistic behavior of quantum programs and the variability introduced by quantum hardware and simulator backends. While existing quantum benchmarks typically offer only static circuit definitions, they often leave researchers responsible for data generation and execution, hindering the creation of standardized and reproducible experiments. In this paper, we introduce QSimBench, a novel, open-source benchmark suite and accompanying Python library designed to address these limitations. QSimBench provides over 20 million precomputed shot-level execution traces:20,000measurements each, across14quantum algorithms, multiple circuit sizes (4−15qubits), and six Qiskit-based backends including both ideal and noise-injected simulations. Each configuration includes not only detailed measurement histories but also the original circuit in OpenQASM, the backend's noise model, and configuration metadata, ensuring full transparency and enabling in-depth analysis. QSimBench empowers researchers to conduct scalable, reproducible, and statistically rigorous evaluations of QSE tools, such as those for monitoring, orchestration, and statistical post-processing, without the need for expensive or difficult-to-reproduce quantum executions. The benchmark is distributed with a lightweight Python API that facilitates fast data retrieval, flexible sampling strategies, and seamless integration into experimental pipelines. QSimBench is publicly available, aiming to foster collaborative and reproducible research within the quantum software community.
Quantum Software Engineering (QSE) is emerging as a critical discipline to make quantum computing accessible to a broader developer community; however, most quantum development environments still require developers to engage with low-level details across the software stack—including problem encoding, circuit construction, algorithm configuration, hardware selection, and result interpretation—making them difficult for classical software engineers to use. To bridge this gap, we present C2 ∣Q⟩ , a hardware-agnostic quantum software development framework that translates specific types of classical specifications into quantum-executable programs while preserving methodological rigor. The framework applies modular software engineering principles by classifying the workflow into three core modules: an encoder that classifies problems, produces Quantum-Compatible Formats (QCFs), and constructs quantum circuits, a deployment module that generates circuits and recommends hardware based on fidelity, runtime, and cost, and a decoder that interprets quantum outputs into classical solutions. This architecture supports systematic evaluation across simulators and Noisy Intermediate-Scale Quantum (NISQ) quantum devices, remaining scalable to new problem classes and algorithms. In evaluation, the encoder module achieved a 93.8% completion rate, the hardware recommendation module consistently selected the appropriate quantum devices for workloads scaling up to 56 qubits. End-to-end experiments on 434 Python programs and 100 JSON problem instances demonstrate that the full C2 ∣Q⟩ workflow executes reliably on simulators and can be deployed successfully on representative real quantum hardware, with empirical runs limited to small- and medium-sized instances consistent with current NISQ capabilities. A proxy-based usability analysis further indicates substantial reductions in handwritten lines of code and explicit configuration steps compared to conventional quantum SDK workflows. These results indicate that C2 ∣Q⟩ lowers the entry barrier to quantum software development by providing a reproducible, extensible toolchain that connects classical specifications to quantum execution. The open-source implementation of C2 ∣Q⟩ is available at https://github.com/C2-Q/C2Q and as a ready-to-use Python package at https://pypi.org/project/c2q-framework/ .
Statistical reproducibility in Noisy Intermediate-Scale Quantum Computing (NISQ) is hindered by hardware variability and by the lack of metadata management practices able to capture calibration, compilation decisions, and data provenance. Although these elements are essential for experimental traceability, the evidence on how they are modeled and operationalized remains scattered. This paper characterizes the state of the art in metadata management for quantum software, focusing on gaps that constrain interoperability, auditability, and statistical reproducibility. We conducted a systematic mapping study that included 40 primary studies. The results reveal a fragmented landscape with limited adoption of formal standards. Existing approaches are mostly providerspecific, conceptually incomplete, or weakly integrated with compilation and execution workflows. Overall, current approaches remain insufficient to support auditable and reproducible quantum experimentation in NISQ environments.
Quantum computing has the potential to solve complex problems that are outside the scope of classical systems, making it a standard shifter. Current devices, which are considered as Noisy Intermediate-Scale Quantum (NISQ) systems, are insufficient in their practical utility due to hardware limitations, error rates, and qubit counts. The dynamic growths wanted to exceed the NISQ era and grow scalable quantum algorithms and architectures that can achieve useful quantum benefit in real-world applications are reviewed in this research. This research gives a comprehensive examination of the complications in scaling quantum systems, adjacent resource efficacy, qubit connectivity, and error correction. In addition to connecting architectural innovations like modular quantum computing, error-resilient circuit design, and distributed quantum systems, the paper provides new hybrid quantum-classical algorithms planned to exploit performance in noisy environments. Results from benchmarking in arenas like machine learning, optimization, and quantum chemistry show how these methods can provide short-term quantum benefit. Lastly, this paper proposes a research roadmap for quantum computing that highlights the requirement of co-designing algorithms, software, and hardware. The goal of this effort is to straighten the progress of quantum systems that can resolve problems in the real world at scale by addressing the inadequacies of NISQ devices and emphasizing scalable solutions.
Quantum software testing (QST) checks whether quantum programs behave according to their intended specifications. A key requirement for QST research is a benchmark that supports rigorous empirical evaluation on programs that are testable and better reflect current software development practices. However, existing studies heavily rely on small hard-coded or circuit-level benchmarks, while available quantum programs are scattered across repositories without clear selection criteria, which limits fair comparison and systematic reproducibility. To this end, we present Qolumbina, a benchmark infrastructure for controlled QST experiments on scalable quantum programs. Qolumbina curates 40 programs from open-source repositories, turns them into test-ready subjects through systematic selection, refactoring, specifications, test case examples, unit tests, and standardized interfaces. We also propose QST-oriented criteria to characterize quantum programs along functionality, output behavior, development complexity, and quantum-specific execution complexity. Using these criteria, our empirical study shows that Qolumbina covers diverse testing-relevant properties and supports scalability analysis beyond fixed-size circuit benchmarks. Through controlled experiments with two recent QST approaches, we demonstrate the feasibility of using Qolumbina for execution-cost and fault-detection studies, and highlight backend-dependent effects that can influence QST result interpretation.
Quantum computers represent a radical technological advancement in the way information is processed by using the principles of quantum mechanics to solve very complex problems that exceed the capabilities of classical systems. However, in the current NISQ era (Noisy Intermediate-Scale Quantum devices), the available hardware presents several limitations, such as a limited number of qubits, high error rates, and reduced coherence times. Efficient management of quantum resources, both physical (qubits, error rates, connectivity) and logical (quantum gates, algorithms, error correction), becomes particularly relevant in the design and deployment of quantum algorithms. In this work, we analyze the role of resources in the various uses of NISQ devices today, identifying their relevance and implications for software engineering focused on the use of quantum computers. We propose a vision for runtime-aware quantum software development, identifying key challenges to its realization, such as limited introspection capabilities and temporal constraints in current platforms. As a proof of concept, we introduce Qonscious, a prototype framework that enables conditional execution of quantum programs based on dynamic resource evaluation. With this contribution, we aim to strengthen the field of Quantum Resource Estimation (QRE) and move towards the development of scalable, reliable, and resource-aware quantum software.