In recent years, quantum, quantum-inspired, and hybrid algorithms are increasingly showing promise for solving software engineering optimization problems. However, best-intended practices for conducting empirical studies have not yet well established. In this paper, based on the primary studies identified from the latest systematic literature review on quantum optimization for software engineering problems, we conducted a systematic analysis on these studies from various aspects including experimental designs, hyperparameter settings, case studies, baselines, tooling, and metrics. We identify key gaps in the current practices such as limited reporting of the number of repetitions, number of shots, and inadequate consideration of noise handling, as well as a lack of standardized evaluation protocols such as the adoption of quality metrics, especially quantum-specific metrics. Based on our analysis, we provide insights for designing empirical studies and highlight the need for more real-world and open case studies to assess cost-effectiveness and practical utility of the three types of approaches: quantum-inspired, quantum, and hybrid. This study is intended to offer an overview of current practices and serve as an initial reference for designing and conducting empirical studies on evaluating and comparing quantum, quantum-inspired, and hybrid algorithms in solving optimization problems in software engineering.
In recent years, quantum, quantum-inspired, and hybrid algorithms are increasingly showing promise for solving software engineering optimization problems. However, best-intended practices for conducting empirical studies have not yet well established. In this paper, based on the primary studies identified from the latest systematic literature review on quantum optimization for software engineering problems, we conducted a systematic analysis on these studies from various aspects including experimental designs, hyperparameter settings, case studies, baselines, tooling, and metrics. We identify key gaps in the current practices such as limited reporting of the number of repetitions, number of shots, and inadequate consideration of noise handling, as well as a lack of standardized evaluation protocols such as the adoption of quality metrics, especially quantum-specific metrics. Based on our analysis, we provide insights for designing empirical studies and highlight the need for more real-world and open case studies to assess cost-effectiveness and practical utility of the three types of approaches: quantum-inspired, quantum, and hybrid. This study is intended to offer an overview of current practices and serve as an initial reference for designing and conducting empirical studies on evaluating and comparing quantum, quantum-inspired, and hybrid algorithms in solving optimization problems in software engineering.
In quantum software engineering (QSE), quantum software testing (QST) has attracted increasing attention as quantum software systems grow in scale and complexity. Since QST evaluates quantum programs through execution under designed test inputs, empirical studies are widely used to assess the effectiveness of testing approaches. However, the design and reporting of empirical studies in QST remain highly diverse, and a shared methodological understanding has yet to emerge, making it difficult to interpret results and compare findings across studies. This paper presents a methodological analysis of empirical studies in QST through a systematic examination of 59 primary studies identified from a literature pool of size 384. We organize our analysis around ten research questions that cover key methodological dimensions of QST empirical studies, including objects under test, baseline comparison, testing setup, experimental configuration, and tool and artifact support. Through cross-study analysis along these dimensions, we characterize current empirical practices in QST, identify recurring limitations and inconsistencies, and highlight open methodological challenges. Based on our findings, we derive insights and recommendations to inform the design, execution, and reporting of future empirical studies in QST.
Quantum computing, particularly in the area of quantum optimization, is steadily progressing toward practical applications, supported by an expanding range of hardware platforms and simulators. While Software Engineering (SE) optimization has a strong foundation, which is exemplified by the active Search-Based Software Engineering (SBSE) community and numerous classical optimization methods, the growing complexity of modern software systems and their engineering processes demands innovative solutions. This Systematic Literature Review (SLR) focuses specifically on studying the literature that applies quantum or quantum-inspired algorithms to solve classical SE optimization problems. We examine 76 primary studies selected from an initial pool of 2083 publications obtained through systematic searches of six digital databases using carefully crafted search strings. Our findings reveal concentrated research efforts in areas such as SE operations and software testing, while exposing significant gaps across other SE activities. Additionally, the SLR uncovers relevant works published outside traditional SE venues, underscoring the necessity of this comprehensive review. Overall, our study provides a broad overview of the research landscape, empowering the SBSE community to leverage quantum advancements in addressing next-generation SE challenges.
The growing intersection of quantum computing and operations research (OR) has opened new avenues for solving high-dimensional optimization problems that remain computationally intractable under classical paradigms. This study presents a systematic mapping and critical evaluation of both quantum and quantum-inspired optimization algorithms, with a particular focus on their scalability, convergence efficiency, and applicability to large-scale operations research challenges. Through a structured review of contemporary literature and empirical implementations, the research categorizes existing algorithms into key families—such as quantum annealing, variational quantum algorithms (VQAs), and hybrid quantum–classical frameworks—while identifying trends in algorithmic performance across industrial logistics, network design, and resource allocation models. The systematic mapping process was conducted collaboratively, emphasizing transparent data collection, inclusion–exclusion criteria, and reproducible methodology following evidence-based review standards. Critical evaluation highlighted the need for unified performance metrics, improved hybridization strategies, and enhanced interpretability in quantum-inspired heuristics. The research also provided valuable insight into refining methodological rigor, including bibliometric analysis, coding frameworks, and academic presentation of findings. Ultimately, this work contributes to the ongoing dialogue between quantum computation and operations research, demonstrating how quantum-enhanced optimization can accelerate decision-making processes in complex, data-intensive environments.
Abstract Quantum optimization has become a leading application for near-term quantumcomputing, and yet many publications compare algorithms against idealizedassumptions and small toy benchmarks. This limits the interpretability, reproducibility,and practical relevance of reported performance gains, particularlyin the noisy intermediate-scale quantum (NISQ) era. In this work, we presenta systematic literature review that investigate quantum optimization beyondtoy benchmarks. Following established SLR protocols, we analyze the literaturealong multiple methodological dimensions, including algorithmic approach,benchmark realism, encoding strategies, hybrid quantum-classical workflows,hardware and noise modeling, evaluation metrics, and reporting practices. Weintroduce a unified taxonomy that captures the interaction between problemformulation, encoding overhead, noise-aware execution, and hybrid optimizationloops. In addition, we propose a reproducibility checklist and scoring rubric toassess reporting completeness and experimental rigor across studies. Instead ofdeveloping new quantum optimization algorithms or making theoretical quantumadvantage claims, the main contribution of this work is that it providesa methodological analysis on benchmarking realism and encoding or evaluationpractices as well as reproducibility rigor in NISQ-era quantum optimization studies.Our review uncovers a continued chasm between algorithmic innovation andevaluation maturity, with the number of publications and diversity of methodsgrowing without corresponding growth in standardized benchmarks, strongclassical baselines, noise-consistent evaluation, or reproducibility.
Abstract Quantum computing holds great promise for solving complex problems that classical computing cannot address, with applications in various industries and sectors. However, developing efficient quantum software remains a challenge. Quantum software engineering (QSE) has emerged to address this, adapting classical software engineering practices for quantum systems. Design patterns, widely used in classical software, can provide reusable solutions for common quantum development issues, but their use in quantum software remains underexplored. This paper presents an empirical study investigating the use of design patterns in 2610 Qiskit programs from GitHub. Using the QCPD Tool to detect four design patterns (initialization, superposition, entanglement, and oracle) and QMetrics to compute software metrics, the study creates a dataset linking patterns with code characteristics. Three research questions guide the study: RQ1 examines the prevalence of design patterns in quantum software, RQ2 explores the relationship between design patterns and code metrics, and RQ3 analyzes the combinations of patterns that occur together. The findings provide insights into quantum software development, offering developers practical guidance on applying specific patterns. The results contribute to QSE by revealing key relationships between patterns and metrics, which can inform future research and tool development. These findings support improved performance, maintainability, and scalability, fostering the broader adoption of quantum computing.
Context: Quantum software development is a complex and intricate process that diverges significantly from traditional software development. Quantum computing and quantum software are deeply entangled with quantum mechanics, which introduces a different level of abstraction and a deep dependence on quantum physical properties. The classical requirements engineering methods must be adapted to encompass the essential quantum features in this new paradigm. Aim: This study aims to systematically identify and analyze challenges, opportunities, developments, and new lines of research in requirements engineering for quantum computing. Method: We conducted a systematic literature review, including three research questions. This study included 105 papers published from 2017 to 2024. Results: The main results include the identification of problems associated with defining specific requirements for quantum software and hybrid system requirements. In addition, we identified challenges related to the absence of standards for quantum requirements engineering. Finally, we can see the advances in developing programming languages and simulation tools for developing software in hybrid systems. Conclusions: This study presents the challenges and opportunities in quantum computing requirements engineering, emphasizing the need for new methodologies and tools. It proposes a roadmap for future research to develop a standardized framework, contributing to theoretical foundations and practical applications.