Feature selection is a pivotal step in machine learning, aimed at reducing feature dimensionality and improving model performance. Conventional feature selection methods, typically framed as NP-hard problems, focus primarily on selecting non-redundant and discriminative features but neglect how these selected features influence downstream machine learning tasks. To bridge this gap, we propose a mixed-integer non-linear programming-based integrated feature selection and logistic regression (IFSLR) model that jointly optimizes feature selection and classifier parameters. To solve the proposed IFSLR model efficiently, we reformulate it into two quantum-computing-compatible quadratic unconstrained binary optimization (QUBO) models via discretization of continuous variables and Taylor expansions, and provide a theoretical approximation guarantee for the proposed QUBO models. To adaptively optimize both the discretization step length optimization sub-problems and the parameterized QUBO sub-problems, we develop a dual-level Q-learning-guided hybrid quantum-classical algorithm that dynamically selects which sub-problem to solve. At the upper level, an adaptive Q learning agent is employed to select appropriate sub-problems during the search process. At the lower level, each iteration addresses either discretization step length optimization sub-problems or parameterized QUBO sub-problems. Experiments on the German credit dataset demonstrate that our method achieves a 10.83% improvement in accuracy and a 4.54-fold enhancement in computational efficiency on average compared with state-of-the-art methods. It also achievesF1 score improvements of 2.86% on the Wisconsin diagnostic breast cancer dataset and 7.65% on the Portuguese bank marketing dataset. The improvements further validate the effectiveness and generalizability of the proposed model and algorithm.
Feature selection is a pivotal step in machine learning, aimed at reducing feature dimensionality and improving model performance. Conventional feature selection methods, typically framed as NP-hard problems, focus primarily on selecting non-redundant and discriminative features but neglect how these selected features influence downstream machine learning tasks. To bridge this gap, we propose a mixed-integer non-linear programming-based integrated feature selection and logistic regression (IFSLR) model that jointly optimizes feature selection and classifier parameters. To solve the proposed IFSLR model efficiently, we reformulate it into two quantum-computing-compatible quadratic unconstrained binary optimization (QUBO) models via discretization of continuous variables and Taylor expansions, and provide a theoretical approximation guarantee for the proposed QUBO models. To adaptively optimize both the discretization step length optimization sub-problems and the parameterized QUBO sub-problems, we develop a dual-level Q-learning-guided hybrid quantum-classical algorithm that dynamically selects which sub-problem to solve. At the upper level, an adaptive Q learning agent is employed to select appropriate sub-problems during the search process. At the lower level, each iteration addresses either discretization step length optimization sub-problems or parameterized QUBO sub-problems. Experiments on the German credit dataset demonstrate that our method achieves a 10.83% improvement in accuracy and a 4.54-fold enhancement in computational efficiency on average compared with state-of-the-art methods. It also achievesF1 score improvements of 2.86% on the Wisconsin diagnostic breast cancer dataset and 7.65% on the Portuguese bank marketing dataset. The improvements further validate the effectiveness and generalizability of the proposed model and algorithm.
One of the biggest obstacles to creating precise machine learning models is choosing representative and pertinent characteristics from big, unbalanced datasets. While too many features raise the risk of overfitting and computational expense, class imbalance frequently results in decreased accuracy and bias. The Simulated Annealing technique is used in this study to tackle a Quadratic Unconstrained Binary Optimization (QUBO) problem that is formulated as a quantum-assisted feature selection method to handle these problems. The technique seeks to reduce inter-feature redundancy and the number of selected features. There are 102,487 samples in the majority class and 11,239 in the minority class, totaling 28 characteristics in the experimental dataset. Nine ideal features were found during the feature selection method (12, 14, 15, 22, 23, 24, 25, 27, and 28). Ten-fold cross-validation was used to assess a Random Forest Classifier that was trained using an 80:20 split. With precision, recall, f1-score, and accuracy all hitting 1.00, the suggested QUBO+SMOTE method demonstrated exceptional performance. Comparatively, QUBO without SMOTE performed worse with accuracy 0.95 and minority-class f1-score of only 0.71, whereas a traditional Recursive Feature Elimination (RFE) approach obtained accuracy 0.97 with minority-class f1-score of 0.94. These findings indicate that QUBO can reduce dimensionality and address class imbalance which requires its integration with SMOTE. This study demonstrates how quantum computing can enhance the effectiveness and efficiency of machine learning, especially for large-scale imbalanced datasets
In this study, the performance of classical and quantum machine learning models was compared using the Breast Cancer dataset, which consists of diagnostic data aimed at classifying breast tumor types. Breast cancer, being one of the most common and life-threatening cancers in women, requires accurate diagnostic tools for early detection and effective treatment. The primary objective of this study is to evaluate the accuracy of quantum-assisted models through quantum feature selection methods. Initially, classical machine learning algorithms such as Support Vector Machines (SVM), Decision Trees, Random Forests, and Logistic Regression were applied to the dataset for baseline analysis. Subsequently, a quantum feature map was constructed using the Cirq library, enabling feature transformation based on this map. The classification was performed using the SVM model with the quantum-transformed features. The Logistic Regression and SVM models demonstrated the highest performance among classical machine learning models, achieving an accuracy rate of 96.49%, followed by Random Forest at 94.74% and Decision Tree at 92.11%. In the context of quantum feature transformation, the model utilizing the top five selected features achieved an accuracy rate of 94.74%, in contrast to 98.25% for the model trained with all features. These findings underscore the potential of quantum feature maps in enhancing model performance compared to classical techniques. The results suggest that quantum computing may offer significant advantages when integrated into machine learning frameworks, particularly in domains such as medical diagnostics, where high accuracy is crucial.
This paper presents a hybrid quantum logistic regression model for binary data classification that combines classical logistic regression methods with quantum computing to improve efficiency and speed up the learning process. Quantum computing is used to represent and process feature and weight vectors as quantum states and to efficiently compute the dot product underlying the logistic function. The use of normalized amplitude coding allows for compact representation of feature vectors and weights, reducing the number of qubits required and potentially speeding up computations for high-dimensional classification problems where classical methods may encounter scalability limitations. The quantum dot product computation is performed using the Hadamard test, which provides a statistical estimate of the probability of measuring an auxiliary qubit. The auxiliary qubit is first put into a superposition state, and after performing controlled unitary operations, amplitude interference is formed, which depends on the scalar product of the feature vector and weight coefficients. The obtained measurement probability is used to estimate the scalar product of the feature vectors and weights. The classical part of the model provides for the execution of operations that are difficult to implement on modern quantum devices, in particular, vector normalization, application of the sigmoid function to estimate the probability of an object belonging to a certain class, calculation of the loss function and iterative update of weight coefficients by the gradient descent method. The purpose of classification is to predict the binary class for an unknown sample based on the training sample. To demonstrate the model, a numerical example is given with a step-by-step calculation of one iteration of logistic regression training. In addition, computer modeling of the hybrid algorithm training process was performed, taking into account the influence of noise on the results of quantum state measurements. A quantum noise model was developed based on the probability of confidence in the measurement result, which is interpreted as an assessment of its reliability in a Bernoulli experiment. Comparison of the obtained results with the classical approach confirms the practical suitability of the hybrid algorithm for binary classification problems. The hybrid model preserves the key properties of classical logistic regression – parameter interpretability and optimization stability, while allowing for the use of quantum advantages for compact data encoding and scalar product computation. This makes the study of hybrid logistic regression relevant for the development of machine learning algorithms on modern quantum platforms.
As the volume of data for classification tasks in machine learning grows, feature selection plays an increasingly crucial role in enhancing the efficiency and effectiveness of these tasks. Existing classical feature selection algorithms often encounter challenges, such as high computational complexity and vulnerability to local optima. Quantum algorithms have been proposed to overcome these limitations. Here, our work proposes a new approach that combines the Quantum Approximate Optimization Algorithm (QAOA) with the classical feature selection algorithm Maximum Relevance-Minimum Redundancy (MRMR). First, we transform the feature selection problem into the Quadratic Unconstrained Binary Optimization (QUBO) problem. Then, we map the QUBO formulation to Hamiltonian with the Ising model. Finally, we employ QAOA with a shallow circuit to search this Hamiltonian's ground state, which matches the specific solution for feature selection. We conducted numerical experiments on eight real-world datasets for classification tasks from diverse domains, including finance, medicine, and natural hazards. Experiment results suggest that QAOA-selected feature subsets size are typically less than half the complete set while achieving comparable or better classification accuracy than classical MRMR, demonstrating its preference for smaller, more efficient feature subsets.
Feature selection plays an important role in improving the efficiency of machine learning models for credit card fraud and default detection. We formulate the feature selection problem as a Quadratic Unconstrained Binary Optimization (QUBO) problem, which we solve using quantum annealers. We propose three new formulations based on this framework that improve the efficiency and flexibility of machine learning models. We benchmark the proposed methods, existing approaches from the literature, and also compare with classical feature-selection methods such as Random Forest feature importance and a combination of mutual information and Spearman correlation. Extensive experiments show that feature selection using quantum computers consistently performs better than the classical methods. Our experiments show the promise of using quantum computers in machine learning tasks in financial risk assessment applications.
Feature selection is crucial for enhancing the accuracy and efficiency of machine learning models. This work investigates the possibility of the usefulness of a quantum annealer to tackle the challenge of selecting features based on maximizing mutual information (MI) and conditional mutual information (CMI). Calculating the optimal feature set for maximum MI and CMI remains computationally intractable for large datasets on classical computers, even with approximation methods. This study employs a Mutual Information Quadratic Unconstrained Binary Optimization (MIQUBO) formulation, enabling its solution on a quantum annealer. Importantly, the study demonstrates the relevancy of this approach in identifying the best feature combinations that maximizes the MI and CMI. To showcase its real-world applicability, we apply MIQUBO to forecasting the price of used excavators. Our results demonstrate that using the MIQUBO approach there is an improvement in the prediction of machine learning models for datasets, with a smaller MI concentration on a subset of all features. • Problem Addressed: We tackle the computationally difficult problem of feature selection for machine learning models, specifically by maximizing mutual information (MI) and conditional mutual information (CMI) between features and the target variable. • Proposed Solution: We employ the adiabatic quantum computation paradigm to tackle this problem. The study utilizes a hybrid annealing approach, specifically the D-Wave Kerberos framework, which combines classical and quantum algorithms to solve the complex optimization problem. The feature selection task is reformulated as a Mutual Information Quadratic Unconstrained Binary Optimization (MIQUBO) problem, which can be solved on a quantum annealer. • Real-World Application: The MIQUBO method is applied to a practical, real-world scenario: forecasting the prices of used excavators. • Key Findings: The research demonstrates that using the MIQUBO approach leads to an improvement in the prediction performance of machine learning models. This is particularly true for datasets where the mutual information is less concentrated on a small subset of features.