TLDR: This research paper proposes a framework for selecting the most suitable machine learning (ML) algorithms across healthcare, telecommunication, and marketing sectors. The framework involves four phases: input analysis, model building, model evaluation using performance metrics (accuracy, precision, recall, F-measure) and Akaike Information Criteria (AIC), and finally, model recommendation. The study found that eager learners generally perform well based on accuracy, with SVM excelling in healthcare and Decision Trees in telecommunication and marketing. However, when considering AIC, lazy learners like KNN were often more suitable for healthcare and telecommunication, indicating the importance of balancing performance with model complexity.
In today’s data-driven world, choosing the most effective machine learning (ML) algorithm for a specific task can be a complex challenge. A recent research paper introduces a comprehensive framework designed to simplify this selection process, particularly for applications in the healthcare, telecommunication, and marketing sectors. The study, authored by A. K. Hamisu and K. Jasleen, provides a structured approach to identify the best-performing ML algorithms based on a combination of performance metrics and the Akaike Information Criteria (AIC).
The proposed framework is divided into four distinct phases, guiding users from initial data understanding to final algorithm recommendation. It begins with the Input Analysis Phase, where the characteristics of the dataset, such as its size, type, and the relationships between attributes, are thoroughly examined. This initial analysis helps in pre-selecting a suitable set of ML algorithms for further evaluation.
Following this, the Model Building Phase involves collecting and preparing data, extracting relevant features, and then training various ML algorithms. For this research, a diverse set of 13 algorithms were categorized into eager learners (like Decision Trees, Support Vector Machines, and Neural Networks), lazy learners (such as K-Nearest Neighbors and Lazy Naïve Bayes), and hybrid learners (combinations of eager and lazy methods).
The Model Evaluation Phase is crucial for assessing how well each algorithm performs. Here, standard performance metrics like accuracy, precision, recall, and F-measure are calculated. Additionally, the Akaike Information Criteria (AIC) score is used as a model selection parameter, which helps in balancing model complexity with its goodness of fit. A lower AIC score generally indicates a better model.
Finally, the Model Recommendation Phase synthesizes all the gathered information. Based on a weighted average of performance parameters and AIC scores, the framework recommends the most suitable ML algorithm for the specific input attributes and problem domain.
For experimentation, the researchers utilized eight datasets across the three target sectors. In the marketing sector, datasets like Avocado prices and Bank Marketing were used. Telecommunication datasets included Telecom and Cell2cell train, while healthcare focused on Cardio-Vascular, Fetal Health, and Health Heart datasets. The findings revealed interesting insights into algorithm performance across these diverse fields.
When evaluated solely on performance metrics like accuracy, precision, recall, and F-measure, eager learner algorithms generally demonstrated superior performance across all three sectors. For instance, eager learners achieved an average accuracy ranging from 90% to 94%. Specifically, Support Vector Machine (SVM) emerged as the top performer in healthcare, while Decision Tree (DT) proved most effective for telecommunication and marketing datasets.
However, when the Akaike Information Criteria (AIC) score was considered, a different picture emerged. For the marketing sector, eager learners still reported the lowest AIC score, with SVM being the most suitable algorithm. But for telecommunication and healthcare, lazy learner algorithms, particularly K-Nearest Neighbors (KNN), showed the lowest AIC scores, suggesting they might offer a better balance of fit and simplicity for these domains. This highlights the importance of considering both performance and model selection criteria.
Also Read:
- ClustRecNet: Automating Clustering Algorithm Selection with Deep Learning
- Assessing LLM Reliability in Tabular Feature Engineering: A Multi-level Approach
This framework offers valuable guidance for practitioners and researchers in selecting appropriate ML algorithms, moving beyond a trial-and-error approach. By systematically analyzing data attributes and evaluating models using a dual set of criteria, it aims to enhance the efficiency and effectiveness of machine learning applications in critical sectors. You can read the full paper for more details: A Framework for Selection of Machine Learning Algorithms Based on Performance Metrices and Akaike Information Criteria in Healthcare, Telecommunication, and Marketing Sector.
