APPLICATION OF THE SUPPORT VECTOR MACHINE, LIGHT GRADIENT BOOSTING MACHINE, ADAPTIVE BOOSTING, AND HYBRID ADABOOST-SVM MODEL ON CUSTOMERS CHURN DATA
Abstract
A service provider is a business that provides services or the expertise of an individual in a certain sector. A service provider’s customer flow could be very dynamic, with both new and churning customers. For the purpose of minimizing the number of churning customers, the company should perform a customer churn analysis. Customer churn analysis is the process of identifying a pattern or trend in churning customers. In order to classify and predict churning customers, machine learning techniques are required to build the classifier model. This paper will use the Support Vector Machine (SVM), Light Gradient Boosting Machine (LightGBM), and hybrid Adaptive Boosting-SVM (AdaBoost-SVM) model. The hybrid AdaBoost-SVM model is a boosting model which uses SVM as its basis classifier instead of a decision tree. The models will be implemented using airlines and telecommunication customers churn data. The usage of oversampling technique is required to balance the number of observations in both classes of training data. Furthermore, a model comparison will be conducted using the F1-Score and the AUC score as the evaluation metric. The analysis shows that LightGBM performs the best result in both dataset with the highest F1-Score and the shortest computational time. In addition, the boosting model AdaBoost-SVM has a better performance than the SVM model due to the boosting algorithm which always minimizes the model error in each iteration. Despite having a better result, AdaBoost-SVM performs in the longest computational time, making it computationally expensive for large datasets. Additionally, the imbalanced nature of the datasets presents challenges in model performance, requiring the application of oversampling techniques to mitigate bias towards the majority class. In conclusion, LightGBM is the best model to classify churning customers based on the higher F1-Score, AUC score, and the shortest computational time.
Downloads
References
B. Huang, M. T. Kechadi and B. Buckley, "CUSTOMER CHURN PREDICTION IN TELECOMMUNICATIONS," Expert Systems with Applications, vol. 39, no. 1, pp. 1414-1425, 2012. doi: https://doi.org/10.1016/j.eswa.2011.08.024
S. Nurhaliza, K. Sadik and A. Saefuddin, "A COMPARISON OF COX PROPORTIONAL HAZARD AND RANDOM SURVIVAL FOREST MODELS IN PREDICTING CHURN OF THE TELECOMMUNICATION INDUSTRY CUSTOMER," BAREKENG: Jurnal Ilmu Matematika dan Terapan, vol. 16, no. 4, pp. 1433-1440, 2022. doi: https://doi.org/10.30598/barekengvol16iss4pp1433-1440
A. Shi, C. Y. Lim and S. L. Ang, "CUSTOMER CHURN ANALYSIS FOR LIVE STREAM E-COMMERCE PLATFORMS BY USING DECISION TREE METHOD," in International Conference on Advanced Communication and Intelligent Systems, Warsaw, 2023. doi: https://doi.org/10.1007/978-3-031-45124-9_13
M. Wazid, A. K. Das, V. Chamola and Y. Park, "UNITING CYBER SECURITY AND MACHINE LEARNING: ADVANTAGES, CHALLENGES AND FUTURE RESEARCH," ICT express, vol. 8, no. 3, pp. 313-321, 2022. doi: https://doi.org/10.1016/j.icte.2022.04.007
D. S. Sisodia, S. Vishwakarma and A. Pujahari, "EVALUATION OF MACHINE LEARNING MODELS FOR EMPLOYEE CHURN PREDICTION," in 2017 international conference on inventive computing and informatics (icici), Coimbatore, 2017.doi: https://doi.org/10.1109/ICICI.2017.8365293
S. Dutta, P. Bose, S. Bandyopadhyay and M. Janarthanan, "A HYBRID MACHINE LEARNING MODEL FOR BANK CUSTOMER CHURN PREDICTION," International Journal of Engineering Trends and Technology, vol. 70, pp. 13-23, 2022.doi: https://doi.org/10.14445/22315381/IJETT-V70I6P202
N. T. M. Sagala and S. D. Permai, "ENHANCED CHURN PREDICTION MODEL WITH BOOSTED TREES ALGORITHMS IN THE BANKING SECTOR," in 2021 International Conference on Data Science and Its Applications (ICoDSA), Bandung, 2021. doi: https://doi.org/10.1109/ICoDSA53588.2021.9617503
Á. Periáñez, A. Saas, A. Guitart and C. Magne, "CHURN PREDICTION IN MOBILE SOCIAL GAMES: TOWARDS A COMPLETE ASSESSMENT USING SURVIVAL ENSEMBLES," in 2016 IEEE international conference on data science and advanced analytics (DSAA), Montreal, 2016. doi: https://doi.org/10.1109/DSAA.2016.84
A. Apicella, F. Isgrò and R. Prevete, "Don't Push the Button! Exploring Data Leakage Risks in Machine Learning and Transfer Learning," arXiv preprint arXiv:2401.13796, 2024. doi: https://doi.org/10.2139/ssrn.4733889
R. Mohammed, J. Rawashdeh and M. Abdullah, "MACHINE LEARNING WITH OVERSAMPLING AND UNDERSAMPLING TECHNIQUES: OVERVIEW STUDY AND EXPERIMENTAL RESULTS," in 2020 11th international conference on information and communication systems (ICICS), Irbid, 2020. doi: https://doi.org/10.1109/ICICS49469.2020.239556
M. Grandini, E. Bagli and G. Visani, "METRICS FOR MULTI-CLASS CLASSIFICATION: AN OVERVIEW," arXiv preprint arXiv:2008.05756, 2020.
A. Kowalczyk, "SUPPORT VECTOR MACHINES SUCCINTCTLY, SYNCFUSION," Succinctly E-Book Series, vol. 114, 2017.
S. Anam, A. N. Guci, F. Widhiatmoko, M. Kurniawaty and K. A. A. Wijaya, "DEVELOPMENT OF HEALTH INSURANCE CLAIM PREDICTION METHOD BASED ON SUPPORT VECTOR MACHINE AND BAT ALGORITHM," BAREKENG: Jurnal Ilmu Matematika dan Terapan, vol. 17, no. 4, pp. 2281-2292, 2023. doi: https://doi.org/10.30598/barekengvol17iss4pp2281-2292
Y. Ren, F. Hu and H. Miao, "THE OPTIMIZATION OF KERNEL FUNCTION AND ITS PARAMETERS FOR SVM IN WELL-LOGGING," in 2016 13th International Conference on Service Systems and Service Management (ICSSSM), Kunming, 2016.
I. S. Al-Mejibli, J. K. Alwan and D. H. Abd, "THE EFFECT OF GAMMA VALUE ON SUPPORT VECTOR MACHINE PERFORMANCE WITH DIFFERENT KERNELS," International Journal of Electrical and Computer Engineering, vol. 10, no. 5, pp. 5497-5506, 2020. doi: https://doi.org/10.11591/ijece.v10i5.pp5497-5506
J. Wang, P. Li, R. Ran, Y. Che and Y. Zhou, "A short-term photovoltaic power prediction model based on the gradient boost decision tree," A short-term photovoltaic power prediction model based on the gradient boost decision tree, vol. 8, no. 5, 2018.
B. Schölkopf, Z. Luo and V. Vovk, EMPIRICAL INFERENCE: FESTSCHRIFT IN HONOR OF VLADIMIR N. VAPNIK, Berlin: Springer Science & Business Media, 2013.
J. H. Friedman, "GREEDY FUNCTION APPROXIMATION: A GRADIENT BOOSTING MACHINE," Annals of statistics, vol. 29, no. 5, pp. 1189-1232, 2001. doi: https://doi.org/10.1214/aos/1013203451
A. Kadiyala and A. Kumar, "APPLICATIONS OF PYTHON TO EVALUATE THE PERFORMANCE OF DECISION TREE-BASED BOOSTING ALGORITHMS," Environmental Progress & Sustainable Energy, vol. 37, no. 2, pp. 618-623, 2018. doi: https://doi.org/10.1002/ep.12888
E. Kristiani, Y.-T. Tsan, P.-Y. Liu, N. Y. Yen and C.-T. Yang, "BINARY AND MULTI-CLASS ASSESSMENT OF FACE MASK CLASSIFICATION ON EDGE AI USING CNN AND TRANSFER LEARNING," Human-centric Computing and Information Sciences, vol. 12, no. 53, 2022.
G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye and T.-Y. Liu, "LIGHTGBM: A HIGHLY EFFICIENT GRADIENT BOOSTING DECISION TREE," Advances in neural information processing systems, vol. 30, 2017.
M. A. Mohammed, S. M. Kadhem and A. A. Maisa'a, "INSIDER ATTACKER DETECTION USING LIGHT GRADIENT BOOSTING MACHINE," Tech-Knowledge, vol. 1, no. 1, pp. 67-76, 2021.
C. Zhang and Y. Ma, ENSEMBLE MACHINE LEARNINg, New York: Springer, 2012.
Copyright (c) 2025 Felice Elena, Robyn Irawan, Benny Yong
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this Journal agree to the following terms:
- Author retain copyright and grant the journal right of first publication with the work simultaneously licensed under a creative commons attribution license that allow others to share the work within an acknowledgement of the work’s authorship and initial publication of this journal.
- Authors are able to enter into separate, additional contractual arrangement for the non-exclusive distribution of the journal’s published version of the work (e.g. acknowledgement of its initial publication in this journal).
- Authors are permitted and encouraged to post their work online (e.g. in institutional repositories or on their websites) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published works.
1.gif)
