A HYBRID BERT-BILSTM-ATTENTION MODEL FOR PUBLIC SENTIMENT ANALYSIS ON A NATIONAL SOCIAL INSURANCE PROVIDER IN INDONESIA
Abstract
Sentiment analysis in Indonesian social media presents significant challenges due to informal language, code-mixing, and sentiment ambiguity in multi-clause expressions. While transformer-based models such as IndoBERT effectively capture contextual semantics, they may struggle to model sentiment transitions and resolve conflicting polarity in noisy and heterogeneous text. To address this limitation, this study employs a hybrid BERT–BiLSTM–Attention model that integrates contextual, sequential, and attention-based representations. Although the architecture itself is not novel, the contribution lies in its systematic integration and empirical evaluation under realistic conditions. Experimental results show that the proposed model achieves an accuracy of 0.85 and a Macro-F1 score of 0.85, outperforming the IndoBERT baseline (0.83) by approximately 2.4%. This improvement is statistically significant (p = 0.027) and supported by effect size analysis, indicating meaningful performance gains. Robustness evaluation under controlled perturbations—including slang injection, character elongation, emoji usage, code-mixing, and class imbalance (up to 30% minority downsampling)—shows only minor performance degradation (0.02–0.03 Macro-F1), demonstrating stable generalization under noisy conditions. These findings provide empirical evidence that the integration of sequential modeling and attention improves the handling of sentiment transitions and multi-clause structures beyond transformer-only approaches, offering practical value for real-world sentiment analysis in Indonesian social media.
Downloads
References
P. W. Handayani, G. A. Zagatti, H. Kéfi, and S. Bressan, “IMPACT OF SOCIAL MEDIA USAGE ON USERS’ COVID-19 PROTECTIVE BEHAVIOR: SURVEY STUDY IN INDONESIA,” 2023. doi: https://doi.org/10.2196/46661.
C. A. de Castro, “THEMATIC ANALYSIS IN SOCIAL MEDIA INFLUENCERS: WHO ARE THEY FOLLOWING AND WHY?,” 2023. doi: https://doi.org/10.3389/fcomm.2023.1217684.
S. Bin Naeem and M. N. K. Boulos, “COVID-19 MISINFORMATION ONLINE AND HEALTH LITERACY: A BRIEF OVERVIEW,” 2021. doi: https://doi.org/10.3390/ijerph18158091.
Z. Zakaria, K. Kusrini, and D. Ariatmanto, “SENTIMENT ANALYSIS TO MEASURE PUBLIC TRUST IN THE GOVERNMENT DUE TO THE INCREASE IN FUEL PRICES USING NAIVE BAYES AND SUPPORT VECTOR MACHINE,” International Journal of Artificial Intelligence & Robotics (Ijair), 2023. doi: https://doi.org/10.25139/ijair.v5i2.7167.
N. Ma, G. Yu, X. Jin, and X. Zhu, “QUANTIFIED MULTIDIMENSIONAL PUBLIC SENTIMENT CHARACTERISTICS ON SOCIAL MEDIA FOR PUBLIC OPINION MANAGEMENT: EVIDENCE FROM THE COVID-19 PANDEMIC,” Front. Public Health, 2023. doi: https://doi.org/10.3389/fpubh.2023.1097796.
A. Kukkar, R. Mohana, A. Sharma, A. Nayyar, and M. A. Shah, “IMPROVING SENTIMENT ANALYSIS IN SOCIAL MEDIA BY HANDLING LENGTHENED WORDS,” Ieee Access, 2023. doi: https://doi.org/10.1109/ACCESS.2023.3238366
M. Slijepčević, N. P. Šević, G. Ašanin, and M. Ilić, “INFLUENCE OF CSR ACTIVITIES ON OPINIONS OF INSURANCE SERVICE USERS WHEN PURCHASING INSURANCE,” Tokovi Osiguranja, 2023. doi: https://doi.org/10.5937/TokOsig2302202S.
J. Park, W. Choi, and S.-U. Jung, “EXPLORING TRENDS IN ENVIRONMENTAL, SOCIAL, AND GOVERNANCE THEMES AND THEIR SENTIMENTAL VALUE OVER TIME,” 2022. doi: https://doi.org/10.3389/fpsyg.2022.890435.
F. Rozy, H. Harmain, and J. Nasution, “ANALYSIS OF THE EFFECTIVENESS OF MOTOR VEHICLE INSURANCE AT PT. JASA RAHARJA,” 2023. doi: https://doi.org/10.62398/probis.v14i6.384.
N. Phan Thi Hang and N. Kim Quoc Trung, “SERVICE QUALITY, CUSTOMER SATISFACTION AND LOYALTY: A CASE STUDY IN VIETNAMESE SMES,” Cogent Business and Management, vol. 11, no. 1, 2024. doi: https://doi.org/10.1080/23311975.2024.2377769.
S. Sinta and S. Suyanto, “KOMUNIKASI INOVASI JASA RAHARJA CABANG RIAU DALAM MENGIMPLEMENTASIKAN PELAYANAN SANTUNAN ONLINE KORBAN KECELAKAAN DI KOTA PEKANBARU,” 2024. doi: https://doi.org/10.62383/aktivisme.v1i4.469.
M. Janssen and G. Kuk, “THE CHALLENGES AND LIMITS OF BIG DATA ALGORITHMS IN TECHNOCRATIC GOVERNANCE,” Gov. Inf. Q., vol. 33, no. 3, pp. 371–377, 2016. doi: https://doi.org/10.1016/j.giq.2016.08.011.
M. Rifaí, R. Buaton, and I. G. Prahmana, “SENTIMENT ANALYSIS USING TEXT MINING TECHNIQUES ON SOCIAL MEDIA USING THE SUPPORT VECTOR MACHINE METHOD CASE STUDY SEAGAMES 2023 FOOTBALL FINAL,” J. Of Artif. Intell. And Eng. Appl., 2023. doi: https://doi.org/10.59934/jaiea.v3i1.274.
N. Z. Al Habesyah, R. Herteno, F. Indriani, I. Budiman, and D. Kartini, “SENTIMENT ANALYSIS OF TIKTOK SHOP CLOSURE IN INDONESIA ON TWITTER USING SUPERVISED MACHINE LEARNING,” Journal of Electronics Electromedical Engineering and Medical Informatics, 2024. doi: https://doi.org/10.35882/jeeemi.v6i2.381.
S. Rahman, N. Jahan, F. Sadia, and I. Mahmud, “SOCIAL CRISIS DETECTION USING TWITTER BASED TEXT MINING-A MACHINE LEARNING APPROACH,” Bulletin of Electrical Engineering and Informatics, 2023. doi: https://doi.org/10.11591/eei.v12i2.3957.
A. D. Herlia and M. Andriansyah, “SENTIMENT ANALYSIS OF FLO APPLICATIONS FOR WOMEN’S NEEDS USING THE CNN AND LSTM ALGORITHMS,” Eduvest - Journal of Universal Studies, 2024. doi: https://doi.org/10.59188/eduvest.v4i5.1182.
D. Olaniyan, R. O. Ogundokun, O. P. Bernard, J. Olaniyan, R. Maskeliūnas, and H. B. Akande, “UTILIZING AN ATTENTION-BASED LSTM MODEL FOR DETECTING SARCASM AND IRONY IN SOCIAL MEDIA,” Computers, 2023. doi: https://doi.org/10.3390/computers12110231.
J. Huang, G. Li, and G. Li, “RESEARCH ON SENTIMENT RECOGNITION IN ONLINE SHOPPING REVIEWS USING AN INTEGRATED BERT AND BILSTM APPROACH,” 2024. doi: https://doi.org/10.1117/12.3035076.
A. Romadhony, S. Al Faraby, R. Rismala, U. N. Wisesty, and A. Arifianto, “SENTIMENT ANALYSIS ON A LARGE INDONESIAN PRODUCT REVIEW DATASET,” 2024. doi: https://doi.org/10.20473/jisebi.10.1.167-178.
A. Kumar, N. K. Gupta, and A. Shrivastava, “A CRITICAL REVIEW ON SENTIMENT ANALYSIS BASED ON DEEP LEARNING TECHNIQUES,” 2024. doi: https://doi.org/10.36948/ijfmr.2024.v06i05.28572.
Z. Anwar, H. Afzal, T. Al‐Shehari, M. Al‐Razgan, T. Alfakih, and R. Nawaz, “MINING THE OPINIONS OF SOFTWARE DEVELOPERS FOR IMPROVED PROJECT INSIGHTS: HARNESSING THE POWER OF TRANSFER LEARNING,” 2024. doi: https://doi.org/10.1109/ACCESS.2024.3397211.
W. She, “A REVIEW OF DEEP LEARNING-BASED TEXT SENTIMENT ANALYSIS RESEARCH,” 2024. doi: https://doi.org/10.54254/2755-2721/32/20230204.
B. He, R. Zhao, L. Wang, and Y. Yang, “ASPECT-BASED SENTIMENT ANALYSIS BASED ON FEATURE EXTRACTION AND ATTENTION,” 2024. doi: https://doi.org/10.54097/da7qvd31.
A. G. Ganie and S. Dadvandipour, “EXPLORING THE IMPACT OF INFORMAL LANGUAGE ON SENTIMENT ANALYSIS MODELS FOR SOCIAL MEDIA TEXT USING CONVOLUTIONAL NEURAL NETWORKS,” Multidiszciplináris Tudományok, 2023. doi: https://doi.org/10.35925/j.multi.2023.1.17.
M. Aeri and A. Professor, “ANALYSIS OF PRODUCT REVIEWS BASED ON SENTIMENT ANALYSIS BY USING MACHINE LEARNING AND DEEP LEARNING,” 2023. doi: https://doi.org/10.48047/NQ.2022.20.3.NQ22956.
R. Das and T. D. Singh, “MULTIMODAL SENTIMENT ANALYSIS: A SURVEY OF METHODS, TRENDS, AND CHALLENGES,” 2023. doi: https://doi.org/10.1145/3586075.
S. M. Hashemi, R. M. Botez, and G. Ghazi, “BIDIRECTIONAL LONG SHORT-TERM MEMORY DEVELOPMENT FOR AIRCRAFT TRAJECTORY PREDICTION APPLICATIONS TO THE UAS-S4 EHÉCATL,” 2024. doi: https://doi.org/10.3390/aerospace11080625.
H. P. Pandit, “REGULAR PAPER COMPARATIVE ANALYSIS OF DEEP LEARNING MODELS FOR SENTIMENT ANALYSIS ON IMDB REVIEWS,” 2024. doi: https://doi.org/10.52783/jes.1345
R. A. Hameed, W. J. Abed, and A. T. Sadiq, “EVALUATION OF HOTEL PERFORMANCE WITH SENTIMENT ANALYSIS BY DEEP LEARNING TECHNIQUES,” 2023. doi: https://doi.org/10.3991/ijim.v17i09.38755.
S. Goswami and A. Kumar, “BILSTM_SAE:A HYBRID DEEP LEARNING FRAMEWORK FOR PREDICTIVE DATA ANALYTICS SYSTEM IN TRAFFIC MODELING,” 2023. doi: https://doi.org/10.21203/rs.3.rs-2422617/v1.
A. He and M. Abisado, “TEXT SENTIMENT ANALYSIS OF DOUBAN FILM SHORT COMMENTS BASED ON BERT-CNN-BILSTM-ATT MODEL,” Ieee Access, 2024. doi: https://doi.org/10.1109/ACCESS.2024.3381515.
N. Chen, Y. Sun, and Y. Yan, “SENTIMENT ANALYSIS AND RESEARCH BASED ON TWO‐CHANNEL PARALLEL HYBRID NEURAL NETWORK MODEL WITH ATTENTION MECHANISM,” Iet Control Theory and Applications, 2023. doi: https://doi.org/10.1049/cth2.12463.
A. V. Ramana, “NONLINEAR DEEP LEARNING FRAMEWORK FOR PRECISE SENTIMENT CLASSIFICATION IN TEXTUAL DATA,” CANA, 2024. doi: https://doi.org/10.52783/cana.v32.2345.
X. Li, L. Chen, B. Chen, and X. Ge, “BERT-BILSTM-ATTENTION MODEL FOR SENTIMENT ANALYSIS ON CHINESE STOCK REVIEWS,” 2024. doi: https://doi.org/10.21203/rs.3.rs-4023113/v1.
M. R. Alifi, D. C. Utomo Lieharyani, B. P. Sudimulya, and M. R. Maulidhan, “IMPLEMENTATION OF INDONLU PRE-TRAINED MODEL FOR ASPECT-BASED SENTIMENT ANALYSIS OF INDONESIAN STOCK NEWS,” Jurnal Teknik Informatika, 2023. doi: https://doi.org/10.15408/jti.v16i2.33791.
H. S. Wicaksana, R. Kusumaningrum, and R. Gernowo, “DETERMINING COMMUNITY HAPPINESS INDEX WITH TRANSFORMERS AND ATTENTION-BASED DEEP LEARNING,” Iaes International Journal of Artificial Intelligence (Ij-Ai), 2024. doi: https://doi.org/10.11591/ijai.v13.i2.pp1753-1761.
X. Zhao, “EXPLORING THE PERFORMANCE OF THE CNN-LSTM MODEL IN STOCK PREDICTION,” Highlights in Business Economics and Management, 2024. doi: https://doi.org/10.54097/x9m1cm10.
H. WANG and S. Zhang, “RESEARCH ON THE APPLICATION OF IMPROVED BERT‐DPCNN MODEL IN CHINESE NEWS TEXT CLASSIFICATION,” Concurr. Comput., 2024. doi: https://doi.org/10.1002/cpe.8338.
N. Sureja, N. Chaudhari, P. Patel, J. Bhatt, T. Desai, and V. Parikh, “HYPER-TUNED SWARM INTELLIGENCE MACHINE LEARNING-BASED SENTIMENT ANALYSIS OF SOCIAL MEDIA,” Engineering Technology & Applied Science Research, 2024. doi: https://doi.org/10.48084/etasr.7818.
F. Liu, C. Yuan, H. Chen, and F. Yang, “PREDICTION OF LINEAR B-CELL EPITOPES BASED ON PROTEIN SEQUENCE FEATURES AND BERT EMBEDDINGS,” Sci. Rep., 2024. doi: https://doi.org/10.1038/s41598-024-53028-w.
B. Zeng, “PARTICLE SWARM OPTIMIZATION-BASED NLP METHODS FOR OPTIMIZING AUTOMATIC DOCUMENT CLASSIFICATION AND RETRIEVAL,” PLoS One, 2025. doi: https://doi.org/10.1371/journal.pone.0325851.
S. Riyanto, I. S. Sitanggang, T. Djatna, and T. D. Atikah, “PLANT-DISEASE RELATION MODEL THROUGH BERT-BILSTM-CRF APPROACH,” Indonesian Journal of Electrical Engineering and Informatics (Ijeei), 2024. doi: https://doi.org/10.52549/ijeei.v12i1.5154.
F. Wang and Z. Liu, “POWER LOAD PREDICTION BASED ON IGWO‐BILSTM NETWORK,” Math. Probl. Eng., 2023. doi: https://doi.org/10.1155/2023/8996138.
C. Wang and J. Qiao, “CONSTRUCTION PROJECT COST PREDICTION METHOD BASED ON IMPROVED BILSTM,” Applied Sciences, 2024. doi: https://doi.org/10.3390/app14030978.
W. Sun, Y. Wang, X. You, D. Zhang, J. Zhang, and X. Zhao, “OPTIMIZATION OF VARIATIONAL MODE DECOMPOSITION-CONVOLUTIONAL NEURAL NETWORK-BIDIRECTIONAL LONG SHORT TERM MEMORY ROLLING BEARING FAULT DIAGNOSIS MODEL BASED ON IMPROVED DUNG BEETLE OPTIMIZER ALGORITHM,” Lubricants, 2024. doi: https://doi.org/10.3390/lubricants12070239.
Y. Yue, Y. Peng, and D. Wang, “DEEP LEARNING SHORT TEXT SENTIMENT ANALYSIS BASED ON IMPROVED PARTICLE SWARM OPTIMIZATION,” Electronics (Basel)., 2023. doi: https://doi.org/10.3390/electronics12194119.
L. Wang, H. Wang, T. Lu, and C. Wang, “REACTIVE POWER OUTPUT MODELING OF SYNCHRONOUS CONDENSER IN UHVDC CONVERTER STATION BASED ON INTERLACED SUPERPOSITION CNN-BILSTM,” International Journal of Computational Intelligence Systems, 2023. doi: https://doi.org/10.1007/s44196-023-00363-x.
W. Ma, Y. Lei, B. Yang, and P. Guo, “A MULTI-MECHANISM FUSION METHOD FOR ROBUST STATE OF CHARGE ESTIMATION VIA BIDIRECTIONAL LONG SHORT-TERM MEMORY MODEL WITH MIXTURE KERNEL MEAN P-POWER ERROR LOSS OPTIMIZED BY GOLDEN JACKAL OPTIMIZATION ALGORITHM,” J. Electrochem. Soc., 2024. doi: https://doi.org/10.1149/1945-7111/ad7c7f.
C. Shi, A. K. Chiu, and H. Xu, “EVALUATING DESIGNS FOR HYPERPARAMETER TUNING IN DEEP NEURAL NETWORKS,” The New England Journal of Statistics in Data Science, 2023. doi: https://doi.org/10.51387/23-NEJSDS26.
D. Qian, “ANALYSIS OF THE HYPERPARAMETER SELECTION IN MACHINE LEARNING,” Applied and Computational Engineering, 2024. doi: https://doi.org/10.54254/2755-2721/104/20241140.
R. Malhotra and M. Cherukuri, “A SYSTEMATIC REVIEW OF HYPERPARAMETER TUNING TECHNIQUES FOR SOFTWARE QUALITY PREDICTION MODELS,” Intelligent Data Analysis, 2024. doi: https://doi.org/10.3233/IDA-230653.
C. Xu, P. Coen-Pirani, and X. Jiang, “EMPIRICAL STUDY OF OVERFITTING IN DEEP LEARNING FOR PREDICTING BREAST CANCER METASTASIS,” Cancers (Basel)., 2023. doi: https://doi.org/10.3390/cancers15071969.
J. A. Ilemobayo et al., “HYPERPARAMETER TUNING IN MACHINE LEARNING: A COMPREHENSIVE REVIEW,” Journal of Engineering Research and Reports, 2024. doi: https://doi.org/10.9734/jerr/2024/v26i61188.
X. Chang, S. Yang, S. Li, and X. Gu, “ROLLING ELEMENT BEARING FAULT DIAGNOSIS BASED ON MULTI-OBJECTIVE OPTIMIZED DEEP AUTO-ENCODER,” Meas. Sci. Technol., 2024. doi: https://doi.org/10.1088/1361-6501/ad5460.
A. Simanjuntak et al., “RESEARCH AND ANALYSIS OF INDOBERT HYPERPARAMETER TUNING IN FAKE NEWS DETECTION,” Jurnal Nasional Teknik Elektro Dan Teknologi Informasi (Jnteti), 2024. doi: https://doi.org/10.22146/jnteti.v13i1.8532.
Copyright (c) 2026 Syaiful Anam, Nur Atiqah Sia Abdullah, Hilmi Aziz Bukhori, Avin Maulana, Regina Vincentia
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this Journal agree to the following terms:
- Author retain copyright and grant the journal right of first publication with the work simultaneously licensed under a creative commons attribution license that allow others to share the work within an acknowledgement of the work’s authorship and initial publication of this journal.
- Authors are able to enter into separate, additional contractual arrangement for the non-exclusive distribution of the journal’s published version of the work (e.g. acknowledgement of its initial publication in this journal).
- Authors are permitted and encouraged to post their work online (e.g. in institutional repositories or on their websites) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published works.
1.gif)
