Pemodelan Emosi Pengguna Berbasis Ulasan Digital Melalui Integrasi Natural Language Processing dan Ilmu Sosial Komputasional

Adinda Riska Safitri(1), Andi Arniaty Arsyad(2),


(1) Universitas Al-Azhar Indonesia
(2) Universitas Al-Azhar Indonesia
(*) Corresponding Author

Abstract


Internal company apps such as PT Astra Honda Motor’s AHM Mobile receive a wide range of reviews on the Google Play Store that contain users’ emotional expressions. However, the emotional dimension of these reviews has rarely been systematically analyzed, as most previous studies have focused only on positive, negative, and neutral sentiments. This study aims to analyze the emotions of AHM Mobile users through the integration of Natural Language Processing (NLP) and Computational Social Science perspectives. The dataset consists of 2.117 reviews obtained via web scraping and classified into six emotional categories: angry, sad, afraid, neutral, surprised, and happy. The annotation process was conducted by two annotators in the fields of clinical psychology and linguistics using a Seniority-Based Tie-Breaking mechanism with a Cohen’s Kappa value of 0.636. Emotion classification was performed using a combination of TF-IDF and Logistic Regression as classical models, as well as IndoBERT as the main model. Evaluation results show that the classical model achieved an accuracy of 0.43 and a macro-F1 score of 0.183, while IndoBERT reached an accuracy of 0.7831 and a macro-F1 score of 0.5582. Collective emotion analysis indicates that anger dominates user reviews and is strongly correlated with user ratings, as evidenced by a Spearman correlation coefficient of 0.7141. These results indicate that sentiment analysis using the IndoBERT model can provide more effective insights for evaluating the quality of usage of internal corporate applications in Indonesia.

Keywords - Computational Social Science (CSS), Emotion Classification, IndoBERT, Natural Language Processing (NLP), User Reviews of the AHM Mobile Application.


Full Text:

PDF

References


A. A. Vărzaru and C. G. Bocean, “Digital Transformation and Innovation: The Influence of Digital Technologies on Turnover from Innovation Activities and Types of Innovation,” Systems, vol. 12, no. 9, 2024, doi: 10.3390/systems12090359.

Welovehonda, “AHM Mobile,” Google Play Store. [Online]. Available: https://play.google.com/store/apps/details?id=com.ahm.hr

S. K. Lubis, M. H. Dar, and F. A. Nasution, “Analisis Sentimen Ulasan Pengguna Aplikasi pada Google Play Store Menggunakan Algoritma Support Vector Machine,” J. Surya Inform., vol. 11, no. 2, pp. 120–128, 2024, doi: 10.36987/informatika.v11i2.5860.

N. P. Husain, S. Sukirman, and S. SAJIAH, “Analisis Sentimen Ulasan Pengguna Tiktok pada Google Play Store Berbasis TF-IDF dan Support Vector Machine,” J. Syst. Comput. Eng., vol. 5, no. 1, pp. 91–102, 2024, doi: 10.61628/jsce.v5i1.1105.

N. F. Hilmi and F. Irwiensyah, “Analisis Sentimen Terhadap Aplikasi Tiktok Dari Ulasan Pada Google Playstore Menggunakan Metode Naïve Bayes,” SMATIKA STIKI Inform. J., vol. 14, no. 01, pp. 146–156, 2024, doi: 10.32664/smatika.v14i01.1210.

D. Lazer et al., “Computational Social Science,” Science (80-. )., vol. 323, pp. 721–723, 2009, [Online]. Available: https://www.science.org/doi/10.1126/science.1167742

P. C. Ncr et al., “Step-by-step data mining guide,” 2000. [Online]. Available: https://www.kde.cs.uni-kassel.de/wp-content/uploads/lehre/ws2012-13/kdd/files/CRISPWP-0800.pdf

M. Z. E. Putra, “Representasi Emosi Dalam Cuitan Twitter Komunitas Marah-Marah: Kajian Psikolinguistik,” J. Pendidik. Bhs. dan Sastra Indones., vol. 14, pp. 349–358, 2024, doi: https://doi.org/10.23887/jpbsi.v14i3.82042.

N. Zahra, Y. Sonia, S. Adilla, R. A. Mardiyah, and D. Amelia, “Semantik Dalam Bahasa Indonesia,” Morfol. J. Ilmu Pendidikan, Bahasa, Sastra dan Budaya, vol. 2, no. 6, pp. 156–164, 2024, doi: 10.61132/morfologi.v2i6.1163.

N. F. Ramadhani, R. A. Hamzah, and R. Arief, “Struktur Kebahasaan Bahasa Indonesia Sebagai Rujukan Penggunaan Bahasa (Sitaksis),” Kande J. Ilm. Pendidik. Bhs. dan Sastra Indones., vol. 06, no. 01, pp. 1–14, 2025, doi: https://doi.org/10.29103/jk.v6i1.21412.

R. Parlika, R. F. Taufiqurrahman, H. I. Farhana, and R. Dimas, “Pengukuran Validitas Aplikasi Akademik Siswa Berbasis Website Menggunakan Metode R-Tabel Dan Cohen’s Kappa,” J. Teknol. Inf., vol. 6, no. 1, 2022, [Online]. Available: https://jurnal.una.ac.id/index.php/jurti/article/view/2617

J. Klie and R. E. De Castilho, “Analyzing Dataset Annotation Quality Management in the Wild,” Comput. Linguist., 2024, doi: https://doi.org/10.1162/coli_a_00516.

M. V. S. Reddy, B. C. Samantha, B. Madhavi, S. Likhita, and K. Sumathi, “Convert Emoji Into Text Using Python,” Int. J. Progress. Res. Eng. Manag. Sci., pp. 1711–1715, 2025, [Online]. Available: https://www.ijprems.com/research-paper/convert-emoji-into-text-using-python

F. Stahlberg and S. Kumar, “Seq2Edits: Sequence transduction using span-level edit operations,” in EMNLP 2020 - 2020 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference, 2020, pp. 5147–5159. doi: 10.18653/v1/2020.emnlp-main.418.

P. M. S. Ardinata, A. A. J. Permana, and I. N. S. W. Wijaya, “Identifikasi Dan Normalisasi Teks Slang Dengan Fasttext Pada Twitter Dalam Bahasa Indonesia,” J. Pendidik. Teknol. dan Kejuru., vol. 21, no. 1, pp. 34–44, 2024, [Online]. Available: https://ejournal.undiksha.ac.id/index.php/JPTK/article/view/66381

K. E. Saputra, “Multilabel multiclass sentiment and emotion dataset from indonesian mobile application review,” Data Br., vol. 50, 2023, doi: 10.1016/j.dib.2023.109576.

I. B. Jatiarso and Y. Azhar, “Klasifikasi Emosi Pada Tweet Pengguna Platform X Menggunakan Metode LSTM-GloVe Berbasis SMOTE,” J. Repos., vol. 7, no. 3, pp. 401–406, 2025, [Online]. Available: https://ejournal.umm.ac.id/index.php/repositor/article/view/36608

I. I. A. Nadhiroh, M. Z. Sarwani, and M. Udin, “Mendeteksi Emosi Berdasarkan Postingan Sosial Media X Menggunakan Algoritma Long Short-Term Memory,” Techno.Com, vol. 24, no. 3, pp. 745–755, 2025, doi: 10.62411/tc.v24i3.13509.

J. Wu, M. Barthet, D. Melhart, and G. N. Yannakakis, “Emotions as Ambiguity-aware Ordinal Representations,” in International Conference on Affective Computing and Intelligent Interaction (ACII) Emotions, 2025.

A. F. Suherman, P. P. Lisnaeni, S. A. Izqiatullailiyah, T. Herlinawati, and Ahman, “A Comparative Analysis of Spearman and Pearson Correlation Using SPSS,” Guid. J. Guid. Couns., vol. 6, no. 1, pp. 1–10, 2016, [Online]. Available: https://ejournal.upi.edu/index.php/optima/article/view/79415

C. Lotfi, S. Srinivasan, M. Ertz, and I. Latrous, “Web Scraping Techniques and Applications: A Literature Review,” in Scrs Conference Proceedings on Intelligent Systems, 2021, pp. 381–394. doi: 10.52458/978-93-91842-08-6-38.




DOI: http://dx.doi.org/10.36722/sst.v11i2.5425

Refbacks

  • There are currently no refbacks.


LP2M (Lembaga Penelitian dan Pengembangan Masyarakat)

Universitas AL-AZHAR INDONESIA, Lt.2 Ruang 207

Kompleks Masjid Agung Al Azhar

Jl. Sisingamangaraja, Kebayoran Baru

Jakarta Selatan 12110

Visitor

 This work is licensed under CC BY 4.0