Enhancing Text Classification Performance: A Comparative Study of RNN and GRU Architectures with Attention Mechanisms
DOI:
https://doi.org/10.61098/jarcis.v2i2.187Keywords:
Text Sentiment Prediction, Attention Mechanism, Preprocessing, Sentiment AnalysisAbstract
Text classification plays a crucial role in natural language processing, and enhancing its performance is an ongoing area of research. This study investigates the impact of integrating attention mechanisms into a recurrent neural network (RNN) based architectures, including RNN, LSTM, GRU, and their bidirectional variants (BiLSTM and BiGRU), for text sentiment analysis. Three attention mechanisms Multihead Attention, Self Attention, and Adaptive Attention are applied to evaluate their effectiveness in improving model accuracy. The results reveal that attention mechanisms significantly enhance performance by enabling models to focus on the most relevant parts of the input text. Among the tested configurations, the LSTM model with Multihead Attention achieved the highest accuracy of 68.34%. The findings underscore the critical role of attention mechanisms in overcoming traditional RNN limitations, such as difficulty in capturing long-term dependencies, and highlight the potential for their application in broader text classification tasks.
Downloads
References
Y. A. Putri Gabriella, “Optimasi Penerimaan Siswa Baru Dengan Penerapan Algortima Text Mining Dan Tf-Idf,” J. Comput. Informatics Res., vol. 2, no. 3, pp. 110–117, 2023, doi: 10.47065/comforch.v2i3.941.
T. E. Hidayat and A. Rosid, “Analysis of Community Sentiments Regarding Plans to Relocate National Capital Using the Naïve Bayes Method Analisa Sentimen Masyarakat Tentang Rencana Pemindahan Ibukota Negara Dengan Metode Naïve Bayes,” Network, Comput. Sci. |, vol. 3, no. 2, pp. 43–49, 2020.
R. Siringoringo and J. Jamaludin, “Text Mining dan Klasterisasi Sentimen Pada Ulasan Produk Toko Online,” J. Teknol. dan Ilmu Komput. Prima, vol. 2, no. 1, pp. 41–48, 2019, doi: 10.34012/jutikomp.v2i1.456.
T. A. Zuraiyah, D. K. Utami, and D. Herlambang, “Implementasi Chatbot Pada Pendaftaran Mahasiswa Baru Menggunakan Recurrent Neural Network,” J. Ilm. Teknol. dan Rekayasa, vol. 24, no. 2, pp. 91– 101, 2019, doi: 10.35760/tr.2019.v24i2.2388.
D. Sintia Amelia, N. Cahyana Aminuallah, and S. Informasi, “Teks Dan Analisis Sentimen Pada Chat Grup Whatsapp Menggunakan Long Short Term Memory (Lstm),” J. Teknol. Terkini, vol. 3, no. 2, p. 1, 2023, [Online]. Available: http://teknologiterkini.org/index.php/terkini/article/view/354
I. N. Husada and H. Toba, “Pengaruh Metode Penyeimbangan Kelas Terhadap Tingkat Akurasi Analisis Sentimen pada Tweets Berbahasa Indonesia,” J. Tek. Inform. dan Sist. Inf., vol. 6, no. 2, pp. 400–413, 2020, doi: 10.28932/jutisi.v6i2.2743.
F. A. Verilia, R. Firdaus, and H. D. Septama, “Pengembangan Pengenalan Aktivitas Manusia Secara Real Time Menggunakan Metode Convolutional Neural Network dan Deep Gated Recurrent Unit,” ULIL ALBAB J. Ilm. Multidisiplin, vol. 2, no. 2, pp. 899–909, 2023, [Online]. Available: https://journal-nusantara.com/index.php/JIM/article/view/1351
S. J. Pipin, R. Purba, and H. Kurniawan, “Prediksi Saham Menggunakan Recurrent Neural Network (RNN-LSTM) dengan Optimasi Adaptive Moment Estimation,” J. Comput. Syst. Informatics, vol. 4, no. 4, pp. 806–815, 2023, doi: 10.47065/josyc.v4i4.4014.
A. Wijanarko, A. N. Al Haura, I. Puspitaningrum, D. Intan, and S. Saputra, “Model Recurrent Neural Network-Gated Recurrent Unit untuk Membangun Mesin Penerjemah Bahasa Indonesia- Banyumasan,” J. Eksplora Inform., pp. 218–226, 2023, doi: 10.30864/eksplora.v13i2.977.
Z. Wahyuzi, “Analisis dan Prediksi Konsumsi Listrik Smart Office Berbasis IoT Terhadap Faktor Internal dan Eksternal Menggunakan Deep Learning,” 2024.
W. Astriningsih and D. H. Fudholi, “Multi Aspek Sentimen Analisis pada Review Hotel Menggunakan Deep learning,” JATISI (Jurnal Tek. Inform. dan Sist. Informasi), vol. 10, no. 3, p. 433, 2023, [Online]. Available: https://jurnal.mdp.ac.id/index.php/jatisi/article/view/5321
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Yulita Ayu Wardani, Mery Oktaviyanti Puspitaningtyas, Happid Ridwan Ilmi, Onesinus Saut Parulian
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
