Deep Transform Ensemble Model for Sentiment Analysis
Corressponding author's email:
trangpth@hcmute.edu.vnDOI:
https://doi.org/10.54644/jte.2025.1897Keywords:
BERT-CNN, BERT-BiLSTM, Deep transform ensemble, Ensemble model, Sentiment analysisAbstract
The ensemble method is a technique that has garnered significant attention in recent years, particularly in the field of sentiment analysis. It leverages the strengths of multiple models to enhance overall performance. Although many ensemble methods for sentiment analysis have been proposed, few have incorporated deep learning models. In this study, we propose an ensemble model based on transformers and deep learning to improve sentiment analysis performance. The proposed model comprises the following main components: (i) an embedding layer, which converts input sentences into vector matrices; (ii) a BERT-LSTM-based sentiment classifier, which extracts and learns global and contextual features from the embedding layer; (iii) a BERT-CNN-based sentiment classifier, which extracts and learns local and semantic features from the embedding layer; (iv) an ensemble layer, which combines the extracted features; and (v) an ensemble classifier layer, which classifies the sentiment of the input sentences. The model is evaluated on four benchmark datasets. Experimental results show that it improves sentiment analysis performance by at least 0.02 and up to 0.05.
Downloads: 0
References
P. Meel, P. Chawla, S. Jain, and U. Rai, "Web text content credibility analysis using max voting and stacking ensemble classifiers," in 2020 Advanced Computing and Communication Technologies for High Performance Applications (ACCTHPA), 2020, pp. 157-161. DOI: https://doi.org/10.1109/ACCTHPA49271.2020.9213234
P. K. Jain, V. Saravanan, and R. Pamula, "A hybrid CNN-LSTM: A deep learning approach for consumer sentiment analysis using qualitative user-generated contents," Transactions on Asian and Low-Resource Language Information Processing, vol. 20, no. 5, pp. 1-15, 2021. DOI: https://doi.org/10.1145/3457206
M. Rhanoui, M. Mikram, S. Yousfi, and S. Barzali, "A CNN-BiLSTM model for document-level sentiment analysis," Machine Learning and Knowledge Extraction, vol. 1, no. 3, pp. 832-847, 2019. DOI: https://doi.org/10.3390/make1030048
K. L. Tan, C. P. Lee, K. S. M. Anbananthen, and K. M. Lim, "RoBERTa-LSTM: a hybrid model for sentiment analysis with transformer and recurrent neural network," IEEE Access, vol. 10, pp. 21517-21525, 2022. DOI: https://doi.org/10.1109/ACCESS.2022.3152828
K. L. Tan, C. P. Lee, K. M. Lim, and K. S. M. Anbananthen, "Sentiment analysis with ensemble hybrid deep learning model," IEEE Access, vol. 10, pp. 103694-103704, 2022. DOI: https://doi.org/10.1109/ACCESS.2022.3210182
M. S. Hossen, A. H. Jony, T. Tabassum, M. T. Islam, M. M. Rahman, and T. Khatun, "Hotel review analysis for the prediction of business using deep learning approach," in 2021 international conference on artificial intelligence and smart systems (ICAIS), 2021. DOI: https://doi.org/10.1109/ICAIS50930.2021.9395757
A. Garg and R. K. Kaliyar, "PSent20: An effective political sentiment analysis with deep learning using real-time social media tweets," in 2020 5th IEEE international conference on recent advances and innovations in engineering (ICRAIE), 2020. DOI: https://doi.org/10.1109/ICRAIE51050.2020.9358379
D. Kotzias, M. Denil, N. De Freitas, and P. Smyth, "From group to individual labels using deep features," in Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, 2015. DOI: https://doi.org/10.1145/2783258.2783380
A. U. Rehman, A. K. Malik, B. Raza, and W. Ali, "A hybrid CNN-LSTM model for improving accuracy of movie reviews sentiment analysis," Multimedia Tools and Applications, vol. 78, pp. 26597-26613, 2019. DOI: https://doi.org/10.1007/s11042-019-07788-7
O. Araque, I. Corcuera-Platas, J. F. Snchez-Rada, and C. A. Iglesias, "Enhancing deep learning sentiment analysis with ensemble techniques in social applications," Expert Systems with Applications, vol. 77, pp. 236-246, 2017. DOI: https://doi.org/10.1016/j.eswa.2017.02.002
S. Minaee, E. Azimi, and A. Abdolrashidi, "Deep-sentiment: Sentiment analysis using ensemble of cnn and bi-lstm models," arXiv preprint arXiv:1904.04206, 2019.
J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, "Bert: Pre-training of deep bidirectional transformers for language understanding," in Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, 2019.
H. T. Phan, N. T. Nguyen, and D. Hwang, "Convolutional attention neural network over graph structures for improving the performance of aspect-level sentiment analysis," Information Sciences, vol. 589, pp. 416-439, 2022. DOI: https://doi.org/10.1016/j.ins.2021.12.127
Y. Kim, "Convolutional neural networks for sentence classification. arXiv 2014," arXiv preprint arXiv:1408.5882, 2014.
H. T. Phan, N. T. Nguyen, D. Hwang, and Y. S. Seo, "M2SA: A novel dataset for multi-level and multi-domain sentiment analysis," Journal of Information and Telecommunication, vol. 7, no. 4, pp. 494-512, 2023. DOI: https://doi.org/10.1080/24751839.2023.2229700
Downloads
Published
How to Cite
Issue
Section
Categories
License
Copyright (c) 2025 Journal of Technical Education Science
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright © JTE.
