Multitask Sentiment Analysis and Topic Classification Using BERT
DOI:
https://doi.org/10.4108/eetsis.5287Keywords:
BERT, Analyzing Sentiments, Categorizing Topics, Multitasking in Learning, Processing Natural Language, Machine Learning Techniques, News Dataset, Retrieval InformationAbstract
In this study, a multitask model is proposed to perform simultaneous news category and sentiment classification of a diverse dataset comprising 3263 news records spanning across eight categories, including environment, health, education, tech, sports, business, lifestyle, and science. Leveraging the power of Bidirectional Encoder Representations from Transformers (BERT), the algorithm demonstrates remarkable results in both tasks. For topic classification, it achieves an accuracy of 98% along with balanced precision and recall, substantiating its proficiency in categorizing news articles. For sentiment analysis, the model maintains strong accuracy at 94%, distinguishing positive from negative sentiment effectively. This multitask approach showcases the model's versatility and its potential to comprehensively understand and classify news articles based on content and sentiment. This multitask model not only enhances classification accuracy but also improves the efficiency of handling extensive news datasets. Consequently, it empowers news agencies, content recommendation systems, and information retrieval services to offer more personalized and pertinent content to their users.
References
Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). BERT: Bidirectional Encoder Representations from Transformers. arXiv preprint arXiv:1810.04805.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. In Advances in neural information processing systems (pp. 30-48).
Howard, J., & Ruder, S. (2018). Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., ... & Zettlemoyer, L. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
Xie, Z., Xiao, Y., Wang, T., Zhou, B., Lin, Z., & An, L. (2021). Towards Transparent and Controllable Attention Mechanisms in NLP. arXiv preprint arXiv:2102.11941.
Kang, J. Y., Lee, S. H., & Jang, S. (2021). A personalized news recommendation model based on multi-task learning. Expert Systems with Applications, 165, 114118
R. Khandelwal, A. Nayak, H. Harkous and K. Fawaz. "CookieEnforcer: Automated Cookie Notice Analysis and Enforcement". Jan. 2022.
H. Zankadi, A. Idrissi, N. Daoudi and I. Hilal. "Identifying learners’ topical interests from social media content to enrich their course preferences in MOOCs using topic modeling and NLP techniques". Nov. 2022.
M. F. Mridha, M. A. H. Wadud, A. Hamid, M. M. Monowar, M. Abdullah-Al-Wadud and A. Alamri. "L-Boost: Identifying Offensive Texts From Social Media Post in Bengali". Jan. 2021.
M. Schirmer, U. Kruschwitz and G. Donabauer. "A New Dataset for Topic-Based Paragraph Classification in Genocide-Related Court Transcripts". Jan. 2022.
Y. Wang, Q. Chen, and W. Wang, ‘Multi-task BERT for Aspect-based Sentiment Analysis’, in 2021 IEEE International Conference on Smart Computing (SMARTCOMP), 8 2021.
Shiwang Huang, Xiaoyu Wang, Xiaohan, Ji, Jing Xie, and Qin Tang, ‘Network News Sentiment Analysis Based on BERT’.
Sarojadevi Palani, P. Rajagopal, and Sidharth Pancholi, ‘T-BERT - Model for Sentiment Analysis of Micro-blogs Integrating Topic Model and BERT’.
M. F. Abdussalam, D. Richasdy, and M. A. Bijaksana, ‘BERT Implementation on News Sentiment Analysis and Analysis Benefits on Branding’, JURNAL MEDIA INFORMATIKA BUDIDARMA, vol. 6, no. 4, p. 2064, Oct. 2022.
G. Li et al., ‘A BERT-based Text Sentiment Classification Algorithm through Web Data’, in 2022 International Conference on Computer Engineering and Artificial Intelligence (ICCEAI), 7 2022.
Samir, S. M. Elkaffas, and M. M. Madbouly, ‘Twitter Sentiment Analysis Using BERT’, in 2021 31st International Conference on Computer Theory and Applications (ICCTA), 2021.
Z. Gao, A. Feng, X. Song, and X. Wu, ‘Target-Dependent Sentiment Classification With BERT’, IEEE Access, vol. 7, pp. 154290–154299, 2019.
‘Online News Monitoring and Sentiment Analysis using BERT Approach’, International Journal of
Advanced Research in Technology and Innovation, Jan. 2023.
S. R. Pingili and L. Li, ‘Target-Based Sentiment Analysis using a BERT Embedded Model’, in 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI), 11 2020.
J. Lei, Q. Zhang, J. Wang, and H. Luo, ‘BERT Based Hierarchical Sequence Classification for Context-Aware Microblog Sentiment Analysis’, in Neural Information Processing, Springer International Publishing, 2019, pp. 376–386.
V. Yadav and S. Shakya, ‘Sentiment Analysis and Topic Modeling on News Headlines’, Journal of Ubiquitous Computing and Communication Technologies, vol. 4, no. 3, pp. 204–218, Sep. 2022.
R. Man and K. Lin, ‘Sentiment Analysis Algorithm Based on BERT and Convolutional Neural Network’, in 2021 IEEE Asia-Pacific Conference on Image Processing, Electronics and Computers (IPEC), 2021.
G. Wei, ‘Research on Internet Text Sentiment Classification Based on BERT and CNN-BiGRU’, in 2022 11th International Conference on Communications, Circuits and Systems (ICCCAS), 2022.
Jinbin Cai, Fei Chen, and Si-xuan Chen, ‘Sentiment Classification Based On BERT’.
Florian Bütow, Florian Schultze, and Leopold Strauch, ‘Semantic Search : Sentiment Analysis with Machine Learning Algorithms on German News Articles’. .
M. Agarwal, P. K. Chaudhary, S. K. Singh, and C. Vij, ‘Sentiment Analysis Dashboard for Socia Media comments using BERT’, in 2023 International Conference on Advancement in Computation & Computer Technologies (InCACCT), 2023.
Bello, S.-C. Ng, and M.-F. Leung, ‘A BERT Framework to Sentiment Analysis of Tweets’, Sensors, vol. 23, no. 1, p. 506, Jan. 2023.
Xiaohong Cai, Hui Cao, and Jin-gang Ma, ‘Sentiment Analysis of E-commerce Comments Based on BERT’.
‘Performance based Machine Learning Algorithm for Topic Oriented Text Categorization’, International Journal of Recent Technology and Engineering, vol. 8, no. 2S11, pp. 3501–3506, Nov. 2019.
C. Wu, F. Wu, T. Qi, Y. Huang, and X. Xie, ‘Title-Aware Neural News Topic Prediction’, in Lecture
Notes in Computer Science, Springer International Publishing, 2019, pp. 181–193.
K. Cai, S. Spangler, Y. Chen, and L. Zhang, ‘Leveraging sentiment analysis for topic detection’, Web Intelligence and Agent Systems: An International Journal, vol. 8, no. 3, pp. 291–302, 2010.
D. Rajput and S. Verma, ‘An Attention Arousal Space for Mapping Twitter Data’, in Lecture Notes in Electrical Engineering, Springer Singapore, 2020, pp. 381–395.
S. D. Tembhurnikar and N. N. Patil, ‘Topic detection using BNgram method and sentiment analysis on twitter dataset’, in 2015 4th International Conference on Reliability, Infocom Technologies and Optimization (ICRITO) (Trends and Future Directions), 9 2015.
Singh and G. Jain, ‘Sentiment Analysis of News Headlines Using Simple Transformers’, in 2021 Asian Conference on Innovation in Technology (ASIANCON), 2021.
D.Deepa, ‘Bidirectional Encoder Representations from Transformers (BERT) Language Model for Sentiment Analysis task: Review’.
S. S. Hossain, Y. Arafat, and M. E. Hossain, ‘Context-Based News Headlines Analysis: A Comparative Study of Machine Learning and Deep Learning Algorithms’, Vietnam Journal of Computer Science, vol. 08, no. 04, pp. 513–527, Apr. 2021.
H. Batra, N. S. Punn, S. K. Sonbhadra, and S. Agarwal, ‘BERT-Based Sentiment Analysis: A Software Engineering Perspective’, in Lecture Notes in Computer Science, Springer International Publishing, 2021, pp. 138–148.
J. Zheng, X. Chen, Y. Du, X. Li, and J. Zhang, ‘Short Text Sentiment Analysis of Micro-blog Based on BERT’, in Lecture Notes in Electrical Engineering, Springer Singapore, 2019, pp. 390–396.
S. Kaman, ‘News Sentiment Analysis By Using Deep Learning Framework’, May 2020.
P. Liu, J. A. Gulla, and L. Zhang, ‘Dynamic Topic-Based Sentiment Analysis of Large-Scale Online News’, in Web Information Systems Engineering -- WISE 2016, Springer International Publishing, 2016, pp. 3–18.
S. Rahman, S. S. Hossain, S. Islam, M. I. Chowdhury, F. B. Rafiq, and K. B. M. Badruzzaman, ‘Context-Based News Headlines Analysis Using Machine Learning Approach’, in Computational Collective Intelligence, Springer International Publishing, 2019, pp. 167–178.
P. Liu, J. A. Gulla, and L. Zhang, ‘RETRACTED ARTICLE: A joint model for analyzing topic and sentiment dynamics from large-scale online news’, World Wide Web, vol. 21, no. 4, pp. 1117–1139, Jul. 2017.
X. Zhang, Z. Wu, K. Liu, Z. Zhao, J. Wang, and C. Wu, ‘Text Sentiment Classification Based on BERT Embedding and Sliced Multi-Head Self-Attention Bi-GRU’, Sensors, vol. 23, no. 3, p. 1481, Jan. 2023.
Xiao, L., Xue, Y., Wang, H., Hu, X., Gu, D., & Zhu, Y. (2022). Exploring fine-grained syntactic information for aspect-based sentiment classification with dual graph neural networks. Neurocomputing, 471, 48-59.
Chandraprabha, M., and Rajesh Kumar Dhanraj. "Ensemble Deep Learning Algorithm for Forecasting of Rice Crop Yield based on Soil Nutrition Levels." EAI Endorsed Transactions on Scalable Information Systems 10, no. 4 (2023).
Singh, R., Subramani, S., Du, J., Zhang, Y., Wang, H., Miao, Y., & Ahmed, K. (2023). Antisocial Behavior Identification from Twitter Feeds Using Traditional Machine Learning Algorithms and Deep Learning. EAI Endorsed Transactions on Scalable Information Systems, 10(4).
Li, L. (2023). Deep Learning Algorithm Aided E-Commerce Logistics Node Layout Optimization Based on Internet of Things Network. EAI Endorsed Transactions on Scalable Information Systems, 10(4), e16-e16.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Parita Shah, Hiren Patel, Priya Swaminarayan
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This is an open access article distributed under the terms of the CC BY-NC-SA 4.0, which permits copying, redistributing, remixing, transformation, and building upon the material in any medium so long as the original work is properly cited.
