AI-Powered Fake News Detection Using Natural Language Processing
Abstract
Automated detection of fake news, meaning fabricated news-style content that may circulate as misinformation or disinformation, is motivated by the speed and volume at which online content spreads and by the limited scalability of manual fact-checking. This paper treats fake-news detection as binary text classification and evaluates a hybrid model that fuses a fine-tuned BERT encoder, a convolutional branch operating over BERT token representations, and an auxiliary branch built from TF-IDF/SVD features and seven handcrafted linguistic features. The model is compared with Naive Bayes, logistic regression, a linear SVM, a word-embedding CNN, an LSTM and fine-tuned BERT on the WELFake dataset [1] (62,833 articles after cleaning; 55.36% real and 44.64% fake), using a stratified 80/10/10 split repeated with random seeds 42, 43 and 44. Over the three runs, the hybrid obtained mean test accuracy, precision, recall and F1-score of 99.62%, 99.64%, 99.51% and 99.58%, compared with 99.45%, 99.42%, 99.35% and 99.38% for fine-tuned BERT; its mean ROC-AUC (0.99968) was slightly lower than that of BERT (0.99984). No significance test was performed, and with three runs the differences should be interpreted cautiously. The principal limitation is that evaluation is restricted to a single benchmark, so robustness to unseen publishers, topics and time periods is untested. Ablation of the individual branches, cross-dataset evaluation, inference-latency measurement and prediction-level explanation are left for future work.
Keywords: Fake news detection, Natural language processing, BERT, Text classification, Deep learning, Misinformation, Linguistic features, WELFake.
References
- [1] P. K. Verma, P. Agrawal, and R. Prodan, "WELFake dataset for fake news detection in text data," Zenodo, 2021, doi: 10.5281/zenodo.4561253.
- [2] S. Vosoughi, D. Roy, and S. Aral, "The spread of true and false news online," Science, vol. 359, no. 6380, pp. 1146–1151, 2018.
- [3] H. Allcott and M. Gentzkow, "Social media and fake news in the 2016 election," J. Econ. Perspect., vol. 31, no. 2, pp. 211–236, 2017, doi: 10.1257/jep.31.2.211.
- [4] N. Grinberg, K. Joseph, L. Friedland, B. Swire-Thompson, and D. Lazer, "Fake news on Twitter during the 2016 U.S. presidential election," Science, vol. 363, no. 6425, pp. 374–378, 2019.
- [5] D. M. J. Lazer et al., "The science of fake news," Science, vol. 359, no. 6380, pp. 1094–1096, 2018, doi: 10.1126/science.aao2998.
- [6] K. Shu, A. Sliva, S. Wang, J. Tang, and H. Liu, "Fake news detection on social media: A data mining perspective," ACM SIGKDD Explor. Newsl., vol. 19, no. 1, pp. 22–36, 2017, doi: 10.1145/3137597.3137600.
- [7] X. Zhou and R. Zafarani, "A survey of fake news: Fundamental theories, detection methods, and opportunities," ACM Comput. Surv., vol. 53, no. 5, Art. no. 109, pp. 1–40, 2020.
- [8] R. Oshikawa, J. Qian, and W. Y. Wang, "A survey on natural language processing for fake news detection," in Proc. 12th Language Resources and Evaluation Conf. (LREC), 2020, pp. 6086–6093.
- [9] H. Ahmed, I. Traore, and S. Saad, "Detection of online fake news using N-gram analysis and machine learning techniques," in Intelligent, Secure, and Dependable Systems in Distributed and Cloud Environments, Springer, 2017, pp. 127–138.
- [10] V. Pérez-Rosas, B. Kleinberg, A. Lefevre, and R. Mihalcea, "Automatic detection of fake news," in Proc. 27th Int. Conf. Computational Linguistics, 2018, pp. 3391–3401.
- [11] B. D. Horne and S. Adali, "This just in: Fake news packs a lot in title, uses simpler, repetitive content in text body, more similar to satire than real news," in Proc. 2nd Int. Workshop on News and Public Opinion, 2017, pp. 1–9.
- [12] Y. Kim, "Convolutional neural networks for sentence classification," in Proc. EMNLP, 2014, pp. 1746–1751.
- [13] N. Ruchansky, S. Seo, and Y. Liu, "CSI: A hybrid deep model for fake news detection," in Proc. ACM CIKM, 2017, pp. 797–806.
- [14] J. A. Nasir, O. S. Khan, and I. Varlamis, "Fake news detection: A hybrid CNN-RNN based deep learning approach," Int. J. Inf. Manage. Data Insights, vol. 1, no. 1, 100007, 2021.
- [15] R. K. Kaliyar, A. Goswami, P. Narang, and S. Sinha, "FNDNet — A deep convolutional neural network for fake news detection," Cognit. Syst. Res., vol. 61, pp. 32–44, 2020, doi: 10.1016/j.cogsys.2019.12.005.
- [16] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of deep bidirectional transformers for language understanding," in Proc. NAACL-HLT, 2019, pp. 4171–4186.
- [17] R. K. Kaliyar, A. Goswami, and P. Narang, "FakeBERT: Fake news detection in social media with a BERT-based deep learning approach," Multimed. Tools Appl., vol. 80, pp. 11765–11788, 2021, doi: 10.1007/s11042-020-10183-2.
- [18] P. K. Verma, P. Agrawal, V. Madaan, and R. Prodan, "MCred: Multi-modal message credibility for fake news detection using BERT and CNN," J. Ambient Intell. Humaniz. Comput., published online Jul. 27, 2022, doi: 10.1007/s12652-022-04338-2.
- [19] A. J. Keya, H. H. Shajeeb, M. S. Rahman, and M. F. Mridha, "FakeStack: Hierarchical Tri-BERT-CNN-LSTM stacked model for effective fake news detection," PLOS ONE, vol. 18, no. 12, e0294701, 2023, doi: 10.1371/journal.pone.0294701.
- [20] F. Al-Quayed, D. Javed, N. Z. Jhanjhi, M. Humayun, and T. S. Alnusairi, "A hybrid transformer-based model for optimizing fake news detection," IEEE Access, vol. 12, pp. 160822–160834, 2024, doi: 10.1109/ACCESS.2024.3476432.
- [21] P. K. Verma, P. Agrawal, I. Amorim, and R. Prodan, "WELFake: Word embedding over linguistic features for fake news detection," IEEE Trans. Comput. Soc. Syst., vol. 8, no. 4, pp. 881–893, 2021, doi: 10.1109/TCSS.2021.3068519.
- [22] H. Ahmed, I. Traore, and S. Saad, "Detecting opinion spams and fake news using text classification," Security and Privacy, vol. 1, no. 1, e9, 2018.
- [23] G. Gravanis, A. Vakali, K. Diamantaras, and P. Karadais, "Behind the cues: A benchmarking study for fake news detection," Expert Syst. Appl., vol. 128, pp. 201–213, 2019.
- [24] S. Kwon, M. Cha, K. Jung, W. Chen, and Y. Wang, "Prominent features of rumor propagation in online social media," in Proc. IEEE 13th Int. Conf. Data Mining, 2013, pp. 1103–1108.
- [25] X. Dong, U. Victor, and L. Qian, "Two-path deep semi-supervised learning for timely fake news detection," IEEE Trans. Comput. Soc. Syst., 2020 (preprint: arXiv:2002.00763).
- [26] W. Y. Wang, "'Liar, liar pants on fire': A new benchmark dataset for fake news detection," in Proc. 55th Annu. Meeting Assoc. Comput. Linguistics, 2017, pp. 422–426.
- [27] A. Zubiaga, M. Liakata, R. Procter, G. Wong Sak Hoi, and P. Tolmie, "Analysing how people orient to and spread rumours in social media by looking at conversational threads," PLoS ONE, vol. 11, no. 3, e0150989, 2016.
- [28] F. K. A. Salem, R. Al Feel, S. Elbassuoni, M. Jaber, and M. Farah, "FA-KES: A fake news dataset around the Syrian war," in Proc. Int. AAAI Conf. Web and Social Media, vol. 13, 2019, pp. 573–582.
- [29] W. Lifferth, "Fake News," Kaggle Dataset, 2018. [Online]. Available: https://www.kaggle.com/c/fake-news/data
- [30] K. Shu, D. Mahudeswaran, S. Wang, D. Lee, and H. Liu, "FakeNewsNet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media," Big Data, vol. 8, no. 3, pp. 171–188, 2020.
- [31] R. Zellers, A. Holtzman, H. Rashkin, Y. Bisk, A. Farhadi, F. Roesner, and Y. Choi, "Defending against neural fake news," in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 32, 2019.
- [32] J. Thorne, A. Vlachos, C. Christodoulopoulos, and A. Mittal, "FEVER: A large-scale dataset for fact extraction and VERification," in Proc. NAACL-HLT, 2018, pp. 809–819.
Explore Our Related Journals
Looking for the right journal for your next manuscript? Explore our international peer-reviewed journals covering engineering, management, computer science, artificial intelligence and multidisciplinary research.