A Review of Document Classification Techniques Using Machine Learning and Deep Learning
DOI:
https://doi.org/10.53032/tvcr/2025.v7n2.23Keywords:
Document, Classification, Deep Learning, RNN, BERTAbstract
The study shows different machine learning and natural language processing techniques are used to address fully automated text classification of extensive datasets. The research looks at multiple studies which employ probabilistic models with deep learning approaches and established machine learning methods to identify documents. The discussion evaluates target model advantages against disadvantages while exploring future development paths in order to resolve the need for highly accurate scalable classification systems. This research evaluates how transformer-based models recently developed will affect classification model outcomes.
References
Sebastiani, F. (2002). Machine learning in automated text categorization. ACM Computing Surveys (CSUR), 34(1), 1–47. https://doi.org/10.1145/505282.505283
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS) (pp. 5998–6008). Curran Associates, Inc.
Devlin, J., Chang, M. W., Lee, K., Toutanova, K. (2018). BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. https://arxiv.org/abs/1810.04805
Minaee, S., Kalchbrenner, N., Cambria, E., Nikzad, N., Chenaghlu, M., Gao, J. (2021). Deep learning-based text classification: A comprehensive review. ACM Computing Surveys (CSUR), 54(3), 1–40. https://doi.org/10.1145/3439726
Baygin, M. (2018). Classification of text documents based on naïve Bayes using n-gram features. Journal of Information Science and Engineering, 34(4), 987–1002.
Deshmukh, R., Patil, M., Bhosale, R. (2019). A document classification using NLP and recurrent neural network. International Journal of Computer Applications, 181(5), 23–29. https://doi.org/10.5120/ijca2019918562
Schwenk, H., Li, X. (2018). A corpus for multilingual document classification in eight languages. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC).
Cheng, Y. (2019). Document classification based on convolutional neural network and hierarchical attention network. Neural Networks, 110, 56–64. https://doi.org/10.1016/j.neunet.2019.07.004
Adhikari, A., Ram, A., Tang, R., Lin, J. (2019). DocBERT: BERT for document classification. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP), 961–969. https://doi.org/10.18653/v1/D19-1094
Huang, X., Paul, M. J. (2018). Examining temporality in document classification Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 394–404. https://doi.org/10.18653/v1/N18-1036
Aggarwal, C. C., Zhai, C. (2012). Mining text data. Springer. https://doi.org/10.1007/978-1-4614-3223-4
Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems (NeurIPS) (pp. 3111–3119).
Howard, J., Ruder, S. (2018). Universal language model fine-tuning for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL) (pp. 328–339). Association for Computational Linguistics. https://doi.org/10.18653/v1/P18-1031
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P. J. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140), 1–67.
Samek, W., Wiegand, T., Müller, K. R. (2019). Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. arXiv preprint arXiv:1708.08296. https://arxiv.org/abs/1708.08296
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 The Voice of Creative Research
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
