Study on the Classification of Chinese Black Humor Texts based on Transformer, Word2Vec, and BERT Techniques
DOI:
https://doi.org/10.54097/qrnqm918Keywords:
Transformer, BERT, Word2Vec, Black Humor, Sentiment ClassificationAbstract
With the rapid development of information technology, the number of Chinese black humor texts has surged. Efficiently and accurately classifying their sentiment orientation has become a key research focus in natural language processing (NLP). This study delves into the performance of three models-BERT, Transformer, and Word2Vec-on the task of Chinese black humor text classification. By collecting and organizing relevant corpora from online forums, black humor expressions, and GitHub repositories, two types of datasets were constructed: single-sentence and question-answer datasets. Comparative analyses of the three models reveal that the BERT model, leveraging its bidirectional encoding mechanism and pre-training capabilities, outperforms others in understanding nuanced emotional tendencies and complex contextual relationships in Chinese black humor texts. This research not only provides novel solutions for Chinese black humor text classification but also enhances our understanding of pre-trained language models in domain-specific applications.
Downloads
References
[1] Turing, A. M. (2009). Computing machinery and intelligence. Springer Netherlands. pp. 23-65
[2] Devlin, J. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arxiv preprint arxiv:1810.04805.
[3] Ahuja, V., Bali, T., & Singh, N. (2018). What makes us laugh? Investigations into automatic humor classification. In Proceedings of the Second Workshop on Computational Modeling of People’s Opinions, Personality, and Emotions in Social Media. pp. 1-9.
[4] Chen, P. Y., & Soo, V. W. (2018). Humor recognition using deep learning. In Proceedings of the 2018 conference of the north american chapter of the association for computational linguistics: Human language technologies, 2, 113-117.
[5] Samson, M., & Gifu, D. (2021). FII FUNNY at SemEval-2021 Task 7: HaHackathon: Detecting and rating Humor and Offense. In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021). pp. 1226-1231.
[6] Mikolov, T. (2013). Efficient estimation of word representations in vector space. arxiv preprint arxiv:1301.3781, 3781.
[7] Lilleberg, J., Zhu, Y., & Zhang, Y. (2015). Support vector machines and word2vec for text classification with semantic features. In 2015 IEEE 14th International Conference on Cognitive Informatics & Cognitive Computing (ICCI* CC). pp. 136-140.
[8] Sidan, M., & Dongsu, L. (2019). Text classification method based on weighted Word2Vec. Information Science, 11, 38–42.
[9] Li, S. (2020). Subword-level Chinese text classification method based on BERT. Computational Science and Applications, 10, 12677.
[10] Yu, Q., Wang, Z., & Jiang, K. (2021). Research on text classification based on bert-bigru model. In Journal of Physics: Conference Series. 1746(1), 012019.
[11] Thomsen, D. B., de la Broıse, J. B., & Mielonen, E. Recognizing Humor and Predicting Humor Ratings in Short Texts.
[12] Li, Y., Zhou, Q., Luo, Y., Ma, S., Li, Y., Zheng, H. T., ... & Yu, P. S. (2024). When llms meet cunning questions: A fallacy understanding benchmark for large language models. arxiv preprint arxiv:2402.11100.
[13] Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention is all you need. Proceedings of the Neural Information Processing Systems (NeurIPS), 1–11.
[14] He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770–778.
[15] Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., ... & Liu, P. J. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140), 1-67.
[16] Dai, Z. (2019). Transformer-xl: Attentive language models beyond a fixed-length context. arxiv preprint arxiv:1901.02860.
[17] Lan, Z. (2019). Albert: A lite bert for self-supervised learning of language representations. arxiv preprint arxiv:1909.11942.
[18] Nair, V., & Hinton, G. E. (2010). Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (ICML-10). pp. 807-814.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 International Journal of Education and Humanities
This work is licensed under a Creative Commons Attribution 4.0 International License.
