Empirical Study on the Effectiveness of DistilBERT Fine-tuning on IMDb Sentiment Classification Outperforming CNN/LSTM

Abstract

Sentiment analysis of user reviews is a core Natural Language Processing (NLP) task with practical uses in real scenarios like recommendation. However, the traditional approach like training neural networks models such as CNN, RNN, and LSTM, has faced challenges in improving recognition accuracy. With the development of Language Models (LM) with their self-attention mechanism for contextual understanding, this research wants to see if Language Models exceed Neural Networks on this task. This study conducts a controlled comparison on the IMDb Large Movie Review Dataset (50K reviews) for binary sentiment classification of long movie reviews. The evaluation of four models is based on the dataset that has been cleaned with max sequence length 256, and a stratified 8:1:1 train/validation/test split with multi-seeds. On the IMDb test set, TextCNN performs with an accuracy of 0.873, LSTM reaches 0.862, while DistilBERT achieves 0.911, consistently outperforming strong CNN/LSTM baselines by about 4%. In addition, an accuracy and latency trade-off is observed: DistilBERT offers the best quality with moderate runtime, while TextCNN/LSTM deliver lower latency. Overall, results confirm that a compact pretrained model with fine-tuning provides clear quality gains on long, nuanced reviews, while traditional approaches remain attractive when speed and simplicity are the priority.