IMDb-Sentimentanalyse unter Verwendung von DistilBERT LoRA und TF-IDF-Baselines: Kalibrierung, Interpretierbarkeit und semi-überwachtes Lernen

We configure the Colab environment, install the required libraries, apply the PEFT–torchao compatibility fix, and set deterministic seeds for reproducible experiments. We load the Stanford IMDb dataset, shuffle and subsample the train and test splits, and inspect class balance, review-length distributions, duplicate leakage, and HTML artifacts. We also visualize review lengths and label frequencies so we understand the dataset structure before building any models.

We train a strong TF-IDF and Logistic Regression baseline and inspect the most influential positive and negative n-grams to establish an interpretable reference point. We then tokenize the IMDb reviews and configure DistilBERT with LoRA adapters that update only a small subset of model parameters while keeping the backbone largely frozen. We use the Hugging Face Trainer with dynamic padding, early stopping, mixed precision, and multiple evaluation metrics to fine-tune the transformer efficiently.

We evaluate the fine-tuned DistilBERT-LoRA model using classification metrics, a confusion matrix, and ROC curves while directly comparing its ROC-AUC performance with the TF-IDF baseline. We sweep classification thresholds to determine whether the default probability cutoff of 0.5 gives the best accuracy on our evaluation set. We also calculate Expected Calibration Error and construct a reliability diagram to measure how closely the model’s predicted confidence corresponds to its actual correctness.

We examine the model’s most confident incorrect predictions and group reviews by length to identify truncation-related failure patterns and difficult examples. We merge the LoRA adapters into the underlying model and apply leave-one-word-out occlusion to estimate which words push individual predictions toward positive or negative sentiment. We then compare predictions based on the beginning and ending portions of long reviews to determine where the strongest sentiment information resides.

We use the fine-tuned transformer to generate high-confidence pseudo-labels for examples from IMDb’s unlabeled split and add these examples to the TF-IDF training corpus. We compare the augmented classifier against the original baseline to measure whether semi-supervised self-training improves predictive accuracy. Finally, we save the merged DistilBERT model and tokenizer, run sentiment inference on custom reviews, and summarize the performance of all models developed throughout the tutorial.

In conclusion, we developed a rigorous sentiment classification pipeline that goes well beyond simply fine-tuning a transformer and reporting accuracy. We established a competitive TF-IDF baseline, train DistilBERT efficiently with LoRA, and evaluate both predictive quality and probability reliability while identifying how review length, truncation, and highly confident mistakes influence real-world performance. We also interpreted individual predictions through occlusion-based saliency, tested whether sentiment information is concentrated near the beginning or end of long reviews, and extended supervised learning with high-confidence pseudo-labels from the unlabeled dataset.

Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us ：https://forms.gle/wbash1wF6efRj8G58

Sana Hassan, a consulting intern at Marktechpost and dual-degree student at IIT Madras, is passionate about applying technology and AI to address real-world challenges. With a keen interest in solving practical problems, he brings a fresh perspective to the intersection of AI and real-life solutions.