TLDR: A study fine-tuned Large Language Models (LLMs) and BERT models for sentiment analysis of e-commerce reviews in Bangladesh, which often contain a mix of Bangla and English. The fine-tuned Llama-3.1-8B model achieved 95.5% accuracy, outperforming other models, demonstrating LLMs’ effectiveness for sentiment classification in low-resource, mixed-language contexts using parameter-efficient fine-tuning.
Understanding customer feedback is crucial for e-commerce businesses, especially in a rapidly growing market like Bangladesh. However, analyzing sentiment from online reviews in this region presents unique challenges due to the diverse linguistic landscape, which often includes a mix of Bangla, English, and “Banglish” (code-mixed Bangla-English).
A recent study by Sumaiya Tabassum, titled “LLM Based Sentiment Classification From Bangladesh E-Commerce Reviews,” explores the effectiveness of Large Language Models (LLMs) and transformer-based BERT models in accurately classifying sentiment from these complex reviews. The research highlights how advanced AI can bridge the gap in natural language processing (NLP) for low-resource languages and mixed-language contexts.
The core of the research involved fine-tuning various LLMs and BERT models on a specific dataset. The dataset itself is quite comprehensive, comprising 78,130 product reviews from major Bangladeshi e-commerce platforms like Daraz and Pickaboo. For the experiments, a subset of 4,000 samples was used, focusing on binary sentiment classification (positive or negative).
The methodology employed two main approaches: one utilizing cutting-edge LLMs such as Llama-3.1-8B, Phi-3.5-mini-instruct, and Mistral-7B-v0.1, and the other using BERT-based models like DistilBERT-multilingual, mBERT, and XLM-R-base. A key aspect of the LLM fine-tuning was the use of Parameter-Efficient Fine-Tuning (PEFT) with Low-Rank Adaptation (LoRA). This technique is vital for reducing computational demands, making it feasible to train large models even in environments with limited resources.
The results were particularly promising. The fine-tuned Llama-3.1-8B model emerged as the top performer among all evaluated models. It achieved an impressive overall accuracy of 95.5%, along with a precision of 93%, recall of 88%, and an F1 score of 90%. While it performed exceptionally well on positive sentiment classification (F1-score: 0.97), its performance on negative sentiment was slightly lower (F1-score: 0.83).
This superior performance demonstrates the significant potential of LLMs, especially when combined with efficient fine-tuning methods, to handle the intricacies of mixed-language sentiment analysis in contexts like Bangladesh. The study not only provides a robust solution for accurate sentiment prediction but also emphasizes the applicability of LLMs in advancing NLP for languages that traditionally have fewer digital resources.
Also Read:
- Wayfair Uses AI to Summarize Customer Reviews, Boosting Shopper Engagement
- AI Models Show Strong Potential for Automating Metaphor Identification in Texts
In conclusion, this research underscores the transformative power of LLMs in sentiment analysis for diverse linguistic environments. It offers a scalable and efficient approach that can be further refined with larger datasets and more nuanced sentiment categories in future studies, ultimately leading to better understanding of customer opinions in the e-commerce sector.
