TLDR: A new framework called TM-Rephrase uses large language models (LLMs) to rephrase informal social media texts into more standardized language before topic modeling. This significantly improves the quality, coherence, uniqueness, and diversity of extracted topics, especially for COVID-19 related tweets. The ‘colloquial-to-formal’ rephrasing strategy showed the most substantial gains, making topics more interpretable and valuable for public discourse analysis, particularly in health communications.
A new study by Wangjiaxuan Xin, Shuhua Yin, Shi Chen, and Yaorong Ge from the University of North Carolina at Charlotte introduces a novel framework called TM-Rephrase, designed to significantly improve how we understand public conversations on social media, especially during critical times like the COVID-19 pandemic. The research, titled Improving Topic Modeling of Social Media Short Texts with Rephrasing: A Case Study of COVID-19 Related Tweets, addresses the long-standing challenge of analyzing short, informal, and often noisy social media texts.
Social media platforms like Twitter (now X) are rich sources of public opinion, but their characteristic brevity, use of slang, and abbreviations make it difficult for traditional topic modeling methods to extract clear and meaningful themes. This often results in topics that are hard to interpret, redundant, or lack coherence.
The TM-Rephrase Solution
To overcome these hurdles, the researchers developed TM-Rephrase, a framework that uses large language models (LLMs) to rephrase raw social media posts into more standardized and formal language before any topic modeling is performed. This approach doesn’t change the topic modeling algorithms themselves; instead, it enhances the quality of the input data, making it much easier for these algorithms to find coherent and distinct topics.
The study explored two main rephrasing strategies: ‘general rephrasing’ and ‘colloquial-to-formal rephrasing’. General rephrasing focuses on basic grammar correction and clarity, while colloquial-to-formal rephrasing aims to transform informal expressions into professional statements, similar to what you might find in an official report.
A Case Study with COVID-19 Tweets
The team applied TM-Rephrase to a dataset of over 25,000 COVID-19-related tweets that were replies to official CDC accounts between May 2020 and November 2022. They then evaluated the impact of rephrasing on several topic modeling methods, including classic ones like Latent Dirichlet Allocation (LDA) and more modern deep learning approaches such as BERTopic and FASTopic.
Key Findings and Improvements
The results were compelling. TM-Rephrase consistently improved three key metrics for topic modeling performance: topic coherence (how semantically related words in a topic are), topic uniqueness (how distinct topics are from each other), and topic diversity (the range of unique words across all topics). It also reduced topic redundancy, meaning fewer overlapping themes.
Notably, the ‘colloquial-to-formal’ rephrasing strategy often yielded the greatest performance gains, especially for the LDA algorithm. For instance, LDA with colloquial-to-formal rephrasing achieved a significantly higher coherence score compared to using original, un-rephrased tweets. This suggests that making the text more formal and structured provides a substantial boost to models that are sensitive to word-level noise.
Beyond the numbers, a qualitative assessment revealed that topics derived from rephrased tweets were much more interpretable and aligned with human understanding. For example, a topic on public health measures that was initially a mix of informal language and general actions, transformed into a highly coherent topic focused on public health policy with professional terminology after colloquial-to-formal rephrasing.
Also Read:
- Mapping Scientific Trends: An LLM Approach to Engineering Research in PNAS
- Unpacking Research Funding: How Location and Gender Shape Scientific Focus in Canada
Implications for Public Health and Beyond
This research has significant practical implications, particularly for public health agencies. By providing clearer, more precise insights into public concerns (such as vaccine side effects or policy impacts), TM-Rephrase can help agencies design more effective communication strategies. The framework is also model-agnostic, meaning it can be applied to various topic modeling algorithms, making it a versatile tool.
The study highlights that while general rephrasing offers a balanced improvement, the colloquial-to-formal scheme acts as a more potent tool for creating highly distinct and semantically pure topics. However, researchers must consider a trade-off: while formal rephrasing improves clarity, it might sometimes reduce the emotional or stylistic nuance of the original text. The choice of rephrasing strategy should align with the specific goals of the analysis.
Looking ahead, the researchers suggest exploring the framework’s applicability in other domains beyond public health, comparing different LLMs, and investigating how rephrasing interacts with various model settings. This work represents a significant step towards making social media analysis more reliable and interpretable, offering valuable insights for policymakers, researchers, and practitioners alike.
