TLDR: LOLGORITHM is a novel AI system that generates diverse and platform-appropriate comments for short videos. Developed by Xuan Ouyang et al., this modular multi-agent system uses a multimodal large language model to produce comments in six distinct styles, including puns, rhyming, and memes. Evaluated on a bilingual dataset from Douyin and YouTube, LOLGORITHM significantly outperforms baseline models, achieving over 90% user preference on Douyin and 87.55% on YouTube, demonstrating its ability to create highly engaging and contextually relevant content.
Short video platforms have become a central part of the internet, transforming how we share information and interact. A key element of this engagement is the comment section, where users express themselves, foster community, and even drive content trends. However, creating comments that are not only relevant but also stylistically diverse and platform-appropriate has been a significant challenge for artificial intelligence.
A new research paper, titled “Laugh, Relate, Engage: Stylized Comment Generation for Short Videos,” introduces an innovative solution to this problem. The paper, authored by Xuan Ouyang, Senan Wang, Bozhou Wang, Siyuan Xiahou, Jinrong Zhou, and Yuekang Li, presents a system called LOLGORITHM.
LOLGORITHM is a modular multi-agent system designed specifically for generating controllable and stylized comments for short videos. Unlike previous attempts that focused on video summarization or live-streaming comments, LOLGORITHM aims to capture the rich and diverse stylistic forms found in short video comments, such as memes, irony, rhyming, and puns.
The system supports six distinct comment styles: puns (homophones), rhyming, meme application, sarcasm (irony), plain humor, and content extraction. This fine-grained style control is achieved through explicit prompt markers and few-shot examples, powered by a multimodal large language model (MLLM) that directly processes video inputs.
The development of LOLGORITHM involved creating a unique bilingual dataset from popular platforms, Douyin (Chinese) and YouTube (English), using their official APIs. This dataset covers five popular video genres: comedy skits, daily life jokes, funny animal videos, humorous commentary, and talk shows. This cross-cultural and cross-platform approach ensures that LOLGORITHM can adapt to different linguistic nuances and community norms.
The comment generation process within LOLGORITHM is structured into three main modules: video content extraction, video content classification, and the actual comment generation. The system first processes raw video into structured text descriptions, identifying highlights, extracting frames, and transcribing audio. This detailed description is then used to classify the video into one of the predefined categories. Finally, the comment generation module selects a style template based on the video’s category and uses a large language model to generate an original comment that matches the template’s structure, tone, and rhythm, while being semantically relevant to the video content.
To evaluate LOLGORITHM’s effectiveness, the researchers employed a dual approach: automated metrics and a large-scale human preference study. The automated scoring framework assessed comments based on originality, relevance, and style conformity. LOLGORITHM consistently achieved leading scores on both Douyin and YouTube, demonstrating its ability to balance these crucial dimensions.
The human preference study was even more compelling. Involving 40 videos and 105 participants, the study asked respondents to choose the comment they were most likely to “like” or engage with from a selection generated by LOLGORITHM and three baseline models (V2Xum-LLM, LiveChat, and GPT-4o Direct). The results showed an overwhelming preference for LOLGORITHM-generated comments, with selection rates exceeding 90% on Douyin and reaching 87.55% on YouTube. This significantly outperformed all baseline models, which often struggled with length, formality, or relevance.
Also Read:
- Agent-Omni: Enabling Flexible AI Reasoning Across Text, Image, Audio, and Video
- AI-Powered System Generates Groundbreaking Research Ideas
This work marks a significant step forward in stylized comment generation for short video platforms. LOLGORITHM introduces a scalable and culturally adaptive framework that promises to enhance user engagement and interaction. The researchers have also made their code and raw experimental data publicly available to facilitate further research. You can find more details about this groundbreaking work in the full research paper available here.
