TLDR: AI Blob! is an experimental system that uses AI, including Large Language Models and semantic embeddings, to recontextualize Italian television archives. Inspired by the satirical Italian TV show ‘Blob’, it processes video audio, extracts and scores sentences for irony and relevance, and then algorithmically arranges them into new, thematically coherent, and often ironic video montages, offering a novel way to engage with historical media.
A new experimental system called AI Blob! is transforming how we interact with vast archives of television footage. Drawing inspiration from the iconic Italian television program ‘Blob’, which is known for its satirical montages, AI Blob! uses advanced artificial intelligence to recontextualize and reinterpret old TV content, creating new narrative sequences.
What is AI Blob! and How Does It Work?
At its core, AI Blob! is designed to move beyond traditional ways of cataloging and retrieving archival material. Instead of relying on static descriptions, it uses semantic understanding to find and combine video clips in meaningful and often ironic ways. The system processes a curated collection of 1,547 Italian television videos. First, it transcribes the audio from these videos into text, breaking it down into individual sentences. These sentences are then converted into “embeddings,” which are numerical representations that capture their meaning, and stored in a special database called a vector database.
When a user provides a theme or prompt, a Large Language Model (LLM) takes over. It generates a variety of related queries, which are then used to search the vector database for relevant sentences. The system intelligently selects unique sentences, avoiding repetition. These selected sentences are then evaluated by the LLM for their potential to create ironic or thematically relevant connections when taken out of their original context. Each sentence is given an “irony score” and a “thematic relevance score” from 1 to 10.
Crafting New Narratives
The magic happens in how AI Blob! constructs new narratives. Based on the irony and thematic relevance scores, the system algorithmically segments the chosen sentences into distinct narrative sections: an introduction, a build-up, a climax, and a conclusion. For example, sentences with lower irony but high thematic relevance might form the introduction, while those with the highest irony scores are reserved for the climax. Within each segment, the LLM strategically orders the sentences to maximize ironic and semantic contrasts, creating a compelling and often humorous effect, much like the original ‘Blob’ program.
Finally, the system retrieves the actual video clips corresponding to the selected sentences using their original timestamps. It then stitches these clips together, adding smooth audio transitions and an introductory sequence, to produce a cohesive and recontextualized audiovisual montage. This process demonstrates how AI can facilitate new approaches to engaging with archives, enabling automated storytelling and cultural analysis.
Behind the Scenes: Data and Technology
The dataset for AI Blob! was compiled from two main sources: the ITTV dataset and the Indimenticabile TV YouTube channel. The audio transcription was performed using WhisperX, a highly efficient speech recognition tool. For processing and embedding the sentences, the system utilized advanced models like Punkt Sentence Tokenizer, xlm-roberta_punctuation_fullstop_truecase, and Cohere’s Embed Multilingual V3. The embeddings are stored in ChromaDB, a database optimized for quick semantic searches.
Also Read:
- Automating Insights from Oral Histories with Language Models
- LoSemB: Enhancing AI Tool Discovery with Logic-Guided Retrieval
Looking Ahead
While AI Blob! shows great promise, the researchers acknowledge some limitations. These include occasional inaccuracies in speech recognition, challenges in consistently achieving perfect ironic contrast, and the current inability to fully analyze visual elements, which are crucial for television’s multimodal language. The relatively small dataset size also limits the depth of satirical montages. Future work aims to address these by incorporating multimodal embeddings to better integrate visual content and by significantly expanding the dataset.
AI Blob! represents a significant step in using AI for archival research and creative reuse. It offers a new model for computational creativity, respecting the legacy of television while embracing the potential of artificial intelligence. You can learn more about this project by reading the full research paper: AI Blob! LLM-Driven Recontextualization of Italian Television Archives.
