TLDR: A new research paper introduces the first multimodal benchmark dataset for analyzing how oil and gas companies frame their advertising in videos, aiming to detect greenwashing. The dataset, comprising expert-annotated Facebook and YouTube ads, evaluates vision-language models (VLMs) on their ability to identify specific framing types related to climate obstruction and corporate impressions. Findings show promising VLM performance but highlight challenges in handling video length, cultural contexts, and subtle framing, offering a crucial tool for systematic greenwashing detection and social science research.
In an era where corporate image is meticulously crafted, a new research paper introduces a groundbreaking tool to scrutinize the public relations campaigns of oil and gas (O&G) companies. Titled “A Multimodal Benchmark for Framing of Oil & Gas Advertising and Potential Greenwashing Detection,” this study by Gaku Morio, Harri Rowlands, Dominik Stammbach, Christopher D. Manning, and Peter Henderson, delves into how these companies frame their messaging, particularly concerning environmental initiatives, and aims to detect potential greenwashing.
The core of this research is a novel benchmark dataset, the first of its kind, specifically designed for analyzing framing in O&G video advertisements. Unlike previous datasets that focused solely on text, this new dataset incorporates both visual and linguistic elements from videos, recognizing that imagery plays a crucial role in conveying messages, especially when no spoken language is present (which accounts for about 30% of videos in their dataset).
Understanding Framing and Greenwashing
Framing is a powerful communication technique that selects certain aspects of reality to promote a specific interpretation or evaluation. O&G companies have long used careful messaging to shape public perception. Often, these messages emphasize contributions to reducing emissions or highlight their essential role in maintaining critical infrastructure, even while relying heavily on fossil fuels. Such communications are frequently labeled as greenwashing – a practice where a company overstates its environmental contributions or distracts from genuine climate action.
The researchers explain that accurately interpreting framing at scale can significantly aid social science and policymaking, particularly in systematically understanding and monitoring greenwashing. The new dataset addresses a critical gap by providing a multimodal approach, acknowledging that video imagery can contain strategic and misleading framings.
The Benchmark Dataset: Two Domains
The dataset comprises expert-annotated video ads from two distinct domains: Facebook and YouTube. This dual-domain approach allows for robust evaluation of vision-language models (VLMs) across different contexts.
- Facebook Domain: This subset includes short video ads from O&G companies and advocacy groups, primarily used in political campaigns in the United States. These videos are annotated for seven greenwashing-related framing types, often associated with “climate obstruction framing.” Examples include emphasizing economic benefits (jobs, tax revenue), promoting “clean” gas, or stressing the necessity of fossil fuels for power systems and manufactured goods.
- YouTube Domain: This larger subset features marketing videos from 42 O&G-related companies across 20 countries. These videos are generally longer and aim to convey specific “impressions” to viewers. Six framing types are annotated here, such as ‘Community and Life’ (contributions to daily life, culture, charity), ‘Economy and Business’ (economic prosperity, tax revenue), ‘Work’ (job creation, reliable workplaces), ‘Environment’ (reducing emissions, supporting renewables), ‘Green Innovation’ (visions for a green future, new energy tech), and ‘Patriotism’ (national pride, energy independence).
The dataset is substantial, featuring 706 videos, over 35,000 seconds of footage, and more than 1,100 expert annotations. It covers more than 50 entities across 20 countries, with videos published from 2010 to 2025, offering rich diversity in cultural context and evolving framing strategies.
Benchmarking Vision-Language Models
The study benchmarks the performance of six cutting-edge VLMs, including GPT-4.1, GPT-4o-mini, Qwen2.5-VL, InternVL2, and DeepSeek-VL2. The task is a multi-label classification, where models predict a set of framing types for each video. Experiments were conducted in both zero-shot (no training examples) and one-shot (one training example provided) settings.
Results show promising capabilities, with GPT-4.1 performing best in the YouTube domain. The one-shot setting generally improved performance for open-weight models, highlighting the importance of providing relevant examples. However, challenges remain, particularly with low-resource labels like ‘Patriotism’ and ‘Green Innovation’, which often rely on more subjective and subtle interpretations.
The research also identified several challenges for VLMs, such as handling videos of varying lengths (shorter videos proved more challenging), different cultural backgrounds (models showed varying performance across regions), and the tendency of models to over- or under-label certain categories. The importance of both visual and transcript information was also highlighted, with transcript input significantly impacting performance in some cases.
Also Read:
- ADL Report: Generative AI Video Tools Struggle to Block Antisemitic Content
- UCLA Engineers Pioneer Light-Based System for Energy-Efficient Generative AI
Towards Multimodal Greenwashing Detection
This benchmark is a significant step towards automated greenwashing detection. The framing types identified in both Facebook and YouTube videos directly relate to known forms of greenwashing, such as selective disclosure or vague environmental claims. By analyzing these framings, the dataset allows researchers to evaluate how accurately VLMs can detect such deceptive practices.
Pilot studies conducted by the authors demonstrate the practical potential of their work. A temporal trend analysis revealed that the model could capture the increasing trend of ‘Environment’ labels in O&G videos after 2020, which could signal industry-wide greenwashing. Company-level analysis also showed that the model could replicate gold label distributions, identifying companies with a high proportion of ‘Environment’ labels or those emphasizing ‘Community and Life’ alongside vague environmental content.
This research offers a powerful computational tool to assist in understanding corporate messaging, profiling company strategies, and conducting social science research on energy and climate communication. The dataset and code are openly available, encouraging further exploration and development in this critical area. You can find more details about this research paper here: A Multimodal Benchmark for Framing of Oil & Gas Advertising and Potential Greenwashing Detection.
