TLDR: A research paper compares federated learning strategies for video violence detection, finding that lightweight 3D CNNs are more energy-efficient and offer better calibration for binary detection, while Vision-Language Models (VLMs) provide richer reasoning, especially with semantic grouping for multiclass tasks. The study proposes a hybrid deployment model where CNNs handle routine screening and VLMs are used for complex events, significantly reducing energy consumption and carbon footprint while maintaining high accuracy and privacy.
In an era where video surveillance is increasingly common for public safety, the deployment of deep learning systems raises significant concerns about privacy and environmental impact. Traditional centralized systems often require transferring sensitive video footage off-device, posing privacy risks and incurring substantial computational and energy costs. A recent research paper explores how to address these challenges by comparing different federated learning strategies for video violence detection, focusing on energy efficiency and privacy preservation.
Federated learning (FL) offers a promising solution by allowing AI models to be trained on local devices without the need to centralize raw data. This approach keeps sensitive information on-device, enhancing privacy. However, the integration of large Vision-Language Models (VLMs), which are powerful but computationally intensive, into federated learning environments introduces new energy and sustainability hurdles.
The paper, titled Federated Learning for Video Violence Detection: Complementary Roles of Lightweight CNNs and Vision-Language Models for Energy-Efficient Use, investigates three distinct strategies for federated violence detection under realistic conditions. These strategies include using pre-trained VLMs for zero-shot inference, fine-tuning VLMs like LLaVA-NeXT-Video-7B with a technique called LoRA (Low-Rank Adaptation), and employing personalized federated learning with a lightweight 3D Convolutional Neural Network (CNN).
The researchers evaluated these methods on standard datasets (RWF-2000 and RLVS) for binary violence detection and on the UCF-Crime dataset for multiclass anomaly recognition. A key finding for binary classification was that all methods achieved over 90% accuracy. However, the lightweight 3D CNN demonstrated superior calibration and significantly lower energy consumption, using roughly half the energy (240 Wh) compared to federated LoRA fine-tuning (570 Wh). While CNNs proved more energy-efficient for routine tasks, VLMs offered richer multimodal reasoning capabilities, which are crucial for understanding complex scenarios.
For multiclass classification, VLMs were specifically assessed. The study found that by grouping semantically similar crime categories (e.g., combining ‘Arson’ and ‘Explosion’ into ‘Destruction’), the VLM’s accuracy improved substantially from 65.31% to 81% on the UCF-Crime dataset. This hierarchical grouping helped the models overcome difficulties in distinguishing between very similar events, such as different types of property crimes or interpersonal violence.
The research also included a systematic quantification of energy use and CO2e emissions, highlighting the environmental impact of different AI architectures. The results suggest a hybrid deployment strategy: efficient 3D CNNs could be used for continuous, low-cost screening on edge devices, while VLMs would be selectively engaged for more complex contextual reasoning or when detailed natural language descriptions of incidents are required. This tiered approach aims to balance accuracy, privacy, and sustainability, reducing the overall energy footprint without compromising situational awareness.
Also Read:
- Charting a Sustainable Path for Artificial Intelligence: A Deep Dive into Green AI Practices
- OvA-LP: A Minimalist Framework for Robust Federated Fine-Tuning on Diverse Data
This comparative study provides valuable insights for developing sustainable and privacy-preserving video surveillance systems, guiding the selection and adaptation of AI models to optimize both performance and environmental responsibility.
