TLDR: PHORECAST is a novel multimodal dataset enabling AI models to predict diverse human responses to public health campaigns. It includes over 30,000 responses from 1,000+ U.S. participants, capturing demographics, personality traits, and reactions to 37 health posters across 7 topics. Training with PHORECAST significantly improves VLM accuracy in emulating human language and predicting opinions, highlighting the critical roles of education, personality, and visual stimuli. The research paves the way for AI-driven personalized health communication strategies, though it notes limitations in global applicability and temporal dynamics.
A new research paper introduces PHORECAST, a groundbreaking multimodal dataset designed to help artificial intelligence (AI) better understand how diverse individuals and communities react to public health messages. This dataset aims to bridge the gap between advanced AI models and the nuanced, varied human responses crucial for effective public health outreach.
The creators of PHORECAST highlight that while large vision and language models (VLMs) show promise, their ability to accurately simulate heterogeneous human responses, especially in critical areas like public health, has been limited by a lack of comprehensive, multimodal data. PHORECAST, which stands for Public Health Outreach Receptivity and Campaign Signal Tracking, addresses this by providing a rich dataset for predicting both individual behavioral responses and community-wide engagement patterns to health messaging.
Building the PHORECAST Dataset
The dataset was compiled from a large-scale study involving over 1,000 participants, yielding more than 30,000 detailed responses to 37 public health posters. These posters covered seven urgent health topics, including COPD, Mental Health, Nutrition, Vaccination, Substance Abuse, Sexual Practices, and Chronic Diseases. Each response captures sentiment, emotional reactions, and behavioral intent, offering deep insights into how message design, individual differences, and community engagement interact.
Participants in the study, all U.S. residents aged 18 or older, completed a 30-minute anonymized survey. This survey collected extensive information, including:
- Demographics: Age, gender, religious/cultural affiliation, political affiliation, race, ethnicity, language, educational attainment, employment status, profession, income, marital status, family status, health conditions, and zip code.
- Personality Traits: Measured using The Big Five Inventory-2 (BFI-2), which assesses Extraversion, Agreeableness, Conscientiousness, Negative Emotionality, and Open-Mindedness, along with 15 specific facets.
- Locus of Control: This measures an individual’s belief about whether they control events in their life (internal locus) or if external factors do (external locus).
- Baseline Opinions: Participants rated their pre-existing concern, motivation, harm perception, and openness on the five randomly assigned health topics before viewing any campaigns.
- Reactions to Campaigns: After viewing specific public health campaigns, participants provided emotional responses (sadness, anger, fear, guilt, disgust, worry, shame, hope) on a Likert scale, re-evaluated their concern, motivation, harm perception, and openness in the context of the campaign, and offered free-form text responses detailing their thoughts.
Training AI Models with PHORECAST
The researchers used PHORECAST to fine-tune state-of-the-art VLMs, specifically Llama 3.2-Vision 11B and Gemma 3 12B. The goal was to train these models to simulate human responses to public health messages based on an individual’s demographic and psychological profile. A key aspect of the training involved a feature randomization protocol, where models were forced to operate with partial user data, mimicking real-world scenarios where complete profiles might not be available.
Key Findings and Model Performance
The results demonstrated significant improvements in VLM performance after training with PHORECAST. Llama 11B, for instance, showed a remarkable 2.4 times increase in exact-match accuracy for predicting opinion indicators. Both models, when trained, learned to emulate real human language more effectively, capturing the subtle distinctions in human responses that were often missed by untrained models.
The study also provided insights into the importance of different factors:
- Demographics: Education level was identified as the most critical demographic factor for accurate prediction of responses.
- Personality: Providing personality information, especially the combination of personality facets and Locus of Control, led to higher prediction accuracy than using demographics alone.
- In-Context Responses: The individual’s immediate reactions to the stimuli (e.g., harm perception, concern) were found to be highly predictive of their overall opinion.
- Visual Stimuli: Images played a critical role in achieving exact-match accuracy, particularly for motivation and concern. However, in scenarios with minimal input, textual signals from stable traits (personality, demographics, Locus of Control) became more dominant for nuanced opinion prediction.
Also Read:
- Unlocking Personality: A New Framework for Text-Based Assessment from Social Media
- Large Vision-Language Models Enhance Carotid Plaque Risk Stratification
Generalization and Future Applications
The trained models showed strong generalization capabilities, performing well on unseen subpopulations and even on health messages employing novel communication strategies (such as self-efficacy, informational/educational, or threatening/fear-driven approaches). This suggests that PHORECAST can help AI systems adapt to new public health campaigns effectively.
Looking ahead, the researchers envision a practical use case for PHORECAST: a VLM-enabled system for recommending health communication strategies. This system could tailor public health messages to specific personality and demographic groups, maximizing their effectiveness. By aggregating predicted responses across individuals within a group, the AI could identify the most impactful messaging approach for a given health topic.
While the dataset currently focuses on English-speaking, U.S.-based participants, the researchers acknowledge the need for future work to broaden the participant base to include diverse cultural, linguistic, and geopolitical factors. This will further enhance the global applicability of AI models in public health. You can read the full research paper here.
