TLDR: This paper introduces the first AI/ML system for automatically assessing building damage from drone (sUAS) imagery that has been deployed in real-world federally declared disasters (Hurricanes Debby and Helene). Facing an overwhelming volume of drone imagery after disasters, the system was developed using the largest known dataset of post-disaster sUAS imagery. The best-performing model, an Attention UNet, was used to assess 415 buildings in approximately 18 minutes during the hurricane responses. The work highlights the development process, operator training, and crucial lessons learned from operational deployment, paving the way for future AI/ML applications in disaster response.
In the aftermath of major disasters, rapid and accurate damage assessment is crucial for effective response and recovery efforts. Traditionally, this involves teams collecting vast amounts of imagery from small uncrewed aerial systems (sUAS), or drones. However, the sheer volume of data—ranging from 47GB to 369GB per day—often overwhelms human experts, leading to significant delays in decision-making.
A groundbreaking AI/ML system has been developed and operationally deployed to address this challenge, marking the first time such a system for sUAS imagery has been used during federally declared disasters, specifically Hurricanes Debby and Helene. This innovation aims to alleviate the ‘data avalanche’ by automating the assessment of building damage, thereby accelerating the disaster response timeline.
The development of this AI/ML system followed a comprehensive four-phase cycle: data curation, model development, operator training, and deployment. Each phase contributed significantly to establishing a new state of practice for sUAS-based damage assessment.
Building the Foundation: Data and Models
The data curation phase was pivotal, leveraging the newly released CRASAR-U-DROIDs dataset. This dataset is the largest known collection of post-disaster sUAS imagery, containing 21,716 building damage labels from 10 different federally declared disasters. Buildings were meticulously labeled by 130 annotators using the Joint Damage Scale (JDS), a schema developed in consultation with federal agencies like FEMA, ensuring alignment with practical needs.
For model development, the problem of building damage assessment was framed as an image segmentation task. Eight baseline models were trained and evaluated on this extensive dataset. Among them, the Attention UNet model emerged as the most performant, achieving a macro F1 score of 0.511. This model was specifically chosen for its ability to generalize to unseen disasters, a critical factor for real-world applications.
Human Integration and Real-World Deployment
Recognizing that technology alone is not enough, the project included a crucial operator training phase. 91 disaster practitioners from 57 agencies were formally trained on the AI/ML system. Interestingly, practitioners showed less interest in raw model accuracy and more in how the system’s outputs could be seamlessly integrated into their existing workflows and data products. This feedback was vital for refining the system’s practical utility.
The culmination of this work was the operational deployment of the Attention UNet system during Hurricanes Debby and Helene. During these events, the system successfully assessed a combined 415 buildings in approximately 18 minutes. For Hurricane Debby, it processed 14.10 GB of imagery and assessed 222 buildings in Suwannee County, Florida. Following Hurricane Helene, it inferred 7.025 GB of imagery and assessed 193 buildings in Lafayette and Taylor counties, Florida. This deployment demonstrated the system’s capability to provide rapid assessments in real disaster scenarios.
Also Read:
- Unlocking Urban Insights: Federated Learning’s Role in Thermal Feature Detection
- Enhancing Disaster Assessment with Multimodal Data Augmentation
Lessons Learned and Future Path
The initial deployments highlighted several practical challenges. These included varying resolutions of sUAS imagery, misalignments between spatial data and imagery, limited wireless connectivity impacting data transmission, and the need for model outputs in formats compatible with existing decision-making tools like ArcGIS. These observations are crucial for future improvements.
Looking ahead, the path to future deployment involves establishing new performance metrics that align with operator concerns (such as advancing decision-making times), integrating ground-level damage assessments to ensure accuracy, and hardening the system for autonomous deployment without constant oversight from the research team. This work provides a valuable template for future efforts in deploying AI/ML systems for disaster response operations. You can read the full research paper here.
