TLDR: This study explores using Midjourney, a generative AI platform, to create 12,000 synthetic images for training deep neural networks (DNNs) to detect construction workers. The approach addresses data scarcity in the construction domain. The DNN model, trained primarily on these synthetic images, showed promising results on real construction datasets, demonstrating the potential of generative AI to supplement real-world data for AI training, despite some limitations like the need for manual labeling.
The construction industry is increasingly embracing digitization and automation, with robots playing a growing role in tasks like collecting site information and performing repetitive labor. A key challenge for these advancements, particularly for visual AI systems like Deep Neural Networks (DNNs) used in worker recognition, is the scarcity of diverse and voluminous training data. Unlike other fields with millions of images, construction-specific visual datasets are often limited, leading to issues like overfitting and bias in AI models.
To address this critical data shortage, a recent study explores a novel approach: leveraging generative AI, specifically the Midjourney platform, to create synthetic images for training DNNs. The core idea is that for effective DNN training, the origin of an image (real or synthetic) is less important than its visual characteristics and realistic context. This method significantly reduces the time and cost associated with collecting and labeling real-world construction site images, while also eliminating privacy concerns.
The researchers developed a comprehensive dataset of 12,000 synthetic images focusing on construction workers. This was achieved by formulating 3,000 unique text prompts, which were then fed into Midjourney. The prompts were carefully engineered to generate diverse scenarios, varying in worker numbers, distances, lighting conditions, and scene contexts. After generation, these images underwent a crucial step: manual labeling with 2D bounding boxes to identify worker instances. This process resulted in 11,992 usable images containing over 36,000 worker instances, which were then split into training, validation, and testing sets.
For the DNN model training, YOLOv7 was chosen due to its balance of speed and accuracy. The model was trained for 800 epochs on the newly created synthetic dataset. The evaluation of this model yielded promising results. When tested on a real construction image dataset, the model achieved an average precision (AP) of 0.937 at an Intersection over Union (IoU) threshold of 0.5, and 0.642 for IoU thresholds ranging from 0.5 to 0.95. These figures indicate a high degree of accuracy and robustness in detecting workers in real-world scenarios, even with the inherent complexities of construction sites.
Notably, the model demonstrated near-perfect performance when evaluated on the synthetic dataset itself, achieving an AP of 0.994 at an IoU threshold of 0.5 and 0.919 across the 0.5-0.95 range. This high performance on synthetic data suggests its effectiveness for DNN training. The study also highlighted that the model trained on Midjourney-generated images showed comparable, and in some metrics, superior performance to a previous model trained on traditional real-world datasets like SODA and MOCS.
While this innovative approach offers significant advantages, such as cost-effectiveness and overcoming data collection hurdles, it does have limitations. The current necessity for manual labeling of the synthetic images remains a time-consuming and labor-intensive process, introducing potential human error. Furthermore, despite their realism, synthetic images may not fully capture the dynamic and unpredictable nature of real construction sites, including subtle human behaviors and unique environmental changes. For more details on the methodology and results, you can refer to the full research paper.
Also Read:
- Building Specialized Image Datasets: The MNIST-Gen Approach
- DALI-PD: Accelerating ML in Chip Design with Synthetic Layout Heatmaps
Future work aims to integrate synthetic data more deeply into the training process, refine the model’s performance on real-world discrepancies, and explore ways to automate the labeling of generative AI outputs. The scalability and transferability of DNN models trained on synthetic images across various construction environments will also be a key area of investigation. This research underscores the significant potential of generative AI in expanding and diversifying training datasets for DNNs, paving the way for more robust and scalable AI applications in construction safety and automation.
