TLDR: The paper introduces a three-phase generative AI system for industrial contour detection, achieving CAD-level precision. It uses data preprocessing, Pix2Pix for initial contour generation, and a Vision-Language Model (GPT-image-1) for refinement guided by natural language prompts and human feedback. This system significantly improves contour fidelity, reducing manual tracing and outperforming other VLMs like Gemini 2.0 Flash in accuracy for industrial applications.
Industrial manufacturing relies heavily on computer vision (CV) systems for tasks like inspection and maintenance. However, these systems often struggle with the unpredictable conditions of real factory floors, such as varying materials, lighting, and noise. This makes it difficult to accurately detect contours, which are crucial for tasks like reusing leftover materials and creating digital models of parts.
Traditional contour detection methods, including advanced deep learning models, often produce incomplete or unstable outlines when faced with reflective surfaces, irregular shapes, and low-contrast edges common in industrial settings. This leads to a bottleneck in manufacturing workflows, as manual tracing of these contours is slow, inconsistent, and expensive.
A New Approach with Generative AI
Researchers have developed a novel, language-guided generative vision system designed to overcome these challenges and achieve CAD-level precision in contour detection. This system focuses on digitizing “remnants” – leftover raw material sheets – so they can be efficiently reused, contributing to cost savings and sustainability.
The proposed system operates in three distinct phases:
1. Data Preprocessing: This initial phase involves collecting remnant images from real industrial sites, capturing a wide range of materials, shapes, and lighting conditions. These images are then standardized and augmented (e.g., rotated, flipped, brightness adjusted) to ensure consistent training and improve the system’s ability to generalize to new scenarios.
2. Contour Generation with Pix2Pix: In the second phase, a conditional Generative Adversarial Network (GAN) called Pix2Pix is used. This model is trained on a combination of synthetic and real-world images, allowing it to learn both the ideal structural regularities and the complexities of actual industrial imagery. The output of this phase is a set of binary contour masks that are coherent and denoised, but not yet precise enough for direct use in CAD systems.
3. VLM-based Contour Refinement: The final and critical phase involves refining these contours using Vision-Language Modeling (VLM). The Pix2Pix output masks are combined with standardized text instructions (multimodal prompts) and fed into an image generation model. This step is crucial for achieving the necessary CAD-level precision. A human-in-the-loop chatbot interface allows domain experts to provide natural language requests (e.g., “remove noise,” “make all holes uniform”), which are then translated into prompts to guide the refinement process iteratively.
Performance and Key Findings
The system was evaluated on proprietary datasets from FabTrack, reflecting authentic industrial variability. While the Pix2Pix model showed high visual similarity to ground truth, it had moderate geometric errors. The VLM-based refinement phase significantly improved this. Several VLMs were considered, but OpenAI’s GPT-image-1 consistently outperformed Google’s Gemini 2.0 Flash in both structural accuracy and perceptual quality. GPT-image-1 achieved higher structural similarity, lower visual distortion, and better geometric fidelity in contour reconstruction.
This research demonstrates that combining generative AI with language-guided refinement can advance industrial computer vision beyond the limitations of traditional methods. By producing sharper, more consistent, and accurately aligned contours, the system reduces the need for manual tracing and moves contour detection closer to the precision required for engineering and manufacturing applications. For more in-depth technical details, you can refer to the original research paper here.
Also Read:
- Boosting 3D CAD Part Retrieval with Reflective AI: A Training-Free Framework
- From Text to 3D Layout: An AI Workflow for Architects
Future Directions
Looking ahead, the researchers suggest expanding the dataset to include more industrial materials, incorporating active learning to allow the system to adapt continuously, and developing “agentic multimodal AI systems.” These future systems could autonomously manage complex vision tasks and dynamically adjust to diverse imaging conditions, further accelerating progress towards Industry 4.0 goals.
