TLDR: CADmium is a new method that uses GPT-4.1 to create a large dataset of human-like text descriptions for CAD models. It then fine-tunes a code-focused LLM (Qwen 2.5-Coder) to generate CAD designs in JSON format directly from these text descriptions. The research also introduces new metrics for evaluating 3D object quality, demonstrating that this text-to-text approach effectively automates and speeds up CAD design.
Computer-aided design, or CAD, is fundamental to creating 2D and 3D objects across various engineering and manufacturing fields, from cars to airplanes. Despite its widespread use, CAD modeling often remains a labor-intensive and manual process. While some efforts have been made to automate this using smaller AI models, the full potential of large language models (LLMs) for sequential CAD design has largely been unexplored.
A new research paper introduces “CADmium,” a novel approach that aims to significantly automate and speed up CAD design. The core idea is to transform CAD generation into a purely text-to-text task, making it more accessible and efficient.
A New Approach to CAD Design
The researchers behind CADmium identified two main challenges in text-conditioned CAD object generation: the scarcity of high-quality, human-like textual descriptions for CAD models, and the lack of suitable ways to represent CAD design histories for effective use with pre-trained language models. Existing machine-generated annotations often struggle to balance natural language fluency with the precise geometric details needed to define a 3D object unambiguously.
To overcome these hurdles, CADmium introduces a new large-scale dataset comprising over 170,000 CAD models. What makes this dataset unique are its high-quality, human-like descriptions. These descriptions were generated using a sophisticated pipeline based on GPT-4.1, a powerful multimodal AI model. This pipeline processes multi-view images of 3D objects along with their construction sequences to create annotations that are both natural-sounding and geometrically precise.
Leveraging Code-Focused AI Models
With this new dataset, the CADmium team fine-tuned Qwen 2.5-Coder-14B, a state-of-the-art instruction-tuned code LLM. The goal was to enable this model to generate CAD sequences in a JSON-based format directly from natural language descriptions. This approach leverages the inherent capabilities of pre-trained code models, eliminating the need for specialized embedding layers that often require significant computational resources.
The research demonstrates that fine-tuning LLMs can be highly effective for generating code used in visual content creation, extending their versatility to applications like CAD. The CADmium pipeline effectively reformulates the entire CAD generation process as a simple text-to-text translation.
Enhanced Evaluation Metrics
One of the significant contributions of CADmium is the introduction of new metrics for evaluating the quality of generated CAD models. Traditional metrics often fall short in reflecting the true quality of complex 3D objects, especially regarding their internal structure. CADmium introduces geometric and topological metrics based on sphericity, mean curvature, and Euler characteristic, along with watertightness. These provide richer structural insights, allowing for a more comprehensive assessment of the generated designs.
Also Read:
- M2-CODER: Advancing AI Code Generation with Visual Design Understanding
- AI Agents Reshaping Conceptual Engineering Design with Structured Language Models
Promising Results and Future Outlook
Experiments conducted on both synthetic and human-annotated data show that CADmium can automate CAD design, drastically accelerating the creation of new objects. The fine-tuned LLM achieved competitive performance against existing state-of-the-art models like Text2CAD, with improvements observed across several evaluation metrics as the model size increased. The quality of CADmium’s expert-level text annotations was also highlighted, being more human-like, concise, and diverse compared to previous methods.
While the initial results are highly promising, the researchers acknowledge limitations, such as the need for further investigation into the generalization of GPT-generated prompts to a wider variety of human prompts. Future work aims to extend the approach with a multi-modal framework to facilitate editing CAD designs. The dataset, code, and fine-tuned models are openly available online, paving the way for further advancements in this field. You can find the full research paper here: CADmium: Fine-Tuning Code Language Models for Text-Driven Sequential CAD Design.
