TLDR: Researchers have introduced COde, a large-scale multimodal dataset for AI in dentistry. It comprises 8775 dental checkups from 4800 patients, including 50000 intraoral images, 8056 radiographs, and detailed textual records. This dataset addresses the shortage of comprehensive resources for training Vision-Language Models (VLMs) and Large Multimodal Models (LMMs) in oral healthcare. Fine-tuning state-of-the-art VLMs on COde demonstrated significant performance gains in classifying oro-dental anomalies and generating diagnostic reports, proving its effectiveness in advancing AI-driven dental solutions. The dataset is publicly available to foster future research.
The field of artificial intelligence (AI) is rapidly transforming healthcare, and dentistry is no exception. AI holds immense potential to improve diagnostic accuracy, streamline clinical processes, and enhance patient care. However, the full realization of AI’s capabilities in oral healthcare, particularly with advanced models like Vision-Language Models (VLMs) and Large Multimodal Models (LMMs), has been hampered by a critical shortage of comprehensive, publicly available datasets.
Traditional deep learning methods often require highly structured and extensively annotated data, limiting their adaptability to new or varied clinical scenarios. These models tend to struggle with generalization, meaning a model trained on specific dental conditions might fail when encountering previously unseen anomalies. In contrast, LMMs possess inherent generalization abilities, allowing them to adapt to new tasks and recognize categories not explicitly part of their initial training. They are also resilient to incomplete input data, a common occurrence in clinical practice, and can process diverse formats like images, X-rays, and textual notes without extensive preprocessing.
Existing multimodal datasets for dentistry have been limited in scope. Some focus solely on radiographs, while others combine radiographs with textual reports but lack the scale or essential diagnostic records needed for advanced VLM tasks. This gap highlighted a pressing need for a large, comprehensive multimodal dataset to propel generative AI in dentistry forward.
Introducing COde: A New Benchmark for Dental AI
To address these limitations, researchers have introduced COde (Casangels Oro-dental), a groundbreaking multimodal dataset designed to advance intelligent dentistry. COde is a comprehensive collection of 8775 dental checkups from 4800 patients, spanning eight years from 2018 to 2025. This rich dataset includes 50000 intraoral RGB images, 8056 radiographs (Panoramic X-rays, Periapical X-rays, and 2D Cone-Beam Computed Tomography slices), and detailed patient records. These textual records encompass diagnoses, treatment plans, follow-up notes, and medical histories, offering an unprecedented level of detail.
The data for COde was meticulously collected from routine patient visits at the Suzhou Dental Doctor Outpatient Clinic. Each check-up involved multiple intraoral photographs taken with a Canon D60 DSLR camera, corresponding radiographic images from standard dental X-ray units and a Sirona Galileos CBCT system, and comprehensive diagnostic reports written by attending dental doctors. All data was securely managed using a cloud-based dental practice management system, E-KanYa9, and collected under strict ethical guidelines with patient informed consent and anonymization.
Data Processing and Ethical Considerations
Before inclusion, the dataset underwent rigorous preprocessing. Duplicate or irrelevant images were removed, and incomplete records were excluded. Teeth numbers, originally in Palmer notation, were standardized to the FDI system. All images were converted to JPEG format and resized. Crucially, all original Chinese clinical reports were translated into English using GPT-4o, creating a bilingual dataset that enhances usability for a global research community while preserving the original text for fidelity.
Key fields recorded for each check-up include a unique Checkup ID, anonymized Patient ID, Age, Gender, filenames for Photographs and Radiographs, and detailed textual entries such as Patient Record, Chief Complaint, Present Illness, Past Medical Record, Examination findings, Radiographs Examination findings, Diagnosis, Treatment Plan, Management, Medical Instructions, and Remarks.
Patient privacy and ethical compliance were paramount throughout the data collection. All patients, or their legal guardians for minors, provided written informed consent. The study protocol was approved by a local institutional ethics committee, and all collected data were de-identified, with personal identifiers replaced by random numeric codes, ensuring compliance with health data privacy regulations.
Also Read:
- Advancing Cancer Diagnosis: A New End-to-End Approach for Whole Slide Image Analysis
- New AI Model JWTH Enhances Biomarker Detection by Fusing Global and Cellular Pathology Insights
Benchmarking AI Dental Assistants
To demonstrate COde’s utility, the dataset was annotated for two key benchmarking tasks: classification and generation. The classification benchmark evaluates a model’s ability to detect six common oro-dental anomalies (Caries, Gingivitis, Malocclusion, Pulpitis, Tooth Loss, or Tooth Structure Loss) from multimodal inputs including age, gender, chief complaint, radiographs, and photographs. The generation benchmark assesses a model’s capacity to produce detailed diagnostic reports, emulating a dental professional’s writing, from inputs like age, gender, chief complaint, past medical history, radiographs, and photographs.
For technical validation, state-of-the-art large vision-language models, Qwen-VL 3B and 7B, were fine-tuned on the COde dataset. Their performance was then compared against their base (non-fine-tuned) counterparts and GPT-4o, using both zero-shot and few-shot prompting strategies. The results were compelling: the fine-tuned models, particularly Qwen-7B, achieved substantial gains over the baselines. For classification, fine-tuned Qwen-7B reached an accuracy of 78.92% and an F1-score of 79.39%, significantly outperforming GPT-4o’s 55.83% accuracy among baselines. In the diagnostic report generation task, fine-tuned Qwen-7B delivered the best performance with an average cosine similarity score of 71.46%, indicating its outputs were remarkably close to dentist-written reports in content and style.
These results unequivocally validate the COde dataset’s effectiveness in enabling VLMs to acquire specialized knowledge for more accurate anomaly classification and human-like report generation. The dataset is publicly available on Hugging Face, providing an essential resource for future research in AI dentistry. Researchers can access the dataset in compressed ZIP format at https://huggingface.co/datasets/zirak-ai/COde. It is provided in both CSV and JSON formats, with a ready-to-use version in ShareGPT format for seamless integration into machine learning pipelines.
The introduction of COde marks a significant step forward for AI in oral healthcare, offering a robust foundation for developing more intelligent, accurate, and adaptable AI dental assistants that can ultimately enhance patient care globally.
