TLDR: Swin-TUNA is a new Parameter Efficient Fine-Tuning (PEFT) method for food image segmentation. It integrates multi-scale trainable adapters into the Swin Transformer, updating only 4% of parameters (8.13M) while outperforming larger models like FoodSAM (632.75M) in accuracy. Swin-TUNA achieves mIoU of 50.56% on FoodSeg103 and 74.94% on UECFoodPix Complete, demonstrating faster convergence and better generalization in low-data scenarios, offering a lightweight solution for industrial food processing applications.
In the rapidly evolving world of digital and intelligent technologies, processing food images has become a significant area of research. Techniques like image classification, object detection, and especially semantic segmentation are vital for various industrial applications, such as quality control on assembly lines, dietary assessment, and food composition analysis.
However, current large-scale AI models, particularly those based on Transformer architectures like FoodSAM, face considerable hurdles. These models often have an enormous number of parameters and demand extensive computational resources, making them challenging to deploy in practical, real-world scenarios. For instance, FoodSAM, while achieving good accuracy, requires a massive 632.75 million parameters, which translates to significant computing power and time for training.
Addressing these challenges, researchers have introduced a novel approach called TUNable Adapter module, or Swin-TUNA. This method falls under the category of Parameter Efficient Fine-Tuning (PEFT), which aims to adapt large pre-trained models for specific tasks by updating only a small fraction of their parameters. Swin-TUNA integrates multi-scale trainable adapters into the Swin Transformer architecture, specifically designed for high-performance food image segmentation.
The core innovation of Swin-TUNA lies in its hierarchical feature adaptation mechanism. It uses separable convolutions and dimensional mappings that vary in scale to better handle the differences in features found in shallow (high-frequency details) and deep (global semantic information) parts of the network. This is combined with a dynamic strategy to balance general features learned during pre-training with task-specific features needed for food segmentation.
The results are quite impressive. Swin-TUNA achieves a mean Intersection over Union (mIoU) of 50.56% on the FoodSeg103 dataset and 74.94% on the UECFoodPix Complete dataset. What’s remarkable is that it surpasses the fully parameterized FoodSAM model in performance while reducing the parameter count by a staggering 98.7%, bringing it down to only 8.13 million trainable parameters. This makes Swin-TUNA a significantly more efficient solution.
Beyond its efficiency, Swin-TUNA also demonstrates faster convergence during training and stronger generalization capabilities, especially in scenarios where only a limited amount of data is available. This is crucial for food datasets, which can often be small. The method effectively retains the vast knowledge from the pre-trained backbone network while only updating the newly injected, task-specific components.
The design of Swin-TUNA involves a plug-and-play module that is inserted into the Swin Transformer Block. This module consists of a downsampling component, a depth-wise convolution (as a non-linear function), and an upsampling component. Unlike other adapter methods that might use a uniform structure, Swin-TUNA’s hierarchical adaptation means that the convolution kernels and mapping dimensions vary across different layers of the network, allowing it to better integrate both high-frequency texture details and global semantic information.
Experiments have shown that Swin-TUNA consistently outperforms other PEFT methods in terms of both accuracy and convergence speed. Ablation studies confirmed that its parallel structure and the hierarchical feature adaptation are key contributors to its superior performance. This work marks a significant step, being the first time a PEFT method has achieved state-of-the-art results for food image segmentation while using such a small number of adjustable parameters.
Also Read:
- Measuring Dining Hall Food Waste Using AI Imaging
- Gaze-Guided Robots: Enhancing Efficiency and Robustness with Human-Inspired Vision
For more technical details, the research paper can be accessed here. The code for Swin-TUNA is also publicly available, enabling further research and application in the field.
