TLDR: A new research paper introduces ThermoCycleNet, a deep neural network designed to adapt thermal imaging analysis from treadmill running to ergometer cycling. The study demonstrates that combining a large volume of automatically generated thermal labels with a small amount of high-quality manual labels is highly effective for training the network. This approach allows for a rapid transition of AI models to new use cases in sports medicine, enabling accurate segmentation of body parts like calves during different physical exercises.
Infrared thermography (IRT) is becoming an increasingly valuable tool in sports medicine. It allows researchers and practitioners to assess thermal radiation during physical exercise and analyze specific anatomical regions, such as the calves, which are well-exposed during activities. This technology helps in understanding how the body responds thermally during different types of exercise.
Building on previous advancements in automatic annotation methods, a new study aimed to transfer a stereo- and multimodal-based labeling approach from treadmill running to ergometer cycling. The challenge lies in adapting a specialized deep neural network, initially trained for one use case, to a new one without starting from scratch.
The researchers developed a system called StereoThermoLegs, which previously generated labels for treadmill running using a combination of infrared, time-of-flight, and visual cameras. These labels, created automatically in the visual domain, were then transformed into the IRT domain using stereo properties and depth information. The goal of the new study was to adapt this advanced stereo label generation method for ergometer cycling and simultaneously develop a new model, ThermoCycleNet, capable of accurately segmenting moving legs in both running and cycling scenarios.
The core hypothesis was that automatically generated stereo labels would enable a rapid transition of the deep neural network from running to cycling, and that only a small amount of manually labeled data would be sufficient for fine-tuning the network to achieve high performance.
To test this, the study utilized the existing StereoThermoLegs dataset in conjunction with a new manual dataset. They analyzed five different models: Model M, trained exclusively with manual data, and models S1-S4, which underwent stereo pre-training followed by fine-tuning with varying fractions of the manual dataset (0%, 10%, 50%, and 100%). A new manual dataset was also created specifically for cycling. The new ThermoCycleNet was then trained on all stereo-labeled data and fine-tuned using the manual datasets.
The results were insightful. While Model M, trained solely on manual data, performed adequately, models that incorporated stereo pre-training showed significant improvements, especially when fine-tuned with even a small fraction of manual data. Specifically, models S3 and S4, which used 50% and 100% manual data for fine-tuning respectively, outperformed the baseline Model M in overall and target class performance. ThermoCycleNet, when applied to the new stereo-labeled cycling dataset (comprising 1600 thermograms and 560 manually annotated training images), achieved a mean Intersection over Union (IoU) of 0.51, with strong performance on target classes like the left and right calf (0.75 and 0.72 IoU respectively).
The study concluded that incorporating a large quantity of automatically generated, lower-quality stereo labels allows for a fast transition of the model from one exercise type (running) to another (cycling). However, achieving high model performance comparable to previous benchmarks still requires fine-tuning with high-quality manual labels. Crucially, the research demonstrates that combining numerous automatic labels with a small set of manual labels is sufficient for ThermoCycleNet to effectively segment the calves of both runners and cyclists, accelerating the adaptation of deep neural networks to new applications.
Also Read:
- TopoTTA: Enhancing Segmentation of Tubular Structures Across Diverse Datasets
- OmniUnet: Enhancing Rover Navigation with Multimodal Terrain Perception
For more detailed information, you can refer to the full research paper available at arXiv.org.
