TLDR: FoQuS is a new coreset selection framework for Automatic Modulation Recognition (AMR) that significantly reduces deep learning training time and resources. It achieves this by intelligently selecting a small, representative subset of data using three novel importance metrics (Forgetting, Persistent Error, Quality Scores) tailored for signal data, outperforming existing methods and showing strong generalization across different model architectures.
Deep learning models have revolutionized many fields, including Automatic Modulation Recognition (AMR), which is crucial for identifying different types of radio signals. These models, however, often demand vast amounts of labeled data and significant computational resources for training. This can lead to lengthy development cycles and high costs, especially when designing new models or fine-tuning existing ones.
To tackle these challenges, researchers have introduced a novel framework called FoQuS: A Forgetting-Quality Coreset Selection Framework for Automatic Modulation Recognition. This innovative approach aims to drastically cut down training overhead by selecting a small, yet highly representative, subset of data—known as a coreset—from the original, larger dataset. The goal is to achieve performance comparable to training on the full dataset, but with much less time and energy.
Existing coreset selection methods, while effective in areas like image processing, haven’t been directly applicable to AMR tasks. This is primarily due to the fundamental differences in data structure between signals and images. Signal data, unlike images, possesses critical physical properties such as the signal-to-noise ratio (SNR). Ignoring SNR during coreset selection can lead to an overrepresentation of easily learned, high-SNR samples, while neglecting crucial low-SNR samples, ultimately degrading performance in noisy environments.
FoQuS addresses these unique challenges by first training a model on the entire dataset and meticulously recording the prediction trajectory of each sample. From this, it constructs three distinct and complementary importance metrics:
- Forgetting Score: This metric identifies samples that are close to the decision boundary and are easily ‘forgotten’ by the model during training, indicating their instability and importance.
- Persistent Error Score: This score targets stubborn examples that the model consistently struggles to learn, highlighting their difficulty and the need for their inclusion.
- Quality Score: This metric focuses on samples that are partially learned but still provide valuable gradients, contributing meaningfully to the model’s learning process.
Instead of simply picking the top-scoring samples, FoQuS normalizes these three scores and combines them into a final FoQuS score. The dataset is then intelligently partitioned into three tiers based on these scores, and a specific proportion of samples is drawn from each tier to form a diverse and effective coreset. This tiered selection strategy ensures a balanced representation of different types of samples, including those with varying SNR characteristics.
Extensive experiments were conducted on three widely-used AMR datasets: RML2016.10a, Sig2019-12, and RML2018.01a. FoQuS was compared against ten existing coreset selection methods across various sampling rates (1% to 30% of the original data). The results consistently showed that FoQuS significantly outperforms traditional baselines, especially at very low sampling rates, while maintaining a leading advantage at medium and high rates. Furthermore, FoQuS demonstrated excellent cross-architecture generalization, meaning the coresets selected by one model architecture performed well when evaluated with different model structures, including more advanced AMR models like SigNet and IQFormer.
Also Read:
- Deep Learning Powers Next-Generation Integrated Sensing and Communication
- Enhancing Radar Detection and Tracking with Integrated Feature Learning
An ablation study confirmed the importance of each component of the FoQuS score, showing that the combination of all three metrics yields the best performance. This research marks a significant step forward in making deep learning for Automatic Modulation Recognition more efficient and accessible, reducing the computational burden without sacrificing accuracy. For more technical details, you can refer to the full research paper here.
