TLDR: A new method called Hierarchical Identity Learning (HIL) improves unsupervised person re-identification across visible and infrared cameras. It uses a two-stage clustering process to capture both broad and fine details of a person’s identity, combined with a multi-center contrastive learning strategy and a bidirectional matching mechanism to accurately link identities across different camera types without needing any manual annotations.
Visible-infrared person re-identification (VI-ReID) is a crucial area in computer vision that focuses on matching images of the same individual captured by both visible light and infrared cameras. This task is inherently challenging due to the significant differences, or ‘modality gap,’ between visible and infrared images. Unlike single-modality re-identification, VI-ReID faces the added complexity of bridging this gap while accurately identifying people. A major hurdle in developing such systems is the immense cost and time involved in manually annotating large cross-modal datasets, which has led researchers to explore unsupervised methods that can learn without relying on expensive human labels.
Existing unsupervised VI-ReID approaches often use a technique called cluster-based contrastive learning. These methods typically represent a person’s identity using a single cluster center, which helps in finding common features among images within that cluster. However, this approach often overlooks the subtle, finer-grained differences that might exist among images of the same person. For instance, variations in pose, lighting, or background could create distinct sub-patterns within a single identity cluster, which a single-center representation might fail to capture. While some methods have introduced multiple memory representations to address this, they sometimes risk overfitting by not adequately preserving the broader, coarse-grained information about an identity.
To overcome these limitations, a new framework called Hierarchical Identity Learning (HIL) has been proposed. This innovative approach recognizes that a single, broad cluster for a person might actually contain several smaller, more specific sub-clusters that reflect these fine-grained variations. HIL is designed to integrate both coarse-grained (broader identity) and fine-grained (subtle variations) pseudo-labels, providing a more comprehensive understanding of a person’s identity. You can find more details about this research in the paper: Hierarchical Identity Learning for Unsupervised Visible-Infrared Person Re-Identification.
How the HIL Framework Works
The HIL framework is built upon a dual-path transformer architecture and incorporates three main components:
First, the core of HIL involves a **two-stage clustering strategy** to generate hierarchical identity information. In the initial stage, a clustering algorithm called DBSCAN is used to group visible and infrared instance features into coarse-grained pseudo-labels, essentially identifying initial person clusters. Following this, a secondary clustering step, typically using K-means, is applied *within* each of these coarse-grained clusters. This second stage helps uncover the finer-grained variations and sub-clusters that a single clustering pass might miss, leading to a richer, multi-level representation of each person’s identity.
Second, the framework introduces **Multi-Center Contrastive Learning (MCCL)**. With the hierarchical identity information now available (both coarse and fine-grained cluster centers), MCCL leverages these multiple centers as reference points. It constructs robust sets of positive and negative samples, allowing the model to refine its feature representations. This process enhances the quality of clustering within each modality (e.g., visible-to-visible or infrared-to-infrared) and significantly reduces the discrepancies between the visible and infrared modalities, making the features more consistent across different camera types.
Third, to further improve cross-modal matching, HIL employs a **Bidirectional Reverse Selection Transmission (BRST)** mechanism. This mechanism establishes reliable correspondences between visible and infrared pseudo-labels by performing matching in both directions. Crucially, it includes a reverse selection process that filters out unreliable pseudo-label matches. This ensures that only the most confident and accurate cross-modal links are retained, thereby enhancing the overall quality and robustness of the identity alignment across modalities.
Also Read:
- SignClip: A New Framework for Accurate Sign Language Translation Using Gestures and Mouthing
- Advancing Open-Vocabulary Segmentation for Remote Sensing Images
Performance and Impact
Extensive experiments were conducted on two widely-used datasets, SYSU-MM01 and RegDB, to evaluate the HIL framework. The results demonstrated that HIL significantly outperforms existing state-of-the-art unsupervised VI-ReID methods. For instance, on the SYSU-MM01 dataset (All Search mode), HIL achieved a notable improvement in both mAP (mean Average Precision) and Rank-1 accuracy. Similar substantial gains were observed on the RegDB dataset (Visible-to-Infrared mode).
Remarkably, HIL also showed competitive performance when compared to some supervised methods, which benefit from precise manual annotations. This highlights HIL’s robustness and its ability to achieve high accuracy without the need for any labeled data, thereby drastically reducing the dependency on human annotation efforts. Ablation studies further confirmed the effectiveness of each individual component (MCCL and BRST), showing that each contributes significantly to the overall performance improvements.
In conclusion, the Hierarchical Identity Learning (HIL) framework represents a significant advancement in unsupervised visible-infrared person re-identification. By intelligently learning hierarchical identity information and employing sophisticated mechanisms for multi-center contrastive learning and bidirectional cross-modal matching, HIL effectively addresses the challenges of modality gaps and noisy pseudo-labels, paving the way for more practical and scalable person re-identification systems.
