TLDR: The paper introduces LIRR-SDA, a Supervised Domain Adaptation (SDA) framework for robust 6-DoF spacecraft pose estimation. It tackles the ‘synthetic-to-real domain gap’ by jointly optimizing domain-invariant representations and task-specific risk using both labeled synthetic and limited labeled real data. Evaluated on the SPEED+ benchmark, LIRR-SDA consistently outperforms existing baselines, achieving superior keypoint regression accuracy and overall pose estimation, even with minimal real-world labels. The framework is lightweight and computationally efficient, offering a practical solution for autonomous space operations.
Spacecraft Pose Estimation (SPE) is a critical technology for a variety of autonomous space operations, including rendezvous, docking, and in-orbit servicing. This involves accurately determining a target spacecraft’s position and orientation, known as its six-degree-of-freedom (6-DoF) pose, from images captured by a chaser spacecraft. However, a significant challenge arises because models trained on abundant synthetic images often perform poorly when faced with real-world or laboratory-generated imagery. This discrepancy is known as the ‘synthetic-to-real domain gap,’ caused by differences in illumination, specular reflections, limited texture, and variations in apparent size.
Traditional methods for SPE, such as geometric computer vision techniques, are often sensitive to noise and environmental changes, limiting their robustness. While deep learning approaches have emerged, directly mapping raw images to pose parameters, they typically demand vast amounts of labeled data and can be less adaptable to new spacecraft designs. Hybrid modular approaches, which combine data-driven feature extraction with geometric model-based solvers, have shown promise. These pipelines typically involve three stages: object detection to locate the spacecraft, keypoint regression to identify specific 2D points on the spacecraft, and a Perspective-n-Point (PnP) solver to compute the 6-DoF pose from these 2D-3D correspondences.
Despite the effectiveness of hybrid modular methods on synthetic datasets like SPEED and SPEED+, their performance degrades sharply on real imagery due to the domain gap. Current domain adaptation techniques primarily focus on unsupervised scenarios where no labeled real-world data is available. However, in many practical situations, a limited number of labeled real target-domain images can be acquired, offering an opportunity for more effective adaptation.
A new research paper, titled “BRIDGING THE SYNTHETIC–REAL GAP: SUPERVISED DOMAIN ADAPTATION FOR ROBUST SPACECRAFT 6-DOF POSE ESTIMATION”, introduces the first Supervised Domain Adaptation (SDA) framework specifically designed for SPE keypoint regression. Authored by Inder Pal Singh, Nidhal Eddine Chenni, Abd El Rahman Shabayek, Arunkumar Rathinam, and Djamila Aouada from SnT, University of Luxembourg, this work addresses the limitations of existing methods by leveraging both labeled synthetic and a modest amount of labeled real data.
The LIRR-SDA Framework
The proposed method, called LIRR-SDA (Learning Invariant Representation and Risk for Supervised Domain Adaptation), builds upon the LIRR paradigm. It jointly optimizes two key objectives: learning domain-invariant representations and minimizing task-specific risk. This means the model is trained to extract features from images that are consistent regardless of whether the image is synthetic or real, while also ensuring that its predictions (e.g., keypoint locations) are accurate across both domains.
In simpler terms, the framework works by making the feature extractor learn to ‘fool’ a domain classifier, so the features it produces are indistinguishable between synthetic and real images. Simultaneously, it uses two prediction heads – one domain-invariant and one domain-dependent – to ensure that the final keypoint predictions are robust and accurate across different domains. This joint optimization allows the model to effectively bridge the synthetic-to-real gap, even with limited real-world supervision.
Experimental Validation and Results
The researchers conducted extensive experiments on the SPEED+ benchmark dataset, which includes a large synthetic source domain and two real target domains (Sunlamp and Lightbox). They compared their LIRR-SDA approach against several baselines: ‘Source-only’ (trained only on synthetic data), ‘Oracle’ (trained only on limited real data), ‘Fine-tune’ (pre-trained on synthetic then fine-tuned on real), and ‘SDA’ (a general supervised domain adaptation method).
The results demonstrated that LIRR-SDA consistently outperformed all other baselines across various metrics, including Keypoint Error, Percentage of Correct Keypoints (PCK), Range Error, Attitude Error, and the overall ESA Score. Remarkably, with only 5% of labeled target data, their method matched or even surpassed the performance of the ‘Oracle’ baseline, which was trained on larger fractions of labeled data. This highlights the efficiency and effectiveness of their approach in utilizing scarce real-world annotations.
The framework is also described as lightweight, backbone-agnostic (meaning it can be easily integrated with different neural network architectures), and computationally efficient. These characteristics make it a practical solution for deployment in real-world space environments, offering a pathway toward more robust and reliable spacecraft pose estimation.
Also Read:
- Advancing Open-Vocabulary Segmentation for Remote Sensing Images
- Advanced AI Framework Boosts Ship Wake Detection in Radar Imagery
Conclusion and Future Directions
This research underscores the importance of carefully designed adaptation objectives when labeled target data is limited but not entirely absent. By effectively aligning invariant representations with task-specific risk, the LIRR-SDA framework significantly minimizes the synthetic-to-real gap in spacecraft pose estimation. While the current work focuses on keypoint regression, future research will explore challenges such as extreme illumination changes, occlusions, and motion blur, as well as investigating temporal modeling across image sequences and reducing annotation requirements through self- or weakly supervised learning. This study provides valuable insights for practical deployment and lays a foundation for further advancements in generalizable on-orbit perception.
