M²Former: Enhancing Event-Based RT-DETR for Robust and Lightweight Space Object Detection

Ruitao Pan, Chenxi Wang*, Bing Han, Xinyu Zhang, Zhi Zhai,
Jinxin Liu, Naijin Liu, Xuefeng Chen, Senior Member, IEEE
IAMIE, Xi'an Jiaotong University
Accepted by IEEE Transactions on Geoscience and Remote Sensing
*Corresponding author (wangchenxi@xjtu.edu.cn)
Pape Code Dataset

Performance comparison on the E-SPARK dataset. M²Former* denotes the variant of M²Former trained with the improved data augmentation strategy. RT-DETR-R18* denotes the variant of RT-DETR-R18 trained with the default data augmentation strategy. The size of each circle indicates the number of model parameters.

Abstract

With increasing human space activities, detecting resident space objects (RSOs) has become critical for space monitoring and on-obrit missions. Traditional optical sensors struggle in space environments due to extreme illumination variations and motion blur. Event cameras, bio-inspired sensors that asynchronously record per-pixel brightness changes, offer high temporal resolution, wide dynamic range, and low power consumption, making them promising for orbital sensing yet underexplored in this context. In this work, we present the first systematic study of event-based space object detection. To address the scarcity of event data, we construct E-SPARK, a large-scale dataset generated with affine transformations and advanced simulators. Building upon this dataset, we propose M²Former, a lightweight multi-scale MetaFormer backbone, together with an area-aware loss (AAL) tailored for small object detection. These are integrated into the RT-DETR framework, a Transformer-based detector known for its robustness but higher computational cost compared to YOLO models. Our design reduces parameters and complexity by over 50% while maintaining comparable detection accuracy. In addition, we design an improved data augmentation strategy that enriches supervision density and data diversity, further boosting detection performance. Experiments on both synthetic and real event data demonstrate that our method achieves state-of-the-art performance and strong generalization. These results highlight the potential of event cameras as a reliable sensing modality for spaceborne detection.

Qualitative Results

RepViT demonstrates the success of the MetaFormer paradigm within convolutional networks. Inspired by this, we redesign the RT-DETR backbone into a MetaFormer-style structure. The proposed M²Former enables efficient convolutional and downsampling operations with multi-scale feature extraction capability. It consists of Res2Net, spatial attention, channel attention, and SPD-Conv modules. Detection results on the proposed E-SPARK dataset are shown below.

Quantitative Results

We adopt standard COCO metrics for object detection benchmarks. AP@50 represents mean Average Precision at IoU threshold 0.5, which measures overall detection accuracy. AP@50:95 averages AP over multiple IoU thresholds (0.5 to 0.95 with a step of 0.05), offering a more stringent indicator of localization precision. APs denotes average precision for small objects, highlighting performance in challenging small object detecion. In zero-shot detection, we consider it as a category-agnostic task.

  • Table I lists the augmentation strategies used during training, including Mosaic, Mixup, translation, scaling, and flipping, which enhance data diversity and generalization.
  • Table II compares event representations. Although Event Volume yields slightly better accuracy, Event Histogram is selected for its strong performance and faster preprocessing.
  • Table III shows that M²Former with the AAL and the improved data augmentation strategry achieves the best accuracy, especially for small objects, validating the effectiveness of our design.
  • Table IV highlights that M²Former reduces parameters and GFLOPs by over 50% compared to RT-DETR-R18 while maintaining real-time performance.
  • Table V presents ablation results of the M²Former, showing the impacts of each module.
  • Table VI confirms that Mosaic and Mixup significantly improve performance, supporting the use of tailored augmentations.
  • Table VII demonstrates that M²Former remains robust even at low input resolution, achieving the highest performance.
  • Table VIII shows that M²Former outperforms other models in zero-shot generalization across lighting conditions.

Generalization to Real data

Zero-shot validation is conducted on real-world data captured from a ground-based testbed, using models trained exclusively on synthetic data without any fine-tuning. We collecte a total of 300 samples using the testbed, covering three illumination conditions: normal exposure, overexposure, and underexposure, with 100 samples per condition. For each sample, RGB images and event data are synchronously recorded by the DAVIS346 camera at the resolution of 346×260. We adopt YOLOv8s as the representative RGB-based detector due to its high detection accuracy and moderate model complexity.
Due to motion blur, color degradation, and texture loss, RGB-based detectors struggle to localize targets under extreme illumination, especially failing completely under underexposure. In contrast, both YOLOv8s and M²Former exhibit stronger robustness when using event data, which is less affected by illumination artifacts. Notably, although M²Former generates more detection boxes than YOLOv8s, these boxes remain consistently close to the target, indicating higher recall and better localization. This highlights a key trade-off: M²Former maintains superior robustness and recall under challenging conditions, whereas YOLOv8s sacrifices recall to preserve precision—often leading to missed detections.
In summary, these results underscore the robustness of event data under extreme lighting conditions and demonstrate the strong sim-to-real generalization capability of the M²Former model, making it reliable for object detection in real-world space scenarios.

Simulation testbed for real data collection and its hardware components.

Detection results under varying illumination conditions. Columns represent normal, overexposed, and underexposed scenes; rows correspond to YOLOv8s (RGB), YOLOv8s (event), RT-DETR-R18 (event), and M²Former (event), respectively. Bounding box colors represent predicted classes, but the task is treated as class-agnostic.

BibTeX

@ARTICLE{11263950,
  author={Pan, Ruitao and Wang, Chenxi and Han, Bin and Zhang, Xinyu and Zhai, Zhi and Liu, Jinxin and Liu, Naijin and Chen, Xuefeng},
  journal={IEEE Transactions on Geoscience and Remote Sensing}, 
  title={M2Former: Enhancing Event-Based RT-DETR for Robust and Lightweight Space Object Detection}, 
  year={2025},
  volume={63},
  pages={1-16},
  keywords={Space vehicles;Cameras;YOLO;Event detection;Transformers;Data augmentation;Computer architecture;Computational modeling;Training;Lighting;Event-based vision;multiscale MetaFormer design;real-time detection Transformer (RT-DETR);space object detection},
  doi={10.1109/TGRS.2025.3636122}}