An End-to-End Object Detector with Spatiotemporal Context Learning for Machine-Assisted Rehabilitation

Abstract

Recently, object detection technologies applied in rehabilitation systems are mainly based on the ready-made technology of CNNs. This paper proposes an DETR-based detector which is an end-to-end object detector with spatiotemporal context learning for machine-assisted rehabilitation. To improve the performance of small object detection, first, the multi-level features of the RepVGG are fused with the SE attention mechanism to build a SEFP-RepVGG. To make the encoder-decoder structure more suitable, next, the value of the encoder is generated by using feature maps with more detailed information than key/query. To reduce computation, Patch Merging is finally imported to modify the feature map scale of the input encoder. The proposed detector has higher real-time performance than DETR and obtains the competitive detection accuracy on the ImageNet VID benchmark. Some typical samples from the NTU RGB-D 60 dataset are selected to build a new limb-detection dataset for further evaluation. The results show the effectiveness of the proposed detector in the rehabilitation scenarios.