FR-DETR: End-to-End Flowchart Recognition With Precision and Robustness

Abstract

Traditional flowchart recognition methods have difficulties in detecting newly added symbols and distinguishing targets from complex backgrounds like line texture. Existing deep-learning-based object detectors and line segment detectors are promising in recognizing and distinguishing targets from texture backgrounds. However, using two separate detectors will inevitably cause unnecessary training and inference costs. Moreover, the insufficient volume and diversity of currently available dataset limit the effectiveness of model training. To address these issues, this paper proposes an end-to-end multi-task network FR-DETR (Flowchart Recognition DEtection TRansformer) and a new dataset for precise and robust flowchart recognition. FR-DETR comprises a CNN backbone and a shared multi-scale Transformer structure to perform symbol detection and edge detection using shared feature maps and respective prediction heads in a coarse-to-fine refinement process. The coarse stage analyzes features with low resolution and suggests candidate regions that contain potential targets for the fine stage to produce accurate predictions using features with high resolution. Meanwhile, a new dataset is constructed to provide more symbol types and complex backgrounds for network training and evaluation. It contains more than 1000 machine-generated flowchart images, 25K+ symbol instances with nine categories, and 20K+ line segments. The experiments show that FR-DETR achieves an overall precision and recall of 94.0% and 93.1% on the proposed dataset, and 98.7% and 98.1% on the CLEF-IP dataset, respectively, which all outperform the prior methods.