CurT: End-to-End Text Line Detection in Historical Documents with Transformers

Abstract

We present the curve transformer (CurT), a novel method of direct baseline detection that models document text line detection as set prediction of cubic Bézier curves, simplifying the layout analysis pipeline by removing the need for the laboriously hand-crafted postprocessing algorithms that are necessary with the current state of the art. CurT combines multiple appealing features: direct prediction enabling processing of material that is ill-suited for the prevailing methods adapting semantic segmentation backbones, a conceptually simple Transformer-based encoder-decoder architecture that can be extended to additional tasks beyond baseline detection, and increased computational efficiency in comparison to older approaches. In addition, we demonstrate that CurT achieves metrics that are competitive with methods based on semantic segmentation. Training and inference code is available under Apache 2.0 license at https://github.com/mittagessen/curt.