Multiphase Focal Liver Lesions Classification with Combined N-gram and BoVW

Abstract

The bag-of-visual-words (BoVW) model has emerged as an effective approach to represent features for focal liver lesions (FLLs). However, most of the previous methods have the limitation of insufficient consideration of the spatiotemporal co-occurrence information, which provokes the low descriptive power of classic visual words. In contrast to previous work, we propose a novel model for multiphase medical image feature generation named the Bi-gram bag-of-spatiotemporal words (Bi-gram BoSTW) to capture the temporal information, as well as, the spatial co-occurrence relationship of the lesion. First, temporal co-occurrence images from multiphase images are constructed. Second, BoVW is employed to extract temporal features from the temporal co-occurrence images and generates the visual words. Finally, we introduce the N-gram schema to add spatial relation to local descriptors. To the best of our knowledge, this is the first work that introduces visual N-grams scheme to contrast-enhanced CT images, which integrates temporal information with spatial co-occurrence relationship and improves the classification performance. The effectiveness of the proposed model is verified on 132 FLLs with confirmed pathology type. The experimental results indicate that (1) the N-gram enriches the semantics and provides more complete representation; (2) the proposed model achieves the best accuracy (83%) with highest training speed (1.5 min) among several well-known methods based on BoVW model.