Multi-scale Global Reasoning Unit for Semantic Segmentation

Abstract

Obtaining context information in a scene is an essential ability for semantic segmentation. GloRe [1] learns to infer the context from a graph-based feature constructed by the Global Reasoning unit. The graph nodes are features that are segmented into regions in image space, and the edges are relationships between nodes. Therefore, a failure to construct the graph results in poor performance. In this study, to resolve this problem, we propose a novel unit to construct the graph using multi-scale information. We call it Multi-scale Global Reasoning Unit. It considers the relationship between each region that retains detailed multi-scale spatial information. Specifically, the proposed unit consists of a Feature Aggregation Module and a Global Reasoning Module. The former selects the features required to construct the graph using the multi-scale features. The latter uses GloRe to infer the relationship from the features. The unit is trained in an end-to-end manner. In experiments, we evaluate the effectiveness of the proposed method on Cityscapes and Pascal-context datasets. As a result, we confirmed that the proposed method outperforms the original GloRe.