Smartphone-based estimation of sidewalk surface type via deep learning

Abstract

In this study, we develop a method of estimating the type of sidewalk surface on which a user walks using three-axis acceleration sensor data measured by a user’s smartphone’s accelerometer. If the shape of the sidewalk surface can be estimated automatically, various sidewalk information, such as walkable sidewalks and sidewalks where pedestrians cannot easily walk, can be collected by simply having many users carry their smartphones. We, therefore, propose a method of estimating the sidewalk surface type by applying a convolutional neural network (CNN) based on the VGG16 architecture to sensor data. In addition, we combined VGG16 with hand-crafted features (HCFs) validated in preliminary experiments. During training, the model was pretrained with the human activity sensing consortium (HASC) dataset, a large benchmark for human activity recognition, as a source domain, and we applied fine-tuning (FT) for the sidewalk surfaces as a target domain. We conducted experiments on seven subjects and evaluated the accuracy of our proposed method using leave-one-subject-out cross-validation (LOSO-CV). The experimental results showed that our proposed method achieved the highest accuracy among all the compared methods. Specifically, our proposed method improved the accuracies of some subjects by more than 20% compared with the baseline method.