Structuralizing disaster-scene data through auto-captioning

Abstract

Disaster-scene images documenting the magnitude and effects of natural disasters nowadays can be easily collected through crowdsourcing aided by mobile technologies (e.g., smartphones or drones). One challenging issue that confronts the first-responders who desire the use of such data is the non-structured nature of these crowdsourced images. Among other techniques, one natural way is to structuralize disaster-scene images through captioning. Through captioning, their imagery contents are augmented by descriptive captions that further enable more effective search and query (S&Q). This work presents a preliminary test by exploiting an end-to-end deep learning framework with a linked CNN-LSTM architecture. Demonstration of the results and quantitative evaluation are presented that showcase the validity of the proposed concept.