Estimating comic content from the book cover information using fine-tuned VGG model for comic search

Abstract

The purpose of this research is to realize retrieval of comic based on content information. Resources of the contents information of existing comics were only the comics itself and review. However, these pieces of information have drawbacks that they can not sufficiently extract information necessary for searching, and that they contain a lot of unnecessary information. In order to solve this problem, we proposed to use the book cover of comics as a resource to grasp the contents of comics. In the proposed method, we estimate the age and cultural background of comics expressed by clothes and belongings written on the cover of comics from the reasoning model which performed fine-tuning from the VGG-16 model. Also, we associated comics with each other based on the obtained semantic vectors and tags. As a result of the experiment, the accuracy of the model was 0.693, and the reproducibility of the tag to the correct data was 0.918. Furthermore, we observed unity in the comics related by the obtained information.