Transfer Learning for Improving Lifelog Image Retrieval

Abstract

With lifelogging devices; such as wearable camera, smart watches, audio recorder or standalone smartphone applications; capturing daily moments becomes easier. In recent years, many workshops and panels have emerged and proposed benchmarks to face challenges in organizing, analyzing, managing, indexing and retrieving specific moments in the huge amount of multi-modal lifelog dataset. Recent advances in deep neural networks have given rise to new approaches to deep learning-based image retrieval. However, using deep neural networks in lifelog context systems is continuously rising challenges: relying on a convolutional neural network which is trained on images not related to the retrieval dataset reduced the performance to extract features. In this paper, we propose a novel fine-tuned Convolutional Neural Network approach based on a Long Short Term Memory processing for improving lifelog image retrieval. The experimental results show the feasibility and effectiveness of our approach with encouraging performance by reaching third place in the ImageCLEF Lifelog Moment Retrieval Task 2018.