MemoriEase 2.0: A Conversational Lifelog Retrieve System for LSC'24

Abstract

Lifelog retrieval plays an important role in memory support for lifeloggers. It helps the lifeloggers to browse, search and navigate their life moments from the lifelog data. However, the volume and variety of lifelog data are enormous and range in multiple modalities so they impose a big challenge to retrieve accurate lifelog moments. The Lifelog Search Challenges (LSCs) are a benchmark challenge for evaluating lifelog retrieval systems in different tasks. In this paper, we introduce the MemoriEase 2.0 lifelog retrieval system that participates in LSC'24. This system not only inherits core functions from the precedent system but also incorporates new components such as conversational search, visual similarity search and retrieval-augmented generation for question-answering tasks. The new functions are expected to help expert and novice users solve all topics in three tasks of LSC'24. We evaluate MemoriEase 2.0 in KIS topics in LSC'23 and the system achieves promising results with Recall@1 is 40% at the first hint and it solves 8 over 10 topics.