A-posteriori novelty assessments for sequential design sessions

Abstract

Among the various novelty metrics available in literature, those developed by Shah and Vargas- Hernandez are frequently used for academic purposes. More precisely, their "a-posteriori" version is often used for assessing ideation effectiveness of idea generation methods. We observed that when in presence of sequential idea generation sessions of the same task, the application of the recalled metric could lead to misleading results. In this paper, we argue about this problem, and we also provide useful suggestions for a correct "a-posteriori" assessment for sequential idea generation sessions.