The reliability of recall measurement in assessing migrant reintegration: Evidence from Ethiopia, Somalia, and Sudan

Abstract

This article asks whether and under what conditions the use of retrospective or “recall” measurement is reliable. Migration researchers are often forced to retrospectively measure baselines when evaluating the impact of interventions due to the transitory nature of migration, developing country contexts, and hastily assembled policy programmes, a situation exacerbated by Covid-19. This article first theoretically considers the extent to which this approach is reliable and likely to result in biased estimates, as well as its broader advantages, disadvantages, and recommendations for best practice. It then considers the case of the “IMPACT” evaluation of the EU-IOM Joint Initiative for Migrant Protection and Reintegration in which 1774 Ethiopian, Somalia, and Sudanese migrant returnees in 2021 were assessed on a range of reintegration measures, 1095 of whom were measured retrospectively. Regression analyses demonstrate that those measured retrospectively give more negative scores on several “Reintegration Sustainability Scores” in line with some theoretical expectations but contrary to others. However, this—mostly non-statistically significant—effect is largely diminished when the small minority who report finding it difficult to remember the baseline period are removed—suggesting that any retrospective measurement effect results from memory bias rather than, for example, consistency bias. No evidence is found to support several theoretically derived interaction effects. Determinants of self-reported memory are demonstrated and recommendations for usage of retrospective measurement are provided, based on these findings.