Are self-assessment of search ability and performance reliable?

Abstract

This study examined users' self-rated search ability with their objective search performance, their subjective assessed search performance, and the accuracy of subjective assessments. Thirty students were recruited for the lab mode online search competition. Participants were asked to rate their search abilities before searching, and predict search performance after searching. The results showed that users' self-rated search ability was not related to their objective search performance in this search competition. As for subjective evaluation of their search performance, no significant difference was found on predicted-as-right, predicted-as-unsureness rate, or the correct and incorrect prediction rates. Besides 70% of correct predictions, all searchers tended to overestimate their search performance, but self-rated low searchers had more chance to predict their answers to the search tasks as wrong and underestimated their search performance sometimes. This study is an exploratory examination on users' self-rated search ability, and has implications to understand how searchers evaluate their search abilities and search performance.