Self-reported and objectively assessed knowledge of evidence-based practice terminology among healthcare students: A cross-sectional study

Abstract

Background Self-reported scales and objective measurement tools are used to evaluate self-perceived and objective knowledge of evidence-based practice (EBP). Agreement between self-perceived and objective knowledge of EBP terminology has not been widely investigated among healthcare students. Aim The aim of this study was to examine agreement between self-reported and objectively assessed knowledge of EBP terminology among healthcare students. A secondary objective was to explore this agreement between students with different levels of EBP exposure. Methods Students in various healthcare disciplines and at different academic levels from Norway (n = 336) and Canada (n = 154) were invited to answer the Terminology domain items of the Evidence-Based Practice Profile (EBP2) questionnaire (self-reported), an additional item of ‘evidence based practice’ and six random open-ended questions (objective). The open-ended questions were scored on a five-level scoring rubric. Interrater agreement between self-reported and objective items was investigated with weighted kappa (Kw). Intraclass correlation coefficient (ICC) was used to estimate overall agreement. Results Mean self-reported scores varied across items from 1.99 (‘forest plot’) to 4.33 (‘evidence-based practice’). Mean assessed open-ended answers varied from 1.23 (‘publication bias’) to 2.74 (‘evidence-based practice’). For all items, mean self-reported knowledge was higher than that assessed from open-ended answers (p<0.001). Interrater agreement between self-reported and assessed open-ended items varied (Kw = 0.04–0.69). The overall agreement for the EBP2 Terminology domain was poor (ICC = 0.29). The self-reported EBP2 Terminology domain discriminated between levels of EBP exposure. Conclusion An overall low agreement was found between healthcare students’ self-reported and objectively assessed knowledge of EBP terminology. As a measurement tool, the EBP2 Terminology scale may be useful to differentiate between levels of EBP exposure. When using the scale as a discriminatory tool, for the purpose of academic promotion or clinical certification, users should be aware that self-ratings would be higher than objectively assessed knowledge.