Reliability and validity testing of the medicines related - consultation assessment tool for assessing pharmacists’ consultations

Abstract

Background: Demonstrating a person-centred approach in a consultation is a key component of delivering high-quality healthcare. To support development of such an approach requires training underpinned by valid assessment tools. Given the lack of a suitable pharmacy-specific tool, a new global consultation skills assessment tool: the medicines related-consultation assessment tool (MR-CAT) was designed and tested. Aim: This study aimed to test the validity and reliability of the MR-CAT using psychometric methods. Method: Psychometric testing involved analysis of participants’ (n = 13) assessment of fifteen pre-recorded simulated consultations using the MR-CAT. Analysis included discriminant validity testing, intrarater and interrater reliability testing for each of the five sections of the MR-CAT and for the overall global assessment of the consultation. Analysis also included internal consistency testing for the whole tool. Results: Internal consistency for the overall global assessment of the consultation was good (Cronbach’s alpha = 0.97). The MR-CAT discriminated well for the overall global assessment of the consultation (p < 0.001). Moderate to high intrarater reliability was observed for the overall global assessment of the consultation and for all five sections of the MR-CAT (rho = 0.64–0.84) in the test–retest analysis. Moderate to good interrater reliability (Kendall’s W = 0.68–0.90) was observed for the overall global assessment of the consultation and for all five sections of the MR-CAT. Conclusion: The MR-CAT is a valid and reliable tool for assessing person-centred pharmacist’s consultations. Moreover, its unique design means that the MR-CAT can be used in both formative and summative assessment.