Questionnaires for 360-degree assessment of consultant psychiatrists: Development and psychometric properties

Abstract

Background: Expert clinical judgement combines technical proficiency with humanistic qualities. Aims: To test the psychometric properties of questionnaires to assess the humanistic qualities of working with colleagues and relating to patients using multisource feedback. Method: Analysis of self-ratings by 347 consultant psychiatrists and ratings by 4422 colleagues and 6657 patients. Results: Mean effectiveness as rated by self, colleagues and patients, was 4.6, 5.0 and 5.2 respectively (where 1=very low and 6=excellent). The instruments are internally consistent (Cronbach's alpha >0.95). Principal components analysis of the colleague questionnaire yielded seven factors that explain 70.2% of the variance and accord with the domain structure. Colleague and patient ratings correlate with one another (r=0.39, P < 0.001) but not with the self-rating. Ratings from 13 colleagues and 25 patients are required to achieve a generalisability coefficient (Ep2) of 0.75. Conclusions: Reliable 360-degree assessment of humane judgement is feasible for psychiatrists who work in large multiprofessional teams and who have large case-loads.