An occupation-based, metacognitive approach to assessing error performance and online awareness

Abstract

Background/aim: Online awareness is the ability to self-monitor, identify and self-correct errors while engaged in an activity. Current assessments of online awareness involve observing and classifying error behaviour during structured, uniform tasks. However, during rehabilitation, practitioners typically work towards improving performance in individually meaningful tasks unique to the client. This article presents a metacognitive, task analytic approach to assessing online awareness involving observation and classification of errors during meaningful occupations determined after client-centred goal setting with two male clients with severe traumatic brain injury (aged 22 and 23). Study aims were to describe the approach, evaluate its feasibility and determine inter-rater agreement for error detection and error categorisation by two experienced occupational therapists. Furthermore, the error profiles and cognitive impairments of the participants on standardised neuropsychological assessment were examined to explore the validity of the assessment. Methods: Individualised assessment tasks included snack preparation, budgeting, timetabling, hot-drink preparation and use of a computer program, which were administered repeatedly over two to three months and audio-visual recordings taken. Independent ratings of two trained occupational therapists were compared using exact percent agreement. Results: Overall agreement about errors was 76%, for which there was 65% agreement about error categorisation and 100% agreement about error correction. Conclusions: There was fair inter-rater agreement between two trained occupational therapists of error behaviour and error correction when using the described occupation-based approach to assessing online awareness. This approach has promise, particularly when combined with standardised, neuropsychological assessments, for providing an in-depth understanding of error behaviour and awareness of errors during meaningful occupations.