Automated extraction of quality indicators for treatment of children with complex developmental disorders: A feasibility study using the example of attention-deficit/hyperactivity disorder

Abstract

Quality issue: Quality assessment is challenging in children with developmental disorders. Previously, a set of quality indicators (QIs) was developed and analyzed in terms of feasibility of use with patients with attention-deficit/hyperactivity disorder (ADHD). QI assessment turned out to be possible but highly complex. Thus, we compared different technologies for automated extraction of data for assessment of QIs. Choice of solution: Four automated extraction technologies (regular expressions, Apache Solr, Apache Mahout, Apache OpenNLP) were compared with respect to their properties regarding the complexity of implementing the QI, the complexity of implementing a check module, the reliability and quality of results, the complexity of preparation of interdisciplinary medical reports, and the complexity of deployment and installation. Implementation: Twenty medical reports from different institutions were reviewed for compliance with three QIs by these technologies and compared with expert opinions. Evaluation: Among the four technologies, Apache Solr had the best overall performance. For manual extraction of the three QIs, at least 77 s were necessary per medical report, whereas the prototype evaluated and extracted the QIs automatically in 8 s on average. Unexpectedly, different assessments of the degree of compliance by the experts turned out to be one of the stumbling blocks. An in-depth evaluation compared results on a semantic level. Lessons learned: It is possible to extract QIs by post-processing automated technologies. This approach can also be applied to other developmental disorders. However, a more uniform documentation throughout institutions involved will be necessary in order to implement this method in daily practice.