Recognizing and avoiding bias to improve child custody evaluations: Convergent data are not sufficient for scientific assessment

Abstract

Numerous academics have raised concerns about the scientific validity of custody evaluations. To improve the quality of evaluations, the root cause of highly trained people frequently producing reports with limited scientific value (no scientifically based nexus between the data and the conclusion) needs to be understood. Review of 50 evaluations found violations of scientific methodology including: cherry picking data, arbitrary determinations of contested facts, using ad hoc hypotheses rather than the results of empirical research, presenting highly speculative inferences as expert opinions, misinterpretation of psychological testing, incorrect interpretation of the behavior of abused children, and reliance on the presence of convergent data rather than hypothesis testing to reach conclusions. The key finding was that errors almost always helped the same parent, indicating that bias drove many of the errors. Combining cognitive psychology and behavioral economics provides an understanding of the high frequency and power of bias in the reports. Intuition, which is highly susceptible to bias, forges our initial opinions, and then confirmation bias affects the vetting, valuing, remembering and interpretation of the data used in abortive attempts at rigorous analysis. To improve the scientific validity of evaluations several steps are needed including (1) better oversight of evaluations, (2) educating evaluators and courts about scientific methodology, the high risk of bias, and the results of empirically based research, and (3) adoption of more structured approaches to custody evaluations.