Judges are from mars, pro se litigants are from venus: Predicting decisions from lay text

Abstract

Access to justice could be significantly expanded if decision support systems were able to accurately interpret statements of fact by pro se (self-represented) litigants. Prior research, which has demonstrated that case decisions can often be predicted by machine-learning models trained on judges' statements of facts, suggests the hypothesis that these same learning algorithms could be effectively applied to pro se litigants' fact statements. However, there has been a dearth of corpora on which to test this hypothesis. This paper describes an experiment testing the ability to predict the outcome of pro se litigants' complaints on a corpus of 5,842 cases initiated by citizen complaints. The results of this experiment were strikingly negative, suggesting that fact statements by unguided pro se litigants are far less amenable to simple machine-learning techniques than judges' texts and appearing to disconfirm the hypothesis above.