Detecting sections and entities in court decisions using HMM and CRF graphical models

Abstract

Court decisions are legal documents that undergo careful analysis by lawyers in order to understand how judges make decisions. Such analyses can indeed provide invaluable insight into application of the law for the purpose of conducting many types of studies. As an example, a decision analysis may facilitate the handling of future cases and detect variations in judicial decision-making with respect to specific variables, like court location. This paper presents a set of results and lessons learned during a project intended to address a number of challenges related to searching and analyzing a large body of French court decisions. In particular, this paper focuses on a concrete and detailed application of the HMM and CRF sequence labeling models for the tasks of: (i) sectioning decisions, and (ii) detecting entities of interest in their content (e.g. locations, dates, participants, rules of law). The effect of several key design and fine-tuning features is studied for both task categories. Moreover, the present study covers steps that often receive little discussion yet remain critical to the practical application of sequence labeling models, i.e.: candidate feature definition, selection of good feature subsets, segment representations, and impact of the training dataset size on model performance.