Some languages seem easier to parse because their treebanks leak

Abstract

Cross-language differences in dependency parsing performance are mostly attributed to treebank size, average sentence length, average dependency length, morphological complexity, and domain differences. In this paper I point to a factor not previously discussed: If we abstract away from words and dependency labels, how many graphs in the test data were seen in the training data? I discuss how to compute graph isomorphisms, and show that, treebank size aside, overlap between training and test graphs explains more of the observed variation than standard explanations such as the above.