Poisonous Datasets, Poisonous Trees

Abstract

Machine learning gives rise to concerns about “algorithmic bias,” arising from bias in the training dataset. A dataset necessarily reflects a past state of affairs, but we may anticipate that the future will be different, or desire it to be so. Law has struggled with an analogous problem: how to deal with “bad evidence” and prejudice. Holmes’s broad view of experience suggests that once undesirable data has been revealed, its potential for mischief is there, and it is futile to pretend it does not exist. Instead, law has developed several strategies to deal with these negative influences. Under the doctrine of the “fruit of the poisonous tree,” in American jurisprudence, evidence that stems from improper actions by public authorities is inadmissible in court. Or, a judge can give instructions to the jury to guard against improper inferences. Or, a jury verdict may be struck down on appeal. Strategies analogous to these might help to make machine learning a tool for better outcomes, rather than a trap that entrenches past mistakes and prejudice.