The illiterate editor: Metadata-driven revert detection in wikipedia

Abstract

As the community depends more heavily on Wikipedia as a source of reliable information, the ability to quickly detect and remove detrimental information becomes increasingly important. The longer incorrect or malicious information lingers in a source perceived as reputable, the more likely that information will be accepted as correct and the greater the loss to source reputation. We present The Illiterate Edi- Tor (IllEdit), a content-agnostic, metadata-driven classica- Tion approach to Wikipedia revert detection. Our primary contribution is in building a metadata-based feature set for detecting edit quality, which is then fed into a Support Vec- Tor Machine for edit classication. By analyzing edit histo- ries, the IllEdit system builds a prole of user behavior, es- Timates expertise and spheres of knowledge, and determines whether or not a given edit is likely to be eventually re- verted. The success of the system in revert detection (0.844 F-measure) as well as its disjoint feature set as compared to existing, content-analyzing vandalism detection systems, shows promise in the synergistic usage of IllEdit for increas- ing the reliability of community information. Copyright 2010 ACM.