Searching massive data streams using multipattern regular expressions

Abstract

This paper describes the design and implementation of lightgrep, a multipattern regular expression search tool that efficiently searches massive data streams. lightgrep addresses several shortcomings of existing digital forensic tools by taking advantage of recent developments in automata theory. The tool directly simulates a nondeterministic finite automaton, and incorporates a number of practical optimizations related to searching with large pattern sets.