MithriLog: Near-storage accelerator for high-performance log analytics

Abstract

This paper presents MithriLog, a log analytics platform with nearstorage accelerators for high-performance, cost- and power-efficient unstructured log processing. MithriLog offloads log analytics queries to an efficient near-storage FPGA implementation of a token querying engine, which can take advantage of the high internal bandwidth of storage devices within the available chip resource limitations. This engine is flexible enough to handle complex queries including template search based on user-defined tree-based template libraries, as well as concurrent execution of multiple queries. MithriLog also uses a log-optimized version of a simple, highthroughput compression algorithm in order to further improve the effective bandwidth of backing storage. Evaluated with complex search queries on large real-world log datasets, MithriLog achieves an order of magnitude higher performance over software systems, even against more expensive machines with enough DRAM to stage the entire dataset. Furthermore, MithriLog delivers constant performance regardless of query complexity, resulting in further improved performance benefits with more complex queries. By replacing costly DRAM with storage and power-hungry CPU threads with FPGAs, MithriLog dramatically improves the cost-effectiveness and accessibility of log analytics.