Engineering choices for open world provenance

Abstract

This work outlines engineering decisions required to support a provenance system in an open world where systems are not under any common control and use many different technologies. Real U.S. government applications have shown us the need for specialized identity techniques, flexible storage, scalability testing, protection of sensitive information, and customizable provenance queries. We analyze tradeoffs for approaches to each area, focusing more on maintaining graph connectivity and breadth of capture, rather than on finegrained/detailed capture as in other works. We implement each technique in the PLUS system, test its real-time efficiency, and describe the results.