Computer software that parses electronic files is often vulnerable to maliciously crafted input data. Rather than relying on developers to implement ad hoc defenses against such data, the Language-theoretic security (LangSec) philosophy offers formally correct and verifiable input handling throughout the software development lifecycle. Whether developing from a specification or deriving parsers from samples, LangSec parser developers require wide-reach corpora of their target file format in order to identify key edge cases or common deviations from the format's specification. In this research report, we provide the details of several methods we have used to gather approximately 30 million files, extract features and make these features amenable to search and use in analytics. Additionally, we provide documentation on opportunities and limitations of some popular open-source datasets and annotation tools that will benefit researchers which need to efficiently gather a large file corpus for the purposes of LangSec parser development.
Mendeley helps you to discover research relevant for your work.
CITATION STYLE
Allison, T., Burke, W., Constantinou, V., Goh, E., Mattmann, C., Mensikova, A., … Timmaraju, V. (2020). Research report: Building a wide reach corpus for secure parser development. In Proceedings - 2020 IEEE Symposium on Security and Privacy Workshops, SPW 2020 (pp. 318–326). Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/SPW50608.2020.00066