RESTTESTGEN: Automated Black-Box Testing of RESTful APIs

Abstract

RESTful APIs (or REST APIs for short) represent a mainstream approach to design and develop Web APIs using the REpresentational State Transfer architectural style. When their source code is not (or just partially) available or the analysis across many dynamically allocated distributed components (typical of a micro-services architecture) poses obstacles to white-box testing, black-box testing becomes a viable option. Black-box testing, in fact, only assumes access to the system under test with a specific interface. This paper presents RESTTESTGEN, a novel approach to automatically generate test cases for REST APIs, based on their interface definition (in Swagger). Input values and requests are generated for each operation of the API under test, with the twofold objective of testing nominal execution scenarios and of testing error scenarios. Two distinct oracles are deployed to detect when test cases reveal implementation defects. Our empirical investigation shows that this approach is effective in revealing actual faults on 87 real-world REST APIs.