Behaviour-driven development applied to the conformance testing of INSPIRE web services

Abstract

The implementation of the INSPIRE directive requires to check the conformity of a large number of network services with the implementing rules of INSPIRE. The evaluation whether a service is fully conformant with INSPIRE is complex and requires the use of specialized testing tools that should report how verification has been made and should identify non-conformances. The use of these tools requires a high degree of technical knowledge. This fact makes very difficult for non-technical stakeholders (end users, managers, domain experts, etc.) to participate effectively in conformance testing, hinders stakeholders understanding of the causes and consequences of non-conformant results and may cause in some stakeholders disinterest in conformance testing. This work explores the suitability of a behaviour-driven development (BDD) approach to the conformance testing of OGC Web services in the context of the INSPIRE directive. BDD emphasizes the participation of non-technical parties in the design of acceptance tests by means of automatable abstract tests expressed in a human readable format. Using this idea as base, this work describes a BDD based workflow to derive abstract test suites and executable test suites from INSPIRE implementation requirements that can be written in the language used by non-technical stakeholders. This work also analyses if BDD and popular BDD tools, such as Gherkin and Cucumber, are compatible with ISO 19105:2000 testing methodology. As demonstration, we present an online conformance tool for INSPIRE View and Discovery services that executes BDD test suites.