Pilot Study on the Intercalibration of a Categorisation System for FAIRer Digital Objects Related to Sensitive Data in the Life Sciences

Abstract

Sharing sensitive data is a specific challenge for research infrastructures in the field of life sciences. For that reason a toolbox has been developed, providing resources for researchers who wish to share and use sensitive data, to support the workflows for handling these kinds of digital objects. Common and community approved annotations are required to be compliant with FAIR principles (Findability, Accessibility, Interoperability, Reusability). The toolbox makes use of a tagging (categorisation) system, allowing consistent labelling and categorisation of digital objects, in terms relevant to data sharing tasks and activities. A pilot study was performed within the Horizon 2020 project EOSC-Life, in which 2 experts from 6 life sciences research infrastructures were recruited to independently assign tags to the same set of 10 to 25 resources related to sensitive data management and data sharing (in total 110). Summary statistics of agreement and observer variation per research infrastructure are provided. The pilot study has shown that experts were able to attribute tags but in most cases with a considerable observer variation between experts. In the context of CWFR (Canonical Workflow Frameworks for Research), this indicates the necessity for careful definition, evaluation and validation of parameters and processes related to workflow descriptions. The results from this pilot study were used to tackle this issue by revising the categorisation system and providing an updated version.