End-to-end model-transformation comprehension through fine-grained traceability information

Abstract

The construction and maintenance of model-to-model and model-to-text transformations pose numerous challenges to novice and expert developers. A key challenge involves tracing dependency relationships between artifacts of a transformation ecosystem. This is required to assess the impact of metamodel evolution, to determine metamodel coverage, and to debug complex transformation expressions. This paper presents an empirical study that investigates the performance of developers reflecting on the execution semantics of model-to-model and model-to-text transformations. We measured the accuracy and efficiency of 25 developers completing a variety of traceability-driven tasks in two model-based code generators. We compared the performance of developers using ChainTracker, a traceability analysis environment developed by our team, and that of developers using Eclipse Modeling. We present statistically significant evidence that ChainTracker improves the performance of developers reflecting on the execution semantics of transformation ecosystems. We discuss how developers supported by off-the-shelf development environments are unable to effectively identify dependency relationships in nontrivial model-transformation chains.