Using MQTT and Node-RED to monitor the ATLAS Metadata Interface (AMI) stack and define metadata aggregation tasks in a pipelined way

Abstract

ATLAS Metadata Interface (AMI) is a generic ecosystem for metadata aggregation, transformation and cataloging. Each sub-system of the stack has recently been improved in order to acquire messaging/telemetry capabilities. This paper describes the whole stack monitoring with the Message Queuing Telemetry Transport (MQTT) protocol and Node-RED, a tool for wiring together hardware/software devices. Finally, this paper shows how Node-RED is used to graphically define metadata aggregation tasks, in a pipelined way, without introducing any single point of failure.