RemOrphan: Object Storage Sustainability Through Removing Offline-Processed Orphan Garbage Data

Abstract

Nowadays, extremely large amounts of structured and unstructured types of data are stored in public, private, and hybrid cloud storage using object storage systems. Among these, storing multimedia data such as image, video, and audio pose unique challenges and long-Term effects on object storage Sustainability. Three such challenges are smoother and more efficient video streaming, middleware placement for media processing, and lastly, management of orphan garbage data. In order to tackle these challenges, this paper presents a generalized architecture for smooth and efficient management as well as retrieval of multimedia data in cloud systems. To do so, first, we propose a new middleware package in the object server for supporting smooth video streaming and on-demand playable video segments. Here, we demonstrate that video segment download time improves by up to 30% when segmentation is done in the object server rather than in the proxy server. After, we focus on how to find orphan garbage data on media cloud storage and to what extent they can hamper data retrieval. Specifically, we present a generalized architecture named 'RemOrphan' for detecting the orphan garbage data using OpenStack Swift hash Ring and scripts. We deploy a private media cloud SPMS and find that around 35% data can be orphan garbage data. Due to the huge amount of orphan data, rsync replication needs higher time and more network overhead which hampers the system's sustainability. We lower around 25% sync delay and 30% network overhead after deploying a deletion daemon to remove the orphan garbage data.