Predicting File Lifetimes with Machine Learning

Abstract

In this article, we show how machine learning methods, namely random forests and convolutional neural networks, can be used to predict file lifetimes from their absolute path with a high reliability in an HPC filesystem context. The file lifetime is defined in this article as the time between the creation of the file and the last time it is read. Such results can be applied to the design of smart data placement policies, especially for hierarchical storage systems.