Partial migration in an 8mm tape based tertiary storage file system and its performance evaluation through satellite image processing applications

Abstract

Recent attention on global environmental changes has stimulated the development of large scale global information systems. Satellite images play a very important role for understanding these global changes. However, the data size is very large and current commercial hierarchical file management systems are not efficient enough to handle such huge data files. Migration of a whole file from tape to disk takes a very long time. Usually users are not interested in a whole image but in only a small portion of it. Thus, there is no need for full migration. We designed and implemented a partially migratable file system based on 8mm tape robotics. The file system migrates only the necessary portion of a file onto the disk. Two real application programs: radiometric/geometric correction and NDVI (Normalized Difference Vegetation Index) generation, were chosen and executed using our experimental file system. Large performance improvements were achieved when compared to the conventional file level migration scheme.