IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 1
Semantic-Aware Metadata Organization
Paradigm in Next-Generation File Systems
Yu Hua, Member, IEEE, Hong Jiang, Senior Member, IEEE, Yifeng Zhu, Member, IEEE,
Dan Feng, Member, IEEE, and Lei Tian
Abstract—Existing data storage systems based on the hierarchical directory-tree organization do not meet the scalability and
functionality requirements for exponentially growing datasets and increasingly complex metadata queries in large-scale, Exabyte-level
file systems with billions of files. This paper proposes a novel decentralized semantic-aware metadata organization, called SmartStore,
which exploits semantics of files’ metadata to judiciously aggregate correlated files into semantic-aware groups by using information
retrieval tools. The key idea of SmartStore is to limit the search scope of a complex metadata query to a single or a minimal number of
semantically correlated groups and avoid or alleviate brute-force search in the entire system. The decentralized design of SmartStore
can improve system scalability and reduce query latency for complex queries (including range and top-k queries). Moreover, it is also
conducive to constructing semantic-aware caching, and conventional filename-based point query. We have implemented a prototype
of SmartStore and extensive experiments based on real-world traces show that SmartStore significantly improves system scalability
and reduces query latency over database approaches. To the best of our knowledge, this is the first study on the implementation of
complex queries in large-scale file systems.
Index Terms—File systems, Metadata management, Scalability, Performance evaluation.
✦
1 INTRODUCTION
Fast and flexible metadata retrieving is a critical requirement
in the next-generation data storage systems serving high-end
computing. As the storage capacity is approaching Exabytes
and the number of files stored is reaching billions, directory-
tree based metadata management widely deployed in conven-
tional file systems [1] can no longer meet the requirements
of scalability and functionality. For the next-generation large-
scale storage systems, new metadata organization schemes are
desired to meet two critical goals: (1) to serve a large number
of concurrent accesses with low latency and (2) to provide
flexible I/O interfaces to allow users to perform advanced
metadata queries, such as range and top-k queries, to further
decrease query latency.
Although existing distributed database systems can work
well in some real-world data-intensive applications, they are
inefficient in very large-scale file systems due to four main
reasons. First, as the storage system is scaling up rapidly,
a very large-scale file system, the main concern of this
paper, generally consists of thousands of server nodes, con-
tains trillions of files and reaches exabyte-data-volume (EB).
Unfortunately, existing distributed databases fail to achieve
efficient management of petabytes of data and thousands of
concurrent requests [2]. Second, for heterogeneous execution
environments, devices of file systems are heterogeneous, such
as supercomputers, clusters of PCs via Ethernet, InfiniBand
Y. Hua, D. Feng and L. Tian are with the School of Computer Science
and Technology, Wuhan National Lab for Optoelectronics, Huazhong Uni-
versity of Science and Technology, Wuhan, China. E-mail: {csyhua, dfeng,
ltian}@hust.edu.cn
H. Jiang is with the Department of Computer Science and Engineering,
University of Nebraska-Lincoln, Lincoln, NE, USA. E-mail: jiang@cse.unl.edu
Y. Zhu is with the Department of Electrical and Computer Engineering,
University of Maine, Orono, ME, USA, Email: zhu@eece.maine.edu
and Fibers, and cloud storage via Internet,. Instead, DBMS
often assumes homogeneous and dedicated high-performance
hardware devices. Recently, the database research community
has become aware of this problem and agreed that existing
DBMS for general-purpose applications would not be a “one
size fit all” solution [3]. This issue has also been observed
by file system researchers [4]. Third, for heterogeneous data
types, their metadata in file systems are also heterogeneous.
The metadata may be structured, semi-structured or even un-
structured, since they come from different operational system
platforms and support various real-world applications. This
is often ignored by existing database solutions. Last but not
the least, existing file systems only provide filename-based
interface and allow users to query a given file, which severely
limits the flexibility and ease of use of file systems.
In the next-generation file systems, metadata accesses
will very likely become a severe performance bottleneck as
metadata-based transactions not only account for over 50%
of all file system operations [5] but also result in billions of
pieces of metadata in directories. Given the sheer scale and
complexity of the data and metadata in such systems, we must
seriously ponder a few critical research problems [6] such as
“How to efficiently extract useful knowledge from an ocean of
data?”, “How to manage the enormous number of files that
have multi-dimensional or increasingly higher dimensional
attributes?”, and “How to effectively and expeditiously extract
small but relevant subsets from large datasets to construct
accurate and efficient data caches to facilitate high-end and
complex applications?”. We approach the above problems by
first postulating the following.
First, while a high-end or next-generation storage system
can provide a Petabyte-scale or even Exabyte-scale storage
capacity containing an ocean of data, what the users really
Digital Object Indentifier 10.1109/TPDS.2011.169 1045-9219/11/$26.00 © 2011 IEEE
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication.