An efficient data access method exploiting quadtrees on MapReduce frameworks

Abstract

Due to the advance of diverse techniques such as social networks and sensor networks, the volume of data to be processed has rapidly increased. After Google proposed the MapReduce framework which processes big data using large clusters of commodity machines, the MapReduce framework is considered as an effective processing paradigm for a massive data set. However, in the view of the performance, a problem of the MapReduce framework is that an efficient access method (i.e., an index) is not supported. Thus, whole data should be retrieved even though a user wants to access a small portion of data. In this paper, we propose an efficient method constructing quadtrees on the MapReduce framework. Our technique reduces the index construction time utilizing a sampling technique to partition a data set. In addition, using the constructed quadtree as well as the MapReduce framework, a subset of data to be retrieved is easily identified and is processed in parallel. Our experimental result demonstrates the efficiency of our proposed algorithm with diverse environments. © Springer-Verlag 2013.