The schema last approach to data fusion

Abstract

Big Data presents new challenges that require new and novel approaches in order to resolve the problems associated with the variability and variety of data obtained from multiple sources. This paper focuses on how to manage variety and the eclectic nature of big data using a technique known as 'Schema Last'. The 'Schema Last' approach is a frame work which defers the application of a descriptive model until it is required. This paper also provides a formal definition of the 'Schema Last' methodology and demonstrates the effectiveness over the more traditional Extract- Transform-Load methodologies employed in many organizations. The 'Schema Last' approach can be used as input to Map Reduction, Index creation and various data mining techniques. Ultimately, the Schema Last approach provides the frame-work to 'fuse' semistructured data into a single coherent view.