Lowering data dimensionality in big data for the benefit of precision agriculture

Abstract

Predictive analytics can be used to make smarter decisions in farming by collecting real-time data on weather, soil and air quality, crop maturity and even equipment and labor costs and availability. This is known as precision agriculture. Big data is expected to play an important role in precision agriculture for managing real-time data analysis with massive streaming data. The data analysis efficiency and throughput would be a challenge with the massive increase in size of big data. The unstructured streaming data received from different agricultural sources would contain multiple dimensions and not the entire content is needed for performing analysis. The core data which is small but that alone enough to represent the entire content should be extracted. This paper explains how to systematically reduce the size of big data by applying a tensor based feature reduction model. The data decomposition and core value extraction is done with the help of IHOSVD algorithm. This way it reduces the overall file size by eliminating unwanted data dimensions. The time involved in data analysis and CPU usage will be significantly reduced when dimensionality reduced data is used in place of raw (unprocessed) data.