Agent-based big data mining

Abstract

Big data is the term used to describe a data, which is difficult to process, manage and analyze using traditional databases or data mining algorithms. Useful knowledge can be extracted from this big data with the help of data mining. Due to the volume, variety, and velocity of data, traditional techniques of data mining may be unsuitable to work with big data. As a result, there is a basic need to create powerful and productive enormous information mining methods. Characterization is one of the information mining strategies that is capable of processing a large amount of data and is growing in popularity. It is used to map a data item into one of several predefined classes or categories. Healthcare data is one form of big data not only for its sheer volume, but also for its complexity, diversity, and speed at which it is generated and must be managed. In this paper, we present the problem of mining the big data using software agent. The main goal of this paper is to develop and implement an agent-based big data classification model that can predict the severity of diabetes disease. Results proved that using agent technology in the preprocessing stage saved the memory storage from 8.66 TB to 5 GB memory space. The transfer data time is reduced from about 12 days to about 10 minutes after preprocessing data remotely using the agent. Regarding classification accuracy, the proposed model has proven 87% accuracy and 65% reliability.