Dimensionality Reduction using Machine Learning and Big Data Technologies

Abstract

Machine learning and big data models are most useful constraints in software technologies. But these systems need very less data at processing time, also technology wise data dimensionality increases day by day. Any algorithm applicable for high dimensional data requires more processing time and storage resources. The curse of dimensionality refers to all the problems that arise when working with data in the higher dimensions that did not exist in the lower dimensions. Our paper attempts to deal with the issue of safety for information at low dimensionality. Addressing this trouble is equivalent to addressing the safety problem of the hardware and software platform. Decision tree (DT) ML model is helpful for these dimensional and clustering problems. DTML model has been reduced the duplicate data size and clustering achieved efficiency 94.3% and reduction ratio by 32.4%..