Big data: Challenges, opportunities and cloud based solutions

Abstract

We are living in an era of information explosion. There are challenges with large and complex amount of data generated every day by social networks, wikis, blogs, emails, traffic system, bridges, airplanes and engine, satellites and weather sensors. 90% of current data in the world has been created in the last two years. Our smart planet becomes more and more intelligent. Besides the challenges posed by such vast amount of data including storage, search, sharing, analysis, and visualization, there are also much opportunities for the world as it becomes more and more digitalized. This study presents Big Data and highlights its key concepts and state-of-the-art implementation as well as research challenges and suggests research directions for future. IT log analytics, Fraud detection pattern, social media pattern and modeling and management patterns are some of opportunities. Hadoop is a cloud based and open source solution for Big Data Analytics which has been written by java. Hadoop solution is currently still immature. In this paper, three topics are suggested for research direction: Security issues in Big Data, context-aware information retrieval, and integrating ontology with Big Data.