The Implementation of Stream Architecture for Handling Big Data Velocity in Social Media

Abstract

Big data is a term of complex data and difficult to process. It consists of several characteristics called 6 Vs. Many applications generate huge data and grow rapidly in seconds. This kind of data comes from many sources such as social media, Internet of Things, log system, e-commerce and soon. This rapid data should be handled with a different approach in big data solutions. This paper proposes to create stream architecture for big data velocity with open source technologies such as Apache Kafka and NoSQL database. The implementation is to handle massive incoming data from social media with specific keywords from Twitter and ingested to NoSQL Database though stream architecture.Historical data then processed to gain valuable insight for better information.