Big data analytics for network congestion management using flow-based analysis

Abstract

Due to explosive growth of traffic volume, it is hard to accumulate Internet traffic on a single machine. In this paper, a Hadoop-based traffic analysis system accepts input from multiple data traces. Hadoop facilitates scalable data processing and storage services on a distributed computing system. This system accepts input of large scales of trace file generated from traffic measurement tool like Wireshark– identifies flows running on the network from this trace file. Characteristics of flow describe the pattern of network traffic; it helps network operator understand network capacity planning, traffic engineering, and fault handling. The main objective is to design and implement a traffic flow identification system using Hadoop. The traffic flow identification system will be very useful for network administrator to monitor faults and also to plan for the future.