Research and Design of Real Time Big Data Visualization Analysis Platform Based on Flink

Abstract

Real-time big data analysis provides a better value than standard offline big data analysis. Academic groups are still researching and exploring the technologies of real-time big data analysis. Due to the vast scale and poor transmission rate of big data, there are several impediments in the research process, making it hard to fully access the potential value of data.A real-time large data visualization analysis platform based on Flink is presented to overcome this challenge.The platform first collects data in real time and sends it to the message middleware Kafka, and then the big data analysis platform reads the data stream in Kafka in real time and transmits it to the multidimensional operator developed with Flink as the framework for real-time Calculate, and uses WebSocket technology to push the calculation results to the front end in real time, and perform visual display, enabling real-time monitoring and analysis of data flow.The experimental findings suggest that the real-time big data analysis platform based on the integration of Flink and Kafka can accurately, promptly, and clearly portray the real-time data flow process and visual analysis.