Research on Optimized Storage and Analysis System of Web Log Based on Django's MVC Framework

Abstract

Association rule analysis algorithm is widely used in Web log analysis, but the existing association rule analysis algorithm will significantly reduce the analysis and mining performance when the amount of Web log is relatively large. This paper proposes an improved clustering algorithm, which first clusters users with the same interests and hobbies, and then mines association rules for users in the same category, thereby reducing data dispersion. Based on Django's MVC framework, it optimizes the storage and storage of Web logs. In the analysis part, users can configure the support and confidence of association rule mining through the front-end, and at the same time query the results of mining through Hive, and use encryption algorithms in the data transmission process to ensure data security.