Commodity Information Collection and System Construction of e-Commerce Platform Based on Network Data

Abstract

In order to quickly and accurately collect the massive commodities and corresponding transaction data of large-scale e-commerce platforms, and improve the ability of data analysis and mining, this paper proposes a platform commodity information collection system based on splash technology. The system prerenders the javascript code in the product page, combined with the Scrapy crawler framework, to realize a system that quickly and effectively collects product data from different platforms, and uses "mobile phone"as the retrieval keyword to verify the designed system, respectively. The experimental results show that the system can effectively collect up to 60,000 comments and 6,000 system requests. Conclusion. The platform commodity information collection system based on splash technology has certain application value and promotion for the commodity data collection of different platforms of e-commerce.