A distributed item based similarity approach for collaborative filtering on hadoop framework

Abstract

Now a day’s multiple website provides millions of product to their on-line users. Due to the interaction of millions customer with the e-commerce websites creates massive volume of data. Recommendation system is a dynamic data capturing system filters massive volume of information generated through the interaction of users to web-portals & generate suggestion that fits the user expectations. Recommendation framework is data filtering tools that make use of algorithms and user rating data to recommend the most relevant items to a particular user. Collaborative filtering is one of the successful techniques in the recommendation system to recommend the top N-item. In the majority of the Recommendation framework information sparsity, high dimensionality, adaptability which are normal issue in the RS domain have adversely influence the exhibition of CF. The proposed system is developed to resolve the mentioned problems in most of the recommendation system using item based similarities approach in CF with the help of user sub space clustering approach on hadoop framework. In the proposed system user subspaces formed by considering the interest of the users in the items like Interested, Neither Interested nor Uninterested (NIU), and Uninterested. After the user subspace clustering the neighbor item tree is constructed. To find out the similarities between the items the similarities measures is developed from the neighbor item tree. It observed that in traditional item-based collaborative filtering method requires more computational cost but the computation of item similarities is performed off line & computational cost required for on line prediction is less. In proposed work to improve the computation speed of off line item-item similarity the computation is performed on various nodes in cluster in the hadoop distributed system. The similarity between the two items is used to predict the rating provided by user on the target items. The proposed method tested on the Movie lens 100K, Movie lens 1M in order to make comparisons with the existing techniques. The proposed method improves the performance of the recommendation systems by resolving the issues like scalability, high dimensionality, data sparsity etc.