Detection of Fraudulent Credit Card Transactions in Real Time Using SparkML and Kafka

Abstract

As a result of the rapid expansion and development of E-Commerce, the use of credit card for online shopping and purchases has increased dramatically, and this rapid increase has led to a surge in fraudulent credit cards. Now being the most popular mode of payment in both offline and online purchases, credit card fraud has become a major menace. With the recent advancement in artificial intelligence, machine learning, data mining, sequence alignment, genetic programming, and fuzzy logic, innovative ways for detecting various credit card fraudulent activities have evolved. In our work, we have utilized SparkML and Scikit-Learn to develop and train several machine learning models for distinguishing fraudulent and legitimate transactions. We have also employed several data preprocessing techniques, such as class imbalance removal, which were implemented using various Spark packages. These machine learning models are then utilized to make predictions on Kafka-generated real-time data streams. Finally, we have used Streamlit in order to build an interface for displaying these predictions in real time.