AI-Generated Phishing: Combining Human Behavior with Post Content to Assess Susceptibility

Abstract

As Large Language Models (LLMs) generate increasingly sophisticated text-image pairs, the risk of LLMs being used for phishing is growing, posing significant challenges for Internet security. We conduct a detailed, IRB-approved study of pairs (uc,pc) where uc is a user property (e.g., demographic or behavioral) and pc is a property of the post content (e.g., topic, emotion, and recency). Unlike past research that focuses either on user properties alone or post content alone, we conduct a comprehensive statistical analysis of more than 1,400 hypotheses involving such (uc,pc) pairs and identify the conditions on user-content pairs that have a significantly higher click probability compared with pairs that do not satisfy the condition. The results include highly nuanced findings connecting susceptibility to phishing with the combination of user properties and post content properties. They also cast some light on contradictory findings from prior work on understanding human susceptibility to phishing.