The Impact of Prompting Techniques on the Security of the LLMs and the Systems to Which They Belong

Abstract

Large language models have demonstrated impressive capabilities. The recent research conducted in the field of prompt engineering showed that their base performance is just a glimpse of their full abilities. Enhanced with auxiliary tools and provided with examples of how to solve the tasks, their adoption into our applications seems trivial. In this context, we ask an uncomfortable question. Are the models secure enough to be adopted in our systems, or do they represent Trojan horses? The idea of prompt injection and jailbreak attacks does not seem to bother the adopters too much. Even though there are a lot of studies that look into the benefits of the prompting techniques, none address their possible downside in regard to the security. We want take a step further and investigate the impact of the most popular prompting techniques on this aspect of large language models and implicitly the systems to which they belong. Using three of the most deployed GPT models to date, we conducted a few of the most popular attacks in different setup scenarios and demonstrate that prompting techniques can have a negative impact on the security of the LLMs. More than that, they also expose other system components that otherwise would have been less exposed. In the end, we try to come up with possible solutions and present future research perspectives.