Large language models in information security and penetration testing: a systematic review of application possibilities

Abstract

The development of artificial intelligence technologies, in particular, large language models (LLM), has led to changes in many areas of human life and activity. Information security (IS) has also undergone significant changes. Penetration testing (pentest) allows evaluating the security system in practice in “combat” conditions. LLMs can take practical security analysis to a qualitatively new level in terms of automation and the ability to generate non-standard attack patterns. The presented systematic review is aimed at determining the already known ways of applying LLM in cybersecurity, as well as identifying “blank spots” in the development of technology. The selection of literature sources was carried out in accordance with the multi-stage PRISMA guidelines based on the analysis of abstracts and keywords of publications. The resulting sample was supplemented using the “snowball” method and manual search of articles. The total number of publications was 50 works from January 2023 to March 2024. The conducted research allowed to analyze the ways of using LLM in the field of information security (goal setting and decision-making support, pentest automation, security analysis of LLM models and program code), determine the LLM architectures (GPT-4, GPT-3.5, Bard, LLaMA, LLaMA 2, BERT, Mixtral 8×7B Instruct, FLAN, Bloom) and software solutions based on LLM used in the field of information security (GAIL-PT, AutoAttacker, NetSecGame, Cyber Sentinel, Microsoft Counterfit, GARD project, GPTFUZZER, VuRLE), to establish limitations (finite “lifetime” of data for LLM training, insufficient cognitive abilities of language models, lack of independent goal setting and difficulties in adapting LLM to new task parameters), identify potential growth points and development of technology in the context of cyber defense (elimination of “hallucinations” of models and ensuring protection of LLM from jailbreaks, implementation of integration of known disparate solutions and software automation of tasks in the field of information security using LLM). The presented results can be useful in developing theoretical and practical solutions, educational and training datasets, software packages and tools for penetration testing, new approaches to building LLM and improving their cognitive abilities, taking into account aspects of working with jailbreaks and “hallucinations”, as well as for independent further multilateral study of the issue.