Large Language Models in Cybersecurity Threats, Exposure and Mitigation

This open access book provides cybersecurity practitioners with the knowledge needed to understand the risks of the increased availability of powerful large language models (LLMs) and how they can be mitigated. It attempts to outrun the malicious attackers by anticipating what they could do. It also alerts LLM developers to understand their work's risks for cybersecurity and provides them with tools to mitigate those risks.

The book starts in Part I with a general introduction to LLMs and their main application areas. Part II collects a description of the most salient threats LLMs represent in cybersecurity, be they as tools for cybercriminals or as novel attack surfaces if integrated into existing software. Part III focuses on attempting to forecast the exposure and the development of technologies and science underpinning LLMs, as well as macro levers available to regulators to further cybersecurity in the age of LLMs. Eventually, in Part IV, mitigation techniques that should allow safe and secure development and deployment of LLMs are presented. The book concludes with two final chapters in Part V, one speculating what a secure design and integration of LLMs from first principles would look like and the other presenting a summary of the duality of LLMs in cyber-security.

This book represents the second in a series published by the Technology Monitoring (TM) team of the Cyber-Defence Campus. The first book entitled "Trends in Data Protection and Encryption Technologies" appeared in 2023. This book series provides technology and trend anticipation for government, industry, and academic decision-makers as well as technical experts.

Part I: Introduction.- 1. From Deep Neural Language Models to LLMs.- 2. Adapting LLMs to Downstream Applications.- 3. Overview of Existing LLM Families.- 4. Conversational Agents.- 5. Fundamental Limitations of Generative LLMs.- 6. Tasks for LLMs and their Evaluation.- Part II: LLMs in Cybersecurity.- 7. Private Information Leakage in LLMs.- 8. Phishing and Social Engineering in the Age of LLMs.- 9. Vulnerabilities Introduced by LLMs through Code Suggestions.- 10. LLM Controls Execution Flow Hijacking.- 11. LLM-Aided Social Media Influence Operations.- 12. Deep(er)Web Indexing with LLMs.- Part III: Tracking and Forecasting Exposure.- 13. LLM Adoption Trends and Associated Risks.- 14. The Flow of Investments in the LLM Space.- 15. Insurance Outlook for LLM-Induced Risk.- 16. Copyright-Related Risks in the Creation and Use of ML/AI Systems.- 17. Monitoring Emerging Trends in LLM Research.- Part IV: Mitigation.- 18. Enhancing Security Awareness and Education for LLMs.- 19. Towards Privacy Preserving LLMs Training.- 20. Adversarial Evasion on LLMs.- 21. Robust and Private Federated Learning on LLMs.- 22. LLM Detectors.- 23. On-Site Deployment of LLMs.- 24. LLMs Red Teaming.- 25. Standards for LLM Security.- Part V: Conclusion.- 26. Exploring the Dual Role of LLMs in Cybersecurity: Threats and Defenses.- 27. Towards Safe LLMs Integration.

Andrei Kucharavy is the co-director of the Generative Learning Center at HES-SO Valais-Wallis. He holds a PhD from University of Paris-Sorbonne (2017), and is an engineer of Ecole Polytechnique (2013) and EPFL. Prior to this position he worked on counter-measures to the use of generative machine learning in offensive cyber-operations as a Distinguished Post-Doctoral Fellow at the Cyber-Defence Campus of armasuisse Science and Technology (S+T).

Octave Plancherel is a study coordinator at the Cyber-Defence Campus of armasuisse S+T. He holds a Bachelor (2022) degree in Business Informatics from the University of Fribourg.

Valentin Mulder is a Scientific Project Manager at the Cyber-Defence Campus of armasuisse S+T. He holds a Master (2022) degree in Legal Issues, Crime, and Security of Information Technologies from the University of Lausanne. Before his current position, he worked in the banking industry, particularly in the area of onlinefraud. In 2023, he co-edited the book “Trends in Data Protection and Encryption Technologies” published by Springer.

Vincent Lenders is the founding Director of the Cyber-Defence Campus from armasuisse S+T. He holds a Master (2001) and PhD (2006) degree in electrical engineering and information technologies from ETH Zurich. He has contributed to developing and implementing various national cyber strategies at the Swiss Government and has published more than 150 technical papers on cyber security, data science and networking. In 2023, he co-edited the book “Trends in Data Protection and Encryption Technologies” published by Springer.