Google Research has announced Project ShieldAI, an automated red-teaming framework designed specifically for testing the security of large language models (LLMs). The framework can detect prompt injection attacks, data leakage vulnerabilities, and jailbreak attempts with high accuracy.
ShieldAI uses a combination of adversarial testing, fuzzing, and behavioral analysis to probe LLMs for security weaknesses. Early testing has revealed that over 60% of popular open-source LLMs contain at least one critical vulnerability that could be exploited.
The framework is being open-sourced to help the AI community build more secure models.