garak is an open-source LLM vulnerability scanner, whose full name is Generative AI Red-teaming & Assessment Kit. It is mainly used to evaluate the security of models or systems, helping to improve LLM security by probing for vulnerabilities.
Main capabilities: garak includes dozens of plugins and thousands of prompts, and can perform security testing on large language models. Its goal is to raise the LLM security poverty line, so as open-source software it is continuously updated and jointly maintained by NVIDIA and the community.
How to use: garak provides a command-line interface, and the tool strives to maintain a friendly command-line experience. Users can refer to the user guide and reference documentation for detailed support. Developers are active in the garak Discord community, and GitHub issues are also actively responded to and handled. For further help, you can contact us through the provided NVIDIA email.
Applicable scenarios: suitable for scenarios that require evaluating the security of LLMs or systems. Many companies have built successful products based on garak, and it has also been recommended by independent articles as an industry standard.
Limitations: currently garak is supported by NVIDIA, but as a research-led project, it has not yet been featured on NVIDIA's homepage. The website content uses the Apache 2.0 license.