Automated red teaming for your LLM app

Show HN: Automated red teaming for your LLM app

Operating in show hn launch, Automated red teaming for your LLM app operates.

Profile

Hi HN,I built this open-source LLM red teaming tool based on my experience scaling LLMs at a big co to millions of users... and seeing all the bad things people did.How it works:- Uses an unaligned model to create toxic inputs- Runs these inputs through your app using different techniques: raw, prompt injection, and a chain-of-thought jailbreak that tries to re-frame the request to trick the LLM.- Probes a bunch of other failure cases (e.g. will your customer support bot recommend a competitor? Does it think it can process a refund when it can't? Will it leak your user's address?)- Built on top of promptfoo, a popular eval toolOne interesting thing about my approach is that almost none of th

Its activity falls under show hn launch; the profile below is assembled from public directory data.

It is recorded at the Show HN launch stage.

Website: www.promptfoo.dev/docs/red-team