AUSTA | Adversarial Intelligence

Open-Weight Guardrails

Llama Prompt Guard 2: 86M vs 22M, the 512-Token Window, and How to Test It

Meta's Llama Prompt Guard 2 is the open-weight prompt injection classifier many self-hosted stacks start with. The two sizes differ in ways the headline AUC numbers hide, every input is cut at 512 tokens, and the model flags only one kind of injection. Here are the numbers from Meta's model card and a test plan for October 2026.

By Austa · Published · Facts checked against vendor sources as of October 2026 · ~9 min read

TL;DR (October 2026):

  • Llama Prompt Guard 2 came out on April 29, 2025 alongside LlamaFirewall and Llama Guard 4. As of October 2026 it's still the current Prompt Guard release in Meta's PurpleLlama repository.
  • Both sizes are binary classifiers (benign or malicious) with a 512-token context window. Longer input has to be split and scanned in segments.
  • 86M (mDeBERTa-base): AUC 0.998 English, 0.995 multilingual, 97.5% recall at 1% false positive rate, 92.4 ms per 512-token classification on an A100.
  • 22M (DeBERTa-xsmall): AUC 0.995 English, but only 0.942 multilingual, 88.7% recall at 1% FPR, 19.3 ms. Meta says it cuts latency and compute by 75%.
  • It flags text that explicitly tries to override instructions. It isn't designed to catch injected content that steers an agent without saying so.

Prompt Guard 2 is the usual starting point for teams that want injection detection they host themselves, with no per-call fees and no data leaving their environment. It's small, fast and well benchmarked. It's also narrow in ways the model card states plainly and integrations tend to forget. This page sets out the numbers from Meta's model card, explains how to choose between the two sizes, and lists what to test before you rely on it.

86M vs 22M: the numbers

MetricPrompt Guard 1Prompt Guard 2 86MPrompt Guard 2 22M
Base modelmDeBERTa-v3mDeBERTa-v3 baseDeBERTa-v3 xsmall
Backbone parameters86M86M22M
AUC, English (Meta private jailbreak benchmark)0.9870.9980.995
Recall at 1% FPR, English21.2%97.5%88.7%
AUC, multilingual0.9830.9950.942
Latency per classification (A100, 512 tokens)92.4 ms92.4 ms19.3 ms
AgentDojo attack prevention at 3% utility loss67.6%81.2%78.4%
Context window512 tokens512 tokens512 tokens
Labels3 classes (benign, injection, jailbreak)2 (benign, malicious)2 (benign, malicious)
Total parameters listed on Hugging Facen/aabout 279Mabout 71M

The last row trips people up when sizing hardware. "86M" and "22M" are backbone parameter counts, as the model card's table header says. The Hugging Face listing reports 278,810,882 and 70,830,722 total parameters, because the token vocabularies add large embedding tables: Microsoft's base model cards list 250K tokens for mDeBERTa-v3-base and 128K for DeBERTa-v3-xsmall. Size memory from the total.

For comparison, Meta's AgentDojo table lists other open classifiers at 22.2% (ProtectAI), 13.5% (Deepset) and 12.9% (LLM Warden) on the same metric. These are Meta's numbers on Meta's chosen benchmark. Treat them as a starting point, not as results for your traffic.

Which size should you run?

What Prompt Guard 2 is trained to flag, and what it isn't

The model card's definition is narrow on purpose. A prompt is malicious if it "explicitly attempts to override prior instructions," whether or not it's harmful and whether or not it's likely to work. Prompt Guard 1 had a broader "injection" label for content that might cause unintended instruction-following. Meta dropped it in version 2 because it found that objective "too broad to be useful."

That choice is why the false positive rate is low, and it's also the main limitation. In practice:

Meta also changed the tokenizer to resist adversarial tokenization tricks such as extra whitespace and fragmented tokens. That raises the bar for cheap evasion. It doesn't help with paraphrase, which is still the route that works for any classifier, as we explain in why prompt-injection filters keep failing.

The 512-token window and how to scan long inputs

Anything past 512 tokens is truncated unless you split it. Meta's guidance is to "split prompts into segments and scan them in parallel." Two details decide whether that holds up:

  1. Overlap the segments. If chunks sit back to back, a payload that crosses a boundary is split in two, and neither half may score. Overlap by at least the length of the longest payload you expect.
  2. Aggregate with max, not mean. One malicious segment in a long clean document should flag the whole input. Averaging dilutes it to nothing.
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

MODEL_ID = "meta-llama/Llama-Prompt-Guard-2-86M"
tok = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID).eval()
MAL = next(i for i, l in model.config.id2label.items() if l.upper() == "MALICIOUS")

def max_malicious_score(text, window=480, stride=240):
    ids = tok(text, add_special_tokens=False)["input_ids"]
    scores, start = [], 0
    while True:
        chunk = tok.decode(ids[start:start + window])
        enc = tok(chunk, return_tensors="pt", truncation=True, max_length=512)
        with torch.no_grad():
            probs = torch.softmax(model(**enc).logits, dim=-1)[0]
        scores.append(probs[MAL].item())
        if start + window >= len(ids):
            return max(scores)
        start += stride

A 480-token window with a 240-token stride means any payload of up to 240 tokens appears whole in at least one window, and each token is classified twice. Meta's inference utilities in the Llama cookbook do a similar job. Whichever you use, check that the windows actually overlap.

Picking a threshold

The model gives a malicious probability, and you set the threshold. Meta reports recall at 1% FPR because its custom energy-based loss pushes known injection patterns to the top of the score range, so a fairly high threshold still catches most of them. Don't copy a number from a blog post, this one included. Run 1,000 to 10,000 of your own benign messages through the model, set the threshold where false positives fall to the rate you can live with, then measure recall with your own attack set. Meta also recommends fine-tuning on domain-specific prompts. Support-desk text, code and tool-call JSON all shift the benign distribution.

Deployment notes

How to test a Prompt Guard 2 deployment

  1. Find which inputs are scanned (user turns, retrieved chunks, tool results) and whether long ones are split or silently truncated at 512 tokens.
  2. Put a known payload at token 600 of a long clean document. If it isn't flagged, nothing is being split.
  3. Put a payload across a chunk boundary to test for overlap.
  4. Rewrite detected payloads as calm statements of policy or fact, with no override wording, and measure whether the agent still acts on them.
  5. Run the same set in each language your users write in, and compare 22M against 86M if 22M is deployed.
  6. Measure what the agent does, not only what the classifier says. Detection is one layer among several, as in the 2026 LLM security checklist.

FAQ

What is the difference between Prompt Guard 2 86M and 22M?
The 86M model is built on mDeBERTa-base and scores 0.998 AUC in English, 0.995 multilingual, and 97.5% recall at 1% false positive rate, at 92.4 ms per classification on an A100. The 22M model is built on DeBERTa-xsmall and scores 0.995 AUC in English, 0.942 multilingual, and 88.7% recall at 1% false positive rate, at 19.3 ms. Meta says the 22M model reduces latency and compute costs by 75%.
What is the context length of Llama Prompt Guard 2?
Both Prompt Guard 2 models support a 512-token context window. Meta recommends splitting longer inputs into segments and scanning them in parallel. Use overlapping segments and take the maximum malicious score so a payload on a segment boundary is not missed.
Does Prompt Guard 2 detect indirect prompt injection?
It detects text that explicitly attempts to override prior instructions, whether that text arrives from a user or from third-party content. Meta removed Prompt Guard 1's broader injection label because it was too broad to be useful, so injected content that steers an agent without explicit override language is outside what the model is trained to flag.
When was Llama Prompt Guard 2 released?
Meta released Llama Prompt Guard 2 86M and 22M on April 29, 2025, together with Llama Guard 4 and LlamaFirewall in the PurpleLlama repository. As of October 2026 it remains the current Prompt Guard release in that repository.
What languages does Prompt Guard 2 support?
Both models were evaluated for attack detection in English, French, German, Hindi, Italian, Portuguese, Spanish and Thai. Only the 86M model uses a multilingual base model; the 22M model has a larger performance gap on multilingual data because no multilingual DeBERTa-xsmall exists.
What license is Prompt Guard 2 under?
The Hugging Face model pages carry the Llama 4 Community License, with a license effective date of April 5, 2025, and access to the weights is gated behind accepting it. The base models it builds on are MIT-licensed Microsoft DeBERTa models.

Meta's full benchmark tables and limitations are in the Llama Prompt Guard 2 model card.