VeriSwarm
About
DocsPricingAgent Skill
LoginRegister
  1. Home
  2. /Learn
  3. /Prompt injection detection ai agents
VeriSwarm

Product

  • Pricing
  • Docs
  • API
  • Agent Skill
  • OATS Spec

Trust

  • Trust Center
  • Security
  • Compliance
  • Status
  • Changelog

Company

  • About
  • Blog
  • Open Source
  • Investors
  • Press

Legal

  • Terms
  • Privacy
  • SLA
  • DPA
Runtime Security

Prompt Injection Detection for AI Agents

Scan each turn for injection before a tool fires, not just the inbound prompt. VeriSwarm Guard runs two detection layers — fast structural pattern analysis, then a self-hosted DeBERTa classifier — entirely inside its own infrastructure. No third-party detection API, no request payload leaving your boundary to get a verdict.

Why the prompt isn’t where injection detection should stop

A hardened system prompt and a clean red-team eval don’t stop the attack that actually matters in production. An agent reads a support ticket, and a hidden instruction inside the ticket body tells it to query a billing record and forward the result to a webhook. The model never violates its system prompt — it calls a tool. The tool runs. The data leaves. Detection has to run on the text that’s about to drive a tool call, not just on what the user typed.

The full argument for why prompt-layer defenses miss this attack path is in Prompt Injection Doesn’t Stop at the LLM. It Flows Through Tool Calls..

Two detection layers

Structural analysis

Regex-based, zero model cost. Catches protocol-level manipulation — chat-format delimiter injection (fake <|im_start|>, [INST], and similar markers meant to confuse the model about where one message ends and another begins) and payloads smuggled inside base64 or similar encodings. These are structural signals, not keyword matches, so they fire on protocol tricks a wordlist would miss entirely.

Self-hosted ML classifier

A DeBERTa-based model (protectai/deberta-v3-base-prompt-injection-v2) loaded once and served locally via ONNX Runtime. It reads intent rather than surface patterns, which is what catches a rephrased attack, a multilingual injection, or an evasion technique that’s new enough not to match any known pattern. If the model fails to load, detection falls back to structural analysis rather than failing open.

The two layers run in order — structural first, because it’s cheaper and catches the loudest attacks outright, then the classifier for everything structural analysis doesn’t already flag.

The API

POST /v1/suite/guard/scan takes a body of { "text": "...", "scan_injection": true, "scan_moderation": true } and returns { "flagged": bool, "injection": {...}, "moderation": {...}, "summary": "..." }. The injectionobject carries the matched category, a confidence score, and which layer produced the verdict. Call it on any text that could drive a tool call before that call fires — a user message, content retrieved from a knowledge base, a webhook payload from an upstream integration.

Wired into Guard Proxy automatically

Guard Proxy is VeriSwarm’s transparent MCP interception layer, and injection scanning is a fixed step in its pipeline: every response coming back from a tool server is checked for injection attempts before it reaches the agent, alongside PII tokenization and policy enforcement. Point your agent’s MCP client at the proxy instead of the tool server directly — zero agent-side code changes.

Defense in depth: session-level detection

Per-turn scanning catches an injection attempt that shows up in a single message. Some attacks don’t — they accumulate risk across a whole conversation, a slow-drip attempt to extract a system prompt or exfiltrate a canary value one small step at a time. Session Sentry runs as a complementary layer that scores risk across the session rather than per-turn, alongside — not instead of — the structural and DeBERTa layers described above.

Related: the data those tool calls can leak

Injection detection stops a manipulated tool call from firing. It’s a related but distinct problem from what happens when a legitimatetool call carries PII it shouldn’t — covered in How to Prevent an AI Agent from Leaking Data. Guard runs both checks in the same pipeline.

Frequently asked questions

Why scan for prompt injection before a tool fires, not just at the prompt?

Because the payload doesn't have to look like a jailbreak to be dangerous. A hidden instruction in a support ticket or a scraped web page can leave the model's own system prompt untouched and still change which tool gets called, with which arguments, on whose data. By the time the call reaches the tool server, the model is out of the loop and the protocol is what executes. Scanning the response before the tool call fires catches that class of attack; scanning only the inbound prompt does not.

What are the two detection layers?

Layer one is structural analysis — regex-based detection of protocol-level manipulation like chat-format delimiter injection (fake <|im_start|> or [INST] markers meant to confuse the model about message boundaries) and base64/encoding smuggling. It's fast and catches attacks that don't even need semantic understanding. Layer two is a machine-learning classifier that reads the actual intent of the text, catching rephrased attacks, multilingual injections, and novel evasion techniques that pattern-matching alone misses.

What model powers the ML layer, and where does it run?

A DeBERTa-based classifier (protectai/deberta-v3-base-prompt-injection-v2), loaded locally and served via ONNX Runtime inside VeriSwarm's own infrastructure. There's no call out to a third-party injection-detection API and no request payload leaves your tenant boundary to get a verdict — detection is fully self-contained. If the model fails to load for any reason, detection falls back to structural analysis only rather than failing open.

Is there an API I can call directly?

Yes — POST /v1/suite/guard/scan takes { "text": "...", "scan_injection": true, "scan_moderation": true } and returns { "flagged": bool, "injection": {...}, "moderation": {...}, "summary": "..." }. The injection field carries the category, a confidence score, and which layer (structural or ml) produced the verdict. Call it on any untrusted text — a user message, RAG content, a webhook payload — before it reaches a step that can trigger a tool call.

Do I have to wire this up myself for every tool call?

Not if you're already running Guard Proxy. Guard Proxy is VeriSwarm's transparent MCP interception layer, and prompt-injection scanning is one step in its fixed pipeline: every response coming back from a tool server is checked for injection attempts before it reaches the agent, alongside PII tokenization and policy enforcement — zero agent-side code changes. The direct scan-session API is there for cases where you want to check text yourself, outside the proxy path.

Does VeriSwarm also detect multi-turn or session-level attacks?

Yes, as a separate, complementary layer. Session Sentry accumulates risk signals across a whole conversation — canary-token exposure, system-prompt extraction attempts — rather than scoring a single turn in isolation, which catches a slow-drip exfiltration attempt that no individual message would trip on its own. It runs alongside the per-turn structural and DeBERTa layers, not instead of them.

What plan is prompt injection detection on?

Guard — including scan-based injection detection and Guard Proxy's automatic scanning — is a Max-plan feature. Gate's free tier gets you trust scoring and event visibility first; Guard is where the enforcement turns on.

Scan before the tool call fires

Guard’s injection detection — the API, and Guard Proxy’s automatic scanning — is a Max-plan feature. Gate’s free tier gives you trust scoring and event visibility first.

Try the demoStart free