Most of the money wasted on an LLM penetration test is wasted before testing starts, at scoping. Scope too narrow and you get a clean report on a chatbot while the agent wired into your database goes untested. Scope too broad and you pay for depth on components that were never exposed. Either way the problem is the same: the scope described a generic “AI app” instead of your architecture.
LLM applications are harder to scope than traditional web apps because the risk doesn’t live where you’d expect. Two chatbots that look identical to a user can have completely different attack surfaces depending on what’s behind them. One answers from a static prompt; the other retrieves from a corpus outsiders can write to and can call three internal APIs. Same UI, wildly different scope.
This guide walks through the information a good scope needs, the factors that actually drive cost, a checklist you can fill in before you talk to a vendor, and a sample statement of work you can adapt. If you want the methodology those tests run against, start with our LLM penetration testing overview.
Answer these before you scope anything
Scoping an LLM engagement is mostly a matter of understanding your own architecture. Five questions do most of the work:
1. What model, and where does it run? A self-hosted open-weight model, a hosted provider API (OpenAI, Anthropic, etc.), or a mix. This determines what’s in scope to test versus what belongs to the provider. In almost every case the exploitable risk is in how you wired the model in, not the base model itself, so “we just use GPT” does not mean there’s nothing to test.
2. Is there retrieval, and who can influence the corpus? If your app does RAG, the corpus is attack surface. The critical follow-up is who can write to it. A corpus built only from vetted internal docs is one risk profile; one that ingests user uploads, support tickets, or scraped web content is a much bigger one, because an attacker can plant content that hijacks the model on someone else’s turn. This is the difference between a small scope and a real one.
3. What can the model do? List every tool, function, plugin, and integration the model can invoke. A model that only produces text has limited blast radius. A model that can query a database, send an email, hit an internal API, or execute code has the blast radius of whatever those tools touch, with whatever privileges they run under. This is usually the single biggest driver of both risk and scope.
4. Who are the users, and what roles exist? Anonymous public users, authenticated customers, multiple tenants, internal staff. Multi-tenant applications need cross-tenant testing (can one customer’s input reach another’s data?), which adds scope. Role boundaries are things attackers try to cross.
5. What does the output touch? Where does model output go? Rendered as HTML or markdown in a browser? Passed into a shell, a query, or another system? Output handling is where a jailbreak turns into XSS, SSRF, or command injection, and it’s invisible if you only test the model in isolation.
If you can answer those five, you can scope the engagement. If you can’t, that’s the first thing a good tester will help you work out.
What actually drives cost
LLM pentest pricing tracks attack surface, not model size. The things that move it:
- Number and power of tools/integrations. Testing an agent with ten tools that reach production systems is a different engagement than testing a single-turn Q&A bot.
- Retrieval and corpus exposure. A writable, externally-influenced corpus adds indirect-injection and poisoning testing that a static prompt doesn’t need.
- Number of user roles and tenants. Each trust boundary is more to test.
- Output pathways. Every place model output flows into another system is a potential insecure-output-handling finding to chase down.
- Depth and retest. Whether you want a point-in-time assessment or validated remediation with a retest cycle.
A note on the “just run an automated scanner” temptation: automated tools are useful for coverage and regression, but the findings that matter in LLM testing (a chained indirect injection that triggers a tool call that exfiltrates another tenant’s data) come from a human reasoning about your architecture. Scanners don’t chain. Scope for the manual testing where the real risk is, and use automation to widen the net around it.
Right-sizing is the whole game: you want to pay for depth exactly where your architecture warrants it and nowhere it doesn’t. That’s what scoping is for.
The scoping checklist
Fill this in before your first scoping call. It’s most of what a tester needs.
Model and hosting
- Which model(s) power the application?
- Self-hosted, provider API, or hybrid?
- Is there a system prompt? Is it considered sensitive?
Retrieval / RAG
- Does the app retrieve from a corpus or knowledge base?
- What’s in the corpus, and where does it come from?
- Who can add or edit corpus content? Can any of them be an outsider?
- What vector store / retrieval stack is used?
Tools and agency
- What tools, functions, or plugins can the model call?
- What does each tool touch (DB, email, internal API, filesystem, code exec)?
- What privileges do those tools run with?
- Can the model chain tool calls autonomously?
Users and access
- What user roles exist (anonymous, authenticated, admin, multi-tenant)?
- Is it multi-tenant? Is tenant isolation in scope?
- Will you provide test accounts for each role?
Output handling
- Where does model output get rendered or sent?
- Is output ever passed to a browser, shell, query, or downstream system?
Engagement logistics
- Test environment: production, staging, or a dedicated instance?
- Are there rate limits or WAFs that will interfere with testing?
- Point-in-time assessment, or with a remediation retest?
- Which standards do you need mapping to (OWASP LLM Top 10, NIST AI RMF, ISO 42001, MITRE ATLAS)?
- Authorization: who signs off that testing is permitted?
What a good SOW contains
The statement of work is where scope becomes a contract. A solid LLM pentest SOW spells out:
- Objectives. What the test is trying to establish.
- In-scope assets. The specific application, endpoints, roles, and components.
- Out-of-scope. Explicitly, including the provider’s base model and any third-party systems.
- Testing approach. The methodology and the attack classes covered.
- Environment and access. Where testing happens and what the client provides.
- Rules of engagement. Timing, rate limits, data handling, what to do on a critical finding.
- Deliverables. Reports, evidence, standards mapping, retest.
- Timeline and authorization.
Sample statement of work
Adapt the bracketed fields. This is a starting template, not legal advice; have your own agreement reviewed.
STATEMENT OF WORK: LLM / AI APPLICATION PENETRATION TEST
Client: [Client name]
Provider: Brackish Security, LLC
Engagement dates: [Start] to [End]
Prepared by: [Name], Brackish Security
1. OBJECTIVES
Assess the security of [Application name], an LLM-powered
[chatbot / assistant / agent], to identify vulnerabilities in the
model integration, retrieval pipeline, tool usage, and output
handling. Determine whether an attacker can cause the application to
disclose sensitive data, take unauthorized actions, or harm its
users, and provide prioritized remediation.
2. SCOPE: IN
- Application: [URL / endpoint / interface]
- Model integration: [model(s), hosting]
- Retrieval pipeline: [corpus, vector store], where present
- Tools / functions the model can invoke: [list]
- User roles under test: [anonymous / authenticated / admin / tenants]
- Output-handling paths: [browser render / downstream systems]
3. SCOPE: OUT
- The base model provider's underlying model and infrastructure
- Third-party systems not owned by the Client
- [Any explicitly excluded components]
- Denial-of-service and load testing
4. TESTING APPROACH
Manual, threat-informed testing following the Brackish LLM
methodology, covering: direct and indirect prompt injection,
jailbreaking, system prompt leakage, excessive agency / tool abuse,
sensitive information disclosure, retrieval and embedding attacks,
and insecure output handling. Findings mapped to the OWASP Top 10
for LLM Applications and [MITRE ATLAS / NIST AI RMF / ISO 42001].
5. ENVIRONMENT & ACCESS
- Environment: [production / staging / dedicated test instance]
- Client to provide: test accounts per role, architecture overview,
and written authorization to test.
6. RULES OF ENGAGEMENT
- Testing window: [dates / hours]
- Rate limits / WAF: [coordinate or disable for testing]
- Data handling: findings and any accessed data handled per
[confidentiality / data-handling terms].
- Critical findings reported immediately, not held for the report.
7. DELIVERABLES
- Executive summary (business impact)
- Technical report with reproducible steps and PoC evidence
- Prioritized, layer-specific remediation guidance
- Standards mapping: [OWASP LLM Top 10 / MITRE ATLAS / NIST AI RMF /
ISO 42001]
- Remediation retest of identified findings
- Attestation letter
8. TIMELINE
- Scoping & threat modeling: [dates]
- Testing: [dates]
- Reporting: [dates]
- Retest: [dates, after remediation]
9. AUTHORIZATION
Testing is authorized by [name / title] on behalf of [Client].
Signed: ____________________ Date: __________
Common scoping mistakes
- Scoping the UI, not the architecture. The chatbot is the part you can see; the tools and corpus behind it are where the risk is.
- Excluding “the AI part” because it’s a hosted model. Your integration is in scope even when the base model isn’t.
- Forgetting the corpus is an input. If outsiders can influence retrieved content, that’s an attack surface, and it’s the one most often missed.
- Testing in an environment that isn’t representative. A dedicated instance with no tools wired up won’t surface the excessive-agency findings that matter.
- Skipping the retest. A finding you can’t confirm is closed isn’t really closed.
Ready to scope yours?
If you can answer the five questions at the top of this page, you’re most of the way to a scope. If you want help working out where your real attack surface is, and a fixed, right-sized scope for testing it, get in touch. We’ll walk your architecture and build the SOW with you.
This is one guide in the Brackish LLM penetration testing series.