How Much Tool Documentation Do AI Agents Actually Need?

Multi-agent platforms pay a hidden tax on every request: agent clients serialize the available tool definitions, schemas and description prose included, into the model context before the user’s task begins. Catalogs of dozens of tools turn that prose into a recurring line item: billed in tokens on every call, and paid again as degraded attention on every long prompt.

The industry learned this at the prompt level. Anthropic removed over 80% of the Claude Code system prompt with no measurable loss for Opus 5 and Fable 5 (Shihipar, 2026). The unexamined next layer sits one level down: the tool definitions themselves. Anthropic’s own tool-use documentation says only that when a required parameter is missing, Opus tends to ask a clarifying question while less capable models may infer a value (Anthropic, 2026). Because the documentation offers no evidence either way, teams keep paying for prose rather than risk accuracy on cheaper models.

We tested that premise with two pre-registered experiments and 480 blind-scored tool calls across two model tiers, a compact one and a frontier one. The headline finding: across 480 blind-scored calls, trimming tool-level prose and consolidating response-shaping parameters produced no statistically detectable accuracy loss at either tier, while the primary prose trim cut mean cold-call input tokens up to 13.8%. Those are cold-call figures: cached multi-turn sessions read the schema from a discounted prefix, so the dollar saving depends on the cache hit rate while the attention-budget saving applies to every call. The full dataset, including all 480 blind-scored calls and the pre-registered rubric, is openly available (Clouatre, 2026). For a CTO weighing model mix, schema prose is a cost lever, and capability spend is the accuracy lever.

Table of contents

Why Does Schema Documentation Cost So Much?

The Model Context Protocol (MCP), the open standard for connecting AI agents to external tools, exposes tool definitions containing a name, description, and input schema, parameter descriptions included. Agent clients typically serialize those definitions into the model’s tool context, where every descriptive word consumes context. A server exposing forty tools with verbose documentation can spend thousands of tokens before the user’s question arrives.

The cost compounds through a known failure mode. Chroma Research (2025) found that every frontier model tested degrades as input length grows. Longer tool catalogs mean more rot-prone context, so schema bloat taxes both cost and quality. The token bill is visible on any invoice; the attention tax hides inside degraded answer quality and is harder to attribute.

The spec’s history shows the community guessed at the fix without evidence. SEP-1382 proposed splitting tool-description and param-description roles as a best practice, but the proposal closed dormant for lack of supporting data (MCP, 2026). Practitioners were told which knobs exist, yet nobody had measured whether either knob moves accuracy, or whether the effect depends on model size.

Three studies anchor this evidence. Hasan et al. (2026) documented description smells across 856 tools on 103 MCP servers; a companion study evaluated smell-aware rewrites of server descriptions (Wang et al., 2026); and Yu et al. (2026) benchmarked tool-call completion on cloud-network APIs. Table 1 summarizes what the published record shows.

Hasan’s gain came from added content, not trimmed prose. None of these studies isolated documentation placement across model sizes; that is the gap this post fills.

How Did We Test It?

Experiment 1 used a 2x2 ablation on one production MCP server tool, analyze_symbol, alongside two distractor tools to keep tool selection honest. With two distractors, this design tests parameter extraction and execution fidelity under mild selection pressure, not semantic disambiguation across a wide catalog. Four cells crossed documentation richness at two levels: rich versus lean tool descriptions, each combined with rich versus sparse parameter descriptions (Figure 1). The primary comparison was Cell C, which reflects current production text, against Cell A, a reconstruction of the historical rich baseline. Cells B and D completed the grid. Two models ran the grid: claude-haiku-4-5 and claude-sonnet-5, a compact tier and a frontier tier one generation apart. 8 prompts, 5 runs, 4 cells, 2 models: 320 calls.

Every response was blind-scored against a pre-registered rubric (Clouatre, 2026). Results were tested with two-tailed Mann-Whitney U at alpha 0.05, a rank-based test that makes no normality assumption and suits ordinal rubric scores. Input tokens were logged per call, giving a paired cost measurement alongside accuracy.

Experiment 2 probed structure rather than prose. It collapsed four response-shaping parameters into a single response_format enum, running 160 calls under the same protocol, prompts, and models. The question: does consolidating parameters into structural choice preserve accuracy while trimming tokens? Four separate boolean and string parameters became one enumerated option, shrinking both the parameter list and the prose describing each option (Code Snippet 1).

[
  {
    "arm": "control",
    "response_format": {
      "summary": true,
      "fields": ["functions"],
      "mode": "callgraph",
      "implOnly": false
    }
  },
  { "arm": "treatment", "response_format": "functions" }
]response-format-collapse.json

What Did 480 Calls Show?

Cell C versus Cell A on param-fill score gave p=0.372 for Haiku (U=720, r=0.10) and p=0.569 for Sonnet (U=820, r=-0.025), a null result at both tiers. A finer check-level composite score confirmed the null at p=0.504 and p=0.569 respectively.

How Much Did Trimming Save in Tokens?

Mean input tokens fell 12.4% on Haiku and 13.8% on Sonnet. The top-level description fell from 1,856 to 465 characters, a 75% cut; across the full serialized tool definition, the measured prompt contribution fell roughly 45%. Neither trim produced a statistically detectable accuracy difference in any comparison.

Experiment 2 matched the pattern: accuracy null at p=0.335 for Haiku and p=0.242 for Sonnet, with token savings of 3.3-4.3%. The four parameters encoded a small set of valid response modes, and the enum covered every supported combination, so the savings came from structure, not lost function.

What Drove Accuracy Instead?

In this workload, the observed gap between model tiers was larger than the observed effect of documentation richness. Sonnet sat near ceiling, with mean param-fill scores of 0.825 to 0.975 in every cell. Haiku stayed uniformly weaker at 0.475 to 0.625 across every cell, meaning its failures tracked capability limits. Tool-selection accuracy was equally flat: 0.575 to 0.6 for Haiku and 0.95 to 0.975 for Sonnet in every cell, so the trim moved neither filling nor selection.

The budget implication: hours spent polishing tool descriptions to rescue a small model target the wrong lever. The same hours spent on prompt design, validation layers, or escalation to a larger model target the gap the data actually shows.

Caveats apply. Each cell held n=40 calls: the study rules out large regressions on this workload but remains underpowered to exclude small accuracy effects, especially given Sonnet’s score ceiling. Experiment 2’s post-hoc sensitivity analysis, dropping one trap check, flips direction for Sonnet at p=0.0427; that result is explicitly exploratory and carries multiple-comparisons risk. Cell A is a reconstruction, not literal historical text. Token figures are cold-call measurements. Finally, the results characterize one production tool deeply, not a broad tool catalog.

What Does This Unlock?

The results suggest that, for this workload, retained parameter semantics and schema structure were sufficient once broad tool-level prose was trimmed. That points toward two architecture questions beyond schema hygiene. The first targets the schema payload itself. If lean manifests and on-demand tool inspection preserve selection and execution quality in a given workload, an agent may not need the full tool catalog in every model request: it can inspect tool APIs dynamically inside an isolated runtime and generate deterministic scripts that call them. The catalog becomes a cache, fetched on demand. Security teams should note the trade: script generation moves risk from context injection into execution, making sandboxing, capability-scoped credentials, egress controls, and audit trails first-class design requirements.

The protocol work to support this is underway. Tool manifests for incremental catalog synchronization, proposed in PR 2636, let clients sync catalogs without re-sending full definitions (MCP, 2026). Routing economics reinforce the case (Dekoninck et al., 2024): a cheap model handles routing and extraction, and a frontier model takes only what is genuinely ambiguous.

Where Do Small Models Fit?

Haiku’s lower but structured profile motivates testing compact models on narrowly bounded, verifiable subtasks such as routing and extraction, paired with validation and escalation to frontier models for ambiguous planning. This study does not establish that a compact model can safely own those tasks across a broad production catalog.

Typed-judgment systems make this concrete: TypeSafe’s System One models return typed answers with probabilities that ordinary code composes into pipelines (TypeSafe, 2026). Its re-ranking cookbook shows a small model with the right output contract beating a larger one. Tool routing, with a bounded enum of choices, fits that pattern closely.

Production infrastructure is already converging on this shape. OpenRouter’s alpha decisions endpoint accepts typed questions, boolean (noul), choice, or score, and routes them to a compact judgment model (OpenRouter, 2026):

{
  "model": "typesafe/jev-1.13",
  "questions": {
    "is_bug": {
      "type": "noul",
      "instructions": "Is the customer reporting a software defect?"
    },
    "urgency": {
      "type": "score",
      "criteria": ["Can wait", "Fix this week", "Blocking revenue"]
    }
  },
  "state": { "ticket": "Checkout page is blank after Pay." }
}alpha-decisions-request.json

Table 3 collects the external signals that small, contract-bound models are ready for the tool layer.

This is a direction of travel, and each component exists today; the experiments here do not test the composition, and each piece needs its own security and reliability evaluation.

What Should Platform Teams Do?

The immediate move is editorial. Trimming description strings now is supported by the direct finding: the production trim cut the analyze_symbol description from 1,856 to 465 characters, and input tokens fell 12-14% on cold calls with no statistically detectable accuracy loss at two model tiers. The change requires no runtime work and compounds across every entry in the catalog.

[
  {
    "name": "deploy_service",
    "description": "Deploy a service to a target environment.",
    "inputSchema": {
      "type": "object",
      "properties": {
        "env": {
          "type": "string",
          "enum": ["staging", "prod"],
          "description": "Target environment. Use prod only when approved."
        },
        "ref": {
          "type": "string",
          "description": "Git ref to deploy; defaults to main."
        }
      },
      "required": ["env"]
    }
  }
]lean-tool-definition.json

The rule the snippet encodes: if a constraint matters for correctness, express it where the parameter is defined, not in tool-level prose. Parameter semantics carried the load in these experiments; vague tool-level narrative did not.

Before rolling a trim out widely, measure with your own prompts; a day of pre-registered scoring on a real workload buys confidence the literature cannot. When reading the token numbers, treat them as a cold-call ceiling for the cost benefit: cached multi-turn sessions read the schema from a discounted prefix. The attention benefit does not amortize. For a monthly estimate, weight the token cut by the cold/warm split and apply your provider’s cache-read discount (0.1x on some plans):

savings = N * P_in * (
  p_cold * delta_cold +
  d_cache * (1 - p_cold) * delta_warm
)savings-formula.txt

Where N is tool-enabled calls per month, P_in the baseline per-call input cost used for a planning estimate (mean input tokens times price; apply the cache multiplier only to the provider-defined cacheable prefix for precise billing forecasts), and p_cold the fraction of calls without a cache hit. delta_cold is the measured fractional cold-call cut, 0.124 to 0.138; delta_warm is the fractional reduction in the cacheable warm prefix, and d_cache is your provider’s cache-read price multiplier, 0.1 on some plans but model- and billing-specific; verify it for your deployment. Every turn still pays full price in context budget, and the context-rot decay documented earlier grows with input length, so shorter catalogs leave more room before it arrives.

Takeaways

1. Trimming showed no detectable accuracy loss at either tier. Lean production text matched the rich baseline (p=0.372, p=0.569, primary comparison).
2. Trimming paid in tokens. Mean input tokens fell 12.4% on Haiku and 13.8% on Sonnet on cold calls.
3. Model tier mattered more than documentation richness. In this workload, the observed capability gap dwarfed any documentation effect.
4. Structure consolidations compound the savings. Collapsing four parameters into one enum saved 3.3-4.3% more tokens with accuracy null at both tiers.
5. The next shrink target is the catalog itself. Code-driven execution against synced manifests, per PR 2636, creates a testable path to smaller tool payloads, a composition the experiments here did not evaluate.
6. Small models are candidates for bounded tool-layer tasks. Typed contracts, validation, and escalation can make compact models economically attractive for narrowly scoped decisions.

References

• Anthropic, “Tool use overview” (2026) — https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview
• Chroma Research, “Context Rot” (2025) — https://www.trychroma.com/research/context-rot
• Clouatre, H., “MCP Parameter-Description Ablation” (2026) — https://doi.org/10.5281/zenodo.22844431
• Dekoninck, J., et al., “RouteLLM: Learning to Route LLMs with Preference Data” (2024) — https://arxiv.org/abs/2406.18665
• Hasan, M. M., et al., “Model Context Protocol (MCP) Tool Descriptions Are Smelly! Towards Improving AI Agent Efficiency with Augmented MCP Tool Descriptions” (2026) — https://arxiv.org/abs/2602.14878
• MCP, SEP-1382 “Split Tool-Description and Param-Description Roles” (2026) — https://github.com/modelcontextprotocol/modelcontextprotocol/issues/1382
• MCP, PR 2636 “Tool Manifests for Incremental Catalog Synchronization” (2026) — https://github.com/modelcontextprotocol/modelcontextprotocol/pull/2636
• OpenRouter, “alpha.decisions: Submit a Decisions (questions and answers) request” (2026) — https://openrouter.ai/docs/api/api-reference/alphadecisions/submit-a-decisions-questions-and-answers-request
• Shihipar, A., “The new rules of context engineering for Claude 5 generation models” (2026) — https://claude.com/blog/the-new-rules-of-context-engineering-for-claude-5-generation-models
• TypeSafe, “Re-ranking with TypeSafe” cookbook (2026) — https://docs.typesafe.ai/cookbooks/rerank_typesafe
• TypeSafe, “System One” docs (2026) — https://docs.typesafe.ai/concepts/system-one
• Wang, P., et al., “From Docs to Descriptions: Smell-Aware Evaluation of MCP Server Descriptions” (2026) — https://arxiv.org/abs/2602.18914
• Yu, G., et al., “Getting the Parameters Right: A Difficulty-Graded Benchmark and Probe-Guided Training for LLM Tool Calls” (2026) — https://arxiv.org/abs/2608.03071