---
title: "When Should an AI Agent Enumerate, and When Delegate?"
description: "A sealed 240-call exhibit grounds the boundary between decision spaces an agent should enumerate exactly and those it should delegate to a typed judge."
author: "Hugues Clouâtre"
pubDatetime: 2026-10-16T20:00:00.000Z
modDatetime: 2026-10-17T21:40:00.000Z
url: https://clouatre.ca/posts/ai-agent-enumerate-or-delegate/
tags:
  - "agentic-ai"
  - "architecture"
  - "case-studies"
  - "evaluation"
---

When a decision space is small and its rules are explicit, an AI agent should enumerate it exactly and verify the answer. When the criteria are fuzzy, the agent should delegate to a typed judge and record the uncertainty. Most agent architectures do neither; they prompt and hope. A sealed, pre-registered exhibit now bounds what the judge branch of that rule can claim: across 240 judge calls over two passes, the selection never flipped, yet confidence drifted pass to pass by a mean of 0.0245. A decision can be stable while the probability behind it is not, and that distinction is what routing is for. The routing decision, not model quality, determines whether an agent's choices are cheap, verifiable, and correct.

## Table of contents

## What Does Exact Enumeration Look Like at Small Scale?

Exact enumeration at small scale means walking every candidate and keeping the provably best one.

### The Blending Problem

An AI datacenter operator contracts power from three carbon-free resources: solar is cheap in daylight hours and produces nothing at night, wind is cheaper on average but volatile, and small-modular nuclear baseload is the most expensive per megawatt yet runs around the clock. Meeting demand while honoring an hourly carbon-free matching commitment is a blending problem, the classic operations-research question of combining inputs of varying cost and availability at least expense. The business setting follows [Orrick, 2025](https://www.orrick.com/en/Insights/2025/11/Powering-Data-Centers), which reports datacenter operators contracting power from mixed carbon-free portfolios.

In a three-resource illustration, the constraints compress to seven: three resource-capacity ceilings, one blending constraint enforcing the carbon-free floor, the matching commitment subsuming any separate demand row, and three non-negativity bounds. Seven constraints taken three at a time produce C(7,3) = 35 candidate triples. Fifteen are singular systems with no unique intersection; of the 20 that solve, ten fail a constraint and ten are feasible, deduplicating to seven distinct vertices. Exact enumeration over rational arithmetic walks all 35 triples, keeps the feasible seven, and returns the highest-value mix that honors the carbon-free floor (Figure 1). The result is provably correct because every candidate was checked.

```mermaid
%% Funnel diagram: 35 constraint triples split into 15 singular systems and 20 solved intersections, the feasibility filter rejects 10, the 10 feasible intersections deduplicate to 7 distinct vertices, and argmax selects the optimum worth 14,480
flowchart TD
    all["35 triples<br/>C(7,3)"] --> solve{"Solve 3x3"}
    solve -->|"singular"| sing["15 singular"]
    solve -->|"solved"| filter{"Feasible?"}
    filter -->|"reject"| infeas["10 infeasible"]
    filter -->|"pass"| feas["10 feasible"]
    feas -->|"deduplicate"| dedupe["7 unique vertices"]
    dedupe -->|"argmax"| opt["Optimum: 14,480"]
    classDef primary fill:#075985,stroke:#282728,stroke-width:2px,color:#fff
    classDef secondary fill:#e6e6e6,stroke:#ece9e9,stroke-width:2px,color:#282728
    classDef accent fill:#38bdf8,stroke:#075985,stroke-width:2px,color:#fff
    classDef muted fill:#fdfdfd,stroke:#ece9e9,stroke-width:1px,color:#282728
    class feas,dedupe primary
    class sing,infeas secondary
    class solve,filter muted
    class all,opt accent
    linkStyle default stroke-width:3px
```

*Figure 1: The enumeration funnel for the blending problem; 15 of the 35 triples are singular, the filter rejects 10 of the remaining 20, and the 10 feasible intersections deduplicate to 7 distinct vertices.*

*Code Snippet 1: Enumeration core, abridged from the public artifacts repo; the load-bearing check is the feasibility guard, which tests capacity bounds and non-negativity in exact rational arithmetic, and the exact 3x3 solve lives in solve3x3.py.*

```python file="judge-enumeration-exhibit/enumerate.py"
def optimal_vertex(constraints, objective):
    feasible_points = []
    for triple in combinations(constraints, 3):
        x = solve3x3(triple)  # exact rational 3x3 solve
        # feasibility guard: Ax <= b and x >= 0, exact arithmetic
        if x is not None and is_feasible(x, constraints):  # [!code highlight]
            feasible_points.append((x, dot(objective, x)))
    return max(feasible_points, key=lambda p: p[1])
```

### Why the Feasibility Filter Is Load-Bearing

An enumeration that skipped the feasibility check would score infeasible points, and constraint checking is exactly what a learned judge does not do by construction. The verifiability transfers to agent design: any reviewer can replay 35 checks in milliseconds, and the audit trail is the code itself. At small scale, exhaustive enumeration beats cleverness because it closes the decision instead of estimating it.

## What Does Rounding a Solver Output Cost?

Rounding loses measurable value. A whole-unit variant of the contract problem scores 8,208 in linear relaxation, and rounding its fractional solution yields 8,192.8, a silent loss of 15.2. The relaxation's optimum is a bound, not a plan; the rounded point is executable but not optimal, so value leaks between the bound and the nearest rounded allocation.

The phrase "almost optimal" covers two mechanisms that fail differently. Integer-grid loss appears when discrete allocation is weighted by continuous values; rounding moves the plan off the optimum, and the loss is quantified, the 15.2 above. Calibration loss appears when a judge score is thresholded at a cutoff such as 0.8. The failure sits at the boundary, where the score's own uncertainty dominates, and no rounding is involved at all.

An agent that rounds a judge output into a decision inherits the first mechanism silently. The remedies differ as much as the mechanisms. Integer-grid loss calls for re-solving on the lattice; calibration loss calls for consuming the probability as a probability, with the cutoff chosen against measured error rates. The fix in both cases is routing. Small formal spaces get enumeration, where no rounding exists, and fuzzy spaces get a judge whose probability is recorded and consumed as a measurement, not thresholded as a promise.

## Where Do Typed Judges Win?

Judges win where no formal model exists: natural-language criteria, ordered preference levels, decisions that resist a constraint matrix. A typed judge answers three primitives:

- a yes-or-no probability
- a choice among named options
- a score on ordered levels

The answer arrives as a typed value with a probability over the options (Code Snippet 2). Two properties matter in production: calibration, so the probability reads as a measurement, and a documented fallback envelope. A judge that returns a recorded fallback can be monitored; a silent fallback is indistinguishable from a skip. The reranker in that same validation silently kept the original order whenever it could not parse the model's ranking, an unrecorded skip of exactly this kind.

*Code Snippet 2: A frozen judge request, abridged from the archived record; the typed choice contract and pinned model version are visible, the archived state carries the objective vector and candidates but no per-candidate objective values, and the result probabilities are rounded for display. The judge exhibit is a distinct problem instance from the blending illustration.*

```json file="judge-enumeration-exhibit/results/raw/v2/run-001.json"
{
  "problem": "p-001", "stratum": "S1",
  "question": {
    "type": "choice",  // [!code highlight]
    "prompt": "Which candidate plan should be selected ..."
  },
  "model": "jev-1.13.0",  // [!code highlight]
  "result": { "selected": "B", "prob": 0.9, "conf": 0.87 }
}
```

Judges remain probabilistic instruments. [Shi et al. (2025)](https://aclanthology.org/2025.ijcnlp-long.18/) documented position bias across 15 large language model (LLM) judges and roughly 150,000 evaluation instances, with verdicts flipping when candidate order swaps. The consumer response is measurement discipline: record the probability, monitor calibration against outcomes, and treat any single verdict as one draw from a distribution. That posture works for fuzzy decisions and is impossible for verifiable ones, which is precisely why the two belong in different branches of an architecture.

The specification cost is also low, which matters for governance. A judge call takes a JSON state and a question with explicit per-option criteria. The exact wording can be version-stamped and reviewed before the first call.

## What Routing Rule Should an Agent Apply?

Three branches cover the space (Table 1). Formalizable and small: enumerate exactly, with the feasibility check visible in the code. Formalizable and large: hand the model to a mixed-integer linear-programming (MILP) solver, with the LLM confined to translating business language into constraints. Not formalizable: delegate to a typed judge and record the probability.

*Table 1: The routing rule; the verification column is what each branch guarantees.*

| Branch | When to Use | Verification |
| --- | --- | --- |
| Enumerate | Formalizable, small | Replayable audit trail |
| Formulate and solve | Formalizable, large | Solver log, dual prices |
| Typed judge | Not formalizable | Recorded probability, calibration monitoring |

Language models fit the formulator role. [AhmadiTeshnizi et al. (2024)](https://arxiv.org/abs/2402.10172) built OptiMUS to formulate and solve linear and mixed-integer programs from natural language, solving 67% more problems than standard prompting. Model strength is no substitute for routing: in a companion retrieval validation, upgrading every helper call to a model at 20 times the per-token price left hit rates flat, and 7 of 10 misses traced to how the pipeline built its candidate pool, not to the model. Industrial scale defeats the direct approach: [Li et al. (2026)](https://arxiv.org/abs/2602.10450) found that fine-tuned models scoring high on textbook benchmarks fall to 0-5.5% on industrial instances, and [Fu et al. (2026)](https://arxiv.org/abs/2607.10768) documented constraint omission on business briefs.

Stakes and reversibility set the escalation path. An enumerated or judged decision that crosses a reversibility boundary should halt for approval. The three-tier gate design in [AI Approval Gates: Engineering Oversight at Machine Speed](/posts/ai-approval-gates/) classifies each action at execution time, so routing composes with the escalation rule instead of duplicating it (Figure 2).

```mermaid
%% Decision tree: a formalizable check splits small spaces to exhaustive enumeration and large spaces to an LLM formulator feeding a MILP solver, non-formalizable decisions go to a typed judge, and every branch passes a reversibility check routing irreversible actions to an approval gate before execution
flowchart TD
    q1{"Formalizable?"} -->|"no"| judge["Typed judge"]
    q1 -->|"yes"| q2{"Small space?"}
    q2 -->|"yes"| enum["Exact enumeration"]
    q2 -->|"no"| solver["Formulate + MILP"]
    judge --> q3{"Reversible?"}
    enum --> q3
    solver --> q3
    q3 -->|"yes"| exec(["Execute"])
    q3 -->|"no"| gate["Approval gate"]
    gate -->|"after approval"| exec
    classDef primary fill:#075985,stroke:#282728,stroke-width:2px,color:#fff
    classDef secondary fill:#e6e6e6,stroke:#ece9e9,stroke-width:2px,color:#282728
    classDef accent fill:#38bdf8,stroke:#075985,stroke-width:2px,color:#fff
    classDef muted fill:#fdfdfd,stroke:#ece9e9,stroke-width:1px,color:#282728
    class judge,enum,solver primary
    class exec secondary
    class gate accent
    class q1,q2,q3 muted
    linkStyle default stroke-width:3px
```

*Figure 2: The routing rule with its escalation path; reversibility is checked after routing, so the approval gate composes with every branch.*

## What Does a Sealed Exhibit Measure at the Boundary?

A sealed exhibit measures the stability of a real judge at the boundary ([Clouatre, 2026](https://doi.org/10.5281/zenodo.23306444)).

### Pre-Registration Pins the Question

It pre-registered its question, reading criterion, and endpoints before any call and pinned the model to a fixed version, jev-1.13.0, the typed judge under test. The exhibit then froze 120 three-variable menus stratified by relative optimality gap: easy, at least 20%; medium, 5-20%; hard, 0.5-5%. Each menu holds four feasible vertices plus one infeasible super-optimal decoy, and the menu carries no per-candidate objective values, so the judge must check feasibility itself. The question, locked before any call, reads: "Which candidate production plan should be selected for this production decision, given the stated constraints and objective?" Two passes produced 240 calls, following the sealed-label pattern of [What a Null Result Taught Us About AI Agent Evaluation](/posts/prompt-repetition-agent-evaluation/). A pinned judge is a scheduled liability: the vendor announced 60 days' warning before this version's formal deprecation ([TypeSafe, 2026](https://typesafe.ai/blog/jev-1-14)), so version pinning needs a re-baselining policy, not a one-time freeze.

### Endpoints and the Saturation Ceiling

*Table 2: Sealed endpoints, 240 calls over 120 frozen menus, model version pinned.*

| Endpoint | Result (Clouatre, 2026) | Reading |
| --- | --- | --- |
| Both-call agreement | 120/120, Wilson 95% CI [0.969, 1.000] | Stable across passes |
| Infeasible decoy selected | 0/240 | Feasibility respected |
| Brier score | 0.0251 | Accurate probabilities |
| Expected calibration error | 0.1366 | Conservative probabilities |
| Position-bias check | Agreement 1.000 across positions | Order-insensitive (see text) |

The instrument saturated: the judge answered all 120 problems correctly in both passes (Table 2), so the pre-specified stratum comparison (p = 1.0) is a ceiling artifact, not evidence of equal difficulty. The ECE of 0.1366 coexists with perfect accuracy because rankings stayed well separated while probabilities ran conservative, roughly 0.10-0.16 below accuracy within each stratum.

### A Post-Hoc Confidence Gradient

A labeled post-hoc addendum found a graded confidence gradient in the same raw records (Table 3). Mean selected-option probability descends 0.894, 0.858, 0.838 across the strata; the S1-versus-S3 contrast, Welch t(155) = 4.63, p < 0.001. Figure 3 plots all 240 raw calls behind the stratum means.

*Table 3: Post-hoc stratified calibration, 80 calls per stratum; exploratory, not pre-registered.*

| Stratum | Selected Prob., Mean (sd) | Stated Conf., Mean (sd) | Runner-Up | Entropy (bits) |
| --- | --- | --- | --- | --- |
| Easy (S1) | 0.894 (0.071) | 0.867 (0.089) | 0.053 | 0.625 |
| Medium (S2) | 0.858 (0.079) | 0.821 (0.099) | 0.069 | 0.784 |
| Hard (S3) | 0.838 (0.082) | 0.797 (0.102) | 0.082 | 0.859 |

**Confidence descends as menus tighten; selections never flip**

| Series | Easy (gap >= 20%) | Medium (5-20%) | Hard (0.5-5%) |
| --- | --- | --- | --- |
| Selected-option probability per call mean | 0.894 (95% CI 0.878-0.909) | 0.858 (95% CI 0.84-0.875) | 0.838 (95% CI 0.82-0.856) |

- Selected-option probability per call observations 1/3 (n=80): 0.81, 0.87, 0.82, 0.93, 0.92, 0.81, 0.8, 0.58, 0.9, 0.94, 0.89, 0.86, 0.98, 0.93, 0.9, 0.93, 0.96, 0.93, 0.88, 0.77, 0.92, 0.96, 0.91, 0.88, 0.93, 0.93, 0.97, 0.78, 0.93, 0.93, 0.95, 0.97, 0.78, 0.95, 0.81, 0.91, 0.96, 0.8, 0.9, 0.87, 0.87, 0.93, 0.92, 0.66, 0.97, 0.99, 0.93, 0.88, 0.8, 0.9, 0.99, 0.91, 0.87, 0.92, 0.91, 0.84, 0.85, 0.92, 0.84, 0.93, 0.91, 0.85, 0.91, 0.91, 0.93, 0.95, 0.9, 0.96, 0.95, 0.91, 0.86, 0.95, 0.96, 0.91, 0.94, 0.77, 0.97, 0.84, 0.94, 0.93
- Selected-option probability per call observations 2/3 (n=80): 0.93, 0.9, 0.66, 0.96, 0.94, 0.86, 0.85, 0.81, 0.82, 0.93, 0.9, 0.93, 0.86, 0.95, 0.89, 0.8, 0.83, 0.88, 0.84, 0.95, 0.86, 0.89, 0.8, 0.87, 0.85, 0.81, 0.91, 0.86, 0.95, 0.86, 0.66, 0.93, 0.7, 0.91, 0.83, 0.74, 0.91, 0.84, 0.91, 0.81, 0.67, 0.89, 0.95, 0.92, 0.65, 0.7, 0.93, 0.76, 0.81, 0.92, 0.92, 0.95, 0.86, 0.86, 0.93, 0.71, 0.86, 0.81, 0.87, 0.72, 0.88, 0.81, 0.94, 0.82, 0.8, 0.85, 0.87, 0.9, 0.9, 0.87, 0.92, 0.98, 0.78, 0.86, 0.92, 0.93, 0.94, 0.75, 0.88, 0.96
- Selected-option probability per call observations 3/3 (n=80): 0.91, 0.82, 0.95, 0.91, 0.9, 0.92, 0.85, 0.78, 0.76, 0.83, 0.98, 0.77, 0.77, 0.91, 0.89, 0.86, 0.8, 0.84, 0.93, 0.74, 0.93, 0.8, 0.89, 0.86, 0.82, 0.69, 0.96, 0.86, 0.73, 0.82, 0.79, 0.78, 0.88, 0.87, 0.88, 0.8, 0.95, 0.68, 0.86, 0.74, 0.88, 0.88, 0.82, 0.93, 0.93, 0.83, 0.66, 0.85, 0.93, 0.89, 0.73, 0.88, 0.77, 0.9, 0.85, 0.9, 0.65, 0.91, 0.76, 0.67, 0.86, 0.77, 0.74, 0.94, 0.8, 0.93, 0.8, 0.87, 0.83, 0.98, 0.77, 0.77, 0.87, 0.63, 0.87, 0.96, 0.84, 0.83, 0.91, 0.75

All 240 v2 calls plotted; post-hoc addendum, not preregistered

*Figure 3: Selected-option probability for every v2 call, 240 raw observations by stratum with mean ticks on bootstrap 95% CI bars; the gradient descends while no selection flipped across the two passes.*

The exhibit supports a stability claim only. Two passes per problem bound that claim, the menus were pre-enumerated by the oracle so nothing here tests the judge as an enumerator, and two passes do not calibrate the judge. The position check varies only the candidate ordering defined in the archived protocol, so it bounds order sensitivity rather than replicating the systematic rotation of Shi et al. The tool exposes no temperature control, so provider defaults apply. The protocol author maintains the judge under test; the full protocol, raw records, and scripts are archived for independent verification.

## What Do Dual Prices Offer Capacity Planning?

Enumeration and solving return something a judge cannot: dual prices, the marginal worth of each constraint. A dual price answers the question every capacity planner asks: what one more unit of a binding resource is worth. In the contract illustration, the dual price of the carbon-free floor is exactly the value each megawatt-hour of mandated clean supply adds over the unconstrained mix that ignores it. A slack constraint prices at zero, by complementary slackness.

An agent that routes small formal decisions to enumeration gets this reading for free, and it extends to the agent's own operation. When the binding constraint is a review queue, a token budget, or a compute allocation, the dual price turns agent capacity planning into an optimization with a readable answer. A judge returns a probability and no price, so capacity questions routed to the judge branch get a confidence but no marginal worth.

## Takeaways

1. **Route by structure, not by model quality.** Small and formal means enumeration with the feasibility check visible; large and formal means a solver with the LLM as formulator; everything else goes to a typed judge with the probability recorded.
2. **Route before you round.** Integer-grid loss and calibration loss fail differently and remedy differently; both are routing failures, not rounding errors.
3. **Treat judge output as measurement.** The exhibit's judge never selected the infeasible decoy in 240 calls, yet its confidence descended with difficulty and drifted pass to pass, so read each verdict as one draw.
4. **Read the dual prices.** The same solve that returns the plan returns what each binding constraint is worth, the prices agent capacity planning otherwise lacks.

## References

- AhmadiTeshnizi, A., Gao, W., Udell, M., "OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models" (2024) -- https://arxiv.org/abs/2402.10172
- Clouatre, H., "Sealed Judge Enumeration Exhibit" (2026) -- https://doi.org/10.5281/zenodo.23306444
- Fu, Y., et al., "Opti-Agent-Bench: Benchmarking End-to-End Optimization R&D Agents on Real-World Business Problems" (2026) -- https://arxiv.org/abs/2607.10768
- Li, Z., et al., "Constructing Industrial-Scale Optimization Modeling Benchmark" (2026) -- https://arxiv.org/abs/2602.10450
- Orrick, "Megawatts to Megabytes: Powering Data Centers" (2025) -- https://www.orrick.com/en/Insights/2025/11/Powering-Data-Centers
- Shi, L., Ma, C., Liang, W., Diao, X., Ma, W., Vosoughi, S., "Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge" (2025) -- https://aclanthology.org/2025.ijcnlp-long.18/
- TypeSafe, "Jev: For the builders AND dreamers" (2026) -- https://typesafe.ai/blog/jev-1-14