---
title: "Structured Grounding Did Not Fix Reading-Chain Recall"
description: "A sequence-match re-run scored exact reading-chain recall: 0/103 with KG grounding, 0/103 without (n=206). Grounding changed failure quality, not success."
author: "Hugues Clouâtre"
pubDatetime: 2026-10-30T12:00:00.000Z
modDatetime: 2026-10-30T19:28:00.000Z
url: https://clouatre.ca/experiments/structured-exhibit-grounding-reading-chains/
---

This experiment never published a result, because its first run nearly published a wrong one. That run reported a 41-point completion gap favoring structured exhibit grounding, but its metric measured schema-key presence, not correctness: an agent could pass while returning a nonsense sequence, as long as a specific key appeared. A pre-publication audit caught the flaw, and the experiment was re-run under a different metric. The re-run scores each run by sequence-match, meaning a run succeeds only when the returned sequence deep-equals the runner-resolved ground truth.

## What did the re-run find?

Zero exact matches in both arms. The KG-grounded arm retrieved real slugs from knowledge-graph.json, but got the order wrong and substituted a third slug that is a real post on this site, just not a member of this chain. 95 of its 103 runs returned the identical wrong answer, `["mcp-tool-docs", "agent-ready-website", "orchestrating-ai-agents-subagent-architecture"]`, against the ground truth `["agent-ready-website", "mcp-tool-docs", "webmcp-beyond-checkout"]`. The ungrounded arm degenerated, word-splitting the chain name into `["agent", "tool", "surface", "chain"]` in 72 of 103 runs and echoing the chain id as the sole element in 31. Grounding changed failure quality, not success: a grounded agent produces plausible but mis-ordered sequences, while an ungrounded one produces noise.

**Grounded runs reached the right slugs; ungrounded runs returned noise**

| Series | Value |
| --- | --- |
| Arm a: right slugs, wrong order | 95 runs |
| Arm a: first two correct, third substituted | 8 runs |
| Arm b: word-split the chain name | 72 runs |
| Arm b: chain id as sole element | 31 runs |

No run in either arm matched the ground-truth sequence (0/206 exact matches)

*Figure 1: Dominant response per group of runs, n=103 per arm; no run in either arm matched the ground-truth sequence.*

Both arms sit at a rate of 0.0 with a Wilson 95% interval of [0.0, 0.036]. With zero successes in 103 runs, the interval says the true rate is at most about 3.6% per arm. The two-proportion score test is degenerate here, with statistic 0 and p = 1, because both rates are 0 and the test carries no information.

*Table 1: Exact reading-chain recall per grounding arm; sequence-match scoring, n=103 per arm.*

| Arm | Grounding | n | Exact matches | Rate | 95% CI |
|---|---|---|---|---|---|
| a | KG-grounded | 103 | 0 | 0.0 | [0.0, 0.036] |
| b | Ungrounded | 103 | 0 | 0.0 | [0.0, 0.036] |

## How was it measured?

Each run asks the agent to list the posts of the agent-tool-surface-chain reading path in order, and the run counts only when the returned slug array deep-equals the ground truth the runner fetches at scoring time: agent-ready-website, then mcp-tool-docs, then webmcp-beyond-checkout. Arm a receives KG-grounded prompting, fetching knowledge-graph.json and reading the readingChains entry before answering. Arm b receives no machine-readable surface and must recall the path from the prompt alone. Both arms run the same model, gemma-4-26b-a4b-it, with 103 samples per arm in interleaved single attempts.

*Code Snippet 1: Protocol sub-object from the committed data file.*

```json file="agent-idea-2026-10-29-0-rerun.json"
{
  "protocol": {
    "summary": "Two-arm comparison of reading-chain recall by an AI agent on this site, scored by sequence-match: a run succeeds only when the returned sequence deep-equals the runner-resolved ground-truth ordered slug array [agent-ready-website, mcp-tool-docs, webmcp-beyond-checkout]. Arm a is KG-grounded: the agent fetches https://clouatre.ca/knowledge-graph.json, reads the readingChains entry, and lists the sequence in order. Arm b is ungrounded: no machine-readable surface, asked to list the posts of the agent-tool-surface-chain reading path in order. Both arms respond with the same JSON shape {\"sequence\": [<slugs>]}.",
    "metric_name": "sequence-match",
    "metric_type": "binary",
    "sequence_chain": "agent-tool-surface-chain",
    "ground_truth_sequence": ["agent-ready-website", "mcp-tool-docs", "webmcp-beyond-checkout"],
    "samples_per_arm": 103,
    "model": "@cf/google/gemma-4-26b-a4b-it"
  }
}
```

The chain itself is stable ground truth. Its sequence is unchanged in knowledge-graph.json throughout, so there is no ground-truth drift between the two runs. What changed between them is the metric, not the target.

## What are the caveats?

The metric is strict. Exact-match scoring gives no partial credit, and a partial-credit metric would score the grounded arm above zero, since its dominant response carries two correct slugs. The result covers a single model on a single site, so it is a case study rather than a domain-general law. Two large observational studies point the other way: structured, statistics-rich content earns more AI citations ([Aggarwal et al., 2024](https://arxiv.org/abs/2311.09735)), and across 174,048 pages cited in AI Overviews, length barely predicted citation, with format matching query intent mattering more ([Ahrefs, 2025](https://ahrefs.com/blog/short-vs-long-content-in-ai-overviews/)). Those studies measure what gets cited; this run measures whether an agent completes a traversal, and the two can disagree. At n = 103 per arm the design detects only large effects, and here the effect estimate is exactly zero, with the Wilson intervals ruling out anything above roughly 3.6% per arm. The two-proportion score test adds nothing in this configuration; its statistic is 0 and its p-value is 1, a degenerate output when both arms fail completely.

## What should a publisher take away?

Agents that fetch the structured exhibit still mis-order and embellish sequences, so exact-order recall is the open problem, not surface formatting. In this run, a knowledge graph got the model to the right slugs, and that is real, but ordering three known items defeated both arms. If agents cannot order a three-page reading path, structured surfaces alone will not make a site legible to them; the deeper question is how agents plan and verify sequences, which is the thread the [agent-ready website thesis](/posts/agent-ready-website/) follows. Treat any experiment measuring agent comprehension by key presence rather than correctness with suspicion, because the two can disagree completely.

## References

- Aggarwal, P. et al., "GEO: Generative Engine Optimization" (2024) -- https://arxiv.org/abs/2311.09735
- Ahrefs, "Short vs. Long Content in AI Overviews: The Data Says Both Work" (2025) -- https://ahrefs.com/blog/short-vs-long-content-in-ai-overviews/