--- title: "AI SDLC Governance: Three Layers for Engineering Leaders" description: "A three-layer governance stack (comprehension, review gate, and observability) gives engineering leaders measurable control over AI-generated code volume." author: "Hugues Clouâtre" pubDatetime: 2026-08-04T09:00:00.000Z modDatetime: 2026-10-05T23:02:52.000Z url: https://clouatre.ca/posts/ai-sdlc-governance-stack/ tags: - "ai-engineering" - "ai-governance" - "implementation-guide" - "security" --- The same AI tooling. Two engineering organizations. Opposite outcomes: one cut pull request (PR) cycle time 31.8% while AI-generated code volume scaled 750x over six months ([Kumar et al., 2025](https://arxiv.org/abs/2509.19708)); the other saw PR closure time climb 42%, from 5 hours 52 minutes to 8 hours 20 minutes, partly from deeper engagement with automated feedback ([Cihan et al., 2025](https://arxiv.org/abs/2412.18531)). Neither study isolates governance maturity as the causal variable. Both show that AI impact is highly context-dependent; organization-level workflow, review, and platform conditions are plausible contributors worth measuring rather than assuming away. This post organizes those conditions into a three-layer model (comprehension, review gate, observability), with patterns from two production open-source repositories: [aptu-coder](https://github.com/clouatre-labs/aptu-coder), an MCP (Model Context Protocol) comprehension server, and [aptu](https://github.com/apps/aptu-dev), a GitHub App for automated PR review and security scanning. ## Table of contents ## What Is a Governance Stack for AI-Generated Code? A governance stack for AI-generated code has three layers, each addressing a distinct failure mode that appears once that code moves from an editor into a shared codebase. ### Three Layers, Three Failure Modes The comprehension layer provides structured analysis before any code is generated or reviewed. Without a symbol table, call graph, or AST (abstract syntax tree), agents operate on raw file bytes, a condition of context starvation that compounds with codebase scale. The review gate layer combines automated review with reversibility-calibrated escalation, not binary approve/reject, at the PR boundary. When that gate is shallow, verification gaps accumulate, a pattern examined in [AI Approval Gates](/posts/ai-approval-gates). The observability layer captures provenance an agent cannot self-verify. Without it, the stack produces no reconstructable audit trail, examined further in [AI Observability Gaps](/posts/ai-observability-gaps). Each layer functions independently; the stack is what their integration produces, extending the accountability-layer thesis in [AI-Assisted Development](/posts/ai-assisted-development). ```mermaid %% Three-layer AI SDLC governance model mapping comprehension, review gate, and observability to their failure modes graph TD INPUT["AI Change"] C["Comprehension"] R["Review Gate"] O["Observability"] FAIL1["Context Starvation"] FAIL2["Verification Gap"] FAIL3["Silent Failure"] INPUT --> C C --> R R --> O C -.->|"failure mode"| FAIL1 R -.->|"failure mode"| FAIL2 O -.->|"failure mode"| FAIL3 classDef primary fill:#075985,stroke:#282728,stroke-width:2px,color:#fff classDef secondary fill:#e6e6e6,stroke:#ece9e9,stroke-width:2px,color:#282728 classDef muted fill:#fdfdfd,stroke:#ece9e9,stroke-width:1px,color:#282728 linkStyle default stroke-width:3px class C,R,O primary class INPUT secondary class FAIL1,FAIL2,FAIL3 muted ``` *Figure 1: The three-layer governance model. Comprehension prevents context starvation; the review gate closes verification gaps; observability produces a reconstructable audit trail.* A fourth risk sits at the integration boundary: drawn from 1,600+ annotated traces across seven frameworks, UC Berkeley's Multi-Agent System Failure Taxonomy (MAST) finds that review layers added without defined input/output contracts reproduce specification errors in CI rather than eliminating them ([Cemri et al., 2025](https://arxiv.org/abs/2503.13657)). ## How Does the Comprehension Layer Work? The comprehension layer gives agents and reviewers a structured understanding of a codebase before either generates or reviews a line of code. aptu-coder, an MCP server written in Rust, is a concrete implementation. ### Structured Analysis Over Raw File Reads aptu-coder provides structured analysis across 18 languages, exposing four analysis tools over a streamable HTTP transport that multiple delegate agents share concurrently. An agent receives pre-parsed symbol tables and call graphs directly, rather than re-deriving structure from raw file bytes on every call. Token-cost evidence for this approach, benchmarked against Django and OpenFAST, is documented in [AI Adoption in Engineering](/posts/ai-adoption-engineering), referenced here rather than reprinted. ### Comprehension-Generation Asymmetry and Why It Matters A survey of more than 1,400 papers identified a core finding: models process complex input context far more reliably than they produce equivalent long-form output, an asymmetry the survey terms comprehension-generation asymmetry ([Mei et al., 2025](https://arxiv.org/abs/2507.13334)). aptu-coder's structured output exploits this deliberately: an agent receiving pre-parsed AST, call graphs, and symbol tables works in the domain where models perform best, instead of expanding into an unrestricted repository scan. The Agentic Context Engineering (ACE) framework reinforces this: it matches a top-ranked production agent using a smaller open-source model by controlling context evolution instead of upgrading model scale ([Q. Zhang et al., 2025](https://arxiv.org/abs/2510.04618)). Context management, not model size, is the cost lever both findings point to. ## How Does the Review Gate Layer Work? The review gate layer sits at the pull request boundary. [aptu](https://github.com/apps/aptu-dev), a GitHub App, implements this layer as configuration-as-code rather than a mandatory pipeline step. The review verdict is a typed judgment, not binary approve/reject: an approve/block probability and per-finding severity scores feed the reversibility-calibrated escalation, mirroring the typed policy checkpoints now appearing at agent boundaries ([Shlomov et al., 2026](https://arxiv.org/abs/2605.20874)). ### Configuration as Code: The Opt-In Governance Model Configuration enforces opt-in per repository via `.github/aptu.yml`, with precise path scoping. Credentials follow a tiered model: allowlisted organizations share managed AI credentials; external installations bring their own API key (BYOK) as a repository secret. Install via the [aptu GitHub App](https://github.com/apps/aptu-dev). *Code Snippet 1: Minimal `.github/aptu.yml` enabling triage and PR review. `review.paths` dispatches only when a PR touches `src/**` or `crates/**`, and ignores Markdown-only changes.* ```yaml file=".github/aptu.yml" {8-11} version: 1 triage: enabled: true review: enabled: true skip-labeled: true instructions-file: .github/instructions/pr-review.md paths: - "src/**" - "crates/**" - "!**/*.md" ai: provider: openrouter model: google/gemma-4-26b-a4b-it ``` ### What the Benchmark Actually Measures A self-reported internal benchmark compared aptu+mercury-2 against raw claude-opus-4.6 across six fixtures, one run per fixture, directional only and not independently verified. Scores reflect the author's assessment against a fixed rubric: issue coverage, specificity of feedback, absence of hallucinated findings, and actionable framing. *Table 1: PR review quality by fixture, aptu+mercury-2 (structured context + schema) vs. raw claude-opus-4.6 (generic two-sentence prompt, no schema) (n=6, one run per fixture, directional).* | Fixture | aptu+mercury-2 | Raw opus-4.6 | |---|---|---| | Security patch | 5.0/5 | 2.0/5 | | Refactor PR | 4.5/5 | 2.5/5 | | New feature | 5.0/5 | 2.0/5 | | Bug fix | 5.0/5 | 2.5/5 | | Test-only PR | 4.5/5 | 2.0/5 | | Dependency bump | 4.8/5 | 2.0/5 | The comparison is architecturally unequal by design: the raw opus baseline used a generic two-sentence prompt with no schema, rubric, or AST context. The result illustrates the architecture pattern, not a model capability claim. *Table 2: PR review cost per call by fixture, aptu+mercury-2 (structured) vs. raw claude-opus-4.6 (generic prompt), with savings vs. baseline (n=6, directional).* | Fixture | aptu+mercury-2 | Raw opus-4.6 | Savings | |---|---|---|---| | Security patch | $0.0012 | $0.0201 | 94% | | Refactor PR | $0.0009 | $0.0184 | 95% | | New feature | $0.0013 | $0.0197 | 93% | | Bug fix | $0.0010 | $0.0189 | 95% | | Test-only PR | $0.0008 | $0.0192 | 96% | | Dependency bump | $0.0014 | $0.0195 | 93% | The narrow savings range (93-96%) across fixture types reflects that review cost is dominated by fixed context size, not fixture complexity. ByteDance's BitsAI-CR confirms at industrial scale that purpose-built structured pipelines outperform generic prompting regardless of model choice ([Sun et al., 2025](https://conf.researchr.org/details/fse-2025/fse-2025-industry-papers/24/BitsAI-CR-Automated-Code-Review-via-LLM-in-Practice)). ## What Does the CI/CD Harness Look Like in Practice? The CI/CD harness turns the review and observability controls into enforceable gates. ### A Multi-Job Pipeline Where Every Gate Has a Purpose The CI pipeline (Figure 2) has a defined dependency graph: a path filter gates all downstream jobs; quality and security jobs run in parallel where possible. The local `just lint` recipe omits the cognitive-complexity flag that CI enforces, so CI is authoritative. ```mermaid %% CI/CD harness flow showing path filtering through parallel jobs to ci-result aggregator and GPG attestation gate graph TD CHANGES["Path Filter"] QUALITY["Quality Gates"] SECURITY["Security Scan"] TEST["Test"] BUILD["Build"] INTEGRATION["Integration"] SCAN["Self Scan"] RESULT["CI Result"] ATTEST["GPG + Attestation"] CHANGES --> QUALITY CHANGES --> SECURITY CHANGES --> TEST TEST --> BUILD BUILD --> INTEGRATION BUILD --> SCAN QUALITY & SECURITY & INTEGRATION & SCAN --> RESULT RESULT -->|"release only"| ATTEST classDef primary fill:#075985,stroke:#282728,stroke-width:2px,color:#fff classDef secondary fill:#e6e6e6,stroke:#ece9e9,stroke-width:2px,color:#282728 classDef accent fill:#38bdf8,stroke:#075985,stroke-width:2px,color:#fff classDef muted fill:#fdfdfd,stroke:#ece9e9,stroke-width:1px,color:#282728 linkStyle default stroke-width:3px class CHANGES,RESULT primary class QUALITY,SECURITY,TEST,BUILD secondary class INTEGRATION,SCAN accent class ATTEST muted ``` *Figure 2: The CI pipeline dependency graph. Path filtering gates all downstream jobs; ci-result is the sole required status check; the GPG tag gate and attestation steps run only on release.* The `scan` block in `.github/aptu.yml` enables App-dispatched security scanning. The `fail-on` field fails the PR on critical and high findings; the App uploads a permanent SARIF (Static Analysis Results Interchange Format) record to Code Scanning and reports a commit status to the PR head. *Code Snippet 2: `scan` block in `.github/aptu.yml` enabling App-dispatched security scanning on every pull request.* ```yaml file=".github/aptu.yml" {6-8} version: 1 triage: enabled: true review: enabled: true scan: enabled: true fail-on: critical,high ``` Repositories not using the App can replicate the same gate with a manual CI job. The scan step combines PR annotation and a persistent Code Scanning record in a single invocation: `--output github-annotations` fails the PR on critical and high findings, and `--sarif-output` writes the SARIF file uploaded in the next step. *Code Snippet 3: `scan-self` job in `.github/workflows/ci.yml`, the manual equivalent for repositories not using the App.* ```yaml file=".github/workflows/ci.yml" {14-15} scan-self: permissions: contents: read security-events: write needs: [changes, build] if: needs.changes.outputs.code == 'true' && needs.build.result == 'success' # ... checkout and artifact download omitted steps: - name: Run security scan and generate SARIF if: always() run: | ./target/ci/aptu scan-security crates/ \ --output github-annotations \ --sarif-output findings.sarif \ --fail-on critical,high - name: Upload SARIF report if: always() uses: github/codeql-action/upload-sarif@18420e3271f74589575af831a523c833acda327f with: sarif_file: findings.sarif category: aptu-scan-security ``` ### Code Complexity as a First-Class CI Gate The check job (Code Snippet 4) enforces the cognitive-complexity lint on every PR. `-D warnings` promotes every clippy warning to a hard error, so a function exceeding the complexity threshold fails the PR exactly as a failing test would, rejecting hard-to-reason-about code. :::warning Teams adopting this pattern should be explicit about the local-CI gap, or developers may believe they passed locally when only CI enforces it. ::: *Code Snippet 4: `.github/workflows/ci.yml`, the `check` job enforcing formatting, Clippy lint-as-error, and the cognitive-complexity hard gate.* ```yaml file=".github/workflows/ci.yml" {11,14-16,18} check: name: Check Format & Lint runs-on: ubuntu-24.04-arm needs: changes timeout-minutes: 10 if: (needs.changes.outputs.code == 'true' || github.event_name != 'pull_request') && github.actor != 'renovate[bot]' steps: # checkout, rust-toolchain, rust-cache omitted - name: Check formatting run: cargo fmt --check - name: Run Clippy lints run: | cargo clippy --locked --profile ci \ -- -D warnings \ -W clippy::cognitive_complexity - name: Build examples run: cargo build --examples --locked -p aptu-core --profile ci ``` ### Dual-Mode Security Scanning: Shift-Left and Persistent Record PR scanning and the post-merge scan reuse the same binary: one gates the PR with transient, diff-scoped feedback; the other persists findings to Code Scanning as a queryable audit trail. The AI review step operates on structured scanner output, not raw diffs, consistent with the Tier 1 boundary in [AI-Augmented CI/CD](/posts/ai-augmented-cicd): AI analyzes linter and scanner JSON, never untrusted code directly. This is the shift-left-plus-persistent-record pattern with supply-chain provenance, as argued in [AI Supply Chain Attack Vectors](/posts/ai-supply-chain-attack-vectors). ### Local Guardrails: The Pre-CI Line of Defence Local hooks are one enforcement point. A more complete guardrail pairs them with a recipe and an AGENTS.md file. [Goose](https://github.com/aaif-goose/goose) validates its own pipeline by running `goose run --recipe goose-self-test.yaml` before any release; its AGENTS.md defines contribution invariants every agent inherits. The aptu repository takes a CI-authoritative approach: a `just check` recipe approximates the CI gate locally, while CI remains the final enforcer. Both patterns are defensible; local guardrails reduce what CI has to catch, and CI enforces what local discipline cannot. For the structured handoff chain that makes decisions reconstructible, see [Orchestrating AI Agents: A Subagent Architecture](/posts/orchestrating-ai-agents-subagent-architecture). ## How Does the Attestation Chain Close the Loop? The comprehension, review, and CI layers converge at release time. The attestation chain seals the evidence linking each artifact to its source, build, and required approvals. ### From Commit to Release: A Named Human at Every Step The aptu repository's AI_POLICY.md, its machine-readable policy manifest enforced by CI, states: "This policy is backed by enforced controls: GPG-signed commits, Developer Certificate of Origin (DCO), required code owner review, SLSA (Supply-chain Levels for Software Artifacts) Level 3 build provenance, and OpenSSF Best Practices Silver. These are not decorative. They ensure that a named, verified human is accountable for every change that reaches users." Each control maps to a named CI gate: a verify-tag-signature job blocks lightweight or unverified tags before release runs, CODEOWNERS enforces reviewer accountability, and an isolated reusable workflow produces SLSA Level 3 provenance. ```mermaid %% Release attestation chain from GPG-signed tag through CI build to cosign signature and SLSA provenance predicate graph TD TAG["GPG-Signed Tag"] VERIFY["Verify Tag"] BUILD["CI Build"] ARTIFACT["Artifact + SHA256"] COSIGN["cosign sign-blob"] SLSA["SLSA Provenance"] RELEASE["Verified Release"] TAG --> VERIFY VERIFY --> BUILD BUILD --> ARTIFACT ARTIFACT --> COSIGN BUILD --> SLSA COSIGN --> RELEASE SLSA --> RELEASE classDef primary fill:#075985,stroke:#282728,stroke-width:2px,color:#fff classDef secondary fill:#e6e6e6,stroke:#ece9e9,stroke-width:2px,color:#282728 classDef accent fill:#38bdf8,stroke:#075985,stroke-width:2px,color:#fff classDef muted fill:#fdfdfd,stroke:#ece9e9,stroke-width:1px,color:#282728 linkStyle default stroke-width:3px class TAG,RELEASE primary class VERIFY,BUILD secondary class ARTIFACT,COSIGN,SLSA accent ``` *Figure 3: The release attestation chain. Two signatures with distinct verification properties: cosign writes to a public transparency log; GitHub native attestation produces a queryable SLSA provenance predicate.* ### Two Independent Signatures Per Release Artifact Each release artifact carries two independently verifiable signatures (Figure 3): cosign's resolves against the public transparency log, while GitHub's native attestation resolves against a queryable SLSA provenance predicate. A user downloading any release binary can verify that the artifact matches what the CI build produced and that the signing event was publicly logged, addressing the risk covered in [AI Supply Chain Attack Vectors](/posts/ai-supply-chain-attack-vectors). *Code Snippet 5: `.github/workflows/build-and-attest.yml`, artifact signing via cosign sign-blob (transparency log) and SLSA Level 3 build provenance attestation.* ```yaml file=".github/workflows/build-and-attest.yml" {3-5,14} - name: Sign tarball with cosign run: | cosign sign-blob --yes \ --bundle "${{ steps.upload-cli.outputs.tar }}.bundle" \ "${{ steps.upload-cli.outputs.tar }}" - name: Upload bundle to release env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} run: | gh release upload $RELEASE_TAG \ "${{ steps.upload-cli.outputs.tar }}.bundle" --clobber # actions/attest-build-provenance v4 (full commit SHA pin) - name: Attest build provenance uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373 with: subject-path: ${{ steps.upload-cli.outputs.tar }} ``` ## How Should Engineering Leaders Measure the Stack? Engineering leaders should measure the governance stack with provenance-backed KPIs, not raw AI activity metrics. Two studies that appear contradictory resolve once governance scaffolding maturity is accounted for. ### Reconciling the METR and Enterprise Longitudinal Results A 19% measured slowdown ([Becker et al., 2025](https://arxiv.org/abs/2507.09089)) and a 31.8% cycle-time reduction ([Kumar et al., 2025](https://arxiv.org/abs/2509.19708)) are not contradictory. METR measured experienced open-source contributors working in their own repositories with early-2025 tooling; Kumar measured a mature in-house platform over a full year with adoption ramp-up and later models. Differences in scaffolding maturity and model generation are plausible contributors to the gap; neither study isolates either variable. The direction is consistent: governance infrastructure compounds over time. ### Five KPIs That Require Provenance, Not Just Metrics Five observable KPIs track the stack's effect rather than the technology's raw output. Each requires decision provenance, the [AI Observability Gaps](/posts/ai-observability-gaps) pattern of event-level logs reconstructing who reviewed a change, under what context, and with what outcome. A minimal event record covers the review-side essentials: `{pr_id, actor, decision, context_tokens, cost_usd, timestamp}`; specification errors can pass CI while still breaking end to end. Without that infrastructure, these become lagging indicators no one can act on. Vendor-cited leaderboard scores are directional signals, not procurement criteria. SWE-bench Pro shows frontier models reaching at most 23% success on long-horizon tasks, against 70%+ on the prior easier benchmark ([Deng et al., 2025](https://arxiv.org/pdf/2509.16941)). These KPIs are designed to surface that gap. *Table 3: Five observable KPIs for AI SDLC governance stack measurement. Inference cost trend: [Gundlach et al., 2025](https://arxiv.org/abs/2511.23455).* | KPI | What it measures | Data source | Maturity signal | |---|---|---|---| | AI-generated code fraction by layer | Feature, infra, or test code share | PR metadata plus review labels | Rising, tracked not capped | | Review escalation rate | Auto-approve vs. human-review vs. reject share | Automated review output (approve / escalate / reject) | Stable or falling | | Cost-per-resolved-PR | AI review and comprehension cost per PR | Review plus comprehension cost telemetry | Falling with inference costs | | Security gate pass rate | First-submission SARIF pass rate, 30-day rolling | SARIF plus Code Scanning first-submission data | Rising with maturity | | Specification error rate | Inter-agent misalignment and handoff failures at integration boundaries | CI failure logs and stage-boundary contract violations | Falling with contract maturity | The KPIs also surface two organizational shifts worth tracking: real-world productivity gains typically run 5-15%, not 10x ([getDX, 2026](https://getdx.com/blog/ai-productivity-gains-are-10-percent-not-10x/)). Results can be higher in bounded workflows with explicit measurement, and the bottleneck moves from code production to specification quality and reviewer capacity. Planning sessions should shift time from task estimation to problem framing and output contracts. These five KPIs are scoped to the governance stack; burnout and adoption signals belong to the program layer covered in [AI Adoption for Engineering Leaders](/posts/ai-adoption-engineering/). ## Where Does the Governance Stack Go Next? Engineering leaders should instrument decision provenance, cap automated iteration depth, and track cost-per-resolved-PR before expanding AI code generation volume. ### Three Concrete Steps for Engineering Leaders First, instrument before automating: deploy decision-provenance observability along the [AI Observability Gaps](/posts/ai-observability-gaps) pattern before expanding AI code generation volume, since the KPIs above depend on that infrastructure already being in place. Second, gate iteration depth: enforce a bounded refinement cap, for example three refinement cycles per PR, in CI before a change is escalated; security-degradation research documents a 37.6% increase in critical vulnerabilities across five iterative AI refinement rounds and recommends human validation between iterations ([Shukla et al., 2025](https://arxiv.org/abs/2506.11022)). Add the cap as a hard CI failure condition, not a cultural norm that erodes under deadline pressure. Third, price the stack explicitly: track cost-per-resolved-PR over time. Absolute cost will fall as inference costs decline ([Gundlach et al., 2025](https://arxiv.org/abs/2511.23455)), but the value of review discipline embedded in the stack will not. ### The Governance Stack Is the Scaffolding, Not the Tax AI code tooling is not the bottleneck; the quality of its governance scaffolding is. Confirm three conditions before scaling further: decision provenance is instrumented; automated iteration is capped at three refinement cycles per PR before escalation; and cost-per-resolved-PR is tracked as a longitudinal baseline. Leaders who can answer yes to all three can scale with evidence. Those who cannot are scaling risk. ## References - Becker, J. et al. (METR), "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity" (2025): https://arxiv.org/abs/2507.09089 - Cemri, M. et al. (UC Berkeley), "Why Do Multi-Agent LLM Systems Fail?" (2025): https://arxiv.org/abs/2503.13657 - Cihan, U. et al., "Automated Code Review In Practice" (ICSE 2025): https://arxiv.org/abs/2412.18531 - Deng, X. et al., "SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?" (2025): https://arxiv.org/pdf/2509.16941 - getDX (Noda, A. & Reock, J.), "AI productivity gains are 10%, not 10x" (2026): https://getdx.com/blog/ai-productivity-gains-are-10-percent-not-10x/ - Gundlach, H. et al., "The Price of Progress: Algorithmic Efficiency and the Falling Cost of AI Inference" (2025): https://arxiv.org/abs/2511.23455 - Kumar, A. et al., "Intuition to Evidence: Measuring AI's True Impact on Developer Productivity" (2025): https://arxiv.org/abs/2509.19708 - Mei, L. et al., "A Survey of Context Engineering for Large Language Models" (2025): https://arxiv.org/abs/2507.13334 - Shlomov, S. et al., "Governance by Construction for Generalist Agents" (2026): https://arxiv.org/abs/2605.20874 - Shukla, S. et al., "Security Degradation in Iterative AI Code Generation: A Systematic Analysis of the Paradox" (2025): https://arxiv.org/abs/2506.11022 - Sun, T. et al. (ByteDance), "BitsAI-CR: Automated Code Review via LLM in Practice" (FSE 2025): https://conf.researchr.org/details/fse-2025/fse-2025-industry-papers/24/BitsAI-CR-Automated-Code-Review-via-LLM-in-Practice - Zhang, Q. et al., "Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models" (2025): https://arxiv.org/abs/2510.04618