How Warden works.
Warden is a context governance and verification layer for AI coding agents. It manages context across eight layers — code intelligence, context selection, tool output pruning, agent memory, response compression, file compression, MCP description compression, and session continuity — and verifies every optimization is safe via a shadow-mode eval gate. Every cut is reversible via CCR.
The eight layers
Each layer targets a different source of token waste. Together they cover the full context lifecycle — from understanding the codebase before work starts, to handing off state between sessions.
Code intelligence
Warden indexes your project's code structure — functions, classes, imports, call sites — using tree-sitter WASM for structural parsing. 30+ languages including TypeScript, JavaScript, Python, Go, Rust, Java, C, C++, C#, Ruby, PHP, Swift, Kotlin, and more. One call replaces dozens of grep/read cycles: who calls a function, what's affected by a change, the project's architecture in one shot. Incremental: only changed files are re-parsed.
Context selection
Given a task, Warden scans the project, reads the top-ranked files, and extracts the sections relevant to your task. Works on code (function/class blocks), markdown (sections), JSON/YAML (key context windows), and text (paragraph matching). The agent gets verbatim content + structural outlines of the rest — not a file list, but the cut content itself. When the code index is available, 2-hop symbol expansion auto-includes dependency signatures. Typically 80%+ smaller than reading full files.
Tool output pruning
Every tool call goes through Warden's wrapper tools. Content-aware routing auto-detects JSON, grep output, test logs, and source code — routing each to its optimal pruning module. Grep returns relevant matches (with duplicate deduplication), file reads return slices + AST-based outlines (when code index is available), test runs keep failures. Code is never rewritten. Every cut is reversible: CCR caches the original in SQLite with a hash key, and the agent can retrieve the full output or a slice of it (around a symbol, or explicit line range) via warden_retrieve. Auto-expires after 7 days.
Agent memory
When the agent makes a durable decision, Warden stores it in local SQLite with FTS5 full-text search. On future tasks, relevant memories are recalled automatically — decisions persist across sessions. Memories auto-surface in warden_status at session start. Conflict detection warns when new decisions contradict old ones. Failed tasks auto-save as memory patterns so the agent doesn't repeat mistakes.
Response compression
Warden's rules files tell the agent to drop filler, pleasantries, and self-narration from its responses. Code, commands, errors, and technical terms stay verbatim. The agent states the fix, not the reasoning process. Max compression, always on — no config, no levels. Includes auto-clarity for safety warnings: switches to full prose for security warnings, irreversible actions, and confused users.
File compression
Memory files (CLAUDE.md, AGENTS.md) load into context every session. Verbose ones waste tokens forever. warden compress strips filler deterministically — no LLM call, free, instant, offline. Sentence scoring drops low-value prose, keeps technical content. Code blocks, paths, commands preserved byte-for-byte. Original backed up automatically. Up to 32% reduction. No config needed.
MCP tool description compression
Warden compresses its own tool descriptions before sending them to the agent. Tool descriptions sit in context for the entire session, so compressing them saves input tokens on every turn. Equivalent to caveman-shrink, but built in. No proxy, no middleware, no extra package.
Session continuity
Sessions end. Context windows fill up. The rules file tells the agent to call warden_handoff with read=true at session start (returns the previous session's handoff) and warden_handoff without read at session end or before compaction (generates a new one). The handoff document covers decisions made, task outcomes, files touched, and pruning decisions — under 300 words. Incremental windowing: each handoff covers only the window since the previous one. The next session starts with context, not from scratch.
Verification
The real problem Warden solves: developers can't tell whether context optimization silently degraded their agent. Warden provides compression with evidence, not compression with assumptions.
Trust guard
Every pruned output is verified: every non-annotation line must appear verbatim in the raw output, or the raw is shipped instead. No silent rewrites, ever. If a pruning module has a bug, the guard catches it and falls back to raw.
Eval gate (shadow → canary → active)
Every pruning rule starts in shadow mode. Warden runs it in parallel with raw output, scores parity, and only promotes once confidence is proven. Built-in rules are active by default — savings from first install. Enterprise mode starts in shadow.
Task outcome tracking
After each task, the agent reports success/failure. Warden correlates this with whether pruning was active. If pruned success rate drops below raw, that's a regression — auto-reverted. This is the evidence that compression didn't degrade outcomes.
Regression watchdog
A continuous watchdog re-runs canary tasks. If a model update, caching bug, or over-aggressive prune rule causes a regression, Warden auto-reverts to the last known-good config. You don't notice — Warden already fixed it.
How do I know it's working?
Six ways to see what Warden is doing — from inline annotations on every tool call to full audit trail exports.
Status line at session start
The agent calls warden_status and prints: "Warden active — X tokens saved (Y% reduction), Z rules live." You see this every time you start a session.
Memory recall at session start
The agent calls warden_memory_recall with your task query and prints any relevant past decisions. You see what Warden remembers from previous sessions.
Per-call savings annotations
Every pruned tool output includes a summary line showing tokens saved. You see the savings on every grep, file read, and test run.
warden doctor — full health check
Checks MCP registration, rules files, pruning engine, memory system, FTS5 index, code index, CCR cache, outcome tracking, budget tracker, and database integrity. 10 checks, clear pass/fail report.
warden status — detailed stats
Shows every rule, its stage, confidence, tokens saved, percentage, and call count. Plus total saved, total processed, and reduction percentage.
warden outcomes — regression proof
Shows task success rates with and without pruning. If pruning is causing regressions, it tells you. This is the evidence that compression didn't degrade outcomes.
Ready to install?
Five steps. One command does everything. See the getting started guide for the full walkthrough.