Miadi Review

Distilling Agent Session Traces into Scalable Institutional Memory

Oct 17, 2026 · model: manual-edit · channel: Boundary

Watch source

Distilling Agent Session Traces into Scalable Institutional Memory

Source: Why AI Coding Agents Need a Memory Pipeline — Boundary https://youtu.be/mAKUL12h1n8

Summary

Dexter Horthy and Aaron Villalpando demonstrate how to extract persistent memory, architectural rules, and tribal knowledge from thousands of AI coding session traces. By pairing custom web tooling with BAML's typed prompt pipelines, aggressive message pruning, SQLite caching, and specialized small classifier models, they turn costly multi-turn agent mistakes into concise, repo-specific instructions for cents per session.

Intended Use & Continuation

The medicine wheel memory and its relationship obviously with talking Circle and wasn’t need to be part of the memory of your relieve lacking that. All agents currently working with JEV should have to consider thinking about this of the guys are doing it

Aspects

Build Human Tooling Before Building Pipelines (13:50-14:55)

Rather than relying on notebooks or generic observability dashboards, the team built bespoke web interfaces to inspect raw messages, token usage, and filtering cutoffs. Visualizing agent trajectories directly enabled them to identify high-signal human corrections and spot data truncation bugs before formalizing the memory pipeline.

Explore with Agents, Ship a Workflow (15:05-15:45)

Rather than deploying an autonomous agent with open-ended MCP tools across the entire dataset, Aaron used Claude Code to explore session logs. Once manual agent exploration revealed stable investigative patterns, they froze that path into a deterministic four-stage typed pipeline inside BAML.

Zero-Cost Prompt Iteration via Request Hashing (18:20-19:35)

Re-running multi-step LLM pipelines across gigabytes of traces quickly becomes unaffordable. By hashing incoming prompt schemas and storing raw completions on disk with SQLite, developers could iterate on single prompts or downstream evaluators without repaying for upstream extractions.

Selective Pruning Preserves High-Recall Knowledge (20:30-22:15)

Naively dumping entire agent chat histories bloats token budgets. The authors trimmed verbose tool calls and truncated long messages in the middle, preserving initial prompts and concluding summaries. This retained 100% of crucial repo discoveries while cutting overall processed token volume by 40%.

End-results and Core Aspect

Core Aspect — Automated Distillation of Institutional Repo Knowledge (07:45-08:45)

The pipeline analyzes multi-session developer interactions to distill recurring agent inquiries, human corrections, and codebase navigation maps into structured CLAUDE.md files, eliminating redundant tool exploration and steering future coding agent runs.

Results

  • 97% Cost Reduction via Small Model Classification (36:35-37:50): Delegating initial filtering, scoring, and classification tasks to specialized low-cost models like Jev handled 97% of pipeline calls for just 5% of the total API bill, reducing whole-repo memory extraction to cents per run.
  • Targeted Context Shards Mapped to Submodules (08:00-09:35): The engine synthesized localized rules and directory guidance mapped specifically to sub-crates. Instead of overloading global context, agents receive localized instructions when navigating difficult or non-standard compiler paths.
  • SQL-Queryable Program Observability via BAML Query (38:05-39:50): By automatically logging step spans and model interactions into a local SQLite database, developers could run ad-hoc SQL queries across execution traces to power bespoke debug viewers and inspect intermediate outputs without manual telemetry code.