Progressive note loading — compact index with on-demand full content

[bm-clawd]

Idea

Inspired by OpenViking's L0/L1/L2 progressive loading and Hermes Agent's "hot memory vs cold recall" architecture.

Problem

When agents work with BM notes, they currently load full note content. For large knowledge graphs, this means either:

• Stuffing too many tokens into context (expensive, slow)
• Loading too few notes and missing relevant context

Proposal

Implement tiered note loading through the MCP tools:

L0 — Index (always cheap to inject)

• Title, type, tags, status
• Relation summary (connected entity names)
• ~50-100 tokens per note

L1 — Observations (medium cost)

• All observations with categories
• Key relations with types
• ~200-500 tokens per note

L2 — Full content (on demand)

• Complete markdown body
• All relations with context
• Attachments/references

The search_notes and build_context tools could return L0 by default, with a depth or detail parameter to request L1/L2. This keeps agent prompts compact while making the full graph accessible on demand.

Benefits

• Dramatically fewer tokens for knowledge graph navigation
• Agents can scan more notes before deciding which to read fully
• Better prompt cache stability (compact index stays stable)
• Aligns with how humans browse — scan titles, then dive in

References

• OpenViking L0/L1/L2: github.com/volcengine/OpenViking
• Hermes Agent memory layers (hot prompt memory + cold FTS recall)
• Relates to Momentum UI progressive rendering

[claude]

Claude finished @bm-clawd's task in 1m 47s —— View job

---

Issue Triage Complete

• Read issue and understand request
• Explore relevant codebase sections
• Check for duplicate issues
• Apply labels and complete triage

---

Classification

Field	Assessment
Type	Feature Request
Priority	Medium
Component	MCP Tools (search_notes, build_context) + Schemas
Complexity	Complex

Labels applied: enhancement

---

Analysis

No duplicates found. Issue #689 (memory budget indicators) is related but different.

Current state of relevant tools:

• search_notes — returns full search results with matched_chunk excerpts and complete entity data. No detail parameter exists. Already returns somewhat compact data (matched chunks, not full content).
• build_context — already has a depth parameter (graph traversal hops, not content detail level) and timeframe. Returns hierarchical context with observations, relations, and related entities — always full content.

What the proposal asks for:
A new detail (or similar) parameter on search_notes and build_context that controls how much of each note's content is returned:

• L0: Title, tags, type, relation summary (~50-100 tokens)
• L1: Observations + key relations (~200-500 tokens)
• L2: Full content (current behavior)

---

Implementation Considerations

This is a complex feature requiring changes across multiple layers:

1. New schema types — EntitySummaryL0, EntitySummaryL1 response variants, or a unified schema with optional fields controlled by detail level
2. MCP tool parameters — add detail: Literal["L0", "L1", "L2"] = "L2" (or similar) to search_notes and build_context
3. Service/repository layer — queries need to optionally suppress content fields or the formatting needs to truncate/omit based on level
4. Prompt/text formatting — the output_format="text" serializers need to respect the detail level
5. Tests — unit + integration tests for each detail level
6. Note: build_context already has a depth parameter (hops), so the new parameter must be named distinctly (e.g., detail or content_level) to avoid confusion

Key design question: Should L0/L1/L2 be an enum or should it be a boolean flag like compact=True? The L0/L1/L2 naming is clear for agents but could be confusing. An alternative would be content_detail: Literal["index", "summary", "full"].

---

Recommended Next Steps

1. Design the schema changes and parameter naming
2. Prototype L0 output for search_notes (easiest entry point — already returns partial data)
3. Add content_detail parameter to build_context (careful not to conflict with existing depth)
4. Update text formatters accordingly
5. Add tests