Skip to content

[TASK] Extend storage backend to scheduled jobs, plans, and shell history #485

@edenreich

Description

@edenreich

Summary

Today only conversations and session groups respect cfg.Storage.Type
(jsonl / sqlite / postgres / redis / memory). Every other piece of
app-managed state — scheduled jobs, plan-mode plans, and shell
history
— is hard-coded to the local filesystem regardless of backend.
That breaks the expectations of users who pick Postgres/Redis for
multi-host or ephemeral deployments: their jobs and history silently stay
on whichever box ran the agent last, and a containerised run loses
everything on restart.

This refactor extends the storage backend to cover the three remaining
app-managed artifact types. User-edited YAML/JSON configs (config.yaml,
prompts.yaml, channels.yaml, agents.yaml, keybindings.json,
shortcuts/*.yaml) stay on disk by design — humans need to edit them.
Logs and screenshots stay on disk — they are large/streaming and the
filesystem is the right tool. Session groups are already backend-aware
(verified all five backends implement SessionGroupStorage).

Current state

Artifact Path Code
Scheduled jobs ~/.infer/schedules/<uuid>.yaml (fsnotify-watched) internal/services/scheduler/store.go, internal/services/scheduler/scheduler.go
Plan-mode plans <configDir>/plans/<ts>-<slug>.md internal/agent/tools/request_plan_approval.go (writePlanFile)
Shell history .infer/history (append-only) internal/ui/history/shell_history.go

Proposed design

  1. Follow the SessionGroupStorage pattern. Add three new typed
    interfaces in internal/infra/storage/interfaces.go:

    • ScheduledJobStorageSaveJob, LoadJob, ListJobs,
      DeleteJob, Watch(ctx) <-chan ScheduledJobChangeEvent.
    • PlanStorageSavePlan, LoadPlan, ListPlans, DeletePlan.
      Plan record: {ID, Title, Slug, CreatedAt, Body string}.
    • ShellHistoryStorageAppendHistory, LoadHistory(limit int).
      Implement each in all five backends.
  2. Single global backend. cfg.Storage.Type drives all four storage
    domains. No new top-level config keys. Users who want
    schedules/plans/history on disk can pick the jsonl backend (the
    default) and keep file-based behaviour.

  3. Refactor storage.NewStorage to return a Stores aggregate holding
    ConversationStorage, SessionGroupStorage, ScheduledJobStorage,
    PlanStorage, ShellHistoryStorage. Update internal/container/container.go
    and cmd/export.go call sites.

  4. Scheduler change-notification: per-backend strategy.

    • JSONL backend: keep the existing fsnotify watcher on the storage
      directory — wrap it inside the JSONL backend's
      ScheduledJobStorage.Watch impl. Zero behavioural change for the
      default install.
    • SQLite / Postgres: 2 s polling reconcile against updated_at.
    • Postgres (optional follow-up): LISTEN/NOTIFY for lower latency.
    • Redis: PSUBSCRIBE on __keyspace@*__:infer:schedules:*.
    • Memory: in-process broadcast on Save/Delete.

    internal/services/scheduler/scheduler.go becomes backend-agnostic and
    just consumes events from the channel. internal/services/scheduler/store.go
    gets deleted; its YAML+atomic-write logic moves into the JSONL backend.
    internal/agent/tools/schedule.go stops calling scheduler.NewStore and
    uses the injected ScheduledJobStorage instead.

  5. Plans return a stable URI, not a path. Currently
    RequestPlanApproval returns the absolute markdown path in its tool
    result. Migrate to infer://plans/<id> plus an inline preview snippet.
    Add a new infer plans show <id> command (cmd/plans.go) so users on
    a DB backend can read plans without poking at storage. JSONL backend
    continues to write the file at the historical path, so existing
    workflows that cat plan files keep working on the default backend.
    Inject PlanStorage into the tool via the existing tool-config struct.

  6. ShellHistory becomes a thin shim around ShellHistoryStorage.
    The public ShellHistoryProvider interface in
    internal/ui/history/shell_history.go stays unchanged so callers
    in internal/ui/... don't move.

  7. Migration on first run. Add internal/infra/storage/migrate.go
    exposing MigrateLegacyArtifacts(ctx, cfg, stores) error. Idempotent.
    On startup of infer channels-manager, infer agent, and infer chat:

    • If ~/.infer/schedules/*.yaml files exist AND the configured backend
      is not jsonl, import them via ScheduledJobStorage.Save, then
      rename the directory to ~/.infer/schedules.migrated-<timestamp>/.
      Don't delete — leaves a recoverable backup.
    • Same pattern for <configDir>/plans/*.md and .infer/history.

Files to modify

  • internal/infra/storage/interfaces.go — add 3 new interfaces + change-event type
  • internal/infra/storage/factory.go — return Stores aggregate
  • internal/infra/storage/{jsonl,sqlite,postgres,redis,memory}.go — implement them
  • internal/infra/storage/migrations/{sqlite,postgres}_migrations.go — schema migrations for scheduled_jobs, plans, shell_history
  • internal/infra/storage/migrate.gonew
  • internal/services/scheduler/scheduler.go — drop fsnotify, consume Watch
  • internal/services/scheduler/store.godelete
  • internal/agent/tools/schedule.go — use injected ScheduledJobStorage
  • internal/agent/tools/request_plan_approval.go — use PlanStorage, return URI
  • internal/ui/history/shell_history.go — shim over ShellHistoryStorage
  • internal/container/container.go — wire 3 new stores
  • cmd/channels.go — pass backend store to scheduler
  • cmd/export.go — update storage.NewStorage call site
  • cmd/plans.gonew, infer plans show <id>
  • tests/mocks/ — regenerate counterfeiter mocks for the new interfaces
  • docs/scheduling.md, docs/plan-mode.md, CLAUDE.md — describe backend-aware storage and migration

Out of scope

  • Session groups (already done — confirmed all 5 backends implement SessionGroupStorage).
  • All *.yaml / *.json user-editable configs (humans edit them).
  • Logs and screenshots (filesystem is the right tool).
  • Conversation token/cost stats (already covered by ConversationMetadata).

Acceptance Criteria

  • ScheduledJobStorage, PlanStorage, ShellHistoryStorage interfaces exist in internal/infra/storage/interfaces.go and follow the SessionGroupStorage pattern.
  • All five backends (jsonl, sqlite, postgres, redis, memory) implement the three new interfaces; storage.NewStorage returns an aggregate Stores struct.
  • Schedule tool, scheduler service, plan-approval tool, and shell history all read/write through the injected backend stores; internal/services/scheduler/store.go is deleted.
  • On JSONL backend, all three artifact types continue to land at their historical paths (~/.infer/schedules/<id>.yaml, <configDir>/plans/<ts>-<slug>.md, .infer/history) — zero behavioural change for default installs.
  • On SQLite/Postgres/Redis, scheduling a job from infer agent is observed by a separate infer channels-manager process within ~2 s and fires on schedule (cross-process Watch works).
  • RequestPlanApproval returns an infer://plans/<id> URI in addition to (or instead of, on non-JSONL backends) the file path; a new infer plans show <id> command renders the markdown body from any backend.
  • MigrateLegacyArtifacts imports legacy on-disk schedules/plans/history into the configured backend on first run and renames the legacy directory with a .migrated-<ts> suffix; idempotent on subsequent runs.
  • Backend-conformance test suite parameterised over all five backends covers each new interface; existing scheduler/plan-tool/shell-history tests pass against fakes for the new interfaces.
  • task fmt && task lint && task test is clean; counterfeiter mocks regenerated.
  • docs/scheduling.md, docs/plan-mode.md, and CLAUDE.md describe backend-aware storage and migration behaviour.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions