Skip to content

development.md

Latest commit

 

History

History
604 lines (460 loc) · 22.9 KB

development.md

File metadata and controls

604 lines (460 loc) · 22.9 KB
Raw

Development Guide

Development setup and workflows for get-tbd (the tbd CLI).

Prerequisites

  • Node.js 22.12.0 or newer
  • pnpm (will be installed automatically via corepack)

Setup

# Enable corepack (includes pnpm)
corepack enable

# Install dependencies
pnpm install

# Install git hooks
pnpm prepare

Development Workflow

Running the CLI from source

During development, run the CLI directly from TypeScript source (no build needed):

pnpm tbd --help
pnpm tbd list
pnpm tbd create "My issue" -t bug

Running the built CLI

To test the production build:

pnpm build
pnpm tbd:bin --help

Testing new shortcuts, guidelines, or templates

Important: When adding new documentation files (shortcuts, guidelines, templates) to packages/tbd/docs/, you must test with the local build, not the globally installed tbd.

The globally installed tbd has its own bundled docs from the published npm package. Running tbd setup --auto with the global installation won’t include your new files.

# 1. Add your new file to packages/tbd/docs/shortcuts/standard/my-shortcut.md

# 2. Build to bundle the new file into dist/docs/
pnpm build

# 3. Run setup with the LOCAL build (from repo root)
node packages/tbd/dist/bin.mjs setup --auto

# 4. Verify the file was copied to .tbd/docs/
ls .tbd/docs/shortcuts/standard/my-shortcut.md

# 5. Test the shortcut with the local build
node packages/tbd/dist/bin.mjs shortcut my-shortcut

Never manually create files in .tbd/docs/—always add them to packages/tbd/docs/ and let setup --auto copy them. This ensures the setup process works correctly for users.

Testing the packaged installation

To test the CLI exactly as users would install it from npm:

# Build, pack, and install globally (like npm install -g get-tbd)
pnpm test:install

# Test the installed binary
tbd --help

# Uninstall when done
pnpm test:uninstall

This creates an npm tarball and installs from it, validating the full package structure.

Developing tbd web

Build the production artifact, then run the same entry point the package publishes:

pnpm --filter get-tbd build
node packages/tbd/dist/bin.mjs web

The command prints the selected loopback URL and stays in the foreground. The server implementation lives in packages/tbd/src/cli/web/, the Commander handler in packages/tbd/src/cli/commands/web.ts, and the strict browser client in packages/tbd/src/web/. packages/tbd/scripts/stitch-web.mjs inlines the browser IIFE and CSS into the single published dist/web/index.html artifact. The component, typography, color, tree, conditional-facet, tooltip, and bounded-sort rules are documented beside their implementations in the authoritative design-system inventory at the top of packages/tbd/src/web/styles.css.

Preserve the agent interaction contract across those surfaces: a natural request to see beads in a browser routes the installed skill to tbd web --open; the agent owns the foreground process and all ordinary tbd mutations; the page is a live viewer, not an editor; and starting it never performs remote exchange. When the target project is outside the agent’s current working directory, the skill must use tbd web <path> --open. The CLI accepts a repository or subdirectory, supports an initialized repository with no beads, and applies the standard initialization error to an existing non-tbd directory. The source skill tiers, welcome-user, README, CLI manual, design, setup output, command startup output, and browser copy must state that contract consistently. integration-files.test.ts, setup-flows.test.ts, golden-output.test.ts, cli-web.tryscript.md, and web-server.test.ts pin the route from packaged docs through installed and rendered artifacts.

Run the focused behavior and package proofs while iterating:

pnpm --filter get-tbd exec vitest run \
  tests/web-*.test.ts tests/cli-web.test.ts tests/bead-web-css.test.ts
pnpm --filter get-tbd exec tryscript run tests/cli-web.tryscript.md
pnpm --filter get-tbd qa:web-package
pnpm --filter get-tbd qa:upgrade-package

The spawned-process acceptance test creates real writer and viewer clones, proves the viewer does not fetch a remote change, runs explicit tbd sync, consumes the resulting local SSE update, and verifies Git isolation and signal cleanup. Unit coverage also proves the one-second reconciliation fallback, metadata-only publication, graph/state-version race ordering, ref-rewind-safe event replay, explicit queued-byte backpressure, closed-stream race isolation, bounded local delta detail, and browser recovery across an observer restart. The package proof launches tbd web from an extracted npm tarball in a disposable, initialized repository and verifies that its self-contained page and APIs work without changing the source checkout. In a release job, TBD_VERSION_OVERRIDE also makes the proof require the packed CLI to report the exact package/tag version. Design and implementation details are in plan-2026-08-10-tbd-web-live-bead-view.md.

Developing external tracker integrations

Build first and exercise the published entry point; integration acceptance tests spawn that same binary in a real repository:

pnpm --filter get-tbd build
pnpm --filter get-tbd exec vitest run tests/integrations-*.test.ts
pnpm --filter get-tbd exec vitest run tests/integration-cli-e2e.test.ts
node packages/tbd/dist/bin.mjs integration status --offline

Provider-neutral policy, links, reconciliation, intents, and orchestration live in packages/tbd/src/integrations/core/; provider transports and mappings live below integrations/<provider>/; packages/tbd/src/cli/commands/integration.ts owns CLI translation only. The engine must validate duplicate external IDs before journal replay or provider I/O, quarantine every holder, and continue unrelated pairs. Never put a credential in process.env, committed bridge state, output, or a test fixture.

Live Linear validation is API-driven and follows packages/tbd/tests/qa/linear-integration.qa.md. Run pnpm --filter get-tbd qa:linear-live -- --team <KEY> --project <NAME> with a gitignored LINEAR_API_KEY; the runner exercises the built CLI in a disposable git repository, archives its provider fixtures, and emits the stable scenario IDs enforced by the import-safe scripts/provider-live-qa-contract.ts checklist shared by future provider drivers. --project is required so project-isolation proof cannot be skipped. The outbound field scenario also reads the provider description back through GraphQL and verifies the current managed-block delimiters while rejecting the legacy format. Manual UI testing is a parity check, not release evidence. The complete Linear RC, GitHub issue/PR, and read-only web projection work map is plan-2026-08-10-external-tracker-integrations.md.

Building

# Build all packages
pnpm build

# Watch mode for development
pnpm --filter get-tbd dev

Testing

# Run tests
pnpm test

# Run tests with coverage
pnpm test:coverage

# Watch mode
pnpm --filter get-tbd test:watch

Formatting and Linting

# Format code (auto-fix)
pnpm format

# Check formatting (CI)
pnpm format:check

# Lint with auto-fix
pnpm lint

# Lint check only (CI)
pnpm lint:check

# Type check
pnpm typecheck

Validating Package

# Validate package.json exports
pnpm publint

Git Hooks

Git hooks are managed by lefthook and run automatically:

  • pre-commit: Format, lint, and typecheck staged files
  • pre-push: Build (if needed), run tests, and—when a package.json is staged—enforce the 14-day package-age rule via pnpm check:package-age.

To skip hooks (emergency only):

git commit --no-verify
git push --no-verify

Dependency Hygiene: The 14-Day Package-Age Rule

This repo enforces the 14-day package-age rule documented in packages/tbd/docs/guidelines/pnpm-monorepo-patterns.md: do not install or upgrade to any package version less than 14 days old.

  • pnpm upgrade:check, pnpm upgrade, and pnpm upgrade:major are wired to ncu --cooldown 14; they will refuse to bump to versions inside the window.
  • pnpm check:package-age (also wired into pre-push) scans every package.json in the repo, queries the npm registry for each pinned version’s publish time, and exits non-zero on any pin under 14 days. Add --warn to report without failing.
  • Exceptions (CVE patches inside the window) must be documented in the commit message or PR description with CVE ID, upstream link, and a Reviewed-by: line.

The check requires registry access (https://registry.npmjs.org); skip it with LEFTHOOK_EXCLUDE=package-age git push only if you’re pushing infrastructure changes that do not touch dependencies.

Commit Conventions

We use Conventional Commits for commit messages.

Format

<type>: <description>

[optional body]

[optional footer(s)]

Types

  • feat: New feature
  • fix: Bug fix
  • docs: Documentation for the product or main codebase (not process docs)
  • process: Documentation or tooling changes to agent or human development processes
  • style: Code style (formatting, no logic change)
  • refactor: Code change that neither fixes a bug nor adds a feature
  • test: Adding or updating tests
  • chore: Maintenance tasks (deps, config, etc.)

Examples

feat: Add support for custom labels

fix: Handle empty issue list gracefully

docs: Update CLI usage examples

process: Add TDD guidelines for agent workflows

test: Add golden tests for sync command

chore: Update dependencies

Notes

  • No scope by default: Don’t include a scope like fix(tbd): for the main codebase. Only use a scope when it provides key disambiguation or clarification (e.g., fix(parser): vs fix(cli): when the distinction matters).
  • Keep the first line under 72 characters
  • Use imperative mood ("Add feature" not “Added feature”)

Creating Releases

Releases are tag-triggered and assembled from clean conventional commits—we do not use Changesets (no .changeset/ files, no “Version Packages” PR). get-tbd is a single published package, so the per-PR changeset ceremony isn’t worth it; release notes are composed from the commits since the last tag at release time.

.github/workflows/release.yml runs on a v* tag push: it builds, runs publint, publishes get-tbd to npm, and creates a GitHub Release whose body is the matching ## X.Y.Z section of packages/tbd/CHANGELOG.md.

Release process

  1. Open one release train and obtain explicit human approval for one published version. A merged fix does not trigger a release. Batch related fixes into one candidate; if validation finds another defect, fix that candidate and restart validation instead of publishing an intermediate patch. Any additional version requires new explicit approval. See publishing.md §Step 0 for the emergency exception.
  2. From clean main, review git log <last-tag>..HEAD and choose the version by the substance of the user-facing change, not the commit-type label: a new CLI capability → minor; fixes, docs, and guidance-content changes → patch (even when a commit is labeled feat); breaking → major. Note for 0.x a semver minor is 0.MINOR.0. See publishing.md §Step 2 for details.
  3. On a claude/release-vX.Y.Z branch: bump version in packages/tbd/package.json and prepend a ## X.Y.Z section to packages/tbd/CHANGELOG.md with notes written per release-notes-guidelines.
  4. Run pnpm qa:upgrade-package, pnpm release:verify, and pnpm test. For setup, launcher, installation, fallback, format, or upgrade changes, validate the packed candidate in a fresh first-party downstream checkout before opening the release PR. Record the evidence in the PR and rerun it after any candidate change.
  5. Open the release PR and merge once CI is green.
  6. Gate before tagging: wait until main CI has reached conclusion=success on the merge commit itself (filter the run by that SHA—right after a merge an unfiltered query can return the previous run). Only then tag vX.Y.Z on that exact commit and push it—the Release workflow publishes to npm and creates the GitHub Release. See publishing.md §Step 6 for the exact gate commands.

For the full step-by-step (including the version-bump heuristic, supply-chain review, and verification), see publishing.md.

CI and GitHub Actions

Keep logic out of workflow YAML. Do not put non-trivial shell—multi-line awk, sed, jq pipelines, regex parsing, conditional logic—inline in a GitHub Actions run: step. Inline CI shell cannot be tested or debugged in isolation; the only way to exercise it is to push a tag or branch and wait for the runner, which is slow and error-prone. A real bug shipped this way: the release workflow’s inline awk changelog extractor silently produced an empty body on every release (v0.1.30 and v0.2.0 both went out with the fallback Release vX.Y.Z string) because it exited 0 either way.

Instead, write a clean, unit-tested script and invoke it by reference:

  • Put the pure logic in a src/ module with an exported function and a thin scripts/*.ts CLI wrapper (run via tsx). See src/utils/changelog.ts, its wrapper scripts/extract-changelog.ts, and the test tests/extract-changelog.test.ts. Import source from tests as ../src/...js (avoid importing .mjs from a test—it resolves inconsistently under vitest on Windows).
  • Cover it with a normal vitest test so the behavior is locked in and debuggable locally.
  • In the workflow, the run: step should only call the script (pnpm exec tsx packages/tbd/scripts/extract-changelog.ts …) and wire its output; keep any remaining shell to trivial plumbing (e.g. the GITHUB_OUTPUT heredoc).

This repository uses TypeScript for CI-support programs because Node, tsx, and Vitest are already pinned project tools; it is not a general preference for Node over Python. See ci-and-gates-rules for the language-neutral selection rule.

If you find yourself reaching for awk/sed in a workflow, that is the signal to move it into a script.

Project Structure

tbd/
├── packages/
│   └── tbd/               # Main CLI package
│       ├── src/
│       │   ├── index.ts   # Library entry (node-free)
│       │   ├── cli/       # CLI-specific code
│       │   │   ├── bin.ts
│       │   │   ├── cli.ts
│       │   │   ├── commands/
│       │   │   └── lib/
│       │   ├── lib/       # Core library (schemas, types)
│       │   ├── file/      # File layer
│       │   └── git/       # Git layer
│       └── tests/
├── scripts/               # Development scripts
├── docs/                  # Documentation
└── .github/workflows/     # CI/CD (release.yml publishes on v* tags)

Architecture

See tbd-design.md for the full design document.

Key concepts:

  • File Layer: Markdown and YAML front matter format
  • Git Layer: Sync via dedicated tbd-sync branch
  • CLI Layer: Commander.js with Beads-compatible commands

CLI Patterns

The CLI follows patterns from research-modern-typescript-cli-patterns.md:

  • Base Command pattern for shared functionality
  • Dual output mode (text and JSON)
  • OutputManager for consistent output handling
  • Proper stdout/stderr separation

Worktree Architecture

tbd uses a hidden git worktree to store issue data on the tbd-sync branch while keeping the user’s working directory clean. The sync worktree is anchored under Git’s common directory so the main checkout and any linked worktrees created by tools like Codex all share the same local issue state. See tbd-design.md §2.3 for the full specification.

Why Worktree?

  • Fast search: ripgrep can search issues without git plumbing commands
  • Direct file access: Read/write issues as regular files, no git show/git cat-file
  • Isolated from main: Issues don’t pollute working directory or affect main branch
  • Conflict-free across linked worktrees: One shared worktree owns tbd-sync, and a repo-scoped lock serializes mutations

Path Conventions

.tbd/                               # Config directory (on main branch)
│
│ Committed to the repo:
├── config.yml                      # Project configuration
├── .gitignore                      # Controls what's gitignored below
├── doc-forks/                      # Fork manifest + base snapshots (f05; tbd-design.md §2.9)
├── workspaces/                     # Persistent state (outbox, named workspaces)
│   └── outbox/                     # Sync failure recovery data
│
│ Gitignored (local only):
├── state.yml                       # Local state
├── docs/                           # Installed documentation (regenerated on setup)
└── backups/                        # Legacy local backups

$GIT_COMMON_DIR/tbd/                # Shared by all linked worktrees of this repo
├── layout.yml                      # Common-dir layout metadata (mirrors config's tbd_format)
├── locks/
│   └── data-sync.lock/             # mkdir-based repo-scoped lock
├── backups/                        # Shared migration/repair backups
└── data-sync-worktree/             # Hidden worktree
    └── .tbd/data-sync/             # Actual issue storage (on tbd-sync branch)
        ├── issues/
        ├── mappings/
        ├── attic/
        └── meta.yml

.tbd/doc-forks/ is committed and holds only fork tracking state: the forks.yml manifest plus base/ snapshots that tbd docs update three-way merges against. The doc fork dir itself lives deliberately outside .tbd/ — default docs/tbd/, tracked in git like any other docs.

CRITICAL: Issues must be written to the worktree path ($GIT_COMMON_DIR/tbd/data-sync-worktree/.tbd/data-sync/issues/), NOT the direct path (.tbd/data-sync/issues/). The direct path is gitignored and exists only as a legacy diagnostic/migration location.

Format Upgrades and Rollback

A tbd_format bump writes exactly two stamps: the tracked .tbd/config.yml and the machine-local $GIT_COMMON_DIR/tbd/layout.yml (plus, only when tbd setup --auto is run, the tracked agent-surface markers). It never touches issue data, so any upgrade can be aborted: restore the tracked files from git and delete layout.yml (it regenerates from the config). The full state inventory and abort recipe are user-facing in tbd-docs.md §Troubleshooting → “Aborting a Format Upgrade”; the migrate → revert → repeat loop and both interrupted-upgrade partial states are pinned by tests in tests/common-dir-layout-doctor.test.ts (“f04 → f05 upgrade”).

Key Source Files

  • packages/tbd/src/lib/paths.ts - Path constants and resolveDataSyncDir()
  • packages/tbd/src/file/git.ts - Worktree init/health/repair functions
  • packages/tbd/src/cli/commands/sync.ts - Sync command with worktree checks
  • packages/tbd/src/cli/commands/doctor.ts - Health checks and repair

Worktree Health States

State Description Fix
valid Worktree exists and has correct branch None needed
missing Worktree directory doesn’t exist tbd doctor --fix
prunable Directory deleted but git tracks it tbd sync --fix
corrupted Missing .git file or wrong branch tbd doctor --fix

Common Failure Modes

  1. Worktree deleted manually: User or tool deletes $GIT_COMMON_DIR/tbd/data-sync-worktree/. Git may still track it (prunable state). Fix: tbd sync --fix or tbd doctor --fix.

  2. Data in wrong location: Bug or old code writes to .tbd/data-sync/ instead of worktree. Fix: tbd doctor --fix migrates data to worktree.

  3. Fresh clone: Repo cloned but worktree not created. tbd setup --auto or first sync creates it.

  4. Git version mismatch: Orphan worktree requires Git 2.42+. Check: git --version, update if needed.

Debugging Tips

# Check worktree health
tbd doctor

# Verbose sync for debugging
tbd sync --debug

# List git worktrees
git worktree list

# Check what git thinks about the worktree
git worktree list --porcelain

# Manually prune stale worktree entries
git worktree prune

# Enable debug logging for path resolution
DEBUG=1 tbd sync
# or
TBD_DEBUG=1 tbd sync

Testing Worktree Code

Run the worktree health tests:

npx vitest run tests/worktree-health.test.ts

Run the e2e worktree scenarios:

npx tryscript run tests/cli-sync-worktree-scenarios.tryscript.md

Testing Forkable Docs

Forkable-docs behavior (fork/unfork/update/diff/status) is covered by tests/cli-docs-fork.tryscript.md, tests/cli-docs-update.tryscript.md, and tests/fork-cross-platform-e2e.test.ts (run from packages/tbd/).

Testing Watch and Changes

tbd changes and tbd watch are read-only observers of the sync branch (tbd-design.md §3.7 and §4.14). Unit coverage lives in tests/issue-changes.test.ts (the pure diff engine), tests/bead-watch.test.ts (the poll loop, deadlines, and private-ref handling), tests/cli-changes.test.ts, tests/cli-watch.test.ts, and tests/watch-beads-shortcut.test.ts (the shipped worker recipe, executed as Bash). tbd watch --ready observes Git transitions into readiness; complete worker loops also scan tbd ready at startup and periodically for an existing backlog and clock-only deferred_until expiry.

Unit tests cannot prove Git isolation, so a real-topology smoke test does:

pnpm qa:watch-release            # builds, then runs against dist/
TBD_QA_BIN=/path/to/tbd pnpm --filter get-tbd qa:watch-release:built   # packed candidate

scripts/validate-watch-release.ts creates a bare remote with two clones, runs concurrent watchers beside sync, list, and ready, and snapshots the caller worktree, sync refs, remote-tracking refs, FETCH_HEAD, the hidden worktree, and private refs before and after. CI runs the built-candidate form on Ubuntu, macOS, and Windows. Any change to fetch flags, ref naming, or timeout handling should be validated here, not only in unit tests: the --refmap= isolation bug was invisible to unit coverage. The manual release playbook is tests/qa/watch-infrastructure-release.qa.md.