A collection of updates about Jules

.jules/bolt.md

@@ -0,0 +1,55 @@
+# Bolt's Performance Journal
+
+⚡ Focus on measurably speeding up the application. Keep optimizations clean, documented, and non-breaking.
+
+## 2026-04-06 - Batched Index Appends in Bulk Session Upsertion
+**Learning:** Parallelizing individual database or cache writes using `Promise.all` can cause massive lock contention, redundant file system descriptor usage, and I/O bottlenecks when each concurrent task appends independently to a single shared log or index file. Writing individual session state files concurrently preserves atomic I/O performance, while aggregating all metadata entries in memory to execute a single, collective `fs.appendFile` batch operation on the shared index file reduces system call overhead from $O(N)$ to $O(1)$ and guarantees clean write alignment.
+**Action:** Always batch append operations to shared flat files (such as index, registry, or JSONL databases) when performing bulk state operations, while keeping distinct, non-overlapping document-level writes concurrent.
+
+## 2026-03-30 - Optimized Top-Level Document Projections Fast-Path
+**Learning:** During multi-document database queries, projecting document fields to lightweight subsets (especially default session/activity projections) is extremely frequent. Invoking a generic projection engine that splits paths, scans for wildcards/exclusions, checks nested structures, and deep-clones primitives introduces major redundant CPU and GC overhead. Pre-calculating a boolean flag `isSimpleTopLevel` in a cached query plan can safely bypass the entire heavy projection engine for flat queries.
+**Action:** Always check if a highly repetitive document/object mapping or projection can be pre-calculated and cached. When a selection is strictly simple and flat, implement an O(1) loop copy with direct primitive mapping to achieve up to ~50% CPU performance improvements in hot paths.
+
+## 2026-03-31 - Optimized Query Selection Path & Avoided Sorter Object Allocations
+**Learning:** Performing `new Date()` construction and parsing inside an $O(N \log N)$ sort comparator is a significant performance bottleneck due to thousands of repeated string parsings and heap allocations. Pre-parsing timestamps in the $O(N)$ hydration/projection phase and caching them as numbers in the `_sortKey` object completely eliminates this overhead. Additionally, calling `Object.entries()` inside loops that run on every matched document causes substantial GC pressure; replacing it with a simple `for...in` loop prevents those redundant array allocations entirely.
+**Action:** Never perform date parsing or string operations inside high-frequency sort comparators. Precompute sort keys as primitives during object mapping. Always replace key/value array helper functions with native loops in high-frequency matching hot paths.
+
+## 2026-04-01 - Avoided Heap Allocation via Primitive-Returning Date.parse & Date.now
+**Learning:** Instantiating `new Date()` objects repeatedly inside high-frequency search and scan loops (such as query selection sorting fallbacks, high-watermarking scan loops, cache tier evaluations, and session asking loops) introduces major garbage collection overhead and memory churn. Using standard native primitive-returning helpers like `Date.parse()` and `Date.now()` completely bypasses V8 heap allocation of date objects, making date comparisons up to ~10x faster and eliminating GC pauses.
+**Action:** Always prefer `Date.parse(string)` and `Date.now()` over `new Date(string)` or `new Date()` for date comparisons, subtraction, and arithmetic operations in hot execution paths.
+
+## 2026-04-02 - Optimized Deep Query Projections and String Sort Collations
+**Learning:** Recursive query projection routines (`getPath`, `setPath`, `deletePath`) split and slice path arrays extensively on every nested field lookup. Replacing sub-path array slicing and destructuring with an explicit `index` traversal parameter completely eliminates redundant array allocations and garbage collection pressure in the query pipeline. Additionally, using standard comparison operators (`<`, `>`) instead of locale-sensitive `localeCompare` inside hot sort comparisons is up to 10x faster for simple alphanumeric identifiers.
+**Action:** Avoid allocating sub-arrays/slices in recursive traversal functions. Always prefer direct comparison operators over slow collation methods like `localeCompare` for sorting non-localized key and ID fields.
+
+## 2026-04-03 - Optimized Unified Diff Parsing via Single-Pass State Machine
+**Learning:** Parsing unified git diff files frequently in activity-syncing pipelines introduces heavy CPU and garbage collection overhead when using regex-based split operations (`patch.split(/^diff --git /m)`). Switching to a single-pass, line-by-line state machine completely eliminates sub-string filtering, nested splits, and multiple line scans, yielding a massive performance boost. Furthermore, handling Windows CRLF `\r\n` line endings by explicitly sanitizing trailing `\r` prevents critical parsing regressions across cross-platform environments.
+**Action:** Always parse structured sequential files (like diffs, logs, or JSONL) in a single sequential line iteration instead of multiple high-level split passes. Ensure cross-platform line termination safety by removing trailing `\r` carriage returns.
+
+## 2026-04-04 - Pre-compiled Query Projection Plans
+**Learning:** Standard document projection pipelines recursively traverse object/array hierarchies and dynamically map, join, and filter nested inclusion and exclusion paths on every matched document. Allocating arrays for `.filter()`, `.map()`, and string `.join('.')` inside hot scanning/hydration loops introduces immense heap churn and garbage collection pressure. Pre-compiling nested select selectors and exclusion sub-paths into cached, query-level plan structures completely eliminates allocation overhead during runtime document iteration.
+**Action:** Move all structural conversions, filtering, and array-mapping operations of metadata patterns up to the query compilation/planning phase and cache them, ensuring document iteration loops only perform O(1) dictionary lookups.
+
+## 2026-04-05 - Compiled Query Where-Clause Filters
+**Learning:** In high-frequency query scanning, filtering, and document hydration loops, repeatedly parsing raw filter structures (e.g., checking `typeof filter`, splitting nested dot-notation paths on `.` to traverse sub-properties, lowercasing substring filters on every comparison, and scanning arrays linearly via `.includes`) causes significant CPU overhead and garbage collection pressure due to frequent string and object allocations. Pre-compiling raw query filters into structured compiled filters with pre-split paths, pre-lowercased patterns, and `Set` collections for $O(1)$ inclusion checks completely eliminates redundant string splits and heap allocations.
+**Action:** Always pre-compile query/where clauses and search metadata patterns once before entering hot execution/iteration loops over collections of documents, and use non-allocating native loops rather than closure-based helpers inside highly repetitive matching logic.
+
+## 2026-04-06 - Bypassed Hydration Mapping to Prevent Object and Array Allocation
+**Learning:** Repeatedly mapping arrays of nested objects (such as activity artifacts) to hydrate them into rich class instances introduces substantial memory allocation and garbage collection pressure. Even when all objects are already hydrated instances, standard `.map()` calls inevitably allocate a new array and clone the parent object. Pre-checking if any elements actually require hydration allows bypassing the entire mapping, array allocation, and cloning pipeline entirely, achieving reference-equal O(1) returns.
+**Action:** Before performing mapping, copying, or hydration on collections of objects, always pre-check if the collection is already in its final desired state. If no transformation is needed, return the original parent object directly.
+
+## 2026-04-07 - Optimized Array Elements Projection Fast-Path
+**Learning:** When projecting array properties in a query (such as activities, outputs, or artifacts) to include all fields without specific nested sub-property paths or exclusions, the projection engine was instantiating empty object structures and traversing fields property-by-property via `getPath`/`setPath` for every element in the array. Bypassing this engine for arrays when `subPaths` only consists of empty subpaths (`subPaths.length === 1 && subPaths[0].length === 0`) and no exclusions are specified, and instead performing a direct fast-path element copy via `deepClone(item)`, completely avoids unnecessary CPU processing and garbage collection allocations.
+**Action:** Always check if a loop mapping or deep traversal can be safely bypassed or fast-pathed when the target transformation evaluates to copying whole items as-is.
+
+## 2026-04-08 - Optimized Concurrency Mapping and Fast-Path Collections in Parallel Execution
+**Learning:** Core concurrency utility functions (such as parallel maps or batch workers) can introduce massive promise allocation and garbage collection churn if they construct worker pools that exceed the collection size or rely on array allocation helper chains (like `new Array(N).fill(0).map(...)`). Limiting active worker pool allocation to `Math.min(concurrency, items.length)` completely avoids redundant, idle worker promises. Furthermore, implementing direct fast-paths for empty and single-item inputs completely bypasses entire worker pool construction, scheduler loops, and error-aggregation array mappings, reducing runtime scheduling overhead from $O(N)$ to $O(1)$.
+**Action:** Always optimize parallel mapper / concurrency pool utilities by providing O(1) fast-paths for empty and single-item arrays, avoiding overhead and allocation entirely.
+
+## 2026-08-11 - Batched Ingestion & Metadata Caching for Immutable Activities
+**Learning:** Repeatedly appending immutable records (such as activities) in sequence triggers a massive I/O bottleneck when each write operation performs independent read-modify-write loops on a centralized metadata file, plus discrete file appends. Moving metadata state to an in-memory cache completely eliminates redundant $O(N)$ reads, while exposing a batch `appendMany` endpoint allows aggregating all log appends and committing the metadata update precisely once on disk per batch, reducing system call and disk I/O overhead from $O(N)$ to $O(1)$.
+**Action:** Always provide batch write APIs for log/index appending, and cache repeatedly read configuration or count files in-memory to prevent disk thrashing during high-frequency sync loops.
+
+## 2026-08-13 - Resolving Leftover Redundant Blocks and Duplicate Member Fields in Merged Code
+**Learning:** During complex refactorings or merges, leftover redundant blocks (such as calling the duplicate appending block with the undefined `toAppend` variable in the hydration pipeline) and duplicate field definitions (such as `metadataCache` in `NodeFileStorage`) can slip through, causing critical runtime crashes and TypeScript compile-time errors. Locating and cleanly removing these leftover duplicate blocks completely resolves the exceptions, restoring 100% test coverage and compile correctness.
+**Action:** Always verify compilation via `bun run build` and run full unit/integration test suites to identify any leftover/duplicate rebase variables or class members, and cleanly prune them to ensure correctness.

.jules/palette.md

@@ -0,0 +1,43 @@
+# Palette's Journal - CLI UX & Accessibility
+
+This journal records critical, reusable UX and accessibility insights specific to this codebase.
+
+## 2026-03-01 - Interactive TTY Styling for Node CLI Orchestration
+**Learning:** Users of rich CLI tools (such as orchestrators) need clear visual hierarchies to quickly scan long logs. Plain-text logs without colored status indicators (like checkmarks or warning symbols) increase cognitive load. Adding green, red, yellow, and dim ANSI escapes in interactive TTY environments significantly improves readability, while maintaining clean unstyled text in CI logs preserves log grepability and compatibility.
+**Action:** Next time when rendering CLI/TUI outputs, always introduce conditional ANSI styles that detect TTY/non-CI environments to color-code success checkmarks (green), errors/failures (red), warnings (yellow), and secondary metadata/fractions (dim).
+
+## 2026-03-05 - Backtick Command Highlighting in CLI Suggestions and Errors
+**Learning:** Highlighting inline code and command names (wrapped in backticks) in terminal suggestions drastically reduces the user's cognitive friction when dealing with errors. It makes actionable instructions (such as "Use `jules-fleet configure` to update settings") immediately stand out visually in interactive terminals without adding noise in plain CI/log environments.
+**Action:** Implement conditional regex-based ANSI formatting on terminal error suggestions to highlight backticked segments in yellow or bold colors when process/stdout is interactive, falling back to clean plain-text when run in CI.
+
+## 2026-03-08 - Informative CLI Resource Creation Feedback
+**Learning:** Long-running CLI orchestration tasks that create remote resources (like Git repositories) require continuous visual feedback (spinners for in-flight tasks, and clear success/warning/error states on completion). Leaving key domain events silent increases user anxiety and cognitive load. Providing clickable, descriptive OSC 8 terminal hyperlinks (like "View Repository") dramatically improves user discoverability and workflow efficiency directly inside TTY environments.
+**Action:** Always ensure all domain-specific lifecycle events (especially resource creation and fallback states) are fully handled in rendering paths, using clear visual states (spinners, warning icons, and descriptive links instead of raw URLs).
+
+## 2026-03-12 - Clickable Wizard Hyperlinks and Standardized Failure Indicators
+**Learning:** In interactive CLI applications (such as setup wizards), raw URLs lead to cluttered terminal screens, wrapping issues, and increased cognitive load. Using descriptive, semantic labels for terminal hyperlinks via OSC 8 makes clickable links look clean and highly discoverable. Additionally, standardizing colored failure status indicators (such as a red '✗') across all subcommand error paths provides unified visual scanning and consistency with existing checkmarks ('✓') and warning symbols ('⊘').
+**Action:** Always use semantic labels for `ansiLink` instead of passing raw URLs as labels in CLI prompts, and consistently prepend red '✗' to failure status outputs in terminal render blocks.
+
+## 2026-03-15 - Standardizing CLI Spinner Completion States with Status Symbols
+**Learning:** In terminal UIs, replacing a running spinner with plain-text completion logs lacks immediate, scannable feedback. Appending standardized, color-coded completion status symbols (such as a green '✓' for success, red '✗' for errors/failures, and yellow '⊘' for timeout/skipped states) directly to the stopSpinner message creates a visually cohesive and satisfying experience that is instantly decodable.
+**Action:** Always append standard, colored status indicators (such as `ansiGreen('✓')`, `ansiRed('✗')`, and `ansiYellow('⊘')`) to `ctx.stopSpinner` logs for completed terminal tasks.
+
+## 2026-03-16 - Unified Backtick Highlighting in CLI Setup Prompts and Status Outputs
+**Learning:** In interactive CLI applications and wizard setups, user-specific variables (such as repository names, auth identities, and secret keys) printed in plain text lack distinct visual separation from instructions. Formatting these variables inside backticks and passing the strings through the `ansiHighlight` utility creates a highly professional, cohesive theme that naturally directs the user's attention to key parameters and command suggestions.
+**Action:** Ensure that user-supplied inputs, paths, or keys in both interactive prompts and domain-specific status event logs (such as skipped/failed reasons) are formatted with backticks and highlighted using `ansiHighlight`.
+
+## 2026-03-17 - Interactive Password Validation in CLI Setup Wizards
+**Learning:** In interactive setup wizards, missing input validation on password or token prompts allows users to accidentally submit empty or whitespace-only values (e.g., by hitting Enter too quickly). This leads to silent configuration issues or obscure downstream auth errors. Implementing robust inline validation for sensitive password prompts ensures immediate interactive feedback and prevents invalid setups.
+**Action:** Always add inline `validate` check functions to both required and optional password/token prompts in CLI wizard flows to confirm they are non-empty and non-whitespace.
+
+## 2026-03-18 - Graceful Parsing and Guidance in CLI Interactive Prompts
+**Learning:** Forcing users to enter exact strings (like `owner/repo` repository slugs) in interactive wizards often leads to validation errors and friction when they copy-paste full Git clone URLs (e.g., HTTPS/SSH links or with `.git` suffixes). Designing prompts to automatically and gracefully parse/clean input URL formats into standard identifiers, complemented by clear placeholder examples, provides an extremely smooth, error-free keyboard navigation and setup experience.
+**Action:** Always clean and normalize complex identifiers or path inputs directly from the user's interactive clipboard/paste actions inside wizard text prompts before passing them to validators, and provide helpful, standardized `placeholder` example structures.
+
+## 2026-08-11 - Descriptive Option Hints in Interactive CLI Wizards
+**Learning:** Selecting from multiple options in interactive CLI setup prompts (like choosing between GitHub PAT or App authentication) can be ambiguous or daunting for users who do not know the exact trade-offs of each option. Adding descriptive `hint` properties to selection options provides key contextual guidance that helps users make informed choices, while refactoring duplicate select blocks to reuse helper prompt functions ensures consistency across all Wizard flows.
+**Action:** Always provide descriptive and highly informative `hint` messages for selection list options in interactive wizard prompts, and reuse core prompt functions to keep setup CLI interfaces unified and modular.
+
+## 2026-08-12 - Backtick Highlighting for Dynamic Repository Names in Creation Logs
+**Learning:** Rendering raw repository slugs (such as `owner/repo`) in dynamic CLI creation event logs without formatting causes them to blend in with adjacent static log text. Formatting repository names in backticks and applying `ansiHighlight` creates visual separation and visual consistency across all CLI subcommands and setup steps.
+**Action:** Always format repository slugs and dynamic resource identifiers in backticks and wrap them with `ansiHighlight` across all CLI event renderers.