Skip to content

Conversation

@ThomasRooney
Copy link
Member

@ThomasRooney ThomasRooney commented Nov 27, 2025

The Persistent Edits feature enables arbitrary code changes on top of any speakeasy generation target. Just make whatever changes you want on top of Speakeasy's target files and we'll start prompting you if you want us to maintain those changes by enabling before we overwrite them.

If this feature is enabled, conflicts may happen when Speakeasy Generation wants to touch the same lines as something with a custom modification, and are expected to be resolved locally after speakeasy run (and then re-running it with speakeasy run --skip-versioning).


Git interface → effective commands

Method Git command
HasObject(hash) git cat-file -e <hash>
ReadBlob(hash) git cat-file -p <hash>
WriteObject(content) git hash-object -w --stdin
CreateSnapshotTree(map) GIT_INDEX_FILE=.git/speakeasy_temp_index git update-index --add --cacheinfo ...; git write-tree
CommitSnapshot(tree,parent) git commit-tree <tree> [-p <parent>]
FetchSnapshot(uuid) git fetch origin refs/speakeasy/gen/<uuid>
PushSnapshot(commit, uuid) git push origin <commit>:refs/speakeasy/gen/<uuid>

Caution

Enabling this (it's default OFF) does rely on an assumption that git push and git fetch interactions with origin are possible and can occur without prompting. It does NOT however interact with any branches or non-speakeasy-generated code: it purely makes low-level git refs to be able to detect any user changes on speakeasy-tracked files.


Git Ref structure

  • Each generation is stored in its pristine form at refs/speakeasy/gen/<generation_id>. Subsequent generations will reference this to determine the user patch f we detect that the generated files are no longer equivalent via an integrity hash check
  • Note that this is not under refs/heads/ so there won't be any visible branches in GitHub; ref keeps objects reachable so git gc won't prune them. Similarly this shouldn't cause local repository bloat as we only need to fetch the most recent one on generation.
  • Lockfile tracks generation_id, pristine_commit_hash, pristine_tree_hash to help make it all work.

Algorithm (4 steps)

  1. Immediate blobbing

    • Render Targets into a VirtualFiles system. I.e. during generation if the option is enabled, files no longer get updated on file-system directly but in-memory.
    • For each file: git hash-object -w → build map path → blobHash
  2. No-op detection

    • Build tree via temp index + git write-tree
    • If tree hash == pristine_tree_hash from lockfile: exit early, no changes (we don't want a fresh no-op tree commit as it's noisy)
  3. Merge loop (per file)

    • Pre-flight scan for // @generated-id: <uuid> headers to detect moves
    • If clean (disk_checksum == last_write_checksum): overwrite with new content
    • If dirty: 3-way merge between base (git cat-file -p <git_object>), disk, and new generated
    • Conflicts use standard git markers. We also set a flag in git to help any IDE (and git status) know that there's a conflict for the user to resolve. Semantically equivalent to git apply where there's a conflict:
      <<<<<<< Current (Your changes)
      ... user edits ...
      =======
      ... new generated content ...
      >>>>>>> New (Generated by Speakeasy)
      
  4. Finalization

    • git commit-tree <new_tree> [-p <pristine_commit_hash>]
    • git push origin <commit>:refs/speakeasy/gen/<new-uuid> (fire-and-forget)
    • Update lockfile

Multi-target handling

Each target in a configured monorepo (go-sdk/, ts-sdk/) has its own trackedFiles section in gen.lock. The GitAdapter takes a baseDir param to translate logical paths (models/pet.go) to repo paths (go-sdk/models/pet.go). All targets share the same git object database and ref namespace. Because we're committing independent git trees and indexing them via gen.lock, should work in any arbitrary git repo.

Lockfile structure

persistentEdits:
  generation_id: "abc-123"           # Maps to refs/speakeasy/gen/abc-123
  pristine_commit_hash: "deadbeef"   # Parent for next commit
  pristine_tree_hash: "cafebabe"     # For no-op detection

trackedFiles:
  sdk.go:
    id: "a1b2c3d4e5f6"               # @generated-id header value
    last_write_checksum: "sha1:..."  # For dirty detection
    git_object: "blobhash"           # Blob hash for 3-way merge base
    moved_to: "pkg/sdk.go"           # If user moved file
    deleted: true                    # If user deleted file

Healer / offline

Before a git fetch, check if pristine_commit_hash exists locally (git cat-file -e). If yes, no need to fetch it.

Test plan

  • Unit: Git adapter command mapping, lockfile round-trip, merge logic (clean vs dirty, conflict markers)
  • Integration:
    • No-op: same inputs twice → no new commit
    • Moved file: @generated-id detection works
    • Dirty merge: 3-way merge with conflicts
    • Multi-target: isolated lockfile sections, shared git objects
    • Offline/healer: fetch failure handling
  • Manual: git show-ref | grep refs/speakeasy/gen/ to inspect refs, manual execution under a bunch of different cases with some test repos.

@ThomasRooney ThomasRooney force-pushed the feat/custom-code branch 2 times, most recently from a390fd2 to a07d32e Compare December 3, 2025 14:42
@mfbx9da4 mfbx9da4 merged commit 0d511ca into main Dec 5, 2025
3 of 4 checks passed
@mfbx9da4 mfbx9da4 deleted the feat/custom-code branch December 5, 2025 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants