feat: cloud sync via GitHub Gist + group-scoped pinned tabs - #1362
Open
Eger37 wants to merge 214 commits into
Open
feat: cloud sync via GitHub Gist + group-scoped pinned tabs#1362Eger37 wants to merge 214 commits into
Eger37 wants to merge 214 commits into
Conversation
added 30 commits
June 28, 2026 00:52
… Gist Adds an opt-in cloud sync that keeps tab groups consistent across devices via a private GitHub Gist, using a compacted base snapshot plus per-device delta logs (disjoint keys, never clobbered): - pure engine: delta-log, delta-capture, replay/merge with conflict rules, plan-sync, lazy compaction, url-sync (incl. about: stub), container identity mapping, group-relative indexing, tab-sleep, applied-navigation echo guard; - transport: multi-file Gist API with ETag conditional GET (304 fast path), XHR upload progress, and an advisory server-clock lock to serialize cycles; - safety: per-device watermark dedup, deferred self-truncation (a clobbered snapshot never loses data), user-priority lock so user actions beat sync; - wiring: background/constants/utils + sync settings UI (options).
Pin a tab within an active group (browser-pinned only while its group is shown). Adds the groupPinned session flag and ordering, native tab-menu and context-menu toggles, group-pin-aware moves (tab-move-split), and supporting cache/menu/migration changes. Includes the thumbtack asset and locale strings.
Render the group-pin thumbtack badge in every tab list (popup main + focused views, normal + archived; mirrored in Manage). Plus popup resilience for the sync apply path: null-guard stale group lookups against cross-channel races and contain async broadcast-handler rejections locally.
Pure-unit coverage for the sync engine: replay/merge conflict rules, planSync no-data-loss invariants, compaction + deferred truncation, advisory lock, conditional fetch (304), container map, group-relative index, url-sync, tab sleep, user-priority lock, delta-log hydration, favicon-bloat bound, and end-to-end convergence.
Keep *.test.mjs out of the webpack build so the delta-sync test suite is not shipped in the extension package.
Route the sync trigger to deltaSynchronization() when USE_DELTA_SYNC is set, falling back to the legacy synchronization() only for explicit-revision pulls. Without this the delta pipeline was never invoked and every sync ran the legacy URL-based path.
Separate "Enable sync to the cloud" (manual sync + gist setup) from a new autoSyncEnable ("Enable auto sync to the cloud") that gates the periodic sync alarm and the "Cloud sync every…" interval picker. The sync alarm now requires both flags.
…le from auto-sync interval
…4ik#7) The manual Sync button and the auto/retry alarm always route through the delta pipeline. Remove the USE_DELTA_SYNC entry-point switch and the trust/revision arguments from the sync trigger path; the legacy URL-based synchronization() stays reachable only via explicit Cloud backup entries.
Add background entries cloudBackupPush (TRUST_LOCAL push of the current local full state) and cloudBackupRestore (TRUST_CLOUD restore of a chosen gist revision), both wrapping the legacy Cloud.synchronization. Extract the browser-action progress wiring shared with cloudSync into withCloudActionProgress so backup and sync drive the toolbar progress identically; the legacy synchronization() itself is left untouched.
Add the local-only options syncBackupFilePath (template, default keeps the
bounded {slot} round-robin) and syncBackupLocation (downloads/host, default
downloads). delta-sync maybeBackupBeforeApply reads both and createBackup
honors a location override on the file-path-override path, so the sync
pre-apply backup uses its own name and destination, independent of the
auto-backup settings, with the same placeholder system.
Replace the single github-gist.vue panel with three focused blocks wired into Options.vue in order access -> sync -> backup: - github-gist-access.vue: GitHub Gist token + file plus the local/FF-Sync storage-location selector, reading the existing githubGistToken/ githubGistFileName keys (no migration). - github-gist-sync.vue: delta-only cloud sync controls (sync button without the trust dropdown, reset-sync-state, sleep/favicon/pre-apply toggles). - github-gist-backup.vue: Create backup (BG.cloudBackupPush) and Restore revision dropdown (BG.cloudBackupRestore) with no trust dropdown. Shared token/area resolution and gist-history loading move into sync-area.mixin.js. The sync trigger now calls BG.cloudSync with no trust/revision args. The pre-apply sync backup location/path settings live in the backup block, reusing the autoBackup location widgets via a new pathKey prop bound to syncBackupLocation/syncBackupFilePath.
Add keys for the GitHub Gist access / cloud sync / cloud backup block titles, the storage-location selector label, the Create backup button, and the pre-apply sync backup settings (en authoritative; ru/uk translated). Remove the now-unused trust-dropdown keys (syncStartTrustLocal, syncStartTrustCloud, syncDataInCloudCanBeDifferent).
…ist access / sync / backup)
…backup into Cloud sync
… backup under Cloud sync
Rename the delta-sync gist layout files so they share a clear STG-sync-* namespace, distinct from the legacy/backup gist files: - STG-snapshot.json -> STG-sync-snapshot.json - STG-delta-<id>.json -> STG-sync-delta-<id>.json - STG-lock.json -> STG-sync-lock.json The gist-discovery predicate in githubgist.js matches via the layout constants, so it follows the new names automatically. Comments and test fixtures naming the old files are updated to match.
…seed Delta sync no longer reads the legacy STG-backup.json to seed its initial snapshot. resolveBaseSnapshot now returns an empty snapshot when no STG-sync-snapshot.json exists; the first sync's bootstrap step (gatherLocalPending/computeBootstrapEvents) then emits synthetic add events for every local group/tab/pinned/option, so a brand-new gist initializes entirely from the CURRENT LOCAL app state. Removes the now-dead seed module and its test, the LEGACY_BACKUP_FILE_NAME constant, and its gist-discovery match. STG-backup.json no longer plays any role in sync.
Add a githubGistBackupFileName sync option (default STG-cloud-backup.json) so the legacy full-state Cloud backup writes/reads its own gist file instead of sharing githubGistFileName with the delta-sync engine. cloudBackupPush/cloudBackupRestore route through Cloud.synchronization with useBackupFile, which builds the GithubGist provider from githubGistBackupFileName (resolved from the same option bag, honoring syncOptionsLocation). The delta-sync path is unaffected; it uses the fixed STG-sync-* layout names.
…seed, separate cloud-backup file
…kup filename, change warnings
Switch gist discovery from matching a file name to matching the user's gist name (the gist description), with a per-name cache of the resolved gist id for a fast GET /gists/:id path. The one named gist holds both the delta layout files (STG-sync-*) and the Cloud backup file; reads/writes still target explicit file names, only discovery changed. Renaming the gist keys a different (empty) cache slot, so a name change cleanly re-discovers or creates a fresh gist and forgets the old one.
The sync pre-apply backup location radio was bound but never saved: the key is a string (not in ONLY_BOOL_OPTION_KEYS) and had no watcher. Register an optionsWatch mirroring autoBackupLocation so the selection persists.
autoSyncEnable lacks a sync/autoBackup prefix, so isRoaming treated it as a synced option and it roamed across devices, inconsistent with syncEnable. Add it to the explicit LOCAL_ONLY_OPTION_KEYS set so it stays per-device.
The settings page mounts three components (gist access / sync / backup) that each fetch gist info on created(), causing ~3x getInfo() calls for the same gist. Cache the in-flight getInfo() promise at module scope keyed by token+name so the concurrent reads share one round-trip; a reload after a user action passes useCache=false to force a fresh fetch.
The cloud-backup error notification deep-linked to #backup/sync, scrolling to the sync block instead of the backup block. Point it at #backup/backup and map the regrouped cloud sub-block ids (cloud/access/sync/backup) in goToBlock so the settings page scrolls to the right section.
…eview fixes A/B/C/D
Move the Cloud sync title to head the entire sync sub-block in Options.vue, with Enable sync to the cloud as the first control inside it. The auto-sync, interval, inner sync controls, sync button and the pre-apply backup unit now all render inside the same Cloud sync box. github-gist-sync renders only the inner controls.
added 30 commits
July 11, 2026 01:25
…n inbound delta create
…on hidden Route the pinned-group branch of apply() through the same windowsWithLoadingGroups lock as the normal group path so overlapping show/hide invocations can no longer interleave. Guard the context-menu togglePinnedGroup() path with the same someGroupAreLoading loading guard applyGroup uses and await the message so rapid clicks cannot re-fire mid-flight. Carry 'hidden' on UI tab objects and decide the menu label on !tab.hidden, matching the background's getPinnedGroupShownWindowId.
clearAddon wipes localStorage sync marks (deviceId, lastPushedSeq) but leaves the IndexedDB delta log intact. After a clear-before-restore with sync enabled, the surviving log is re-read on the post-reload init with lastPushedSeq reset to 0, and startup reconcile synthesizes tab/pinned removals for every stale uid absent from the restored set, deleting cloud and other-device data. Call resetSyncState() before the reload on the clear path to clear the delta log and local marks (cloud untouched), matching the state produced by the manual sync reset.
…r pinned-group tabs by pin tier
…eaf modules Move Groups.getTitle/getIconUrl/tabsCountMessage/createTitle/getDefaults/ create/extractUId/getEmojiIcon, Tabs.getTitle/extractId/isLoading/ normalizeFavIcon, Windows.get and the cloud broadcast helpers verbatim into groups-helpers.js, tabs-helpers.js, windows-helpers.js and sync/cloud/cloud-helpers.js. The original modules re-export them, so background callers are unaffected, while pages can import the helpers without pulling the full background module graph.
…ckground modules Shared mixins and popup components now use groups-helpers/tabs-helpers/ windows-helpers/cloud-helpers plus broadcast channel() subscriptions, so opening the popup no longer parses groups.js, tabs.js, windows.js, cloud.js and their transitive graph (migration, githubgist, menus-*, delta capture) or runs their listener side effects. The single-tab mute toggle goes through sendMessageModule like the group-level one already did. Manage and options pages share the same mixins and get the same diet; their own direct imports are untouched.
The color picker component is only needed when the group edit dialog renders, so load it with a dynamic import; webpack emits it as a shared async chunk instead of inlining it into every page bundle. Its css stays in the static entry css.
loadTabUid no longer mints on the pure read path used by sync; it reads in-memory then the browser.sessions value and returns empty when neither is set. A new ensureTabUid reads the session first and mints (via setTabUid) only when truly empty. setTabUid mirrors every persisted uid into a compact storage.local tabUidBackup map; loadTabUid reconstructs a lost session value from that backup by re-persisting it instead of minting a fresh uid, so sync identity is never spuriously re-minted.
Route every mint-if-missing site through Cache.ensureTabUid so the persisted session/backup value is read before a new uid is minted: tabs.js onCreated (pinned/normal), onUpdated pin-absorb and onAttached; groups.js absorbNativePinnedTabs; delta-capture resolveUid; local-state getLivePinnedTabs. Drop the durable backup entry on real tab close in tabs.js onRemoved.
…lpers and broadcast channels
… duplicating tabs
Enable webpack minification with keep_fnames/keep_classnames so logger output that reads Function.prototype.name stays intact. The minimizer plugin processes every emitted JS asset, including files copied verbatim by copy-webpack-plugin, so the externals /js tree and background.js are minified too. Import specifiers, including query-string module parameters, are preserved.
Favicons (including data: blobs) now travel in a separate per-device gist file (STG-sync-favicons-<deviceId>.json) mapping tab uid to favIconUrl, never in the append-only delta log or the snapshot. This removes the O(events) favicon surface that previously drove a multi-GB syncDeltaLog. - layout: add the favicon file name + device-id helpers (reserved prefix) - url-sync: add sanitizeFavIconUrlForFile (keep data: within a size cap), drop the now-unused sanitizeFavIconUrl, and strip all tab favicons from synced group records so events stay favicon-free - favicon-map: build the uid->favicon map from the full model (live group tabs, archived group tabs, pinned tabs) with a per-favicon + per-file budget; serialize/merge helpers - favicon-file: apply a merged map on pull via Cache.setTabFavIcon for live/pinned tabs and into archived-group tab records - delta-capture/local-state: stop carrying favicons into events/snapshot - delta-sync: read peer favicon files on pull and apply them; rebuild the self map each push and write only when it changed vs the last written map (wholesale overwrite drops dead uids); wired into the fast path too
Records/snapshot no longer carry favicons; the per-device favicon file carries data: favicons keyed by uid. Rework favicon-bloat + url-sync + delta-sync-helpers tests to assert the new invariants (no favicon in events, data: favicons kept only in the file within caps).
New events never carry favicons, so the historical favicon-stripping insurance (stripHistoricalTabFavicon and the favicon part of stripEventBloat) is no longer needed; the owner clears delta logs on update. Thumbnail and oversized-icon (iconUrl) stripping stay intact as separate still-relevant blob classes. Update hydration tests to exercise the surviving thumbnail/icon strip and assert favicons are now preserved through migration.
…yed per-device favicon file
get-startup-data stripped favIconUrl from every windows-payload tab because grouped tabs get icons from the groups payload; unsynced tabs are in neither, so they rendered a placeholder on first open. Strip only tabs with a truthy groupId, matching the loadUnsyncedTabs criterion (!tab.groupId).
…s in startup windows payload
The favicon-sync feature only admitted data: blobs into the per-device favicon map, silently dropping the http(s)/moz-extension favIconUrl that nearly every live tab carries. Favicons stopped propagating between devices. Admit http(s)/moz-extension (and data:, size-capped) into sanitizeFavIconUrlForFile so the map carries real favicons. On the receive side, add Cache.setSyncedTabFavIcon to persist a synced favicon of any scheme to session storage so unloaded/discarded/hidden tabs still render an icon; the apply path uses it and skips writes when the stored value already matches. Keep favicons out of the delta log and snapshot. Relax prepareForSaveTab to persist non-data favIconUrls so archived group tab records carry http favicons for the map build to offer.
buildFavIconMap dropped entries by lexical uid order when over the file budget, so a few huge data: blobs could evict many cheap http URLs and the truncation was silent. Sort candidates by cost ascending so the cheapest favicons are kept first, and surface an overflow warning (mirroring the delta-log cap warn) via an injected callback that keeps the module pure.
…through the sync favicon file
…tent hash
Real data: favicons routinely exceed the old 30000-char per-favicon cap and
the arbitrary 512000-byte file cap, so they were silently dropped and never
synced. Both limits were set out of fear of the delta-LOG blowup, but the
favicon file is O(tabs) and overwritten in place, not O(events) append.
- Derive caps from the real GitHub gist per-file API ceiling (GIST_FILE_MAX_BYTES
= 10 MiB; larger needs a git clone). File budget = half that; per-favicon cap =
a sixteenth. Ordinary embedded favicons now pass; only pathological blobs drop.
- sanitizeFavIconUrlForFile validates scheme only; all size gating moved into
buildFavIconMap where every drop is reported via onOverflow (never silent).
- Content-hash dedup: favicon file is now {tabs:{uid->hash}, blobs:{hash->url}}
so tabs sharing an icon store the blob once. mergeFavIconMaps resolves back to
a flat uid->url map (and tolerates the legacy flat format).
Favicons still never ride the delta log/snapshot; http(s)/moz-extension icons
remain admitted alongside data:.
…s and dedup the favicon file by content hash
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reworks cloud sync to a delta model. The existing single-file Gist sync is
replaced with a compacted snapshot + per-device delta logs, for reliable
multi-device convergence. Per-device watermark makes replay idempotent;
deferred truncation means a clobbered snapshot is re-derivable (no data loss).
ETag conditional GET (304 fast path), advisory server-clock lock, user-priority
lock so user actions beat sync. Data still lives only in the user's own Gist.
Adds group-scoped pinned tabs. Pin a tab within a group (browser-pinned
only while its group is active) — session flag, ordering, native + context-menu
toggles, thumbtack badge in popup/Manage.