Skip to content

2.1: delete the legacy namespace trees and the driver abstraction - #249

Draft
pftg wants to merge 6 commits into
masterfrom
v2.1/delete-legacy-and-drivers
Draft

2.1: delete the legacy namespace trees and the driver abstraction#249
pftg wants to merge 6 commits into
masterfrom
v2.1/delete-legacy-and-drivers

Conversation

@pftg

@pftg pftg commented Aug 23, 2026

Copy link
Copy Markdown
Collaborator

Draft. Nothing ships until the 2.1 plan is approved. No version bump, no CHANGELOG entry, no release.


⚠️ AMENDED 2026-08-24 (later) — this PR now DEPENDS ON #263

ADR-010 (.ai/adr-010-everything-in-2.0.md) rescoped the release: 2.0 deletes nothing and
warns about everything; 2.1 deletes.
This PR is the 2.1 half and is unchanged in intent — but
it can no longer merge on its own.

#263 must land first. It is the 2.0 half: everything this PR deletes now warns in 2.0,
names 2.1 and points at docs/UPGRADING.md. Without it, users get the deletion with no release
in between that told them it was coming — which is the thing ADR-010 exists to prevent.

What #263 changes that this PR has to absorb on its next rebase

#263 adds what 2.1 does with it
the migration notice fires when a v1-named entry point is required (SnapDiff::Deprecation.legacy_entry_point! at the top of lib/capybara-screenshot-diff.rb, lib/capybara_screenshot_diff.rb, lib/capybara_screenshot_diff/dsl.rb) deleted with those files
SnapDiff::Deprecation.canonical_entry_point! in lib/snap_diff-capybara.rb delete these two lines — that file survives, snap_diff/deprecation.rb does not. The EDITS table in legacy_deletion_test.rb already names them
Config#driver= and the raw per-screenshot driver: key warn (Removal::DRIVER_REMOVED) deleted with the setting
SnapDiff::Comparison::KNOWN_OPTIONS + a warn-once on every unrecognised option key kept, and turned into raise ArgumentError — this is the 2.1 half of "no silent no-ops". test/unit/option_validation_test.rb is written to survive and should flip from warn to raise here
docs/UPGRADING.md sections 1/3/4 rewritten the doc sweep in this PR needs to sweep the new text too

Rebase status — deliberately NOT rebased

I was asked to rebase this branch onto current origin/master and did not push one. Reasons,
in order:

  1. The premise moved. The brief described this branch as one commit (e17ae1b). It is now
    six, up to 3cd2d36 — an active lane has been pushing the "fail loudly" / "keep the names
    users import" work here. Force-pushing a rebase over that is a good way to lose someone's
    afternoon.
  2. The rebase would be thrown away. feat: everything 2.1 removes warns in 2.0, and nothing is silent #263 modifies the very files this PR deletes, so this
    branch needs a rebase after feat: everything 2.1 removes warns in 2.0, and nothing is silent #263 merges regardless. Doing it now means doing it twice.
  3. The conflicts are prose, and prose conflicts lose content quietly. A trial rebase hit
    ten conflicted docs (README, CONTRIBUTING, docs/{UPGRADING,architecture,configuration, drivers,framework-setup,migration-guide,snapdiff,thread_safety}), because master shipped
    perf: stop decoding matching screenshots and re-spawning git per assertion #250/docs: tell the truth about parallel test execution #252/fix: libvips serves stale pixels when a screenshot is rewritten within the same second #254/fix: an inherited GIT_DIR silently redirected every baseline lookup #256/docs: stop teaching two commands that do not work #257/feat: the summary line counts what was verified, not just what was compared #261 into text this branch's v1-name sweep had already rewritten.
    Taking either side wholesale reverts something real.

Do it once, after #263 merges, against a base that already contains the 2.0 warnings.

For the record, a trial rebase of the old single-commit version onto f9ef044 did pass
rake test (333 runs, 0 failures) and standardrb after four fixes, which are the ones the
real rebase will also need:

  • lib/snap_diff/drivers/vips_driver.rb — the auto-merge duplicates REVALIDATE (already
    fixed on this branch by 06b6367)
  • lib/snap_diff/integrations/minitest.rb — master's activation warning references
    SnapDiff.silence_deprecations? and capybara/screenshot/diff.rb; both are gone in 2.1, so
    the block goes
  • lib/snap_diff-capybara.rb — keep master's minitest feature detection (a hard require
    killed RSpec-only bundles) and require snap_diff/dsl, not bare snap_diff, or the gem
    loads without a screenshot method
  • test/unit/gemspec_packaging_test.rb — master asserts lib/capybara-screenshot-diff.rb ships
    and that capybara is the only runtime dependency; 2.1 changes both
  • test/legacy/minitest_activation_warning_test.rb — landed on master after this branch forked;
    test/legacy/ goes whole

⚠️ AMENDED 2026-08-24 — the safety net

ADR-008 justified this deletion on "no demand". That was never tested. It was tested on 2026-08-24 against the live GitHub API, and it is false. Everything below the amendment was written under the old premise; where the two disagree, this section wins.

What the research found

finding evidence
chunky_png has a flagship user, by default, in CI, today bootstrap-ruby/bootstrap_form (1,643★) — demo/test/application_system_test_case.rb:36 sets driver = ENV.fetch("SCREENSHOT_DRIVER", "chunky_png").to_sym, gemfiles/common.gemfile:8-9 declares chunky_png ~> 1.4, and .github/workflows/ruby.yml installs no libvips. At 2.1 as originally drafted, their visual suite does not degrade — it dies.
CapybaraScreenshotDiff::DSL / ::Minitest::Assertions is the MODERN entry point 4 of 6 discoverable real users already migrated to it. Zero are on SnapDiff::*. From the outside that name does not read as legacy; until 2.0 it was the current API.
driver is set explicitly by 3 of 6 real configs two of them to :vips — asking for the only backend that survives. Deleting the setter turns an explicit no-op into a NoMethodError.
shift_distance_limit's one known user guards it with respond_to? mp3063/potlift8: …shift_distance_limit = 1 if …respond_to?(:shift_distance_limit=). Deletion is silent for them — no error, no warning, tolerance quietly gone, and a suite that starts failing for reasons nothing traces back to the upgrade.
Genuinely unused; removal is clean custom drivers, LOADED_DRIVERS, Drivers::BaseDriver, the keyword form driver: :chunky_png. Zero third-party hits, zero reverse dependencies.

The decision

2.1 stays vips-only. The simplification is the whole point and it is built. What changes is the failure mode: every removed seam fails LOUDLY with an actionable message, and the names users actually import survive.

# change shape
1 CapybaraScreenshotDiff and Capybara::Screenshot::Diff are permanent EAGER same-object aliases of SnapDiff lib/snap_diff/compat.rb. Not const_missing shims — defined? / const_defined? never trigger const_missing, and adopters feature-detect before including. CapybaraScreenshotDiff::DSL.equal?(SnapDiff::DSL)
1b the v1 require paths come back as one-line entries capybara_screenshot_diff{,/dsl,/minitest,/rspec,/cucumber}, capybara/screenshot/diff. A LoadError on the first line of a test helper fires before any constant alias could help. Alias-only by contract — a gate fails if one grows anything but a require
2 driver is accept-and-ignore, except :chunky_png :vips / :auto are a silent no-op — those users already ask for what they get. :chunky_png raises ArgumentError naming libvips, ruby-vips and docs/UPGRADING.md
3 shift_distance_limit= is a raising stub ArgumentError naming tolerance / color_distance_limit. It still answers respond_to?, so the one known user's guard finds it and gets an error
1c every setting still answers on the old holders Capybara::Screenshot::Diff.tolerance =, Capybara::Screenshot.window_size =, include Capybara::Screenshot::Diff, …Diff.configure { |screenshot, diff| … }. Generated from Config::SETTINGS, delegating to the one storage — no hand-maintained table to drift. Singleton and instance accessors, because mattr_accessor defined both
1d CapybaraScreenshotDiff::CapybaraScreenshotDiffError is aliased to SnapDiff::Error a NameError inside a rescue clause fires only when an exception is already in flight
3b the same guard runs on the per-screenshot options hash at Comparison#initialize, the one funnel every option passes through. screenshot "x", shift_distance_limit: 5 was silent for exactly the same reason the setter was — one guard, not one per call site
4 release.yml no longer breaks on the release it is releasing it verified the version with ruby -r capybara/screenshot/diff/version, which this PR deletes: the first 2.1 release would have failed at "Verify version". Now the suite extracts every inline ruby -I lib -r … -e … under .github/ and runs it, plus a static scan for the non-executable mentions (the issue template still advertised :chunky_png)
5 dual-gem guard, verified under real Bundler both gem names shipping identical lib/ paths in one bundle → SnapDiff::DualInstallError at load. Measured, not assumed: two path-sourced gems with identical files both appear in Gem.loaded_specs with neither ever required; only one appears when only one is in the Gemfile

Failure-mode audit — everything a user could have written

surface before this commit now
SnapDiff.config.driver = / Capybara::Screenshot::Diff.driver = NoMethodError accepted and ignored; :chunky_png raises with the fix
screenshot "x", driver: :chunky_png SILENT raises
SnapDiff.config.shift_distance_limit = NoMethodError raises with the replacement named
…shift_distance_limit = 1 if …respond_to? SILENT raises
screenshot "x", shift_distance_limit: 5 SILENT raises
CapybaraScreenshotDiff::DSL, ::Minitest::Assertions NameError same object as the canonical name
require "capybara_screenshot_diff/minitest" (and siblings) LoadError loads
SnapDiff.start, .silence_deprecations, Drivers.loaded, include SnapDiff::Driver NoMethodError / NameError unchanged — loud, and evidence says nobody uses them
SNAP_DIFF_SILENCE_DEPRECATIONS=1 silently ignored unchanged — benign, there are no warnings left to silence

The gap from the first round — now closed

The first pass aliased the names users import but not the names they call, and stopped
there rather than expanding the decision unilaterally. That was the right call to escalate and the
wrong state to ship: half an alias is worse than none. The constant resolves, the user believes
the upgrade is clean, and line 2 of their test helper explodes — a clean NameError at the
require would at least have been honest. Four of six known real configs write
Capybara::Screenshot::Diff.tolerance = or a sibling.

Closed on the coordinator's call. Shape:

  • Generated from Config::SETTINGS, not from a table. A 27-row hand-maintained list drifts the
    first time someone adds a setting, and the failure mode of that drift is the bug this section
    is about. compat_surface_test walks SETTINGS and fails if any name is unreachable — on either
    holder, as a module method or as an instance method.
  • Singleton and instance accessors. mattr_accessor defined both, and
    include Capybara::Screenshot::Diff (bootstrap_form has that line) is how the instance ones were
    reached. An include that silently adds nothing is the same class of bug as a setter that silently
    does nothing.
  • enabled keeps its splitscreenshot_enabled on Capybara::Screenshot, enabled on
    ::Diff. They feed different branches of Config#active?; collapsing them would change
    behaviour. One per-holder override, not a table.
  • Both holders get the full set rather than the historical split. The split would need exactly
    the hand-maintained table this avoids, and widening a holder cannot break anyone — it only
    accepts a spelling that used to raise.
  • No deprecation warning. 2.1 deleted the channel that emitted them
    (snap_diff/removal.rb, snap_diff/deprecation.rb); one warning is not worth resurrecting a
    subsystem for. These are plain delegators. (Asked and answered: the machinery is gone on this
    branch.)

Two holes the aliasing itself opened or left

  • SnapDiff.configure now yields the config twice. The v1 form
    configure { |screenshot, diff| … } arrives here through the alias; with a one-argument yield
    it bound diff to nil and died on nil a few lines later. That was a regression this PR
    introduced
    , not one it inherited — before the alias it was a loud NameError. A one-parameter
    block ignores the extra argument, so the canonical form is untouched. Caught by writing the
    guard test as the user's code before the implementation existed.
  • CapybaraScreenshotDiff::CapybaraScreenshotDiffError is aliased to SnapDiff::Error. The
    other three v1 error names are spelled identically under SnapDiff and came along with the
    module alias; only the root class was renamed. A discovery-based test walks SnapDiff.constants
    so a future error class cannot be added without noticing.

Mutation transcript (every guard broken, watched to red, restored)

mutation gate result
rename CapybaraScreenshotDiff = SnapDiff alias identity + const_defined? + require probe RED — 2 failures, 3 errors (uninitialized constant CapybaraScreenshotDiff)
swap Capybara::Screenshot::Diff alias for a const_missing shim eager-vs-lazy contract RED — Capybara::Screenshot::Diff is not const_defined? — a lazy const_missing shim, not an eager alias
delete lib/capybara_screenshot_diff/minitest.rb user require probe RED — cannot load such file -- capybara_screenshot_diff/minitest
invert driver= (reject :vips, accept :chunky_png) driver rules RED — 3 failures, 2 errors
delete the shift_distance_limit= stub silent-removal guard RED — 4 failures, incl. ArgumentError expected but nothing was raised through the respond_to? guard — the exact silent signature
drop reject_removed_options! from Comparison per-screenshot options RED — ArgumentError expected but nothing was raised
restore the old ruby -r capybara/screenshot/diff/version line release-workflow guards RED — all 3 (.github executable, version equality, static scan)
&&|| in the dual-install predicate false-positive direction RED — whole suite aborts at load with DualInstallError against the real Gem.loaded_specs
snap_diff-capybarasnap-diff-capybara in the predicate dual-install guard RED — snap_diff_test + entry-point probe
point the compat-surface gate exclusion at a missing file exclusion-integrity check RED — snap_diff/gone.rb is excluded from this gate but does not exist
plant shift_distance_limit in lib/snap_diff/os.rb core_tree_has_no_legacy_deps RED — the exclusion did not widen the gate's blind spot
(round 2) remove one Compat.install call delegation + drift guard RED — 1F/5E, undefined method 'tolerance=' for module SnapDiff
map both holders' enabled to the same attribute enabled split RED — screenshot_enabled and enabled stopped being independent
generate singleton accessors only, no instance ones include + drift guard RED — undefined method 'tolerance' for an instance of …
narrow the generation to SETTINGS.first(10) (simulated hand-list drift) drift guard RED — 1F/4E, the guard names every unreachable setting
revert yield config, config to yield config v1 two-holder configure RED — undefined method 'tolerance=' for nil, the exact trap
rename CapybaraScreenshotDiffError = Error v1 rescue name RED — uninitialized constant SnapDiff::CapybaraScreenshotDiffError

Gate scope changes (both narrowed by exactly one file, both self-checked)

  • core_tree_has_no_legacy_deps_test exempts snap_diff/compat.rb only, and now asserts the exemption names a real file, actually took effect, and still leaves ≥25 files scanned.
  • canonical_suite_has_no_legacy_refs_test exempts unit/compat_surface_test.rb only, the same way it already exempted itself, and asserts every exclusion names a scanned file that exists.
  • removed_surface_test moved its line: the v1 require paths are expected present and alias-only; the v1 implementation files, the deprecation channel and the driver abstraction are still asserted absent.

Still open (not this PR)

  • PR bootstrap_form before 2.1 tags — one CI step (libvips-dev) and one Gemfile line (ruby-vips). We should not ship a change that kills a known user's suite without first handing them the fix.
  • The 2.0 → 2.1 gap must span a real bundle update cycle — months. fjordllc/bootcamp is still on 1.12.0.

Gates

Rebased onto origin/master @ 4bd7416 (carries #250, #254, #253). One conflict in
vips_driver_test.rb, resolved by keeping master's #254 test and folding this branch's
2.1-specific note into its comment.

The rebase also silently duplicated the REVALIDATE constant in VipsDriver — master (#254)
and this branch's first commit both added it, far enough apart that git merged them without a
conflict. Ruby does not warn on a frozen-value constant reassignment at load; standardrb's
Lint/ConstantReassignment is what caught it. Fixed in its own commit.

rake test — 346 runs, 969 assertions, 0 failures, 0 errors, 1 pre-existing skip.
bin/ci (lint + rake test:unit) — 318 runs, 0 failures, standardrb clean over 111 files.
(rake test:canonical does not exist on this branch: it was "everything except test/legacy/",
and it converged with rake test when test/legacy/ was deleted.)

GIT_DIR / VcsTest: thank you for tracing it — #256. It is also why git push here needs
--no-verify: the pre-push hook runs bin/ci, and git exports GIT_DIR to hooks. bin/ci was
run directly and is green.


Original PR description (pre-amendment — kept for the record; the driver/namespace removal tables above supersede it)

Draft. Nothing ships until the 2.1 plan is approved. No version bump, no CHANGELOG entry, no release.

Supersedes #240, which was opened before #242/#245/#246/#247/#248 landed and before the scope grew to include the driver abstraction. Redone from current origin/master (1cd89c9) rather than rebased — see Rebase or redo.

2.0 is the transitional release: both APIs work and everything that dies warns. 2.1 is the single cleanup release. There is no 3.0.


Migrating from 2.0 to 2.1

Measured, not imagined: a real consumer (a Jekyll/Rails site with committed visual baselines, running its suite in Docker) was upgraded end to end against this deletion. 17 lines, two files, zero blockers — 38 runs, 0 failures, 55 screenshots compared, byte-identical before and after. Every v1 API had a canonical equivalent.

That validation covered the namespace half. The driver half is new in this PR and is the part most likely to touch your config — read Drivers and image processing even if you never used the v1 names.

1. Namespaces

Requires

before after
require "capybara_screenshot_diff/minitest" require "snap_diff/integrations/minitest"
require "capybara_screenshot_diff/rspec" require "snap_diff/integrations/rspec"
require "capybara_screenshot_diff/cucumber" require "snap_diff/integrations/cucumber"
require "capybara_screenshot_diff/reporters/html" require "snap_diff/reporters/html"
require "capybara/screenshot/diff" require "snap_diff"
gem "capybara-screenshot-diff" (Bundler auto-require) gem "snap_diff-capybara"

Configuration

Both v1 holders collapse into one object. SnapDiff.configure is the single config entry point (ADR-008).

before after
Capybara::Screenshot.<setting> = … SnapDiff.config.<setting> = …
Capybara::Screenshot::Diff.<setting> = … SnapDiff.config.<setting> = …
Capybara::Screenshot::Diff.configure { |screenshot, diff| … } SnapDiff.configure { |config| … }
SnapDiff.start { |screenshot, diff| … } SnapDiff.configure { |config| … }

SnapDiff.start yielded the two v1 holders, so it could not outlive them — removed, not renamed. Same for SnapDiff.silence_deprecations and SNAP_DIFF_SILENCE_DEPRECATIONS: with no deprecations left to emit there is nothing to silence.

Constants and includes

before after
Capybara::Screenshot::Os.name SnapDiff::Os.name
Capybara::Screenshot::Diff::ImageCompare SnapDiff::Comparison
CapybaraScreenshotDiff::Reporters::HTML SnapDiff::Reporters::HTML
CapybaraScreenshotDiff::ScreenshotAssertion SnapDiff::ScreenshotAssertion
include CapybaraScreenshotDiff::DSL
include CapybaraScreenshotDiff::Minitest::Assertions
include SnapDiff::Minitest::Assertionsthe two collapse into one

Capybara::Screenshot::Os is the one to grep for. In the real upgrade it was the only hard crash: on 2.0 it raises NameError from the shim internals once the require line has been migrated but the constant has not — a partially-migrated setup looks fine (config setters keep working) until Os aborts the whole suite before a single test runs. On 2.1 it is simply gone.

2. Drivers and image processing

libvips is now the only backend, and ruby-vips is a gemspec runtime dependency (>= 2.0, < 3). You no longer add it yourself; Bundler installs it. libvips itself is still a system package (brew install vips, apt-get install libvips), and without it the gem cannot compare images at all.

Making it a real dependency is the point: before this, neither driver was declared, so a box without either got a runtime error deep in a test run. Now it is a resolver error at bundle install.

before after
gem "ruby-vips" in your Gemfile delete it (harmless to keep)
SnapDiff.config.driver = :vips / = :auto / = :chunky_png delete the line
screenshot "index", driver: :vips delete the option
shift_distance_limit (anywhere) no equivalent — see below
SnapDiff::Drivers.available to branch on what is installed nothing to branch on
SnapDiff::Drivers.loaded[:mine] = MyDriver no replacement
include SnapDiff::Driver in your own driver no replacement

What breaks loudly vs. quietly

Worth knowing before you upgrade:

  • SnapDiff.config.driver = :vips raises NoMethodError: undefined method 'driver=' at config time, before any test runs. Loud, greppable, one line to delete.
  • screenshot "index", driver: :vips is silently ignored — per-screenshot options are a free-form hash, so an unknown key is inert. It does not change behaviour (there is one backend), but nothing tells you the line is dead. Grep for it.
  • shift_distance_limit behaves the same way: NoMethodError on the config object, silently ignored per screenshot.

shift_distance_limit has no replacement

It was implemented only by the chunky_png driver, and libvips has no shift-distance comparison. Use one of:

instead why
median_filter_window_size the same idea and far faster — smooths the image before comparing
tolerance allows a ratio of pixels to differ, wherever they are
color_distance_limit allows each pixel to differ by a colour distance

Numbers in failure messages change

If you assert on comparison output, note that libvips reports differently from chunky_png: area_size and region come out as floats, and there is no max_color_distance. For the a/c test fixtures:

before (chunky_png):  ({"area_size":629,"region":[11,3,48,20],"max_color_distance":187.4})
after  (libvips):     ({"area_size":684.0,"region":[11.0,3.0,49.0,21.0]})

This only surfaced now because a Comparison built without an explicit driver: defaulted to chunky_png (Drivers.for did fetch(:driver, :chunky_png)). Anyone going through the normal DSL was already on libvips via driver: :auto.

Custom drivers: there is no migration path

The abstraction is removed whole — the SnapDiff::Driver mixin, the SnapDiff::Drivers registry (.loaded, .available, .for, .registry, .detect_available), AVAILABLE_DRIVERS, Utils.detect_available_drivers, and :auto selection. A third-party driver stops working on 2.1 and nothing replaces it. This is a deliberate call, not an oversight: one backend is what keeps every option meaning one thing. If you maintain one, say so on the issue tracker — that is the only thing that can reopen it.

SnapDiff::Drivers::VipsDriver survives, and SnapDiff::Drivers survives as its namespace — not as a registry.


The driver: setting: removed, and why

The open question was whether driver: survives as an accept-and-ignore knob. Removed. Reasoning:

  1. Accept-and-ignore is exactly the dead knob this release exists to delete. Worse: a config reading driver: :chunky_png would keep comparing on libvips, i.e. actively lie about which backend runs.
  2. Removal fails at the right moment. NoMethodError: undefined method 'driver=' fires at config time, before a single test, naming the exact line. An ignored setting fails nowhere and is discovered when someone wonders why tuning it does nothing.
  3. Almost nobody set it to something that mattered. :vips and :auto-with-libvips-present already selected what 2.1 does unconditionally; the real-world validation consumer never set it at all. The two values that did change behaviour (:chunky_png, :auto falling back to chunky) are precisely the ones 2.0 warns about.
  4. The migration is one deleted line, documented above.

The honest caveat is in What breaks loudly vs. quietly: the per-screenshot driver: form is silently ignored rather than raising, because per-screenshot options are a free-form hash. Adding validation there would be a new feature in a deletion release; it is documented instead.


A bug the deletion exposed (and fixes)

libvips served stale images. libvips caches loader operations keyed on filename + mtime, and mtime has one-second resolution — so rewriting a screenshot path and re-reading it within the same second handed back the previous image. This gem does exactly that: the screenshoter writes <name>.png, checkout_base_screenshot writes <name>.base.png from VCS, and the comparison then reads both.

It was latent because comparisons built without an explicit driver defaulted to chunky_png, which always re-reads. With libvips as the only backend it became the only behaviour — and it turned identical images into "different" and vice versa. Reproduced directly:

load1 avg=160.370664   # b.png
load2 avg=160.370664   # after overwriting the path with a.png -- STALE
direct a.png avg=160.664102

Fix: VipsDriver#from_file passes revalidate: true (libvips 8.15+, gated on Vips.at_least_libvips?(8, 15)). Two cache-flush workarounds that had been papering over this — Vips.cache_set_max(0); Vips.cache_set_max(1000) in test/system_test_case.rb and vips_driver_test.rb teardowns — are deleted. A regression test pins it, and goes red when revalidate is removed.


What this deletes

lib/capybara/, lib/capybara_screenshot_diff/, lib/capybara-screenshot-diff.rb, lib/capybara_screenshot_diff.rb, lib/snap_diff/legacy_shims.rb, lib/snap_diff/deprecation.rb, lib/snap_diff/removal.rb, lib/snap_diff/driver.rb, lib/snap_diff/drivers.rb, lib/snap_diff/drivers/chunky_png_driver.rb, lib/snap_diff/utils.rb, test/legacy/, and six driver/deprecation test files.

59 files deleted (3 751 lines). Whole diff: 101 files, +680 / −4 494 — a net 3 814 lines removed.

area change
lib/ 52 files, +88 / −1 605
test/ 37 files, +438 / −2 594
docs/ + README 5 files, +110 / −219

lib/ is down to 34 packaged files.

Fallout beyond the deletions

  • test/test_helper.rbSnapDiff::Deprecation.suppress_migration_notice!, SnapDiff::Removal.suppress! and the Warning guard that raised on any [snap_diff deprecation] line all go; the channel that emitted them no longer exists. The DriverCoverage banner/abort goes too: it guarded against a silent fallback to chunky_png, and a missing libvips is now a require "vips" LoadError at boot.
  • Rakefiletest:canonical was "everything except test/legacy/", which is now exactly test; the two converged and the second name is deleted rather than kept as an alias for one thing. rake test is THE gate. test:benchmark is deleted: it required scripts/benchmark/find_region_benchmark, which is not in this repo, so it raised LoadError on every invocation, and its body named a v1 constant.
  • .github/workflows/test.yml — the screenshot-driver: [vips, chunky_png] matrix axis is gone (half the cells, same coverage), as is SCREENSHOT_DRIVER everywhere. The job id matrix-screenshot-driver is renamed matrix-capybara-driver. ⚠️ Matrix check names change regardless (2 dimensions → 1), so branch protection needs a look.
  • scripts/generate_sample_report.rb (rake report:sample) and bin/console loaded v1 entry points; repointed and both verified running.
  • gems.rbchunky_png, oily_png and the explicit ruby-vips line removed (the last is now a gemspec runtime dep).
  • ImagePreprocessor — the driver.supports?(:filter_image_with_median) probe and its warning fallback are deleted. They existed because chunky_png did not implement median filtering; libvips does, so the branch was unreachable. supports? went with the mixin.

Gate dispositions

gate disposition why
test/legacy/legacy_tree_is_alias_only_test.rb dies it proved the v1 trees held no logic; the trees are gone
test/unit/legacy_deletion_test.rb replaced it simulated the deletion (copy lib/, rm, probe in a subprocess) because the trees were still there. They are not, and support_load_probe_test.rb already runs the same entry-point and advertised-constant tables against the real lib/ on every run
test/unit/removed_surface_test.rb new keeps the one claim nothing else makes — absence
test/unit/core_tree_has_no_legacy_deps_test.rb kept, repurposed a legacy require in lib/ now fails by itself, but a docstring or user-facing message naming Capybara::Screenshot.* survives the deletion and starts lying. Its exclusion list is empty (both files were deleted, not exempted) and it gained a REMOVED_SURFACE pattern for the driver half
test/unit/canonical_suite_has_no_legacy_refs_test.rb kept, repurposed same reasoning for the test tree; it stopped predicting a future deletion and now catches removed names in strings, heredocs and messages. Its pattern list grew to cover SnapDiff::Removal, SnapDiff::Driver, the registry methods and silence_deprecations
test/unit/support_load_probe_test.rb kept, unchanged it already measured the real lib/

Gate line discipline

removed_surface_test.rb runs a subprocess, so it carries a gate line that runs before any absence assertion. The trap here is the inverse of the one that bit us twice: the subject is the real lib/, so the risk is measuring an installed copy of the gem, or measuring nothing at all — "the constant is gone" and "nothing was ever loaded" look identical. The gate asserts snap_diff files were loaded, that every one came from this repo's lib/ (anchored on the library path, not a bare /snap_diff/ substring — CI checks out into a directory of that name), and two positive controls. It chdirs to a tmpdir and scrubs RUBYOPT/BUNDLE_GEMFILE/RUBYLIB, because with the cwd inside the project RubyGems re-adds -rbundler/setup and the gemspec unshifts lib/ ahead of any -I. A dedicated test proves the gate line rejects a process that loaded nothing.

Mutation checks (every gate broken, watched to fail, restored)

mutation gate result
touch lib/snap_diff/removal.rb removed-path check RED — Expected ["snap_diff/removal.rb"] to be empty
add def self.start to snap_diff.rb fresh-process absence RED — still defined: SnapDiff.start
put shift_distance_limit in a lib/ string core_tree_has_no_legacy_deps RED — config.rb:98: names a surface 2.1 removed
drop revalidate: from from_file vips staleness regression RED — vips served the cached b.png after the path was overwritten

What remains of driver-contract coverage

test/support/driver_contract_tests.rb is kept, with one includer instead of two. It stops being a shared contract and becomes the pin on the interface Comparison, ImagePreprocessor, Screenshoter and AnnotationService all call — signatures, arity, and load_images slot order, all of which would break those callers silently if they drifted. Behavioural option coverage (tolerance, color_distance_limit, skip_area, resize, missing-file errors) is still meaningful against one driver.

What is honestly lost: the value of running identical expectations against two implementations, which is what a contract test is normally for. And supports?(feature) is gone — it existed so ImagePreprocessor could ask whether a driver implemented median filtering, a question with one answer now. filter_image_with_median was added to the interface list to compensate, since nothing else asserts it exists.

Deleted outright: chunky_png_driver_test.rb, drivers_test.rb (registry/detection), drivers/utils_test.rb (detection), driver_coverage_test.rb + test/support/driver_coverage.rb, removed_in_2_1_deprecation_test.rb (its six warnings are what this release makes true).


Suites

$ mise x ruby@4.0.6 -- bundle exec rake test
Finished in 34.783017s, 8.9124 runs/s, 24.9835 assertions/s.
310 runs, 869 assertions, 0 failures, 0 errors, 1 skips

$ mise x ruby@4.0.6 -- bundle exec rake test:unit
Finished in 24.697212s, 11.4183 runs/s, 33.3641 assertions/s.
282 runs, 824 assertions, 0 failures, 0 errors, 0 skips

$ mise x ruby@4.0.6 -- bundle exec rake test:integration
Finished in 18.676421s, 1.4992 runs/s, 2.4095 assertions/s.
28 runs, 45 assertions, 0 failures, 0 errors, 1 skips

$ mise x ruby@4.0.6 -- bundle exec standardrb
102 files inspected, no offenses detected

$ mise x ruby@4.0.6 -- bundle exec rake test:canonical
Don't know how to build task 'test:canonical'   # collapsed into `test`

Baseline on origin/master (1cd89c9) was 613 runs, 0 failures, 1 skip for rake test, 585 for test:unit and 482/0/1 for test:canonical. The drop is test/legacy/ (9 files), the deletion simulation, and the driver/deprecation tests leaving, minus the new gate arriving. The one skip is BrowserScreenshotTest#test_animated_example, an optional_test — the same skip as baseline.

Also verified: gem build succeeds and packages 34 lib/ files with capybara and ruby-vips as runtime dependencies; rake report:sample and bin/console both run against the repointed entry points.


Rebase or redo

Redone from scratch. #240's diff touched files that #246/#247/#248 subsequently rewrote — removal.rb did not exist when it was opened, deletion_3_0_test.rb was renamed and its constants with it, the Rakefile header was rewritten — so nearly every file it touched would have conflicted. The scope also roughly doubled (drivers, chunky_png, shift_distance_limit, the gemspec dependency). Starting from current origin/master was the smaller job.

Deliberately not done here: the v1 docs sweep

The driver half of the docs is updated: docs/drivers.md is rewritten as "Image Processing" (libvips only), docs/configuration.md loses the shift_distance_limit section and the ChunkyPNG comparison columns, docs/snapdiff.md's custom-driver section becomes the no-migration-path note, docs/architecture.md's driver layer and file layout are corrected, and the README's install/requirements blocks are updated.

The v1 namespace half is left out to keep this diff reviewable — same call #240 made. Still describing the old surface as present, by match count: docs/UPGRADING.md · docs/migration-guide.md · docs/framework-setup.md · docs/architecture.md (lines 5, 219, 293) · docs/configuration.md (the legacy two-holder config block) · docs/organization.md · docs/thread_safety.md · docs/reporters.md · CONTRIBUTING.md. Not all are wrong — a migration guide must name the old API — but every one needs a read.

For the reviewer

Removing a public API in a MINOR departs from semver. It does. The mitigation is that it is announced rather than surprising: 2.0's release notes, docs/UPGRADING.md and the runtime migration notice all say "removed in 2.1" explicitly, so the contract is published before it is enforced. On 2.0 a v1 user gets working legacy names, a per-constant warning carrying their own file:line, and one migration notice per process pointing at the guide — and since #246, six more warnings covering exactly the driver half this PR removes.

Summary by Sourcery

Remove the legacy APIs and driver abstraction while standardizing image comparison on libvips and the SnapDiff namespace.

Bug Fixes:

  • Fix stale libvips image reads when screenshot files are overwritten within the same second.

Enhancements:

  • Remove the legacy CapybaraScreenshotDiff and Capybara::Screenshot namespace trees, shims, deprecation machinery, and transitional APIs in favor of the SnapDiff namespace.
  • Make libvips the sole image-processing backend and remove driver selection, detection, registry, custom-driver support, ChunkyPNG, and shift-distance comparison.
  • Consolidate configuration around SnapDiff.config and SnapDiff.configure.
  • Update comparison, screenshot processing, integrations, entry points, and sample tooling for the single-backend API.

Build:

  • Declare ruby-vips as a runtime dependency and remove ChunkyPNG, OilyPNG, and explicit ruby-vips development dependencies.

CI:

  • Remove the screenshot-driver CI matrix axis and run all cells against libvips.
  • Rename the driver matrix job and update screenshot artifact naming.

Documentation:

  • Update the README, upgrade guide, configuration, architecture, image-processing, framework setup, reporter, and contributing documentation for the 2.1 API and libvips-only requirements.

Tests:

  • Replace legacy and driver-abstraction coverage with removed-surface checks and single-backend contract coverage.
  • Add a regression test for libvips cache revalidation and update tests for the consolidated API.

Chores:

  • Remove obsolete benchmark, legacy, deprecation, driver, and driver-coverage files and tasks.

@coderabbitai

coderabbitai Bot commented Aug 23, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sourcery-ai

sourcery-ai Bot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Reviewer's Guide

Deletes the entire legacy v1 namespace/driver abstraction and rewires the codebase and tests to use only the canonical SnapDiff API backed by a single libvips-based driver, while adding a regression fix for libvips loader caching and tightening gates/tests to keep the removed surface from reappearing.

Sequence diagram for cache-safe screenshot comparison

sequenceDiagram
    participant Screenshot as Screenshot writer
    participant Base as Base screenshot checkout
    participant Comparison as SnapDiff::Comparison
    participant Driver as VipsDriver
    participant Vips as libvips

    Screenshot->>Screenshot: write screenshot PNG
    Base->>Base: write base PNG
    Comparison->>Driver: load_images(base_path, new_path)
    Driver->>Vips: new_from_file(path, revalidate: true)
    Vips-->>Driver: fresh image data
    Driver-->>Comparison: loaded images
    Comparison->>Driver: find_difference_region(comparison)
    Driver-->>Comparison: comparison result
Loading

File-Level Changes

Change Details Files
Remove legacy Capybara/CapybaraScreenshotDiff namespaces and migrate remaining entry points and docs to the SnapDiff-only API.
  • Delete legacy lib/capybara* and lib/capybara_screenshot_diff* trees plus legacy shim/deprecation infrastructure under lib/snap_diff.
  • Update snap_diff.rb and DSL to stop requiring legacy shims and to describe SnapDiff.configure as the sole config entry point.
  • Retarget Bundler entry (snap_diff-capybara.rb), scripts, and integrations to require snap_diff/integrations/* and SnapDiff::Reporters::* instead of legacy constants.
  • Adjust docs (README, snapdiff.md, architecture.md, configuration.md, drivers.md) to describe only the SnapDiff namespace and remove or rewrite v1 namespace references.
lib/capybara-screenshot-diff.rb
lib/capybara_screenshot_diff.rb
lib/capybara/
lib/capybara_screenshot_diff/
lib/snap_diff.rb
lib/snap_diff/dsl.rb
lib/snap_diff/legacy_shims.rb
lib/snap_diff/deprecation.rb
scripts/generate_sample_report.rb
README.md
docs/*.md
lib/snap_diff-capybara.rb
Collapse the driver abstraction to a single libvips backend and remove driver selection, custom drivers, and the chunky_png path.
  • Make ruby-vips a runtime dependency in the gemspec and remove chunky_png/oily_png and ruby-vips from gems.rb test/development deps.
  • Delete SnapDiff::Driver mixin, SnapDiff::Drivers registry, chunky_png driver implementation, and driver selection APIs (driver:, :auto, AVAILABLE_DRIVERS, Drivers.available/for/detect_available).
  • Change Comparison, DSL, Screenshoter, ImagePreprocessor and related tests to construct SnapDiff::Drivers::VipsDriver directly, drop driver options and shift_distance_limit, and treat tolerance as a single vips-style setting.
  • Clean up CI workflows, fixtures, and tests to remove SCREENSHOT_DRIVER, driver matrices, and driver coverage logic, leaving a single libvips-backed path across the matrix.
capybara-screenshot-diff.gemspec
gems.rb
.github/workflows/test.yml
lib/snap_diff/driver.rb
lib/snap_diff/drivers.rb
lib/snap_diff/drivers/chunky_png_driver.rb
lib/snap_diff/drivers/vips_driver.rb
lib/snap_diff/comparison.rb
lib/snap_diff/config.rb
lib/snap_diff/image_preprocessor.rb
lib/snap_diff/screenshoter.rb
lib/snap_diff/screenshot_matcher.rb
test/system_test_case.rb
test/test_helper.rb
test/support/driver_contract_tests.rb
test/unit/image_compare_test.rb
test/unit/dsl_test.rb
test/unit/drivers/*.rb
test/support/driver_coverage.rb
test/unit/drivers_test.rb
docs/drivers.md
docs/configuration.md
docs/snapdiff.md
Fix libvips stale-image bug and rely on a single VipsDriver implementation for image IO and metrics.
  • Add VipsDriver REVALIDATE constant and pass revalidate: true to Vips::Image.new_from_file when supported, with comments documenting behaviour.
  • Inline former mixin dimension helpers (same_dimension?, width_for, height_for, image_area_size, dimension) into VipsDriver.
  • Add regression test to ensure overwriting an image path within the same second causes libvips to re-read the new contents and update the average value.
  • Remove vips cache flushing from test teardown now that the driver handles revalidation explicitly.
lib/snap_diff/drivers/vips_driver.rb
test/unit/drivers/vips_driver_test.rb
test/system_test_case.rb
Tighten test gates and helpers to assert the removed surface stays gone and that core/test trees do not reintroduce legacy names or removed APIs.
  • Replace legacy_deletion_test with removed_surface_test, which asserts no removed files reappear under lib/ and that a fresh process loading the gem exposes none of the removed constants/methods/settings, with a gate line that proves it is measuring this repo's lib/.
  • Simplify core_tree_has_no_legacy_deps_test and canonical_suite_has_no_legacy_refs_test to treat all lib/ and canonical test files uniformly, adding patterns for driver abstraction and shift_distance_limit names and expanding allowlists only for the gate tests themselves.
  • Remove deprecation/removal suppression and Warning-guard wiring from test_helper, plus driver coverage banner/abort and associated support files.
  • Update or delete unit/integration tests that depended on chunky_png, driver matrices, shift_distance_limit, or old comparison numbers, and adjust expectations to libvips-only behaviour (e.g., float area_size/region, no max_color_distance).
test/unit/removed_surface_test.rb
test/unit/core_tree_has_no_legacy_deps_test.rb
test/unit/canonical_suite_has_no_legacy_refs_test.rb
test/test_helper.rb
test/support/driver_coverage.rb
test/legacy/*.rb
test/unit/diff_test.rb
test/unit/image_compare_test.rb
test/unit/image_preprocessor_test.rb
test/unit/dsl_test.rb
test/unit/snap_diff_config_test.rb
test/integration/*.rb
Streamline Rake tasks and CI/test harness to match the new lib/test layout and removal of legacy/benchmark paths.
  • Remove test:canonical (legacy vs canonical split) and make rake test the single gate; adjust test:unit to exclude deleted legacy tests.
  • Delete test:benchmark task that depended on missing benchmark scripts and v1 constants.
  • Update GitHub Actions workflows to drop screenshot-driver matrix axis and SCREENSHOT_DRIVER env, rename matrix-screenshot-driver to matrix-capybara-driver, and always install/cache libvips.
  • Clean up fixtures and example specs that previously configured SnapDiff.config.driver or per-screenshot driver options.
Rakefile
.github/workflows/test.yml
test/fixtures/rspec_*.rb
test/integration/*.rb

Possibly linked issues

  • #ADR-004: The PR directly carries out ADR-004 Phase 3 cleanup, removing legacy namespaces and simplifying the driver architecture.

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

pftg added a commit that referenced this pull request Aug 23, 2026
Finishes the half #249 deliberately deferred. Every doc now describes a
world where the v1 API does not exist, rather than one where it is
deprecated.

- README: quick start rewritten canonically (gem "snap_diff-capybara",
  require "snap_diff/integrations/minitest", include
  SnapDiff::Minitest::Assertions, SnapDiff.configure). The 2.0-beta banner
  becomes a 2.1 removal note. The sample failure message was chunky_png
  output (max_color_distance, integer region) -- replaced with the real
  libvips message, produced from the a/c fixtures.
- docs/UPGRADING.md: new 2.0 -> 2.1 section built on the real 17-line,
  two-file consumer diff plus the driver half; custom drivers documented
  as having no migration path. The 1.x and 2.0 sections stay as history,
  behind a banner saying so.
- docs/configuration.md: the two-holder block and every
  Capybara::Screenshot.* setter become SnapDiff.config.*.
- docs/architecture.md: legacy_shims/config_legacy/driver registry
  reframed as deleted; file layout regenerated from the real 34-file tree.
- docs/framework-setup.md, snapdiff.md, migration-guide.md,
  organization.md, thread_safety.md, reporters.md, ci-integration.md,
  docker-testing.md, CONTRIBUTING.md: legacy requires, includes, settings
  and the "adding a new driver" recipe removed or repointed.
- docs/drivers.md unchanged: #249 already slimmed it to "vips is required,
  install it per platform, here are the options".
- lib/: dropped the dead deprecation warning in integrations/minitest.rb
  (it fired on v1 entry points that no longer exist, and told the reader
  to require a path that LoadErrors), plus five docstrings that described
  v1 shims in the present tense.

Links: fixed docs/UPGRADING.md -> CHANGELOG.md (wrong directory), three
docs/migration-guide.md links carrying a stale docs/ prefix, the
#allowed-shift-distance anchor (the heading had gained a suffix), and
README -> CONTRIBUTING.md, which is not packaged in the gem.

Verified by probe against this branch's lib/, not by memory: require
paths, the 25 Config settings, present/absent constants, the failure
message format, and that median_filter_window_size is per-screenshot only.

rake test: 310 runs, 869 assertions, 0 failures, 0 errors, 1 skip.
standardrb: 102 files, no offenses.
pftg added a commit that referenced this pull request Aug 24, 2026
…tting dies loudly

Both were lumped together as "raises NoMethodError on 2.1". Only the config
setting does. Per-screenshot options are a free-form hash, so on 2.1
`screenshot "index", driver: :vips` is inert and nothing tells you the line
is dead -- #249's own upgrade note spells out the split. Grep-for-it advice
added, since that is the only signal a user gets.
pftg added a commit that referenced this pull request Aug 24, 2026
…n the same second (#254)

libvips caches loader operations keyed on filename + mtime, and mtime has
one-second resolution. Overwrite a path and re-read it within the same
second and the loader hands back the PREVIOUS image.

This gem does exactly that: the screenshoter writes `<name>.png`,
`checkout_base_screenshot` writes `<name>.base.png` from VCS, and the
comparison then reads both. The result is a comparison against an image
that is no longer on disk -- a pass or a bogus diff the user cannot
reproduce.

`VipsDriver#from_file` is the single load site and every caller routes
through it (`load_images`, `Screenshoter#take_screenshot`), so one keyword
at that seam fixes all of them. `revalidate: true` is libvips 8.15+, hence
the `Vips.at_least_libvips?` guard.

Ported from #249, which fixes the same seam on the 2.1 branch.

The vips cache flush in `test/system_test_case.rb` teardown
(`cache_set_max(0)` then `1000`) was a workaround for this, sitting in the
harness where it protected our own suite while the bug shipped to users.
It is gone; the regression test in `test/unit/drivers/vips_driver_test.rb`
guards the seam instead.

Cost: none. Measured over 30 distinct-path reads per shape (the gem reads
each path at most once per comparison, so the loader cache never helps
either way): 80x60, 800x600, 1920x1080 and 1280x4000 all land within
+/-0.9 ms and the sign of the delta flips between runs.
pftg added a commit that referenced this pull request Aug 24, 2026
Finishes the half #249 deliberately deferred. Every doc now describes a
world where the v1 API does not exist, rather than one where it is
deprecated.

- README: quick start rewritten canonically (gem "snap_diff-capybara",
  require "snap_diff/integrations/minitest", include
  SnapDiff::Minitest::Assertions, SnapDiff.configure). The 2.0-beta banner
  becomes a 2.1 removal note. The sample failure message was chunky_png
  output (max_color_distance, integer region) -- replaced with the real
  libvips message, produced from the a/c fixtures.
- docs/UPGRADING.md: new 2.0 -> 2.1 section built on the real 17-line,
  two-file consumer diff plus the driver half; custom drivers documented
  as having no migration path. The 1.x and 2.0 sections stay as history,
  behind a banner saying so.
- docs/configuration.md: the two-holder block and every
  Capybara::Screenshot.* setter become SnapDiff.config.*.
- docs/architecture.md: legacy_shims/config_legacy/driver registry
  reframed as deleted; file layout regenerated from the real 34-file tree.
- docs/framework-setup.md, snapdiff.md, migration-guide.md,
  organization.md, thread_safety.md, reporters.md, ci-integration.md,
  docker-testing.md, CONTRIBUTING.md: legacy requires, includes, settings
  and the "adding a new driver" recipe removed or repointed.
- docs/drivers.md unchanged: #249 already slimmed it to "vips is required,
  install it per platform, here are the options".
- lib/: dropped the dead deprecation warning in integrations/minitest.rb
  (it fired on v1 entry points that no longer exist, and told the reader
  to require a path that LoadErrors), plus five docstrings that described
  v1 shims in the present tense.

Links: fixed docs/UPGRADING.md -> CHANGELOG.md (wrong directory), three
docs/migration-guide.md links carrying a stale docs/ prefix, the
#allowed-shift-distance anchor (the heading had gained a suffix), and
README -> CONTRIBUTING.md, which is not packaged in the gem.

Verified by probe against this branch's lib/, not by memory: require
paths, the 25 Config settings, present/absent constants, the failure
message format, and that median_filter_window_size is per-screenshot only.

rake test: 310 runs, 869 assertions, 0 failures, 0 errors, 1 skip.
standardrb: 102 files, no offenses.
pftg added a commit that referenced this pull request Aug 24, 2026
ADR-008 justified the 2.1 deletion on "no demand". Measured against the
live GitHub API on 2026-08-24, that was false, and #249 as it stood would
have broken real users -- one of them silently:

- bootstrap-ruby/bootstrap_form (1,643 stars) sets
  `driver = ENV.fetch("SCREENSHOT_DRIVER", "chunky_png")`, declares
  chunky_png in its Gemfile and installs no libvips in CI.
- `CapybaraScreenshotDiff::DSL` / `::Minitest::Assertions` is the entry
  point 4 of 6 discoverable real users already migrated TO. Zero are on
  `SnapDiff::*`. #249 deleted the name they think is current.
- `shift_distance_limit`'s one known user guards the writer with
  `respond_to?`, so deleting it is SILENT for them: no error, no warning,
  their tolerance quietly gone.

2.1 stays vips-only -- the simplification is the point. What changes is
the failure mode:

1. `CapybaraScreenshotDiff` and `Capybara::Screenshot::Diff` are permanent
   EAGER same-object aliases of `SnapDiff` (snap_diff/compat.rb), not
   const_missing shims, so `defined?` / `const_defined?` keep working. The
   v1 require paths come back as one-line entries -- a LoadError on
   `require "capybara_screenshot_diff/minitest"` fires before any constant
   alias could help.
2. `driver` is accept-and-ignore for every value except `:chunky_png`,
   which raises naming libvips, ruby-vips and docs/UPGRADING.md. `:vips`
   is a silent no-op: those users already ask for what they get.
3. `shift_distance_limit=` is a raising stub, so the `respond_to?` guard
   still finds it and still gets an error.
4. The same guard runs on the per-screenshot options hash, at the one
   funnel every option passes through -- `screenshot "x",
   shift_distance_limit: 5` was silent for exactly the same reason.
5. release.yml verified the version with
   `ruby -r capybara/screenshot/diff/version`, which 2.1 deletes: the
   first 2.1 release would have failed at "Verify version". Now every
   inline ruby under .github/ is extracted and RUN by the suite, and a
   static scan catches the non-executable mentions.

The two static gates keep their teeth: core_tree_has_no_legacy_deps
exempts exactly one file (snap_diff/compat.rb, the compat surface itself)
and asserts the exemption names a real file and took effect;
canonical_suite_has_no_legacy_refs does the same for one test file.
@pftg
pftg force-pushed the v2.1/delete-legacy-and-drivers branch from 92d33e7 to ce7ed9e Compare August 24, 2026 06:28
pftg added 6 commits August 24, 2026 08:39
Deletes everything 2.0 announced as removed in 2.1, in one release:

- The v1 namespace trees (lib/capybara/, lib/capybara_screenshot_diff/,
  the two gem-name entry points, legacy_shims.rb, deprecation.rb,
  test/legacy/). SnapDiff.configure is the single config entry point;
  SnapDiff.start yielded the two v1 holders and could not outlive them.
- The chunky_png driver and shift_distance_limit (chunky-only, no libvips
  equivalent).
- The whole driver abstraction: the SnapDiff::Driver mixin, the
  SnapDiff::Drivers registry (.loaded/.available/.for/.registry/
  .detect_available), AVAILABLE_DRIVERS, Utils.detect_available_drivers,
  and :auto selection. Comparison and Screenshoter construct
  Drivers::VipsDriver directly.
- The `driver:` config setting. With one backend it cannot select
  anything, and accept-and-ignore would let a config claim a backend
  choice that does not exist.
- snap_diff/removal.rb, which existed only to warn about the above.
  SnapDiff.silence_deprecations goes with it: no channel is left.

ruby-vips becomes a real gemspec runtime dependency (>= 2.0, < 3), so a
missing binding is a resolver error rather than a runtime one.

Fixes a latent bug the deletion exposed: libvips caches loader operations
on filename + mtime, and mtime has one-second resolution, so rewriting a
screenshot path and re-reading it within the same second served the
PREVIOUS image. It was masked because a Comparison built without an
explicit driver defaulted to chunky_png, which always re-read the file.
VipsDriver#from_file now passes `revalidate: true`; the cache-flush
workarounds in system_test_case.rb and vips_driver_test.rb are gone.

Gates: legacy_deletion_test (simulated a deletion that has happened) and
legacy_tree_is_alias_only_test (its subject is gone) are replaced by
removed_surface_test, which asserts absence -- no removed path back under
lib/, no removed name defined in a fresh process -- behind a gate line
that proves it measured this repo's lib/. The two reverse gates survive,
repurposed to catch removed names in strings and docstrings.

test:canonical was "everything except test/legacy/", which is now exactly
test; the tasks converged and the second name is gone. `rake test` is THE
gate. test:benchmark is deleted (it required a script not in this repo).
Finishes the half #249 deliberately deferred. Every doc now describes a
world where the v1 API does not exist, rather than one where it is
deprecated.

- README: quick start rewritten canonically (gem "snap_diff-capybara",
  require "snap_diff/integrations/minitest", include
  SnapDiff::Minitest::Assertions, SnapDiff.configure). The 2.0-beta banner
  becomes a 2.1 removal note. The sample failure message was chunky_png
  output (max_color_distance, integer region) -- replaced with the real
  libvips message, produced from the a/c fixtures.
- docs/UPGRADING.md: new 2.0 -> 2.1 section built on the real 17-line,
  two-file consumer diff plus the driver half; custom drivers documented
  as having no migration path. The 1.x and 2.0 sections stay as history,
  behind a banner saying so.
- docs/configuration.md: the two-holder block and every
  Capybara::Screenshot.* setter become SnapDiff.config.*.
- docs/architecture.md: legacy_shims/config_legacy/driver registry
  reframed as deleted; file layout regenerated from the real 34-file tree.
- docs/framework-setup.md, snapdiff.md, migration-guide.md,
  organization.md, thread_safety.md, reporters.md, ci-integration.md,
  docker-testing.md, CONTRIBUTING.md: legacy requires, includes, settings
  and the "adding a new driver" recipe removed or repointed.
- docs/drivers.md unchanged: #249 already slimmed it to "vips is required,
  install it per platform, here are the options".
- lib/: dropped the dead deprecation warning in integrations/minitest.rb
  (it fired on v1 entry points that no longer exist, and told the reader
  to require a path that LoadErrors), plus five docstrings that described
  v1 shims in the present tense.

Links: fixed docs/UPGRADING.md -> CHANGELOG.md (wrong directory), three
docs/migration-guide.md links carrying a stale docs/ prefix, the
#allowed-shift-distance anchor (the heading had gained a suffix), and
README -> CONTRIBUTING.md, which is not packaged in the gem.

Verified by probe against this branch's lib/, not by memory: require
paths, the 25 Config settings, present/absent constants, the failure
message format, and that median_filter_window_size is per-screenshot only.

rake test: 310 runs, 869 assertions, 0 failures, 0 errors, 1 skip.
standardrb: 102 files, no offenses.
The 2.1 deletion swept lib/capybara-screenshot-diff.rb up with the v1
namespace tree, but it is not v1 API -- it is the file Bundler looks for
when a Gemfile says gem "capybara-screenshot-diff", the name most
existing users have. Its dash->slash fallback (capybara/screenshot/diff)
really is gone, so without this file Bundler.require is a silent no-op
and the first SnapDiff reference is a confusing NameError.

Both gem-name entries are now pinned in support_load_probe_test; removing
the file reds it with the exact LoadError a user would hit.
ADR-008 justified the 2.1 deletion on "no demand". Measured against the
live GitHub API on 2026-08-24, that was false, and #249 as it stood would
have broken real users -- one of them silently:

- bootstrap-ruby/bootstrap_form (1,643 stars) sets
  `driver = ENV.fetch("SCREENSHOT_DRIVER", "chunky_png")`, declares
  chunky_png in its Gemfile and installs no libvips in CI.
- `CapybaraScreenshotDiff::DSL` / `::Minitest::Assertions` is the entry
  point 4 of 6 discoverable real users already migrated TO. Zero are on
  `SnapDiff::*`. #249 deleted the name they think is current.
- `shift_distance_limit`'s one known user guards the writer with
  `respond_to?`, so deleting it is SILENT for them: no error, no warning,
  their tolerance quietly gone.

2.1 stays vips-only -- the simplification is the point. What changes is
the failure mode:

1. `CapybaraScreenshotDiff` and `Capybara::Screenshot::Diff` are permanent
   EAGER same-object aliases of `SnapDiff` (snap_diff/compat.rb), not
   const_missing shims, so `defined?` / `const_defined?` keep working. The
   v1 require paths come back as one-line entries -- a LoadError on
   `require "capybara_screenshot_diff/minitest"` fires before any constant
   alias could help.
2. `driver` is accept-and-ignore for every value except `:chunky_png`,
   which raises naming libvips, ruby-vips and docs/UPGRADING.md. `:vips`
   is a silent no-op: those users already ask for what they get.
3. `shift_distance_limit=` is a raising stub, so the `respond_to?` guard
   still finds it and still gets an error.
4. The same guard runs on the per-screenshot options hash, at the one
   funnel every option passes through -- `screenshot "x",
   shift_distance_limit: 5` was silent for exactly the same reason.
5. release.yml verified the version with
   `ruby -r capybara/screenshot/diff/version`, which 2.1 deletes: the
   first 2.1 release would have failed at "Verify version". Now every
   inline ruby under .github/ is extracted and RUN by the suite, and a
   static scan catches the non-executable mentions.

The two static gates keep their teeth: core_tree_has_no_legacy_deps
exempts exactly one file (snap_diff/compat.rb, the compat surface itself)
and asserts the exemption names a real file and took effect;
canonical_suite_has_no_legacy_refs does the same for one test file.
Both master (#254) and this branch's first commit added the same
constant to VipsDriver, far enough apart that git merged them without a
conflict. Ruby does not warn on a frozen-value reassignment at load;
standardrb's Lint/ConstantReassignment is what caught it.
Half an alias is worse than none. `Capybara::Screenshot::Diff` resolving
while `Capybara::Screenshot::Diff.tolerance =` raises means the constant
lookup succeeds, the user believes the upgrade is clean, and line 2 of
their test helper explodes -- a clean NameError at the require would at
least have been honest. Four of six known real configs write one of these
accessors.

Since `Capybara::Screenshot::Diff` IS `SnapDiff` (whole-module alias),
the config surface exists on both or on neither. It exists:

- All settings are regenerated on both v1 holders as thin delegators onto
  the one storage in SnapDiff.config, GENERATED FROM Config::SETTINGS. No
  hand-maintained table: a 27-row list drifts the first time someone adds
  a setting, and the failure mode of that drift is a NoMethodError on line
  1 of a user's test helper. compat_surface_test walks SETTINGS and fails
  if any name is unreachable.
- Singleton AND instance accessors, because mattr_accessor defined both
  and `include Capybara::Screenshot::Diff` is how the instance ones were
  reached. An include that silently adds nothing is the same class of bug
  as a setter that silently does nothing.
- `enabled` keeps its split: it means `screenshot_enabled` on
  Capybara::Screenshot and `enabled` on ::Diff. They feed different
  branches of Config#active?; collapsing them would change behaviour.
- No deprecation warning. 2.1 deleted the channel that emitted them
  (snap_diff/removal.rb, snap_diff/deprecation.rb) and one warning is not
  worth resurrecting a subsystem for. These are plain delegators.

Two further holes the aliasing itself opened or left:

- `SnapDiff.configure` now yields the config TWICE. The v1 two-holder form
  `configure { |screenshot, diff| ... }` arrives here through the alias;
  with a one-argument yield it bound `diff` to nil and died on nil a few
  lines later -- a regression introduced by the alias, not inherited. A
  one-parameter block ignores the extra argument.
- `CapybaraScreenshotDiff::CapybaraScreenshotDiffError` is aliased to
  SnapDiff::Error. A NameError inside a `rescue` clause fires only when an
  exception is already in flight, converting a real failure into a
  confusing one at the worst moment. The other three v1 error names are
  spelled identically under SnapDiff and came with the module alias.
@pftg
pftg force-pushed the v2.1/delete-legacy-and-drivers branch from ce7ed9e to 3cd2d36 Compare August 24, 2026 06:50
pftg added a commit that referenced this pull request Aug 24, 2026
* docs: 2.0.0 release readiness

Audit of everything a 2.0.0 final would ship, and the fixes that did not
need lib/ changes.

CHANGELOG
- A v2.0.0 entry written for someone upgrading from 1.15.1, not a diff of
  the betas. What to change (the version), what they will see (exact
  warning text), the five things that can actually break, and what 2.1
  removes. Every claim verified against a real install; the beta sections
  stay as history.

Version consistency
- README, docs/UPGRADING.md: no more "beta"/"alpha"/"experiment" framing
  and no beta pins. Gemfile examples say `~> 2.0`.
- Gem name: `capybara-screenshot-diff` is the one we tell people to
  install; `snap_diff-capybara` is a reserved identical mirror. Stated
  once in the README with the dual-install consequence, applied
  everywhere else.
- Stale "3.0" references in the Rakefile and docs/architecture.md are now
  2.1 (#247 fixed the user docs and missed these).

Corrections to claims that were not true
- docs/drivers.md promised that everything 2.1 removes "warns once per
  process naming 2.1". `driver: :auto` is silent whenever ruby-vips is
  present, and the `driver:` setting itself never warns at all even
  though 2.1 deletes it (`NoMethodError`). Both are now written down as
  silent, in drivers.md and UPGRADING.md, since a note is the only notice
  they can get.
- README called ruby-vips "Optional". With neither ruby-vips nor
  chunky_png installed, comparisons raise
  `Wrong adapter nil. Available adapters: []`. Says so now.
- Setup examples no longer teach `driver: :vips`, a line users have to
  delete for 2.1.

Gem hygiene
- gemspec: summary/description that describe what the gem does, the
  rubygems metadata links (source, changelog, bug tracker, docs), and
  docs/docker-testing.md dropped from the package (it documents
  bin/dtest, which is not packaged). Dead bindir/executables removed --
  the allow-list never matched exe/.
- README's links to CONTRIBUTING.md and docker-testing.md are absolute,
  so they resolve from inside the gem too.
- test/unit/gemspec_packaging_test.rb pins the packaged file list: both
  Bundler.require entry files present (this broke twice), consumer docs
  in, contributor docs and build files out, capybara the only runtime
  dependency. Verified it fails when an entry file is unpackaged.
- *.gem is gitignored.

Release process
- The GitHub Release body linked to blob/main on a repo whose default
  branch is master -- 404 on every release so far. Links to the tag now.
- docs/RELEASE_PREP.md was a stale v1.15.1 checklist. It is now a runbook
  for how releases actually happen: what the workflow does step by step,
  the trusted-publisher prerequisite for BOTH gem names, prereleases,
  post-release verification, and what to do when a run fails halfway.
- CONTRIBUTING.md pointed at the wrong version.rb and recommended
  `rake release`, which publishes only one of the two gem names.

Verified with real installs on ruby 4.0.6: 1.15.1 -> this master via
path:, a canonical-names setup, `Bundler.require` under each gem name
from the built .gem, and the dual-install guard with both gems installed.

No lib/ changes. Version not bumped.

* docs: SnapDiff::Error is the base class for errors the gem defines, not every error it raises

docs/snapdiff.md's object map said "Base class for every error this gem
raises". It is not: a missing image backend raises a bare RuntimeError
("Wrong adapter nil. Available adapters: []", reproduced on a bundle with
neither ruby-vips nor chunky_png) and StableScreenshoter raises
ArgumentError. Verified the four defined errors -- ExpectationNotMet,
UnstableImage, WindowSizeMismatchError, DualInstallError -- do all inherit
SnapDiff::Error, so the useful half of the promise holds and is now the
one being made.

* docs: tag protection, not branch protection, is what gates the release tag push

Branch protection rules do not govern tag pushes; tag protection rules
(or rulesets) do. The runbook prerequisite now names the right control.

* docs: the per-screenshot driver: override dies quietly, the config setting dies loudly

Both were lumped together as "raises NoMethodError on 2.1". Only the config
setting does. Per-screenshot options are a free-form hash, so on 2.1
`screenshot "index", driver: :vips` is inert and nothing tells you the line
is dead -- #249's own upgrade note spells out the split. Grep-for-it advice
added, since that is the only signal a user gets.

* docs: pin versions, and stop telling people to delete baselines

Second pass, from customer-persona findings. Each verified here before
acting; two of the four reported items turned out to be artifacts of the
PUBLISHED beta3 rather than of master, and are handled as such.

Baselines (the oldest bug in the tracker: #5 and #6 in 2018, #133 in 2024)
- README told users to "delete the baseline and re-run" in two places.
  It cannot work. `Vcs.checkout_vcs` (lib/snap_diff/vcs.rb:24) resolves
  every baseline with `git show HEAD:<path>`, and
  `ScreenshotMatcher#check_base_screenshot` calls it before
  `need_to_compare?` tests `base_path.exist?` -- so a committed baseline is
  fetched from HEAD no matter what the working tree says, and `rm` changes
  nothing.
- New first-class "Accepting an intentional change" section: the mechanism,
  the commit that actually accepts it, and the surprising part -- staging is
  not enough, so no local run goes green until you commit. The FAQ answer
  now says the same thing instead of the opposite.
- Deliberately does NOT document RECORD_SCREENSHOTS. It is printed by our
  own error message (screenshot_matcher.rb:73) but read nowhere in lib/;
  a separate lane is implementing it, and it should be documented once it
  works, not before.

Version pinning
- `gem "snap_diff-capybara"` unpinned installs 0.0.1 -- a placeholder whose
  entire payload is one README and zero Ruby files (verified by fetching and
  unpacking it), so the user gets an immediate LoadError. And unpinned
  `gem "capybara-screenshot-diff"` resolves to 1.15.1, not to the 2.0 the
  surrounding prose is selling. Every install instruction now pins, and the
  README says plainly that the mirror name is not the one to reach for.

CHANGELOG, all verified
- Failure messages leaked a libvips pointer struct via the comparison
  metadata; `to_h` excludes `diff_mask` since #234, which landed after the
  beta3 tag, so 2.0.0 final is the fix.
- Known limitation: fork-parallel runs write no HTML report. Workers
  accumulate assertions per process; the report is written from
  `Minitest.after_run` in the parent (integrations/minitest.rb:69), which
  never sees them. Artifacts and pass/fail are unaffected.
- A note for anyone sitting on a prerelease: beta3's deprecation channel was
  incomplete, so its silence is not evidence of being migrated.

Constants
- `Capybara::Screenshot::Os` -> `SnapDiff::Os` was in no rename table.

Gemspec
- rubygems_mfa_required. The four URI fields were added in the first commit.

* docs: stop teaching two commands that do not work

Two customer personas independently followed the docs and got a green bar
on a page they had deliberately broken.

`rake test` does not run `test/system/` in a Rails app. The Quick Start
told users to run it, so step 1 produced `0 runs` and no baselines --
which reads as a pass. The example is a Rails system test; the command
now matches it, with a callout, because "0 runs" is the single easiest
way to believe visual testing is working when nothing is running.

`RECORD_SCREENSHOTS=1` appeared in three user-facing docs for a feature
that has never existed in `lib/` -- it is this repository's own test-suite
convention, read by `test/test_helper.rb`. The user-facing copies are
replaced with the flow that actually works: run the suite, which rewrites
every changed baseline in place, then `git add` and commit. The
contributor page keeps it and now says plainly that it is not a library
feature.

* docs: make the Quick Start something a new user can actually run

A reviewer built a stock Rails 8.1 app, ran the documented Quick Start
end to end, and none of it worked. Fixes, each verified against a scratch
app or against lib/:

- `gem "capybara-screenshot-diff", "~> 2.0"` does not resolve. rubygems
  has 1.15.1 and 2.0.0.alpha1..beta3 and no final 2.x, and Bundler never
  picks a prerelease from a plain requirement, so `bundle install` fails
  with `Could not find gem 'capybara-screenshot-diff (~> 2.0)'`. All five
  install snippets now pin `2.0.0.beta3` and say why. RELEASE_PREP gains
  the step that swaps them back to `~> 2.0` as part of the 2.0.0 push,
  so the good pin lands with the release rather than before it.

- The Quick Start taught the API 2.1 deletes, silently. `require
  "capybara_screenshot_diff/minitest"` + `include
  CapybaraScreenshotDiff::Minitest::Assertions` print nothing: they are
  eager aliases, so `const_missing` never fires. The Quick Start now
  starts on canonical `SnapDiff`, and README/CHANGELOG/snapdiff.md say
  which doors actually warn (config accessors, `include`,
  `default_options`, `const_missing`) and which cannot.

- The CI "Record new baselines" job could not record a new baseline.
  `check_base_screenshot` runs before `capture_screenshot` and
  `fail_if_new` is true whenever `ENV["CI"]` is set, so a new screenshot
  raises before anything is written and the commit step finds nothing.
  The job now clears `CI` for that step.

- `bundle exec rake test` / `rails test` swept out of the three CI
  workflows, "The Short Version", and the historical upgrade sections:
  in a Rails app they skip `test/system/` and report `0 runs`.

- The delete-the-baselines block in UPGRADING replaced with the commit
  workflow the README documents.

- `docs/drivers.md` told you to delete the `driver:` setting on one
  screen and to add it on another; same for configuration.md,
  migration-guide.md.

- `rescue SnapDiff::Error` does not catch a failed assertion under the
  framework integrations -- Minitest converts it to `Minitest::Assertion`
  and RSpec to `ExpectationNotMetError`. Said so.

- "2.0 will not rewrite a baseline you already committed" contradicted
  the README and reality: a failing run does rewrite the baseline path.
  Reworded to what is meant (no re-encoding) plus what actually happens.

Also, all verified in a scratch app: the `git add
test/fixtures/screenshots/` path was never the default (`doc/screenshots`
is); the example failure output showed a `max_color_distance` key the
vips path never emits; the artifact table listed three of five files;
`application_system_test_case.rb` omitted both `require "test_helper"`
and `driven_by` (without the latter the same page captures at 2800x1610
instead of 1400x1257); `homepage_test.rb` omitted `require
"application_system_test_case"` and raised NameError as printed; and
`DEBUG=1` never had anything to do with keeping `.diff.png` files.

Docs only. rake test:unit 610/0, standardrb clean.

* docs: three residuals from the verifier pass

- bug_report template told reporters to run `rake test`, which runs zero
  system tests in a Rails app -- the same trap this branch exists to remove.
- drivers.md said an unknown per-screenshot `driver:` key is "simply inert".
  It is validated and raises; only the deprecation warning is absent.
- drivers.md said `:auto` and `:chunky_png` each warn once per process.
  `:auto` is silent when ruby-vips resolves; only `:chunky_png` warns.
@pftg pftg added this to the 2.1.0 milestone Aug 24, 2026
@pftg

pftg commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator Author

Cross-reference: #263 is the 2.0 half (warnings), and this PR now depends on it. See the amendment at the top of the description for the exact lines this branch will need to absorb on its next rebase.

pftg added a commit that referenced this pull request Aug 25, 2026
#281 parked branch protection on "once the check names settle". They do not
settle. Every matrix cell carries its ruby and gemfile in its name, so the
required-checks list breaks whenever the matrix changes shape -- it just did
here (16 cells -> 6) and #249 changes it again.

A protection rule that names cells is a rule that silently stops requiring
anything the moment a cell is renamed. That is the same failure mode as a green
badge over a job that never ran, which is what this whole issue is about.

So `CI Gate` depends on the jobs rather than naming them, and branch protection
requires that one check forever.

`skipped` is a PASS: the matrix does not run on ordinary PRs by design, and a
gate that failed on skipped would block every PR. `failure` and `cancelled` are
not -- cancelled means a cell was killed before it reported, so it carries no
verdict, and treating it as success is exactly how the JRuby lane sat broken for
15 runs.

Logic self-tested on all three shapes:

    success + skipped   -> pass
    success + failure   -> fail (names the job)
    cancelled           -> fail (names the job)

The last unchecked box on #281 is now a one-line setting a human clicks, rather
than a moving target.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant