fix: libvips serves stale pixels when a screenshot is rewritten within the same second

[pftg]

The bug

libvips caches loader operations keyed on filename + mtime, and mtime has one-second resolution. Overwrite a path and re-read it within the same second, and the loader hands back the previous image.

This gem does exactly that: the screenshoter writes <name>.png, checkout_base_screenshot writes <name>.base.png from VCS, and the comparison then reads both. The result is a comparison against an image that is no longer on disk — a pass, or a bogus diff, that the user cannot reproduce.

Reproduced on unmodified master (d5b7a9e), ruby-vips 2.3.0 / libvips 8.18.5:

truth: a.png avg=160.6641015625  b.png avg=160.3706640625
elapsed since first read: 3.9 ms (same wall-clock second: true)
read 1 (file was b.png): avg=160.3706640625
read 2 (file is  a.png): avg=160.3706640625
STALE - got b.png (the PREVIOUS image)

Not a red assertion — the driver returned b.png's pixels while a.png was on disk.

The fix

REVALIDATE = Vips.at_least_libvips?(8, 15) ? {revalidate: true}.freeze : {}.freeze

applied at the single load site, VipsDriver#from_file. Every caller routes through it (load_images, Screenshoter#take_screenshot), so one keyword at that seam fixes all of them. revalidate is libvips 8.15+, hence the version guard; :revalidate ("Don't use a cached result for this operation") and at_least_libvips?(major, minor) are both confirmed against the ruby-vips docs.

Ported from #249, which fixes the same seam on the 2.1 branch.

Mutation check

state	result
fix absent (master)	1 failures — Expected 160.3706640625 to not be equal to 160.3706640625 (b.png's average, the stale image)
fix present	51 runs, 106 assertions, 0 failures
fix reverted again with a targeted edit	red again, same reason

Perf

revalidate: true costs nothing. The gem reads each path at most once per comparison, so the loader cache can never help it — the honest measurement gives every read a distinct path. 30 reads per shape, each forced through .avg:

shape	plain (ms)	revalidate	delta	delta (repeat run)
80x60 (fixture)	1.550	1.147	-0.403	-0.229
800x600	6.680	6.663	-0.017	+0.661
1920x1080	20.104	20.783	+0.679	+0.093
1280x4000 (long page)	50.811	49.936	-0.875	+0.192

All four within ±0.9 ms, and the sign of the delta flips between runs. Matches #249's finding.

(A benchmark that re-reads one path N times appears to show revalidate costing up to +19 ms on 1920x1080 — but that baseline is serving stale cache hits, which is the bug itself, not a workload this gem has.)

#250's byte-identical short-circuit runs before any decode, so it is untouched by this change.

The test-harness workaround

test/system_test_case.rb flushed the vips cache in teardown (cache_set_max(0) then 1000) — a workaround for this exact bug, sitting in the harness where it protected our own suite while the bug shipped to users. Removed here, as in #249.

Evidence for the removal, all with SCREENSHOT_DRIVER=vips:

workaround	fix	result
present	mutated out	485 runs, 1 failure — only the new unit test
removed	mutated out	485 runs, 1 failure — only the new unit test
removed	present	485 runs, 0 failures

Which existing tests would have caught a regression if the workaround had stayed? None — and that is the point. With the workaround in place or removed, the mutated fix reds exactly one test: the new #from_file re-reads a path that was overwritten within the same second. The system suite never caught this, because within one test the two paths (<name>.png and <name>.base.png) are distinct and each is read once; the flush was guarding a cross-test rewrite of the same path that the suite does not deterministically hit. Removing it is safe, and the new unit test is now the only thing standing between this bug and users.

Gates

• rake test:unit — 588 runs, 0 failures (also green under SCREENSHOT_DRIVER=vips)
• rake test:canonical — 485 runs, 0 failures, 1 skip (also green under SCREENSHOT_DRIVER=vips)
• standardrb lib test — 151 files, no offenses

Untouched: CHANGELOG.md, README.md, docs/, version.rb.

🤖 Generated with Claude Code

https://claude.ai/code/session_014BQJX6eWzBj2UTm5zQsjEs

Summary by Sourcery

Ensure Vips image loads revalidate rewritten screenshot files so comparisons always use the pixels currently on disk.

Bug Fixes:

• Prevent the Vips driver from returning stale image pixels when a screenshot file is overwritten and reread within the same second.

Enhancements:

• Remove the system-test cache-flushing workaround now that stale reads are handled at the image-loading boundary.

Tests:

• Add a regression test covering rereading an overwritten image path within the same second.

[sourcery-ai]

Sorry @pftg, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

[coderabbitai]

Warning

Review limit reached

Next included review available in 40 minutes.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: ea357ca3-c5b1-432a-a069-d64979d8f664

📥 Commits

Reviewing files that changed from the base of the PR and between d5b7a9e and db07eea.

📒 Files selected for processing (3)

• lib/snap_diff/drivers/vips_driver.rb
• test/system_test_case.rb
• test/unit/drivers/vips_driver_test.rb

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[sourcery-ai]

Reviewer's Guide

Ensure VipsDriver always loads fresh image data instead of stale cached pixels when a screenshot file is overwritten and re-read within the same second, and remove a now-unnecessary test harness cache flush while adding a focused regression test for this behavior.

Sequence diagram for fresh Vips image loading after overwrite

sequenceDiagram
    participant Screenshoter
    participant VipsDriver
    participant Libvips
    participant Filesystem

    Screenshoter->>Filesystem: write screenshot path
    Screenshoter->>VipsDriver: from_file(filename)
    VipsDriver->>Libvips: Image.new_from_file(filename, **REVALIDATE)
    Libvips->>Filesystem: read current pixels
    Filesystem-->>Libvips: latest image data
    Libvips-->>VipsDriver: fresh image
    VipsDriver-->>Screenshoter: image for comparison

Loading

File-Level Changes

Change	Details	Files
Force libvips to bypass stale loader cache entries when reading images through the VipsDriver.	Introduce a REVALIDATE constant that conditionally sets the libvips revalidate: true option for versions >= 8.15. Update the VipsDriver#from_file method to pass **REVALIDATE when constructing a new Vips::Image from a file. Document in-code the libvips filename+mtime caching behavior and why revalidate is required for this driver.	lib/snap_diff/drivers/vips_driver.rb
Remove the vips cache flush workaround from the system test teardown, relying instead on the driver-level fix.	Delete the conditional Vips.cache_set_max calls from SystemTestCase#teardown that previously reset the libvips cache after tests using the vips driver. Add comments explaining that the former workaround is no longer needed because VipsDriver#from_file now uses revalidate: true and that coverage is provided by a regression test.	test/system_test_case.rb
Add a regression unit test that proves #from_file returns fresh pixels when a path is overwritten within the same second.	Remove the per-test vips cache flush from VipsDriverTest teardown since the driver fix makes it unnecessary. Add a new test that copies b.png to a probe path, reads and averages it, overwrites the same path with a.png within the same second, then re-reads via #from_file and asserts the averages differ and the second read matches a.png. Ensure the probe file is cleaned up with FileUtils.rm_f in an ensure block to keep the test isolated.	test/unit/drivers/vips_driver_test.rb

Possibly linked issues

• #unknown: The PR fixes stale libvips pixels after screenshot rewrites, plausibly causing the issue’s invisible differences and unexpected screenshot modifications.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.