Skip to content

configuration.md

Latest commit

 

History

History
701 lines (515 loc) · 27.4 KB

configuration.md

File metadata and controls

701 lines (515 loc) · 27.4 KB
Raw

Configuration Reference

Quick Setup

Canonical (v2): every setting lives on one flat object, SnapDiff.config.

# In test_helper.rb or rails_helper.rb
SnapDiff.configure do |config|
  config.window_size = [1280, 1024]
  config.stability_time_limit = 1
  config.blur_active_element = true
  config.hide_caret = true
  config.tolerance = 0.0005
  config.color_distance_limit = 15
end

Legacy (still supported): the two-holder block, split across Capybara::Screenshot and Capybara::Screenshot::Diff.

Capybara::Screenshot::Diff.configure do |screenshot, diff|
  screenshot.window_size = [1280, 1024]
  screenshot.stability_time_limit = 1
  screenshot.blur_active_element = true
  screenshot.hide_caret = true
  diff.tolerance = 0.0005
  diff.color_distance_limit = 15
end

driver: is deliberately absent from both examples. The setting is removed in 2.1 (NoMethodError at config time) and 2.0 cannot warn about it. Add gem "ruby-vips" and leave the selection alone — see Drivers.

SnapDiff::Config is the storage; the legacy accessors are thin delegators onto it. There is one source of truth, so a write through either surface is visible through the other — mixing them is safe, and you can migrate a suite one line at a time:

SnapDiff.config.window_size = [1280, 1024]
Capybara::Screenshot.window_size    # => [1280, 1024]

Every option name below is identical on both surfaces — only the receiver changes. The one exception: Capybara::Screenshot.enabled is SnapDiff.config.screenshot_enabled, because SnapDiff.config.enabled is taken by Capybara::Screenshot::Diff.enabled. See SnapDiff — the canonical API for the full SnapDiff-native surface.

Note: Setting SnapDiff.config.screenshot_enabled = false is sufficient to disable all screenshots. There is no need to define no-op modules or monkey-patch the gem.

Record modes — accepting changes

record is the single setting for what happens when a screenshot has no committed baseline, or when you want to accept the ones that changed. It replaces fail_if_new, which is removed in 2.1.

SnapDiff.config.record = :once   # what a local run already does; see Precedence below for CI
Mode Missing baseline Baseline present Reach for it when
:once recorded, not compared compared the default — you want a normal run
:none fails with the git add command attached compared every screenshot must already be recorded
:all recorded re-recorded, not compared you changed the UI on purpose and want the new rendering to become the baseline

:all — the bulk-accept verb

After an intentional redesign that changed forty screenshots, re-record them in one run rather than accepting them one failure at a time:

# test_helper.rb. There is no CLI flag — the gem has no runner to hang one on —
# so gate it on an environment variable of your own if you want one:
SnapDiff.config.record = ENV["ACCEPT_SCREENSHOTS"] ? :all : :once
ACCEPT_SCREENSHOTS=1 bin/rails test:system     # with the line above in test_helper.rb

Every screenshot is written to its baseline path and nothing is compared, so git status lists exactly what changed. Review the images, then commit them — see Accepting an intentional change.

At the end of the run the gem names what it accepted:

[snap_diff] record: :all re-recorded 3 screenshots WITHOUT comparing: checkout/cart, checkout/payment, checkout/review. Review the result before committing -- an unintended change is accepted just as silently.

:all refuses to run under CI (when ENV['CI'] is set to a non-empty value). It accepts every rendering by design, so a mode left in a committed config file would buy you a build that compares nothing and passes forever, with the "recorded" screenshots discarded when the runner is torn down. Re-record locally, where you can look at the result.

A CI job that needs to record screenshots with no baseline yet does not need :all: record = :once records those and still compares everything that has a baseline. See CI integration.

Per screenshot

assert_matches_screenshot "flaky_widget", record: :none   # this one must already exist

Precedence

An explicitly set mode outranks fail_if_new; fail_if_new decides only when no mode was set. That is the same rule fail_if_new itself has over the CI sniff — explicit outranks implicit, all the way down. Per screenshot outranks the config; the config outranks fail_if_new.

With nothing set, record reads back as :none under CI and :once off it — which is exactly what fail_if_new has always done, so a suite that never mentions record behaves as it always did. The missing-baseline default is deliberately unchanged: failing only under CI is what Jest, AVA, Vitest, testthat and jest-image-snapshot all chose, and a screenshot baseline recorded on your laptop is often worthless on another OS. :none makes strictness an explicit choice instead.

You wrote record reads Missing baseline
nothing, off CI :once recorded
nothing, under CI :none fails
record = :once :once recorded, on CI too
record = :none :none fails, off CI too
record = :once, fail_if_new = true :once recorded — the mode wins
record = nil, fail_if_new = true :none fails — nil hands it back

A misspelt mode raises ArgumentError at the point you set it, rather than reading back as "nobody said".

Recommended tolerance values

Use Case VIPS tolerance ChunkyPNG color_distance_limit stability_time_limit
Animated/complex pages 0.01 30 2s
Standard Rails apps 0.001 (default) 15 1s
Pixel-perfect design tests 0.0001 5 1s

Note: VIPS defaults to tolerance: 0.001 (allows 0.1% pixel difference). ChunkyPNG has no default tolerance.

Choosing the Right Color Comparison Method

Important: perceptual_threshold, color_distance_limit, and tolerance serve different purposes. Use this decision tree:

Step 1: Choose color comparison method (pick ONE)

Method Scale Driver Best for
perceptual_threshold 0-100+ (dE00) VIPS only Cross-OS/browser font rendering, anti-aliasing
color_distance_limit 0-510 (RGBA Euclidean) VIPS, ChunkyPNG Legacy setups, fine-grained RGB control

Recommendation: Use perceptual_threshold: 2.0 for most cases. It matches human perception and needs less tuning.

⚠️ Color comparison methods are exclusive: perceptual_threshold and color_distance_limit cannot both be active — if you set both, perceptual_threshold wins and color_distance_limit is ignored. However, tolerance works with both methods and is applied by default for VIPS (0.001). This means even with perceptual_threshold: 2.0, the tolerance: 0.001 default still filters results.

Step 2: Set tolerance (optional, independent)

Setting What it does Scale
tolerance Maximum allowed ratio of different pixels (VIPS) or diff bounding box (ChunkyPNG) 0.0-1.0

Example: tolerance: 0.001 allows 0.1% of the image to differ (e.g., 125 pixels in a 1280×1024 screenshot).

Key difference:

  • perceptual_threshold / color_distance_limit → "how different can a pixel be?"
  • tolerance → "how many pixels can differ?"

⚠️ Driver difference: VIPS counts actual different pixels. ChunkyPNG counts the bounding box area around differences — a single pixel diff creates a box, and the entire box area counts against tolerance. This makes ChunkyPNG stricter with the same tolerance value.

Quick start

# Modern approach (recommended)
screenshot 'dashboard', perceptual_threshold: 2.0

# Allow small noise regions
screenshot 'dashboard', perceptual_threshold: 2.0, tolerance: 0.001

# Legacy ChunkyPNG setup
screenshot 'dashboard', color_distance_limit: 15

Configuration Tiers

Tier 1 — Zero config (works immediately): blur_active_element and hide_caret are on by default, and record behaves as :none in CI (a missing baseline fails) and :once off it. Just require 'snap_diff/integrations/minitest' (legacy: capybara_screenshot_diff/minitest) and call screenshot.

Tier 2 — Set when tests are flaky:

Setting When to use
window_size Screenshots differ between machines due to different browser sizes
tolerance Sub-pixel rendering differences cause false positives
skip_area Dynamic content (timestamps, ads) changes between runs
stability_time_limit Animations or loading states cause inconsistent captures

Tier 3 — Advanced tuning:

Setting When to use
perceptual_threshold Anti-aliasing false positives across OS/browser versions
shift_distance_limit Content shifts by a few pixels (ChunkyPNG only — removed in 2.1)
area_size_limit Allow small diff regions below a pixel count
color_distance_limit Fine-tune raw RGB channel tolerance
median_filter_window_size Smooth noise before comparison (VIPS only)

Common Options

Screen size

You can specify the desired screen size using

Capybara::Screenshot.window_size = [1024, 768]

This will force the screen shots to the given size, and skip taking screen shots unless the desired window size can be achieved.

Disabling screen shots

If you want to skip taking screen shots, set

SnapDiff.config.screenshot_enabled = false

You can of course set this by an environment variable

SnapDiff.config.screenshot_enabled = ENV['TAKE_SCREENSHOTS']

A disabled screenshot is not an assertion, and Minitest is told so: a test whose only assertion was a screenshot reports as missing assertions rather than as a pass over nothing.

Disabling diff

If you want to skip the assertion for change in the screen shot, set

Capybara::Screenshot::Diff.enabled = false

Using an environment variable

Capybara::Screenshot::Diff.enabled = ENV['COMPARE_SCREENSHOTS']

Tolerate screenshot differences

Removed in 2.1. A screenshot that differs from its baseline fails — that is what the gem is for. To accept a difference, re-record it: record = :all. It keeps working for the whole 2.x line and warns once per process.

To allow screenshot differences, but still fail on functional errors, you can set the following option:

Capybara::Screenshot::Diff.fail_on_difference = false

It defaults to true. This can be useful in continuous integration to a generate a screenshot difference report while still reporting functional errors.

Does not tolerate new screenshots

Removed in 2.1, superseded by record. record = :none is fail_if_new = true; record = :once is fail_if_new = false. Unlike the boolean, a mode means the same thing on CI and off it. It keeps working for the whole 2.x line and warns once per process.

To fail the test if a new screenshot is taken, set the following option:

Capybara::Screenshot::Diff.fail_if_new = true

If fail_if_new is set to true, the test will fail if a new screenshot is taken that does not have a corresponding previous image to compare against. This can be useful in situations where you want to ensure that every screenshot taken by your tests corresponds to an expected state of your application.

fail_if_new defaults to true in CI environments (when ENV['CI'] is set to a non-empty value). Setting it yourself outranks the environment: fail_if_new = false stays false under CI=true. Assign nil to hand it back to the environment. Setting record outranks it either way.

Marks new screenshots as pending

Removed in 2.1. It skips the test instead of saying what to do about the missing baseline. record = :none fails with the git add command attached; record = :once records the screenshot and names it in the end-of-run summary. It keeps working for the whole 2.x line and warns once per process.

To mark tests as pending (skipped) if a new screenshot is taken without a baseline, set:

Capybara::Screenshot::Diff.pending_if_new = true
# Required in CI, because fail_if_new defaults to true there and raises before
# the pending marker is applied.
Capybara::Screenshot::Diff.fail_if_new = false

If pending_if_new is set to true, the test will be marked as skipped in teardown when a new screenshot has no committed baseline to compare against. This is complementary to fail_if_new (which raises immediately); fail_if_new takes precedence since it raises first. This option is useful when you want to record new screenshots without blocking CI, but still track them as needing review.

Screen shot save path

By default, Capybara::Screenshot::Diff saves screenshots to a doc/screenshots folder, relative to either Rails.root (if you're in Rails), or your current directory otherwise.

If you want to change where screenshots are saved to, then there are two configuration options that that are relevant.

The most likely one you'll want to modify is ...

Capybara::Screenshot.save_path = "other/path"

The save_path option is relative to Capybara::Screenshot.root.

Capybara::Screenshot.root defaults to either Rails.root (if you're in Rails) or your current directory. You can change it to something entirely different if necessary, such as when using an alternative web framework.

Capybara::Screenshot.root = Hanami.root

Screen shot stability

To ensure that animations are finished before saving a screen shot, you can add a stability time limit. If the stability time limit is set, a second screen shot will be taken and compared to the first. This is repeated until two subsequent screen shots are identical.

Capybara::Screenshot.stability_time_limit = 0.1

This can be overridden on a single screenshot:

test 'stability_time_limit' do
  visit '/'
  screenshot 'index', stability_time_limit: 0.5
end

When the page will not settle

The failure names the area that kept changing, and hands you the command that fixes it:

Could not get stable screenshot for 'index-with-ticker' within 1.2s (5 attempts).
  The page kept changing in 1 area, over 4 attempt pairs:
    [67,50,213,68] (left,top,right,bottom edges) -- 0.55% of the 800x600 image, changed in 4 of 4 pairs
  Always the same area, in every pair: that is an animation, clock, carousel or live counter.
  Exclude it and the page is stable without waiting:
    assert_matches_screenshot "index-with-ticker", skip_area: [67,50,213,68]
  <one annotated attempt image per line>

The coordinates are measured, not guessed: they come from the comparisons the gem just ran between consecutive attempts, so pasting the suggested skip_area in works.

One caveat, and it is a real one. The suggested box is the union of what changed across the attempts that ran — a sample of the animation, not a proven bound on it. If the moving thing also changes SIZE between frames (proportional text of varying width in a centred box is the common case), a later frame can render a pixel or two outside the box that was measured, and the masked run fails again with a much smaller region. Paste the new suggestion, or widen the box by a few pixels; it converges. For the usual case — a clock, spinner or counter repainting inside a fixed element — the extent does not move and the first suggestion is the whole fix.

Read the "changed in N of N pairs" line before acting on it:

  • N of N — one area, every single pair. Something is animating in one place: a clock, a carousel, a spinner, a live counter. skip_area is the fix, and since masking no longer waits it costs nothing.
  • Fewer than N, or several areas each changing once. The page is still rendering, not animating. Masking those areas would hide real content. The message says so and suggests nothing to mask — settle the page in a readiness block instead, or raise wait:.

Knowing what the waiting cost

Every run that waited for stability ends with what it actually paid:

[snap_diff] 34 screenshots waited for the page to settle: 0.19s and 2 attempts at worst. Every screenshot settled on its first retry, so a lower stability_time_limit would cost less per screenshot.

Two attempts is the floor — one capture, plus the retry that matched it. Hitting the floor across the whole run means no page was ever still moving when the retry was taken, so every stability_time_limit sleep was spent on a page that had already stopped. That is the evidence for tuning it down; without it, lowering the setting is guesswork, and guesswork loses to sleep.

Run-level and silent when nothing waited, for the same reason as the never-matched-selector line: a line printed on every screenshot is a line people learn to skip.

Maximum wait limit

When the stability_time_limit is set, but no stable screenshot can be taken, a timeout occurs. The timeout occurs after Capybara.default_max_wait_time, but can be overridden by an option.

test 'max wait time' do
  visit '/'
  screenshot 'index', wait: 20.seconds
end

Hiding the caret for active input elements

In Chrome the screenshot includes the blinking input cursor. This can make it impossible to get a stable screenshot. To get around this you can set the hide caret option:

Capybara::Screenshot.hide_caret = true

This will make the cursor (caret) transparent (invisible), so the blinking does not delay the screen shot.

Removing focus from the active element

Another way to avoid the cursor blinking is to set the blur_active_element option:

Capybara::Screenshot.blur_active_element = true

This will remove the focus from the active element, removing the blinking cursor.

Allowed color distance

Sometimes you want to allow small differences in the images. For example, Chrome renders the same page slightly differently sometimes. You can set set the color difference threshold for the comparison using the color_distance_limit option to the screenshot method:

test 'color threshold' do
  visit '/'
  screenshot 'index', color_distance_limit: 30
end

The difference is calculated as the euclidean distance. You can also set this globally:

Capybara::Screenshot::Diff.color_distance_limit = 42

Allowed shift distance

Removed in 2.1. shift_distance_limit is implemented only by the ChunkyPNG driver, and 2.1 removes that driver — libvips becomes the only backend. Setting it anywhere (SnapDiff.config.shift_distance_limit =, the legacy Capybara::Screenshot::Diff.shift_distance_limit =, or screenshot 'index', shift_distance_limit: 2) warns once per process in 2.0. There is no vips equivalent: use median_filter_window_size (the faster answer to the same problem — see Drivers), tolerance, or color_distance_limit.

Sometimes you want to allow small movements in the images. For example, jquery-tablesorter renders the same table slightly differently sometimes. You can set set the shift distance threshold for the comparison using the shift_distance_limit option to the screenshot method:

test 'color threshold' do
  visit '/'
  screenshot 'index', shift_distance_limit: 2
end

The difference is calculated as maximum distance in either the X or the Y axis. You can also set this globally:

Capybara::Screenshot::Diff.shift_distance_limit = 1

Note: For each increase in shift_distance_limit more pixels are searched for a matching color value, and this will impact performance severely if a match cannot be found.

If shift_distance_limit is nil shift distance is not measured. If shift_distance_limit is set, even to 0, shift distance is measured and reported on image differences.

Allowed difference size

You can set set a threshold for the differing area size for the comparison using the area_size_limit option to the screenshot method:

test 'area threshold' do
  visit '/'
  screenshot 'index', area_size_limit: 17
end

The difference is calculated as width * height. You can also set this globally:

Capybara::Screenshot::Diff.area_size_limit = 42

Skipping an area

Sometimes you have expected change that you want to ignore. You can use the skip_area option with [left, top, right, bottom] or css selector like '#footer' or '.container .skipped_element' to the screenshot method to ignore an area.

test 'unstable area' do
  visit '/'
  screenshot 'index', skip_area: [[17, 6, 27, 16], '.container .skipped_element', '#footer']
end

skip_area masks what is on the page at assertion time — it does not wait. A selector is resolved against the DOM as it is when the screenshot is taken; one that matches nothing masks nothing, immediately. (Until 2.0 it blocked for Capybara.default_max_wait_time per unmatched selector — 5s each, on every screenshot. That wait is gone.)

So content that arrives late — lazy-loaded images, JS-injected widgets, anything behind an unresolved fetch — has to be settled before the assertion, or its mask will be empty and the unstable region will be compared. Settle it in the readiness block described below; the two cases people hit most (webfonts and lazy images) are written out in Recipes.

If a selector matched nothing in every screenshot of a run, the end-of-run summary names it:

[snap_diff] 1 selector never matched anything in this run: "artcile img". A selector that matches nothing masks nothing -- check for a typo or a stale selector.

That is a run-level fact on purpose. Per screenshot the gem cannot tell a typo from a page that legitimately has no images, so it says nothing; a selector that matched somewhere is doing its job and is never mentioned. Nothing is printed when every selector matched.

The readiness block

assert_matches_screenshot and capture_screenshot (and the screenshot / assert_no_screenshot_changes wrappers) take an optional block. It runs once, after the enabled check and before the capture:

test 'gallery' do
  visit '/gallery'
  assert_matches_screenshot 'gallery', skip_area: ['article img'] do
    scroll_to :bottom
    assert_text 'End of gallery'
    scroll_to :top
  end
end

Why a block rather than the line above it: work in the block is skipped when screenshots are off. Both methods return immediately when SnapDiff.config.enabled (or screenshot_enabled) is false, so a preload_all_images written on the preceding line still pays for its browser round-trips — scroll, wait, scroll back — for a screenshot that is never taken. Inside the block it costs nothing, which is what makes turning visual tests off actually free.

Errors raised in the block are yours and propagate unchanged. It is not a hook: there is no configuration-level equivalent, no after-block, and it runs once per assertion rather than once per stability retry.

In RSpec, call assert_matches_screenshot directly rather than through the match_screenshot matcher — expect(page).to match_screenshot('x') { ... } binds the block by Ruby's {}/do...end precedence rather than by intent, so the matcher does not take one. In Cucumber the DSL is in the World, so step definitions pass a block the same way a Minitest test does.

Recipes

These are the workarounds people hand-roll anyway. The block is where they belong, because that is the only place they are skipped when screenshots are off.

Webfonts. A font swapping in mid-capture reflows every line of text that uses it, so the same page renders two different ways depending on when the screenshot lands — the classic "it only fails on CI" flake. People usually paper over it with a skip_area, a loosened tolerance and a retry, all three of which weaken the comparison everywhere. Wait for the swap instead:

assert_matches_screenshot 'home' do
  page.evaluate_async_script(
    'var done = arguments[0]; document.fonts.ready.then(function(){ done(true) })'
  )
end

document.fonts.ready resolves once every font used by the current layout has loaded (or failed) — and on a warm cache that has already happened, so the call returns on the first round trip. It is a Font Loading API promise, supported in every browser Capybara drives.

There is deliberately no built-in font wait: it would be a browser round trip imposed on every screenshot in every suite, and a driver-compatibility surface the gem would own forever, in exchange for one line you can write yourself.

Lazy-loaded images. Anything behind loading="lazy", an IntersectionObserver or an unresolved fetch is simply not there when the screenshot is taken. Force it in, then come back:

assert_matches_screenshot 'gallery', skip_area: ['article img'] do
  scroll_to :bottom
  assert_text 'End of gallery'
  scroll_to :top
end

Note the order: skip_area: ['article img'] masks what exists at assertion time, so the images have to be in the DOM before the mask is resolved. A selector for content that has not loaded yet produces an empty mask and the unstable region is compared anyway — see Skipping an area.

What does not belong here. Waits that every screenshot needs regardless (disable_animations, hide_caret) are configuration, not readiness. And the block runs once per assertion, not once per stability retry, so it cannot be used to nudge a page that keeps moving — for that, see When the page will not settle.

The arguments are [left, top, right, bottom] for the area you want to ignore. You can also set this globally:

Capybara::Screenshot::Diff.skip_area = [0, 0, 64, 48]

If you need to ignore multiple areas:

screenshot 'index', skip_area: [[0, 0, 64, 48], [17, 6, 27, 16], 'css_selector .element']

Skipping stack frames in the error output

If you would like to override the screenshot method or for some other reason would like to skip stack frames when reporting image differences, you can use the skip_stack_frames option:

test 'test visiting the index' do
  visit root_path
  screenshot :index
end

private

def screenshot(name, **options)
  super(name, skip_stack_frames: 1, **options)
end

Screenshot Format

You can specify the format of the screenshots taken by setting the screenshot_format option. By default, the format is set to "png". However, you can change this to any format supported by your image processing driver. For example, to set the format to "webp", you can do the following:

Capybara::Screenshot.screenshot_format = "webp"

Customize Capybara#screenshot options

Allow to bypass screenshot options to Capybara driver.

# To create full page screenshots for Selenium
Capybara::Screenshot.capybara_screenshot_options[:full_page] = true

screenshot('index', median_filter_window_size: 2, capybara_screenshot_options: {full_page: false})

← Back to README