Differences on font/text positioning #815
Description
Within our team we wanted to try setting up visual regression testing on our Elixir liveview application to ensure we don't end up causing UI issues when bringing some modifications.
We set up the common chromeOptions capabilities:
config :wallaby,
max_wait_time: 10_000,
screenshot_dir: "./_screenshots",
js_errors: true,
js_logger: nil,
chromedriver: [
# be careful when disabling headless, because the resolution of your monitor will affect the resulting screenshots.
headless: true,
capabilities: %{
chromeOptions: %{
args: [
"--no-sandbox",
"window-size=1920,1080",
"--force-device-scale-factor=1",
"--disable-gpu",
"--force-color-profile=scrgb",
"--headless",
"--fullscreen",
"--disable-features=HDR",
"--font-render-hinting=none",
"--user-agent=Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36"
]
}
}
]*,
*::after,
*::before {
transition-delay: 0s !important;
transition-duration: 0s !important;
animation-delay: -0.0001s !important;
animation-duration: 0s !important;
animation-play-state: paused !important;
}We set up a threshold of 1% for differences and tried running it. The threshold was never met between running the tests locally to record the screenshots(on a Macbook air M1) and our CI pipeline on Github actions because of the text font. There was always some differences right at the sides of text, we thought that it could be related to anti-aliasing and tried a bunch of chromium command line switches that didn't make things better.
A hardcore solution we tried was setting the font a Minecraft bitmap font, immediately things looked better and much of the text that had a regular font-width: 400 and font-size: 1rem looked exactly similar when recording screenshots on my Macbook and on the CI pipeline, so I added the following css to fix the font size and weight and disable smoothing.
font-family: "Minecraft" !important;
font-smooth: never !important;
-webkit-font-smoothing: none !important;
-moz-osx-font-smoothing: none !important;
font-weight: 400 !important;
font-size: 16px !important; This immediately makes all text exactly alike between the two environments, but the exception is centered text.
It seems when the browser calculates the position of the centered text, it ends up with some subpixel positioning. which causes the text on CI to have a more pixelated look on some of the centered text. the image below displays a look of the differences for reference.
This would be the screenshot recorded at my macbook.
And this would be the picture taken during CI.
The difference might be small in these two pictures, but eventually when a page has more centered text, there's a big difference and that would end up breaking even a 10% threshold. I tried to add "--disable-font-subpixel-positioning" but the difference is still there.
Before we opted for a devopsy solution where we would run our tests in an environment similar to CI to record new screenshots(which is very discouraging because we like the idea of being able to update the screenshots by simply running the tests on your machine) we want to post about it to see if somebody already has a more convenient solution for this case.
For information: We are using chromedriver and chromium version 136.