Skip to content

Update README to address Simplecov intermittently dropping coverage - #123

Open
darronschall wants to merge 2 commits into
briandunn:masterfrom
darronschall:update-readme-for-simplecov
Open

Update README to address Simplecov intermittently dropping coverage#123
darronschall wants to merge 2 commits into
briandunn:masterfrom
darronschall:update-readme-for-simplecov

Conversation

@darronschall

@darronschall darronschall commented May 12, 2026

Copy link
Copy Markdown

Two related fixes to the SimpleCov section of the README, plus removal of the # TODO: possible simplecov fixes block at the bottom, since between them these answer it.

1. The parent / worker merge race

After first integrating flatware, I've seen 2% drops in coverage in every 1-in-3 full suite runs, with a different worker missing from the final Coverage report generated for ... listing each time.

From what I can tell, flatware rspec returns to its top-level exit the moment the DRb sink has seen every worker report its final result — but a worker's at_exit (where SimpleCov stores its resultset slice to disk) runs after the last DRb report goes over the wire. So the parent's own SimpleCov at_exit (which merges all workers' slices into the final report) can fire before the slowest worker finishes writing.

The solution appears to be adding a Process.waitall barrier inside an at_exit registered from before_fork (after require 'rails_helper' has loaded SimpleCov). LIFO at_exit ordering means the barrier fires before SimpleCov's at_exit, so the merge sees the full set of workers' resultsets. The handler is gated to Process.pid == parent_pid; it no-ops in worker forks.

I'm running this setup in production now and have not seen any coverage drops.

This is also what the TODO's item 1 was after — "a process needs to claim to be the last one for simplecov to run the merge" — so I've removed that block.

2. On SimpleCov 1.0, at_fork is already called for you

My previously-added advice in 54232f8 to add SimpleCov.at_fork.call(test_env_number) to after_fork (written for SimpleCov 0.22) is now wrong for most Rails readers, because of two changes in SimpleCov 1.0:

  • The fork hook moved to Process._fork. 0.22 patched Process.fork by alias, which a bare Kernel#fork bypasses — and flatware forks with a bare fork (cli.rb, worker.rb). So on 0.22 the hook never fired for flatware and the manual call was genuinely needed. 1.0 prepends to Process._fork, the official extension point every fork path funnels through, so it now does fire here and calls SimpleCov.at_fork in the child itself.
  • The rails profile enables subprocess merging. The hook is only installed when enabled_for_subprocesses?, and 1.0's rails profile turns that on (0.22's didn't). So SimpleCov.start "rails" opts you in without asking.

Together that means my previously documented call ran the default at_fork twice per worker. Since it names each slice by appending its ordinal to the current command_name, every worker lands in the merged report as RSpec (subprocess: 1) (subprocess: 1).

I've split the guidance by case in c3ce137 rather than dropping the call, since it's still correct whenever the hook isn't installed, and pointed at SimpleCov.enabled_for_subprocesses? as the way to tell which case you're in.

Since 1.0, enabling subprocess merging makes SimpleCov hook
`Process._fork` and call `SimpleCov.at_fork` in the child itself.
`SimpleCov.start "rails"` turns that on, so the README's advice to add
`SimpleCov.at_fork.call(test_env_number)` to `after_fork` now
double-applies the default `at_fork` for most Rails readers. That lambda
names each slice by appending its ordinal to the current command_name,
so every worker shows up in the merged report as
"RSpec (subprocess: 1) (subprocess: 1)".

Two things had to change at once for that to bite, which is why the old
advice was right until now. 0.22 hooked `Process.fork` by alias, and
flatware forks with a bare `Kernel#fork` — that routes through
`Process._fork` but not through `Process.fork`, so 0.22's hook never
fired here and the manual call was genuinely needed. 1.0 also flipped
the "rails" profile to enable subprocess merging, which is what installs
the hook in the first place.

So the guidance is split by case rather than dropped: it is still
correct wherever no hook is installed, with
`SimpleCov.enabled_for_subprocesses?` given as the way to tell which
case you're in.

Two smaller fixes in the same section: the per-file threshold example
used `minimum_coverage_by_file`, which 1.0 deprecates in favour of
`coverage(:line) { minimum_per_file 0 }` (the old form is kept beside it
for 0.22), and the sample's `at_exit` line carried a stray leading space.
@darronschall
darronschall force-pushed the update-readme-for-simplecov branch from e3eb24b to c3ce137 Compare July 30, 2026 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant