Mutation testing: --filter exceeds the kernel's per-argument limit (and PCRE's compile limit) on widely-covered classes

[yeapea]

Filed here because issues are disabled on pestphp/pest-plugin-mutate. The code
referenced below lives in that package.

What happens

On a class that many tests cover, every mutation dies before it runs:

proc_open(): posix_spawn() failed: Argument list too long
  at vendor/symfony/process/Process.php:411

The mutation is reported as not measured, and with a shard-per-directory CI setup the
whole run fails.

Why

MutationTest::start() selects the covering tests by building one regex fragment per test
and joining all of them into a single argv element:

// pest-plugin-mutate/src/MutationTest.php:92
'--filter="'.implode('|', $filters).'"',

Linux caps one argv element at MAX_ARG_STRLEN = PAGE_SIZE * 32
(include/uapi/linux/binfmts.h). x86_64 always has 4 KiB pages, so the cap is exactly
131072 bytes, and it cannot be tuned — ulimit raises the total ARG_MAX, never the
per-element limit. macOS has no per-element cap at all, which is why this reproduces in CI
and not on a developer machine.

Measured on one real mutation, in a small utility class that essentially every component
test reaches (2076 covering tests across 179 classes):

encoding	bytes	vs 131072
one fragment per test (current)	172,779	over
factor the class prefix	124,987	fits, +5% only
collapse each class to Cls::	4,205	fits, 31x headroom

It is over two limits, not one

Worth knowing before picking a fix: PCRE also refuses that pattern.

preg_match(): Compilation failed: regular expression is too large at offset 163235

So even a transport that allowed a 170 KB argument would hand PHPUnit a filter it cannot
compile. The encoding itself has to get shorter; moving the same payload elsewhere is not
enough.

The TODO above the loop cannot fix it as written

// TODO: we should pass the tests to run in another way, maybe via cache, mutation or env variable

A cache or file carrier works. An environment variable does not:
do_execveat_common() runs the same copy_strings() over envp as over argv, and the
MAX_ARG_STRLEN check lives inside it — the environment is under the identical
per-element cap. Flagging it only so that route is not chosen and found wanting later.

What we did, and would gladly contribute

An adaptive encoding, applied in order:

1. Factor the class prefix — always. Cls::(.*)a|Cls::(.*)b becomes Cls::(.*)(a|b).
   Lossless: replayed against the real test list, both forms select the identical set. The
   inner parentheses matter — without them the alternation binds to the whole pattern
   instead of the tail.
2. Collapse the heaviest class to a bare Cls:: if it still does not fit, largest
   first, stopping as soon as it fits. That is a strict superset of the original selection
   (2894 vs 2243 tests on the measured case), so it can only ever run MORE tests — it can
   never turn a killed mutant into a survivor.
3. Budget below the hard cap (we use 96 KiB): a filter sized to the exact limit is one new
   test away from failing again.
4. Throw with the measured length if even a full collapse does not fit, rather than
   dropping the filter. Dropping it would run the whole suite per mutant and look like a
   fast green.
5. Print whenever a collapse fires — which classes, how many tests it widened, the
   resulting length. A silent widening would make the score certify more than the run
   measured.

One caveat worth documenting alongside it: widening is safe only while the ordinary
suite is green. A test already failing for unrelated reasons counts as a kill.

Result on the case above: 163,246 → 95,614 bytes, four classes collapsed, no test lost.

We run this as a composer-patches patch today and would happily send it as a PR if the
approach looks right. We did not want to open with an unsolicited patch to a file that
already carries a TODO about redesigning this exact thing.

Environment

• pestphp/pest-plugin-mutate v4.0.1, pestphp/pest 4.7.5, PHP 8.4
• Linux amd64 container (fails), macOS arm64 (does not — no per-element cap)

[yeapea]

There is a PR for this now: pestphp/pest-plugin-mutate#43.

Two encodings, applied in order.

Factor the class prefix — Cls::(.*)a|Cls::(.*)b becomes Cls::(.*)(a|b). Lossless, and about a third shorter on a realistic set (52,370 → 34,438 bytes over four classes × 300 tests). The inner parentheses matter: without them the alternation binds to the whole pattern rather than the tail and the filter selects tests it was never given, so there is a test pinning that.

If it still does not fit, collapse a class to its bare Cls:: prefix, heaviest first, stopping the moment it fits. That is a strict superset of the original selection — it can only run more tests, never fewer, so it cannot turn a killed mutant into a survivor. The 2,076-test case from this issue goes from 150,448 bytes to 32.

The collapse is never silent: the run reports which classes widened and by how many tests, once per process. And if even a fully collapsed filter is over budget it throws, because dropping the filter would run the whole suite per mutant and read as a fast green.

Whole suite green (328 tests), pint / phpstan / rector clean, 100% type coverage on the new files. Three behaviours red-proved by reverting the code that guards them.

The algorithm has been running in production in two repositories for several weeks against real mutation runs — as a composer-patches patch in one and an anchored vendor rewrite in the other. Upstreaming it so neither has to re-derive it on every release.

[rappasoft]

Corroborating this on v5.0.2 — same failure, and the two-limit analysis above matches what we measured independently before finding this issue.

Our case is a Laravel service provider: because it executes at application boot, every test in the suite covers it, so the covering set is the suite itself.

• 1,547 covering tests across 93 test classes
• one argv element of 145,136 bytes vs MAX_ARG_STRLEN = 32 × 4096 = 131,072
• verified empirically in the same container: a 131,000-byte argument spawns, 131,072 fails with exactly proc_open(): posix_spawn() failed: Argument list too long

Two additions that might be useful when this gets fixed.

The PCRE ceiling depends on the shape of the pattern, not its length. On PHP 8.4 / PCRE2 10.45, with the same fragment shape the plugin builds:

encoding	bytes	compiles
one alternative per test	33,039	yes
one alternative per test	36,060	no
one alternative per class (Cls::)	33,004	no

The class-collapsed form fails earlier by byte count, because shorter fragments mean more alternation branches per byte. So a fix cannot pick a safe byte budget: whatever it builds has to be compile-checked, or a preg_match size failure will be discovered by the child instead.

The failure is silent, and it scores. NameFilterIterator::accept() calls @preg_match($this->regularExpression, $name) (suppressed), so an uncompilable filter accepts no test at all: PHPUnit prints No tests found. and exits 1. MutationTest::hasFinished() reads a non-zero exit as MutationTestResult::Tested, i.e. a killed mutant. Between roughly 33 KB and 131 KB, then, the run does not crash — it reports mutants as killed by a child process that executed nothing. On a large suite that is a clean 100% that verified nothing, which is a good deal worse than the posix_spawn failure above it.

That makes @yeapea's point 4 (throw rather than drop the filter) the load-bearing one, and worth extending: any filter this code cannot prove usable should raise, never be handed over hopefully.

We are running a composer-patches patch of the same shape — degrade to class level, with usability decided by an actual preg_match of the value as passed (PHPUnit reads the leading " as the pattern delimiter, incidentally), and a throw if even the collapsed form is unusable. Happy to see either encoding land upstream; the prefix factoring in point 1 is strictly better than our collapse and we would switch to it.