Snapshot testing

[OliverJAsh]

We are using fast-check alongside snapshot testing: the data we generate with Arbitrarys and a stable seed ends up being stored in snapshots.

When a new property is added to the Arbitrary, it seems to change the random data for all other properties. For example:

import * as fc from 'fast-check';

const arb = fc.array(
  fc.record({
    age: fc.integer({ min: 0, max: 20 }),
    name: fc.string(),
  }),
  { minLength: 5, maxLength: 5 },
);

console.log(
  fc.sample(arb, {
    seed: 0,
    numRuns: 1,
  })[0],
);

This outputs the following:

[ { age: 19 }, { age: 2 }, { age: 5 }, { age: 6 }, { age: 19 } ]

Now let's say added new property, name: fc.string():

 const arb = fc.array(
   fc.record({
     age: fc.integer({ min: 0, max: 20 }),
+    name: fc.string(),
   }),
   { minLength: 5, maxLength: 5 },
 );

If we run this again, we can see that age is now producing different values:

[
  { age: 19, name: '8][#zL8y' },
  { age: 1, name: `'oR #g|"` },
  { age: 1, name: '?"callc' },
  { age: 6, name: 'l|' },
  { age: 16, name: "IN!%'jX" }
]

As a result of this, our snapshots need updating very frequently. This can be a huge pain.

Is there any way to make the Arbitrary stable so that adding a new property doesn't affect other properties? Or perhaps you would recommend that we shouldn't be using fast-check in this way?

[dubzzz]

So far, there is no way to stabilize it. Each entry added to the record stored in the array will offset the next record "by design". Designed to be efficient for generation cases.