Skip to content

(WIP) use rstest to parameterize test cases#3100

Closed
neunenak wants to merge 3 commits intocasey:masterfrom
neunenak:test-reductions
Closed

(WIP) use rstest to parameterize test cases#3100
neunenak wants to merge 3 commits intocasey:masterfrom
neunenak:test-reductions

Conversation

@neunenak
Copy link
Contributor

Introduce rstest as a dev-dependency, use it to parameterize a number of near-identical test cases in tests/functions.rs

@neunenak
Copy link
Contributor Author

@casey curious what you think of this - I noticed in the process of working on #2426 that just has a lot of test cases and a lot of them are pretty repetitive. I've used rstest before to get test parameterization, I kinda like it, so I asked claude to come up with a .md doc for places where the tests are particularly repetitive and had it take a stab at parameterizing some of the ones in tests/functions.rs, just because that was the easiest to do. I don't think it makes sense to merge this until #2426 gets merged since they would be affecting some of the same tests.

@casey
Copy link
Owner

casey commented Mar 4, 2026

I'm pretty ambivalent about this. I think the rstest code is pretty hard to read. It would be far better to just somehow generate sub-tests inside an existing tests, but rust doesn't support that. As it is, I'd probably not merge this. The savings seem okay, but it introduces a new dependency and makes the tests weird. If a whole lot more deduplication was possible, I'd be more inclined to add it.

@casey casey closed this Mar 4, 2026
@neunenak
Copy link
Contributor Author

neunenak commented Mar 5, 2026

I'm pretty ambivalent about this. I think the rstest code is pretty hard to read. It would be far better to just somehow generate sub-tests inside an existing tests, but rust doesn't support that. As it is, I'd probably not merge this. The savings seem okay, but it introduces a new dependency and makes the tests weird. If a whole lot more deduplication was possible, I'd be more inclined to add it.

I think there's more opportunity for deduplication than the relatively limited amount that's happening here for the sake of a smaller PR, but yeah I can see how rstest introduces a new dependency and makes the actual test methods that get generated kind of hard to read.

Maybe there's a way to extend the existing Test formalism to get some deduplication. The Test builder syntax could accept something pairs of input and expected output, store them internally, and then loop over them in the body of a single test function. Test::status could keep track of what case is currently executing, save test failure conditions internally, and then emit a more detailed panic! describing every test case that failed. So this would end up looking like, a bunch of existing rust test functions getting folded into a smaller number of rust test functions that test for more things.

Another idea is to modify the Test formalism a bit to have Test::success emit some data structure that is basically a vec of the existing Output type, which would represent all the test cases generated by that particular Test instance. Then instead of having Test::new() get called within a named rust test function, wrap it in a simple macro that extracts the cases from the Output and generates named test methods that way.

@casey
Copy link
Owner

casey commented Mar 5, 2026

Extending the Test builder to get some deduplication would be interesting. I have no idea how it would work, but maybe something like a test matrix?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants