[Feat] Add a testing guide sec for C functions

[wenzeslaus]

Add a testing guide for tests which call/test C functions through ctypes or test their wrappers (e.g., grass.pygrass). Raising the issue and pointing to an example is the minimal addition.

However, this may, or rather should, trigger a review of current techniques and compare those to the preferred practice, whatever that is. I tried couple of approaches, although I think I had to clean up some of that because only basic subprocess and code in a string actually worked reliably.

Sadly for pytest, grass.gunittest framework does handle these fairly well since a test runs in a subprocess in an exiting GRASS session which has the environment set up and it robust to crashes and exits since a test is a subprocess (if I recall correctly, this was indeed intentional design for grass.gunittest and one of the reasons to have a customized test framework, so no surprise this works out of the box there). Other than that, pytest is a better match for tests of individual, small functions.

One case the guide does not cover, and it bit me on #7835: a test that calls into C through ctypes should make those calls in a subprocess.

My regression test there called Vect_cat_list_to_array() through ctypes in the pytest process. The bug under test was a null dereference, so on Windows the call died — and because a segfault ends the process, pytest went with it. The Windows suite stopped at 8% with Process completed with exit code 1 and no traceback at all, since pytest's captured output dies with it. @echoix restarted the run twice before I worked out that it was mine and not flakiness.

Running the ctypes calls in a subprocess (subprocess.run([sys.executable, "-c", SCRIPT], env=env, ...) with the env from the standard session fixture, asserting on the return code) turns the same crash into return code -11 in one ordinary test failure, and the rest of the suite survives. That is better even when the test passes, because a suite-killer becomes something you can diagnose.

Worth a short subsection if you agree — the failure mode gives you nothing to go on, so it is hard to work out from first principles. Happy to draft the wording.

Originally posted by @Pranav-error in #7851 (comment)

[Pranav-error]

Thanks for opening it — I'd like to take this one.

What I'd write, based on what actually worked on #7835:

• Why a subprocess, not just how. An in-process ctypes call that aborts takes the interpreter with it, so pytest dies and you get an exit code with no traceback and no captured output. On lib/vector: Fix null pointer dereference for an empty cat_list #7835 that stopped the Windows suite at 8% and looked like flakiness — @echoix restarted it twice before I worked out it was mine. Framed as "one failing test becomes nobody's tests run", the rule justifies itself.
• The pattern: subprocess.run([sys.executable, "-c", SCRIPT], env=env, capture_output=True), with env from the ordinary session fixture, asserting on the return code and putting stdout+stderr in the assertion message. A segfault then reads as return code -11 in a normal failure.
• That it is worth doing even when the test passes — it converts an opaque suite-killer into something diagnosable.
• Your point about gunittest, which I think belongs in the section rather than as a footnote: it already runs each test as a subprocess in a live session, so this is a pytest-specific concern, and it is one of the few places where gunittest is the more robust of the two.

I agree the wider review is the bigger part, and I have not tried enough approaches to have an opinion on which is preferred — I only know that subprocess plus code in a string worked, which matches what you found. I would rather write the narrow version first (rule, example, gunittest note) and leave the comparison to someone who has tried more of them, unless you would rather it wait for the full review.

One practical note so the timing does not look like silence: I am at the repository's open pull request limit with #7835, #7837, #7840, #7844 and #7847, so I will open this once one of them merges or closes. Happy to have the text ready before then.

[Pranav-error]

Drafted it: Pranav-error:docs-testing-guide-c-functions (one commit, doc/development/testing.md only).

It adds a ### Tests which call C functions section after "Tests which run in parallel", covering:

• why the process matters, not just the mechanics — a ctypes call runs in the pytest process, so an abort ends the run with an exit code and no traceback, which reads as an infrastructure problem rather than a failing test;
• that in-process calls are fine when they cannot abort, pointing at lib/vector/rtree/tests/lib_vector_rtree_ctypes_test.py as the case where that holds;
• the subprocess pattern, using session.env as lib/raster/tests/lib_raster_reclass_test.py already does, and asserting on the return code with the output included;
• your point about grass.gunittest already running each test file as a subprocess, so it is the more robust choice for C code expected to crash while pytest suits small individual functions.

I have deliberately kept it to the narrow version and left the wider review of techniques out, as I said above — I have not tried enough approaches to have a view worth writing down.

I still cannot open the pull request: I am at the repository's five open pull request limit (#7835, #7837, #7840, #7844, #7847). The branch is pushed and ready, so it is one click whenever one of those merges or closes. If you would rather not wait, feel free to take the branch or the text into your own PR — I would rather the section existed than that it had my name on it.