[rcore] Process manipulation API for Windows and POSIX-compliant platforms

[konakona418]

I noticed that in the wishlist for raylib7.0 (#5710) there are plans for process manipulation APIs, so I wrote the following functions for posix-compliant platforms:

• InitProcess to launch a new process
• CheckProcess which checks whether a process is still running
• PauseProcess to pause execution
• CloseProcess to terminate a process

And a counterpart for PauseProcess: ResumeProcess

Please let me know if any changes need to be made!

[konakona418]

There have been a few updates, so I will squash them together

[raysan5]

@konakona418 thanks for working on this addition, I think it still requires some work but it's a good start... Windows support should definitely be there...

[konakona418]

Hi @raysan5
Thanks for the quick reply! I'll work on it right away...

[konakona418]

Hello @raysan5

I just implemented the API for Windows except for PauseProcess and ResumeProcess (I'll explain this right away). I tested the functions with MinGW cross compiler and Wine and the functions seem to be working just fine. However I'm not sure if that result will remain the same when it comes to a actual Windows device because sadly, I don't have a Windows laptop around right now...

And about this patch some questions remain:

• I embedded the real struct definitions for calling CreateProcessA inside the InitProcess function, and for declaration of CreateProcessA I used a implicit struct declaration. I hard-coded some Win32 constants and attached their meanings in the comment. I handled this in a way similar to other Win32 imports to prevent polluting the namespace, but I'm not sure if I've done that in a proper manner...
• InitProcess uses a static buffer to concat cmdline arguments, but I believe MAX_PATH * 4 is sufficient for most cases, and there's a bound check which will abort if overflow takes place
• The reason why pause and resume is not implemented yet is that, if we want to achieve that, rather we iterate over all the threads and use API like SuspendThread or dynamically load some ntdll internal functions, and on this topic I really want to ask for your opinion
• I tried to stick to the convention, but please let me know if any persists

Please let me know if more adjustments are required!

[orcmid]

As difficult as it might appear, shouldn't these be handled at the platform level?

Perhaps there is no meaningful POSIX distinction with respect to platforms?

[konakona418]

Hi @orcmid

Currently I followed the existing method applied to a few similar functions in rcore.c, which handles platforms via #if defined(...) macros directly. Regarding POSIX platforms, while there are subtle differences between Linux, macOS and BSDs, the basic APIs used here such as fork(), execvp(), waitpid(), and kill() behave consistently enough for the scope of this process API. If we need more platform-specific or advanced APIs, we will certainly need to introduce more fine-grained distinction.

[konakona418]

Hello @raysan5

I finished working on process suspension and resuming just now, in which i dynamically loaded and used the ntdll internal functions NtSuspendProcess and NtResumeProcess. I also have done some cleanup as well. The implementation appears to be working correctly in my tests.

But still, I'm not sure whether dynamic loading Nt* functions is a good idea for raylib or not. What's your opinion?

[Peter0x44]

SDL3 also has a process management API that might be good for reference

[raysan5]

@konakona418 thank you very much for all the work put on this new feature! It looks great to me but I need some time to review it properly, I also want to check other libraries approaches for reference.

[konakona418]

@raysan5 thanks for the feedback!

Please take your time to review it. If any part needs to be changed, please tell me and I'd be happy to update this PR!

[raysan5]

@konakona418 I've been reviewing other libraries/languages handling of processes, specifically: SDL3, Python, Rust, .NET, Qt and Glib. I also asked AI for some ideas and approaches.

Some concerns about the current API and improvements to be more aligned with raylib style and simplicity:

• raylib usually prioritizes the pass-by-value instead of pass-by-reference, it would be nice if functions could just get Process by value, after all the only needed data is mostly the id and exitCode is only used for exit. It can be returned by other functions.
• int CheckProcessState(Process process) can return state (READY, RUNNING, PAUSED, FINISHED) and/or the exit code (need a bit more thinking)
• I understand current API launches processes asynchronously, that's fine but maybe an int WaitProcess(Process process) blocking functions could be useful, many APIs have that. Still, not sure if it can confuse some users.
• It would be useful some function to get process output data, probably asynchronously, a const char *ReadProcessOutput(Process process) would be useful. Not sure if a sync equivalent version could be needed.
• In that regards, maybe adding a flags parameter to Process InitProcess(const char *command, char *const args[], int flags) could be useful for some process configuration... but not sure if really required and what real use cases it could have (probably most users don't neeed it)

What are your thoughts?

[konakona418]

Hi @raysan5

Thanks for the code review!

• I agree that it's better to pass the Process struct by value, and I will alter the API to accommodate that
• About the return value of CheckProcessState(), maybe we can use a struct containing both exit code and a state? Or let it take a pointer to write exit code to?
• A blocking function can be nice, as long as we make its purpose clear in the docs
• I'll try to work on reading process output
• I'm not sure whether a flag for InitProcess() is needed as well, maybe it can be useful when some users don't want to display a window when launching a process or some other scenarios? Or maybe we can add something like InitProcessEx() later if needed?

[raysan5]

Mmmmh... now Process spawns into two different structs + 1 enum, those are the kind of additions I try to minimize... but let's keep it for now...

[konakona418]

One thing I'm worried about this is that the intention of the naming is kind of unclear...

What's your opinion?

[raysan5]

If this is the new approach, probably a GetProcessInfo() would be more clear

[raysan5]

Could we just return a NULL terminated string so we can avoid length parameter? As per my understanding, it can just fills the internal buffer until next ReadProcessOutput() is called (updating it on the request), right? Or the full output is returned on every call?

[konakona418]

Yes, the current implementation behaves as a stream API, which means the results won't be accumulated. So users need to allocate their own buffer, and copy the content in the returned read-only buffer to their buffer based on length, otherwise previous outputs will be lost.

Other approaches include bool ReadProcessOutput(Process process, void *buf, int bufSize, int *length), which takes a buffer created by user and can prevent extra copy or char *ReadProcessOutput(Process process, int *length) which allocates a buffer dynamically which needs to be MemFree(), but the current implementation just resembles a few text manipulation functions which uses a static buffer as well...

As for the null-terminated string issue, my thought is, for normal ASCII char streams this is sufficient but there are other cases where this approach might cause problems, for example, UTF-8 strings. The pipe read operation basically treats stdout/stderr as a byte stream and does not respect the codepoint boundaries of UTF-8 strings, and inserting NULL terminators may break the strings and cause strange outcomes...

Another case is that users might want to pass raw byte data like large binary objects (though less common), and here's another example. Currently GetClipboardImage() function is not supported on Wayland, and to read clipboard image users can use a utility program called wl-clipboard which contains such features with process API, and read the binary image data directly. In this case, the stream itself may contain NULL bytes which does not stands for the terminator of something (e.g. color black 0 0 0), and adding NULL can also break the struct of the binary object.

[konakona418]

And I'm a bit concerned that maybe static buffer is not a good idea for process API, because this might lead to some confusion easily, for example:

int l1, l2;
TraceLog(LOG_INFO, "p1: %s; p2: %s", ReadProcessOutput(p1, &l1), ReadProcessOutput(p2, &l2));

And this will print identical outputs for p1 and p2 which is read from p2's IO pipe.

What do you think?

[raysan5]

Mmmh... good catch... but otherwise a buffer must be allocated at some point, for example at InitProcess(), inside p1.processData and just fill and return it by ReadProcessOutput() or let the user manage their own buffer with this mentioned limitation (reused static buffer). Actually most Text*() functions deal with same issue and in my experience it has never been a big issue if clearly stated that limitation.

[raysan5]

About the *length, I still prefer to avoid it when dealing/expecting text strings, for library consistency. UTF-8 shouldn't be a problem for NULL terminator.

[raysan5]

I see the need for the ProcessInternal... and it's also platform dependant... it needs a bit more thought... AudioStream also uses and internal struct... 🤔

[raysan5]

@konakona418 Added some further reviews, my comments on mentioned points:

• About the return value of CheckProcessState(), maybe we can use a struct containing both exit code and a state? Or let it take a pointer to write exit code to?

Already reviewed current approach, I think ProcessInfo could be fine...

• I'm not sure whether a flag for InitProcess() is needed as well, maybe it can be useful when some users don't want to display a window when launching a process or some other scenarios? Or maybe we can add something like InitProcessEx() later if needed?

We can avoid that flags parameter for now, probably most users won't need it. In the future a SetProcessFlags() can be added, that's current approach with SetConfigFlags() and SetWindowState(), probably not the best approach but at least consistent with the library.

[Peter0x44]

If they all use a shared buffer, then you can't start multiple processes at once with that API? Sounds like a major limitation.

[konakona418]

@Peter0x44 Actually, you can. That's because the data will only be read to the buffer when ReadProcessOutput() is called. And if you call that function and copy the data to your own buffer right away, it's possible to open several processes and read simultaneously from them.

[orcmid]

if you call that function and copy the data to your own buffer right away, it's possible to open several processes and read simultaneously from them.

Uh, wait, with concurrent processes? And with no semaphore or other guarding? How do you get atomicity between the call of the function and copying of the data?

[konakona418]

It assumes invocations to ReadProcessOutput() and data copying is performed on one certain thread, which is, the main thread. Maybe we can make this clear in the documentation if needed.

By "simultaneously" I meant that multiple processes can be polled sequentially from the same thread, not concurrent access from multiple threads. Sorry if that led to any confusion...