Add opt-in client/server mode to the rdx executable - #869
Conversation
6967765 to
3bf6c65
Compare
49ed15d to
13f37ec
Compare
59e9e4b to
7a54996
Compare
2e6a202 to
bc2a231
Compare
85bbbba to
cc553f6
Compare
f282639 to
173d2e6
Compare
cc69ca7 to
d68b816
Compare
173d2e6 to
5cb5692
Compare
d68b816 to
51b4f05
Compare
5cb5692 to
d319c14
Compare
51b4f05 to
2bb59e0
Compare
d319c14 to
632c830
Compare
2bb59e0 to
1de091c
Compare
632c830 to
519a264
Compare
1de091c to
ae8fcbd
Compare
519a264 to
fbb1b8b
Compare
ae8fcbd to
9e80825
Compare
fbb1b8b to
861101d
Compare
9e80825 to
5fb40e7
Compare
679504a to
b8f407e
Compare
298b5af to
6a0dfca
Compare
6aaf92e to
beda794
Compare
9d24c41 to
1dd0a1f
Compare
`rdx query` pays for indexing and resolving the workspace on every call. This adds a resident server that pays it once, and a `--server` flag that sends the query to it. Without the flag nothing changes, and a platform without `fork` or UNIX sockets keeps running inline. `rdx server start|stop|restart|status` manages the server for a workspace. ## Identity A workspace maps to a runtime directory keyed by an app id, which covers the workspace path, the protocol version, the Ruby version, the rubydex version and a fingerprint of the loaded native extension. A gem upgrade or a protocol change therefore addresses a different directory and forces a fresh server, because a C extension cannot be reloaded in place. The directory holds three files: - `lock` is empty, and no code reads or writes it. The server holds an exclusive lock on it for its whole life, and the file is never deleted. A lock belongs to an inode, so unlinking the path would let a second process lock a new file and start a second server for the same workspace. - `state.json` records the pid, the token, the version and the start time, and a rename replaces it in one step. - `socket` is the UNIX domain socket. The two files stay separate because of Windows: `flock` there is `LockFileEx` over the whole file, and an exclusive lock denies every other process read access to that range. A record inside the locked file would be unreadable exactly while a server runs. The lock answers "does a server own this workspace". The kernel releases it when the holder dies, so a recycled pid cannot look alive. The recorded pid is a display value for a human, and the CLI never signals it. An authenticated request over the socket is the only stop. ## Wire protocol A frame is a decimal length line followed by exactly that many bytes of JSON. The two directions carry opposite risks, so each has its own reader: - The server bounds a whole request with one deadline, because it handles one client at a time and a client that drips bytes would otherwise hold the accept loop. - The client bounds only the gap between chunks, and waits for the first byte without a limit, because its server sends nothing until the query finishes. Reads happen in 64 KiB chunks, since `read_nonblock` allocates its `maxlen` before it reads. The module raises two error types, and a body that is not JSON is one of them, so no caller has to rescue `JSON::ParserError`. ## Failure A resident server belongs to every client, so one bad request must not take it away from the rest. Each connection is rescued on its own: the fault is logged with its backtrace, the client gets status 1, and the accept loop lives on. A server that holds its lock but does not answer is reported rather than waited on. `status` probes it without starting, stopping or restarting anything, and prints the recorded pid and start time so a human can find the process. ## Freshness The graph is a snapshot taken at boot. A file edited afterwards is still answered from the state it had at boot, so `rdx query --server` and `rdx query` disagree about a workspace that changed. Incremental updates for added, changed and deleted files are the follow-up that stacks on this change.
1dd0a1f to
055b037
Compare
| @@ -26,6 +26,14 @@ def start(argv = ARGV) | |||
| require "rubydex" | |||
There was a problem hiding this comment.
Doesn't this defeat the --server fast path? CLI.start still requires rubydex before dispatch, so rdx query --server ... loads the native extension in the client before Query#server_available? runs. That means the client keeps paying the extension load cost and can still fail when the local extension is incompatible, even if a server could answer. I think we need to move the native require into the inline commands/paths (query --schema, query without --server, console/mcp as needed) and keep the top-level dispatcher Ruby-only.
| @ext_fingerprint ||= begin | ||
| lib_dir = File.expand_path("../..", __dir__) | ||
| artifacts = Dir.glob(File.join(lib_dir, "**", "rubydex.{bundle,so}")) + | ||
| Dir.glob(File.join(lib_dir, "librubydex_sys.*")) |
There was a problem hiding this comment.
Looks like this misses the dylib we actually load. lib_dir is .../lib, but extconf copies librubydex_sys.dylib into lib/rubydex/, so this glob is empty. A Rust-only rebuild can leave rubydex.bundle unchanged while changing the dylib, and then app_id/expected_version won't change, so clients can keep talking to a server with stale native code. Can we make this recursive too?
| Dir.glob(File.join(lib_dir, "librubydex_sys.*")) | |
| Dir.glob(File.join(lib_dir, "**", "librubydex_sys.*")) |
Goal
rdx querypays for indexing and resolving the workspace on every call. This adds a resident serverthat pays it once, and a
--serverflag that sends the query to it. Without the flag nothing changes,and a platform without
forkor UNIX sockets keeps running inline.rdx server start|stop|restart|statusmanages the server for a workspace.This pull request is the infrastructure only: the runtime state, the wire protocol, the process
lifecycle and the CLI surface. Keeping the graph in step with a changing workspace is #979, stacked on
this one.
Identity
A workspace maps to a runtime directory keyed by an app id, covering the workspace path, the protocol
version, the Ruby version, the rubydex version and a fingerprint of the loaded native extension. A gem
upgrade or a protocol change therefore addresses a different directory and forces a fresh server,
because a C extension cannot be reloaded in place.
The directory holds three files:
lockstate.jsonsocketThe two files stay separate because of Windows:
flockthere isLockFileExover the whole file, andan exclusive lock denies every other process read access to that range, so a record inside the locked
file would be unreadable exactly while a server runs. The lock file is never unlinked either, because
a lock belongs to an inode and unlinking the path would let a second process lock a new file and start
a second server for the same workspace.
The lock answers "does a server own this workspace". The kernel releases it when the holder dies,
so a recycled pid cannot look alive. The recorded pid is a display value for a human, and the CLI never
signals it. An authenticated request over the socket is the only stop.
Wire protocol
A frame is a decimal length line followed by exactly that many bytes of JSON. The two directions carry
opposite risks, so each has its own reader:
client that drips bytes would otherwise hold the accept loop.
its server sends nothing until the query finishes.
Reads happen in 64 KiB chunks, since
read_nonblockallocates itsmaxlenbefore it reads: a peerdeclaring 300 MiB and sending one byte cost 300 MiB of resident memory before that change. The module
raises two error types, and a body that is not JSON is one of them, so no caller has to rescue
JSON::ParserError.Failure
A resident server belongs to every client, so one bad request must not take it away from the rest. Each
connection is rescued on its own: the fault is logged with its backtrace, the client gets status 1, and
the accept loop lives on.
A server that holds its lock but does not answer is reported rather than waited on.
statusprobes itwithout starting, stopping or restarting anything, and prints the recorded pid and start time so a human
can find the process.
status,stopandrestartall exit 1 in that case instead of hanging.The report claims no more than what the probe saw.
servetakes one request at a time, so a serverworking on a long query is indistinguishable from one that is stuck, and the message says so rather
than telling the reader to kill a process that may be doing exactly what they asked.
Freshness
The graph is a snapshot taken at boot. A file edited afterwards is still answered from the state it had
at boot, so
rdx query --serverandrdx querydisagree about a workspace that changed. #979 fixesthat, and it is stacked on this branch.
Tests
test/server/covers the framing, the runtime state, the client lifecycle and the error handling;test/server/integration_test.rbdrives the real executable as a subprocess. Every test was checkedagainst the behaviour it guards.