Add: verify remote startup failure cleanup

[ChaoWao]

Summary

• Add the failed remote worker ID to open and attach errors while preserving
  TimeoutError classification when the startup budget is exhausted.
• Add deterministic failure injection for remote-open timeout,
  later-remote activation failure, and native endpoint-attach failure.
• Exercise real L4, local SUB, and remote L3 process trees, then assert
  exact child PIDs, remote process groups, shared-memory names, sessions,
  and native resources are reclaimed.
• Verify the same remote daemon accepts a healthy session after rollback.

Testing

• .venv/bin/python -m pytest -q tests/ut/py/test_worker/test_remote_zero_residual.py tests/ut/py/test_worker/test_remote_startup_budget.py tests/ut/py/test_remote_l3_lifecycle.py tests/ut/py/test_callable_identity.py
  • 178 passed after rebasing onto latest upstream/main
• .venv/bin/pre-commit run --files python/simpler/worker.py tests/ut/py/test_worker/test_remote_zero_residual.py tests/ut/py/test_worker/_remote_zero_residual_target.py
  • Passed

[coderabbitai]

📝 Walkthrough

Walkthrough

Remote L3 activation now adds contextual errors for session opening and endpoint attachment failures. Linux acceptance tests verify complete process, SHM, session, and worker cleanup after several failure paths and confirm later daemon reuse.

Changes

Remote session rollback

Layer / File(s)	Summary
Contextual remote activation errors python/simpler/worker.py	Session opening and endpoint attachment failures now raise RuntimeError values that identify the remote worker and preserve the original exception details.
Remote cleanup test harness tests/ut/py/test_worker/_remote_zero_residual_target.py, tests/ut/py/test_worker/test_remote_zero_residual.py	Added a delayed no-op remote target and helpers that inspect process trees, process groups, SHM mappings, sessions, and worker state.
Rollback and daemon reuse scenarios tests/ut/py/test_worker/test_remote_zero_residual.py	Added Linux tests for open timeout, later-session failure, and attachment failure. Each test checks resource reclamation and subsequent daemon reuse.

Estimated code review effort: 3 (Moderate) | ~30 minutes

Possibly related PRs

• hw-native-sys/simpler#1398: Both changes cover remote worker startup, teardown, and failure cleanup.
• hw-native-sys/simpler#1432: Both changes extend remote L3 activation and rollback handling in worker.py.
• hw-native-sys/simpler#1420: Both changes modify remote L3 session activation and startup handling.

Poem

A rabbit watched the workers run,
Then checked each process, every one.
When sessions failed, the paths grew clear,
No stray SHM remained near.
The daemon hopped back into play. 🐇

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly summarizes the added verification for remote startup failure cleanup.
Description check	✅ Passed	The description directly explains the failure-injection tests, resource cleanup checks, error handling, and validation results.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/ut/py/test_worker/test_remote_zero_residual.py`:
- Around line 212-214: Update the cleanup in the test’s finally block after
thread.join(timeout=1.0) to assert that the observer thread is no longer alive
before returning. Use the existing thread object and preserve the stop signal
and timed join order.
- Around line 228-230: Update the assertions before
observed.assert_reclaimed(worker) in the test to require observed.shm_names to
be non-empty, ensuring the failure path captured at least one SHM segment before
cleanup.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 99b60ece-de26-40f2-8288-e0a5c1abbf48

📥 Commits

Reviewing files that changed from the base of the PR and between 06fbd42 and 6f032e1.

📒 Files selected for processing (3)

• python/simpler/worker.py
• tests/ut/py/test_worker/_remote_zero_residual_target.py
• tests/ut/py/test_worker/test_remote_zero_residual.py