Skip to content

feat: async tasks — background PHP work with UI completion callbacks - #228

Open
simonhamp wants to merge 13 commits into
mainfrom
component-async-api
Open

feat: async tasks — background PHP work with UI completion callbacks#228
simonhamp wants to merge 13 commits into
mainfrom
component-async-api

Conversation

@simonhamp

@simonhamp simonhamp commented Jul 24, 2026

Copy link
Copy Markdown
Member

What

Adds a way to run PHP work on a background thread and handle the result back on the UI thread, where the callback can update component state and the screen re-renders:

public function generateReport(): void
{
    $this->generating = true;   // paints immediately — spinner shows

    AsyncTask::dispatch(static function () {
        return ExpensiveReport::build()->toArray();   // separate PHP interpreter
    })->finished(function (array $report) {
        $this->report = $report;                      // back on the UI thread
        $this->generating = false;
    })->failed(fn (\Throwable $e) => $this->error = $e->getMessage());
}

There's a $this->async(...) shorthand on NativeComponent, and an AsyncTask subclass form (BuildReport::dispatch($month)->finished(...)) for anything you'd rather not write inline.

Why this is mostly glue

Both halves already existed and just weren't connected:

  • A real background PHP thread — the queue-worker, ephemeral and per-webview lanes already boot independent interpreters on their own threads with their own TSRM contexts.
  • A completion channel into the parked runloop — the fluent-callback machinery built for Camera (NativeCallbacks + fireNativeCallback + sendNativeEventnphp_element_post_event) already binds a callback to the live component and re-renders. That is ->finished().

What was missing was a way for a background context to say "done" back to the UI runloop, plus the API on top. This adds a dedicated concurrent async lane and wires it through.

Design decisions

  • Static closures enforced. The work runs in a different interpreter, so it can't carry $this. A bound closure is rejected by reflection at dispatch time, in the handler where you can see it — not silently in a background log.
  • Screen-scoped callbacks. finished()/failed() are dropped if the originating screen is no longer topmost — firing a component-bound closure against the wrong live component is a footgun. The origin is held as a WeakReference, since spl_object_ids are recycled and an id comparison could hand a callback to an unrelated screen. ->shared('alias') opts out, delivering a named event any active screen handles via #[On('alias')].
  • Every dispatch reaches exactly one outcome. A spinner that never stops is the worst failure this API can have, so the paths that delivered nothing are all closed: a refused dispatch fails at the call site (AsyncTask.Dispatch's success flag is checked), a non-encodable result fails as an ordinary task failure (the JSON round trip happens in the background context, where it can still be reported), and a hung task times out (default 60s, ->timeout($seconds), 0 to disable).
  • failed() handler, receiving an AsyncTaskException that carries the original message and class. One attempt, no retries — silent re-runs of a button press are surprising.
  • Immediate and concurrent, on its own lane. Not the queue worker (a ≤3s queue:work poll is too slow for interactive work), no database, no standard queue. A bounded pool of reusable PHP contexts; queueing beyond the pool is logged rather than silent.
  • Payload crosses via a temp file, so native stays a dumb courier with just two bridge functions: AsyncTask.Dispatch and AsyncTask.Complete. The timeout travels as a deadline plus the pre-built completion to post if it passes, so even the watchdog doesn't teach native what an event looks like.
  • dispatch() terminology for familiarity, but this is explicitly not a queued job.

Full rationale in docs/async-task-design.md.

Alternative considered

Expose a nativephp_post_event() PHP function in the extension — cleaner and zero native hop, since the background context would post the wake event itself instead of routing through AsyncTask.Complete. Not taken here because the nativephp extension ships prebuilt, so a new extension function can't land from this repo. Worth revisiting when the extension is next rebuilt.

Changes

PHPAsyncTask, PendingAsyncTask, AsyncTaskRunner/Transport/Registry, AsyncTaskFinished/AsyncTaskFailed events, AsyncTaskException, native:async:run command, and async-completion handling in NativeComponent.

Native — an async_php_* lane in both php_bridge.c (Android) and PHP.c/PHP.h (iOS), plus AsyncTaskExecutor and AsyncTask.* bridge functions in Kotlin and Swift, wired into app boot and hot-reload runtime reboots. Both executors stop synchronously — draining in-flight tasks and shutting their contexts down before returning — because callers stop the pool immediately before a runtime reboot that frees Zend state those contexts reference. The lane avoids setenv entirely: it runs several contexts at once, and setenv/getenv are neither thread-safe nor per-thread, so the per-thread $_SERVER entries carry the console environment instead.

Jump — dispatches to a dev-machine subprocess and drains a completion spool into the runloop, since there's no device async lane on a laptop. It also sweeps overdue subprocesses each tick, standing in for the native watchdog.

TestingAsyncTask::fake() runs work inline-synchronously with assertDispatched() / assertNotDispatched() / assertDispatchedTimes() / assertShared(), and applies the same JSON normalization a device does, so a test can't pass on a value that wouldn't survive the real hop.

Testing

  • New Pest tests covering the weak screen scoping, the result round trip, the timeout contract and the encode fallback, on top of the original 18. PHPStan clean; Pint clean.
  • Verified end-to-end against a demo screen in super-native (dispatch → background run → callback → state update → re-render), including the failure path, the static-closure guard, shared() delivery and the screen-scope drop.

⚠️ The native executors have not been compiled or run on a device yet — the C, Kotlin and Swift lanes need a real build (native:run, not hot reload) to shake out build errors. That's still the main thing to check before this leaves draft.

Docs for this land separately in the nativephp.com repo as digging-deeper/async-tasks.

🤖 Generated with Claude Code

Adds AsyncTask::dispatch(static fn () => ...) (and a $this->async() shorthand
on NativeComponent) to run work on a background PHP thread and handle the
result back on the UI thread, where callbacks can mutate component state.

Builds on two things that already existed: the background-interpreter lanes
(worker/ephemeral/webview) and the native event channel that already delivers
camera results into a parked runloop. This wires them together and adds a
dedicated, concurrent async lane.

- Work closure must be static; a $this-bound closure is rejected by reflection
  at dispatch time rather than failing silently in a background log.
- finished()/failed() are rebound to the live component. They are screen-scoped
  and dropped if the user navigated away; shared('alias') opts out by delivering
  a named event any active screen can handle via #[On].
- Runs immediately and concurrently on its own pool of PHP contexts — not the
  queue worker, no SQLite, no queue, one attempt, no retries.
- Payload crosses via a temp file, so native stays a courier with two bridge
  functions: AsyncTask.Dispatch and AsyncTask.Complete.
- Jump runs tasks in a dev-machine subprocess and drains a completion spool.
- AsyncTask::fake() runs work inline for tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@simonhamp

Copy link
Copy Markdown
Member Author

…files

CI's Code Style job flagged 8 files. Each contained a fully-qualified class
reference where the class was imported or in the same namespace, which is
exactly what the fully_qualified_strict_types fixer rewrites.

- Docblock FQCNs replaced with imported short names.
- Inline FQCNs in tests replaced with imports (AsyncTask::clearFake(),
  SerializableClosure).
- NativeComponent: dropped the `use Closure;` added for async(). That import
  retroactively made the file's ten pre-existing `\Closure` usages shortenable,
  so the signature now uses `\Closure` to match the file's own convention —
  a two-line diff instead of churning unrelated lines.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@simonhamp
simonhamp marked this pull request as ready for review July 25, 2026 12:49
@shanerbaner82

Copy link
Copy Markdown
Contributor

Had a proper read through this — the shape of it is right. Framing it as glue between the existing TSRM lanes and the Camera-style callback channel is accurate, and the PHP layer is in good shape. The design doc is genuinely useful too.

The native side is where the work still is, roughly in line with the ⚠️ in the description:

  • Hot reload can crash. AsyncTaskExecutor.stop() doesn't wait for in-flight tasks on Android the way PHPQueueWorker.stop() does, so shutdownPersistentRuntime() can tear PHP down underneath a live context. On iOS stop() bails early if the pool is still booting, which both skips the cleanup and leaks the C slots — after that every dispatch is silently dropped for the life of the app.
  • Concurrency vs. setenv. Four pool threads flipping APP_RUNNING_IN_CONSOLE on and off clobber each other (and setenv/getenv aren't thread-safe). The eval'd code already sets $_SERVER, so the C setenv calls can probably just go.
  • Failures are invisible. The AsyncTask.Dispatch result is discarded, and a non-encodable result makes json_encode return false — either way nothing completes, ->failed() never fires, and the spinner spins forever. Same story for a hung task, since there's no timeout.

Smaller ones worth grabbing whenever this gets picked back up: screen scoping via spl_object_id is exposed to id reuse after a screen is popped (a WeakReference would close it), $nativeActiveComponent is never cleared so it retains a dead component, AsyncTask::fake() doesn't JSON round-trip the result so tests will pass where a device wouldn't, starting the task from __destruct() can throw, and the "never touches SQLite" claim isn't quite true — NativeCallbacks::register() writes to the cache.

Given the amount of native surface here, and that the C/Kotlin/Swift lanes still haven't been compiled or run on a device, we're going to hold this until v4 is out rather than land it late in the cycle. Nothing wrong with the direction — it just deserves a release where the native lanes can get real device time and the extension question (a userland nativephp_post_event(), per the alternative you noted) can be revisited properly.

Leaving it open. Thanks @simonhamp — good writeup.

@shanerbaner82 shanerbaner82 added enhancement New feature or request needs testing Manual testing needed labels Jul 26, 2026
claude added 2 commits July 26, 2026 09:39
Review feedback on #228. The PHP layer was sound; the gaps were in the
native lanes and in the failure paths that silently delivered nothing —
the worst outcome this API can have, since the UI is sitting on a spinner
waiting for a callback that never arrives.

Hot reload / shutdown safety:

- Android `AsyncTaskExecutor.stop()` now joins its pool (and the new
  watchdog thread) before returning, like `PHPQueueWorker.stop()`, and
  reports whether it drained. Callers stop it immediately before
  `shutdownPersistentRuntime()`, which frees Zend state a live async
  context still references — returning early tore PHP down underneath a
  running task.
- iOS `stop()` no longer bails when the pool is mid-boot. It signals the
  boot loop to stop, waits it out, then tears down every slot it finds.
  Bailing left the C slots allocated with `start()` refusing to run twice,
  so every later dispatch was dropped for the life of the app.

Concurrency:

- Dropped the `setenv("APP_RUNNING_IN_CONSOLE"/"PHP_SELF")` calls from the
  async lane in both `php_bridge.c` and `PHP.c`. Four pool threads flipping
  a process-wide, non-thread-safe env var clobber each other and the UI
  lane. The eval'd code sets the per-thread `$_SERVER` entries instead —
  which is what Laravel's `Env` reads — including before the bootstrap runs.

Every dispatch now reaches exactly one outcome:

- `AsyncTask.Dispatch`'s `success` flag is checked. No executor, no slot,
  no bridge, or a Jump subprocess that wouldn't launch fails the dispatch
  at the call site and fires `->failed()`.
- The result is JSON round-tripped in the background context
  (`AsyncTaskRunner::normalizeResult()`), where a non-encodable value can
  still be reported as an ordinary task failure. `encodeCompletion()`
  backstops the envelope itself.
- Every dispatch carries a deadline (default 60s, `->timeout($seconds)`,
  0 to disable) plus the pre-built completion to post if it passes, so the
  native watchdogs stay dumb couriers. Jump sweeps overdue subprocesses on
  each runloop tick instead. The work isn't killed — the timeout unblocks
  the UI, and a late completion for an already-failed task is discarded.

Smaller ones from the same review:

- Screen scoping holds the origin as a `WeakReference`, closing the
  `spl_object_id` reuse hole where a popped screen's id could be handed to
  the next component and deliver its callback to an unrelated screen.
- `$nativeActiveComponent` is weak and restored when a runloop exits, so it
  stops retaining a dead component.
- `AsyncTask::fake()` runs the same JSON normalization as the device, so a
  test can't pass on a value a device would never deliver.
- `__destruct()` can no longer throw; a start failure reports through the
  task's own `failed()` channel and the error log.
- Async callbacks register with `durable: false`, so the "never touches
  SQLite" claim is now true — `NativeCallbacks::register()` was writing a
  cache copy that had nothing to survive to anyway.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TFbiJwd86MZCTLH5kJHZxo
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TFbiJwd86MZCTLH5kJHZxo

Copy link
Copy Markdown
Member Author

Thanks @shanerbaner82 — that's a good read of it, and all of it landed. Pushed to component-async-api.

Happy to hold for v4; the native lanes still need real device time either way. But the three big ones were all latent crashes/hangs rather than polish, so they're fixed now rather than waiting to be re-derived later.

Hot reload / shutdown

  • Android stop() now joins the pool (and the new watchdog thread) before returning and reports whether it drained, like PHPQueueWorker.stop(). Callers stop it immediately before shutdownPersistentRuntime(), so returning early was tearing PHP down underneath a live context. Budget is 5s, matching the queue worker — long enough for a task to land, short enough that onDestroy() can't hang the app. A failed drain logs loudly at the call site.
  • iOS stop() no longer bails mid-boot. It signals the boot loop to stop adding slots, waits it out, then tears down every slot it finds. You were right about the consequence: bailing left the C slots allocated with start() refusing to run twice, so every later dispatch was dropped for good. Tasks dispatched during boot are now held and flushed when the pool comes up, rather than dropped.

Concurrency vs. setenv

Gone from the async lane in both php_bridge.c and PHP.c — your read was right, the eval'd $_SERVER is what Laravel's Env reads anyway. The bootstrap needed the same treatment (it runs before any eval), so there's now a small $_SERVER eval ahead of php_execute_script instead of the setenv pair. Per-thread superglobal, no shared state.

Failures are invisible

Reframed as: every dispatch reaches exactly one outcome. Three paths could deliver nothing, and all three are closed:

  • AsyncTask.Dispatch's success flag is now checked. No executor, no free slot, no bridge, or a Jump subprocess that wouldn't launch fails the dispatch at the call site and fires ->failed() immediately.
  • The result is JSON round-tripped in the background context (AsyncTaskRunner::normalizeResult()), where a non-encodable value can still be reported as an ordinary task failure. encodeCompletion() backstops the envelope itself (invalid UTF-8 in a message or trace).
  • Every dispatch carries a deadline — default 60s, ->timeout($seconds), 0 to disable — plus the pre-built completion to post if it passes, so native holds both and stays the dumb courier rather than learning how a failure event is shaped. Jump has no native watchdog, so the transport sweeps overdue subprocesses on each runloop tick. The work isn't killed on device (can't interrupt an interpreter mid-task safely); the timeout unblocks the UI, and a late completion for an already-failed task is discarded.

The smaller ones

  • Screen scoping holds the origin as a WeakReference — closes the id-reuse hole exactly as you described.
  • $nativeActiveComponent is weak and restored when a runloop exits, so a nested/hot-swapped loop hands the baton back instead of leaving a dead screen pinned.
  • AsyncTask::fake() runs the same normalization as the device, so an object result now arrives as an array in tests too, and a non-encodable result fails the task in a test exactly as it would on a phone.
  • __destruct() can't throw any more: a start failure reports through the task's own failed() channel and the error log. start() called explicitly still throws.
  • The SQLite claim is now true rather than corrected — async callbacks register with durable: false. The tier-2 copy was a cache write with nothing to survive to (the payload and scope metadata are RAM/temp-file bound, so a process kill loses the task regardless), so dropping it also takes a SQLite write off the UI thread per dispatch.

Design doc updated throughout; new Pest coverage for the weak scoping, the JSON round trip, the timeout contract and the encode fallback. The ⚠️ stands — none of the native lanes have been compiled or run on a device, and that's still what this needs most before it goes anywhere.


Generated by Claude Code

claude and others added 3 commits July 26, 2026 09:55
Three follow-ups from review.

drainJumpCompletion() sorted spool files by name, but the names are random
UUIDs — so "oldest first" was actually arbitrary order. Sort by filemtime
with the name as a tiebreak, since mtime is one-second granular on some
filesystems and two tasks can finish inside the same second.

Spool files are written 0600 in a 0700 directory (and a directory left
world-readable by an earlier run is tightened). The payload goes straight
to unserialize() in the runner: a closure is signed with the app key, but
a task subclass's constructor arguments are not.

forTask() rejects a subclass with no handle() at dispatch time, matching
the static-closure guard beside it — otherwise the mistake surfaces as a
generic ->failed() from a background thread reading "Call to undefined
method".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AMnYeU9ZEGk2XR4FpciYRD
On device, AsyncTask.Complete posts a native event that unblocks
nativephp_element_wait_event(). Under Jump there is no such wake path —
the completion spool is just a directory the polyfill checks on entry.

nextEventTimeout() returns -1 (block until the user does something) for
any screen without a #[Poll], which is most of them. So the polyfill
checked an empty spool, blocked indefinitely, and the completion written
a moment later sat there undelivered. From the app it looks like a task
that never finishes; dispatch several and they all land at once the
moment an unrelated tap happens to wake the loop.

While Jump runners are in flight (or a completion is spooled but not yet
drained), clamp the wait to 150ms so the loop comes back and looks. Costs
nothing when no async task is pending — hasPendingJumpRunners() is false
and the timeout is passed through untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AMnYeU9ZEGk2XR4FpciYRD
@shanerbaner82

Copy link
Copy Markdown
Contributor

@simonhamp — took the updated branch for a run on Jump, the iOS simulator and an Android emulator. Most of it holds up well, but § Hot reload found a reproducible SIGABRT on Android that I think blocks merge. Details below, plus three small fixes I've pushed to the branch.

First, credit where it's due: the rework closes almost everything from the last round. WeakReference for the origin scope and the active component, the durable: false callbacks, normalizeResult() shared between the runner and the fake, the dispatch-refusal path, the watchdog, and — the one I'd have bet against being clean — every setenv() gone from the async lane in favour of per-thread $_SERVER. That last one is exactly right: Env reads $_SERVER via ServerConstAdapter, so it works, and it's per-TSRM rather than process-global.

The Android crash

Reproduced by dispatching a 20s task and then triggering hot reload. From logcat:

I AsyncTaskExecutor: Stopping async executor...
E AsyncTaskExecutor: nativephp-async-3 still running after 5000ms — its PHP context is still live
I AsyncTaskExecutor: Async executor stop TIMED OUT

then the app dies. The caller ignores the answer it just asked for:

if (asyncExecutor?.stop() == false) {
    Log.e("HotReload", "Async pool did not drain before runtime reboot")
}

phpBridge.shutdownPersistentRuntime()   // ← proceeds regardless

STOP_TIMEOUT_MS is 5s, and a thread parked in zend_eval_string can't be interrupted — as the comment in stop() says itself. So any task outliving the deadline leaves a live TSRM context, and php_module_shutdown then frees the Zend state underneath it.

The join and the loud log are a genuine improvement — it's gone from a silent race to a detected one. It just isn't prevented yet.

The structural problem is that the stop deadline (5s) and the task timeout (60s default) aren't coupled, so "task longer than 5s" silently means "hot reload is unsafe". Options as I see them:

  1. Don't reboot when the pool won't drain — skip that hot-reload cycle and retry when it does.
  2. Defer the reboot: re-post it until stop() returns true, with a cap. Better UX, a bit more machinery.
  3. Raise the deadline — narrows the window without closing it.

I'd lean 2 falling back to 1, logging plainly that hot reload is waiting on a task rather than dying. Happy to implement whichever you prefer — I have the emulator set up and can verify it — but it changes hot-reload behaviour, so it's your call.

Probably the same root cause: dispatching 6 concurrent tasks after a timed-out stop gave 4 completions and 2 that never returned, with only 3 distinct interpreter contexts where the pool is 4, showing run #2/run #3. That reads as orphaned threads keeping their thread-local contexts and the next pool coming up degraded.

What passed

  • iOS simulator, concurrency: 4 completions at ~1.0s, 2 at ~2.1s. Exactly pool=4 with queueing, out of dispatch order. The whole round trip works.
  • Jump: every section green — dispatch, failure with the original exception class, timeout, shared alias, both dispatch guards, subclass form.

Three fixes pushed (0e3f608, 57b1331)

1. Jump tasks never completed. nextEventTimeout() returns -1 for any screen without a #[Poll], so the polyfill checked the spool once and then blocked forever. On device AsyncTask.Complete posts an event that unblocks the wait; under Jump the spool is passive, so completions sat undelivered until an unrelated tap woke the loop — dispatch several and they all land at once. The wait is now clamped to 150ms while runners are in flight, passed through untouched otherwise.

2. drainJumpCompletion() ordering. sort() on random UUID filenames isn't "oldest first" — now filemtime with a name tiebreak.

3. Spool permissions + a handle() guard. Spool is 0700/0600 (the payload goes straight to unserialize(), and a subclass's constructor args aren't signed the way a closure is), and forTask() rejects a subclass with no handle() at dispatch, matching the static-closure guard beside it.

722 tests, Pint clean, each with a regression test.

Also worth noting for anyone testing this: the native lanes only reach the build via native:installnative:run alone recompiles the existing nativephp/ios|android scaffold and will silently keep using the old Swift/Kotlin. Cost me two builds before I spotted it.

🤖 Generated with Claude Code

@shanerbaner82 shanerbaner82 added the bug Something isn't working label Jul 26, 2026
@shanerbaner82

Copy link
Copy Markdown
Contributor

Native → PHP event channel coalesces concurrent completions

Found while testing this on Android (emulator). The async lane in this PR is correct — the loss is one layer down, in the shared C event channel.

nphp_element_post_event() (build-scriptsshared/nativephp/nphp_element.c:814) is a single slot, not a queue:

uint8_t *p = r->event_heap;        /* always writes at offset 0 */
...
atomic_store(&r->event_size, (uint32_t)pos);
atomic_store(&r->event_count, 1);  /* a flag, never incremented */
pthread_cond_signal(&r->event_cond);

nphp_element_wait_event() reads that one frame and clears the flag. Any second post that lands before PHP drains overwrites the first, silently.

This PR is the first thing to post into that channel from several OS threads at once (4 pool threads + the watchdog), so it's the first thing to expose it.

Repro

Section 5 of the test bench — "Dispatch 6", six 1s tasks against the 4-slot pool:

Run AsyncTask.Complete calls in logcat Landed in the UI Lost
1 6 (i=1..6) 4 (#3 #4 #6 #2) #1, #5
2 6 (i=1..6) 5 (#4 #1 #2 #5 #6) #3

The lost ones are always the ones posted within a few ms of another completion (39.787 / 39.794 / 39.795 → the middle one disappears). The spinner never stops, because count($this->lane) === 6 is never reached.

Run one at a time, everything in the bench passes:

  • Happy path — task ctx differs from the UI ctx, pid matches
  • Failure path — original exception class survives the hop
  • Timeout — fires at 3s, late result correctly discarded
  • Pool identity — 4 distinct ctx values, late tasks reuse a ctx at run #2

So the PHP layer, the Kotlin executor, the watchdog and the boot/stop lifecycle all behave. It's purely the delivery channel dropping frames under concurrency.

Not Android-specific

iOS goes through the identical C function (AsyncFunctions.swiftNativeElementBridge.sendNativeEventnphp_element_post_event). It's a race, and Android loses it more often because it's raw JNI threads with nothing scheduled in between. iOS passing is luck, not immunity.

Fix (deferred)

Make event_heap a real FIFO. Each frame already carries data_size in its header, so append-at-tail / pop-at-head inside the existing event_mutex is a contained change, and event_count becomes an actual count. No per-frame wire-format change, so the Kotlin/Swift readers stay as they are.

Two open decisions when we pick this up:

  1. Bounded vs unbounded queue. Unbounded is simplest, but a screen that stops draining while tasks keep completing grows memory without limit. A cap with drop-oldest + a log line is safer, but needs a number.
  2. Whether to bump nphp_get_format_version. The per-frame format doesn't change, but that version is what guards stale-lib-vs-new-reader mismatches — bumping it makes a mismatch fail loud instead of silently wrong.

Deferring until after v4 ships. This is a build-scripts change, so it needs a full PHP rebuild and a re-ship of the libs into mobile-air — not a v4-window change. Tagging pending.

cc @simonhamp

@shanerbaner82 shanerbaner82 added the pending Deferred — revisit after the current release label Jul 27, 2026
Shane's finding on #228: nphp_element_post_event() is a single slot, not
a queue — a second post landing before PHP drains the first overwrites it.
This lane is the first thing posting into it from several OS threads at
once, so it's the first to expose it; concurrent completions get dropped,
the watchdog's own timeout event included.

Every path above the channel does reach an outcome, so the design section
stays, but claiming it flatly overstates what ships until the FIFO fix
lands in build-scripts (deferred past v4 — needs a PHP rebuild).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TFbiJwd86MZCTLH5kJHZxo

Copy link
Copy Markdown
Member Author

Good find, and I agree with deferring it — that's a build-scripts change with a rebuild and a lib re-ship behind it, not a v4-window one.

One consequence worth naming: this puts a ceiling on the "every dispatch reaches exactly one outcome" property I added last round. Every path above the channel does reach an outcome, but the channel can drop the frame carrying it — and that includes the watchdog's own timeout event, which is the thing meant to be the last line of defence. So the failure mode it was built to prevent (spinner never stops) is still reachable under concurrency, just via a different route. I've caveated the design doc rather than leaving it reading as an unqualified guarantee (e9c58df); the section stands, with the bound stated and the FIFO fix referenced.

Also worth flagging for whoever picks up the FIFO work: this lane posts from five threads, not four — the four pool threads plus the watchdog, which fires from its own thread precisely when a pool thread is stuck and not posting. That's the one poster guaranteed to be concurrent with nothing, so it's a useful case to have in mind when sizing the queue.

On your two open decisions — both look like calls for the build-scripts PR rather than this one, and @simonhamp is cc'd. For what it's worth, from this side: a bounded queue with drop-oldest would silently reintroduce exactly the failure this lane's watchdog exists to catch, so if it's bounded I'd want the drop to be loud enough that a dropped completion is distinguishable from a task that never finished.


Generated by Claude Code

@shanerbaner82 shanerbaner82 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an impressive piece of work — the API shape is right, and the PHP layer is genuinely well built: the static-closure reflection guard failing at the call site, the WeakReference screen-scoping (with the correct object-id-recycling rationale), normalizeResult() running in the background context so a non-encodable result fails loudly, the encodeCompletion() belt-and-braces fallback, the destructor discipline in PendingAsyncTask, and the Windows-safe array-form Symfony\Process under Jump. Cross-language plumbing was verified end-to-end: bridge names (AsyncTask.Dispatch/AsyncTask.Complete), watchdog keys, JNI descriptors, @_silgen_name signatures, and the success flag contract all line up, and the setenv-avoidance claim is true on both platforms. Notably, three independent read-throughs found no compilation blockers in the C, Swift, or Kotlin — the never-compiled code is closer to building than you'd expect.

Requesting changes because the PR's two central guarantees are each broken in several independent places, and a number of stated claims don't match the code. Organized by severity:

1. "Every dispatch reaches exactly one outcome" — six independent holes

1a. PHP — dispatch during mount() scopes to the wrong screen (both runloops). $this->mount() runs at NativeComponent.php:2230 but markActive($this) happens at :2238 (same shape on the Jump runloop at :2405 — its own comment at :2383 confirms mount precedes it). So the canonical "start loading when the screen opens" dispatch captures active() = the previous screen. The completion arrives, origin->get() !== $this, and it's dropped — and the timeout failure is dropped by the same check, so nothing ever unblocks the spinner. Fix: mark active before mount() in both runloops.

1b. iOS — watchdog disarmed on C-call return, not on delivery. AsyncTaskExecutor.swift:187-194 disarms after _async_php_run returns, on the premise that a completion "has already been posted from PHP". Two reachable paths violate that: the early strdup("") return when the slot isn't initialized, and a Zend bailout (memory limit, E_ERROR) that zend_first_try swallows — not a Throwable, so AsyncTaskRunner::run's catch never fires and no completion is posted. The run "ends", the only backstop is cancelled, spinner forever.

1c. Both platforms — stop() drops queued tasks after PHP was told success: true. Android drains the queue and calls inFlight.clear(), discarding the pre-built timeout failure envelopes it is already holding (AsyncTaskExecutor.kt:248-283); iOS stop() does pending.removeAll() + cancels the pending watchdogs its own doc comment says "will unblock the UI" (AsyncTaskExecutor.swift:256-270, also the boot-failure branch at :129). In both cases PendingAsyncTask believes the task is live. The executors should post the failure events they're discarding.

1d. Both platforms — no native watchdog↔completion dedup. PHP posts AsyncTask.Complete inside the run; disarm happens only after request teardown returns. A task finishing at the deadline delivers Finished then Failed for the same id. Single delivery currently survives only because the PHP side forgets on first delivery — which depends on in-order processing of an event channel the last commit's own message says can drop frames. Same fix both sides: disarm in AsyncFunctions.Complete (the payload already carries id; on iOS it's documented as such and never read).

1e. Jump — a subprocess that dies without spooling is silently forgotten. sweepJumpTimeouts() (AsyncTaskTransport.php:223-227, same shape in hasPendingJumpRunners()) drops a dead process's entry and its deadline without checking the exit code or whether a spool file exists. Child OOMs, hits a parse error mid-edit, fails to bootstrap, or is SIGKILLed → no completion, no timeout, runloop returns to timeout: -1. This is the dev-loop path, so it'll be hit constantly. disableOutput() also means there's no stderr to diagnose with (and it makes getErrorOutput() throw, so the fix needs output routed to a file or captured incrementally). Fix shape: on !isRunning() with no spool file, synthesize the failure completion before unsetting.

1f. Jump — end of the hosting request SIGKILLs every child. static::$jumpProcesses holds Symfony Process objects whose __destruct runs stop(0). Hot reload / device-gone give-up / root back-out kills all in-flight tasks with no failure delivered; conversely a completion spooled before death is drained by the next process (drainJumpCompletion() is indiscriminate; on macOS php -S forks a pool, so worker A can drain worker B's completion) into empty AsyncTaskRegistry/NativeCallbacks statics and vanishes silently.

2. The reboot crash this PR exists to prevent is still reachable

2a. Android — the drain result is checked and ignored. MainActivity.kt:941 (and :1032): if (asyncExecutor?.stop() == false) { log } then shutdownPersistentRuntime() anyway. A task past the 5s stop budget is still inside zend_eval_string when php_module_shutdown frees the state its TSRM context references — the exact SIGSEGV the KDoc on the executor says the synchronous wait prevents. Compounding it, native_async_thread_shutdown early-returns unless called from the owning thread (php_bridge.c:1424-1427), so there is no C entrypoint to reap an orphaned context: the gen-N−1 thread eventually finishes and runs php_request_shutdown + ts_free_thread against gen-N (or freed) state.

2b. iOS — the opposite trade-off, also unsafe. stop()'s queue.sync drains the entire per-slot backlog, uncancellable and unbounded — 20 queued 10s tasks block hot reload ~50s per slot (AsyncTaskExecutor.swift:275-279). Meanwhile the 30s bootFinished timeout path proceeds to reboot while the boot loop is still calling async_php_boot, and leaves the group entered (counter skewed for every later cycle) (:72-83, :249-251). An interrupted boot's slot is never reclaimed — in_use stays 1 forever, and at ASYNC_PHP_MAX_SLOTS 4 a few hot reloads with slow boots permanently brick the lane (every subsequent boot returns -2, every dispatch rides the watchdog) (PHP.c:1363-1401, 1453-1471).

2c. Both C lanes — taskId is interpolated into zend_eval_string unescaped (PHP.c:1276-1288, php_bridge.c:1493-1506) while the webview lane in the same files escapes every inlined parameter via webview_escape_php. The UUID invariant is real today but enforced nowhere in C/Swift/Kotlin; any id with a quote yields a swallowed parse error and a generic timeout, and any future caller letting an id cross a trust boundary gets PHP execution. One escape call per side closes the class. (Related: char eval_code[1024] truncates silently; the ephemeral lane uses 4096.)

2d. Races (narrower but real): iOS submit() enqueues after dropping the lock, so a stale block can run against a rebooted pool whose slot index was re-allocated (AsyncTaskExecutor.swift:185-187 vs :275-279); Android dispatch()/stop() share no lock, so a task can land in a queue nobody will drain after Dispatch returned success (AsyncTaskExecutor.kt:119-146 vs :237-250); Android's failed async_embed_init leaves a half-started request that the 500ms retry loop re-enters forever (php_bridge.c:1410-1422 + AsyncTaskExecutor.kt:158-167).

3. Claims vs. code

  • "Queueing beyond the pool is logged rather than silent" — on Android the LinkedBlockingQueue is unbounded so offer() never fails and the "queue full" branch is dead code; the waiting log fires only at 2×poolSize outstanding (AsyncTaskExecutor.kt:62, :133-141). On iOS nothing logs or bounds backlog at all, and round-robin slot selection queues task 5 behind a busy slot 0 while slots 1–3 idle (AsyncTaskExecutor.swift:175-186).
  • Port asymmetries: Android deadlines use System.currentTimeMillis() — a clock-set forward trips every watchdog, backward means never (AsyncTaskExecutor.kt:126); iOS correctly uses monotonic DispatchTime. iOS buffers dispatches during startup; Android hard-fails anything in the 2.5s WORKER_START_DELAY_MS window — so a first-screen dispatch deterministically hits ->failed() (MainActivity.kt:123, :250-255).
  • "The pool boots lazily" (iOS comment) — start() eagerly boots 4 full Laravel contexts at launch and after every reload. Worth either making true or deleting, given the memory footprint.
  • iOS dispatch_semaphore_create pairs are never released across reboot cycles — 8 leaked semaphores per hot reload (PHP.c:1393-1394).

4. Packaging & tests

  • composer.json is missing both new dependencies: laravel/serializable-closure (used unguarded in PendingAsyncTask.php:8, AsyncTaskRunner.php:5) and symfony/process (class_exists-guarded). Works today only because host apps pull them via laravel/framework.
  • The suite's 4th skip is this PR's own test, and it can never run: AsyncTaskTransportTest.php:91 guards on function_exists('nativephp_call'), which the Jump polyfill defines in every testbench boot — so the refused-dispatch cleanup test has never executed, locally or in CI.
  • The fake diverges from the device in two ways that matter: it never serializes the work envelope (captured objects are shared by identity in tests, deep copies on device — AsyncTaskRunner::invoke(unserialize(serialize($this->work))) in startFaked() closes it), and ->shared() delivers nothing under fake (only assertShared() sees it).
  • A resource capture serializes silently to i:0 (no throw in PHP 8.4), so the "fails loudly at dispatch" guard misses the very example its comment names.
  • sweepJumpTimeouts / spawnJumpRunner have zero test coverage; a few tests are effectively vacuous (assertNotDispatched on a fresh fake; the "memory only" test hand-calls NativeCallbacks::register rather than dispatching, so flipping durable: in PendingAsyncTask wouldn't fail it).
  • Security posture worth one look: task ids ride argv (visible in ps on the dev machine) and the payload is unserialize()d without allowed_classes; the 0700 spool dir is the real barrier, and the SerializableClosure signature only exists when APP_KEY is set. Cheap hardening: allowed_classes + HMAC over the payload file. Orphaned payload files also have no GC on any path.

None of this needs a rethink — 1a is a two-line move, 1b/1d are "disarm in Complete", 1c is "post what you're already holding", 1e/1f are exit-code checks and process ownership, and the packaging/test items are mechanical. The on-device pass you already planned should then validate the lot. Happy to pair on any slice of it.

🤖 Review drafted with Claude Code

From @shanerbaner82's review. This is the PHP/packaging slice; the native
executor findings are held pending a decision on the hot-reload trade-off.

1a — a task dispatched from mount() scoped to the WRONG screen. mount()
runs before the runloop marks itself active, and the router calls it
before runLoop() on the hot-swap path, so the canonical 'start loading
when the screen opens' dispatch captured the screen being replaced. Its
completion was then dropped by the origin check — and so was its timeout
failure, by the same check, so nothing could unblock the spinner. Marked
active before mount() in run(), and around mount()/onResume() + runLoop()
in NativeRouter (markActive/restoreActive are @internal-public for it).

1e — a Jump runner that exited without spooling was dropped along with
its deadline, so nothing ever reported on the task. That's the dev loop's
likeliest failure: a parse error from a half-saved file, a fatal in
bootstrap, an OOM, a SIGKILL. Now synthesized as a failure carrying the
exit code and the tail of the runner's stderr, so the cause is visible
instead of a generic timeout a minute later. disableOutput() is gone for
the same reason — it made getErrorOutput() throw at exactly the moment
it was needed.

Packaging — composer.json was missing laravel/serializable-closure (used
unguarded) and symfony/process. Both only resolved via the host app's
laravel/framework.

Tests — the refused-dispatch test guarded on function_exists(
'nativephp_call'), which the Jump polyfill defines in every testbench
boot, so it had never run; it now drives the refusal through FakeBridge,
with the accepted and unanswered cases alongside. The durability test
hand-called NativeCallbacks::register(), so flipping durable: in
PendingAsyncTask wouldn't have failed it; it now goes through a real
dispatch. The fake also round-trips the work envelope through
serialize/unserialize, so captured objects are deep copies in tests as
they are on a device.

Also corrected the serialize-guard comment: PHP 8.4 serializes a captured
resource to i:0 without complaint, so that guard never caught the case
its own comment named.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TFbiJwd86MZCTLH5kJHZxo

Copy link
Copy Markdown
Member Author

Checked every finding against the code — the ones I've acted on I confirmed first-hand rather than taking on trust, and they were all real. 1a in particular is the best catch in the review: I wrote that ordering and read past it twice. mount() is exactly where "start loading when the screen opens" lives, so the canonical dispatch was scoping to the screen being replaced — and because the timeout failure is dropped by the same origin check, the one mechanism meant to guarantee an outcome was disabled precisely for the dispatch most likely to need it.

Pushed (b7a4776) — the PHP and packaging slice:

  • 1a — marked active before mount() in run(), and around mount()/onResume() + runLoop() in NativeRouter (the hot-swap path mounts from the router, so the runloop-side move alone wouldn't have covered it). markActive/restoreActive are @internal-public for that caller.
  • 1e — a Jump runner that exits without spooling now gets a synthesized failure carrying the exit code and the tail of its stderr, so a parse error from a half-saved file reads as a parse error rather than a generic timeout 60s later. disableOutput() is gone for the reason you gave: it made getErrorOutput() throw at exactly the moment it was wanted.
  • Packaginglaravel/serializable-closure and symfony/process added to composer.json. Straightforwardly missing; both were riding in on the host app's laravel/framework.
  • Tests — the refused-dispatch test now drives the refusal through FakeBridge instead of a function_exists('nativephp_call') guard that could never be false in testbench, with the accepted and unanswered cases alongside. The durability test goes through a real dispatch, so flipping durable: in PendingAsyncTask now actually fails it. The fake round-trips the work envelope through serialize/unserialize, so captured objects are deep copies in tests as on device. Plus coverage for the runner-died-silently paths.
  • Corrected the serialize-guard comment — you're right that PHP 8.4 turns a captured resource into i:0 without complaint, so that guard never caught the case its own comment named.

Held, deliberately — the native executor slice (1b, 1c, 1d, 2a, 2b, 2c, 2d):

Not because I disagree; I traced each one and they look right, including the ASYNC_PHP_MAX_SLOTS exhaustion in 2b, which is the nastiest of them because it's permanent rather than per-cycle. But every one lives in code that has still never been compiled or run, and I can't verify a fix for a thread-lifecycle bug by reasoning about it — that's how the first version of stop() got written. You have the emulator up and offered to pair, so that slice is genuinely better in your hands than mine.

Two things from my side if you take it:

  • 2a is a decision, not just a fix. Skip the reboot / defer-and-retry / raise the deadline all change hot-reload behaviour, and it's @simonhamp's call. I'd add one thing to whichever wins: don't construct a new AsyncTaskExecutor while the old one's threads are alive — that's what produces the degraded pool you saw (3 contexts for a pool of 4), and it'll survive any of the three options unless it's fixed explicitly.
  • 1d and the last commit interact. "Disarm in Complete" is the right shape, but it makes single delivery depend on the Complete event actually arriving — and per e9c58df that channel can drop frames under concurrency. Worth making the disarm robust to a dropped Complete (or landing the FIFO fix first), otherwise a dropped completion silently disarms nothing and the watchdog stays the only backstop for a task that already finished.

CI running on b7a4776; I'll drive it green.


Generated by Claude Code

The full envelope round-trip broke 'passes an AsyncTaskException carrying
the original message': unserializing a closure re-evaluates its source in
the namespace ReflectionClosure reports, which for a closure written in a
Pest test file is Pest's compiled namespace rather than the global one it
was written in — so an unqualified 'new RuntimeException' resolved to
P\Tests\Unit\RuntimeException.

That's an artifact of re-evaluating a test-file closure in this process,
not something a device does, so reproducing it here fails tests over a
problem real dispatches don't have. Round-trip the task-subclass args,
which are plain data and where the identity-sharing gap is real, and
document the remaining closure divergence rather than papering over it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TFbiJwd86MZCTLH5kJHZxo

Copy link
Copy Markdown
Member Author

Green on d2982ae (tests, Pint, PHPStan).

One note, since you gave the fix precisely and I didn't land it as written: AsyncTaskRunner::invoke(unserialize(serialize($this->work))) in startFaked() fails the suite. Unserializing a closure re-evaluates its source in the namespace ReflectionClosure reports for it, and for a closure written in a Pest test file that's Pest's compiled namespace rather than the global one it was written in — so the unqualified new RuntimeException('boom') in the failure-path test resolved to P\Tests\Unit\RuntimeException.

That's an artifact of re-evaluating a test-file closure in this process, not something a device does, so reproducing it would fail tests over a problem real dispatches don't have. What landed: task-subclass args are deep-copied (plain data, and the identity-sharing gap you described is real there), and the remaining closure divergence is documented on FakeAsyncTask rather than papered over — assert on what the work returns, not on objects it mutates through a capture. If you see a way to round-trip the closure that survives the test-file namespace, I'd take it; I couldn't find one that wasn't worse than the divergence.

The other four items from §4 are in: composer.json deps, the refused-dispatch test that could never run, the durability test that couldn't fail, and the serialize-guard comment that named a case it didn't catch.


Generated by Claude Code

claude added 2 commits August 7, 2026 13:58
The single-slot channel this documented was fixed on main (7b38826 plus
the extension-side change), so the caveat now describes a bound that no
longer exists. Rewritten as history rather than a live limitation.

Keeps one residual on the record: post_event returns 1 queued / 0 dropped
and drops when PHP has stopped draining. The UI writer discards that by
design, but a completion is the case that should check it — and
AsyncTask.Complete doesn't yet, which leaves a dropped completion leaning
on the watchdog again. Noted as a native follow-up rather than fixed here,
since that lane is still uncompiled.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TFbiJwd86MZCTLH5kJHZxo

Copy link
Copy Markdown
Member Author

Merged main into the branch (8755042) — clean, no conflicts, despite how far it had moved. Worth surfacing what came with it, because it changes two things on this PR.

The event-channel FIFO landed (#265 plus the extension-side change). That's the fix @shanerbaner82 opened as deferred-past-v4 — so the caveat I added to the design doc in e9c58df described a bound that no longer exists. Rewritten as history rather than a live limitation (02a3a57).

It also answers the bounded-vs-unbounded question in that thread, and leaves this PR one loose end. nphp_element_post_event() now returns 1 queued / 0 dropped, dropping only when the queue is over its backlog cap — so the drop is bounded and reported, which is the shape I argued for. The general UI writer discards that return deliberately, and NativeElementBridge.swift is explicit about why it shouldn't be copied:

// A caller that DOES care — an async-task completion, say — should
// check it rather than copy this.
_ = nphp_element_post_event(type, callbackId, nodeId, data, UInt32(max(0, dataSize)))

AsyncTask.Complete is precisely that caller and doesn't check it yet: a dropped completion currently falls back to the watchdog, which is the slow path the completion was supposed to make unnecessary. It's a small change — thread a Bool back through sendNativeEvent on both platforms — but it lands in the native slice that's still uncompiled, so I've recorded it as a follow-up rather than adding an unverified change to that pile. It belongs with 1b/1c/1d whenever those get picked up.

Two knock-on effects on the review:

  • 1d gets easier. My caution there was that "disarm in Complete" makes single delivery depend on a channel that could silently lose the Complete. With a FIFO plus a drop signal, that dependency is now checkable rather than a silent hope.
  • The v4 hold has expired. This was parked until v4 shipped; main now carries v4 and the pinned binaries.

Still outstanding and unchanged: 2a (the hot-reload decision — skip / defer-and-retry / raise the deadline, plus not constructing a new AsyncTaskExecutor while the old one's threads are alive) and who takes the native slice. Both are @simonhamp's call.

CI running on 02a3a57.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request needs testing Manual testing needed pending Deferred — revisit after the current release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants