Skip to content

fix: propagate context into handler workers - #756

Open
zhongkechen wants to merge 21 commits into
mainfrom
fix/otel-handler-context-428
Open

zhongkechen wants to merge 21 commits into
mainfrom
fix/otel-handler-context-428

Conversation

@zhongkechen

@zhongkechen zhongkechen commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #428.

Carry invocation-start context into the SDK-owned handler worker so ordinary user spans and logs inherit the durable trace. Opted-in Invocation/Execution scopes bind the existing Invocation/Workflow span when the ambient parent is absent or unrelated, retain valid same-trace parents and baggage, and restore context on return, failure or suspension.

Isolate plugin-bearing invocation context from the host. Failed startup or scope-entry bindings are discarded before subsequent setup and user code; finalizers retain the Context that owns their tokens. Existing hook threads/order, result/exception identity and the no-plugin path are preserved.

Optional scopes require literal __durable_handler_context_api__ = 1 on the concrete class. The generic base provides no marker/default hook, so old helpers, properties and dynamic attributes remain untouched; custom subclasses explicitly redeclare the marker. Provider API 1, dependency floors, plugin lifetime and checkpoint/replay formats are unchanged. This fix is independent of #737. Supported older cores load the new plugin and retain their existing tracing behavior; the new worker-scope behavior requires the updated core.

Validation:

  • Full core: 1,778 tests. Full OTel: 362 tests. Actual installed minimum core 2.0.0: 314 applicable OTel tests. Repository type checks, formatting and wheel builds pass.
  • Real public worker/step/wait/resume coverage includes both OTel views, plugin order, same/unrelated/absent parents, baggage, success/failure, completed-body suppression and caller restoration. Sampled/unsampled direct scopes also preserve cleanup and exception identity.
  • Negative controls: 27 partial-setup cases, 11 legacy-hook collision cases and 26 Execution-scope cases fail before their respective fixes. The final context/compatibility suite passes 74 cases; the extended OTel scope matrix passes 98 focused cases.
  • Four actual installed core/plugin pairs pass 24 legacy helper/property/dynamic lookup + wait/resume/context-cleanup cases. Imports and final wheel contents are verified.
  • A deterministic concurrent stdout regression fails with concatenated JSON before the fixture output lock; all 11 normal-conformance package tests pass. Expected log records and wait timings remain unchanged. Recurrent layer lookup throttling uses bounded standard retries scoped only to that read; test timeouts and assertions are unchanged.

Cross-SDK conformance coverage is in #758 and aws/aws-durable-execution-conformance-tests#131. Current-head CI and actual AI review generation/posting are monitored separately.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 2, 2026 23:23 — with GitHub Actions Active
@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 2, 2026 23:26 — with GitHub Actions Active
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 2, 2026 23:40 — with GitHub Actions Active
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 3, 2026 01:56 — with GitHub Actions Active
@github-actions

This comment has been minimized.

@zhongkechen zhongkechen changed the title fix: preserve handler worker context fix: propagate context into handler workers Oct 3, 2026
@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 3, 2026 02:21 — with GitHub Actions Active
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 3, 2026 02:58 — with GitHub Actions Active
Comment thread pyproject.toml
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 3, 2026 03:10 — with GitHub Actions Active
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 3, 2026 03:56 — with GitHub Actions Active
@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 4, 2026 01:30 — with GitHub Actions Active
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 4, 2026 01:56 — with GitHub Actions Active
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 5, 2026 23:21 — with GitHub Actions Active
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 6, 2026 03:16 — with GitHub Actions Active
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 6, 2026 03:39 — with GitHub Actions Active
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 6, 2026 04:01 — with GitHub Actions Active
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen had a problem deploying to ai-pr-review-runtime October 7, 2026 02:06 — with GitHub Actions Failure
@zhongkechen
zhongkechen deployed to ai-pr-review-runtime October 7, 2026 02:40 — with GitHub Actions Active
concurrency:
group: otel-conformance-tests
cancel-in-progress: false
queue: max

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex AI review · Finding arf_v1_22d3xyth5kzgqv5ha3lcroqycg

P1 — Remove the unsupported concurrency key. GitHub Actions' concurrency mapping accepts only group and cancel-in-progress; queue: max makes this workflow invalid, so it will not start. The same addition also invalidates cloud-tests.yml and conformance-tests.yml. Remove all three queue entries and, if every run must be preserved, implement serialization using a supported external lock or dispatcher.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked the current official Actions workflow schema: concurrency-mapping supports queue, whose allowed values are single and max. Both workflow-level and job-level concurrency use this mapping.

The reviewed head 74c5486 has also completed Cloud tests and Conformance Tests successfully with these declarations, so the workflows are accepted and running. Keeping queue: max together with cancel-in-progress: false retains pending runs while preserving the active tests.

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Codex AI review

Found one blocking workflow-syntax regression affecting the cloud and conformance test workflows.

Reviewed commit 74c5486bd8779fdf4c15aafc1c7507867f0bc9f8. Workflow run

This branch was successfully deployed

1 active deployment
ai-pr-review-runtime — 74c5486b Deployed Oct 7, 2026 by zhongkechen via ai-pr-review / Codex review / Generate Codex review #1381
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature][OTel]: Move on invocation start and end to worker thread

1 participant