Repository navigation
[OTel]: Complete replay and chained-execution conformance coverage #874
Description
Parent: #873 — Remaining Python OpenTelemetry work.
Track the remaining conformance coverage and qualification represented by PR #796, using dc736d5 as the current reference. Implement through focused conformance PRs; keep runtime and testing-library fixes in their existing issues.
Coverage to deliver
| Case | Required behavior |
|---|---|
| 21 — completed-step replay | A completed step body runs once and returns its saved result after a durable wait/resume. Validate telemetry and replay independently of incidental event timing. |
| 22 — user-function context | Ordinary user spans inherit the SDK-owned handler, step/condition attempt, child, branch, and iteration contexts without supplying a repaired parent. |
| 23 — callback-function context | Callback submitters, wrapped retry callbacks, and virtual-child bodies retain the expected tracing parent in both views. |
| 24 — invocation retry | A retryable invocation error produces RETRYING/UNSET, followed by successful replay using the saved step result. |
| 27 — suspended chained child | A real chained child performs a one-second durable wait, resumes, and returns the original payload; check caller/child invocations, wait identity, parentage, and continuation links. |
Case 11's replay-barrier fix is already delivered by merged #864. Reuse it as a regression control rather than duplicating that implementation. PR #796 currently targets main; its description still contains historical stack text, so use the actual branch state when preparing a smaller PR.
Implementation scope
- The owning package is
packages/aws-durable-execution-sdk-python-conformance-tests-otel: handlers,template.yaml,src/Makefile, README, and local public-handler tests. - Keep requirement-to-handler mappings, deployed function names/targets, and the local catalog assertions synchronized in both Invocation and Execution views.
- Preserve the current shared workflow/catalog defaults to the conformance repository's
main, the manualconformance_test_refoverride, and SDK handlers built from the exact commit under test. Validate the shared workflow's current inputs/secrets contract when changing the caller. - The shared orchestrator runs the Invocation and Execution suites. Existing legacy long-running templates/tests are not a request to re-enable that suite.
- Coordinate language-neutral requirements and runner changes in aws/aws-durable-execution-conformance-tests, including the related work in shared PR #131. Use the public SDK API rather than compensating for a missing SDK capability in a handler.
Deferred callback qualification
Keep cases 25–26 explicitly NOT_IMPLEMENTED with issue references until their prerequisites are ready. Their deferred source and tests remain available at 6484e577.
- Case 25: requires terminal status/event semantics ([OTel]: Report terminal failures and keep exception events within span bounds #853) and empty-error history/runtime consistency ([Testing]: Preserve empty callback error history without changing runtime errors #862).
- Case 26: requires replay-completion telemetry ([OTel]: Export unvisited replay completions with the correct parent context #854), callback updated-operation production ([Testing]: Record callback terminal completions as updated operations #860), and input acknowledgment ([Testing]: Acknowledge invocation input without losing pending updates #861). Preserve the assertion inside the observed step that completion has already reached the exporter, plus raw export counts at both later barriers; recorded timestamps alone do not prove delivery order.
- Additional case 17 error regression: requires error snapshot isolation ([Bug]: Isolate terminal plugin error snapshots from checkpointed state #852) and the applicable callback-error fixes. The existing case 17 handler remains active.
Before restoring these tests, correct their history expectations: included diagnostic payloads are untruncated, while metadata-only history omits payloads and cannot reconstruct error details. The empty callback-failure Error.Payload service shape remains unverified against deployed AWS Lambda and needs evidence or an explicit qualification decision.
Known qualification gap
PR #796 reports that the S3 Execution-view job failed case 21 because WaitSucceeded preceded InvocationCompleted by 63 ms, contrary to a fixed history-order expectation. The execution returned before-after successfully. Determine which event order the service contract guarantees, collect the relevant trace/history evidence, and fix the owning requirement/fixture if the order is legitimately variable. Preserve the saved-step, result, and telemetry assertions; do not count execution success alone as conformance.
Acceptance criteria
- Cases 21–24 and 27 have public-API handlers, template/Makefile registration, documentation, and focused local regressions in both views.
- Case 11 remains covered by fix(otel): restore case 11 replay coverage #864's implementation and tests.
- The complete catalog accounts for active and deferred requirements; unsupported cases are never reported as passing.
- Case 21's cloud history-order failure is resolved or explicitly dispositioned with service evidence and the appropriate shared-repository change.
- Restore deferred cases 25–26 and the additional case 17 regression after their implementation prerequisites land, through separate focused PRs as needed.
- Qualify the intended behavior against the deployed Invocation/Execution backend matrix, including raw export counts and parent/trace identity. Link exact SDK/catalog revisions and run results.
- Update the PR/README to reflect current coverage, dependencies, and validation; identify changed test expectations and their reasons.
Starting points and validation
Reference files at the current PR head:
Use the repository's pinned Hatch version and run the package tests and focused workflow tests, plus package lint/format and commit-message checks. The PR description reports 75 local package tests on Python 3.13.17 and an earlier 10-test workflow validation; those are historical results, not new validation performed while filing this issue. Cloud qualification remains a separate requirement.
Activity