Repository navigation
Upgrade declared wire protocol version from 2 to at least 5 (prerequisite for publish return values) #1345
Description
Context
ably-pubsub-java and ably-cocoa both ship publish return values: publish() returns a PublishResult containing the serials of the published messages, per the specification change in ably/specification#406 (specification 5.0.0, wire protocol version 5 — spec points PBR1/PBR2a, RSL1n, RTL6j, TR4s).
Family implementations:
- java: [AIT-99] feat: protocol v5 + message appends ably-pubsub-java#1182 (protocol v5 prerequisite), [AIT-98] feat: realtime edits and deletes ably-pubsub-java#1183 (PublishResult + ACK
resplumbing), [AIT-84] feat: add support for returning serials onpublishably-pubsub-java#1184 (public publish API; released in 1.6.0), Deprecation methods removal ably-pubsub-java#1247 (2.0 cleanup) - cocoa: [AIT-125, AIT-126] Realtime edits and deletes, new edits REST API, appends API ably-cocoa#2178 (protocol v5 +
resextraction), [AIT-299] Add newpublish()overloads that returnARTPublishResultably-cocoa#2181 (public API; released in 1.2.57)
ably-pubsub-dotnet cannot implement the realtime half of this feature until it declares a wire protocol version of at least 5: the ACK ProtocolMessage's res field (TR4s) is a protocol-v5 feature, and this SDK currently declares 2.
For the REST half: the specification has no per-feature version gate — RSL1n ("On success, returns a PublishResult object... The API response body will contain a superset of the fields of a PublishResult") is written for a client declaring the spec's protocol version (CSV2c/CSV2d). Whether the REST API includes serials in the publish response body for a client declaring version 2 is not specified anywhere, so it needs a quick empirical probe (step 2 below).
Note the specification has since moved on to wire protocol 6 (specification 6.x, LiveObjects partial object sync), and both sibling SDKs now declare 6 (java Defaults.ABLY_PROTOCOL_VERSION = "6", cocoa ARTDefault_apiVersion = @"6"). The target version for this upgrade (5 minimum vs 6 for family parity) is part of step 1.
Current state
The declared version lives in one constant, used in three places (paths as of the integration/v2 stack, PR #1343 head):
src/Ably.PubSub.Shared/Defaults.cs:40—public const string ProtocolVersion = "2"; // CSV2src/Ably.PubSub.Shared/Transport/TransportParams.cs:183— sent as thevquery param on every realtime connection (RTN2f)src/Ably.PubSub.Shared/Http/AblyHttpRequester.cs:76-78— sent as theX-Ably-Versionheader on every REST request (RSC7e)
The gap: what each protocol version introduced
From the ably/specification version history (meta.yaml bumps):
| Protocol | Spec version | Introduced | dotnet status |
|---|---|---|---|
| 3 | 3.0.0 (2024-01) | Untyped/flattened stats API | Audit: current Stats model may predate the untyped-stats shape |
| 4 | 4.0.0 (2025-09) | Message structure rework (TM2*): serial (TM2r), version object with default-population rules (TM2s), annotations/summary (TM2u, TM8; TM8a1 requires tolerating unknown summary structures); supersedes the 3.x-era TM2k/TM2p/TM2o/TM2n/TM2q fields |
Missing entirely — Message has only id/clientId/connectionId/connectionKey/name/timestamp/data/extras/encoding; no message action enum on Message at all |
| 5 | 5.0.0 (2025-12) | PublishResult + ACK.res (TR4s, PBR*); MESSAGE_APPEND action (TM5); update/delete/append APIs returning UpdateDeleteResult (RSL15, RTL32); batch-publish serials (BPR2c) |
Missing entirely — ProtocolMessage has no res; no mutable-message API |
| 6 | 6.0.0 (2026-02) | LiveObjects partial object sync (object messages, protocol 6+) | N/A while the SDK has no LiveObjects support; ProtocolMessage has no state field and its action enum tops out at Auth = 17 (no OBJECT/OBJECT_SYNC) — inbound tolerance to verify |
Family adoption pattern (from commit history): both siblings skipped declaring v3 and moved 2 → 4 together with the message-structure modeling (java 55da3464 "implement protocol v4 support...", cocoa 9f514a20 "Implement a nested message structure based on protocol v4"), then 4 → 5 with the append/publish-result work (java 31969a8a inside PR 1182: version constant + MESSAGE_APPEND enum value + append/edit APIs + version-pinning test updates — no separate tolerance work, because v4 had already remodeled Message; cocoa bumped the constant inside the res-extraction commit ceab5e81 in PR 2178). There is no family precedent for a tolerance-only bump — both SDKs modeled the v4 message structure before or with declaring v5, which is why step 1 below needs an explicit answer rather than an assumption. Also of note: cocoa removed RTL6d message bundling during v5 adoption (spec change: ably/specification#408); ably-pubsub-dotnet already sends one ProtocolMessage per publish call and re-sends queued messages individually, so no equivalent work is needed here.
Proposed steps
- Realtime-team confirmation. Is it safe for a client to declare protocol v5 (or v6) while implementing none of the v3–v5 feature surface, provided it tolerates the inbound shapes (unknown JSON fields are ignored by the SDK's Newtonsoft deserialization; unknown enum values need verification)? Also: pick the target version — 5 (minimum for
ACK.res) or 6 (current spec, family parity). This should be bundled with the already-pending RTN2g agent-identification question for the realtime team, since both affect the same connection-params code. - Empirical REST probe. Publish via REST with
X-Ably-Version: 2against sandbox and inspect the response body: does it already containserials? Decides whether the REST half of publish return values works before this upgrade lands. - Tolerance audit per the table above, with a per-item disposition (tolerate-and-ignore with tests / model minimally / out of scope):
- inbound
Messagecarryingserial,action,version,annotations/summaryfields (and non-CREATE actions such asMESSAGE_UPDATE/MESSAGE_DELETE/MESSAGE_SUMMARY/MESSAGE_APPENDarriving on subscribed channels); - unknown
ProtocolMessageaction values aboveAuth = 17; - untyped-stats response shape vs the current
Statsmodel; - anything else the confirmation in step 1 surfaces.
- inbound
- Protocol-bump PR (mirroring the java approach in [AIT-99] feat: protocol v5 + message appends ably-pubsub-java#1182): the
Defaults.csconstant change, the audit outcomes from step 3, and updates to the tests that pin the version string (AblyHttpRequesterSpecs,HttpSpecs,DefaultTests). Coordinate edits toDefaults.cs/TransportParams.cs/Agent.cswith the pending agent-identification change from step 1, since they touch the same lines. As noted in the UTS dependency comment below, the UTS forwards-compatibility spec (uts/realtime/unit/connection/forwards_compatibility_test.md, RTF1/RSF1) should be used as the basis for a manual forwards-compatibility test (written in this SDK's existing test architecture) providing the step-3 audit evidence in this PR. - Publish return values feature PR (separate, stacked on 4):
PublishResulttype,ProtocolMessage.res(TR4s), positional ACK matching, REST publish response-body decoding (RSL1n), and the publish return-type changes (RTL6j/RSL1n).
Acceptance criteria
- Realtime-team answer recorded: target protocol version + whether a tolerance-only declaration is acceptable
- REST probe result recorded (serials present/absent for a v2-declared client)
- Tolerance audit completed with a written disposition per item in the table
- SDK declares the agreed protocol version via
Defaults.ProtocolVersion, exercised on both the realtimevparam and the RESTX-Ably-Versionheader; version-pinning tests updated - Full existing test suite green (JSON protocol mode)
- Publish return values (separate follow-up) unblocked
Activity
sacOO7 commentedon Sep 30, 2026
UTS (Universal Test Suite) dependency on this upgrade
We assessed the UTS realtime/rest spec modules (
uts/realtime,uts/restin the specification repo: 130 spec files — 127 test specs plus 3 mock-helper specs — containing 1125 individual test specs, by direct census (the module README's counts are slightly stale)) for dependence on wire-protocol features newer than the version 2 this SDK declares. Summary: the protocol upgrade gates two deliverables, not one — the publish return values feature (above) and completion of UTS adoption for this SDK.Inventory: UTS specs requiring protocol > 2
Fully blocked spec files (12 files, 77 test specs — the entire mutable-messages / annotations / publish-result feature area):
realtime/unit/channels/channel_get_message.mdgetMessage(RTL28)realtime/unit/channels/channel_message_versions.mdgetMessageVersions(RTL31)realtime/unit/channels/channel_update_delete_message.mdUpdateDeleteResultfrom ACKres(RTL32a–e)ProtocolMessage.resnot parsedrealtime/unit/channels/channel_annotations.mdrealtime/integration/mutable_messages_test.mdrest/unit/channel/get_message.mdrest/unit/channel/message_versions.mdrest/unit/channel/update_delete_message.mdrest/unit/channel/annotations.mdrest/unit/channel/publish_result.mdPublishResultreturn + response-body decoding absentrest/unit/types/mutable_message_types.mdMessagelacks serial/action/version/annotations types entirelyrest/integration/mutable_messages.mdPartially blocked (2 files, 5 of their 40 tests):
realtime/unit/channels/channel_publish.mdRTL6j/publish-result-serials-0,batch-publish-serials-1,incrementing-msg-serial-2,nack-results-error-3)PublishResultfrom ACKres(RTL6j, TR4s)resparsing absent (mock can sendres; SDK discards it)rest/integration/publish.mdRSL1n/publish-result-serials-0)Related but distinct:
rest/unit/batch_publish.md(28 tests) exercisesRestClient#batchPublish(RSC22), which this SDK does not have at all — a version-independent API gap — and 2 of its tests (BPR2c/serials-array-0,BPR2c/serials-null-conflated-0) additionally assert the protocol-5-eraserialsin batch responses.Not blocked, worth knowing:
rest_client.md,channel/publish.md,channel/idempotency.md,encoding/message_encoding.md) use mock 2xx publish responses shaped{"serials": [...]}as fixtures. They translate and run fine at v2 today (the SDK ignores the body), and the fixtures are already the post-upgrade response shape — no rework later.realtime/unit/connection/forwards_compatibility_test.md(RTF1/RSF1, 3 tests — ProtocolMessages/Messages must tolerate unrecognised attributes and unknown enum values) is runnable today and is precisely the tolerance audit step 3 of this issue asks for — translating it should be part of the upgrade PR's test evidence.rest/unit/stats.md,realtime/unit/client/realtime_stats.md,rest/integration/time_stats.md) use standard stats JSON fixtures and are not blocked at v2; the protocol-3 untyped-stats question remains only a model-shape audit item.uts/objectsmodule (LiveObjects) is out of this census; it presupposes protocol 4+/6 features and an objects plugin this SDK does not have.Conclusion
res,Message.serial/version/annotations, unknown actions) blocking the unit tier even under mocks; the declared version alone blocks only the 17 integration-tier tests that need the live sandbox to send protocol-5 shapes. So the constant bump by itself unblocks the integration tier's wire behaviour, while the features it gates (this issue's step 5 and the mutable-message/annotations APIs) unblock the rest.[-]Upgrade declared wire protocol version from 2 to 5 (prerequisite for publish return values)[/-][+]Upgrade declared wire protocol version from 2 to at least 5 (prerequisite for publish return values)[/+]sacOO7 commentedon Oct 1, 2026
The SDK's declared wire protocol version is raised from 2 to 6 in the pull request below.
Basis: parity with ably-pubsub-java (
lib/src/main/java/io/ably/lib/transport/Defaults.java,ABLY_PROTOCOL_VERSION = "6") and ably-cocoa (Source/ARTDefault.m,ARTDefault_apiVersion = @"6"), both currently at 6.No SDK in the family has previously declared a newer version while implementing only wire tolerance for parts of it, so the sandbox tests that were gated on the declared version are now enabled as the empirical check.
The
/statsendpoint stays pinned to the v2 response format.PR: #1349