resumable: clarify handling of conflicts and retry policies #3319
Conversation
See httpwg#3275 for discussion.
guoye-zhang
left a comment
guoye-zhang
left a comment
There was a problem hiding this comment.
Thanks for opening this PR, the text looks good. Left a few minor comments.
| - For a `5xx (Server Error)` status code, the client MAY automatically attempt upload resumption by retrieving the current offset ({{offset-retrieving}}). | ||
|
|
||
| If no final response was received at all due to connectivity issues, the client MAY automatically attempt upload resumption by retrieving the current offset ({{offset-retrieving}}). | ||
| - For a `409 (Conflict)` status code, the client SHOULD attempt to resume the upload by using the offset from the `Upload-Offset` response header field. |
There was a problem hiding this comment.
I think it's better to keep this flow the same as 5xx: recommending doing an offset retrieval before resuming. Otherwise this creates a new code path, and the text in "Upload Append" about picking the offset is also inconsistent with the recommendation here.
There was a problem hiding this comment.
Updated, how does this look? It feels wrong to completely omit the option to leverage the Upload-Offset from the response since it's called out as a MUST for the server to return it just below
| 3. **Optimistic Concurrency Control**: By maintaining the upload's state in a strongly consistent datastore, the server can atomically check if the `Upload-Offset` in a request matches the resource's current state. If they do not match, the server can reject the request with a `409 (Conflict)` status code. | ||
|
|
||
| Since implementing this approach is not always technically possible or feasible, other measures can be considered as well. A simpler approach is that the server only processes a new request to retrieve the offset ({{offset-retrieving}}), append representation data ({{upload-appending}}), or cancellation ({{upload-cancellation}}) once all previous requests have been processed. This effectively implements exclusive access to the upload resource through an access lock. However, since network interruptions can occur in ways that cause the request to hang from the server's perspective, it might take the server significant time to realize the interruption and time out the request. During this period, the client will be unable to access the resource and resume the upload, causing friction for the end users. Therefore, the recommended approach is to terminate previous requests to enable quick resumption of uploads. | ||
| Servers SHOULD choose a strategy that best fits their architecture while fulfilling the requirements of this section. Regardless of the chosen strategy, clients MUST be prepared to handle a `409 (Conflict)` response as a recoverable error, as described in {{upload-appending}}. |
There was a problem hiding this comment.
All retry behaviors are MAY/SHOULD, do we need a MUST here?
There was a problem hiding this comment.
Good catch, fixed to be consistent (SHOULD)
- Addressed feedback around inconsistencies (HAVE vs MUST). - Removed example 409 conflict recovery.
See discussion on httpwg#3319
|
|
||
| 2. **Pessimistic Locking**: The server processes requests for a given upload resource sequentially, effectively creating an exclusive lock on that resource. A new request is only processed after the previous one completes. This can be simpler to implement but may lead to delays if a request hangs from the server's perspective. | ||
|
|
||
| 3. **Optimistic Concurrency Control**: By maintaining the upload's state in a strongly consistent datastore, the server can atomically check if the `Upload-Offset` in a request matches the resource's current state. If they do not match, the server can reject the request with a `409 (Conflict)` status code. |
There was a problem hiding this comment.
I'm not sure if this is enough to guarantee issue-free uploads. What if the server receives two requests for the same (correct) offset? They both match, but which content is appended to the upload resource? Did you mean to say that the content should be appended in one transaction together with the offset check? But that would make this the same as the pessimistic locking, I think.
There was a problem hiding this comment.
I agree with Acconut here. To me approach 2 and 3 seem very similar in how they would work in practice. The server would receive two requests with the same offset (why though, really?) and one is slightly faster than the other i.e. it will get the exclusive lock and the thus the second request would fail. I think the wording of point 2 hints that the requests are queued and processed after each other which I guess is not the case (the offset would most likely never match for the second request)
There was a problem hiding this comment.
I had a go at rewording this to be clearer as it was both too vague and overly specific in different areas - is this any better? The key distinction between 2 & 3 is that servers don't need to acquire an exclusive lock for the resource which means they're able to achieve higher availability during network partitions
4 participants
See #3275 for discussion.