Skip to content

Invalid range in read with get from object_store #654

@nomiero

Description

@nomiero

Describe the bug

We saw this error while using SlateDB with rustfs as the object store.

ObjectStoreError(Generic { store: \"S3\", source: Generic { store: \"S3\", source: RetryError(RetryErrorImpl { method: GET, uri: Some(http://rustfs:9000/cider-data/slatedb/compactions/00000000000000000003.compactions), retries: 1, max_retries: 10, elapsed: 37.399360833s, retry_timeout: 180s, inner: Status { status: 400, body: Some(\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><Error><Code>InvalidArgument</Code><Message>invalid header: range: &quot;bytes=42-41&quot;</Message></Error>\") } }) } }  

Per my understanding from reading the code, this seems to happen because the last chunk of the stream that indicates end of stream was not received and then a retry was issued, which goes through this and reduces the end bound which results in the invalid range (Not fully confident of this tracing).

I'm wondering if a fix can be done during retry where a check if the entire range is downloaded, then no need to retry and it can just exist with success.

To Reproduce
Can't easily reproduce this as it seems to not happen frequently.

Expected behavior
Instead of failing with 400 although the range is fully downloaded, it should just return success.

Additional context
Some related issues have been reported in SlateDB slatedb/slatedb#1019

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions