Skip to content

docs: add URL handling note to HTTPFileSystem class docstring#2024

Merged
martindurant merged 3 commits intofsspec:masterfrom
kenichikawaguchi:docs/http-url-handling-note
Apr 29, 2026
Merged

docs: add URL handling note to HTTPFileSystem class docstring#2024
martindurant merged 3 commits intofsspec:masterfrom
kenichikawaguchi:docs/http-url-handling-note

Conversation

@kenichikawaguchi
Copy link
Copy Markdown
Contributor

Summary

Adds a "Note on URL handling" section to the HTTPFileSystem class docstring,
describing the URL-passthrough behaviour and noting that applications
constructing URLs from user input are responsible for their own validation.

This was discussed in security advisory GHSA-69jp-3788-fh27 and suggested
by the maintainer (martindurant).

What changed

Added 14 lines to the HTTPFileSystem class docstring in
fsspec/implementations/http.py.

Note on URL handling
--------------------
Like any HTTP client library, this class passes URLs directly to the
underlying ``aiohttp`` session without filtering or restricting the
target host. URLs pointing to loopback addresses (``127.x.x.x``),
link-local addresses (``169.254.x.x``, used by cloud Instance Metadata
Services), or private network ranges (RFC 1918) are therefore
reachable, which is intentional for use cases such as accessing
internal S3-compatible storage or local development servers.

Applications that construct URLs from user-supplied input are
responsible for validating those URLs before passing them to
``HTTPFileSystem``. Failing to do so may expose the application to
Server-Side Request Forgery (SSRF).

Tone

The note is intentionally neutral — it describes standard HTTP client
behaviour, not a warning, consistent with the maintainer's feedback.

Related

Comment thread fsspec/implementations/http.py Outdated
Comment thread fsspec/implementations/http.py Outdated
@martindurant martindurant merged commit 0b290bf into fsspec:master Apr 29, 2026
10 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants