Skip to content

Stream SCP downloads into the OpenURI buffer - #93

Open
OskarEichler wants to merge 1 commit into
net-ssh:masterfrom
OskarEichler:codex/stream-open-uri-buffer
Open

Stream SCP downloads into the OpenURI buffer#93
OskarEichler wants to merge 1 commit into
net-ssh:masterfrom
OskarEichler:codex/stream-open-uri-buffer

Conversation

@OskarEichler

Copy link
Copy Markdown

Summary

Write SCP chunks directly into OpenURI's existing buffer so it can spill to a temporary file, instead of first accumulating the complete download in an extra StringIO.

Reproduction and evidence

A bounded local simulation transfers 1,024 chunks of 16 KiB (16 MiB), verifies the resulting byte count and uses /usr/bin/time -l in three fresh Ruby processes per implementation. Median maximum resident size: 70,123,520 → 33,603,584 bytes. Median transfer-buffer time: 0.013746s → 0.010415s. Both produce a Tempfile-backed OpenURI result. These measurements isolate buffering; they are not SSH/network throughput measurements. Combined binary/empty/20,000-byte models and local scp-backed URI integration pass.

Compatibility / breaking changes

The OpenURI result and rewind behavior stay unchanged. The private adapter now passes a write-capable destination to Net::SCP.download! rather than requesting a complete String return. Custom monkey patches of that call must accept an IO destination. OpenURI remains responsible for its own memory-to-tempfile threshold and result lifecycle.

Verification and limits

  • This focused branch passes the existing upstream suite: 36 tests / 70 assertions, Ruby 4.0.6, net-ssh 7.3.3, test-unit 3.7.8 and Mocha 2.1.0; all Ruby commands used rbenv.
  • The combined release-based consumer candidate passes the suite with frozen string literals enabled, 10 focused cases, 2,906 model assertions, 24 URI-component assertions and 19 local scp/URI assertions. Combined evidence includes other separately proposed fixes and existing upstream frozen-string corrections; it is not attributed to this patch alone.
  • No repository tests added/modified under the consuming project's explicit no-new-tests policy. Reproductions/models ran externally. No deployed servers, SSH connections or production data were used.
  • All seven runtime files parse with the Ruby 2.6 parser; actual older-Ruby/Windows/remote-server matrices were not run locally. No upstream lint config exists; targeted lint has the same eight pre-existing warnings, with no new warnings.
  • Existing upstream PRs/issues and contribution guidance were checked. Maintainer CI results are pending; no CI pass is claimed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant