Skip to content

Conversation

@nhz2
Copy link
Member

@nhz2 nhz2 commented Sep 7, 2024

This is part of JuliaIO/TranscodingStreams.jl#239

This PR reduces the allocations needed when decompressing data compressed with transcode.

For example:

julia> using BenchmarkTools

julia> using CodecZstd

julia> c = transcode(ZstdCompressor, rand(UInt8, 10000));

julia> @btime transcode($(ZstdDecompressor()), $(c));

With this PR and JuliaIO/TranscodingStreams.jl#239:
416.579 ns (4 allocations: 10.05 KiB)

Before:
510.451 ns (5 allocations: 19.72 KiB)

@codecov
Copy link

codecov bot commented Sep 7, 2024

Codecov Report

❌ Patch coverage is 93.33333% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 63.11%. Comparing base (eef39ab) to head (db69f63).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
src/libzstd.jl 66.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master      #64      +/-   ##
==========================================
+ Coverage   61.49%   63.11%   +1.61%     
==========================================
  Files           4        4              
  Lines         361      366       +5     
==========================================
+ Hits          222      231       +9     
+ Misses        139      135       -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@nhz2 nhz2 marked this pull request as draft October 5, 2024 16:04
@nhz2 nhz2 marked this pull request as ready for review October 5, 2024 18:26
write(s, b"Hello")
close(s)
buffer3 = take!(sink)
@test CodecZstd.find_decompressed_size(buffer3) == CodecZstd.ZSTD_CONTENTSIZE_UNKNOWN
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

transcode with ZstdCompressor now records the decompressed size.

@nhz2 nhz2 requested a review from mkitti October 5, 2024 19:44
Copy link
Member

@mkitti mkitti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Julia functions should not return error codes. They should throw exceptions.

@nhz2 nhz2 marked this pull request as draft October 6, 2024 00:28
@nhz2 nhz2 marked this pull request as ready for review October 6, 2024 19:11
@nhz2 nhz2 requested a review from mkitti October 6, 2024 19:11
@nhz2 nhz2 marked this pull request as draft September 8, 2025 22:28
@mkitti
Copy link
Member

mkitti commented Nov 11, 2025

What is the status of this pull request at the moment?

@nhz2
Copy link
Member Author

nhz2 commented Nov 12, 2025

It has some minor conflicts that need to be resolved. I also want to try to maintain support for older versions of TranscodingStreams.

@nhz2 nhz2 marked this pull request as ready for review November 14, 2025 22:12
@nhz2 nhz2 merged commit a5f5289 into master Nov 15, 2025
23 checks passed
@nhz2 nhz2 deleted the nz-pledgeinsize branch November 15, 2025 04:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants