Skip to content

out_s3: build the chunk debug message only when debug is on - #485

Merged
kenhys merged 1 commit into
fluent:masterfrom
Watson1978:out_s3-log-debug-block
Aug 20, 2026
Merged

kenhys merged 1 commit into
fluent:masterfrom
Watson1978:out_s3-log-debug-block

Conversation

@Watson1978

Copy link
Copy Markdown
Contributor

out_s3 interpolates its two chunk debug messages at the call site, so dump_unique_id_hex, Fluent::Plugin::Buffer::Metadata#to_s and the concatenation all run on every chunk write — including at the default log level, where Fluent::Log#debug returns on its first line without printing anything. Passing a block moves that work behind the level check.

  • Measured with fluentd 1.19.3 on Ruby 4.0.6 over 200,000 calls with a real Metadata and unique_id: at log_level info the current form costs 12 objects / 1000 B / 1.417 µs per call, the block form 1 object / 40 B / 0.044 µs. At log_level debug the two are identical (22 objects / 3240 B / ~2.3 µs)
  • log.debug { ... } is enough here. log.on_debug { log.debug ... } removes the remaining Proc allocation and reaches 0 B, but 40 bytes does not pay for the extra nesting
  • The logged line is unchanged when debug is on, confirmed through the test driver against the real write path
  • This fires once per chunk write rather than per record, so the absolute saving is small. It is included because the fix costs nothing and the allocation is pure waste

The message was interpolated at the call site, so dump_unique_id_hex,
Metadata#to_s and the concatenation ran on every chunk write even at the
default log level, where Fluent::Log#debug returns before printing
anything.

- Pass a block instead, which moves that work behind the level check:
  1000 bytes and 12 objects per call become 40 bytes and 1 at log_level
  info, measured with fluentd 1.19.3 on Ruby 4.0.6
- The logged line is unchanged when debug is on

Signed-off-by: Shizuo Fujita <fujita@clear-code.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Watson1978
Watson1978 requested a review from kenhys August 19, 2026 01:53
@kenhys
kenhys merged commit fbba2c7 into fluent:master Aug 20, 2026
9 checks passed
@Watson1978
Watson1978 deleted the out_s3-log-debug-block branch August 24, 2026 03:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants