Repository navigation
config: parse a fractional number before a size or time unit - #5625
Conversation
The size and time types matched only the digits right before the unit, so the integer part of a fractional value was dropped silently: "0.5s" became 5 seconds, "1.5m" 5 minutes and "1.5k" 5120 bytes. Accept a decimal number before the unit. Values without a fractional part are parsed exactly as before and keep returning an Integer. Signed-off-by: Lucas Barros <lucasbarros.kam@gmail.com>
Watson1978
left a comment
There was a problem hiding this comment.
Thanks for the PR and the clear description. Reading 0.5s as 5 seconds is an easy trap to fall into, and the fix makes sense.
One thing to fix before merging: unit_value uses Float arithmetic, which is not exact for many decimal inputs and can shift timekey buckets. Details and a suggestion are inline.
About compatibility: existing configurations that already use such values get a different effective value after upgrading. For example, total_limit_size 2.5g was 5 GiB and becomes 2.5 GiB. So we will ship this in the next minor release as an enhancement, not as a backport to v1.19, and call it out in the release note.
Signed-off-by: Lucas Barros <lucasbarros.kam@gmail.com>
Watson1978
left a comment
There was a problem hiding this comment.
Thanks for the update. unit_value now multiplies with Rational. The new 1.1h and 0.7d cases fail on master and on e41d027, and return the exact integers 3960 and 60480 here.
I also tried two scenarios. With timekey 1.5m, this version restored and flushed a file buffer chunk written by master. In a YAML config with retry_wait 0.5s, the retries went to <secondary> as expected.
As mentioned in the first review, this goes into the next minor release, not v1.19.
Which issue(s) this PR fixes:
None (found while reading
lib/fluent/config/types.rb; no existing issue or PR about it).What this PR does / why we need it:
Config.size_valueandConfig.time_valuelook for the unit with unanchored regexps such as/([0-9]+)s/. With a fractional value, only the digits right before the unit are captured, so the integer part is dropped without any warning:flush_interval 0.5sgives a 5 second flush interval,retry_wait 1.5mwaits 5 minutes,chunk_limit_size 1.5mgives 5 MB. A unitless value such as0.5already parses as 0.5 seconds, so this mostly bites people who add the unit.This accepts an optional decimal part before the unit (
/([0-9]+(?:\.[0-9]+)?)s/, and the same for the other units). The unit is applied withRational, so a whole result is an exact Integer (1.1h-> 3960) and any other result is a Float. Integer values with a unit suffix still produce the same integer result, so10s,10sec,2m,4Gand friends are unchanged. Sizes are truncated to an Integer (1.5k-> 1536), as thesizetype is a byte count.Tests: new data in the
normal casetests ofConfig.size_value(1.5k,0.5m,2.5G) andConfig.time_value(0.5s,1.5m,0.25h,1.5d,1.1h,0.7d), checked with and withoutstrict. They fail on master and pass here:test/config/test_configurable.rb,test_configure_proxy.rb,test_system_config.rb,test/plugin/test_buffer.rbandtest_output_as_buffered_retries.rbpass too.test/config/test_config_parser.rb,test_dsl.rbandtest/test_config.rbhave the same failures with and without this change on my machine (include paths relative to the working directory).Docs Changes:
fluent/fluentd-docs-gitbook#727
Release Note:
config: a fractional number before a size or time unit (
0.5s,1.5m,1.5k) is parsed as such instead of dropping the integer part