palette/moreland: fix floating-point arithmetic error in Palette#799
palette/moreland: fix floating-point arithmetic error in Palette#799sbinet merged 3 commits intogonum:masterfrom Juneezee:issue-798
Conversation
Fixes #798. Signed-off-by: Eng Zer Jun <[email protected]>
sbinet
left a comment
There was a problem hiding this comment.
Thanks for the PR.
I just have minor requests.
See below:
Reference: #799 (review) Signed-off-by: Eng Zer Jun <[email protected]>
Reference: #799 (review) Signed-off-by: Eng Zer Jun <[email protected]>
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## master #799 +/- ##
==========================================
+ Coverage 70.78% 72.26% +1.48%
==========================================
Files 60 59 -1
Lines 5291 7353 +2062
==========================================
+ Hits 3745 5314 +1569
- Misses 1350 1852 +502
+ Partials 196 187 -9 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
kortschak
left a comment
There was a problem hiding this comment.
Thanks.
Please send a change to gonum/gonum with the commit message "A+C: add Eng Zer Jun" (please correct this if I got your name wrong) adding yourself to AUTHORS and CONTRIBUTORS.
@kortschak My name is already in the AUTHORS and CONTRIBUTORS file 😊 |
|
@Juneezee Ah, thanks. You show up as first time contributor here, but this is because the repo is different. Thanks for the change. |
Fixes #798.
This PR fixes a panic in the
Palettemethod caused by a floating-point precision error, where the computed valuevslightly exceeds the upper boundl.max.Debugger screenshot
Problem
As shown in the above screenshot, we have the following values:
l.min = 0.3402859786606234l.max = 15.322841335211892n = 15delta = 1.0701825254679478i = 14v = 15.322841335211894(note herevis0.000000000000002greater thanl.max)The expected calculation of
delta:This confirms that
deltais correct and has no floating-point error.Next, the expected calculation of
v:Ideally,
vshould be exactly equal tol.max. However, due to a small floating-point precision error of0.000000000000002,vbecomes slightly greater thanl.max, triggering the panic incheckRangefunction:plot/palette/moreland/luminance.go
Lines 116 to 118 in 2815ea1
Solution
Use the built-in
minfunction to ensurevdoes not exceedl.max. Ifvbecomes greater thanl.maxdue to floating-point precision errors, it will be capped atl.max.