Conversation
fix: Preallocation bug in example. docs: Add performance-related hints. docs: Use new built-in value decoding methods.
FuzzReadNumber cross-checks ReadNumber against refScan, a naive reference implementation of the RFC 8259 number grammar. It asserts the number of consumed bytes, the ReturnCodeInteger/ReturnCodeNumber classification and that the string and []byte instantiations agree. Also close three gaps in the table tests: - Exponents spanning more than one 8-byte batch. - Negative variants of the TestReadNumberErr inputs. - TestReadNumberEmpty for the panic on empty input.
The signed parsers inferred overflow from the sign of the result, which only holds if the value wraps once. The longest accepted digit strings can wrap several times and land back in the correct sign, so Token.Int8 returned (44, nil) for the JSON value 300 instead of ErrOverflow. The cases that can overflow now compute in the next wider type and bound-check before narrowing. U8, U16, U32, U64 and I64 were unaffected. test: Add FuzzAtoi, exhaustive 8/16-bit and boundary tests against strconv, and regression rows for the affected ranges.
Every type switch on the type parameter is now exhaustive, which removes
the unreachable default branches in Scan, ScanOne, Validate and
ValidateOne and fixes variantCheckAndReplace, which silently skipped
escaping for derived types.
!fix: Unescape object keys before RFC-6901 encoding them. A pointer
references the decoded member name, hence `{"a\/b":1}` and `{"a/b":1}`
now both yield /a~1b.
test: Add exhaustive, boundary and fuzz tests for keyescape.
BREAKING CHANGE: Types derived from string or []byte such as
json.RawMessage no longer satisfy the type parameter constraint and must
be converted at the call site. Pointer and ViewPointer now encode the
decoded key, hence keys containing JSON escape sequences produce
different pointers than before.
Convert the source to a string once at the API boundary, run the engines on it, then convert back. toStr and fromStr copy nothing and allocate nothing. The public API and its behavior don't change. This drops the []byte copies of the engines and of jsonnum.ReadNumber and strfind.EndOfWhitespaceSeq. validate takes no callback, so it becomes one non-generic function. scan and tokenize stay generic, but both their copies are now string-sized and identical. Machine code shrinks from 55675 to 29711 bytes (-47%). Validation is ~10% faster on []byte and ~6% on string. Scan and tokenize gain a few percent on both.
- Parser -> Scanner - NewParser -> NewScanner - DefaultStackSizeIterator -> DefaultStackSizeScanner
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add tokenization capabilities for higher efficiency decoding.