Conversation
…rsing Key optimizations: - Rewrite block string parser: direct string processing instead of sub-lexer + Vec<Vec<Token>> (~10%) - Compact Span: u32 start+len (8 bytes) instead of Range<usize> (16 bytes), add Copy (~3%) - Field: consume-then-check for alias instead of peek(1) (~2%) - Optimize next_if_* methods: peek+consume in single buffer operation (~1%) - Lazy depth_limiter.bump(): only bump when optional elements exist - Add Copy to DepthLimiter + preallocate Vec capacity in DefinitionDocument Also adds benchmarks for ExecutableDocument parsing with a large fixture, and updates downstream crates to use Copy semantics on Span (clone → deref). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
| let arguments = if VariableArguments::is_match(tokens) { | ||
| Some(VariableArguments::from_tokens( | ||
| tokens, | ||
| depth_limiter.bump()?, | ||
| )?) | ||
| } else { | ||
| None | ||
| }; |
There was a problem hiding this comment.
Should we just change the return type of TryFromTokens::try_from_tokens to return the transposed type and have this be the implementation? I think we call transpose almost every time we call this
There was a problem hiding this comment.
one caller in directives.rs uses Option<Result<...>> 😓 so we could do that but we'd have to update the usage globally. I don't think this is so bad because it makes it obvious that we're manually avoiding calling depth_limiter.bump() when the branch isn't taken?
There was a problem hiding this comment.
Could we change the implementation of TryFromTokens to do the bump internally?
There was a problem hiding this comment.
It becomes a bit messy since there's a few cases which don't bump so it would simplify most of the callers but make a few more complex. Just changing the return type from Option -> Result we can at least get rid of the manual transpose()
There was a problem hiding this comment.
Refactored this a bit
| start: u32, | ||
| len: u32, |
There was a problem hiding this comment.
Why u32 instead of usize?
There was a problem hiding this comment.
Makes the struct smaller so all the .clone() calls become Copy and avoids heap allocations
6c780f0 to
6e20145
Compare
~16% faster schema parsing, ~12% faster executable parsing
Key optimizations:
Also adds benchmarks for ExecutableDocument parsing with a large fixture, and updates downstream crates to use Copy semantics on Span (clone → deref).
Note: this is a manually curated (with the help of Claude) and cleaned up version of an
/autoresearchrun