Skip to content

Fix JSONL parsing warnings and add missing model pricing#127

Merged
mike1858 merged 1 commit intomainfrom
fix/jsonl-parsing-and-model-pricing
Mar 8, 2026
Merged

Fix JSONL parsing warnings and add missing model pricing#127
mike1858 merged 1 commit intomainfrom
fix/jsonl-parsing-and-model-pricing

Conversation

@mike1858
Copy link
Member

@mike1858 mike1858 commented Mar 8, 2026

Claude Code analyzer:

  • Add catch-all #[serde(other)] variant to ClaudeCodeEntry enum to silently skip unknown entry types like 'last-prompt'
  • Add catch-all #[serde(other)] variant to ContentBlock enum to handle new block types like 'tool_reference' and 'redacted_thinking'
  • Make ToolResult.content optional to handle tool results with no content field
  • Add deserialize_u64_or_null helper to handle null token counts in Usage struct (OpenRouter sends null instead of omitting fields)

Model pricing:

  • Add model name normalization to get_model_info() that strips provider prefixes (minimax/, z-ai/, openrouter/, etc.) and handles :free and -free suffixes with $0 pricing
  • Add pricing for GPT-5.4, GLM-5, GLM-5-Code, GLM-4.5-Air, MiniMax M2.5, Step 3.5 Flash, Solar Pro 3, Aurora Alpha
  • Add date-suffixed aliases for new models

Reduces startup warnings from 130+ to 2 (unrelated to Claude Code).

Closes #126.

Summary by CodeRabbit

  • New Features
    • Added many new AI models and aliases plus free-tier support with automatic zero-cost handling.
  • Bug Fixes
    • More robust handling of missing/unknown content and entry types in API responses.
    • Token counts now tolerate null values (treated as zero) to avoid parsing errors.
  • Refactor
    • Navigation and input-handling logic tightened to reduce accidental actions and improve UI consistency.

@coderabbitai
Copy link

coderabbitai bot commented Mar 8, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 24a6391c-efe9-4c82-9628-ea40fd932f68

📥 Commits

Reviewing files that changed from the base of the PR and between bff9595 and c4f606b.

📒 Files selected for processing (5)
  • src/analyzers/claude_code.rs
  • src/analyzers/codex_cli.rs
  • src/analyzers/copilot.rs
  • src/models.rs
  • src/tui.rs
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/analyzers/codex_cli.rs

📝 Walkthrough

Walkthrough

Made Claude Code deserialization tolerant of null/unknown fields and optional tool result content. Expanded the in-code model registry with many new models, aliases, and free-tier handling via a refactored lookup. Minor pattern/guard refactors in several analyzers and the TUI; behavior preserved.

Changes

Cohort / File(s) Summary
Claude Code Deserialization
src/analyzers/claude_code.rs
ToolResult.content now Option<Content> with #[serde(default)]; added deserialize_u64_or_null to treat null as 0 and applied to Usage token fields; added #[serde(other)] Other variants for ContentBlock and ClaudeCodeEntry.
Model Index & Lookup
src/models.rs
Added many new model entries and aliases; introduced a private FREE_MODEL_INFO constant; added lookup_model helper, provider-prefix normalization, and suffix handling (e.g., :free, :extended, -free) to resolve or return zero-cost info.
Codex CLI Parsing (formatting)
src/analyzers/codex_cli.rs
Reformatted assistant-arm match/guard structure (syntactic rework only); emitted messages and cost logic unchanged.
Copilot Analyzer (pattern guard)
src/analyzers/copilot.rs
Replaced String match inner-block with a guard-based pattern to skip metadata prefixes (e.g., vscode-, file://, ssh-remote); behavior preserved.
TUI Navigation Guards & Flow
src/tui.rs
Switched navigation and Ctrl+T handling to guard-based patterns, moved per-tab reads and target_len calculations inside guarded scopes, and ensured redraw flag consistency; behavioral intent unchanged.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested reviewers

  • bl-ue

Poem

🐰 I hopped through JSON, nulls, and names,

stripped prefixes, mapped free-tier claims,
new models sprouted like carrots in rows,
unknown blocks welcomed where mystery grows,
a cheerful nibble — the parser happily tames.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Out of Scope Changes check ⚠️ Warning PR includes changes beyond #126: JSONL parsing fixes (claude_code.rs, codex_cli.rs, copilot.rs) and guard-based refactoring in tui.rs not mentioned in linked issue requirements. Review whether JSONL parsing improvements and tui.rs refactoring are intentional scope expansions or should be separated into distinct PRs with appropriate issue tracking.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed Title accurately describes the primary changes: fixing JSONL parsing issues and adding missing model pricing entries.
Linked Issues check ✅ Passed PR implements GPT-5.4 pricing with correct tiered structure ($2.50/$5.00 input, $15.00/$22.50 output, $0.25/$0.50 caching) and adds multiple related models as noted in objectives.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch fix/jsonl-parsing-and-model-pricing

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (1)
src/analyzers/claude_code.rs (1)

313-315: Consider keeping one low-noise signal for skipped schema types.

These catch-alls solve the warning storm, but they also make future Claude schema drift invisible. A per-run counter or a single aggregated warn_once when Other is hit would preserve observability without bringing back the old log spam.

Also applies to: 428-430

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/analyzers/claude_code.rs`:
- Around line 331-350: The current deserialize_u64_or_null function turns
explicit JSON nulls into 0 on deserialization, which causes Usage fields in
struct Usage to be treated as real zeroes during fingerprinting/merge; change
the model to preserve "unknown" vs zero by making these Usage fields Option<u64>
(keep deserialize_u64_or_null or rename to deserialize_option_u64 to return
Option<u64>), update deserialize logic to return None for null/missing and
Some(n) for numbers, and then adjust the dedup fingerprint and merge path
(functions like merge_message_into and any fingerprint computation) to use
presence/absence (None vs Some(0)) rather than raw numeric 0 when deciding
deduplication and summation so unknown counts are not treated as zeroes during
merge.

In `@src/models.rs`:
- Around line 997-1007: Add missing alias entries in MODEL_ALIASES to mirror the
newly added models in MODEL_INDEX: add self-referencing mappings for
"glm-5-code", "step-3.5-flash", "solar-pro-3", and "aurora-alpha" and also add
date-suffixed variants (e.g., "glm-5-code-YYYYMMDD" => "glm-5-code",
"step-3.5-flash-YYYYMMDD" => "step-3.5-flash", etc.) following the existing
alias patterns used for entries like "glm-5-20260211" => "glm-5"; update
MODEL_ALIASES in src/models.rs so lookups for both canonical and date-tagged
names resolve to the canonical model keys.
- Around line 1065-1068: The suffix branch that handles
base_name.strip_suffix("-free") currently returns
lookup_model(base).unwrap_or(&FREE_MODEL_INFO), which incorrectly uses the base
model's pricing when the lookup succeeds; change it so that whenever a "-free"
suffix is detected the function returns FREE_MODEL_INFO unconditionally (i.e.,
always return Some(&FREE_MODEL_INFO) from the strip_suffix("-free") branch
instead of calling lookup_model).
- Around line 1049-1053: The current branch handling
after_slash.strip_suffix(":free") calls
lookup_model(base).unwrap_or(&FREE_MODEL_INFO), which returns the base model's
real pricing when found; change it so :free always yields $0 pricing by either
(A) returning FREE_MODEL_INFO unconditionally instead of
lookup_model(...).unwrap_or(...), or (B) if you need to preserve other metadata
from lookup_model(base), create/return a copy of the found model with its
pricing fields zeroed (e.g., lookup_model(base).map(|m|
m.with_zero_pricing()).unwrap_or(&FREE_MODEL_INFO)); update the code around
after_slash.strip_suffix(":free"), lookup_model, and FREE_MODEL_INFO
accordingly.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 7b5554dd-6217-4ddc-929f-4ded08c39bd3

📥 Commits

Reviewing files that changed from the base of the PR and between ac4af92 and 557fa7a.

📒 Files selected for processing (2)
  • src/analyzers/claude_code.rs
  • src/models.rs

@mike1858 mike1858 force-pushed the fix/jsonl-parsing-and-model-pricing branch from 557fa7a to bff9595 Compare March 8, 2026 15:14
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
src/models.rs (1)

1011-1017: Consider adding date-suffixed aliases for consistency.

The new models step-3.5-flash, solar-pro-3, and aurora-alpha only have self-referencing aliases. Other models in the codebase include date-suffixed variants (e.g., glm-5-20260211glm-5). Consider adding similar aliases for consistency, though this is optional if these models don't use date-based versioning.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/models.rs` around lines 1011 - 1017, The three new model alias entries
("step-3.5-flash", "solar-pro-3", "aurora-alpha") currently only map to
themselves; add date-suffixed aliases consistent with the rest of models so
date-versioned names resolve to the base name. In src/models.rs, next to the
existing entries for step-3.5-flash, solar-pro-3, and aurora-alpha, add entries
like "step-3.5-flash-YYYYMMDD" => "step-3.5-flash", "solar-pro-3-YYYYMMDD" =>
"solar-pro-3", and "aurora-alpha-YYYYMMDD" => "aurora-alpha" (use the actual
date patterns used elsewhere), following the same map structure and ordering as
surrounding alias entries so date-suffixed lookups behave consistently.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/models.rs`:
- Around line 347-356: The "gpt-5.4" entry currently uses PricingStructure::Flat
and a single CachingSupport::OpenAI rate but the spec requires tiered pricing
and caching; update the "gpt-5.4" ModelInfo to use PricingStructure::Tiered (or
equivalent) with two tiers: ≤272k context -> input_per_1m: 2.50, output_per_1m:
15.00; >272k context -> input_per_1m: 5.00, output_per_1m: 22.50, and set
CachingSupport::OpenAI with two corresponding cached rates: 0.25 for ≤272k and
0.50 for >272k, preserving is_estimated: false and keeping the same "gpt-5.4"
key and ModelInfo structure.

---

Nitpick comments:
In `@src/models.rs`:
- Around line 1011-1017: The three new model alias entries ("step-3.5-flash",
"solar-pro-3", "aurora-alpha") currently only map to themselves; add
date-suffixed aliases consistent with the rest of models so date-versioned names
resolve to the base name. In src/models.rs, next to the existing entries for
step-3.5-flash, solar-pro-3, and aurora-alpha, add entries like
"step-3.5-flash-YYYYMMDD" => "step-3.5-flash", "solar-pro-3-YYYYMMDD" =>
"solar-pro-3", and "aurora-alpha-YYYYMMDD" => "aurora-alpha" (use the actual
date patterns used elsewhere), following the same map structure and ordering as
surrounding alias entries so date-suffixed lookups behave consistently.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 5a3b5c5a-1287-4835-9a04-ca00e4817215

📥 Commits

Reviewing files that changed from the base of the PR and between 557fa7a and bff9595.

📒 Files selected for processing (5)
  • src/analyzers/claude_code.rs
  • src/analyzers/codex_cli.rs
  • src/analyzers/copilot.rs
  • src/models.rs
  • src/tui.rs

Claude Code analyzer:
- Add catch-all #[serde(other)] variant to ClaudeCodeEntry enum to
  silently skip unknown entry types like 'last-prompt'
- Add catch-all #[serde(other)] variant to ContentBlock enum to handle
  new block types like 'tool_reference' and 'redacted_thinking'
- Make ToolResult.content optional to handle tool results with no
  content field
- Add deserialize_u64_or_null helper to handle null token counts in
  Usage struct (OpenRouter sends null instead of omitting fields)

Model pricing:
- Add model name normalization to get_model_info() that strips provider
  prefixes (minimax/, z-ai/, openrouter/, etc.) and handles :free and
  -free suffixes with $0 pricing
- Add pricing for GPT-5.4, GLM-5, GLM-5-Code, GLM-4.5-Air,
  MiniMax M2.5, Step 3.5 Flash, Solar Pro 3, Aurora Alpha
- Add date-suffixed aliases for new models

Reduces startup warnings from 130+ to 2 (unrelated to Claude Code).
@mike1858 mike1858 force-pushed the fix/jsonl-parsing-and-model-pricing branch from bff9595 to c4f606b Compare March 8, 2026 15:25
@mike1858
Copy link
Member Author

mike1858 commented Mar 8, 2026

@coderabbitai Review.

@coderabbitai
Copy link

coderabbitai bot commented Mar 8, 2026

@mike1858 Sure! I'll start reviewing PR #127 right away.

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@mike1858 mike1858 merged commit 31066b6 into main Mar 8, 2026
6 checks passed
@mike1858 mike1858 deleted the fix/jsonl-parsing-and-model-pricing branch March 8, 2026 16:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add pricing for GPT 5.4

1 participant