feat: Major features v1.3.0 - Cost Tracking, Caching, Multi-User, A/B Testing #7
Conversation
## New Features ### Core Modules Added - **Cost Tracking** (metrics.py): Per-request cost calculation, budget limits, alerts - **Request Caching** (cache.py): LRU cache with TTL, invalidation strategies - **Multi-User Support** (users.py): Per-user token/cost limits, model access control - **A/B Testing** (ab_testing.py): Model comparison with statistical analysis ### Bug Fixes - OAuth graceful fallback when credentials missing - Router initialization race condition fixed - Request metadata store memory leak fixed (TTL + max size) - Model reload thrashing fixed with cooldown - Default config now works out of the box ### Enhancements - Health check endpoint: ccproxy status --health - Shell integration: ccproxy shell-integration --shell [bash|zsh|fish] - OAuth token background refresh - Configuration validation at startup - Global tokenizer cache for performance - Request retry with exponential backoff ### Documentation - docs/troubleshooting.md: Common issues and solutions - docs/architecture.md: System design with ASCII diagrams - docs/examples.md: Configuration examples for various use cases - README.md updated with all new features ### Tests - 356 tests passing - New test files: test_metrics, test_cache, test_users, test_ab_testing, etc. - Coverage improved to 71%
|
Before reviewing 6,000 lines: can you write up, in your own words, what problem you're solving and what value this adds? |
|
Hey, You're right, 6k lines was way too much. My bad. Been thinking about this - wanna break it into small PRs instead. Each one solving a specific thing. First one would be config validation with helpful errors. Fixes that "default config doesn't work" issue. Pretty small change, maybe couple hundred lines, no breaking stuff. Basically when someone messes up the YAML they get an actual helpful message instead of a stack trace. After that got some other stuff lined up (oauth race condition, connection pooling etc) but lets start small. Want me to go ahead with the config one? |
2 participants
Summary
This PR adds major new features to ccproxy v1.3.0, including cost tracking, request caching, multi-user support, and A/B testing framework. All features are fully tested with 356 tests passing.
New Modules
metrics.pycache.pyusers.pyab_testing.pyBug Fixes
Enhancements
ccproxy status --healthfor health metricsccproxy shell-integrationfor shell aliasesDocumentation
docs/troubleshooting.md- Common issuesdocs/architecture.md- System designdocs/examples.md- Configuration examplesTests