Merge pull request #151 from Pyronewbic/dev

deps: bump 6 packages, fix protobufjs/qs vulns; docs for v1.5.0

Author: Pyronewbic

CHANGELOG.md

@@ -1,5 +1,18 @@
 # Changelog
 
+## 1.5.0 (2026-05-25)
+
+- Admin panel: tabbed layout (Keys / Analytics / Errors / Funnel)
+- Analytics dashboard: daily volume chart, by-tier/by-path bars, top queries, avg latency
+- Error monitoring: type filter dropdown, 30-day TTL on error-logs, process-level handlers (unhandledRejection, uncaughtException)
+- User funnel tracking: signup/firstSearch/firstGrade/firstPortfolioAdd milestones, GET /api/funnel
+- Set browser: language filter (?lang=en|jp), 353 sets (up from 238)
+- Card database: on-demand sync (POST /api/card-database/sync), no more TTL-based refresh on cold start
+- PokeWallet integration: JP card rarity enrichment (SAR/IR/UR/HR/SR/RR/AR/ACE/CR)
+- Card ID normalization: deduplicate EN/JA zero-padded IDs for older sets (neo, base)
+- Cold start fix: card DB + set metadata fully loaded before accepting connections
+- 488 tests (313 unit, 104 API, 71 smoke)
+
 ## 1.4.0 (2026-05-20)
 
 - SBOM attestation: Syft SPDX generated from built container image, cosign-attested to digest

README.md

@@ -1,6 +1,6 @@
 # <img src="logos/casecomp-logo.svg" width="32" height="32" alt="Casecomp logo" /> Casecomp
 
-[![Version](https://img.shields.io/badge/version-1.4.0-d9b676)](CHANGELOG.md)
+[![Version](https://img.shields.io/badge/version-1.5.0-d9b676)](CHANGELOG.md)
 [![CI](https://github.com/Pyronewbic/casecomp/actions/workflows/ci.yml/badge.svg)](https://github.com/Pyronewbic/casecomp/actions/workflows/ci.yml)
 [![Deploy](https://github.com/Pyronewbic/casecomp/actions/workflows/deploy.yml/badge.svg)](https://github.com/Pyronewbic/casecomp/actions/workflows/deploy.yml)
 [![License](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
@@ -26,8 +26,8 @@ Search any Pokemon card across four marketplaces in one query. Get live prices,
 - **PSA grading signals** - population data, difficulty, gem 10%, recommended submission tier
 - **Slab comparison** - compare PSA 10 / BGS 9.5 / TAG 10 prices across sources
 - **Portfolio** - track cards, ROI, value over time, grading opportunities
-- **Admin** - API key management, stats KPIs, error log at `/admin`
-- **Set browser** - browse 238 sets with logos, rarity filters, collection tracking (owned/missing)
+- **Admin** - API key management, analytics dashboard, error monitoring with type filters, user funnel tracking at `/admin`
+- **Set browser** - browse 353 sets with logos, rarity filters, language filter (EN/JP), collection tracking (owned/missing)
 - **REST API** - authenticated endpoints with rate limiting, per-key caching, OpenAPI spec
 - **Claude Code skill** - `/casecomp` for plain-English card search
 - **Chrome extension** - queue auto-join for Pokemon Center, Walmart, Costco, Target drops
@@ -84,7 +84,8 @@ Full reference: [api.casecomp.xyz/docs](https://api.casecomp.xyz/docs)
 
 ```bash
 # ── Public (no key) ──────────────────────────────────────────
-curl "https://api.casecomp.xyz/api/sets"                        # browse 238 sets
+curl "https://api.casecomp.xyz/api/sets"                        # browse 353 sets
+curl "https://api.casecomp.xyz/api/sets?lang=en"                 # EN sets only
 curl "https://api.casecomp.xyz/api/sets/sv06"                   # cards in a set
 curl "https://api.casecomp.xyz/api/autocomplete?q=umbreon"      # card search (29K cards)
 curl "https://api.casecomp.xyz/api/health"                      # service status
@@ -117,7 +118,7 @@ curl -X POST -H "Authorization: Bearer $CASECOMP_KEY" \
 
 ### Public endpoints (no key)
 
-`GET /api/health` | `GET /api/demo` | `GET /api/sitemap` | `GET /api/autocomplete` | `GET /api/sets` | `GET /api/sets/:setCode` | `GET /docs` | `GET /docs/spec.json` | `?demo=true` (sample data) on search/sold/arbitrage/price-history
+`GET /api/health` | `GET /api/demo` | `GET /api/sitemap` | `GET /api/autocomplete` | `GET /api/sets` | `GET /api/sets/:setCode` | `GET /api/funnel` | `GET /docs` | `GET /docs/spec.json` | `?demo=true` (sample data) on search/sold/arbitrage/price-history
 
 ## Security
 
@@ -199,7 +200,9 @@ All caches use Firestore (shared across Cloud Run instances, persists across dep
 | `cache-ebay-sold` | 24 hours | eBay sold comp results |
 | `price-history` | permanent | Sold comp prices over time |
 | `api-keys` | permanent | Developer API keys (hashed) |
-| `error-logs` | permanent | API errors with request IDs |
+| `error-logs` | 30 days (TTL) | API errors with request IDs |
+| `card-database-cache` | permanent (on-demand sync) | TCGdex card index (~29K cards) |
+| `user-milestones` | permanent | User funnel milestones (signup, first search, etc.) |
 
 ## Infrastructure
 
@@ -213,12 +216,12 @@ Load unpacked from `extension/` in `chrome://extensions`.
 
 ## Tests
 
-486 tests across three layers. CI required checks: unit + codeql. Smoke is non-blocking.
+488 tests across three layers. CI required checks: unit + codeql. Smoke is non-blocking.
 
 | Suite | Count | Command | Covers |
 |-------|------:|---------|--------|
-| **Unit** | 312 | `yarn test:unit` | Filters, grading, query builder, card identity, condition detection, image preprocessing, email alerts, portfolio ROI, CSV export, autocomplete, JWT auth, price trends, RASP detection |
-| **API** | 103 | `yarn test:api` | Search, sold, PSA, grade, auth, admin keys, arbitrage, price history, alerts, share pages, portfolio CRUD, card view, upload-url, analytics, collection tracking |
+| **Unit** | 313 | `yarn test:unit` | Filters, grading, query builder, card identity, condition detection, image preprocessing, email alerts, portfolio ROI, CSV export, autocomplete, JWT auth, price trends, RASP detection |
+| **API** | 104 | `yarn test:api` | Search, sold, PSA, grade, auth, admin keys, arbitrage, price history, alerts, share pages, portfolio CRUD, card view, upload-url, analytics, collection tracking, lang filter, card-database sync |
 | **Smoke** | 71 | `yarn test:smoke` | API root page, detail panel, tabs, PSA stats, arbitrage, mobile viewport, portfolio, autocomplete, search filters |
 
 ## Contributing

docs/env-vars.md

@@ -56,6 +56,7 @@ Copy **`.env.example`** to **`.env`** and fill in the required values.
 | `GOOGLE_OAUTH_CLIENT_ID` | *(none)* | Google OAuth client ID for sign-in (popup flow) |
 | `TOGETHER_API_KEY` | *(none)* | Together AI key for card detection (GLM-4.6V, falls back to Claude Sonnet) |
 | `RASP_MODE` | `monitor` | RASP enforcement mode: `monitor` (log only) or `block` (reject malicious requests) |
+| `POKEWALLET_API_KEY` | *(none)* | PokeWallet API key for JP card rarity data (free tier: 100 req/hr, 1K/day) |
 
 ## Email notifications
 

docs/internals.md

@@ -23,7 +23,7 @@ lib/
     auth.js           Google OAuth token verification, JWT (HS256)
     api-keys.js       Developer key management
   cards/
-    card-database.js  TCGdex card DB (29K EN+JP cards), set browser, rarity
+    card-database.js  TCGdex card DB (29K EN+JP cards), set browser, rarity, PokeWallet JP enrichment
     card-identity.js  Canonical IDs, set resolution, SET_TOTAL_MAP
     demo.js           Sample data (3 multi-source cards)
     grading-dataset.js  ML slab image collection from sold listings (eBay, magi, search)
@@ -50,8 +50,8 @@ public/admin/         Admin panel (keys, stats, errors)
 extension/            Chrome extension: queue auto-join, drop intel
 terraform/            GCP infra (Cloud Run, Firestore, LB, CDN, Scheduler)
 test/
-  unit-test.js        312 unit tests
-  api-test.js         103 API integration tests
+  unit-test.js        313 unit tests
+  api-test.js         104 API integration tests
   smoke-test.js       71 Playwright UI smoke tests
 ```
 
@@ -66,7 +66,7 @@ test/
 - **Dashboard**: static files from `public/` served at `/` and `/admin`.
 - **Docs**: Swagger UI at `/docs`, spec at `/docs/spec.json`.
 
-On startup: eBay OAuth token pre-fetched, TCGdex card database loaded from Firestore cache (24h TTL), set names + logos loaded in parallel.
+On startup: eBay OAuth token pre-fetched, TCGdex card database loaded from Firestore cache (no TTL, on-demand sync via `POST /api/card-database/sync`), set metadata (names + logos) loaded in parallel. Server waits for card DB and set metadata before accepting connections.
 
 ## Multi-region deployment
 
@@ -98,8 +98,10 @@ All caches use Firestore (shared across Cloud Run instances, single region). No
 | `cache-ebay-sold` | 24 hours | eBay sold comp results |
 | `price-history` | permanent | Sold comp prices over time |
 | `api-keys` | permanent | Developer API keys (hashed) |
-| `error-logs` | permanent | API errors with request IDs |
-| `api-analytics` | 30 days | Request analytics (tier, path, latency) |
+| `error-logs` | 30 days (TTL) | API errors with request IDs, type filter |
+| `api-analytics` | 30 days | Request analytics (tier, path, latency, daily/byStatus) |
+| `card-database-cache` | permanent | TCGdex card index (~29K cards), synced on-demand |
+| `user-milestones` | permanent | User funnel milestones (signup → search → grade → portfolio) |
 | `grading-dataset` | permanent | ML training data: slab images + parsed grades |
 
 Stale-while-revalidate on active listings for owner key. File-based cache (`.json` files) still used by the CLI.
@@ -152,6 +154,8 @@ Use `--refresh` to delete all cache files before a run.
 | `grade-logs` | source + createdAt desc | Filter grades by source |
 | `api-analytics` | userId + ts desc | Per-user analytics |
 | `price-history` | cardKey + recordedAt desc | Card price history |
+| `price-history` | cardId + recordedAt desc | Card price history (by canonical ID) |
+| `error-logs` | type + createdAt desc | Error filtering by type |
 
 **ML dataset pipeline**: graded slab images (PSA/BGS/CGC/TAG) are passively collected into `grading-dataset` Firestore collection from multiple sources: eBay sold (via `track-prices` and `/api/sold`), magi sold (via `track-prices`), and any search with sold results (`/api/search`). Grade is parsed from listing title or grade label. `GET /api/grading-dataset/stats` monitors collection progress.
 
@@ -163,7 +167,7 @@ Three workflows: `ci.yml` (all checks), `deploy.yml` (build + sign + deploy), `t
 
 | Job | What | Required? |
 |-----|------|-----------|
-| unit | 312 unit tests | Yes |
+| unit | 313 unit tests | Yes |
 | smoke | 71 Playwright smoke tests | No (continue-on-error) |
 | codeql | SAST for JavaScript/TypeScript | Yes |
 | scan | SBOM (Syft) + Grype vulnerability scan | No |