Skip to content

Add NVIDIA mirror site#55

Open
KaKituken wants to merge 1 commit into
aiming-lab:mainfrom
KaKituken:add-nvidia-mirror
Open

Add NVIDIA mirror site#55
KaKituken wants to merge 1 commit into
aiming-lab:mainfrom
KaKituken:add-nvidia-mirror

Conversation

@KaKituken

Copy link
Copy Markdown

Mirrored site

NVIDIAhttps://www.nvidia.com/
Added as the 16th WebHarbor site, served on port 40015.

What's included

  • sites/nvidia/ — Flask + SQLAlchemy app modeling NVIDIA's product world: a GPU/hardware catalog with full spec sheets, a spec-comparison tool, a driver-download finder, a news/Newsroom section, scored token-overlap search (every listing + a global /search), accounts, cart/checkout, wishlist, and product reviews. 22 Jinja2 templates in NVIDIA's black + green (#76b900) design.
  • 25 real products across 5 categories with real published specs; 8 real NVIDIA Newsroom articles; 9 driver entries.
  • Idempotent seeding (function-level gates) with the 4 canonical benchmark users (alice.j@test.com, bob.c@test.com, carol.d@test.com, david.k@test.com; password TestPass123!). Byte-identical reset verified.
  • 20 WebVoyager tasks in sites/nvidia/tasks.jsonl.
  • Registered in the three required places: websyn_start.sh, control_server.py, Dockerfile (EXPOSE … 40015).

Seeded rows per major model

model rows model rows
products 25 reviews 5
articles 8 orders 2
drivers 9 order_items 2
users 4 wishlist_items 3

Products by category: GeForce Gaming 10 · Data Center 6 · Studio/Professional 4 · Embedded 3 · Consumer Devices 2 (GeForce RTX 50/40, RTX PRO & Ada workstation, H200/H100/A100/B200/GH200/L40S, Jetson, SHIELD).

Tasks

20 tasks in sites/nvidia/tasks.jsonl (WebVoyager schema), spanning spec lookup, browse/filter, the comparison tool, driver downloads, cart + checkout, account editing, wishlist, reviews, news, and newsletter — including 5 multi-step tasks (e.g. "among all GeForce Gaming cards with ≥16 GB of memory, find the least expensive and add it to your cart") and 2 disambiguation tasks.

Byte-identical reset

$ curl -X POST http://localhost:8201/reset/nvidia
{"site":"nvidia","ready":true,...}

$ docker exec wh-test md5sum /opt/WebSyn/nvidia/instance/nvidia.db \
                            /opt/WebSyn/nvidia/instance_seed/nvidia.db
7bb43da0a67ef9ecce4396a93daaf9df  .../instance/nvidia.db
7bb43da0a67ef9ecce4396a93daaf9df  .../instance_seed/nvidia.db

All 6 seed_*() functions early-return as a whole; all 16 sites return 200.

Visual fidelity vs. the real site

Mirror rendered at localhost:40015 (compare against https://www.nvidia.com/). Real GPU renders and Newsroom thumbnails harvested from nvidia.com.

Home Product detail (specs)
home detail
Products listing Compare tool
products compare
More (drivers, news)

drivers
news

Paired HuggingFace assets PR

https://huggingface.co/datasets/ChilleD/WebHarbor/discussions/38 — uploads nvidia.tar.gz (instance_seed/nvidia.db + static/images: 25 product renders + 8 news thumbnails).

.assets-revision already pins revision: main, so no bump is needed; once the dataset PR merges, scripts/fetch_assets.sh pulls the nvidia assets automatically. Merge the HF dataset PR before this one so a clean-clone build can fetch the assets.

Notes for human review

  • Catalog is 25 products — below the generic 50–200 guideline, but deliberate: this is NVIDIA's actual current lineup. Padding to 200 would mean inventing fake GPUs and hurting fidelity. Breadth checks pass (a "geforce rtx" search returns 14 cards across two series).
  • Two products reuse a same-series render (RTX 4000 Ada → RTX 5000 Ada image; Jetson Orin NX → Orin Nano image) because NVIDIA publishes no standalone render for those SKUs; the families are visually near-identical.
  • News article bodies are 2–3 sentences synthesized from each article's real excerpt (titles, dates, categories, and images are the real scraped values).
  • Fixed during Phase 3: the footer newsletter form was missing its CSRF token (subscriptions silently failed) — now fixed and verified end-to-end via Playwright.

🤖 Generated with Claude Code

New WebHarbor mirror of nvidia.com on port 40015 (16th site).

- sites/nvidia/: Flask + SQLAlchemy app — GPU/hardware catalog (Product with
  full spec sheet, Article, Driver, Review, CartItem, Order, WishlistItem,
  NewsletterSubscriber). Routes: browse/filter, product detail with specs,
  spec-comparison tool, driver-download finder, news, scored token-overlap
  search (all listings + global), auth, account, cart/checkout, wishlist,
  reviews, newsletter. 22 Jinja2 templates, NVIDIA black + green design.
- 25 real products across 5 categories (GeForce RTX 50/40, RTX PRO / Ada
  workstation, data-center H200/H100/A100/B200/GH200/L40S, Jetson, SHIELD)
  with real published specs; 8 real NVIDIA Newsroom articles; 9 drivers.
- Idempotent seeding (function-level gates) with the 4 canonical benchmark
  users (alice.j@test.com et al.); byte-identical reset verified.
- 20 WebVoyager tasks in sites/nvidia/tasks.jsonl.
- Registered in websyn_start.sh, control_server.py, Dockerfile (EXPOSE
  ...40015).
- Re-applies the .gitignore fix: scraped_data/ and instance/ patterns had
  inline comments that gitignore treats as part of the pattern.

Heavy assets (instance_seed/nvidia.db, static/images/) ship via the
ChilleD/WebHarbor HF dataset, not this repo.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@DEM1TASSE

Copy link
Copy Markdown

Reviewer verifiers → submitted as #58

The deterministic verifiers + judge rubrics for this site are up as #58, which bundles
your NVIDIA site commit (authorship preserved) with my verifier commit so both of us stay
credited. Full review below.


Review — nvidia (PR #55)

Reviewer: DEM1TASSE · Site author: @KaKituken · Date: 2026-07-03
Verdict: APPROVE AFTER CHANGES. The build is mechanically sound and the tasks are leak-free and
deterministically gradable, but beyond the 3 release-mechanics gating items there is one
substantive site-fidelity finding: the mirror models nvidia.com as a generic e-commerce store
rather than the marketing/ecosystem site it actually is (see §3, "Site-fidelity caveat").

Reviewed by building the image and driving a real Chromium (Playwright) against the mirror on
alt ports (control plane :8202, sites :41000-41015). Since the paired HF assets PR is not
yet merged, assets were fetched from the HF discussion #38 PR ref for this review.


1. Mechanical checks — PASS (3 gating items)

Check Result
Syntax (py_compile app.py / seed_data.py / _health.py) OK
Idempotent seeding all 6 seed_*() gated at function level (Product.query.count()>0, etc.)
Site isolation no cross-site imports
Path hygiene BASE_DIR-relative, no absolute paths
Port registration nvidia = index 15 → 40015, self-consistent across websyn_start.sh / control_server.py / Dockerfile
Build succeeds, 3.29 GB
HTTP 200 sweep 16/16 sites return 200
Control plane /health ok, nvidia alive (port 40015)
Byte-identical reset md5 match, repeatable (7bb43da…)
reset-all all 16 sites in 0.97 s

Gating items before merge

  1. F1 — Port 40015 collides with the already-merged Merriam-Webster. This branch predates
    Merriam's merge, so both claim index 15 (hence the PR shows CONFLICTING). Rebase onto current
    main and move nvidia to index 16 / port 40016: websyn_start.sh SITES, control_server.py
    SITES, Dockerfile EXPOSE …-40016, and the web port in sites/nvidia/tasks.jsonl
    (40015 → 40016). The verifiers are port-agnostic (they match URL paths, not the host port).
  2. F2 (MAJOR) — .assets-revision pins the mutable revision: main, not an immutable HF
    merge SHA.
    Non-reproducible build. Pin the actual merge commit once HF discussion Add B&H Photo WebHarbor mirror #38 lands.
    (Same issue MufanQiu flagged on phys_org feat(phys_org): add Phys.org mirror site #6.)
  3. F3 — Paired HF assets PR (discussion Add B&H Photo WebHarbor mirror #38) is unmerged, so nvidia.tar.gz is absent from
    HF main; a clean-clone build cannot fetch the DB/images today. Merge the HF PR before
    this one (author acknowledged this in the PR body).

NIT

  • Boot banner in websyn_start.sh hardcodes "Starting 15 sites on ports 40000-40014" but
    actually launches 16 (nvidia → 40015). Cosmetic; update the string.

2. Visual fidelity — PASS

Strong fidelity to nvidia.com. Black + green (#76b900) brand throughout, real product renders
and news thumbnails (no placeholders).

  • Homepage: header + nav (Gaming / Studio / Data Center / Robotics / Drivers / News), search,
    Login, Cart; RTX 5090 hero with a real GPU render; category pills; Featured Products grid with
    real images + prices; Latest News with real thumbnails and dates; 3-column footer.
  • Product detail (e.g. RTX PRO 6000 Blackwell): breadcrumb, image, price, In-Stock,
    Add-to-Cart / Save-to-Wishlist / Compare, a full Tech Specs table (Architecture, CUDA
    Cores, Memory, Bandwidth, TDP, Interface), Reviews, Related Products.
  • Drivers finder (3 dropdowns), Compare tool, Newsroom all render cleanly.

3. Functional depth — PASS on the individual flows; see the site-fidelity caveat below

Every flow driven via Playwright:

  • Auth: alice.j@test.com / TestPass123! logs in → redirects to /account.
  • Search: multi-word (RTX 5080) returns results — not strict-AND broken.
  • Browse: 25 products across 5 categories; detail links resolve.
  • Drivers filter: series/branch/OS dropdowns → returns a specific version (e.g. 566.36).
  • Compare tool: /compare?ids=… renders a side-by-side spec table (CUDA 21,760 vs 16,384).
  • Cart / Wishlist / Checkout / Review / Profile edit / Newsletter: routes present and
    exercised (see grading contract, §5 — DB after-state confirms each write persists).

Site-fidelity caveat (significant — please address)

The individual pages render faithfully, but the site is modeled as a generic e-commerce store,
which is not what nvidia.com is.
The real site — the GeForce section alone spans ~25–30 distinct
page types — is a marketing + ecosystem + support site: a rich per-series landing page (the
real /50-series/ page carries Blackwell-architecture, interactive performance charts, DLSS 4.5,
Reflex 2, RTX-AI-PC, and Creators sections with embedded video), standalone technology pages
(DLSS / Ray Tracing / Reflex / G-SYNC), an ecosystem (GeForce NOW, NVIDIA Studio, Omniverse,
Broadcast, RTX Remix), and device finders (gaming/Studio laptops, desktops, monitors). The
mirror flattens all of that into catalog → product detail → compare → drivers → news.

Two concrete consequences:

  1. Navigation is much easier than on the real site. The real difficulty of nvidia.com is
    choosing among many similarly-named landing pages and digging specs out of expandable
    marketing sections; here the agent just browses a clean store. None of the per-series landing
    pages or technology/ecosystem surfaces exist.
  2. The checkout flow is invented. nvidia.com does not sell GPUs via a cart/checkout — it uses
    "Where to Buy" → third-party retailers. Tasks T11/T12/T17 (add-to-cart / place-order)
    therefore exercise a flow the real site doesn't have — a fidelity mismatch, not just a
    simplification.

Recommendation (not a hard blocker; tasks remain valid and deterministically gradable): add at
least the per-series landing pages (and fold a marketing section or two onto the solve path so
navigation tasks have real depth), and re-anchor the commerce tasks onto a "Where to Buy" /
wishlist flow that matches the real site rather than a direct checkout. This is a larger ask than
the mechanical nits and is the main reason this is APPROVE AFTER changes rather than a clean pass.


4. Task quality — PASS (no leaks, good distractors, solid difficulty spread)

All 20 tasks are solvable through the mirror UI with deterministic, DB-backed answers.

  • No answer leaks. Spec values (memory / CUDA / TDP) appear only on product detail pages,
    never on the /products listing or /search results (verified: 32 GB, GDDR7, 21760,
    CUDA, TDP, 141 GB all absent from the listing; searching 5090 does not surface 32 GB).
    Read tasks therefore require a detail-page hop.
  • Anti knowledge-shortcut. Driver versions (566.36 / 566.14) are the mirror's own frozen
    builds — a frontier model can't recall them, so tasks 9/10 force real navigation.
  • Distractors present. T17 ("cheapest GeForce Gaming with ≥16 GB") = RTX 5060 Ti ($429); the
    $299 RTX 5060 / RTX 4060 (8 GB) and $549 RTX 5070 (12 GB) are genuine near-misses that must be
    excluded — a real reasoning task, not first-result-click.
  • Difficulty spread. Easy single-spec lookups (T0–T2), medium compare/filter (T6–T10),
    multi-step stateful (checkout T11, review T15, constraint-cart T17). ≥3 tasks need ≥5 actions.

Findings for the author

  • T5 homepage leak. The Jetson Orin Nano Super Developer Kit appears at $249 in the
    homepage "Featured Products" grid
    , so the agent can answer T5 ("price of the Jetson Orin
    Nano Super") straight off the homepage without ever visiting the Embedded category. In the
    full agent run the agent did exactly this on every attempt. The verifier's nav-gate + rubric
    correctly reject the homepage shortcut (so grading is safe), but the task is trivially
    shortcuttable — fix: drop the Jetson from the homepage featured grid, or reword T5 onto an
    Embedded-page-specific fact.
  • T9/T10 driver versions are not series-specific (Game Ready Win11 = 566.36 for RTX 30/40/50
    alike), so T9's answer is reachable by picking any Game Ready/Win11 row. Still deterministic
    and navigation-gated — acceptable, but the author could differentiate versions per series.

5. Grading contract (reviewer deliverable)

I authored a deterministic verifier per task under sites/nvidia/verify/ and recorded
verifier_path + judge_rubric in every tasks.jsonl row (no answer key — ground truth
lives only in the verifiers). Design is deterministic-first: (1) trajectory navigation
(anti-shortcut), (2) SQLite DB after-state for the 8 stateful tasks, (3) answer vs frozen
ground truth, with the LLM only as an anchored consistency check.

Frozen answer key (from instance_seed/nvidia.db): T0 32 GB GDDR7 · T1 10,752 · T2 450 W ·
T3 RTX 5060 $299 · T4 H200 · T5 $249 · T6 5090 +5,376 · T7 5080 (960 GB/s) · T8 RTX PRO 6000 96 GB ·
T9 566.36 · T10 566.14 · T11 order w/ 5080 · T12 4060 in cart · T13 5070 Ti wishlist · T14 country=Germany ·
T15 5★ 'Incredible' on Jetson · T16 remove RTX PRO 6000 · T17 5060 Ti in cart · T18 2026-06-16 · T19 gamer42 subscribed.

Validation of the grading contract

Ran the official react agent (agent_demo/agent.py, gpt-4o) across all 20 tasks with a DB reset
before each, then graded every run BOTH ways (deterministic verifier vs LLM judge) and compared.

  • No-op run (homepage only, empty answer, clean DB) → all 20 verifiers FAIL (no false positives). ✅
  • Anti-shortcut — a trajectory with the correct T0 answer but no product-page visit →
    verifier FAILs on nav_5090_detail (exit 1). Recall without navigation cannot pass. ✅
  • Human answer-check — every read task's frozen ground truth was re-checked against the value
    actually rendered on the mirror. All matched except T18, where the article shows
    Jun 16, 2026 (abbreviated month); verify_18 was fixed to accept that on-page form.

Verifier vs LLM-judge consistency (full 20-task agent run): 18/20 agree; the 2 divergences are
both cases where the deterministic verifier is CORRECT and the LLM judge false-negatives.

Task Agent outcome Verifier LLM judge Note
T0 read "32 GB GDDR7" PASS success agree
T2 TDP "450 W" PASS success agree
T4 141GB "H200" PASS success agree
T11 checkout "order placed, #3" PASS success agree — hardest stateful
T14 profile "country → Germany" PASS success agree
T16 wishlist remove "RTX PRO 6000 removed" PASS success agree
T13 wishlist add empty answer, DB mutated PASS FAIL verifier > judge (silent state)
T18 newsroom "Jun 16, 2026" PASS FAIL DIVERGE — verifier right
T19 newsletter "gamer42 subscribed" PASS FAIL DIVERGE — verifier right (DB state a screenshot judge can't see)
T9 drivers (agent empty; gold: nav /drivers + 566.36) PASS verifier passes real completion
T15 review (agent claimed success but no DB row) FAIL FAIL verifier caught hallucinated success; PASS on a genuine submit (DB: rating 5 'Incredible')
T17 constraint (agent empty; driven: 5060 Ti → cart) PASS verifier passes real completion
T1/T3/T5/T6/T7/T8/T10/T12 agent empty/shortcut under load FAIL FAIL agree (see notes)

Findings from the validation:

  1. The deterministic verifier is the correct PRIMARY grader. Every divergence (T13, T18, T19)
    is the LLM judge false-negativing a genuine success the verifier caught via DB after-state or
    exact answer match. There is no case where the verifier was wrong and the judge right.
  2. The verifier catches hallucinated success. On T15 the agent reported posting the review
    but no review row was written — verify_15 FAILed on the DB after-state (and PASSes on a real
    submit). This is the state-mismatch guarantee working.
  3. Empty/wrong answers are agent-capability limits, not task/verifier defects. gpt-4o left
    several tasks empty (worsened by parallel-container load — page-readiness timeouts); the
    verifiers correctly FAIL those and PASS once the task is actually completed. The tasks are
    solvable and deterministically gradable.

Required before merge

Release mechanics (gating):

  1. Rebase onto main; move nvidia to port 40016 (F1).
  2. Merge HF discussion Add B&H Photo WebHarbor mirror #38, then pin .assets-revision to that immutable SHA (F2, F3).

Site fidelity (the main content ask):
3. Add the per-series marketing landing pages and fold at least one marketing/tech section onto
the solve path so navigation has real depth; re-anchor the commerce tasks (T11/T12/T17) onto a
"Where to Buy" / wishlist flow that matches nvidia.com instead of an invented direct checkout (§3).
4. De-leak T5: the Jetson Orin Nano Super is priced on the homepage featured grid, so the task is
answerable without visiting the Embedded category (§4).

Nits:
5. Fix the boot-banner site count ("15 sites"); consider per-series driver versions.

The build is well-engineered and the grading contract is solid — the verifier suite
(sites/nvidia/verify/, 20 tasks) + rubrics are validated and ready to ship. The one thing
holding this back from a clean approve is site-modeling fidelity (§3): it reads as a generic GPU
store, not as nvidia.com. Task definitions and rubrics were independently confirmed correct by the
reviewer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants