Add Craigslist mirror#5
Conversation
…age and allrecipes.category.image
Adds two batch-friendly scripts that pull lead-image thumbnails from the
MediaWiki action API (`prop=pageimages`, `pithumbsize=1280`, batched
`titles=A|B|C`, 1 req/s with exponential 429 backoff) and write the
images under each site's `static/images/.../` plus a `/static/images/...`
web-path into the live mirror DB.
- google_search_topic_wiki.py: 1153 NULL hero_image rows. Heuristic
entity-extractor strips trailing query-style phrases ("Latest Game
Score", "Bio", "Movie Ratings", ...) so "Phoenix Suns Latest Game
Score" maps to "Phoenix Suns". 556 new wiki fills (726/1323 total
filled; top dup 0.3%).
- allrecipes_category_wiki.py: 54 NULL image rows. Per-name candidate
list rewrites generic category names to Wikipedia titles ("Italian"
-> "Italian cuisine", "Chicken" -> "Chicken as food", "Desserts"
-> "Dessert"). 37 new fills (46/54; top dup 2.2%).
Mozilla download UA on upload.wikimedia.org (scrape-real-images
gotcha #1), 8 KB content filter (gotcha aiming-lab#5), diversity gate <5% at end.
Misses leave NULL (no fabricated placeholders).
Review —
|
|
Thanks for the detailed review. I pushed a follow-up and refreshed the HF asset PR. What changed:
HF asset PR update:
Validation rerun: The only remaining blocker is the expected asset-flow step: HF PR #5 is still open, so |
Reviewer verifiers → submitted as #60The deterministic verifiers + judge rubrics for this site are up as #60, which bundles Review —
|
| Check | Result |
|---|---|
Syntax (py_compile app.py) |
OK |
| Site isolation | no cross-site imports |
| Port registration | craigslist = index 16 → 40016 (rebased, Merriam preserved at 40015) |
| Build | succeeds, 17/17 sites return 200 |
| Byte-identical reset | md5 052b5b6a… matches seed, repeatable ✅ |
reset-all |
all 17 sites in ~1 s |
Gating item: .assets-revision pins commit 54882a6a, which does NOT contain craigslist.tar.gz
(a fetch from it extracts 0 sites); the assets are only on the HF discussion #5 PR ref. Repin
.assets-revision to the SHA that actually has the merged craigslist asset.
2. Visual fidelity — PASS (rendered in Chromium)
Faithful to real Craigslist across the pages I screenshotted:
- Homepage: the iconic minimalist layout — purple logo, area tabs (sfc/sby/eby/…), the
multi-column community / services / housing / for-sale / jobs / gigs category grid with real
subcategories, nearby-cl + us-cities sidebar, event calendar, clean footer. - Search: area/category dropdowns, "1-18 of 18", all/owner/broker tabs, list/gallery/map view
toggles, price/type filters, listing cards with price + "no image" boxes (realistic for
Craigslist) + "1d ago oakland" location. - Listing detail: authentic
/d/<slug>-<id>/<id>.htmlURL, breadcrumb, price + title +
neighborhood, reply/flag, an attribute table (condition/brakes/frame/…), a location map with a
pin, "posting body" + the classic "do NOT contact this poster with unsolicited services"
boilerplate, "more from " related listings.
No mojibake, no broken layout. Clean.
Finding — image/content mismatch on listings that carry photos. Furniture listings share a
small pool of ~5-6 desk/chair photos assigned by rotation (each listing's details_json.images
is the same pool at a different offset), not matched to the item. So the "Black office chair"
(id 2) renders a crate-barrel-desk photo, "Dining chair set" (id 3) shows a desk, and the
"Walnut writing desk" (id 7) shows a gaming chair. (Categories without seeded images — e.g. bikes —
correctly show the "no image" box, which is faithful to Craigslist.) The photos are real and there's
no mojibake/layout problem, so this is milder than a broken render, but it reduces fidelity. Fix:
match each listing's image to its actual item (or, if diverse photos aren't available, fall back to
the "no image" box rather than showing a wrong-category photo). Grading is unaffected (the verifiers
use text/DB, not images). (The mismatch is widespread across furniture — e.g. the "Maple bookcase"
id 88 also shows the crate-barrel-desk photo, and the "Standing desk frame, white" id 8 — the
T19 target — shows a non-white crate-barrel-desk photo, so even the color contradicts the title.)
Finding — the favorites feature is labeled three different ways. The same feature (route
/saved) is called "faves" in the header icon (base.html) and homepage sidebar (index.html),
"favorites" in the account/messages/saved sub-nav links, and "saved listings" in the page
title + <h1> (saved.html). Real Craigslist uses one term — "favorites" — and uses neither
"faves" nor "saved listings" for the page. Fix: standardize on "favorites" everywhere. (This also
touches task wording: tasks T0/T3/T13 say "saved listings", which matches the page title but not the
"favorites"/"faves" nav.)
3. Functional depth — PASS
183 listings across 34 categories (for-sale / housing / jobs / services / community / gigs), 15
areas, plus login, account edit, saved listings, saved searches, hidden listings, replies/messages,
and posting. The full surface is exercised by the grading contract below (DB after-state confirms
save / save-search / post / reply / hide / account-edit writes persist).
Finding — the hide feature is one-way (no view/restore of hidden listings). Search cards have a
hide control (trash icon in gallery view / "hide" button in list view → POST /listing/<id>/hide),
and hidden listings correctly drop out of results. But there is no un-hide route and no "hidden
listings" view — once hidden, a listing cannot be seen or restored anywhere in the UI. Real
Craigslist shows a "N hidden" count you can reveal/restore. This makes the hide feature feel broken
(the user has no way to see what they hid) and leaves the hidden_listings state unmanageable.
Fix: add a hidden-listings view + un-hide, or at least a "N hidden — show" affordance on search.
Also, in gallery view the hide control is an unlabeled icon-only button (card-trash icon-only,
only an aria-label), so it's hard to discover that it means "hide" — add a visible label/tooltip
(the list view already has a "hide" text button).
Finding — the post form shows every field regardless of category. post.html renders all fields
unconditionally (no per-category logic), so a bikes posting still shows bedrooms, square feet,
compensation/company, and employment type — fields that only apply to housing or jobs. Real
Craigslist adapts the posting form to the category. Fix: show category-appropriate fields only
(housing → bedrooms/sqft; jobs → compensation/company/employment type; for-sale → condition, etc.).
4. Task quality — PASS with two pre-seed conflicts
Tasks are excellent: synthetic listings (anti-recall), page-specific answers (sqft, pet policy,
compensation, monitor specs, desk dimensions, showing times), and rich stateful coverage
(save / save-search / post / reply / hide / remove-saved / account-edit).
Finding — the T1/T4 sqft leak (MufanQiu's MAJOR) is only half-fixed; it still leaks on keyword
search. The author reported removing square footage from housing result cards, but the guard in
search.html:8 is {% set is_housing = category and category.group_slug == 'housing' %} — it is
True only when browsing a specific housing category page. On a keyword search
(/search?q=studio&area=east+bay, which is exactly how T1/T4 are performed), no category is
selected, so is_housing is False and search.html:312 renders {{ listing.sqft }}ft² on the
cards. Result: the studio cards still show 510ft² / 405ft² / 390ft², so T1's square-footage answer
and T4's rent-per-sqft are both computable from the results page without opening any detail page —
the original leak persists. Fix: gate on the listing's group
(listing.category_group == 'housing'), not the search's category context, so sqft is hidden on
housing cards in keyword search too.
Finding — T0 and T3 save-targets are already pre-saved. alice's seed saved-listings are
[chair id 1, Honda Accord id 13, studio id 21] (set up for T13 to remove the Accord). But:
- T0 asks to save the adjustable-arms chair = id 1, which is already saved → the save is a
no-op and cannot be verified (the DB after-state equals the initial state). - T3 asks to save the 2006 Honda Accord = id 13, which is already saved (it's the item T13
removes) → same no-op conflict.
Fix: point T0/T3 at chairs/Accords that are NOT in alice's pre-seeded saved list (or drop those
two from the seed). My verifiers for T0/T3 check navigation + that the item is saved, but flag that
they cannot confirm the add — the tasks should be corrected.
Finding — T5 asks for a category the save-search UI can't set. The task ("save a search named
'Gaming chair watch' for furniture listings matching gaming chair, max $200") requires the
saved search to carry a furniture category filter. But the save-search form
(search.html:177-184) has no category control — its hidden category_slug is
{{ category.slug if category else '' }}, i.e. inherited only from being on a category-browse page.
On the natural flow (keyword-search "gaming chair", then save), category is empty, so the saved
search stores category_slug='' — the furniture filter is silently dropped. To actually set it, the
agent must first browse into the furniture category page and search within it, which the task never
says. So the task both reads confusingly (why specify a category AND a keyword that already implies
it?) and can't be fully satisfied through the obvious UI. My verifier therefore only checks that a
saved search named "Gaming chair watch" exists. Fix: either add a category selector to the
save-search form, or reword T5 to a single keyword intent (e.g. "save a search for 'gaming chair'
priced up to $200").
Minor: T9 ("moving boxes near Oakland") matches two Oakland listings (id 20, id 52); the verifier
accepts a reply on either.
Finding — image coverage is inconsistent across categories. apartments (12/12), furniture
(16/16 — but content-mismatched, see §2), and cars_trucks (9/9) all have photos, but bikes (0/10)
and electronics (0/10) have none (every listing shows the "no image" box). Whole categories with
zero images is less faithful than real Craigslist (bikes/electronics usually have photos). Combined
with the furniture mismatch (§2), the image handling has three inconsistent states: correctly
matched, wrongly matched, and entirely absent. Fix: give bikes/electronics real (matched) images
or accept the "no image" box as a deliberate choice applied consistently.
5. Grading contract (reviewer deliverable)
A deterministic verifier per task under sites/craigslist/verify/ + verifier_path /
judge_rubric in every tasks.jsonl row (no answer key). Deterministic-first: listing-open
navigation gate + SQLite DB after-state for the stateful tasks (saved_listings / saved_searches /
hidden_listings / messages / new postings / user profile) + answer vs frozen ground truth, LLM only
as an anchored check.
Validation
- No-op run → all 20 verifiers FAIL (no false positives). ✅
- Verifier vs LLM-judge consistency (full 20-task agent run, gpt-4o): 14/20 agree. Genuine
agent-completed PASSes (both graders agree) span read tasks (T1/T7/T14/T17/T19) and stateful ones
(reply T9/T11, remove-saved T13). The 6 divergences:
| Divergence | Meaning |
|---|---|
| T2, T8 | verifier PASS, judge FAIL — verifier right (reply recorded in DB / correct answer; judge false-negative) |
| T5, T15 | verifier FAIL, judge PASS — verifier right (agent didn't complete the save-search / reply; judge false-positive) |
| T0, T3 | verifier PASS (weak), judge FAIL — the T0/T3 task bug, not a verifier win: the verifier can only pass on nav + "already saved" (id 1/13 are pre-seeded), so it can't confirm a real save; here the judge's FAIL is arguably closer to correct. This is exactly why T0/T3 need fixing. |
The remaining empties (T4/T6/T10/T12/T16/T18) are gpt-4o capability limits on Craigslist's
multi-step flows (compare, create-posting, account-edit, hide); the verifiers correctly FAIL those.
Net: every divergence except the two broken tasks favors the deterministic verifier.
Required before merge
Leaks / task feasibility (higher priority):
- Finish the T1/T4 sqft fix — hide sqft on housing cards in keyword search too (gate on
listing.category_group == 'housing', not the search's category context). Still leaks today (§4). - Fix T0/T3 save-targets — they're already in alice's pre-seeded saved list, so the save is a
no-op and unverifiable (§4). - Fix T5 — either add a category control to the save-search form or reword to a single keyword
intent; the requested furniture-category filter can't be set through the obvious UI (§4).
Front-end fidelity / feature completeness:
4. Match listing images to their items (furniture shows wrong-category / wrong-color photos), and
give bikes/electronics images or apply "no image" consistently (§2).
5. Standardize the favorites label (faves / favorites / "saved listings" → one term) (§2).
6. Make the posting form category-aware (a bikes post shouldn't ask for bedrooms/sqft/compensation) (§3).
7. Add a hidden-listings view + un-hide, and a visible label on the gallery hide icon (§3).
Release mechanics:
8. Repin .assets-revision to the SHA that actually contains craigslist.tar.gz (merge HF #5 first);
the current pin 54882a6a extracts 0 sites.
Nit: T9 ("moving boxes near Oakland") has two valid Oakland targets (id 20/52).
The build is faithful at first glance and the grading contract is solid — the verifier suite
(sites/craigslist/verify/, 20 tasks) + rubrics are validated and ship with it — but the site needs
the fixes above before it reads as finished. Credit to @FlowRays for already addressing MufanQiu's
earlier round (T14/T15/T18/T19 leaks, lost+found, AppleDouble); this review builds on that and adds
the newly-found items + the verifiers.
Addresses DEM1TASSE follow-up review on PR aiming-lab#53: - aiming-lab#1: 'Drugs & Medications' nav item now links to /drugs (the real drug list) instead of the empty /section/drugs dead page; removed the redundant 'Drugs A-Z' item. One working, correctly-labeled drugs entry. - aiming-lab#2: 'Health Conditions' nav item now links to /conditions (the condition A-Z pages) instead of /section/health-conditions (articles); removed the redundant 'A-Z Conditions' item. Matches task wording (T3/T17). - aiming-lab#4: /history now shows the section name (e.g. 'Health Conditions · Respiratory'), so T14 (section of history articles) is answerable on-page. Added section_names map to the global context. - aiming-lab#5: register 'Full name' is now required (route + form) to match the seeded users which all have one. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Thanks for the thorough pass. I pushed a follow-up that addresses the requested site/task fixes. What changed:
Updated HF asset PR #5: Validation rerun: I also rechecked the targeted flows locally: no sqft leak on keyword studio search, T0/T3 are no longer pre-favorited, T5 saves the furniture category, hidden listings can be restored, and bike postings ignore housing/job-only fields. The only remaining release step is still the expected one: once HF PR #5 is merged, I will repin |
3 participants
Summary
Adds a Craigslist mirror to WebHarbor as the 17th site on port
40016.Real site mirrored: https://www.craigslist.org/
The mirror covers category browsing, search, listing detail pages, posting, replies, login, account edits, favorites, saved searches, hidden listings, and messages. It includes 20 benchmark tasks across for-sale, housing, jobs, services, community pages, and account workflows.
Latest review fixes are included: the housing sqft leak is closed on keyword searches, T0/T3 now target items that are not pre-favorited, T5 can save a furniture category filter, favorites naming is consistent, hidden listings can be shown/restored, the posting form is category-aware, and furniture/bikes/electronics now use the no-image fallback consistently.
Seed Data
Listings by group:
Assets
HF asset PR: https://huggingface.co/datasets/ChilleD/WebHarbor/discussions/5
Latest asset commit on that PR:
https://huggingface.co/datasets/ChilleD/WebHarbor/commit/7fad33e51e2c3261e59edac5071e769ffa55c9c2
This GitHub PR is still draft while the HF asset PR is pending. After the HF PR is merged,
.assets-revisionshould be pinned to the merged HF commit and this PR can be marked ready.Verification
Targeted review checks also passed locally: keyword studio search no longer leaks sqft, T0/T3 targets are not pre-favorited and can be newly added, T5 saves
category_slug=furniture, hidden listings can be shown and restored, bike postings ignore housing/job-only fields, and the asset tarball has no._*,__MACOSX, or stale furniture-image entries.