Skip to content

Latest commit

 

History

History
128 lines (105 loc) · 9.88 KB

Notes.md

File metadata and controls

128 lines (105 loc) · 9.88 KB

PureCars AI Interview — Study Cheat Sheet Generative AI Creative Engine

  1. Company Context & Mission

Key Numbers to Know $250M/year Annual ad spend managed across thousands of dealerships $7M/year Current cost of manual creative + compliance review — your target to automate 1,000s of dealers Multi-tenant, each with their own inventory, OEM, and jurisdiction Daily/weekly Creative refresh cadence — throughput is a core requirement Meta, Google, Display Multi-platform formats, each with own specs

What You're Building A Generative AI Creative Engine that: • Generates ad assets at scale — images, copy, variants, multi-platform formats • Enforces compliance automatically — OEM rules, state/provincial laws, dealer constraints • Operates at massive throughput — tens of thousands of ads on daily/weekly cycles

OEM Compliance Basics OEMs (Ford, GM, Toyota, Hyundai, etc.) publish strict brand standards. Violations cost dealers their co-op ad funds.

▸ Logo placement & sizing rules ▸ Approved color palettes (exact hex values per OEM) ▸ Trim and price accuracy — no hallucinated specs ▸ Required disclaimer text (must appear verbatim) ▸ Incentive claim rules — claims must be substantiated ▸ Typography and font standards ▸ Vehicle imagery: approved angles and compositions ▸ Co-op eligibility requirements per campaign type

State & Provincial Ad Law Triggers ▸ Price disclosure: must be 'all-in' or clearly itemized (varies by state) ▸ APR format requirements — precise decimal, required context ▸ Lease payment fine print — residual value, money factor disclosure ▸ 'As low as' claim rules — must have qualifying inventory ▸ Quebec bilingual requirements for Canadian campaigns ▸ Bait-and-switch prohibitions — advertised units must be available   2. System Architecture

Frame every architecture answer as four layers: Retrieval → Generation → Compliance Gate → Delivery

The Four-Layer Pipeline Layer 1 — Retrieval Dealer inventory feed → structured DB. OEM rules, state laws, brand assets → vector store. At request time: retrieve relevant rules + dealer-specific constraints via RAG. Layer 2 — Generation Copy: LLM with structured prompt + retrieved context. Images: fine-tuned diffusion model (LoRA per OEM) or ControlNet for vehicle poses. Output: raw asset + structured metadata. Layer 3 — Compliance Gate Rule-based: regex for prices, disclaimers, required text. LLM-as-judge: brand tone, claim accuracy. Vision model: logo placement, color palette, trim accuracy. Fail → regenerate or flag. Layer 4 — Delivery Format resizing per platform spec. A/B variant tagging. CDN storage. Push to Meta/Google APIs. Audit log for compliance trail.

Multi-Tenant Architecture Dealer isolation Separate RAG namespaces per dealer. Rate limiting per tenant. Isolated credential and asset storage in object storage. OEM rule versioning OEM publishes new guidelines → version-controlled rule sets. Each dealer config pinned to OEM version. Audit trail on rule changes. Throughput at scale Async job queue (Celery/SQS). Priority tiers: real-time vs. batch. Horizontal scaling on generation workers. Cache reusable assets. Observability Per-job latency, compliance pass rate, generation cost per dealer. Drift detection on output quality.   3. Key Technical Concepts

RAG — Retrieval-Augmented Generation Augments LLM calls with retrieved documents at inference time. Critical here because OEM rules change constantly and cannot be baked into fine-tuning weights.

▸ Chunk OEM rule PDFs into ~500-token passages ▸ Embed with a dense retriever (e.g., OpenAI text-embedding-3, Cohere) ▸ Top-k retrieval at prompt time, re-ranked for relevance ▸ Separate namespace per OEM + per state jurisdiction ▸ Version documents — when rules update, re-index and re-run impacted assets

Fine-tuning vs. LoRA vs. ControlNet Full fine-tune Updates all model weights. Expensive, slow, risks losing generalization. Use only with massive OEM-specific data. LoRA (recommended) Injects small trainable matrices into attention layers. 10–100x cheaper than full fine-tune. Best for per-OEM image style (Ford vs. Toyota look-and-feel). ControlNet Adds structural conditioning (pose, depth, edge maps) to diffusion. Use to enforce vehicle angle — guarantees 'front 3/4 view' every time. PureCars play LoRA per OEM brand style + ControlNet for vehicle pose + RAG for text/copy rules. Don't over-fine-tune — preserve generalization.

LLM-as-Judge Evaluation Use a separate LLM call to evaluate another LLM's output. Scalable alternative to human review for compliance checks at scale.

▸ Structured scoring rubric in system prompt (not free-form evaluation) ▸ Separate judge model from generator — no self-grading ▸ Calibrate judge against human labels monthly ▸ Output: structured pass/fail + specific reason for failure ▸ Track judge accuracy as a first-class metric

CRITICAL: Pricing & Incentives Must Never Be Generated NEVER let the LLM generate prices, APR, or incentive amounts. Use slot-filling: LLM generates copy with [PRICE] placeholder → inject verified value from inventory feed. Compliance gate verifies final ad matches data source exactly.

Production AI Engineering Checklist ▸ Prompt versioning — git-tracked, tested before deploy ▸ Eval regression suite — runs on every prompt change ▸ Output drift monitoring — detect quality degradation over time ▸ Human-in-loop escalation path for low-confidence outputs ▸ Cost-per-asset tracking per dealer and per OEM ▸ Retry + fallback logic on generation failure ▸ Structured logging and distributed tracing ▸ Shadow mode before full rollout of new models/prompts   4. Scenario Q&A — Know These Cold

Q: How would you ensure a generated vehicle image matches OEM rules? Q: How would you ensure an AI-generated vehicle image matches OEM rules? Multi-layer approach: (1) ControlNet with OEM-approved vehicle pose maps to constrain angles. (2) LoRA fine-tuned on approved OEM brand imagery for visual style. (3) Vision model compliance gate checks logo placement, color accuracy vs. OEM hex palette, required badges. (4) Human escalation queue for low-confidence outputs. Key insight: generation constraints reduce violations; the compliance gate catches what slips through.

Q: How would you enforce state-specific advertising laws automatically? RAG-backed rule store: (1) Ingest all state/provincial ad law summaries into a vector store, tagged by jurisdiction. (2) Each dealer record has a jurisdiction field. (3) At generation time, retrieve relevant state rules and inject into the compliance-checking prompt. (4) Deterministic regex for hard rules (APR format). (5) LLM-as-judge for semantic rules ('does this claim require a disclaimer?'). (6) Version control on rule documents — when a state updates its rules, re-run impacted assets.

Q: How would you scale creative generation for thousands of dealerships? Async job queue architecture: (1) API receives generation request → publishes to SQS/Celery queue. (2) Worker pool scaled horizontally — GPU workers for image, CPU for copy. (3) Priority tiers: real-time vs. batch nightly refresh. (4) Shared asset cache for OEM logos, backgrounds, fonts — don't regenerate reusable assets. (5) Per-dealer config blob in Redis for fast lookup. (6) Output to CDN directly. (7) Track P95 latency and cost-per-asset per dealer.

Q: A dealer's ad got rejected by their OEM. How do you diagnose it? Audit trail first: (1) Pull the generation job log — which rule version was active? What did RAG retrieve? What did the compliance gate score? (2) If the gate passed it but OEM rejected: gap in rule coverage → add rule to rule store and recalibrate the judge. (3) If the gate flagged it but it shipped anyway: escalation path was bypassed — fix the workflow gate. Root cause: is it a rule-store gap, model capability gap, or workflow gap? Each has a different fix.

Q: How would you build an evaluation framework for this system? Three-tier eval: (1) Unit evals — deterministic checks on known cases, run on every prompt change. (2) LLM-as-judge evals — rubric-driven judge scores a held-out eval set weekly, tracks regression. (3) Human evals — monthly sample of 200 outputs reviewed by compliance team; human labels calibrate the judge. Key metrics: compliance pass rate, human override rate, cost-per-asset, P95 latency. Treat eval infra as a first-class product.   5. Quick Reference — Terms & Concepts

RAG Retrieval-Augmented Generation — retrieves relevant docs at inference time to ground LLM output LoRA Low-Rank Adaptation — cheap fine-tuning via small trainable matrices in attention layers ControlNet Structural conditioning layer for diffusion models — enforces pose, depth, edges LLM-as-judge A separate LLM evaluates another LLM's output against a rubric Slot-filling LLM generates copy with placeholders; verified data is injected post-generation Multi-tenant Isolated namespaces, configs, and rate limits per dealer within shared infrastructure Compliance gate Automated review layer (rule-based + model-based) before assets are delivered Drift detection Monitoring for output quality degradation over time as models or data change Shadow mode Running a new model/prompt in parallel without serving output — validates before cutover Co-op funds OEM subsidizes dealer advertising — only compliant ads qualify for reimbursement Vector store Database of embedded document chunks enabling semantic similarity search for RAG OEM Original Equipment Manufacturer (Ford, GM, Toyota, etc.) — sets brand + compliance rules

Interview Framing Reminders Show: (1) You understand the problem — $7M in manual work to automate, legal stakes. (2) You can architect the solution — layer-by-layer pipeline thinking. (3) You can build it end-to-end — not research, production engineering. (4) You can scale it — async queues, caching, horizontal workers. (5) You can make it compliant — slot-filling for prices, RAG for rules, LLM-as-judge for evaluation.