AndreKalberer · Oct 5, 2026
diff --git a/docs/movie-discovery-prototype.md b/docs/movie-discovery-prototype.md
@@ -0,0 +1,123 @@
+# Unloaded movie discovery: bounded metadata prototype
+
+Read-only research, separate from the published coverage audit. No desktop/TV
+search, browse, providers, user profiles, or playback changes. Do not ship this as
+proof of availability. No network requests are implemented in the prototype.
+
+## Sources and scope
+
+Join the [official TMDB daily movie-ID export](https://developer.themoviedb.org/docs/daily-id-exports)
+to VidAPI's [documented daily TMDB movie-ID list](https://vidapi.ru/api).
+Stream the cached gzip and retain only provider-listed records with explicitly
+false adult/video flags. Search original titles and exact IDs in a Node worker.
+No individual embeds, accounts, API keys, or added dependencies are needed.
+Membership remains `playbackStatus: not-tested`.
+
+The cached October 4 export contains exactly `adult`, `id`, `original_title`,
+`popularity`, and `video`. It has no release date/year, poster, genre, language,
+localized title, or alias list. TMDB says these are not full data exports.
+Its [authenticated search workflow](https://developer.themoviedb.org/docs/search-and-query-for-details)
+is the stronger method when legitimate build credentials are available.
+
+VidAPI documents full ID files and latest-added page metadata, but no title-search
+endpoint or per-ID metadata lookup. Its docs currently describe 24 items/page,
+where the earlier live audit observed 20. Read response values rather than assume
+page size. Full-feed crawling would cost thousands of requests and still need
+snapshot consistency handling. Separate IMDb/TMDB lists are not an ID mapping.
+
+## Local checkpoint
+
+October 5, 2026 run reused the audit's October 4 gzip (28,129,840 bytes). One public
+provider-list download returned 91,993 IDs and the same SHA-256 as the audit:
+`ada798678db408cd58165d032fffee044969e274a0a0dd6d3f1f633f07ead00a`.
+The worker scanned 1,252,997 rows / 129,369,990 expanded bytes and retained 90,983
+eligible, titled provider-listed movies. This intersection is not a provider total.
+
+Build took 3.565 seconds. Worker heap used was 68,806,984 bytes; whole-process RSS
+was 154,775,552 bytes, not a worker delta or peak. Complete run CPU was 5.89 seconds
+user / 0.797 seconds system, including build and queries. Across 100 sequential
+warm queries: roundtrip median 23.5ms, p95 30.7ms, max 31.9ms. Parent 10ms heartbeat
+maximum observed gap was 26ms. These are local Node measurements, not UI or TV QA.
+
+After moving byte/line guards ahead of decoding, one justified cached rerun checked
+the changed scanner's performance and full-export compatibility. It retained the
+same 90,983 records and exact 129,369,990-byte count. Build took 5.046 seconds;
+worker heap used 68,245,280 bytes / process RSS 158,298,112 bytes. Warm roundtrip
+median was 26.2ms, p95 31.4ms, max 33.0ms; parent heartbeat max gap was 25.7ms.
+Complete-run CPU was 7.516 seconds user / 0.641 seconds system. Guarded results are
+in `release/discovery-prototype/benchmark-stream-guard.json` in the primary
+workspace. This rerun used only cached inputs, with no additional download.
+
+All five automatically selected audit probes outside baseline built-ins and page 1
+were found. `Resident Evil: Welcome to Raccoon City` returned `460458` without
+loading its browse page. Other probes include upcoming titles: this export cannot
+identify released-film or verified-playable popularity cohorts.
+
+`기생충` and `tmdb:496243` found Parasite's record. `Parasite` returned other IDs,
+not `496243`. `千と千尋の神隠し` and `tmdb:129` found Spirited Away; the English title
+returned nothing. Accent-folded `Amelie` found `194` and other originals. Identity
+must remain the stable TMDB ID; remakes and accent collisions stay separate.
+
+Ignored raw artifacts are in the primary workspace's
+`release/discovery-prototype/benchmark.json` and adjacent `provider-ids.txt`.
+
+## Reproduce offline
+
+```sh
+node tests/movie-discovery-prototype.test.cjs
+node scripts/bench-movie-discovery.cjs <cached-export.json.gz> <provider-ids.txt> <prior-audit.json> <ignored-output.json>
+```
+
+Fixtures cover remakes, accent collisions, CJK original/ID search, unlisted/adult/
+unknown eligibility exclusion, malformed input, duplicate-ID rejection, and reuse
+of the actual renderer's `formatVidSrcMovie` identity. Results omit unsupported
+year/poster fields; that formatter yields `vidsrc-movie-<ID>` and VidAPI preference.
+Merge future candidates with loaded metadata by stable ID, preserving richer
+fields. Apply the separately reviewed provider-listing formatter change on eventual
+integration; this prototype does not require that PR or alter its files. Streaming
+regressions verify oversized unterminated gzip lines and total inflation are
+rejected before compressed/inflated EOF, plus a single 409,600-byte input chunk.
+One-byte chunks preserve UTF-8 titles, CRLF across chunks, standalone CR, and a
+final line without a delimiter. Worker fixtures cover successful startup/query,
+invalid queries, query after termination, malformed gzip, line-limit failure, and
+invalid provider IDs; startup failure terminates its worker.
+
+Bounds: 60MB compressed export, 350MB expanded bytes, 2m rows, 150k provider IDs,
+3MB provider text, 65,536 UTF-8 bytes per line excluding delimiters, 1,000 UTF-16
+code units per title, 200 UTF-16 code units per query, max 50 results. Expanded-byte
+and pending-line limits are checked on raw chunks before line concatenation or
+UTF-8 decoding; total bytes count actual delimiters. A rejected chunk is never
+decoded. No more than 65,536 pending line bytes are retained between chunks; the
+currently supplied source chunk and stream buffering are additional transient
+memory. Worker failure destroys both compressed and decompressed pipeline stages
+and discards the run. Guards are not a production resource-security guarantee.
+Plain numeric queries currently mean IDs; a runtime UI should use the
+explicit `tmdb:<ID>` syntax if numeric film titles need ordinary title matching.
+
+## Proposed runtime scope for parent review
+
+After ownership approval, implement only desktop no-key movie discovery. Lazily
+start one shared background worker, debounce input, ignore superseded responses,
+and return at most 20 candidates. Keep the index out of the renderer; preserve
+stable namespace, original title, unsupported empty fields, source dates, and
+listing-only status. Search must not open players or select playback servers.
+
+Cache raw snapshots and a versioned compact intersection outside the saved
+library/profile. Suggested TTL is 24 hours, one shared in-flight refresh, with a
+maximum seven-day stale fallback labeled stale/offline. An expired provider
+snapshot cannot claim current membership. Keep last valid cache on failure;
+validate complete replacements before atomic rename. Record fetched timestamps,
+chosen TMDB export date, both hashes, schema/normalization version, and counts.
+Hashes prove repeatability, not authenticity. Require dated filename/manifest
+instead of guessing a local gzip date. Production also needs cancellation,
+bounded download/timeouts/retries, and killable worker startup.
+
+This method alone cannot deliver reliable English/global title search. Prefer
+legitimate keyed TMDB metadata for that requirement. An authorized alias dataset
+would need separate license, ID-map, freshness, download-size, and performance
+review. Do not fabricate translations, borrow credentials, or hardcode hundreds
+of titles. Original-title/ID discovery can still improve no-key search honestly.
+
+TV remains deferred: this Node worker and ~65.6MiB worker heap cannot be assumed
+portable to constrained TV browsers. Compact delivery/Web Worker design needs
+separate cross-surface and device QA.
diff --git a/scripts/bench-movie-discovery.cjs b/scripts/bench-movie-discovery.cjs
@@ -0,0 +1,58 @@
+'use strict';
+const fs = require('node:fs');
+const path = require('node:path');
+const { performance } = require('node:perf_hooks');
+const { startWorker } = require('./movie-discovery-prototype.cjs');
+
+// Offline inputs required: matching dated export, provider list, prior audit.
+(async () => {
+  const [exportPath, idsPath, auditPath, outputPath] = process.argv.slice(2);
+  if (!exportPath || !idsPath || !auditPath || !outputPath) throw new Error('Provide export, provider ID list, audit, output paths.');
+  const audit = JSON.parse(fs.readFileSync(auditPath, 'utf8'));
+  const probes = audit.movies.filter(movie => movie.providerListed && !movie.initialProviderPageListed && !movie.desktopBuiltinListed).slice(0, 5);
+  const beforeCpu = process.cpuUsage();
+  const started = performance.now();
+  let ticks = 0, maxTickGapMs = 0, lastTick = started;
+  const heartbeat = setInterval(() => {
+    const now = performance.now();
+    maxTickGapMs = Math.max(maxTickGapMs, now - lastTick);
+    lastTick = now; ticks++;
+  }, 10);
+  let client;
+  try {
+    client = await startWorker(exportPath, idsPath);
+    const observations = [];
+    for (const movie of probes) {
+      const start = performance.now();
+      const result = await client.query(movie.title);
+      observations.push({ query: movie.title, expectedId: movie.id, found: result.results.some(item => item.tmdb_id === movie.id),
+        baselineBuiltinListed: false, baselineInitialPageListed: false, roundtripMs: performance.now() - start, ...result });
+    }
+    const foreign = [];
+    for (const query of ['기생충', 'Parasite', 'tmdb:496243', '千と千尋の神隠し', 'Spirited Away', 'tmdb:129', 'Le Fabuleux Destin d’Amélie Poulain', 'Amelie', 'tmdb:194']) {
+      foreign.push({ query, ...await client.query(query) });
+    }
+    const timings = [];
+    for (let i = 0; i < 100; i++) {
+      const start = performance.now();
+      await client.query(['unlikely movie match xyz', 'the', 'tmdb:129', 'dune'][i % 4]);
+      timings.push(performance.now() - start);
+    }
+    timings.sort((a, b) => a - b);
+    const report = { generatedAt: new Date().toISOString(), exportPath, providerPath: idsPath,
+      cacheNote: 'TMDB cached export reused; provider list independently captured, may differ from audit snapshot.',
+      metadata: client.metadata, heartbeat: { intervalMs: 10, ticks, maxTickGapMs },
+      totalMs: performance.now() - started, cpuMicroseconds: process.cpuUsage(beforeCpu),
+      queryRoundtrip: { count: timings.length, medianMs: timings[50], p95Ms: timings[94], maxMs: timings[99] },
+      observations, foreign, playbackStatus: 'not-tested', uiStatus: 'not-integrated' };
+    fs.mkdirSync(path.dirname(outputPath), { recursive: true });
+    fs.writeFileSync(outputPath, JSON.stringify(report, null, 2) + '\n');
+    console.log(JSON.stringify({ indexed: report.metadata.indexed, buildMs: report.metadata.buildMs,
+      queryRoundtrip: report.queryRoundtrip, heartbeat: report.heartbeat,
+      unloadedProbesFound: observations.filter(item => item.found).length,
+      foreign: foreign.map(({ query, results }) => ({ query, ids: results.map(movie => movie.tmdb_id) })) }, null, 2));
+  } finally {
+    clearInterval(heartbeat);
+    if (client) await client.close();
+  }
+})().catch(error => { console.error(error.message); process.exitCode = 1; });
diff --git a/scripts/movie-discovery-prototype.cjs b/scripts/movie-discovery-prototype.cjs
@@ -0,0 +1,173 @@
+'use strict';
+
+// Isolated metadata experiment. No renderer integration, network, or player URLs.
+const fs = require('node:fs');
+const { createGunzip } = require('node:zlib');
+const { Worker, isMainThread, parentPort, workerData } = require('node:worker_threads');
+const { performance } = require('node:perf_hooks');
+const { createHash } = require('node:crypto');
+
+const MAX_IDS = 150000;
+const MAX_ROWS = 2000000;
+const MAX_EXPANDED_BYTES = 350000000;
+const MAX_LINE_BYTES = 65536;
+const idOf = value => /^[1-9]\d*$/.test(String(value)) && Number.isSafeInteger(Number(value)) ? String(value) : null;
+const normalize = value => String(value).normalize('NFKD').replace(/\p{M}/gu, '').toLocaleLowerCase('en-US').replace(/[^\p{L}\p{N}]+/gu, ' ').trim();
+
+function parseIds(text) {
+  if (Buffer.byteLength(text) > 3000000) throw new Error('Provider ID input exceeds bound.');
+  const ids = new Set();
+  for (const line of text.split(/\r?\n/)) {
+    if (!line.trim()) continue;
+    const id = idOf(line.trim());
+    if (!id) throw new Error('Invalid provider ID.');
+    ids.add(id);
+    if (ids.size > MAX_IDS) throw new Error('Provider ID count exceeds bound.');
+  }
+  if (!ids.size) throw new Error('Empty provider ID snapshot.');
+  return ids;
+}
+
+async function* boundedLines(input, limits = {}) {
+  const maxBytes = limits.maxBytes ?? MAX_EXPANDED_BYTES;
+  const maxLineBytes = limits.maxLineBytes ?? MAX_LINE_BYTES;
+  let bytes = 0, pendingBytes = 0, afterCR = false;
+  let parts = [];
+  const append = part => {
+    pendingBytes += part.length;
+    if (pendingBytes > maxLineBytes) throw new Error('Export line byte bound exceeded.');
+    if (part.length) parts.push(part);
+  };
+  const complete = () => {
+    const line = Buffer.concat(parts, pendingBytes).toString('utf8');
+    parts = []; pendingBytes = 0;
+    return line;
+  };
+  for await (const value of input) {
+    const chunk = Buffer.isBuffer(value) ? value : Buffer.from(value);
+    bytes += chunk.length;
+    if (bytes > maxBytes) throw new Error('Export expanded byte bound exceeded.');
+    let start = 0;
+    for (let offset = 0; offset < chunk.length; offset++) {
+      const byte = chunk[offset];
+      if (afterCR) {
+        afterCR = false;
+        if (byte === 10) { start = offset + 1; continue; }
+      }
+      if (byte === 10 || byte === 13) {
+        append(chunk.subarray(start, offset));
+        yield { line: complete(), bytes };
+        start = offset + 1;
+        afterCR = byte === 13;
+      } else if (pendingBytes + offset - start + 1 > maxLineBytes) {
+        // Reject before any unterminated line is concatenated or decoded.
+        throw new Error('Export line byte bound exceeded.');
+      }
+    }
+    append(chunk.subarray(start));
+  }
+  if (pendingBytes) yield { line: complete(), bytes };
+  // Include trailing blank delimiters in exact expanded-byte accounting.
+  yield { line: '', bytes };
+}
+
+async function buildIndex(input, providerIds, limits) {
+  const byId = new Map();
+  let rows = 0;
+  let bytes = 0;
+  const fields = new Set();
+  try {
+    for await (const entry of boundedLines(input, limits)) {
+      const { line } = entry;
+      bytes = entry.bytes;
+      if (!line.trim()) continue;
+      if (++rows > MAX_ROWS) throw new Error('Export row bound exceeded.');
+      const movie = JSON.parse(line);
+      Object.keys(movie).forEach(key => fields.add(key));
+      const id = idOf(movie.id);
+      if (!id || typeof movie.original_title !== 'string' || movie.original_title.length > 1000
+        || !Number.isFinite(movie.popularity)
+        || !(typeof movie.adult === 'boolean' || movie.adult === null)
+        || !(typeof movie.video === 'boolean' || movie.video === null)) throw new Error('Invalid export record.');
+      if (movie.adult !== false || movie.video !== false || !providerIds.has(id) || !movie.original_title.trim()) continue;
+      if (byId.has(id)) throw new Error('Duplicate eligible TMDB ID.');
+      byId.set(id, { tmdb_id: id, title: movie.original_title, popularity: movie.popularity, normalized: normalize(movie.original_title) });
+    }
+  } finally {
+    input.destroy();
+  }
+  // Stable ID, never normalized title, is identity. Remakes and accent collisions survive.
+  const movies = [...byId.values()].sort((a, b) => b.popularity - a.popularity || Number(a.tmdb_id) - Number(b.tmdb_id));
+  return { movies, byId, rows, bytes, fields: [...fields].sort(), providerIds: providerIds.size };
+}
+
+function search(index, query, limit = 20) {
+  if (typeof query !== 'string' || query.length > 200 || !Number.isInteger(limit) || limit < 1 || limit > 50) throw new Error('Invalid query.');
+  const exactId = /^(?:tmdb:)?([1-9]\d*)$/.exec(query.trim());
+  const clean = movie => ({ tmdb_id: movie.tmdb_id, title: movie.title, popularity: movie.popularity, providerListed: true, playbackStatus: 'not-tested' });
+  if (exactId) return index.byId.has(exactId[1]) ? [clean(index.byId.get(exactId[1]))] : [];
+  const key = normalize(query);
+  if (!key) return [];
+  const exact = [], partial = [];
+  for (const movie of index.movies) {
+    if (movie.normalized === key) exact.push(clean(movie));
+    else if (movie.normalized.includes(key) && partial.length < limit) partial.push(clean(movie));
+    if (exact.length >= limit) break;
+  }
+  return exact.concat(partial).slice(0, limit);
+}
+
+async function startWorker(exportPath, idsPath) {
+  if (fs.statSync(exportPath).size > 60000000 || fs.statSync(idsPath).size > 3000000) throw new Error('Input file size exceeds bound.');
+  const worker = new Worker(__filename, { workerData: { exportPath, idsPath } });
+  let nextId = 0;
+  let closed = false;
+  const pending = new Map();
+  const ready = new Promise((resolve, reject) => {
+    worker.on('message', message => {
+      if (message.ready) resolve(message);
+      else if (pending.has(message.id)) { pending.get(message.id).resolve(message); pending.delete(message.id); }
+    });
+    worker.on('error', error => { closed = true; reject(error); for (const entry of pending.values()) entry.reject(error); pending.clear(); });
+    worker.on('exit', code => {
+      closed = true;
+      reject(new Error('Index worker exited before completion (code ' + code + ').'));
+      for (const entry of pending.values()) entry.reject(new Error('Index worker exited.'));
+      pending.clear();
+    });
+  });
+  let metadata;
+  try { metadata = await ready; }
+  catch (error) { await worker.terminate(); throw error; }
+  return { metadata, query: query => new Promise((resolve, reject) => {
+    if (closed) { reject(new Error('Index worker is closed.')); return; }
+    const id = ++nextId;
+    pending.set(id, { resolve, reject });
+    worker.postMessage({ id, query });
+  }), close: () => { closed = true; return worker.terminate(); } };
+}
+
+if (!isMainThread) {
+  (async () => {
+    const started = performance.now();
+    const idsText = fs.readFileSync(workerData.idsPath, 'utf8');
+    const providerIds = parseIds(idsText);
+    const compressed = fs.createReadStream(workerData.exportPath);
+    const gunzip = createGunzip();
+    compressed.on('error', error => gunzip.destroy(error));
+    compressed.pipe(gunzip);
+    let index;
+    try { index = await buildIndex(gunzip, providerIds); }
+    finally { compressed.destroy(); gunzip.destroy(); }
+    parentPort.postMessage({ ready: true, rows: index.rows, indexed: index.movies.length, fields: index.fields,
+      expandedBytes: index.bytes, buildMs: performance.now() - started, memory: process.memoryUsage(),
+      providerIds: index.providerIds, providerSha256: createHash('sha256').update(idsText).digest('hex') });
+    parentPort.on('message', ({ id, query }) => {
+      const started = performance.now();
+      try { parentPort.postMessage({ id, results: search(index, query), searchMs: performance.now() - started }); }
+      catch (error) { parentPort.postMessage({ id, error: error.message }); }
+    });
+  })().catch(error => { throw error; });
+}
+
+module.exports = { normalize, parseIds, buildIndex, search, startWorker };