ITU publications bibliographic data for Relaton-ITU

This repository holds the combined bibliographic data for both ITU sectors — ITU-R (Radiocommunication) and ITU-T (Telecommunication Standardization) — used by Relaton-ITU. It supersedes the ITU-R-only relaton-data-itu-r repository.

Data is stored in Relaton YAML format in a single flat data/ folder, with the two sectors distinguished by filename prefix (data/itu-r- vs data/itu-t-). A single top-level data/ is what relaton index --flavor itu scans, matching every other relaton-data-* repository:

• data/itu-r- — ITU-R publications. Migrated once from relaton-data-itu-r. ITU removed / WAF-blocked the bulk-enumeration endpoint that harvested this corpus (the old RunSearch search) and there is no known replacement "list all ITU-R publications" source yet, so these files are *preserved as-is and are not re-crawled. They will be refreshed again once an ITU-R enumeration source is restored (see the hand-off notes in crawler.rb).

• data/itu-t- — ITU-T Recommendations (and supplements/amendments/corrigenda). Harvested from the ITU-T recommendation index (mws/api/recommendations/searchRecs, issue relaton-itu#80) and *wiped and recreated on every crawl (via the data/itu-t-* glob).

Crawler

crawler.rb runs daily via .github/workflows/crawler.yml. Each run:

1. wipes the data/itu-t- files and re-derives the ITU-R index rows from the preserved data/itu-r- files;

2. harvests the full ITU-T corpus into data/, merging its rows into the index (Relaton::Itu::DataFetcher.fetch("itu-t"));

3. writes the combined index-v2.yaml and rebuilds index-v2.zip.

The data/itu-r-* files are never wiped, so the un-re-harvestable ITU-R corpus is retained.

Note

refreshing the ITU-T half needs the ITU-T producer (issue relaton-itu#80) merged to relaton main, which the Gemfile pins. Without it a crawl re-derives the ITU-R index but cannot re-harvest ITU-T (the fetch hits the dead RunSearch endpoint and fails); the already-published data/itu-t-* files are left as-is — the collapse guard refuses to republish a wipe.

Index

Both sectors serialize to the same pubid:itu structured index-v2, so one index holds ITU-R and ITU-T rows together:

• index-v2.yaml/.zip — pubid-structured index (rows carry _type: pubid:itu:*), consumed by the Relaton::Itu flavor.

Each row’s id is kept only if it round-trips (from_hash(to_hash).to_hash == to_hash) through ::Pubid::Itu, the same guard Relaton::Itu::DataFetcher#pubid and the index loader’s Relaton::Index::FileIO#id_supported? apply, so producer and consumer stay aligned. A handful of malformed/series-level ids that pubid does not model are written as data files but left unindexed.

Verify the published index with:

bundle exec rspec

Note	index-v2 must be produced by the same relaton/pubid the flavor consumes — see the pins and comments in Gemfile.