Add openh-rf-convert skill + supporting cleanup - #23
Merged
Conversation
waltsims
force-pushed
the
openh-rf-convert-skill
branch
from
June 25, 2026 03:48
5d7dff2 to
efe3efd
Compare
A two-skill workflow: openh-rf-convert prepares contributor submissions
in zea format; the existing openh-rf-submission-eval grades them.
Convert is source-agnostic — HDF5, Verasonics .mat (v7.3), .npz/.npy,
raw .bin + .json sidecars, DICOM, Hugging Face datasets — and produces
a verified convert.py + reconstruct.py + pipeline.yaml against the
openh-rf-latest zea spec. It does NOT write the data card.
skills/openh-rf-convert/
SKILL.md — six-step orchestration
README.md — install + layout
references/source-inspection.md — reading inspect_source.py output
references/field-mapping.md — source-side heuristics + Verasonics→zea
table + RcvData reshape + custom-data
escape hatch (CustomElement)
references/sources/uff.md — USTB UFF source-specific guide with
6 Tips, including the critical
plane-wave t0_delays + per-transmit
initial_times correction
scripts/inspect_source.py — numpy/h5py/scipy inventory tool with
.uff dispatch and recursive
bytes-attr decoding
scripts/tests/ — pytest coverage
skills/openh-rf-shared/ — shared by convert + submission-eval
data-card-template.md
pipeline_template.py
validate_zea_spec.py
(zea-format-notes.md DELETED — superseded by zea.data.spec docstrings
+ https://zea.readthedocs.io/en/openh-rf-latest/, which are the
authoritative target spec; maintaining a parallel copy invited drift
that the eval skill's 01-format.md had already warned about.)
skills/openh-rf-submission-eval/ — all citations migrated from
../openh-rf-shared/zea-format-notes.md
to zea.data.spec.{DataSpec, ScanSpec,
ProbeSpec, MetadataSpec} and the
readthedocs URL. Min-zea-version and
raw_data hard-requirement policies
are now stated inline in SKILL.md
since they're OpenH-RF acceptance
rules, not zea spec.
Notable findings documented in sources/uff.md, validated end-to-end
against PICMUS_numerical_calib_v2.uff and USTB's MATLAB DAS as ground
truth:
Tip 4 — steered plane-wave t0_delays AND initial_times.
USTB uses a wavefront-at-origin t=0 reference and computes tx_delay
analytically. zea computes tx_delay as min-over-elements of
(rx + t0_delays), which only matches USTB's analytical formula when
t0_delays use +(x · sin θ) / c (positive sign) referenced to
wavefront-at-origin. Because zea also requires t0_delays >= 0, we
have to subtract the per-tx min — which moves each transmit's time
origin backward by shift_per_tx. The sample-to-time mapping then
only stays consistent across transmits if initial_times is bumped
forward by the same amount. Without that bump, coherent compound
smears axially (17.8 samples mismatch between 0° and ±16° at
PICMUS's 5.21 MHz IQ rate).
Tip 6 — IQ reconstruction needs a custom pipeline.
zea.Pipeline.from_default()'s Demodulate op crashes on n_ch=2 even
though it documents pass-through. Drop the op for IQ inputs; the
downstream beamformer applies the correct IQ phase rotation when
data has n_ch=2 (beamformer.py lines 267–270).
Plus tips covering UFF's two transmit-sequence shapes (numbered vs
flat sequence/), IQ vs RF data layouts, element_width fallback,
RF demodulation_frequency substitution.
Skill validated end-to-end in two ways:
1. From the conversation context (with full debugging history loaded):
convert.py + reconstruct.py written from the docs, validator
reports compliant:true, B-mode matches USTB ground truth
structurally (cyst, hyperechoic block, point-target rows all in
correct mm positions).
2. From a fresh agent context (no debugging history): same outcome
in one pass; reported the skill as "self-sufficient" with two
small doc gaps that have now been closed (Annotations closed
vocabulary + CustomElement escape hatch).
waltsims
force-pushed
the
openh-rf-convert-skill
branch
from
June 25, 2026 03:53
efe3efd to
8c1023a
Compare
…d/verify Distill the manual convert workflow that produced the existing examples/*/convert.py files into the skill itself: - Step 3 (Scaffold) — say plainly: find the closest existing examples/*/convert.py and copy-modify it. Goal is the smallest convert.py that works — no defensive code, no error handling beyond what zea raises, no boilerplate. Generate for ONE sample first, not the whole dataset. - Step 4 (Verify) — gate scaling explicitly: do not process more samples until the single-sample reconstruction looks right. Validated with an A/B cold-context test on PICMUS: with this tip the generated convert.py drops from 125 to 98 LOC (~22%), defensive code blocks drop from 4 to 1 (the kept one is the Tip-5-sanctioned demod fallback), and the agent opens 5 reference example files instead of 2 — both runs still produce a compliant:true output and a correct B-mode, so the simplification doesn't sacrifice correctness. No adjacent doc changes needed; no contradictions with existing SKILL.md prose.
tristan-deep
suggested changes
Jun 25, 2026
tristan-deep
left a comment
tristan-deep
left a comment
Contributor
There was a problem hiding this comment.
Nice! Haven't tried the skill myself yet, but some small comments from my side to align a bit better with current zea tooling :)
| @@ -0,0 +1 @@ | |||
| ../../skills/openh-rf-convert No newline at end of file | |||
Contributor
There was a problem hiding this comment.
.claude dir was added again?
Collaborator
Author
There was a problem hiding this comment.
it was with an link to the skill. The skill file is in skills/
|
|
||
| For Verasonics: time-in-wavelengths → s via `× (1/fc)`; distance-in-wavelengths → m via `× (c/fc)`. | ||
|
|
||
| ## Custom data (escape hatch) |
Contributor
There was a problem hiding this comment.
For a complete documentation on custom data, custom metadata, and custom elements, see https://zea.readthedocs.io/en/openh-rf-latest/data-acquisition.html#custom-fields
This explanation is a bit off in some parts.
Collaborator
Author
There was a problem hiding this comment.
removed the custome data and added the link
Contributor
|
Additionally I would add to the skill to check for the current installation environment. Mainly to check if the latest |
- Tip 6 (uff.md): replace the manual `[op for op in ... if Demodulate]` filter with the built-in `zea.Pipeline.from_default(baseband=True)`, which is the supported API for IQ inputs. Verified against PICMUS: drops ApplyWindow + Demodulate cleanly, produces an IQ-correct chain (Cast → Beamform → EnvelopeDetect → Normalize → LogCompress) and a matching compliant B-mode. - Custom data section (SKILL.md): trim local prose and point at the canonical zea docs at https://zea.readthedocs.io/en/openh-rf-latest/data-acquisition.html#custom-fields rather than restating the schema. The previous text mis-described some details. - Verasonics block (_default.md): add a pointer to the supported zea.data.convert.verasonics converter so contributors can use it directly rather than re-implementing the manual mapping. - Hard requirements (SKILL.md): add an explicit environment check step — confirm `zea >= 0.1.0` before running the workflow, so outdated installs surface immediately rather than as cryptic API errors downstream. Not changed: the .claude/skills/openh-rf-convert discovery symlink follows the existing pattern set for openh-rf-submission-eval by PR #19; keeping it consistent with main rather than removing only one of two. Happy to remove both as a follow-up if the project is deprecating that convention.
@waltsims on the PR: the prior wording said '>= v0.1.0' which understated the floor. The submission gate is v0.1.0a3; current zea (v0.1.0) is the preferred install because alpha-era API drift causes confusing failures. Reworded the Hard-requirements bullet accordingly.
The local description was redundant with the linked-to docs and was exactly the kind of restatement that drifts. Heading kept as a discoverable hook; body is now the link.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the
openh-rf-convertagentic skill — a source-agnostic adapterthat turns a contributor's raw ultrasound dataset (HDF5, Verasonics
.mat,.npz/.npy, raw.bin+.jsonsidecars, DICOM, HF datasets)into the OpenH-RF (zea) file format, with a confirmed source→zea field
mapping and a verified conversion against
openh-rf-latest. The skilldoes not write the data card; it hands off to the contributor for
that, then to
openh-rf-submission-evalfor grading.What's in it
skills/openh-rf-convert/: SKILL.md (6-step orchestration), references(general field-mapping + per-format
sources/uff.md), andinspect_source.py(numpy/h5py/scipy inventory tool).skills/openh-rf-shared/: moves the validator, pipeline template, anddata-card template into a shared core read by both skills. Removes
zea-format-notes.md— the target spec is now sourced directly fromzea.data.spec.*and https://zea.readthedocs.io/en/openh-rf-latest/,eliminating a parallel-copy drift risk the eval skill already warned about.
skills/openh-rf-submission-eval/: citation-only updates to match theshared layout