Skip to content

The no-B-frames constraint is refused on write but never checked by doctor or the encoder #368

Description

@kstonekuan

Current behavior

Canonical video requires no B-frames (docs/FORMAT.md:153, item 4). Two refusals enforce it on the write side: the transform refuses pass-through video carrying B-frames, and write_access_units_to_mp4 (src/hflow/video.py:592) refuses any B-frame payload, because a -c:v copy remux drops the reorder tail (#255, measured in #250).

Two places state the constraint and do not check it.

hflow doctor does not classify picture coding types. _check_video_payload (src/hflow/doctor.py:96) calls count_h264_pictures at :112 for the picture count and never asks whether that picture is a B picture. docs/FORMAT.md:161-162 says so outright:

hflow doctor still does not classify picture coding types (below), so a clean report does not prove this constraint; the refusals do.

So a non-conforming file can pass hflow doctor and then be refused by the remux behind Episode.video, which is the wrong order for a validator to find out.

The encoder does not assert what its own docstring depends on. encode_images_to_h264 (src/hflow/video.py:132) lists its guarantees as "enforced, raising VideoEncodeError otherwise", and justifies the first one with "no B-frames means decode order == presentation order". _enforce_encode_guarantees at :665 checks the frame count, the keyframe cadence, and SPS/PPS on keyframes. It does not check for B-frames. The property holds only because bframes=0 sits in the x264 parameter string at :156. Edit that string, or meet an ffmpeg build that ignores the flag, and HFlow's own encoder produces output that HFlow's own remux refuses, with the failure surfacing a long way from its cause.

Why now

The instrument did not exist when either of these was written. scan_picture_coding_types (src/hflow/video.py:455) arrived with #345, and after #354 and #358 it is cheap and shares one NAL walk with count_h264_pictures. Measured on main, 640x480 bframes=3:

 300 frames  count_h264_pictures  1.43 ms   scan_picture_coding_types  1.68 ms
3000 frames  count_h264_pictures 14.54 ms   scan_picture_coding_types 16.93 ms

Both readers already walk the same NALs and decode the same two Exp-Golomb fields, so the classification is very nearly free at the point where doctor already pays for the count.

Pattern to copy

Doctor's existing video findings and their documented codes: video-format, video-invalid-slice-header, video-multiple-access-units at doctor.py:96-140, each with a row in the finding-code table at docs/FORMAT.md:197-199. A new code needs a row there too.

_enforce_encode_guarantees already raises VideoEncodeError with a message naming the offending access unit; a B-frame guarantee belongs beside those. Note it receives list[AccessUnit], and each unit carries .data, so the stream is b"".join(unit.data for unit in access_units).

What to build

A doctor finding when a canonical video message carries a B picture, and an encode guarantee that HFlow's own encoder never emits one.

One constraint that is easy to trip over. The two readers report an unparseable slice header differently, and #358 went out of its way to keep count_h264_pictures's two messages byte-identical because hflow doctor displays them:

count_h264_pictures        "slice header has no complete first_mb_in_slice value"
                           "slice header truncates its first_mb_in_slice value"
scan_picture_coding_types  "a slice header is incomplete or truncated; picture coding types
                            cannot be classified"

video-invalid-slice-header's text must not change. Whether you keep count_h264_pictures for the count and add the scan alongside, or take the count from scan_picture_coding_types(...).picture_count and translate its failure back into the existing text, is your call; say which you picked and why.

Definition of done

  1. hflow doctor reports an error finding for a canonical video message whose picture is a B picture, with a new code documented in the docs/FORMAT.md finding-code table.
  2. video-invalid-slice-header's message text is unchanged for both existing failure kinds. The current tests covering them pass untouched.
  3. _enforce_encode_guarantees raises VideoEncodeError when the encoder produces a B picture, naming the access unit.
  4. Both are proven by deleting the new check and watching a test go red, with the result in the PR body.
  5. A conforming canonical episode still produces a clean doctor report, and encode_images_to_h264 still succeeds. No existing fixture starts failing.
  6. Doctor's per-message cost does not regress measurably. It runs this path once per video message, so do not walk the NALs twice; include a before and after measurement.
  7. docs/FORMAT.md:161-162 is updated, since it currently documents this gap as open.

Non-goals

Validation

uv sync --locked --all-extras
uv run ruff check --fix
uv run ruff format
uv run ty check
uv run pytest -q tests/test_video.py tests/test_video_malformed_guards.py \
  tests/test_doctor.py tests/test_processing_regressions.py
uv run pytest -q

Run plain uv run pytest -q from the repo root: pytest tests alone misses packages/hflow-server/tests.

When Markdown changes, run the link check from CONTRIBUTING.md.

Notes

Not a good first issue: it edits the canonical validation path and an error contract that hflow doctor displays, and getting it wrong either misses the constraint or starts failing conforming files.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    advancedNeeds codebase familiarity; not a starter issuebugSomething isn't workinghelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions