Skip to content

Fix/vp8 vp9 codec normalization#501

Merged
siddharthvaddem merged 3 commits intosiddharthvaddem:mainfrom
FabLrc:fix/vp8-vp9-codec-normalization
Apr 29, 2026
Merged

Fix/vp8 vp9 codec normalization#501
siddharthvaddem merged 3 commits intosiddharthvaddem:mainfrom
FabLrc:fix/vp8-vp9-codec-normalization

Conversation

@FabLrc
Copy link
Copy Markdown
Collaborator

@FabLrc FabLrc commented Apr 28, 2026

Description

Fixes H.264 video export failing with "VideoDecoder error: Unknown or ambiguous codec name" / "Unsupported codec".

Motivation

When exporting a recording, the StreamingVideoDecoder calls web-demuxer.getDecoderConfig("video") which returns a codec string and an optional description (the AVCDecoderConfigurationRecord / avcC box). In some cases — particularly WebM containers repaired by @fix-webm-duration/fix — the extradata is missing (description: undefined), and web-demuxer falls back to a malformed codec string like avc1.2420032. This string is rejected by Chromium's WebCodecs VideoDecoder with "Unknown or ambiguous codec name".

The same root cause can also produce bare four-character code strings (av01, vp08, vp09, avc1, h264) that lack the full parametrized form WebCodecs requires.

Type of Change

  • New Feature
  • Bug Fix
  • Refactor / Code Cleanup
  • Documentation Update
  • Other (please specify)

Related Issue(s)

#500

Changes

src/lib/exporter/streamingDecoder.ts

  1. H.264 codec fallback — When VideoDecoder.isConfigSupported() returns false (or configure() throws) for a codec string starting with avc1, the decoder now falls back to avc1.640033 (H.264 High Profile Level 5.1, universally supported on Chromium).
    Rationale: The actual H.264 bitstream profile/level is conveyed in-band via SPS NAL units, so a decode-capable codec string (even a generic one) is sufficient for the decoder to configure itself.

  2. Bare codec string normalization — Added normalization for four-character code strings returned by web-demuxer:

    • vp08vp8
    • vp09vp9
    • avc1avc1.640033
    • h264avc1.640033

    These follow the existing av01 → parametrized AV1 string pattern from PR fix: handle av1 VideoDecoder errors #334 .

  3. Extended shouldPreferSoftwareDecode — Added vp09 and vp9 to the software-decode preference (consistent with AV1, which also benefits from software decode for CPU-intensive codecs).

  4. Diagnostic logging — Added console.log / console.warn statements to surface the raw codec, description, and isConfigSupported result at runtime. These can be removed once the fix is stable.

Testing

  1. Check out the fix/vp8-vp9-codec-normalization branch
  2. Build and run the app (npm run dev)
  3. Record a screen capture (system must not support H.264 hardware acceleration, or produce a WebM with a malformed codec string)
  4. Try to export the recording
  5. Verify export completes without "VideoDecoder error" or "Unsupported codec" in the DevTools console
  6. Confirm the output video plays correctly

Checklist

  • I have performed a self-review of my code.
  • I have added any necessary screenshots or videos.
  • I have linked related issue(s) and updated the changelog if applicable.

Summary by CodeRabbit

  • Bug Fixes
    • Enhanced codec support and normalization for VP8, VP9, AVC1, and H.264 formats to improve video playback compatibility
    • Improved error handling with detailed diagnostic information when decoder configuration fails
    • Added validation of decoder configuration before initialization for more reliable playback

@FabLrc FabLrc requested a review from siddharthvaddem as a code owner April 28, 2026 12:25
@FabLrc FabLrc added the bug Something isn't working label Apr 28, 2026
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 28, 2026

📝 Walkthrough

Walkthrough

The streaming decoder enhances codec handling in decodeAll: normalizing multiple WebDemuxer codec formats (vp08, vp09, avc1, h264) to WebCodecs standards, expanding software decode preference to VP9, validating configurations via VideoDecoder.isConfigSupported before setup, and implementing tiered fallback strategies with codec-specific error logging.

Changes

Cohort / File(s) Summary
Streaming Decoder Codec & Config Logic
src/lib/exporter/streamingDecoder.ts
Expanded AV1 normalization; added support for bare codec string normalization (vp08, vp09, avc1, h264); broadened VP9 software decode preference; added pre-configuration isConfigSupported validation; improved error handling with codec-aware logging; implemented tiered try/catch fallback with software-preference and avc1-specific recovery paths.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested reviewers

  • siddharthvaddem

Poem

🎬 codecs tumble through the night
vp08, avc1, trying to get it right
fallbacks gracefully cascade down
isConfigSupported won't let us drown
software decode keeps the frame rate tight 🎯

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Linked Issues check ⚠️ Warning Linked issue #500 (not #39) is the actual related issue for this codec normalization bug fix. Issue #39 covers performance optimization with requestVideoFrameCallback, which is a different feature unrelated to this codec fix. This PR addresses #500 (codec normalization). Issue #39 (playback-based extraction) is a separate feature and should not be validated against this codec normalization changeset.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed Title refers to codec normalization changes present in the PR, covering VP8/VP9 and related fixes, though it's slightly terse and doesn't mention H.264 fallback.
Description check ✅ Passed Description is thorough: covers purpose, motivation, related issues, detailed changes, and testing steps. All key template sections are filled out appropriately.
Out of Scope Changes check ✅ Passed All changes in streamingDecoder.ts directly address codec normalization, error handling, and H.264 fallback as stated in the linked #500 issue. No unrelated modifications detected.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
src/lib/exporter/streamingDecoder.ts (1)

305-306: Nit: gate verbose decoder diagnostics behind a debug flag

Lowkey useful while debugging, but these logs will fire on every decode path and dump raw description objects. Consider guarding with a debug toggle (or console.debug) to keep prod consoles less cursed.

diff suggestion
-		console.log("[StreamingVideoDecoder] decoderConfig.codec:", decoderConfig.codec);
-		console.log("[StreamingVideoDecoder] decoderConfig.description:", decoderConfig.description);
+		const debugDecode = false;
+		if (debugDecode) {
+			console.debug("[StreamingVideoDecoder] decoderConfig.codec:", decoderConfig.codec);
+			console.debug(
+				"[StreamingVideoDecoder] decoderConfig.description:",
+				decoderConfig.description,
+			);
+		}
...
-			console.log(
-				`[StreamingVideoDecoder] isConfigSupported for "${preferredDecoderConfig.codec}":`,
-				support.supported,
-			);
+			if (debugDecode) {
+				console.debug(
+					`[StreamingVideoDecoder] isConfigSupported for "${preferredDecoderConfig.codec}":`,
+					support.supported,
+				);
+			}

Also applies to: 387-391

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/lib/exporter/streamingDecoder.ts` around lines 305 - 306, The two
console.log calls printing decoderConfig.codec and decoderConfig.description
(and the similar logs at the other spot) are noisy and dump raw objects; wrap
these diagnostics behind a debug toggle or replace with console.debug so they
only run when verbose logging is enabled—e.g., add a check on a DEBUG/verbose
flag or use console.debug inside the StreamingVideoDecoder decode/init path
where decoderConfig is available to prevent spamming production consoles while
keeping the logs accessible for debugging.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/lib/exporter/streamingDecoder.ts`:
- Around line 324-329: The codec normalization currently unconditionally
overwrites decoderConfig.codec to "avc1.640033" for /^avc1$/i or /^h264$/i and
later the fallback path attempts the same configure() again causing a misleading
warning; change both places to only set or attempt the avc1 fallback when
decoderConfig.codec (case-insensitive) is not already "avc1.640033" so you skip
the no-op rewrite and avoid calling configure()/emitting the fallback warning
redundantly.

---

Nitpick comments:
In `@src/lib/exporter/streamingDecoder.ts`:
- Around line 305-306: The two console.log calls printing decoderConfig.codec
and decoderConfig.description (and the similar logs at the other spot) are noisy
and dump raw objects; wrap these diagnostics behind a debug toggle or replace
with console.debug so they only run when verbose logging is enabled—e.g., add a
check on a DEBUG/verbose flag or use console.debug inside the
StreamingVideoDecoder decode/init path where decoderConfig is available to
prevent spamming production consoles while keeping the logs accessible for
debugging.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 67202809-4da3-488b-87de-c26fd29f75a5

📥 Commits

Reviewing files that changed from the base of the PR and between 1fefde8 and f9401f0.

📒 Files selected for processing (1)
  • src/lib/exporter/streamingDecoder.ts

Comment thread src/lib/exporter/streamingDecoder.ts
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f9401f051c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/lib/exporter/streamingDecoder.ts
@Enriquefft
Copy link
Copy Markdown
Contributor

dont forget the debugging console logs

@siddharthvaddem siddharthvaddem merged commit a6fe33a into siddharthvaddem:main Apr 29, 2026
17 checks passed
FabLrc added a commit to FabLrc/openscreen that referenced this pull request Apr 29, 2026
Sync ui-rework with main, which includes:
- fix/vp8-vp9-codec-normalization (PR siddharthvaddem#501)
- fix/cursor-telemetry-session-isolation (PR siddharthvaddem#457)

No source conflicts — changes are in disjoint areas.
@coderabbitai coderabbitai Bot mentioned this pull request Apr 30, 2026
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants