fix: reject truncated Exp-Golomb codes

[alexliyu7352]

Summary

• Reject incomplete unsigned Exp-Golomb code words instead of accepting partial values.
• Propagate parsing failures through H.264, H.265, and H.266 parameter-set updates.
• Add regression coverage for truncated parameter sets without mutating decoder configuration state.

Why

The previous parser could reach the end of a truncated code word without a complete suffix, then use a partial value. Malformed SPS/PPS data could therefore be accepted or trigger invalid shift/assertion behavior.

Impact

Malformed H.264, H.265, and H.266 parameter sets now fail cleanly while valid inputs retain the existing behavior.

Validation

• Clean project build against the current upstream SDK and codec dependencies.
• Default test suite and focused truncated-parameter-set regression checks.

[Copilot]

Pull request overview

This PR hardens Exp-Golomb parsing by rejecting truncated ue(v) codewords, propagates those parse failures through H.264/H.265/H.266 parameter-set update paths, and adds regression tests to ensure malformed parameter sets fail without altering stored decoder configuration arrays.

Changes:

• Update mpeg4_h264_read_ue to return an error for incomplete/truncated ue(v) codewords and propagate failures to SPS/PPS parsing.
• Ensure H.265/H.266 parameter-set copy/update paths treat failed ID parsing as an update error.
• Add regression tests for truncated PPS inputs to verify clean failure without mutating stored arrays/offsets.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
libflv/source/mpeg4-annexbtomp4.c	Makes ue(v) parsing reject truncated codewords and updates AVC SPS/PPS update logic + regression test.
libflv/source/hevc-annexbtomp4.c	Propagates ue(v) parse failures through HEVC SPS/PPS parsing and update flow + regression test.
libflv/source/vvc-annexbtomp4.c	Propagates ue(v) parse failures through VVC PPS parsing and update flow + regression test.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (2)

libflv/source/hevc-annexbtomp4.c:170

• sps_temporal_id_nesting_flag is assigned but never used, which can trigger unused-variable warnings (often treated as build failures under -Werror). Remove the variable/assignment to keep the build warning-free.

	uint8_t sps_max_sub_layers_minus1;
	uint8_t sps_temporal_id_nesting_flag;
	uint8_t conformance_window_flag;
	uint32_t value;

libflv/source/hevc-annexbtomp4.c:320

• hevc_vps_id(...) mutates hevc (profile/tier/level + temporal layer fields). In h265_vps_copy, calling it inside the search loop overwrites the fields parsed from the new VPS with values from each stored VPS. If no stored VPS matches and the new one is added, hevc can end up reflecting the last probed stored VPS rather than the newly added VPS.

	if (0xFF == vpsid)
		return -1;
	for (i = 0; i < hevc->numOfArrays; i++)
	{
		if (H265_NAL_VPS == hevc->nalu[i].type && vpsid == hevc_vps_id(hevc->nalu[i].data, hevc->nalu[i].bytes, hevc, hevc->data + hevc->off, sizeof(hevc->data) - hevc->off))

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.