add TwelveLabs Pegasus video analyser node (opt-in, VP_WITH_LLM)

[mohit-twelvelabs]

Hi! I'm Mohit, I work at TwelveLabs (@mohit-twelvelabs).

What this adds

A new opt-in inference node, vp_pegasus_analyser_node, that integrates TwelveLabs Pegasus for whole-video semantic understanding / structuring.

Unlike the existing vp_mllm_analyser_node (which sends a single frame per request to a multimodal LLM), Pegasus reasons over the entire video on the TwelveLabs cloud and returns a structured semantic description (summary / chapters / highlights / open-ended Q&A). Because Pegasus analyses the whole video rather than per-frame, the node calls the API once, caches the result, and writes it into frame_meta->description for frames flowing through — so it drops straight into existing pipelines and renders with the existing vp_mllm_osd_node.

Files:

• nodes/infers/vp_pegasus_analyser_node.{h,cpp} — the node (HTTP via the bundled cpp_httplib + nlohmann/json, no new third-party dependency).
• samples/pegasus_analyse_sample.cpp + samples/CMakeLists.txt wiring.
• README / SAMPLES / nodes table entries.

Why it helps VideoPipe

VideoPipe already markets video structuring as a core use case and recently added mLLM support. Pegasus extends that from per-frame image description to true whole-video understanding, giving users cloud-grade semantic structuring without running a local model or GPU.

Opt-in / non-breaking

The node and sample are guarded by the existing VP_WITH_LLM flag (the same gate as the mLLM node), so a default cmake .. build is completely unaffected. No defaults change and no existing behavior is touched. It reuses already-bundled dependencies (cpp_httplib, nlohmann/json, OpenSSL — already required when VP_WITH_LLM=ON).

How it was tested

• clang++ -std=c++17 -DVP_WITH_LLM -fsyntax-only on both the node and the sample TU against the real OpenCV4 + OpenSSL + bundled httplib/json include graph — compiles clean (only the repo's pre-existing experimental/filesystem portability quirk on non-Ubuntu libc++).
• Verified the exact request payload the node builds (POST /v1.3/analyze, x-api-key header, {model_name, video|video_id, prompt, max_tokens, stream:false}) against the live TwelveLabs API and confirmed it returns the analysis in the data field.

You can grab a free API key at https://twelvelabs.io — there's a generous free tier.

[taoxm]

邮件已收到，谢谢！