Skip to content

Add realtime video analysis and chat with token IDs scenarios#6

Open
jianbing119 wants to merge 5 commits into
5G-MAG:developmentfrom
jianbing119:vlm_chatToken_scenario
Open

Add realtime video analysis and chat with token IDs scenarios#6
jianbing119 wants to merge 5 commits into
5G-MAG:developmentfrom
jianbing119:vlm_chatToken_scenario

Conversation

@jianbing119
Copy link
Copy Markdown

@jianbing119 jianbing119 commented May 12, 2026

Add new scenarios for AI traffic testbed

  1. Realtime video analysis scenario
  • realtime video scenario implemented
  • vlm (Liquid model) implemented in aiortc lib
  • vlm webrtc server and vlm webrtc client implemented
  1. Chat with token ID scenario
  • chat token scenario implemented
  • chat token client implemented (openAI compatible API)

Please also see README_new.md

peterxuhw added 3 commits May 8, 2026 09:52
1. add realtime video analysis scenario, with local deployed vlm,
integrated in modified aiortc
2. add chat with token IDs scenario, using openAI compatible API
@jordijoangimenez jordijoangimenez added enhancement New feature or request Reviewers wanted Extra attention is needed labels May 13, 2026
@jordijoangimenez
Copy link
Copy Markdown
Contributor

jordijoangimenez commented May 13, 2026

Hello @jianbing119, when you have a moment please add a file called ATTRIBUTION_NOTICE in the root directory with the following content:

This software contains dependencies with the following open-source projects:

- aiortc - library for WebRTC and ORTC in Python
  available at https://github.com/aiortc/aiortc
  Licensed under BSD 3-Clause "New" or "Revised" License (https://github.com/aiortc/aiortc/blob/main/LICENSE)

- Liquid model
  available at https://github.com/FoundationVision/Liquid
  Licensed under MIT License (https://github.com/FoundationVision/Liquid/blob/main/LICENSE)

- StreamingBench
  available at https://github.com/THUNLP-MT/StreamingBench/
  Licensed under MIT License (https://github.com/THUNLP-MT/StreamingBench/blob/main/LICENSE)

Just giving you an example from another repo which contains dependencies: https://github.com/5G-MAG/rt-5gms-application-function/tree/main

@jordijoangimenez jordijoangimenez self-requested a review May 13, 2026 17:37
Comment thread README_new.md
Comment thread README_new.md
Comment thread README_new.md Outdated
Comment thread README_new.md Outdated
Comment thread README_new.md Outdated
Comment thread README_new.md
Comment thread README_new.md
@jordijoangimenez
Copy link
Copy Markdown
Contributor

First round of checks just to ensure build. Thank you @jianbing119 !

now two vlm dataset:
proactive_output
real
update GPU requirements for client and server
@jianbing119 jianbing119 force-pushed the vlm_chatToken_scenario branch from 7a72c78 to d70455b Compare May 26, 2026 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Reviewers wanted Extra attention is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants