Skip to content

[FRONTEND] External phone camera management#3

Open
nicolas-rabault wants to merge 1 commit intomainfrom
external_cam
Open

[FRONTEND] External phone camera management#3
nicolas-rabault wants to merge 1 commit intomainfrom
external_cam

Conversation

@nicolas-rabault
Copy link
Collaborator

What is this PR about

In this PR we try to use the web interface to link local network devices cameras to feed the dataset.

How do we plan to do it

In the camera configuration dropdown, add a special camera option called "Phone".

This option must:

  • Always be visible
  • Be selectable multiple times

When a "Phone" camera is added:

  1. 🔍 Determine frontend base address:

    • If the user accesses the frontend via localhost or 127.0.0.1, detect the machine's local network IP address (e.g., 192.168.x.x).
    • Use this local network address for any mobile connections (QR code, streaming).
    • If the user is already on a LAN or public address, reuse it as-is.
  2. 🔐 Enable HTTPS locally (required for camera access on phones):

    • If the frontend is running locally (e.g., in development), enable HTTPS using:
      • A self-signed certificate (auto-generated if needed)
      • Or use mkcert to generate a trusted local certificate
    • Make sure the frontend is reachable via https://<local_ip>:<port> over Wi-Fi
  3. 🌐 Setup a WebRTC streaming endpoint on the backend:

    • Create a unique WebRTC session for each "Phone" camera.
    • This endpoint accepts incoming WebRTC video streams from phones.
  4. 📱 Generate a mobile streaming link with QR code:

    • Generate a URL: [https://<local_ip>:<port>]/remote_cam/[webrtc_id]
    • This page allows the phone to:
      • Select its camera using getUserMedia
      • Stream the video to the backend using WebRTC
  5. 🖥️ Frontend UI behavior:

    • While no stream is active, show the QR code in place of the camera feed preview.
    • Once the phone connects and starts streaming to the backend:
      • Show the live video preview in the miniature card on the dashboard
  6. 🔁 Stream routing from phone to frontend:

    • Phone streams video → Backend via WebRTC
    • Backend forwards (relays or rebroadcasts) the video to the frontend
    • Use either:
      • A WebRTC relay to forward the stream to the desktop browser
      • Or transcode into MJPEG or low-latency HLS/WebSocket for frontend preview
    • Keep latency low and reuse existing libraries where possible

Implementation constraints:

  • Use existing libraries for:
    • WebRTC signaling
    • QR code generation
    • Local HTTPS (e.g., mkcert, selfsigned, flask-tls)
  • Support multiple simultaneous phone camera streams
  • Keep code modular and simple

Optional:

  • Expire inactive sessions
  • Show connection status and stream quality

Related PRs

@nicolas-rabault
Copy link
Collaborator Author

@jurmy24 @DavidLMS I worked on implementing this very interesting feature, which was the main focus of our hackathon.
I managed to build a dedicated camera configuration interface for recording, and successfully generated a QR code on the miniature. Scanning it with a phone opens an HTTPS page that is trying to starts streaming the phone’s camera to the backend via WebRTC.
However, I'm still facing some connection issues that I haven’t been able to resolve.
Could you have a look.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant