Skip to content

Add FunASR/SenseVoice API backend - #1554

Open
LauraGPT wants to merge 10 commits into
chidiwilliams:mainfrom
LauraGPT:codex/funasr-api-backend
Open

Add FunASR/SenseVoice API backend#1554
LauraGPT wants to merge 10 commits into
chidiwilliams:mainfrom
LauraGPT:codex/funasr-api-backend

Conversation

@LauraGPT

@LauraGPT LauraGPT commented Jul 14, 2026

Copy link
Copy Markdown

Summary

  • add a first-class FunASR API model backed by a separately running FunASR/SenseVoice OpenAI-compatible server
  • support file/folder imports, live recording, and the funasrapi CLI model type
  • add configurable base URL and model settings, with environment overrides for protected or remote endpoints
  • keep FunASR credentials isolated from the saved OpenAI key and enforce the backend's transcription-only contract
  • document server setup and usage across preferences, file import, live recording, and CLI guides

Boundary fixes

  • allow live recording to start without a local model path for FUNASR_API, matching the existing OpenAI API behavior
  • disable unsupported word-level parsing for FunASR responses so a text-only or empty-segment response cannot raise TypeError or discard the transcript
  • omit prompt and timestamp_granularities fields that the FunASR endpoint does not support

Current-main refresh

  • merge current main@3fa37a9d5238d29fa75a66cdb76555fd2820957e without rewriting the already reviewed feature history
  • incorporate the upstream manual-build Python-version fix and completed Chinese translations; neither overlaps the 27-file FunASR change
  • current SSH-signed, DCO-compliant, and GitHub-verified head: 2b77a03619d6ccd9159be70f348bec0b78b6b8ed

Verification

  • TDD live-recording regression failed before the fix because RecordingTranscriber was called 0 times; it now starts once with the expected API options
  • TDD word-timing regression reproduced TypeError: 'NoneType' object is not iterable; it now returns the complete text fallback and sends no word-timing request
  • exact-head focused API, CLI, model-loader, file/recording, preferences, and UI suite: 323 passed
  • exact-head broad repository suite: 1134 passed, 7 skipped
  • frozen uv lock check with uv 0.12.1, Python 3.12, Transformers 5.14.1, and Torch 2.8.0+cu129
  • the current-main and candidate Ruff findings are identical after line-number normalization; fatal rules pass across every changed Python file, and all newly added Python files pass the complete Ruff ruleset
  • changed-file Python compilation, safe pre-commit whitespace/EOF/large-file hooks, submodule cleanliness, conflict-marker scan, lock check, and git diff --check

GitHub currently reports the PR mergeable with no unresolved review threads. The exact-head CI and Snapcraft workflow runs are action_required with zero jobs, so they require upstream approval before GitHub can execute them. The prior and refreshed heads are retained on codex/backup-buzz-1554-086a241-20260803 and codex/backup-buzz-1554-prepush-2b77a03-20260803 in the contributor fork.

Closes #1140

@LauraGPT

LauraGPT commented Jul 14, 2026

Copy link
Copy Markdown
Author

Follow-up for head c7951ec: review found and fixed two contributor-side boundaries before maintainer review:

  • live recording now allows the empty local model path required by FUNASR_API;
  • FunASR file transcription disables unsupported word-level parsing and preserves text-only/empty-segment responses instead of raising TypeError or returning no transcript.

Current local verification is green: 13 focused FunASR tests, 96 recording-widget tests, and the broad repository suite (1037 passed, 4 skipped), plus Ruff on the changed production modules and git diff --check.

The new CI and Snapcraft runs are both action_required with zero jobs, so they need maintainer approval before GitHub can execute them. No rush given your summer schedule; approval and review whenever convenient would be appreciated.

@raivisdejus

Copy link
Copy Markdown
Collaborator

Summer is busy time, but sure, I will review this.

@LauraGPT

Copy link
Copy Markdown
Author

Refreshed this PR onto current main and resolved the only merge conflict in buzz/transcriber/recording_transcriber.py.

Conflict resolution kept the upstream use_transcriptions_endpoint guard for local whisper.cpp and preserved the FunASR API behavior that routes translate requests back to /audio/transcriptions when the configured API does not support translation.

Validation on exact head f1abc011c561:

  • python3 -m py_compile buzz/transcriber/recording_transcriber.py buzz/transcriber/audio_api.py
  • conflict-marker check on buzz/transcriber/recording_transcriber.py
  • QT_QPA_PLATFORM=offscreen PYTHONPATH=demucs_repo:deepmultilingualpunctuation:ctc_forced_aligner:whisper_diarization:$PWD /cpfs_speech/user/zhifu.gzf/codebase/Buzz-funasr-audit-20260714/.venv/bin/python -m pytest tests/transcriber/audio_api_test.py tests/transcriber/recording_transcriber_test.py tests/recording_transcriber_test.py -q -> 61 passed
  • git diff --check

GitHub now reports the PR MERGEABLE; there are no configured CI checks on the PR.

LauraGPT added 2 commits July 27, 2026 01:34
Signed-off-by: LauraGPT <18321252+LauraGPT@users.noreply.github.com>

# Conflicts:
#	docs/docs/index.md
Signed-off-by: LauraGPT <18321252+LauraGPT@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Add FunASR/SenseVoice as alternative transcription engine

2 participants