Add whisper lid stage - #57
Conversation
9c038cc to
8db1a5a
Compare
Signed-off-by: Fan Qian <fqian@nvidia.com>
Signed-off-by: Fan Qian <fqian@nvidia.com>
8db1a5a to
865a5ec
Compare
| msg = "OpenAI Whisper is required for WhisperLangIDStage. Install: pip install openai-whisper" | ||
| raise ImportError(msg) from exc | ||
|
|
||
| _WHISPER_MAX_SAMPLES = 30 * 16_000 # 30 s at 16 kHz |
There was a problem hiding this comment.
max samples from VAD are 40 secs, please change this.
There was a problem hiding this comment.
Whisper has a max audio length of 30s, we can only trim the audio for it to run.
|
|
||
| for task_idx, language_probabilities in zip(chunk_indices, probabilities, strict=True): | ||
| language = max(language_probabilities, key=language_probabilities.get) | ||
| tasks[task_idx].data[self.output_key] = language |
There was a problem hiding this comment.
Store the LID result in LangIDResult dataclass.
| return task | ||
|
|
||
| # Non-Indic: cross-check with Whisper when available. | ||
| if whisper_result is not None: |
There was a problem hiding this comment.
Add whisper as "tertiary" model in run_metadata_pipeline.py
Signed-off-by: Fan Qian <fqian@nvidia.com>
|
|
||
| whisper_grp = ap.add_argument_group("Whisper LID (tertiary)") | ||
| whisper_grp.add_argument( | ||
| "--whisper", |
There was a problem hiding this comment.
Let's run whisper by default, and mark it as secondary or teritary based on indic flag. Please remove this flag.
Signed-off-by: Fan Qian <fqian@nvidia.com>
| batch_size: int = 8 | ||
| max_duration_sec: float = 30.0 # Whisper's input window is 30 s; base defaults to 10 s | ||
|
|
||
| _model: Any = field(default=None, init=False, repr=False) |
There was a problem hiding this comment.
Please add support for inference using local model path as well.
|
|
||
| def _load_model(self, device: str | torch.device) -> Any: | ||
| try: | ||
| import whisper |
There was a problem hiding this comment.
Add whisper package to toml and update lock.
56fbbd2 to
f5e3b7c
Compare
| for the shared waveform/output arguments. | ||
| """ | ||
|
|
||
| tag: str = "tertiary" |
There was a problem hiding this comment.
Sorry, forgot to remove. Fixed now.
|
Please add description. |
3 participants
Description
Usage
# Add snippet demonstrating usageChecklist