Skip to content

cli.md

Latest commit

 

History

History
239 lines (187 loc) · 11.9 KB

cli.md

File metadata and controls

239 lines (187 loc) · 11.9 KB
Raw

CLI Test Harness

BINARY=build/Build/Products/Debug/OpenSuperMLX.app/Contents/MacOS/OpenSuperMLX

# List all commands
$BINARY --help

# Per-command usage
$BINARY help transcribe

Quick Start

# Smoke test — always works, no model needed
$BINARY diagnose --json

# Transcribe a file
$BINARY transcribe audio.wav --json

# Simulate streaming pipeline (exercises ring buffer → inference → events)
$BINARY stream-simulate audio.wav --json

# Run benchmark with accuracy check
$BINARY benchmark audio.wav --reference-text "reference text" --json

All commands accept --json (structured output to stdout), --quiet (suppress stderr progress), and --verbose. stream-simulate --verbose also writes transcription events to stderr.

Command Reference

Command Purpose Main arguments and subcommands
transcribe <file> Transcribe one audio file --language, --no-correction, --temperature
stream-simulate <file> Feed a file through the streaming pipeline --language, --chunk-duration, --disable-repetition-recovery
correct <text> Apply configured LLM correction (pass "" as <text> with --file) --file, --provider, --prompt
config Read or change application settings list, get <key>, set <key> <value>
recordings Inspect and manage recording history list, search <query>, show <id>, delete <id>, regenerate <id>
queue Manage imported-file transcription add <files>..., status, process
mic Inspect or select input devices list, select <device>
model Show the built-in model list
benchmark <file> Measure accuracy, speed, and memory --language, --runs, --wer-threshold, --reference-text, --suite
diagnose Print an environment snapshot No command-specific arguments

Run $BINARY help <command> or $BINARY help <command> <subcommand> for generated usage and defaults.

LLM correction sizing

correct and every app path (streaming stop, file import, queue, regenerate) run the same flow after transcription ends. The provider's Model Limits settings (openAIContextTokens / openAIMaxOutputTokens / openAIThinkingEnabled / openAIThinkingEffort, and the bedrock… equivalents) decide how much text one request can correct:

  • Tokens are estimated as 1 per CJK character, ½ per other non-ASCII character and ¼ per ASCII character.
  • Capacity is the smaller of (max output − thinking reserve) / 1.1 and (context − prompt − thinking reserve) / 2.1, times 0.8.
  • A transcript within capacity is corrected in one request (mode: "single"). A longer one is split at sentence ends into the fewest balanced chunks that fit and corrected one by one (mode: "chunked"). A custom prompt runs once per chunk.
  • If a single request is truncated, times out, returns suspiciously short output, or hits a token-limit error, a transcript of at least 1,000 tokens is retried in at least two chunks.
  • A failed chunk keeps its original text. Network errors stop the remaining chunks; a token-limit error on the first chunk stops with a hint to lower the limits.
  • Thinking parameters the model rejects are dropped automatically and remembered until the app restarts. openAIExtraBody (a JSON object) is merged into every request and disables that fallback when it sets reasoning_effort, reasoning, thinking or output_config itself.

correct "" --file transcript.txt --json reports mode, chunk_count, failed_chunk_count, processing_time_s and, when some chunks failed, warning. It exits with llm_correction_failed only when nothing was corrected.

Streaming repetition recovery

Streaming compares generated text with the actual text-conditioning prefix (up to 150 tokens). The detector uses normalized Unicode scalars, with one rule for mixed scripts: a new span must contain at least 48 UTF-8 bytes and its edit distance must not exceed 15% of the longer compared span. Spaces and punctuation remain present. This detects textual repetition; it is not an acoustic hallucination score.

On detection, the session uses the saved text checkpoint at or before the last 8-second window's start. It preserves that checkpoint's pending tail and replaces the later text with a fresh decode of retained mel frames, using a new encoder cache, empty text conditioning, and new decoder KV. The retry has a 256-token budget and must end normally. Fresh text is preserved in full: adjacent matching words are not deleted merely because they repeat the frozen prefix. Audio queued after the recovery endpoint is processed once. Watchdogs measure consumed audio, so a large feed does not count queued audio as time already spent decoding. Inference resets preserve the frontend, queued mel, and accepted pending tail.

If recovery cannot finish, the checkpointed range is skipped and transcription continues from its end with fresh inference state. The frontend and queued audio are preserved. is_complete stays false, and the CLI result's gaps array reports each skipped range with start_seconds, end_seconds, and reason. These positions refer to the source recording; synthetic tail silence is excluded. Recording continues in the app, which reports the file name, time range, and failure reason in error logs and in the pipeline trace when Debug Mode is enabled. Successful recovery does not guarantee correct words at the boundary or remove misrecognitions already present in the checkpoint.

$BINARY stream-simulate audio.wav --json --verbose
$BINARY stream-simulate audio.wav --disable-repetition-recovery --json --verbose

--chunk-duration controls file feeding, not the 2-second decode cadence. Recovery-attempt mel ranges are inference coordinates (100 frames/second). Failure reports additionally include recording positions that account for capture backpressure drops and inference resets. The existing capture backpressure policy is unchanged. This path does not alter the separate transcribe command.

Neural Engine Audio Encoder

The audio encoder runs on the Neural Engine by default; the text decoder stays on the GPU. Turn the setting off to run the encoder on the GPU instead:

$BINARY stream-simulate audio.wav --json    # data.audio_encoder is "neural_engine"
$BINARY config set useNeuralEngineAudioTower false
$BINARY stream-simulate audio.wav --json    # data.audio_encoder is "gpu"

The app looks for the compiled encoder at mlx-models/coreml/qwen3_asr_audio_tower_int8.mlmodelc and otherwise downloads it from ax0t/Qwen3-ASR-1.7B-CoreML-INT8. If it cannot be loaded, the app falls back to the GPU encoder, shows a toast, and audio_encoder reports "gpu". Scripts/qwen3_audio_tower_coreml.py builds the encoder; Scripts/qwen3_ane_audio_tower_eval.py compares GPU utilization, memory, and speed against a pre-change build.

Transcript MCP Bridge

The transcript MCP bridge is opt-in and binds only to 127.0.0.1.

$BINARY config set transcriptMCPEnabled true
$BINARY config set transcriptMCPPort 17653

Restart the app after changing these settings. Then register the local MCP endpoint from an agent client:

codex mcp add opensupermlx-transcript --url http://127.0.0.1:17653/mcp

Available tools:

Tool Purpose
get_transcript_delta Pull transcript segments after after_cursor
search_transcript Search final transcript text with bounded context
get_transcript_window Fetch nearby transcript context around a cursor
list_transcript_sessions List recent live transcript sessions

Tool responses keep human summaries in content[].text. Machine-readable values such as next_cursor, segments, and matches are returned only in structuredContent.

Error Codes

With --json, command failures are written to stderr as {"status":"error","command":"...","error":{"code":"...","message":"..."}}.

Code Meaning
model_not_found The requested model is not in the catalog
model_not_cached The model must be downloaded before use
model_load_failed The selected model could not be loaded
audio_file_not_found The input path does not exist
audio_format_unsupported The audio file cannot be decoded
transcription_failed Transcription did not complete
stream_timeout Streaming simulation exceeded its time limit
llm_correction_failed Post-transcription correction failed
database_error A recording database operation failed
audio_file_missing A history entry no longer has usable audio
invalid_config_key The configuration key is unknown
invalid_config_value The value does not match the key's type

Running CLI Tests

# All CLI tests
xcodebuild test -scheme OpenSuperMLX -destination 'platform=macOS,arch=arm64' \
  -derivedDataPath build -clonedSourcePackagesDirPath SourcePackages \
  CODE_SIGNING_ALLOWED=NO -only-testing:OpenSuperMLXTests

# Single command test class
xcodebuild test -scheme OpenSuperMLX -destination 'platform=macOS,arch=arm64' \
  -derivedDataPath build -clonedSourcePackagesDirPath SourcePackages \
  CODE_SIGNING_ALLOWED=NO -only-testing:OpenSuperMLXTests/TranscribeCommandTests

Pre-Commit CLI Verification

If a CLI command can exercise the code path you changed, run it before committing.

What you changed Verify with
Transcription, model, ITN transcribe <audio> --json
Streaming pipeline stream-simulate <audio> --json
LLM correction correct "text" --json; long text: correct "" --file transcript.txt --json
Settings / AppPreferences config get <key>
Recordings DB recordings list --json
Audio devices mic list --json
Model catalog model list --json
Any change (minimum bar) diagnose --json

For bug fixes: reproduce via CLI first → fix → verify via CLI → include repro steps in commit message.

End-to-End Streaming Tests

Automated XCTest suite that exercises the full streaming pipeline with a real model and real audio. Requires:

  1. Model downloaded locally (mlx-community/Qwen3-ASR-1.7B-5bit)
  2. Audio file path provided via OPENSUPERMLX_E2E_AUDIO environment variable

Tests skip automatically if either prerequisite is missing.

# Run all E2E streaming tests with a short audio file
OPENSUPERMLX_E2E_AUDIO=/path/to/audio.wav \
  xcodebuild test -scheme OpenSuperMLX -destination 'platform=macOS,arch=arm64' \
  -derivedDataPath build -clonedSourcePackagesDirPath SourcePackages \
  CODE_SIGNING_ALLOWED=NO -only-testing:OpenSuperMLXTests/StreamingE2ETests

# Run a single E2E test
OPENSUPERMLX_E2E_AUDIO=/path/to/audio.wav \
  xcodebuild test -scheme OpenSuperMLX -destination 'platform=macOS,arch=arm64' \
  -derivedDataPath build -clonedSourcePackagesDirPath SourcePackages \
  CODE_SIGNING_ALLOWED=NO \
  -only-testing:OpenSuperMLXTests/StreamingE2ETests/testStreamingProducesNonEmptyText

# Long-duration stability test (audio ≥2 min required, otherwise skipped)
OPENSUPERMLX_E2E_AUDIO=/path/to/long-audio.wav \
  xcodebuild test -scheme OpenSuperMLX -destination 'platform=macOS,arch=arm64' \
  -derivedDataPath build -clonedSourcePackagesDirPath SourcePackages \
  CODE_SIGNING_ALLOWED=NO \
  -only-testing:OpenSuperMLXTests/StreamingE2ETests/testStreamingNoStallOnLongAudio
Test What it verifies Audio requirement
testStreamingProducesNonEmptyText Basic pipeline: audio in → text out Any duration
testStreamingReceivesIntermediateUpdates displayUpdate events flow during streaming Any duration
testStreamingCompletesWithinTimeLimit No hang — processing < 5× audio duration Any duration
testStreamingTextGrowsOverTime Text accumulates across multiple updates Any duration
testStreamingNoStallOnLongAudio No stall >60s between updates (regression for backpressure fix) ≥2 minutes
testStreamingStatsReceived Stats events with peak memory are emitted Any duration