[✨ FEATURE] Remote transcription backend with the ability to run the speech-to-text model on a separate machine

[salimhamed]

Problem

All of FluidVoice's speech engines run on-device. The SpeechModel catalog (FluidAudio/CoreML Parakeet, Qwen3, Cohere, Nemotron, Apple Speech, whisper.cpp) is exclusively local, and every
TranscriptionProvider implementation loads and executes the model on the Mac running the client.

This means transcription quality and latency are capped by the hardware of the machine you're dictating on. As a result, users cannot:

• Use a lightweight or older Mac (including Intel Macs, which are locked out of all the Apple Silicon–only FluidAudio models) as the dictation client while a more powerful machine on the
  LAN does the inference.
• Run a single larger, higher-accuracy model (e.g. Whisper Large, or a GPU-served model) on a home server and share it across several Macs.

The built-in local API server exposes POST /v1/transcribe, so a FluidVoice instance can already serve transcriptions, but it binds only to 127.0.0.1, and the app has no way to consume a remote transcription endpoint.

The AI-enhancement layer already supports custom OpenAI-compatible base URLs for the post-processing LLM, so remote inference is an established pattern in the app, just not for speech to text.

Proposed solution

Add a "Remote server" speech engine option alongside the existing local models:

1. New TranscriptionProvider implementation that sends captured audio to a user-configured HTTP endpoint and returns the transcript.
2. Settings UI under Speech Recognition, including: base URL, optional API key, model name, and a "Test connection" button. This mirrors the custom-endpoint UX that already exists for AI post-processing providers.
3. Wire format would support the OpenAI-compatible POST /v1/audio/transcriptions (multipart WAV upload) so the feature works out of the box with the broad ecosystem of self-hostable servers (e.g., whisper.cpp server, faster-whisper/speaches, LocalAI, vLLM, ONNX/NIM Parakeet servers, etc.)
4. Optionally, allow the existing built-in local API server to bind to a LAN interface (opt-in, with an API token). This would allow one FluidVoice install on a powerful Mac to act as the transcription backend for other FluidVoice clients.

Streaming preview can be out of scope for a first version.

Alternatives considered

No response

[altic-dev]

You can port forward from the mac to any other PC and work on it. do you want to ping from other device?

[salimhamed]

You can port forward from the mac to any other PC and work on it. do you want to ping from other device?

@altic-dev Thanks for the quick reply! I think that's the reverse direction of what I'm asking for.

Port-forwarding would let other devices call the Local API on my Mac, using my Mac as the transcription server.

What I'm asking for is the opposite. I want the Mac running the FluidVoice client to offload the STT inference to a different, more powerful machine, and receive the transcript back.

Here's my concrete use case: I dictate on a MacBook Air running FluidVoice. On my LAN, I have a server with a dedicated GPU running an OpenAI-compatible transcription endpoint. I want FluidVoice to route dictation audio to this external endpoint instead of running inference on the client Mac.

I was thinking this could just be a "Remote server (custom endpoint)" option in the speech model picker, similar to how custom OpenAI-compatible providers already work for AI post-processing.

[grohith327]

Hey @salimhamed, a couple of things:

1. There is a transcribe route in the app right now but its not fully built out, you can't start the server directly from the app GUI but we are looking to build this change this. Make it easier to run the transcription/post-processing server
2. Right now since we are a Mac app, our backend is in Swift and the models use CoreML. We have a windows/linux version coming out soon that should leverage any GPU

Once these two are in, you should be able to achieve what you are looking for.

[salimhamed]

Hi @grohith327, thanks for the reply and details about your roadmap.

The two roadmap items you mentioned are focused on running FluidVoice's transcribe server independently and a Windows/Linux build that could run on custom hardware. Those would let me stand up a FluidVoice instance as the backend on my linux server with a dedicated GPU, which is great.

However, the functionality I'm asking for is slightly different. I'm asking for a client side option, in the Mac app's speech model picker, to route dictation audio out to a configurable endpoint (base URL + optional API key) that is compatible with the OpenAI /v1/audio/transcriptions format. This would let me self-host a transcription backend that is compatible with FluidVoice.

This would decouple model choice from the models bundled with the app. Users could run any model served behind the OpenAI transcription API without waiting for each to be packaged into a release.

Is this client-side "remote endpoint" option something you'd be open to?

[altic-dev]

So plug in any voice model. Cloud/remote just like LLMs in Enhancement session?

…

On Wed, Jul 8, 2026 at 10:19 AM Salim ***@***.***> wrote: *salimhamed* left a comment (altic-dev/FluidVoice#547) <#547 (comment)> Hi @grohith327 <https://github.com/grohith327>, thanks for the reply and details about your roadmap. The two roadmap items you mentioned are focused on running FluidVoice's transcribe server independently and a Windows/Linux build that could run on custom hardware. Those would let me stand up a FluidVoice instance as the backend on my linux server with a dedicated GPU, which is great. However, the functionality I'm asking for is slightly different. I'm asking for a client side option, in the Mac app's speech model picker, to route dictation audio out to a configurable endpoint (base URL + optional API key) that is compatible with the OpenAI /v1/audio/transcriptions format. This would let me self-host a transcription backend that is compatible with FluidVoice. This would decouple model choice from the models bundled with the app. Users could run any model served behind the OpenAI transcription API without waiting for each to be packaged into a release. Is this client-side "remote endpoint" option something you'd be open to? — Reply to this email directly, view it on GitHub <#547?email_source=notifications&email_token=BVSOW2QTJQ7I65MKDJSAY5L5DZ7DPA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIOJRG4ZTQNRZGIYKM4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLDGN5XXIZLSL5RWY2LDNM#issuecomment-4917386920>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BVSOW2VUXIDDDX3IVIMXQZ35DZ7DPAVCNFSNUABGKJSXA33TNF2G64TZHMYTANRRGMZDOMZRGE5US43TOVSTWNBYGMYTEMBWHA2DDILWAI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[salimhamed]

@altic-dev that's exactly right. For my specific use-case, I want to locally host a voice model on linux server with a powerful GPU on my LAN, then point FluidVoice at it. This will free up resources on my Mac and allow me to experiment with other voice models.

[grohith327]

Haven't thought about FluidVoice being client only, you love our interface that much huh 😅. Fair ask

[salimhamed]

you love our interface that much huh

It's just that good! ;)

[nishantkumar1292]

any updates on this? I too would like to use a bigger model, specially on my phone.

[altic-dev]

@nishantkumar1292 part of our plan ;)

Focusin on FV on iOS first and a local model rn - will enhance the experience with some sort of 'Fluid Link' as we go :)

[ben-vargas]

Thanks for listing xAI here, that’s the backend I wanted too.

I’m not planning to open a PR, given the local-first focus. For personal use I added an opt-in Grok Speech engine on a fork:

https://github.com/ben-vargas/ai-fluidvoice/tree/feat/grok-streaming-stt

It’s a Voice Engine option, never the default. Live dictation goes to xAI over wss://api.x.ai/v1/stt (not the OpenAI /v1/audio/transcriptions REST shape). File/meeting transcription uses their REST STT endpoint. The STT credential is separate from the xAI key under AI Enhancement.

Auth is either an xAI API key, or a read-only Grok Build CLI session (~/.grok/auth.json if Grok CLI / Grok Build is already logged in). FluidVoice does not run OAuth and does not write that file and never itself refreshes the token, but will launch the Grok Build CLI to get it to refresh its token. In the current build, hold-to-talk uses the CLI OAuth token; the API-key WebSocket path is still gated.

Not asking for this to land upstream; just leaving a pointer in case anyone else is poking at a cloud engine inside FluidVoice.

[niklasfrick]

+1 on this, and adding a concrete use case.

Setup

• Client: MacBook running FluidVoice
• Server: NVIDIA DGX Spark on my LAN, serving Whisper Large / Parakeet via an OpenAI-compatible transcription endpoint (vLLM / NIM)

I'd like FluidVoice to send the dictation audio to the DGX Spark and just insert the transcript, instead of running the model on the Mac.

Why this still fits "local-first"

Nothing leaves my network. It's the same trust model as a local model, just on a different machine. The AI Enhancement layer already allows exactly this for the LLM (custom base URL), so it would be consistent to allow it for the voice engine too.

What I'd need

A "Custom endpoint" entry in the Voice Engine picker:

• Base URL
• Optional API key
• Model name
• Test connection

Wire format POST /v1/audio/transcriptions (multipart WAV) covers whisper.cpp server, faster-whisper / speaches, LocalAI, vLLM, NIM and similar out of the box. Streaming can come later.

@ben-vargas nice work on the Grok fork. Did you build it as a generic provider, or is it tied to the xAI WebSocket shape? An OpenAI REST variant of that might be the shortest path to a self-hosted option.

@altic-dev good to hear it's on the plan. Would a community PR for an opt-in OpenAI-compatible engine (never default) be welcome, or is "Fluid Link" meant to cover this in a different way?

Happy to test against the DGX Spark once there's anything to try.

[altic-dev]

Hey! Yeah the link is to run FluidVoice anywhere and open up the server to local network. we're working on linux builds n more. Not interested in opening up openai endpoint for voice models yet as it might cause a lot more confusion in general as we're trying to make it simpler. but yeah, some form of native comm is what I am thinking!

[ben-vargas]

The Grok STT WebSocket used is not compatible with OpenAI WebSocket, it would have to be an additional provider using REST at end of dictation, so an additional provider.

Having Fable dig into the Codex CLI repo, they're only doing realtime WebSocket transcription via api key until it supports OAuth, so REST would be the path for now, but not supported in the shape of the xAI WebSocket.