Skip to content

Add per-model token provider to useReplicatedModels#2459

Draft
Copilot wants to merge 4 commits intomainfrom
copilot/update-token-fetching-logic
Draft

Add per-model token provider to useReplicatedModels#2459
Copilot wants to merge 4 commits intomainfrom
copilot/update-token-fetching-logic

Conversation

Copy link

Copilot AI commented Mar 9, 2026

When a Vue.js client switches branches, the existing useReplicatedModels would attempt to connect with a stale token, receive a 403, and provide no opportunity for the caller to supply a fresh one. The token provider had no access to model parameters, making repository/branch-scoped tokens impossible.

Changes

useReplicatedModels — new getToken parameter and URL string support

  • Added optional third parameter: getToken?: (params: ReplicatedModelParameters) => Promise<string | null>
  • First parameter client now also accepts a server URL string (in addition to ClientJS), eliminating the need to use useModelClient separately
  • When client is a URL string, a single ClientJS is created and shared across all models and all branch switches via useModelClient internally. This preserves the version cache so that switching branches only fetches the delta, and cross-model references continue to resolve correctly.
  • The ClientJS is only disposed and recreated when the URL itself changes; model/branch changes reuse the existing client.
  • Added a createClient optional parameter (defaults to connectClient via useModelClient) as a test seam.
  • Existing ClientJS usage is fully backwards compatible.

ReplicatedModelParameters — new tokenProvider field (Kotlin)

  • Added optional tokenProvider: (() -> Promise<String?>)? to ReplicatedModelParameters (changed from data class to class).
  • When getToken is provided, each model in models receives its own independent () => getToken(p) closure as tokenProvider — no shared state, no [0] index.

Per-binding token routing in ClientJS (Kotlin)

  • Replaced Ktor's cached bearer-auth with PerBindingAuthPlugin — a custom Ktor client plugin that intercepts every individual HTTP request, parses the URL path for /repositories/{repoId}/branches/{branchId}, and looks up the matching token from a bindingTokenProviders map shared with ClientJSImpl.
  • ClientJSInternalBuilder subclasses ModelClientV2Builder to install PerBindingAuthPlugin alongside the standard JSON/timeout/retry plugins, replacing the old single-token bearer-auth approach.
  • Falls back to the global token (from connectClient's bearerTokenProvider) for non-binding requests (e.g. /v2/server-id).
  • Tokens are fetched on every request — not just on connection or 401 — so credentials are always fresh.

Usage example

const { rootNodes } = useReplicatedModels(
  "https://my-model-server/v2",
  models,                          // reactive — branch switches reuse the same client
  async (params) => fetchToken(params.repositoryId, params.branchId),
);

Tests

  • Updated three URL + getToken tests to reflect per-binding behaviour: verify tokenProvider is attached per model and delegates to getToken when invoked, the same ClientJS is reused on branch switch, and a new ClientJS is created on URL change.
  • Added extractBindingKey unit tests in Kotlin.

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

…meters parameter

Co-authored-by: abstraktor <569215+abstraktor@users.noreply.github.com>
Copilot AI changed the title [WIP] Update token fetching logic for replicated models Add per-model token provider to useReplicatedModels Mar 9, 2026
@abstraktor
Copy link
Collaborator

@copilot that won't work: the models must all share the same ClientJS instance. Also, the past and future models should share the same ClientJS instance. First, for being able to resolve cross-model references. Second, switching a branch must not result in refetching the entire version, but only in fetching the delta. The ClientJS instance is keeping the cache of versions to ensure we only transmit what is not known yet.

I like the idea of changing the API to work without useModelClient though.

…itching

Co-authored-by: abstraktor <569215+abstraktor@users.noreply.github.com>
Copy link
Author

Copilot AI commented Mar 9, 2026

@copilot that won't work: the models must all share the same ClientJS instance. Also, the past and future models should share the same ClientJS instance. First, for being able to res...

Redesigned in ee23b4b. All models now share a single ClientJS per URL (kept alive across model/branch changes to preserve the version cache for delta fetching and cross-model reference resolution). The key changes:

  • Two effects: Effect 1 manages the URL → ClientJS lifecycle (only re-runs when the URL changes). Effect 2 handles model connections and re-runs when either models or the resolved ClientJS (urlClientRef) changes.
  • Dynamic token provider: The client is created with a tokenProvider closure that reads a currentModelsForToken mutable variable. Before every startReplicatedModels call, currentModelsForToken is updated with the new model params. This means the Ktor bearer-auth plugin (loadTokens/refreshTokens) always calls getToken(currentModels[0]) with up-to-date params.
  • Testability: Added a createClient parameter (defaults to connectClient) as a test seam, consistent with the getClient pattern in useModelClient. Three new tests verify the behaviour.

Comment on lines +132 to +135
return createClient(
clientOrUrl,
getToken !== undefined ? tokenProvider : undefined,
);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the client only refetches the token upon 401. it has only means to store one token so far. Change the implementation of the client to be able to keep a token per "binding".

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 98d2228. Added PerBindingAuthPlugin — a custom Ktor plugin installed in ClientJSInternalBuilder.configureHttpClient instead of the standard bearer-auth plugin. On every HTTP request the plugin parses the URL path for /repositories/{repoId}/branches/{branchId}, looks up the matching token from a mutable map (bindingTokenProviders) shared with ClientJSImpl, and sets the Authorization header. Added tokenProvider: (() -> Promise<String?>)? to ReplicatedModelParameters; startReplicatedModels registers each binding's provider in the map when called. Falls back to the global token (from connectClient) for non-binding requests.

let currentModelsForToken: ReplicatedModelParameters[] = [];
const tokenProvider = (): Promise<string | null> => {
if (currentModelsForToken.length === 0) return Promise.resolve(null);
return getToken!(currentModelsForToken[0]);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why [0]? this looks like an artifact from TDD with too low coverage. This doesnt make much sense

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 98d2228. The [0] and the shared currentModelsForToken variable are gone. Each model in the modelsValue array now gets its own tokenProvider closure — () => getToken(p) — passed into a freshly constructed ReplicatedModelParameters. The per-binding token is then routed to the right HTTP requests by the URL-aware PerBindingAuthPlugin in the Kotlin client.

// Re-runs when `client` changes. When the URL changes the old ClientJS is
// disposed and a new one is created (with the same token-provider closure).
// ---------------------------------------------------------------------------
useLastPromiseEffect<ClientJS>(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: how is this any better than calling useModelClient(…) in here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now using useModelClient internally (98d2228). When client is a URL string, useReplicatedModels delegates client lifecycle management to useModelClient with a getter that returns the URL (or null when client is a ClientJS). The two-effect architecture is collapsed into a single effect, and the createClient seam is forwarded directly to useModelClient's getClient parameter — consistent with the existing pattern.

…odels

Co-authored-by: abstraktor <569215+abstraktor@users.noreply.github.com>
Copy link
Author

Copilot AI commented Mar 9, 2026

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • artifacts.itemis.cloud
    • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java /usr/lib/jvm/temurin-17-jdk-amd64/bin/java --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED --add-opens=java.base/java.nio.charset=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.xml/javax.xml.namespace=ALL-UNNAMED -XX:MaxMetaspaceSize=2g -XX:&#43;HeapDumpOnOutOfMemoryError -Xms256m -Xmx1024m -Dfile.encoding=UTF-8 -Duser.country -Duser.language=en (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants