use nexus proxy instead of provider-specific embeddings by kinjalh · Pull Request #3250 · pluralsh/console

kinjalh · 2026-02-24T15:51:28Z

use nexus proxy instead of provider-specific embeddings. Haven't deleted provider-specific embeddings yet for safety, first I think we make sure this is working before doing any deletions.

Test Plan

unit tests

Checklist

If required, I have updated the Plural documentation accordingly.
I have added tests to cover my changes.
I have added a meaningful title and summary to convey the impact of this PR to a user.

Plural Flow: console

michaeljguarino · 2026-02-24T15:53:20Z

lib/console/ai/nexus.ex

@@ -0,0 +1,143 @@
+defmodule Console.AI.Nexus do


This should basically just wrap up to the the openai provider, like how it's done in the AzureOpenAI one. Also, it needs to be definable in DeploymentSettings via graphql + kubernetes api, and not via env var.

greptile-apps · 2026-03-03T15:54:04Z

Greptile Summary

This PR introduces a Console.AI.Nexus provider module that routes embedding requests through a unified Nexus/Bifrost proxy instead of going directly to provider-specific embedding APIs. Both the Elixir schema (DeploymentSettings.AI.nexus) and the Go CRD (NexusSettings) are extended to hold the proxy URL, an optional bearer token, and an optional embedding model override.

Key changes:

New Console.AI.Nexus module that wraps Console.AI.OpenAI with the Nexus URL/token, following the same delegation pattern used by Console.AI.Azure.
Provider.embeddings/2 now checks for a configured Nexus URL first and short-circuits to it before attempting the normal embedding_provider lookup.
The nexus embed is added to the DeploymentSettings.AI Ecto schema with access_token stored as EncryptedString.
Corresponding Go CRD type (NexusSettings) added with a TokenSecretRef for Kubernetes secret injection, consistent with other provider settings types.
Good test coverage for Nexus.new/1, embeddings/2, error paths, and the Provider.embeddings/2 integration via Settings.update.

Issues found:

The embeddings/2 function in provider.ex ignores the documented opts[:client] parameter when Nexus is configured, creating an interface inconsistency.
Nexus.embeddings/2 lacks a guard for nil URL, which would cause a FunctionClauseError (not return an error tuple) if called directly with a nil URL, violating the @spec contract. The Provider.embeddings/2 call-site guards this, but the function itself should be defensive.

Confidence Score: 3/5

Additive changes to embeddings path are low-risk, but two interface/contract issues should be addressed: nil URL guard and opts parameter inconsistency.
Two verified issues require fixes before merge: (1) Nexus.embeddings/2 lacks a guard for nil URL, causing FunctionClauseError instead of returning an error tuple, violating the @SPEC contract. While upstream code guards this, the function should be defensive. (2) The documented opts[:client] parameter is silently ignored in the Nexus branch, creating an interface inconsistency—either document this behavior or add a comment explaining the design choice. Both are fixable with minor changes; the PR is otherwise well-structured and well-tested.
lib/console/ai/nexus.ex (add nil URL guard), lib/console/ai/provider.ex (clarify opts handling for Nexus)

_{Last reviewed commit: 33d979e}

greptile-apps

_{8 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps · 2026-03-03T15:54:07Z

lib/console/ai/provider.ex

  def embeddings(text, opts \\ []) do
    settings = Settings.cached()
-    with {:ok, %mod{} = client} <- client(settings, opts[:client] || :embedding),
-      do: mod.embeddings(client, text)
+    case settings do
+      %DeploymentSettings{ai: %AI{nexus: %{url: url} = nexus}} when is_binary(url) ->
+        Nexus.new(nexus) |> Nexus.embeddings(text)
+      _ ->
+        with {:ok, %mod{} = client} <- client(settings, opts[:client] || :embedding),
+          do: mod.embeddings(client, text)
+    end
  end


The embeddings/2 function ignores the opts[:client] hint when Nexus is configured. This is inconsistent with the documented behavior in the @doc block (line 125), which lists :client as a valid option.

When a caller explicitly passes client: :embedding or client: :tool, that hint is silently discarded in the Nexus branch. Since the :client option is documented as supported but not actually used by the Nexus path, either:

Update the docs to clarify that :client is ignored when Nexus is configured, or

Modify the logic to only use Nexus when :client is not explicitly set to something other than :embedding (if provider-specific routing is needed in the future)

Current behavior seems correct (Nexus should always take priority for embeddings), but the interface is misleading.

greptile-apps · 2026-03-03T15:54:08Z

lib/console/ai/nexus.ex

+  def embeddings(%__MODULE__{url: url, token: token, embedding_model: model}, text) do
+    OpenAI.new(%{
+      base_url: Path.join(url, "/v1"),
+      access_token: token,
+      embedding_model: model || @default_embedding_model
+    })
+    |> OpenAI.embeddings(text)
+  end


Path.join/2 requires both arguments to be binaries. If url is nil (a valid state since the struct field is String.t() | nil), calling this function directly will raise a FunctionClauseError rather than returning an error tuple, violating the @spec contract.

While Provider.embeddings/2 guards against this with when is_binary(url), direct callers to Nexus.embeddings/2 would not be protected. Add a guard or a separate clause to handle the nil case gracefully:

Suggested change

def embeddings(%__MODULE__{url: url, token: token, embedding_model: model}, text) do

OpenAI.new(%{

base_url: Path.join(url, "/v1"),

access_token: token,

embedding_model: model || @default_embedding_model

})

|> OpenAI.embeddings(text)

end

def embeddings(%__MODULE__{url: url, token: token, embedding_model: model}, text)

when is_binary(url) do

OpenAI.new(%{

base_url: Path.join(url, "/v1"),

access_token: token,

embedding_model: model || @default_embedding_model

})

|> OpenAI.embeddings(text)

end

def embeddings(%__MODULE__{url: nil}, _text),

do: {:error, "Nexus url is not configured"}

michaeljguarino

some of the greptile comments should also be handled I think too

michaeljguarino · 2026-03-04T01:07:24Z

lib/console/ai/nexus.ex

+  """
+  @spec completion(t(), Console.AI.Provider.history(), keyword) :: Console.error
+  def completion(%__MODULE__{}, _messages, _opts) do
+    {:error, "completion not implemented for Nexus provider - use chat endpoints directly"}


you should be able to call into the openai provider for all of the other callbacks here too

michaeljguarino · 2026-03-05T05:05:53Z

lib/console/ai/nexus.ex

+  """
+  def new(opts \\ %{}) do
+    %__MODULE__{
+      url: Map.get(opts, :url),


this is actually deployed as a sidecar, so you should be able to auto-configure it against "http://localhost:8080/...", for instance, you can see it configured here: https://console.plrldemo.onplural.sh/cd/clusters/a1748282-ce8b-48ab-ae7e-326e74fce04e/services/2a95775e-4865-449a-b160-75de27590a0d/pods/plrl-console/console-74dd89c868-jk7v6

(if you auto-configure it you'll need to ensure its using the correct path, can ask sebastian what the path will be for openai endpoints)

michaeljguarino

two minor nits

michaeljguarino · 2026-03-10T16:34:06Z

lib/console/ai/nexus.ex

+    }
+  end
+
+  defp get_url(opts) do


use function head matches here (always more readable and terser)

michaeljguarino · 2026-03-10T16:35:25Z

lib/console/ai/provider.ex

-    with {:ok, %mod{} = client} <- client(settings, opts[:client] || :embedding),
-      do: mod.embeddings(client, text)
+    case settings do
+      %DeploymentSettings{ai: %AI{nexus: %{url: url} = nexus}} when is_binary(url) ->


why does this need to be special cased? It should just use the configured nexus client.

ok i see the issue, the client/2 function below doesn't properly select a nexus client, and my guess is the AI didn't appropriately add it there. Two things:

You'd need more test coverage to actually uncover the issue here, my guess is only embeddings are tested and AI force-fixed here.

Worth actually reading contextual code to catch these prior to review.

kinjalh added the enhancement New feature or request label Feb 24, 2026

michaeljguarino reviewed Feb 24, 2026

View reviewed changes

kinjalh force-pushed the use-nexus-proxy branch 3 times, most recently from 5abe0fd to 33d979e Compare March 3, 2026 15:18

kinjalh marked this pull request as ready for review March 3, 2026 15:48

greptile-apps bot reviewed Mar 3, 2026

View reviewed changes

michaeljguarino reviewed Mar 4, 2026

View reviewed changes

kinjalh force-pushed the use-nexus-proxy branch from 33d979e to d31857a Compare March 4, 2026 19:42

michaeljguarino reviewed Mar 5, 2026

View reviewed changes

kinjalh force-pushed the use-nexus-proxy branch from d31857a to 3191414 Compare March 9, 2026 13:44

michaeljguarino reviewed Mar 10, 2026

View reviewed changes

use nexus proxy instead of provider-specific embeddings

dd41d9d

kinjalh force-pushed the use-nexus-proxy branch from 3191414 to dd41d9d Compare March 13, 2026 14:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

use nexus proxy instead of provider-specific embeddings#3250

use nexus proxy instead of provider-specific embeddings#3250
kinjalh wants to merge 1 commit intomasterfrom
use-nexus-proxy

kinjalh commented Feb 24, 2026

Uh oh!

michaeljguarino Feb 24, 2026

Uh oh!

greptile-apps bot commented Mar 3, 2026

Uh oh!

greptile-apps bot left a comment

Uh oh!

greptile-apps bot Mar 3, 2026

Uh oh!

greptile-apps bot Mar 3, 2026

Uh oh!

michaeljguarino left a comment

Uh oh!

michaeljguarino Mar 4, 2026

Uh oh!

michaeljguarino Mar 5, 2026

Uh oh!

michaeljguarino left a comment

Uh oh!

michaeljguarino Mar 10, 2026

Uh oh!

michaeljguarino Mar 10, 2026

Uh oh!

michaeljguarino Mar 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

kinjalh commented Feb 24, 2026

Test Plan

Checklist

Uh oh!

michaeljguarino Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot commented Mar 3, 2026

Greptile Summary

Confidence Score: 3/5

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

michaeljguarino left a comment

Choose a reason for hiding this comment

Uh oh!

michaeljguarino Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

michaeljguarino Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

michaeljguarino left a comment

Choose a reason for hiding this comment

Uh oh!

michaeljguarino Mar 10, 2026

Choose a reason for hiding this comment

Uh oh!

michaeljguarino Mar 10, 2026

Choose a reason for hiding this comment

Uh oh!

michaeljguarino Mar 10, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants