Nitro Intelligence

The entrypoint to everything AI.

This component aims to consolidate and standardize AI features implemented in the host application.

Configuration

NitroIntelligence is configured via an initializer in the host application. All external dependencies (e.g. inference API credentials, observability settings) must be injected at boot time:

NitroIntelligence.configure do |config|
  # Standard Rails integrations
  config.logger         = Rails.logger           # Logger instance
  config.environment    = Rails.env              # e.g. "production", "test"
  config.cache_provider = Rails.cache            # ActiveSupport cache store

  # Inference (LLM) settings
  config.inference_api_key  = "..."              # API key for the inference service
  config.inference_base_url = "https://..."      # Base URL for the inference service (optional; defaults to the inference gateway)
  config.inference_max_retries = 0               # Retries the SDK makes itself (optional; defaults to 0 - see "Retries and Timeouts")
  config.inference_timeout     = 600             # Request timeout in seconds (optional; defaults to openai-ruby's)

  # Observability (Langfuse) settings
  config.observability_base_url = "https://..."  # Base URL for the observability service (optional; defaults to Cerebro)
  config.observability_projects = [              # Array of project credential hashes
    {
      "slug"       => "my-feature-project",
      "id"         => "project-id",
      "public_key" => "pk-...",
      "secret_key" => "sk-...",
    },
  ]

  # Nitro Intelligence Assistants settings (optional)
  config.assistants_config = {                   # See "Assistants" below
    "base_url" => "https://nip-assistants.example.com",
    "user_id" => "my-app",
    "definitions" => {
      "candidate-concierge" => {},
    }
  }

  # Model configuration
  config.model_config = {
    "default_audio_transcription_model" => "Qwen/Qwen3-ASR-1.7B",
    "default_text_model" => "Qwen/Qwen3.6-35B-A3B",
    "default_image_model" => "google/gemini-3.1-flash-image",
    "default_text_to_speech_model" => "openai/gpt-4o-mini-tts",
    "models" => [
      {
        "name" => "Qwen/Qwen3.6-35B-A3B",
        "type" => "text"
      },
      {
        "name" => "Qwen/Qwen3-ASR-1.7B",
        "type" => "audio_transcription"
      },
      {
        "name" => "google/gemini-3.1-flash-image",
        "type" => "image",
        "aspect_ratios" => ["1:1", "2:3", "3:2", "3:4", "4:3", "9:16", "16:9"],
        "resolutions" => ["512", "1K", "2K"],
        "omit_output_fields" => ["provider_specific_fields.thought_signatures"]
      },
      {
        "name" => "openai/gpt-4o-mini-tts",
        "type" => "text_to_speech",
        "default_voice" => "marin",
        "default_response_format" => "mp3",
        "voices" => ["echo", "nova", "marin", "cedar"],
        "response_formats" => ["mp3", "wav"]
      }
    ]
  }
end

Configuration Keys

Key	Type	Default	Description
logger	Logger	Logger.new($stdout)	Logger used for diagnostic output
environment	String	"test"	Runtime environment name
cache_provider	cache store	NullCache	ActiveSupport-compatible cache store
inference_api_key	String	""	API key for the LLM inference service
inference_base_url	String	"https://inference.powerhome.ai"	Base URL for the LLM inference service. Defaults to the shared inference gateway, so only a host talking to a different one needs to set it
inference_max_retries	Integer	0	Retries the SDK makes itself after a failed request. Off by default — see Retries and Timeouts
inference_timeout	Numeric	nil	Request timeout in seconds. nil keeps openai-ruby's default of 600
observability_base_url	String	"https://cerebro.powerhome.ai"	Base URL for the Langfuse observability service. Defaults to Cerebro, so only a host talking to a different one needs to set it
observability_projects	Array<Hash>	[]	Langfuse project credentials (slug, id, public_key, secret_key)
observability_user_id	String	""	User ID recorded on observed traces when a request passes no user_id parameter — see Users
assistants_config	Hash	{}	Assistants to make addressable by key. base_url (String, default: "https://assistants.powerhome.ai") and user_id (String, default: "default-user") are shared by every entry; definitions (Hash) holds one entry per assistant, keyed by what it is looked up with, each able to override a shared value. Without definitions the hash is read as credentials for a single Assistants.new, which takes the same two defaults — see Assistants
model_config	Hash	{}	Model defaults and per-model settings. Top-level keys: default_text_model, default_audio_transcription_model, default_image_model, default_text_to_speech_model, and models (array of per-model hashes keyed by name and type, with type-specific options like aspect_ratios/resolutions for images or voices/response_formats for TTS)

Basic Usage

OpenAI API/LLM Requests

A simple LLM call can be invoked as follows:

client = NitroIntelligence::Client.new
client.chat(message: "Why is the sky blue?")
content = result.choices.first&.message&.content

This component handles setting defaults for most things, such as model, host, keys, etc.

You may also use openai-ruby compatible syntax with this wrapper by passing the parameters keyword in. For example:

client = NitroIntelligence::Client.new
client.chat(parameters: { model: "Qwen/Qwen3.8-27B", messages: [{ role: "user", content: "Why is the sky blue?" }]})

Providing Parameters

Parameters such as 'max_tokens' and 'temperature' can be passed in under the parameters key.

client = NitroIntelligence::Client.new
client.chat(parameters: { model: "Qwen/Qwen3.8-27B", max_tokens: 1000, temperature: 0.7, messages: [{ role: "user", content: "Why is the sky blue?" }]})

For a full list of supported parameters, see the API reference here.

Retries and Timeouts

The client does not retry failed requests by default. openai-ruby on its own retries a 429, 408, 409 or 5xx twice within about two seconds, but none of those is worth repeating that quickly from here: the inference gateway has already failed over between providers before it answers, and a rate limit or exhausted budget it passes on will not clear within a second. Each retry only adds load, and for billable work such as image generation a retry after a timeout can pay twice. A caller that wants retries decides when, typically in a job with backoff, and can tell a rate limit from an exhausted budget first (see the gateway's error guidance).

To retry anyway, or to change the timeout, set them for every client in the configuration (inference_max_retries, inference_timeout), for one client with the same names, or for one request with openai-ruby's request_options. The narrower setting wins.

# For one client
client = NitroIntelligence::Client.new(inference_max_retries: 2, inference_timeout: 30)

# For one request, with any method
client.chat(message: "Why is the sky blue?", parameters: { request_options: { max_retries: 2, timeout: 30 } })
client.generate_image(message: "A bear installing a window", parameters: { request_options: { timeout: 120 } })

Audio Transcription

Nitro Intelligence can be used to transcribe audio from a file into text.

Basic examples of usage:

client = NitroIntelligence::Client.new
audio = File.open('sample.m4a', 'rb')
result = client.transcribe_audio(audio_file: audio)

result is a OpenAI::Models::Audio::Transcription object:

<OpenAI::Models::Audio::Transcription:0x2fdc8 {:text=>"Hello, how are you doing today?", :usage=>{:input_tokens=>28, :output_tokens=>10, :total_tokens=>38, :type=>:tokens, :input_token_details=>{:audio_tokens=>28, :text_tokens=>0}}}>

To use a prompt, simply initialize your client with the observability_project_slug and pass prompt_name into parameters:

client = NitroIntelligence::Client.new(observability_project_slug: "my-test-project")
audio = File.open('sample.m4a', 'rb')
result = client.transcribe_audio(audio_file: audio, parameters: {prompt_name: "Spanish Converter"})

puts result

# <OpenAI::Models::Audio::Transcription:0x2fd8c {:text=>"Hola, ¿cómo estás hoy?", :usage=>{:input_tokens=>37, :output_tokens=>9, :total_tokens=>46, :type=>:tokens, :input_token_details=>{:audio_tokens=>28, :text_tokens=>9}}}>

Text-to-Speech

Nitro Intelligence can be used to create spoken-word audio files from text.

Basic example of usage:

client = NitroIntelligence::Client.new
tts = client.text_to_speech(message: 'Hello, this is Power Home Remodeling Group.')

# tts is a StringIO object, we can write the file to disk
File.binwrite('tts.mp3', tts.string)

Supplying custom parameters:

client = NitroIntelligence::Client.new
tts = client.text_to_speech(
  message: 'Hello, this is Power Home Remodeling Group.',
  parameters: {
    voice: 'marin',
    response_format: 'mp3',
    instructions: 'Translate the English into Spanish before speaking.',
    speed: 1.25
  }
)

# tts is a StringIO object, we can write the file to disk
File.binwrite('tts.mp3', tts.string)

For a full list of supported parameters, see the API reference here. Note that voice and response_format are further constrained to the voices and response_formats listed for the chosen model in config.model_config.

Image Editing and Generation

Nitro Intelligence can be used for image editing and generation

Basic examples of usage:

Image Generation

client = NitroIntelligence::Client.new
result = client.generate_image(message: "Create an image of a bear installing a window.")

result is a NitroIntelligence::ImageGeneration object, and the generated image can be accessed via result.generated_image which returns a NitroIntelligence::Image.

You may write the file to disk:

File.binwrite("my_generated_image.#{result.generated_image.file_extension}", result.generated_image.byte_string)

Image Editing and Uploading Reference Images

To edit an image, provide your source image as a byte string, along with any references you would like to include.

client = NitroIntelligence::Client.new
house = File.binread("./house.jpg")
siding = File.binread("./siding.png")
result = client.generate_image(message: "Replace the siding in the image of the house with the new siding I have provided.", target_image: house, reference_images: [siding])

Using Prompts

See Observability[## Observability] for more details. Basic usage looks like:

client = NitroIntelligence::Client.new(observability_project_slug: "sample-project-slug")
house = File.binread("./house.jpg")
siding = File.binread("./siding.png")
result = client.generate_image(target_image: house, reference_images: [siding], parameters: {prompt_name: "Sample Prompt Name"})

Image Configuration

You can specify parameters such as model to use, aspect ratio and resolution via the parameters key:

client = NitroIntelligence::Client.new
result = client.generate_image(message: "Create an image of a bear installing a window.", parameters: {aspect_ratio: "4:3", resolution: "512"})

Observability

Setup

If your feature is setup for observability, simply pass the observability project and desired prompt name in when invoking requests. It is also preferable to pass in a source and any other useful metadata for later debugging. For example:

client = NitroIntelligence::Client.new(observability_project_slug: "fake-feature-project")
client.chat(
  message: "Why is the sky blue?",
  parameters: {
    prompt_name: "My Prompt",
    # prompt_label: "debug",
    # prompt_version: "v2",
    metadata: {
      source: self.class,
    },
  }
)

If no prompt_label or prompt_version is provided, the 'production' prompt is used by default.

Prompt Fallbacks

Pass prompt_fallback_name to name a second prompt to use when the one you asked for isn't there. The prompt in prompt_name is tried first; if it doesn't exist, or its lookup fails, the fallback is used instead:

client = NitroIntelligence::Client.new(observability_project_slug: "fake-feature-project")
client.chat(
  message: "Why is the sky blue?",
  parameters: {
    prompt_name: "My Prompt - Scheduling", # used when it exists
    prompt_fallback_name: "My Prompt",     # used when it doesn't
    # prompt_fallback_label: "debug",
    # prompt_fallback_version: "v2",
    prompt_variables: {
      appointment_id: "1234",
    },
  }
)

This is for shipping a variant of a prompt — per domain, per locale, per experiment — without having to create one for every caller. Build the specific name however your feature likes; the fallback is just the prompt you'd have asked for anyway.

In practice the caller composes the specific name from something it already knows. Summarising a call, for example: each queue holds a different kind of conversation, so a queue that needs its own wording can have its own prompt while the rest use the general one.

base = "call_summary"
queue = call.queue.slug # "sales", "support", "collections", ...

client.chat(
  parameters: {
    prompt_name: [base, queue].compact.join("."), # call_summary.collections
    prompt_fallback_name: base,
    prompt_variables: { transcript: call.transcript },
  }
)

The fallback is looked up at its own prompt_fallback_label and prompt_fallback_version, and inherits neither from the prompt you asked for first. They describe a different prompt: a version number is minted per prompt, so version 3 of one name is an unrelated revision of another, and a label the first prompt carries need not exist on the fallback at all.

That means an omitted prompt_fallback_label resolves the fallback the same way any unlabelled lookup does — at the 'production' label. If you want both prompts to come from the same label, name it for both.

Whichever prompt is used is the one linked to the trace and merged into your request. If the fallback is missing too, the error is raised rather than swallowed — a request that asked for a prompt shouldn't quietly run without one.

Custom Trace Names

To provide custom trace names to the observability platform, you can pass 'trace_name' in parameters. Example:

client = NitroIntelligence::Client.new(observability_project_slug: "fake-feature-project")
client.chat(
  message: "Why is the sky blue?",
  parameters: {
    prompt_name: "My Prompt",
    metadata: {
      source: self.class,
    },
    trace_name: "custom-trace-name"
  }
)

Custom Trace IDs

To generate a deterministic trace ID in the observability platform, you can pass trace_seed as a parameter. This is useful when you want a stable identifier that is derived from a specific domain value (e.g., document ID). The same trace_seed will produce the same trace ID, making it easier to correlate multiple events. Each seeded call creates a root observation in that trace, including when another observation is active. With Langfuse v4 event reads, trace input and output come from the most recently ingested root observation. Calls without a seed keep their active parent observation.

document_id = "document-123"
client = NitroIntelligence::Client.new(observability_project_slug: "fake-feature-project")
client.chat(
  message: "summarize the document",
  parameters: {
    trace_seed: document_id,
  }
)

Grouping Traces: Users, Sessions and Tags

user_id attributes traces to a user of your application, session_id groups related traces, and tags label them for filtering. All three are optional. session_id and tags are sent only when you set them.

client = NitroIntelligence::Client.new(observability_project_slug: "fake-feature-project")
client.chat(
  message: "why did this deploy fail?",
  parameters: {
    user_id: current_user.id.to_s,
    session_id: "deploy-9f2c1ab",
    tags: ["deployment-failure-analyzer"],
  }
)

Users

A trace that carries a user_id shows up under that user in Cerebro's Users view, which lists each user with their trace count, token usage and cost, and links through to their traces. The trace list can also be filtered by user.

When a request passes no user_id, the observability_user_id configuration key is used instead, so a host can attribute all of its traces to one identity (the application itself, say) and override it per request for the traces that belong to a person:

NitroIntelligence.configure do |config|
  config.observability_user_id = "nitro-web"
end

Use an identifier that is stable and not itself personal data — a database ID rather than an email address or name — since it is stored on every trace and shown in Cerebro. It must be a String of at most 200 characters; langfuse-rb drops any other value with a warning, so convert numeric IDs with to_s.

user_id is recorded only in Cerebro. It is not sent to the inference gateway, so to attribute gateway spend to a user as well, also put the identifier in metadata (see below).

Correlating With Application Logs and the Inference Gateway

Every observed request is correlated across three systems automatically:

• Observability platform → inference gateway. The trace ID is sent to the gateway as x-litellm-trace-id, and metadata is sent as x-litellm-spend-logs-metadata (dropped if it exceeds 4KB).

  The two are independent. Metadata is sent whenever you set it, observed or not, so gateway spend can be attributed even without observability. The trace ID is sent only on the observed path - it comes from the observation being recorded, never from whatever span happens to be active, so a client built without an observability_project_slug sends none even inside a host application with its own OpenTelemetry instrumentation.

• Inference gateway → observability platform. When a request fails, the gateway's own request identifier is read from the error response and recorded on the observation as litellm_call_id metadata.

• Application users → observability platform. Pass user_id to see a user's traces, usage and cost together in Cerebro — see Users.

• Application logs → observability platform. Put whatever your logs are keyed by into metadata — it lands on the trace and in the gateway's spend logs.

client.chat(
  message: "why did this deploy fail?",
  parameters: {
    trace_seed: deploy_url,
    metadata: {
      source: self.class,
      rails_request_id: request_id,
      job_id: job_id,
    },
  }
)

To log the trace ID from application code, derive it from the same seed with NitroIntelligence::Trace.create_id(seed:).

Failed Requests

A request that raises still produces a usable trace. The observation records:

• the input that was sent (recorded before the request runs)
• level: "ERROR" and a status_message carrying the exception class and message
• litellm_call_id metadata when the gateway returned one

The exception is then re-raised, so this changes what is observed, not how callers handle failures. Image generation is the one exception to input recording: its input carries base64 payloads that are replaced with media references only on success, so it records no input up front.

Scoring

You can use NitroIntelligence::Reporter to evaluate existing traces. Calling NitroIntelligence::Reporter#score lets you attach metrics to a trace in the observability platform.

By Trace ID

reporter = NitroIntelligence::Reporter.new(observability_project_slug: "fake-feature-project")
reporter.score(name: "precision", value: 0.5, trace_id: "2377b0b38acf0f11b95504344fad6152")

By Trace Seed

Knowing the trace seed, you can use NitroIntelligence::Trace.create_id to get the trace ID.

document_id = "document-123"
reporter = NitroIntelligence::Reporter.new(observability_project_slug: "fake-feature-project")
trace_id = NitroIntelligence::Trace.create_id(seed: document_id)
reporter.score(name: "precision", value: 0.5, trace_id:)

Using different data types for scoring

By default, scores submitted on a trace are numeric, but you can use the data_type keyword to change it to :boolean or :categorical. For example:

reporter.score(name: "precision", value: "Partially Correct", trace_id:, data_type: :categorical)

Numeric data is helpful for scores that happen on a sliding scale, like success rate or percent similarity. Boolean data is helpful to convey something that either passes or fails. Categorical data is helpful for scores that have pre-defined categories, like "Correct", "Partially Correct", and "Incorrect".

Prompts

Message Shape

A chat completion needs a user message in order for the model to answer. What you have to supply depends on the type of the prompt you name, and the two types behave differently.

Text prompts

A text prompt in Cerebro is equivalent to the system prompt. System prompts are used to pre-load initial context into the model needed for following call executions.

Pure text prompts require a user message to be supplied. For example:

#  Wrong - Raises ObservedChatPromptError - A prompt alone, provides a system message alone with nothing to answer
client.chat(parameters: { prompt_name: "Assistant" })

# Correct - The prompt supplies the system message, you supply the user message
client.chat(message: "Why is the sky blue?", parameters: { prompt_name: "Assistant" })

Chat prompts

A chat prompt in Cerebro holds a list of role-tagged messages rather than a single string, so it can carry its own user turn. Define one when the prompt structure should represent a conversation flow vs. a single block of static text, e.g. pre-modeling an interaction you want to continue:

[
  {
    "role": "system",
    "content": "You are a helpful support agent for an online bookstore. Be concise and friendly. Our return window is 30 days."
  },
  {
    "role": "user",
    "content": "Hi, do you ship to Canada?"
  },
  {
    "role": "assistant",
    "content": "Yes! We ship to Canada. Delivery usually takes 5–8 business days."
  },
  {
    "role": "user",
    "content": "{{question}}"
  }
]

That prompt is self-sufficient, so no additional message: is needed:

client.chat(parameters: { prompt_name: "Appointment Extractor", prompt_variables: { document: text } })

Any messages you do pass are appended after the prompt's own system message, so a chat prompt can also serve as a preamble to a live conversation.

A chat prompt containing only a system message has the same problem as a text prompt, and is refused in the same way. If you find yourself writing one, either add a user message to it in Cerebro or make it a text prompt and pass the turn from the caller via the message keyword.

Prompt Variables and Config

Prompts are often created with "variables". These variables can be supplied and compiled into the prompt. For example:

client = NitroIntelligence::Client.new(observability_project_slug: "fake-feature-project")
client.chat(
  message: "Where is the appointment?",
  parameters: {
    prompt_name: "My Prompt With Variables",
    prompt_variables: {
      appointment_id: "1234",
    },
    metadata: {
      source: self.class,
    },
  }
)

Prompts can be created with a "config" JSON object. This config stores structured data such as model parameters (like model name, temperature), function/tool parameters, or JSON schemas.

The prompt config is merged into your chat request by default. The prompt config object is merged into the "parameters" hash, overriding any existing keys.

Consider this prompt config:

{
  "model": "Qwen/Qwen3.6-35B-A3B"
}

Invoking this request would result in "Qwen/Qwen3.6-35B-A3B" being used as the model, even if supplied manually:

client = NitroIntelligence::Client.new(observability_project_slug: "fake-feature-project")
client.chat(
  message: "Where is the appointment?",
  parameters: {
    model: "Qwen/Qwen3.8-27B", # Will not be used, will be overridden by config "Qwen/Qwen3.6-35B-A3B"
    prompt_name: "My Prompt With Variables",
    prompt_variables: {
      appointment_id: "1234",
    },
    metadata: {
      source: self.class,
    },
  }
)

To disable the prompt config entirely, which may be useful for debugging/testing, you can supply the "prompt_config_disabled" keyword. For example:

client = NitroIntelligence::Client.new(observability_project_slug: "fake-feature-project")
client.chat(
  message: "Where is the appointment?",
  parameters: {
    model: "Qwen/Qwen3.8-27B", # This will now be used since "prompt_config_disabled" is true
    prompt_name: "My Prompt With Variables",
    prompt_variables: {
      appointment_id: "1234",
    },
    prompt_config_disabled: true,
    metadata: {
      source: self.class,
    },
  }
)

Nitro Intelligence Assistants

NitroIntelligence::Assistants is Nitro Intelligence's lightweight SDK for working with hosted agent threads, runs, and human review flows. It is mainly used to initialize conversation threads, trigger agent runs, inspect agent tool calls pending human approval, and resume interrupted threads after human reviews.

For the full guide, see ASSISTANTS.md. For the service this SDK talks to, see the Nitro Intelligence Assistants documentation.

Assistants

An assistant is addressed by the key it is configured under:

assistant = NitroIntelligence.assistants["candidate-concierge"]

assistant.await_run(thread_id: thread.id, messages: messages)
assistant.thread_state(thread_id: thread.id)

await_run and review_tool_calls supply the assistant's own id, so a caller never passes it -- passing assistant_id: to either raises ArgumentError rather than quietly sending the call to a different assistant, which is what a migrated caller carrying its old argument would otherwise do. Everything else is thread-scoped and delegates to the client untouched. Reach the client directly with assistant.client if you need it.

Configuration

Connection settings shared by every assistant sit at the top level; definitions holds one entry per assistant, keyed by what you look it up with:

config.assistants_config = {
  # Optional. Defaults to the shared Assistants deployment.
  "base_url" => "https://nip-assistants.example.com",
  "user_id" => "my-app",
  "definitions" => {
    "candidate-concierge" => {},
    "home-studio" => {
      # Overrides the shared value for this assistant only.
      "base_url" => "https://pr306.nip-assistants.example.com"
    }
  }
}

An entry may carry keys this gem has no use for — a graph id, or the Cerebro project an assistant reports to — and they are ignored, so one structure can serve both a deployment and the application reading it. That includes a name of its own: an entry's name is a human-readable label for the assistant's record, distinct from the key it is filed under. Assistant#key returns the latter.

Credentials

Each entry supplies its own api_key and assistant_id:

"definitions" => {
  "candidate-concierge" => {
    "api_key" => "...",
    "assistant_id" => "..."
  }
}

Where a host gets them is its own business — an environment variable its deployment mounts, a secrets store, a literal for local work. This gem reads no environment and assumes no naming convention, so nothing here is tied to one deployment's wiring.

A key that resolves without them raises Assistant::ConfigurationError, naming every field it is missing at once so a host resolving them from elsewhere can see which lookup failed. base_url is not among them: an entry that says nothing about where to reach its assistant gets https://assistants.powerhome.ai, so only a host talking to a different deployment — a review environment, a local server — sets it.

Without definitions

assistants_config lacking definitions is read as keyword arguments for a single Assistants client, and NitroIntelligence.assistants returns that client rather than a registry. This is the shape that predates lookup by key; a host still on it is left alone. base_url is optional here too, and a configuration without one reaches the same shared deployment.