Skip to content

LCORE-2861: Sync config doc #2070

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tisnik merged 1 commit into from
Jul 6, 2026
+38 −6
Merged

LCORE-2861: Sync config doc #2070

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions docs/config.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1034,6 +1034,22 @@ <h2 id="inferenceconfiguration">InferenceConfiguration</h2>
default_model / default_provider keep their query-time routing meaning
and are independent of this list.</td>
</tr>
<tr class="odd">
<td>max_infer_iters</td>
<td>integer</td>
<td>Server-side default for the maximum number of inference iterations a
model can perform in a single request. Prevents small models from
looping indefinitely on tool calls. Per-request values take precedence
over this default. Set to None to disable the limit.</td>
</tr>
<tr class="even">
<td>max_tool_calls</td>
<td>integer</td>
<td>Server-side default for the maximum number of tool calls allowed in
a single response. Prevents small models from exhausting the context
window with repeated tool calls. Per-request values take precedence over
this default. Set to None to disable the limit.</td>
</tr>
</tbody>
</table>
<h2 id="jsonpathoperator">JsonPathOperator</h2>
Expand Down
14 changes: 14 additions & 0 deletions docs/config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -896,6 +896,20 @@
},
"title": "High-level inference providers",
"type": "array"
},
"max_infer_iters": {
"type": "integer",
"nullable": true,
"default": 10,
"description": "Server-side default for the maximum number of inference iterations a model can perform in a single request. Prevents small models from looping indefinitely on tool calls. Per-request values take precedence over this default. Set to None to disable the limit.",
"title": "Default max inference iterations"
},
"max_tool_calls": {
"type": "integer",
"nullable": true,
"default": 30,
"description": "Server-side default for the maximum number of tool calls allowed in a single response. Prevents small models from exhausting the context window with repeated tool calls. Per-request values take precedence over this default. Set to None to disable the limit.",
"title": "Default max tool calls"
}
},
"title": "InferenceConfiguration",
Expand Down
14 changes: 8 additions & 6 deletions docs/config.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -332,12 +332,14 @@ In-memory cache configuration.
Inference configuration.


| Field | Type | Description |
|------------------|--------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| default_model | string | Identification of default model used when no other model is specified. |
| default_provider | string | Identification of default provider used when no other model is specified. |
| context_windows | object | Map of fully-qualified model identifier (e.g., "openai/gpt-4o-mini") to context window size in tokens. Used by the conversation compaction trigger to decide when older turns must be summarized before the input exceeds the window. Models absent from this map have no registered window — callers fall back to their own default or skip the token-based trigger. |
| providers | array | Unified-mode synthesis input (Decision S5): a high-level, backend-agnostic list of inference providers the synthesizer expands into Llama Stack provider entries. Lives at the configuration root so it survives a future backend change. A non-empty list signals unified mode. Empty (the default) leaves legacy/remote modes unaffected. The sibling default_model / default_provider keep their query-time routing meaning and are independent of this list. |
| Field | Type | Description |
|------------------|---------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| default_model | string | Identification of default model used when no other model is specified. |
| default_provider | string | Identification of default provider used when no other model is specified. |
| context_windows | object | Map of fully-qualified model identifier (e.g., "openai/gpt-4o-mini") to context window size in tokens. Used by the conversation compaction trigger to decide when older turns must be summarized before the input exceeds the window. Models absent from this map have no registered window — callers fall back to their own default or skip the token-based trigger. |
| providers | array | Unified-mode synthesis input (Decision S5): a high-level, backend-agnostic list of inference providers the synthesizer expands into Llama Stack provider entries. Lives at the configuration root so it survives a future backend change. A non-empty list signals unified mode. Empty (the default) leaves legacy/remote modes unaffected. The sibling default_model / default_provider keep their query-time routing meaning and are independent of this list. |
| max_infer_iters | integer | Server-side default for the maximum number of inference iterations a model can perform in a single request. Prevents small models from looping indefinitely on tool calls. Per-request values take precedence over this default. Set to None to disable the limit. |
| max_tool_calls | integer | Server-side default for the maximum number of tool calls allowed in a single response. Prevents small models from exhausting the context window with repeated tool calls. Per-request values take precedence over this default. Set to None to disable the limit. |


## JsonPathOperator
Expand Down
Loading