feat!: Implement include parameter specifically for adding logprobs in the output message

[s-akhtar-baig]

Problem

As an Application Developer, I want to use the include parameter with the value message.output_text.logprobs, so that I can receive log probabilities for output tokens to assess the model's confidence in its response.

What does this PR do?

• Updates the include parameter in various resource definitions
• Updates the inline provider to return logprobs when "message.output_text.logprobs" is passed in the include parameter
• Converts the logprobs returned by the inference provider from chat completion format to responses format

Closes #4260

Test Plan

• Created a script to explore OpenAI behavior: https://github.com/s-akhtar-baig/llama-stack-examples/blob/main/responses/src/include.py
• Added integration tests and new recordings

[s-akhtar-baig]

Note to the reviewers:

The current behavior when passing include["message.output_text.logprobs"] along with tools is as follows:

Tool type	Current behavior (are logprobs returned?)	OpenAI behavior (are logprobs returned?)
MCP	Yes	Yes
Built-in	Yes	No
Function	No	No

The behavior gets further complicated when passing include["message.output_text.logprobs"] with multiple tools:

• Web search call before mcp tool call: OpenAI generated two assistant messages, and logprobs were returned.
• Mcp tool call before web search call: OpenAI generated one assistant message, and no logprobs were returned.

I wanted to get other opinions to determine how logprobs should be handled with built-in and mcp tools. My preference is to keep it consistent i.e. either return logprobs for both or do not return logprobs for either.

Appreciate any feedback!

Test script: https://github.com/s-akhtar-baig/llama-stack-examples/blob/main/responses/src/include.py

[ashwinb]

This looks good to me. Doing this consistently makes sense!

[mergify]

This pull request has merge conflicts that must be resolved before it can be merged. @s-akhtar-baig please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[github-actions]

✱ Stainless preview builds

This PR will update the llama-stack-client SDKs with the following commit message.

feat: Implement include parameter specifically for adding logprobs in the output message

Edit this comment to update it. It will appear in the SDK's changelogs.

✅ llama-stack-client-node studio · code · diff

Your SDK built successfully.
generate ⚠️ → build ✅ → lint ✅ → test ✅

npm install https://pkg.stainless.com/s/llama-stack-client-node/ca079771633ea253598bf3c96e20112d3c13dab3/dist.tar.gz

✅ llama-stack-client-kotlin studio · code · diff

Your SDK built successfully.
generate ⚠️ → lint ✅ → test ❗

⏳ llama-stack-client-python studio · code · diff

generate ⚠️ → build ⏳ → lint ⏳ → test ⏳

✅ llama-stack-client-go studio · code · diff

Your SDK built successfully.
generate ⚠️ → lint ❗ → test ❗

go get github.com/stainless-sdks/llama-stack-client-go@f350020c226e602777780297a3857ba91cae5774

⏳ These are partial results; builds are still running.

---

This comment is auto-generated by GitHub Actions and is automatically kept up to date as you push.
Last updated: 2025-12-03 20:07:49 UTC

[mergify]

This pull request has merge conflicts that must be resolved before it can be merged. @s-akhtar-baig please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[s-akhtar-baig]

This looks good to me. Doing this consistently makes sense!

Thanks, @ashwinb!

It took a while for me to get the recordings in, sorry about that, but now the pipeline is green. Please let me know if the changes look good. Thanks!

[ashwinb]

@s-akhtar-baig looks like the docker variant is still red somehow :(

[s-akhtar-baig]

@ashwinb, it was failing on acquiring a lock, hopefully it passes successfully this time.

[s-akhtar-baig]

@ashwinb, the check failed again on:

File "/home/runner/work/llama-stack/llama-stack/.venv/lib/python3.12/site-packages/httpcore/_synchronization.py", line 268, in __enter__
    self._lock.acquire()
+++++++++++++++++++++++++++++++++++ Timeout ++++++++++++++++++++++++++++++++++++

Can you please help re-run the check (I don't have the required permissions)?

Re: Don't know why it keeps failing.

[ashwinb]

Something in the PR is triggering a deadlock in the conversations test case