Skip to content

fix(dataset): pair Anthropic tool_result blocks with their tool_use by id - #10293

Open
Zhuoxi2000 wants to merge 1 commit into
modelscope:mainfrom
Zhuoxi2000:fix-anthropic-tool-result-order
Open

Zhuoxi2000 wants to merge 1 commit into
modelscope:mainfrom
Zhuoxi2000:fix-anthropic-tool-result-order

Conversation

@Zhuoxi2000

Copy link
Copy Markdown

PR type

  • Bug Fix
  • New Feature
  • Document Updates
  • More Models or Datasets Support

PR information

Anthropic tool_result blocks are matched to tool_use blocks by tool_use_id, so the results of parallel calls can arrive in a different order from the calls. MessagesPreprocessor.anthropic_to_messages dropped the ids and emitted positional tool_response messages in arrival order. With results [B, A] for calls [A, B], each call was paired with the other call's result. #10174 fixed the same problem for OpenAI tool_calls in normalize_openai_tool_calls; this PR does the same for the Anthropic path.

Changes in swift/dataset/preprocessor/core.py:

  • New MessagesPreprocessor._align_anthropic_tool_results. It takes the tool_use blocks of the previous message and the leading run of tool_result blocks in the current message. If the results form a complete batch (one result per tool_use, distinct non-null ids, the same id set), it sorts them into tool_use order. Other cases are left as they are, including legacy data without ids and partial batches. The guard is the same as in fix: preserve OpenAI parallel tool-result associations #10174.
  • anthropic_to_messages remembers the tool_use blocks of the previous list-content message and aligns the next message's blocks before converting them.

The content blocks are reordered, not the emitted messages. Images inside a tool_result are appended to images while the blocks are converted, so they stay in the same order as their <image> placeholders. Within a reordered batch, that order is the aligned call order. Text blocks after the results (a user follow-up in the same message) stay after the tool responses. The input row is not modified in place.

No separate issue; the bug is described above.

Experiment results

New tests in tests/general/test_data_preprocess.py::TestProviderMessagesPreprocess:

  • test_anthropic_parallel_tool_results_out_of_order passes two results to two calls, in order and reversed, and expects tool_response in call order.
  • test_anthropic_out_of_order_tool_results_keep_images_aligned uses two image-bearing results in reverse order, followed by a user text block. It checks that the <image> placeholders, the images list and the trailing user message all line up.

Before (main @ 02baac9, fix reverted):

$ python -m pytest -q tests/general/test_data_preprocess.py::TestProviderMessagesPreprocess
FAILED ...::test_anthropic_out_of_order_tool_results_keep_images_aligned
SUBFAILED(result_order=['toolu_shanghai', 'toolu_beijing']) ...::test_anthropic_parallel_tool_results_out_of_order
2 failed, 6 passed, 1 subtests passed

After:

$ python -m pytest -q tests/general/test_data_preprocess.py::TestProviderMessagesPreprocess \
    tests/general/test_data_preprocess.py::TestDataPreprocess::test_tool_message \
    tests/general/test_data_preprocess.py::TestDataPreprocess::test_nested_tool_arguments
9 passed, 2 subtests passed

flake8 7.3.0, isort 8.0.1 and yapf 0.43.0 (the versions in .pre-commit-config.yaml) report nothing on the changed files.

Anthropic pairs tool_result blocks with tool_use blocks by tool_use_id, so a
parallel result batch may arrive in a different order from the calls.
anthropic_to_messages discarded the ids and emitted positional tool_response
messages in arrival order, pairing each result with the wrong call.

Align a complete result batch (one result per tool_use, distinct ids) to the
tool_use order before the ids are discarded, as normalize_openai_tool_calls
does for OpenAI tool_calls (modelscope#10174). The blocks are reordered rather than the
emitted messages, so images inside the results stay consistent with their
<image> placeholders. Incomplete or id-less batches are left as-is.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant