When trying to use tools with MLX models (such as the exact weather example in the README), this runs through the function "applyChatTemplate" in swift-transformer Tokenizer. When it hits this block:
if let tools {
context["tools"] = try .array(tools.map { try Value(any: $0) })
}It throws:
runtime("Cannot convert value of type Optional to Jinja Value")
NOTE: The only change I made to the example was updating the description to:
Get the weather forecast for any location worldwide (city, state, or address). ALWAYS use this tool when a user asks about current weather or weather forecasts.
So that Qwen would recognize it MUST run this tool.
Here is the tools object:
Printing description of tools:
▿ Optional<Array<Dictionary<String, Any>>>
▿ some : 1 element
▿ 0 : 2 elements
▿ 0 : 2 elements
- key : "function"
▿ value : 3 elements
▿ 0 : 2 elements
- key : "name"
- value : "get_weather"
▿ 1 : 2 elements
- key : "description"
- value : "Get the weather forecast for any location worldwide (city, state, or address). ALWAYS use this tool when a user asks about current weather or weather forecasts."
▿ 2 : 2 elements
- key : "parameters"
▿ value : JSONValue
▿ object : 6 elements
▿ 0 : 2 elements
- key : "properties"
▿ value : JSONValue
▿ object : 1 element
▿ 0 : 2 elements
- key : "city"
▿ value : JSONValue
▿ object : 2 elements
▿ 0 : 2 elements
- key : "type"
▿ value : JSONValue
- string : "string"
▿ 1 : 2 elements
- key : "description"
▿ value : JSONValue
- string : "The city to fetch the weather for"
▿ 1 : 2 elements
- key : "required"
▿ value : JSONValue
▿ array : 1 element
▿ 0 : JSONValue
- string : "city"
▿ 2 : 2 elements
- key : "description"
▿ value : JSONValue
- string : "Generated Arguments"
▿ 3 : 2 elements
- key : "type"
▿ value : JSONValue
- string : "object"
▿ 4 : 2 elements
- key : "additionalProperties"
▿ value : JSONValue
- bool : false
▿ 5 : 2 elements
- key : "$defs"
▿ value : JSONValue
▿ object : 1 element
▿ 0 : 2 elements
- key : "Infinity_Vaultz.WeatherTool.Arguments"
▿ value : JSONValue
▿ object : 5 elements
▿ 0 : 2 elements
- key : "type"
▿ value : JSONValue
- string : "object"
▿ 1 : 2 elements
- key : "properties"
▿ value : JSONValue
▿ object : 1 element
▿ 0 : 2 elements
- key : "city"
▿ value : JSONValue
▿ object : 2 elements
▿ 0 : 2 elements
- key : "description"
▿ value : JSONValue
- string : "The city to fetch the weather for"
▿ 1 : 2 elements
- key : "type"
▿ value : JSONValue
- string : "string"
▿ 2 : 2 elements
- key : "required"
▿ value : JSONValue
▿ array : 1 element
▿ 0 : JSONValue
- string : "city"
▿ 3 : 2 elements
- key : "additionalProperties"
▿ value : JSONValue
- bool : false
▿ 4 : 2 elements
- key : "description"
▿ value : JSONValue
- string : "Generated Arguments"
▿ 1 : 2 elements
- key : "type"
- value : "function"
This issue is caused by:
private func convertToolToMLXSpec(_ tool: any Tool) -> ToolSpec {
// Convert AnyLanguageModel's GenerationSchema to Sendable dictionary
// using MLXLMCommon.JSONValue which is already Sendable
let parametersValue: JSONValue
do {
let resolvedSchema = tool.parameters.withResolvedRoot() ?? tool.parameters
let data = try JSONEncoder().encode(resolvedSchema)
parametersValue = try JSONDecoder().decode(JSONValue.self, from: data)
} catch {
parametersValue = .object(["type": .string("object"), "properties": .object([:]), "required": .array([])])
}
return [
"type": "function",
"function": [
"name": tool.name,
"description": tool.description,
"parameters": parametersValue,
] as [String: any Sendable],
]
}Suggested change is:
- Add a helper to convert JSONValue → Foundation types:
extension JSONValue {
var foundationObject: Any {
switch self {
case .string(let s): return s
case .number(let n): return n
case .bool(let b): return b
case .array(let a): return a.map { $0.foundationObject }
case .object(let o): return o.mapValues { $0.foundationObject }
case .null: return NSNull()
}
}
}
- Use foundationObject instead:
-
"parameters": parametersValue,
-
"parameters": parametersValue.foundationObject(),
The crash occurs because parameters is a JSONValue enum, which swift-transformers Value(any:) cannot convert. Converting JSONValue recursively to Foundation types before passing to Jinja fixes the crash, and allows nested schemas to work correctly.
When trying to use tools with MLX models (such as the exact weather example in the README), this runs through the function "applyChatTemplate" in swift-transformer Tokenizer. When it hits this block:
It throws:
NOTE: The only change I made to the example was updating the description to:
Get the weather forecast for any location worldwide (city, state, or address). ALWAYS use this tool when a user asks about current weather or weather forecasts.
So that Qwen would recognize it MUST run this tool.
Here is the tools object:
This issue is caused by:
Suggested change is:
The crash occurs because parameters is a JSONValue enum, which swift-transformers Value(any:) cannot convert. Converting JSONValue recursively to Foundation types before passing to Jinja fixes the crash, and allows nested schemas to work correctly.