Tool Calling (Function Calling)

The LLMTR gateway supports tool calling (function calling) fully compatible with OpenAI Chat Completions. You give the model a list of tools, the model returns a tool call when needed, you run the tool and send the result back. The flow runs over https://llmtr.com/v1/chat/completions and works without changes with the OpenAI SDKs, agent frameworks such as LangChain and Flowise, and coding tools (Cursor, Cline).

Flow

You send tools (and optionally tool_choice) in the request.
If the model decides to call a tool, the response returns finish_reason: "tool_calls" with message.tool_calls, and message.content is null.
You execute the tool on your side.
You append the result as a role: "tool" message (with the same tool_call_id) and send the request again. The model produces the final answer.

1. Tool definition and first request

curl https://llmtr.com/v1/chat/completions \
  -H "Authorization: Bearer llmtr-your_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [
      {"role": "user", "content": "What is the weather in Istanbul?"}
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get the current weather for a city",
          "parameters": {
            "type": "object",
            "properties": {
              "location": {"type": "string", "description": "City name"}
            },
            "required": ["location"]
          }
        }
      }
    ],
    "tool_choice": "auto"
  }'

Response: tool_calls

When the model calls a tool, content is null, finish_reason is tool_calls, and a tool_calls array is returned. function.arguments is always a JSON string (not a JSON object), so parse it before use.

{
  "id": "chatcmpl-xxx",
  "object": "chat.completion",
  "model": "openai/gpt-4o-mini",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_abc123",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"location\":\"Istanbul\"}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ],
  "usage": { "prompt_tokens": 62, "completion_tokens": 18, "total_tokens": 80 }
}

2. Sending the tool result back (multi-turn loop)

After running the tool, append two messages to the history: the assistant message the model returned (with tool_calls) and the tool message carrying the result. The tool_call_id in the tool message must match the call id the model returned.

curl https://llmtr.com/v1/chat/completions \
  -H "Authorization: Bearer llmtr-your_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [
      {"role": "user", "content": "What is the weather in Istanbul?"},
      {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_abc123",
            "type": "function",
            "function": {"name": "get_weather", "arguments": "{\"location\":\"Istanbul\"}"}
          }
        ]
      },
      {
        "role": "tool",
        "tool_call_id": "call_abc123",
        "content": "{\"temp_c\":24,\"condition\":\"sunny\"}"
      }
    ]
  }'

The model returns the final answer as plain text (finish_reason: "stop"):

{
  "choices": [
    {
      "message": {"role": "assistant", "content": "It is currently 24°C and sunny in Istanbul."},
      "finish_reason": "stop"
    }
  ]
}

A single turn may return multiple tool calls (parallel tool calls). You must add a separate role: "tool" message for each tool_call; if one is missing, the next request returns an error.

Replaying Hy3 reasoning_content

When tencent/hy3:low or tencent/hy3:high calls a tool, the assistant message may also contain reasoning_content. Send that field back unchanged on the next turn:

{
  "role": "assistant",
  "content": "I will call the weather tool.",
  "reasoning_content": "I am selecting the tool for the city supplied by the user...",
  "tool_calls": [{
    "id": "call_abc123",
    "type": "function",
    "function": {"name": "get_weather", "arguments": "{\"location\":\"Istanbul\"}"}
  }]
}

Append the matching role: "tool" result after this assistant message. Hy3 adaptively executes low requests containing tools with high reasoning.

tool_choice

Use tool_choice to steer the model’s tool usage:

Value	Meaning
`"auto"` (default)	The model decides whether to call a tool
`"none"`	No tool call; plain text is produced
`"required"`	The model is forced to call at least one tool
`{"type": "function", "function": {"name": "get_weather"}}`	A specific tool is forced

Some models accept only auto and none (e.g. the Moonshot Kimi K2 family; kimi-k3 additionally supports required). For provider-specific constraints, see Chat Completions.

Streaming

With stream: true, tool calls flow inside delta.tool_calls. The first chunk carries the tool call (id, name, arguments); the closing chunk arrives with finish_reason: "tool_calls", followed by data: [DONE].

data: {"choices":[{"index":0,"delta":{"role":"assistant","tool_calls":[{"index":0,"id":"call_abc123","type":"function","function":{"name":"get_weather","arguments":"{\"location\":\"Istanbul\"}"}}]},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}]}

data: [DONE]

For general details of the stream format, see Streaming.

SDK examples

The official OpenAI SDKs work with tool calling unchanged once base_url / baseURL points to LLMTR.

from openai import OpenAI

client = OpenAI(api_key="llmtr-your_key", base_url="https://llmtr.com/v1")

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {"location": {"type": "string"}},
            "required": ["location"],
        },
    },
}]

resp = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "What is the weather in Istanbul?"}],
    tools=tools,
)
print(resp.choices[0].message.tool_calls)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "llmtr-your_key",
  baseURL: "https://llmtr.com/v1",
});

const resp = await client.chat.completions.create({
  model: "openai/gpt-4o-mini",
  messages: [{ role: "user", content: "What is the weather in Istanbul?" }],
  tools: [
    {
      type: "function",
      function: {
        name: "get_weather",
        parameters: {
          type: "object",
          properties: { location: { type: "string" } },
          required: ["location"],
        },
      },
    },
  ],
});

console.log(resp.choices[0].message.tool_calls);

Which models support it

Tool calling is supported on the chat models of the global providers: OpenAI (GPT / o-series), Anthropic Claude, Google Gemini, xAI Grok, Tencent Hy3, zai GLM, Qwen, DeepSeek, Moonshot Kimi, MiniMax, Xiaomi MiMo, StepFun, AionLabs and Mistral. A model’s support is marked by the function_calling capability in the catalog / model detail page.

Among the Turkey-hosted models, llmtr/qwen3-5-4b supports tool calling, so you can use it for agent flows whose data must be processed in Turkey.

Models that do not support it:

The other Turkey-hosted first-party models (llmtr/gemma-4, llmtr/qwen3-6-35b, llmtr/trendyol-7b, llmtr/magibu-11b-v8) do not currently parse function calling; sending tools to them does not return structured tool calls.
For poolside/laguna-xs.2, sending tools / tool_choice returns a 400 unsupported_operation.

Verify the target model’s tool calling support before sending production traffic.

Chat Completions - endpoint reference and provider-specific behaviors
Streaming - SSE stream format
Errors - error codes