Agent loop

A coding agent does not work in a single request: it hands the model a set of tools, runs whichever tool the model asks for, feeds the result back, and lets the model decide the next step. That loop is natively supported on /v1/responses.

Shape of the loop

In the Responses API the conversation history is the input array. It holds two kinds of entry:

Role-bearing messages — {"role": "user", "content": [...]}
Native items — no role, discriminated by type: function_call, function_call_output, reasoning

On each turn you append the model's function_call item and your own function_call_output item to the array, then repeat the request.

Turn 1: declare the tool

curl https://llmtr.com/v1/responses \
  -H "Authorization: Bearer llmtr-your_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.3-codex",
    "input": [
      {"role": "user", "content": [{"type": "input_text", "text": "Weather in Istanbul?"}]}
    ],
    "tools": [
      {
        "type": "function",
        "name": "get_weather",
        "description": "Returns the current temperature for a city.",
        "parameters": {
          "type": "object",
          "properties": { "city": { "type": "string" } },
          "required": ["city"]
        }
      }
    ]
  }'

If the model decides to call the tool, a function_call item appears in output:

{
  "output": [
    {
      "type": "function_call",
      "call_id": "call_abc123",
      "name": "get_weather",
      "arguments": "{\"city\":\"Istanbul\"}"
    }
  ]
}

Turn 2: feed the result back

Run the tool on your side, then append two items to the previous input array: the model's function_call and your function_call_output. The call_id must be identical on both.

curl https://llmtr.com/v1/responses \
  -H "Authorization: Bearer llmtr-your_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.3-codex",
    "input": [
      {"role": "user", "content": [{"type": "input_text", "text": "Weather in Istanbul?"}]},
      {
        "type": "function_call",
        "call_id": "call_abc123",
        "name": "get_weather",
        "arguments": "{\"city\":\"Istanbul\"}"
      },
      {
        "type": "function_call_output",
        "call_id": "call_abc123",
        "output": "{\"tempC\":21,\"conditions\":\"clear\"}"
      }
    ],
    "tools": [ ... ]
  }'

The model now answers with the tool result in view. Repeat the loop until the model stops producing function_call items.

With the OpenAI SDK

from openai import OpenAI

client = OpenAI(api_key="llmtr-your_key", base_url="https://llmtr.com/v1")

tools = [{
    "type": "function",
    "name": "get_weather",
    "parameters": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"],
    },
}]

conversation = [{"role": "user", "content": "Weather in Istanbul?"}]

while True:
    response = client.responses.create(
        model="openai/gpt-5.3-codex",
        input=conversation,
        tools=tools,
    )

    calls = [item for item in response.output if item.type == "function_call"]
    if not calls:
        print(response.output_text)
        break

    # Append the model's own items to the history unchanged.
    conversation += response.output

    for call in calls:
        result = run_tool(call.name, call.arguments)
        conversation.append({
            "type": "function_call_output",
            "call_id": call.call_id,
            "output": result,
        })

Reasoning items

Reasoning models may also return a reasoning item in output. Append those to the history unchanged as well: a model that can see its own chain of thought from earlier turns stays consistent across multi-step tasks. The conversation += response.output line above already does this.

Looping with streaming

The agent loop works with stream: true too. While tool arguments are being produced you receive response.function_call_arguments.delta events, followed by response.function_call_arguments.done. The response.output array on the terminal response.completed event contains the same items a non-streaming response would, so you can continue the loop from there.

See the streaming section on the Responses API page for details.

Parallel tool calls

The model may emit several function_call items in one turn. Run them all and append a separate function_call_output for each. A call_id left unanswered causes an error on the provider side.

Chat Completions format

If you use /v1/chat/completions, the same loop is built with the classic tool_calls and role: "tool" messages; the gateway translates them into Responses items for you. See Tool calling for details.

Models that only support the Responses endpoint (the GPT-5 Codex series, gpt-5.5, the gpt-5.6 family) return 400 endpoint_mismatch on /v1/chat/completions; build the loop on /v1/responses for those.

Common errors

Symptom	Cause
`400 invalid_request` — input item rejected	An entry carries neither `role` nor `type`
Provider reports "no tool output found"	A `function_call` was appended without its matching `function_call_output`
`call_id` mismatch	The `call_id` you feed back must match what the model returned, exactly
`400 endpoint_mismatch`	A Responses-only model was sent to `/v1/chat/completions`