Agent workflows · 2026-09-16
Three silent failures on Motif 3 that still return HTTP 200
On Motif 3, tool_choice none, stop sequences and reasoning_effort all appear to work and do not. Read the measured behaviour of each, why they return 200, and what to change in your agent code to prevent them.
What a silent failure is, and why it is the expensive kind
If a request returns 400 your code sees it, logs it, retries or raises an alert. The real problem is a request that returns 200 with a useless answer: the status code is a success, the finish reason is normal, and the data in your hand cannot be used.
Failures like that are invisible to monitoring. Your error-rate chart stays flat, your latency chart looks normal, and the problem surfaces only through a user complaint or a parsing error one layer down. In agent loops the effect compounds, because an empty answer becomes the input to the next step.
Three concrete examples were measured on Motif 3. All three are triggered by common parameters, and all three have a simple fix, as long as you know about them.
One: tool_choice none does not suppress the tool call
The expected behaviour is clear: you attached tool definitions but do not want them used this turn, so you set tool_choice to none and the model answers in plain text.
On this row something else happens. Instead of suppressing the call, the provider writes the raw tool-call template into the reply text and finishes the request normally. The same result came back in three attempts out of three.
LLMTR stops that request before it reaches the provider and returns a 400 that tells you what to do instead. The measurement itself is why that was preferred: a documentation warning is not enough, because agent frameworks build the request from the supported parameters they read on the model card and never see a warning written for a human.
The fix is one sentence: if you do not want tools used, leave the tools field out of the request entirely. With no tools attached, tool_choice none is harmless and is not refused. If you want to steer tool use, use auto, required or a named function — all three work.
The measured response to tool_choice none — Measured through LLMTR, 16 September 2026
request : tool_choice = "none", tools = [get_weather]
response: HTTP 200
finish_reason = "stop"
tool_calls = []
content = "<tool_call>{\"name\": \"get_weather\", ...}</tool_call>"Two: a stop sequence can empty the response completely
This is the hardest of the three to notice, because the stop field genuinely is applied. The problem is what it is applied to.
The model thinks before every answer, and the reasoning trace is produced before the visible reply. If the stop sequence matches that hidden text, generation halts there and the visible answer never begins. The result is HTTP 200, a normal finish reason, and an empty content field.
The measurement shows this directly. On a prompt asking the model to count from one to ten, giving a word the model uses while thinking as the stop sequence returns an empty answer, while an invented sequence the model would never think of leaves the answer intact.
The practical rule: do not use stop on this model. If you must, pick a sequence you can be certain the model will not use while thinking — which you cannot be certain of for any word in natural language. To shorten a response use max_tokens, or bind the output to a schema.
Stop sequence behaviour — Measured through LLMTR, 16 September 2026
prompt: "count from one to ten"
no stop sequence -> "one two three ... ten"
stop: ["five"] -> content: "" finish_reason: "stop" HTTP 200
stop: ["seven"] -> content: "" finish_reason: "stop" HTTP 200
stop: ["ZZQQ"] -> "one two three ... ten"Three: reasoning_effort and the reasoning off switches are accepted and ignored
What makes this case insidious is that the field is validated. The provider rejects none, minimal, xhigh and max with 400, so the field behaves like a real scale. The three values it accepts do not separate: low reasons roughly twice as long as high, and the ranges overlap.
The same is true of attempts to switch reasoning off. Three different methods were tried, five samples each, and not one of the fifteen attempts produced an empty reasoning trace. All three are accepted, and all three change nothing.
This reaches the bill: hidden reasoning tokens count as output. On a metered row that would mean charging a user for tokens under a setting they believe reduces them. On this free row it comes back as latency instead.
That is why LLMTR offers no reasoning level on this row and does not advertise reasoning or include_reasoning as supported parameters on the model card. The fields remain passthrough, so they still reach the provider if you send them; no promise is being made to you about them.
Two more observations: seed and logprobs
The seed field is accepted but does not produce reproducibility. Two identical requests with the same seed returned different answers. Even at temperature zero, three consecutive calls gave three different answers, so do not expect determinism from this model.
The logprobs field returns a populated block, but the tokens in it belong to the reasoning trace rather than the visible answer. You cannot use it to inspect the token probabilities of the reply; if you try, you are reading the statistics of the wrong text.
Two parameters that could be measured do work: temperature visibly affects the output, and top_p, measured as a contrast, pulls a high-temperature output back into coherence. Those two are advertised as supported parameters on the model card; no claim is made about the ones that are not.
- tool_choice none: do not send the tools field at all.
- stop: do not use it on this model; use max_tokens or a schema instead.
- reasoning_effort: not supported on this row, do not send it.
- seed: do not rely on it for reproducibility.
- logprobs: the tokens returned do not belong to the visible answer.
What should change in your agent code
First, make silent failures visible. Add an empty-content check before you use a response, and count an empty content field with a normal finish reason as an error. That single check catches two of the three cases above immediately.
Second, manage tool use through the shape of the request rather than through a parameter. On turns where you want no tools, do not send the tools field at all; that is both the correct behaviour and the only path the gateway does not refuse.
Third, bind the output to a schema. Using json_schema instead of parsing free text both reduces parse failures and moves the detection of an empty or malformed answer from the parsing stage to the validation stage. On this model the schema held even against a prompt written to violate it.
Frequently asked questions
Why does tool_choice none return 400 when it is a standard value?
Not because it is standard, but because it does not work on this row. Instead of suppressing the call, the provider writes the raw tool-call template into the reply text and returns 200. LLMTR stops the request before the provider and returns a 400 telling you what to do; to disable tools, leave the tools field out of the request.
I sent a stop sequence and got an empty answer. Is that a bug?
No, it is measured behaviour. The model thinks before answering, and if the stop sequence matches that hidden text, generation halts there and the visible answer never begins. The safest choice on this model is not to use stop; use max_tokens to shorten a response.
Can I lower reasoning_effort to reduce cost?
No, the field is not supported on this row. The three accepted values did not separate in testing — low actually reasoned longer than high. That is why LLMTR offers no reasoning level on this row.
Do these refused requests come out of my free quota?
A tool_choice none request sent alongside tools never enters the quota, because the gateway refuses it before any allowance is reserved. Requests that are accepted, processed and then fail can spend one.
How do I make the same prompt return the same answer?
On this model you cannot. The seed field did not produce reproducibility, and even at temperature zero three calls gave three different answers. If you need a stable result, bind the answer to a schema and assert field by field rather than comparing text exactly.