Integration guides · 2026-08-28
Ling 3 Tiny Thinking and Instant: reasoning and output budgets
Choose Thinking or Instant for local Ling 3 Tiny, budget generated tokens, detect truncated answers, and distinguish local weights from the retired LLMTR route.
What do Thinking and Instant change?
Ling 3 Tiny can use the same open weights for reasoning or direct answers. Set enable_thinking=true in its local chat template for Thinking, or false for the direct response behavior called Instant. This does not require downloading another model, and enabling reasoning does not guarantee a more accurate answer to every question.
As checked on August 28, 2026, InclusionAI still publishes Tiny weights under the MIT license. The LLMTR identifier inclusionai/ling-3.0-tiny, however, has been retired since August 15, 2026, and returns model_retired. Local inference is separate. LLMTR names the paid inclusionai/ling-3.0-flash as its successor; it does not silently forward Tiny requests to Flash.
Set the mode in one place
The official chat template enables reasoning by default. If the first system message already contains detailed thinking on or detailed thinking off, it preserves that instruction. Avoid carrying an old manual mode instruction alongside a new enable_thinking setting: conflicting controls can produce a prompt you did not intend.
| Environment | Selection | Meaning |
|---|---|---|
| Local Tiny | enable_thinking=true | Thinking enabled; same model weights |
| Local Tiny | enable_thinking=false | Instant; template choice for direct answers |
| Former LLMTR Tiny | :fast | Historical contract; does not reactivate the retired identifier |
| LLMTR Flash | :none or reasoning_effort=none | Disables reasoning on the paid successor |
The output budget includes more than the visible answer
In vLLM, max_tokens caps the generated output sequence. Reasoning tokens can consume that allowance, so do not interpret it as a quota exclusively for the answer shown to the user. Thinking can spend the budget before reaching even a short final answer. Hiding reasoning in the interface does not eliminate its generation.
The value 2048 below is an illustrative test budget, not Tiny’s maximum capacity or a guaranteed final answer allowance. Describe the desired answer format in the prompt and configure the overall generation limit separately. General vLLM documentation describing a separate thinking budget does not establish support in your selected Ling branch. Verify compatibility before adding it.
A single request to local vLLM
This Python example assumes the official model card’s vLLM recipe based on the ling_3_0 branch is already installed, serving the name auto with the ling3 reasoning parser. LING_LOCAL_BASE_URL is the complete local API base, such as http://127.0.0.1:8000/v1. Use LING_LOCAL_API_KEY if authentication is enabled. These are not LLMTR settings; do not expose an unauthenticated server externally.
The example was not executed, and no model response is fabricated. For an Instant trial, change only enable_thinking to False. HTTP failures propagate as errors; successful HTTP responses still undergo completeness checks.
Local request that prints status and usage metadata without printing content
import json
import os
from urllib.request import Request, urlopen
base = os.environ.get(
"LING_LOCAL_BASE_URL", "http://127.0.0.1:8000/v1"
).rstrip("/")
headers = {"Content-Type": "application/json"}
key = os.environ.get("LING_LOCAL_API_KEY")
if key:
headers["Authorization"] = f"Bearer {key}"
payload = {
"model": "auto",
"messages": [{"role": "user", "content": "Calculate 17 * 23. Return only the number."}],
"chat_template_kwargs": {"enable_thinking": True},
"temperature": 1.0,
"top_p": 0.95,
"top_k": 20,
"max_tokens": 2048,
"stream": False
}
request = Request(
base + "/chat/completions",
data=json.dumps(payload).encode("utf-8"),
headers=headers,
method="POST"
)
with urlopen(request, timeout=120) as response:
result = json.load(response)
choice = result["choices"][0]
if choice.get("finish_reason") == "length":
raise RuntimeError("Generation limit reached; review the budget.")
answer = choice["message"].get("content")
if not isinstance(answer, str) or not answer.strip():
raise RuntimeError("No final answer; check the template and parser.")
print({"finish_reason": choice.get("finish_reason"), "usage": result.get("usage")})Distinguish truncated output from empty answers
Do not treat finish_reason=length as a completed result. Reduce the task’s scope or increase the budget within your resource limits. Limit retries rather than repeatedly submitting the same request. Empty content alone does not identify the cause: reasoning may have exhausted the budget, the parser may be wrong, or the chat template may be incompatible.
Current vLLM documentation notes that reasoning_content was renamed reasoning. An empty legacy field therefore does not prove reasoning was disabled; check the server version. Hugging Face and SGLang recipes also differ in parser and NEXTN settings. Do not combine fragments from different recipes.
Choose using criteria from your own workload
Build a small evaluation set containing short classification, date extraction, and planning with several constraints. Define an acceptable answer for each task in advance. Compare both modes using the same model revision and sampling settings; one successful response is insufficient evidence for a general decision.
- Assess correctness, format compliance, and truncation rate separately.
- Measure duration on your hardware; do not estimate usage tokens from character counts.
- Keep prompts and responses out of production logs; record the mode, error category, and available usage counters.
Frequently asked questions
Do I need another Tiny download for Instant?
No. This distinction concerns how the same weights are invoked through the chat template. However, check that your local server actually applies the selected template and enable_thinking value. A changed interface toggle alone is insufficient.
Does disabling reasoning guarantee a faster answer?
There is no fixed latency guarantee. Task length, hardware, and concurrent requests also affect duration. Compare completed, correct answers on your workload; do not count a fast but truncated answer as a success.
Will sending enable_thinking reactivate Tiny on LLMTR?
No. A reasoning setting cannot reopen a retired access route. Running local weights is a separate option. If moving to paid Flash on LLMTR, evaluate the current model identifier, price, and supported reasoning controls together.