EVREN guides · 2026-09-25
EVREN response waiting and streaming experience
EVREN rows accept stream: true, but the answer is delivered in chunks only after it has been completed. How this affects waiting time, what determines that time, and how waiting can be shown in an interface.
Short answer
A request sent to an EVREN row with stream: true does not fail, and the answer arrives in chunks. It is not real streaming, though: the answer is completed on EVREN's side and then delivered in parts. For the user this means the first word appears when generation has finished. The interface has to be designed around that wait.
We recommend using EVREN through LLMTR: you register your own EVREN key once in Settings, and the same code then reaches both EVREN models and the rest of the catalog. The key stays yours, and EVREN's own logging and retention policy still applies unchanged.
Real streaming versus chunked delivery
In real streaming, each token the model produces is sent to the client as soon as it exists. The first word usually appears within a second or two and the text grows on screen; the total time is the same, but the user does not feel the wait.
On EVREN rows nothing reaches the client until the whole answer has been generated. When generation finishes, the chunks are delivered back to back at very short intervals. The data format is the same as real streaming, so client code written for stream: true works without changes. Only the timing differs: the time to the first chunk equals the total generation time.
What determines the wait
Because the first chunk arrives when generation finishes, anything that lengthens generation lengthens the wait. The table shows the main factors.
| Factor | Effect | Counterpart |
|---|---|---|
| Answer length | Every new token adds to generation time | max_tokens is kept at the length the task actually needs |
| Input length | A long document is processed before generation starts | Only the relevant parts are sent |
| Model | Rows with fewer active parameters do less computation per token | The rows EVREN positions as everyday and fast are tried |
| Back-to-back requests | EVREN returns 429 when it applies a rate limit | Requests are queued, then retried after a wait |
Waiting in the interface and the client
Indicators built on real streaming, such as a typing effect, start with a long gap on these rows. A status indicator showing how long the pending request has been running, plus a cancel button, tells the user more accurately what is happening. A thirty-second wait on these rows is not a failure.
On the client side, the timeout is kept long enough to cover the whole generation. Even when the SDK's own timeout is sufficient, proxies and load balancers between the server and the browser often have shorter timeouts and can cut a long answer halfway. The example below carries both a timeout and the user's cancellation on the same request.
Streaming from EVREN with a timeout and cancellation
import OpenAI from "openai";
const client = new OpenAI({ baseURL: process.env.LLMTR_BASE_URL + "/v1", apiKey: process.env.LLMTR_API_KEY });
export async function answer(question: string, signal: AbortSignal, onText: (t: string) => void) {
const started = Date.now();
// On EVREN rows the first chunk arrives when generation has finished, so
// the timeout is long enough to cover the whole generation.
const stream = await client.chat.completions.create(
{ model: "evren/gemma-4-31b", stream: true, max_tokens: 800,
messages: [{ role: "user", content: question }] },
{ timeout: 120_000, signal }
);
for await (const chunk of stream) {
onText(chunk.choices[0]?.delta?.content ?? "");
}
return Date.now() - started;
}Rate limits and retries
LLMTR applies no rate limit of its own to these rows; a 429 comes from EVREN's limit and returns with the type provider_rate_limited. A community-written integration notes a wait of about 5 seconds on consecutive requests. That figure is not a limit EVREN has published, but it shows that parallel requests can come back with 429.
A fault on EVREN's side returns as 502 and can be retried after a short wait. Other 4xx responses concern the request itself; sending the same request again does not change the result. The error-handling article in this series covers every error type.
A background pattern for long jobs
For work that can take minutes, such as writing a report or summarising a long document, keeping the user waiting at the end of an open connection is the most fragile arrangement. If the connection drops, the tab closes or a proxy times out, the generated answer is lost with it.
On these rows the first chunk arrives only when the job is done anyway, so streaming gives long jobs no advantage for the user. Sending the request as a background job on the server, storing the result and showing it to the user when it is ready is the more robust path. Short chat answers can still be awaited on the same screen.
Frequently asked questions
Should I use EVREN directly or through LLMTR?
We recommend using EVREN through LLMTR. Your requests still go to EVREN on your own EVREN key and LLMTR charges nothing for them; the difference is that the same code also reaches the other models in the catalog. EVREN's logging and retention policy is not affected by the choice.
Will I get an error if I send stream: true?
No. The request is accepted and the answer arrives in chunks. The only difference is that the chunks are sent after the answer has been completed.
Why does the first chunk arrive late?
On these rows the answer is delivered after it is completed on EVREN's side, so the time to the first chunk equals the total generation time. A lower max_tokens value and a shorter input shorten it.
Is there a difference if I send stream: false?
The total time is the same. With stream: false the answer arrives in one piece; with stream: true the same answer arrives in chunks after it is complete. On these rows the difference is very small for the user.
Which EVREN row answers fastest?
EVREN positions Gemma 4 31B as its everyday low-latency model and Qwen3.8 Flash Next as its highest-concurrency row. Measuring the time on your own workload gives the most accurate comparison.