Agent workflows · 2026-09-25
Multiple independent questions over one document with Jev
Why asking several Jev questions of the same document in one request is cheaper, what the independence of questions means, when a second request is needed, and how the state and questions share the context limits.
Short answer
In Jev the input of a request is the sum of the state and questions fields, and the charge is on that input only. Sending the document again for every question asked of it means paying for the document's tokens every time. Grouping the questions in one request means sending the document once and paying, for each extra question, only that question's tokens.
Questions in the same request are evaluated independently and in parallel. So adding a question does not noticeably lengthen the response time and does not change the results of the other questions.
The cost difference
The table compares a hypothetical case, 20 questions of about 50 tokens each asked of an 8,000-token document, in two arrangements. The price is $0.042 per million input tokens; output is free, but it is still counted in the usage field of the response.
| Arrangement | Requests | Input tokens | Cost |
|---|---|---|---|
| 20 questions in one request | 1 | 9,000 | $0.000378 |
| Each question in its own request | 20 | 161,000 | $0.006762 |
What independence means
TypeSafe states it plainly: one question's answer is not hidden context for another. Every question sees the same state, but none sees the other questions' answers. That means a conditional question such as "if the document is a lease, does it have a deposit clause" cannot be expressed in one request.
In most cases this does not call for a second request. The conditional question goes into the same request, and when the condition is not met its answer is simply not read in code. The tokens of a few extra questions are almost always cheaper than sending the document a second time. A second request is needed when the question itself or its options cannot be written without the first answer: for example, when the first request determines the document type and the second uses an option list specific to that type.
Structured data as state
state does not have to be plain text; structured data in the form of an object or an array is accepted too. The fields of an application form, the lines of an order or the messages of a conversation can be sent with their field names instead of being pasted into one text. Field names make it easier to tell which information came from where: "phone" and "emergency phone" stay two separate fields.
Structured data also counts against the budget. Field names and formatting use tokens, so leaving out fields unrelated to the questions reduces both cost and budget use.
Question keys and matching answers
questions is a map whose keys you choose. The keys are not sent to the model; answers come back under the same keys and the code reads them by key. What the model sees is the instructions field, so each question's instruction must make sense on its own. The example below asks four questions of a contract in one request and reads the two conditional answers only if the document really is a lease.
Four independent questions over one contract
import os, requests
# Keys are only for matching answers; they are not sent to the model.
QUESTIONS = {
"is_lease": {"type": "noul", "instructions": "Is this document a lease agreement?"},
"has_deposit": {"type": "noul", "instructions": "Does the document contain a deposit or security clause?"},
"notice_period": {
"type": "choice",
"instructions": "Which termination notice period does the document state?",
"criteria": {"30_days": None, "60_days": None, "90_days": None, "not_stated": None, "other": None},
},
"risk": {
"type": "score",
"instructions": "How one-sided is the agreement from the tenant's point of view?",
"criteria": ["Balanced", "Partly one-sided", "Clearly one-sided"],
},
}
res = requests.post(
os.environ["LLMTR_BASE_URL"] + "/v1/systemone",
headers={"Authorization": "Bearer " + os.environ["LLMTR_API_KEY"]},
json={"model": "typesafe/jev", "state": contract_text, "questions": QUESTIONS},
timeout=60,
)
body = res.json()
answers = body["answers"]
if answers["is_lease"]["noul"] >= 0.9:
# For non-lease documents these two answers are ignored; asking is cheaper than a second request.
print(answers["has_deposit"]["noul"], answers["notice_period"]["choice"])
print(body["usage"]["input_tokens"])Budget and limits
Context is measured against two limits: each question together with the state must fit 32K tokens, and the state with all questions 64K tokens; with the TypeSafe-Danger-Zone-Long-Context: true header both become 262K. A request accepts at most 1,000 questions and anything beyond is refused with 400 invalid_request, but in practice the context limit is what binds: even a question with empty instructions uses tokens, so 1,000 questions exceed 64K long before. A document over the limit is split into parts; each part becomes its own state and the questions see only that part.
The optional max_budget_usd field is a pre-request estimate: if the estimated cost exceeds it, the request is never sent. The exact charge is calculated from the token count the provider reports and is not checked against that cap again. For a hard upper limit, a per-key spending limit is used. The endpoint does not stream; the answer comes back in one piece.
Growing the question set
The independence of questions has one more practical consequence: adding a new question to a set does not change the results of the existing ones. A flow that starts with four questions for a document type can grow to twenty over time, and the thresholds of the first four do not have to be measured again because of it.
As the number of questions grows, keeping them in one definition file rather than spreading them across the code makes it easy to see which question was added when and which threshold it is tied to. The usage field of the response also shows how input tokens change as the set grows.
Frequently asked questions
Do questions in the same request affect each other?
No. Every question sees the same state but not the other questions' answers. Adding or removing a question does not change the others' results.
When is a second request needed?
When a question's instruction or options cannot be written without the first answer. If only the use of the answer depends on a condition, putting the question in the same request and ignoring the answer in code is cheaper.
How many questions can one request carry?
At most 1,000 questions are accepted. In practice the limit is set by the context: each question with the state must fit 32K tokens, the whole request 64K.
Is max_budget_usd a hard spending cap?
No. It is an estimate made before the request; the exact charge is not checked against it afterwards. For a hard limit, a per-key spending limit is used.
Does state have to be plain text?
No. state can be plain text or structured data in the form of an object or an array. Field names use tokens too, so only the fields relevant to the questions are sent.