Structured Decisions (System One)

When you ask a language model for a decision your code will branch on, you are doing two jobs at once: telling the model to produce the answer, then parsing the text it produced back into your own types. The parsing step is the fragile one — the model changes format, adds a preamble, truncates the JSON.

System One models remove that step. They take a state (the content to judge) and the typed questions you define, and return a structured answer for each question. No free text is generated, so there is nothing to parse. Every answer arrives with its probability distribution, which means the threshold is yours to set in code.

Model

Model	Context	Input / Output ($/1M tokens)
typesafe/jev	32,000	0.042 / 0

To list the System One models in the catalog:

curl -s "$LLMTR_BASE_URL/v1/models" \
  | jq -r '.data[] | select(.supported_operations[] == "SYSTEM_ONE").id'

Request

curl "$LLMTR_BASE_URL/v1/systemone" \
  -H "Authorization: Bearer llmtr-your_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "typesafe/jev",
    "state": "I was charged twice on my card and I want the duplicate refunded.",
    "questions": {
      "refund_requested": {
        "type": "noul",
        "instructions": "Is the customer requesting a refund?"
      },
      "department": {
        "type": "choice",
        "instructions": "Which team should handle this request?",
        "criteria": {
          "billing": "Payments, invoicing, refunds",
          "technical": "Bugs, outages, integrations",
          "sales": "Pricing, new accounts"
        }
      }
    }
  }'

Field	Required	Description
model	Yes	The System One model id.
state	Yes	The content to evaluate. Plain text or structured data (an object or an array).
questions	Yes	A map of questions whose keys you choose. Answers come back under the same keys.
max_budget_usd	No	A pre-flight estimated-cost check. See the note below.

Question keys are not sent to the model; they only match answers back to questions. Write the complete question in instructions, however self-explanatory the key looks — the key is not text the model sees.

max_budget_usd is a pre-flight check: the request is refused before it reaches the upstream when the cost estimated from the body exceeds this value. The final charge is computed from the token count the upstream reports and is not re-checked against this ceiling, so input that produces more tokens per character than expected can be billed above the estimate. Use the per-API-key spending limit when you need a hard cap.

Questions in one request are evaluated independently and at the same time. One answer never becomes context for another, so adding questions does not noticeably lengthen the response and costs only that question's tokens. Putting a question you will not need on every input into the same request and ignoring it in code is cheaper than making a second request.

Question types

All three share type and instructions; criteria differs by type.

Noul

A yes/no question. Returns the probability that the answer is yes, between 0 and 1. criteria is optional and only clarifies what each end means.

{
  "is_urgent": {
    "type": "noul",
    "instructions": "Does this message convey urgency?",
    "criteria": {
      "true": "Explicitly time-sensitive",
      "false": "No urgency expressed"
    }
  }
}

Choice

Picks one of the options you define. Returns the selected option, the probability distribution across every option, and a confidence value. criteria maps each option name to a description; use null when an option needs no further detail.

Add an option such as other when your inputs might not fit the list; otherwise the model always returns the closest option in it.

{
  "department": {
    "type": "choice",
    "instructions": "Which team should handle this request?",
    "criteria": {
      "billing": "Payments, invoicing, refunds",
      "technical": "Bugs, outages, integrations",
      "other": null
    }
  }
}

Score

Returns a position along ordered levels. criteria is an ordered array of at least two levels. The returned score can fall between two of them.

{
  "frustration": {
    "type": "score",
    "instructions": "How frustrated does the customer appear?",
    "criteria": ["Calm", "Frustrated but civil", "Very angry"]
  }
}

Do not confuse a yes/no judgment with measuring a level. For "is this candidate strong in Python?", a value of 0.5 does not mean the candidate is mid-level; it means the model gives yes and no equal probability. Use Score to measure a level, and a Noul with a clearly defined condition when you need a decision.

Response

{
  "model": "typesafe/jev",
  "answers": {
    "refund_requested": {
      "type": "noul",
      "noul": 0.94
    },
    "department": {
      "type": "choice",
      "choice": "billing",
      "probabilities": { "billing": 0.88, "technical": 0.07, "sales": 0.05 },
      "confidence": 0.85
    }
  },
  "usage": { "input_tokens": 312, "output_tokens": 61 }
}

Every key in answers is a question key you supplied. Each answer carries its own type, so which fields to read is unambiguous: noul for a Noul, choice + probabilities + confidence for a Choice, score + legend + probabilities + confidence for a Score.

The model field returns the LLMTR catalog id. The token counts above are illustrative; output_tokens normally comes back greater than zero — what is free is the price of output, not the count. Do not assume this field is zero when building your own token accounting.

Choosing a threshold

confidence summarizes how peaked the probability distribution is; where to act automatically is your decision. If you only need the most likely option, you do not need a threshold at all — reading choice is enough. Thresholds earn their place where you separate acting automatically from handing a case to a person.

Keep thresholds and question text together in one file in your code. Those are what a reviewer needs to look at; threshold constants scattered across a codebase are not found again later.

Billing

Only input tokens are billed; output tokens are free. The usage field in the response carries both counts.

Input is the sum of state and questions. Asking many questions about the same state is therefore cheaper than sending that state again in a second request — combine questions into one call.

Platform margin is not added to model prices; the margin applies to credit top-ups only. See Billing for details.

Limits

state and questions share a single context budget: 32,000 tokens for typesafe/jev, roughly 150,000 characters of English text.

A single request accepts at most 1,000 questions; more are refused with 400 invalid_request. In practice the budget binds first, not that cap: even a question with an empty instruction costs tokens, so 1,000 questions exceed the 32,000-token budget well before the limit is reached.

This endpoint does not support streaming. The response arrives in one piece; there is no generated text to stream.

System One models do not appear in the Playground and cannot be called on chat endpoints. typesafe/jev answers on /v1/systemone only.

Errors

Status	Meaning
400 invalid_request	The request body does not match the schema. The message names the offending field.
404 model_not_found	The model id is not in the catalog.
400 unsupported_operation	The model does not support this endpoint. The response lists the endpoints it does support.
422 provider_error	A question definition is in a shape the model does not accept. The message carries which field is at fault.
429 rate_limit_error	The API key's request limit was exceeded.
503 model_unavailable	The model is temporarily unable to answer. If the response carries a Retry-After header, wait that long.

See Errors for the error format.