Audio decisions (Hume Audio Decision Model)

hume/audio-decision-model is Hume's decision model for audio. It takes one recording (MP3 or WAV) and the questions you define, and answers each question with probabilities instead of text. A question can be about how speech sounds (a caller's impatience, how natural a synthetic voice is), what is said (a request to cancel a subscription) or other sounds (background noise, a phone ringing). It returns no transcript, explanation or reply; your code combines the probabilities with thresholds and weights.

The model is called on the same POST /v1/systemone endpoint as Structured Decisions, with your existing LLMTR API key. The request shape is the same; the recording comes in the audio field, and one question type (span) exists on this model only.

Model	Input / Output ($/1M tokens)
`hume/audio-decision-model`	0.47 / 0

Request

AUDIO_B64=$(base64 < call.wav | tr -d '\n')

curl "$LLMTR_BASE_URL/v1/systemone" \
  -H "Authorization: Bearer llmtr-your_key" \
  -H "Content-Type: application/json" \
  -d @- <<EOF
{
  "model": "hume/audio-decision-model",
  "state": "A support call to an internet provider. The agent answers the call, and the caller reports an outage.",
  "audio": {"format": "wav", "data_base64": "$AUDIO_B64"},
  "questions": {
    "agent_tone": {
      "type": "choice",
      "instructions": "Which best describes the tone of the agent?",
      "criteria": {
        "warm": "Warm and friendly toward the caller",
        "neutral": "Polite but impersonal",
        "curt": "Short or dismissive with the caller"
      }
    },
    "caller_impatience": {
      "type": "score",
      "instructions": "How impatient does the caller sound?",
      "criteria": ["Not at all", "2", "3", "4", "Extremely"]
    },
    "background_noise": {
      "type": "noul",
      "instructions": "Is there distracting background noise?"
    },
    "cancel_moment": {
      "type": "span",
      "instructions": "When does the caller say they want to cancel their service?"
    }
  }
}
EOF

curl reads the body from standard input, so the base64 of a five-minute recording never has to fit on the command line.

import base64, os, requests

with open("call.wav", "rb") as f:
    audio_b64 = base64.b64encode(f.read()).decode("ascii")

res = requests.post(
    "https://llmtr.com/v1/systemone",
    headers={"Authorization": f"Bearer {os.environ['LLMTR_API_KEY']}"},
    json={
        "model": "hume/audio-decision-model",
        "state": "A support call to an internet provider.",
        "audio": {"format": "wav", "data_base64": audio_b64},
        "questions": {
            "background_noise": {
                "type": "noul",
                "instructions": "Is there distracting background noise?",
            },
            "cancel_moment": {
                "type": "span",
                "instructions": "When does the caller say they want to cancel their service?",
            },
        },
    },
    timeout=120,
)
res.raise_for_status()

answers = res.json()["answers"]
if answers["background_noise"]["noul"] > 0.7:
    print("noisy recording")
for span in answers["cancel_moment"]["spans"]:
    print(span["audio_start_ms"], span["audio_end_ms"], span["probability"])

import { readFileSync } from "node:fs";

const res = await fetch("https://llmtr.com/v1/systemone", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.LLMTR_API_KEY}`,
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    model: "hume/audio-decision-model",
    state: "A test call with a voice agent. Both voices are synthetic, and the agent speaks first.",
    audio: { format: "wav", data_base64: readFileSync("test_call.wav").toString("base64") },
    questions: {
      naturalness: {
        type: "score",
        instructions: "How natural does the agent sound?",
        criteria: ["Not at all", "2", "3", "4", "Extremely"]
      },
      talks_over: { type: "noul", instructions: "Does the agent talk over the caller?" }
    }
  })
});

const { answers, usage } = await res.json();
console.log(answers.naturalness.score, answers.talks_over.noul, usage.input_tokens);

Field	Required	Description
`model`	Yes	`hume/audio-decision-model`
`audio`	Yes	`{"format": "wav" or "mp3", "data_base64": "..."}`. Standard base64, without a `data:` prefix. The file itself must be a WAV or MP3 file; a URL is not accepted.
`state`	Yes	A short description of the recording, plain text, up to 1,000 characters: the setting, who is speaking, the script that was read. Put the text your questions depend on here, each piece on its own line with a label such as `Script:`.
`questions`	Yes	A map of questions keyed by IDs you choose (1-256 characters). Answers come back under the same keys.
`max_budget_usd`	No	A pre-request check on the estimated cost. On this model the estimate is made for a five-minute recording whatever the real length, so for short recordings the real charge is far below it. A value under the estimate returns `400 budget_too_low`.

The assets and images fields and the box question type are not used on this model.

Question types

Ask each question for one judgment. Split a decision into several questions (the agent's tone, the caller's impatience, a request to cancel) and combine the answers in your code.

Type	Asks	`criteria`	Answer
`noul`	Whether something is true of the recording	Optional: `{"true": "...", "false": "..."}`, each 1-300 characters	`noul`: the estimated share of listeners who would answer yes (0-1)
`choice`	Which of several options applies	Option name → description; 2-20 options, name 1-50 and description 1-100 characters (a description may be `null`)	`choice`, `confidence`, and `probabilities` for each option
`score`	Where the recording falls on an ordered scale	2-10 levels from lowest to highest, each 1-150 characters	`score` (the probability-weighted mean level), `confidence`, `probabilities`, `legend`
`span`	When something happens	None; describe the event in full in `instructions`	`spans`: `audio_start_ms`, `audio_end_ms`, `probability`; an empty list means the event was not found

For a scale, use five levels with labelled ends: ["Not at all", "2", "3", "4", "Extremely"] or ["Strongly disagree", "Disagree", "Neutral", "Agree", "Strongly agree"].

A probability is the estimated share of human listeners who would give that answer: a value near 0.5 means listeners would split, not that the model is unsure.

The questions are answered together. On this model all the questions in a request are answered in one inference, so adding, removing or reordering questions can change the other answers. Send the same questions in the same order for every recording you compare. If one answer decides your next question, send a second request with the same recording.

Response

{
  "model": "hume/audio-decision-model",
  "answers": {
    "agent_tone": {
      "type": "choice",
      "choice": "warm",
      "confidence": 0.3072,
      "probabilities": { "warm": 0.5381, "neutral": 0.3926, "curt": 0.0693 }
    },
    "caller_impatience": {
      "type": "score",
      "score": 2.7195,
      "confidence": 0.4229,
      "probabilities": { "0": 0.0218, "1": 0.0874, "2": 0.2463, "3": 0.4385, "4": 0.206 },
      "legend": { "0": "Not at all", "1": "2", "2": "3", "3": "4", "4": "Extremely" }
    },
    "background_noise": { "type": "noul", "noul": 0.1274 },
    "cancel_moment": {
      "type": "span",
      "spans": [{ "audio_start_ms": 38120, "audio_end_ms": 38940, "probability": 0.7734 }]
    }
  },
  "usage": { "input_tokens": 11683, "output_tokens": 0 }
}

Treat an answer with a type you do not recognize as unknown, not as an error.

Preparing the recording

A recording must be 5 seconds to 5 minutes long. Split a longer one at pauses rather than at fixed intervals, ask every part the same questions and combine the answers in your code (the mean for an overall level, the maximum for whether something happens at all). Each part must be at least 5 seconds long too.
The model hears audio as 16 kHz, 16-bit, one channel; a two-channel file is mixed into one. To judge one person on their own, send that person's channel or turns as a separate file.
Because the recording travels as base64 in the JSON body, the file can be up to about 18 MB. Convert WAV files to 16 kHz, 16-bit mono: five minutes is then about 9.6 MB.
Skip silent or nearly silent files; the model judges silence like any other audio and can answer it with high confidence.
With phone recordings, ask about a specific problem (such as background noise) rather than overall quality; phone audio carries less high-frequency sound, so the model hears it as lower in quality and "Does the audio sound muffled?" comes out high for nearly every call.
Questions about music (where the chorus starts), the direction of a sound (left, right) and open questions ("Why is the caller calling?") are not answered well. Instead of an open question, ask a choice question that lists the likely reasons.
Hume publishes no list of supported languages. Before relying on a question, run it on recordings of your own whose answer you know, at the thresholds you plan to use.

Billing

Only input tokens are billed: 1 million input tokens cost $0.47, and output is free. The input token count depends mostly on the recording's length; the text of the questions and of state counts too. In the example in Hume's documentation, a 135-second call with four questions came to 11,683 input tokens, about $0.0055. The billed count is in the response's usage.input_tokens.

Limits

Limit	Value
Recording length	5 seconds - 5 minutes
Format	MP3 or WAV, one file per request
File size	about 18 MB (before base64)
Questions	1-25: `noul`, `choice` and `score` together at most 20, `span` at most 5
`instructions`	1-500 characters
`state`	up to 1,000 characters

Errors

Status	Meaning
`400 invalid_request`	The request is not in the shape this model accepts. The message says what to change: `audio` is missing or the file is not MP3 or WAV; the recording is shorter than 5 seconds or longer than 5 minutes; the file could not be decoded; `state` or `instructions` is not text; a `box` question, `assets` or `images` was sent; a question or `state` is outside the limits above (the message names the field).
`400 budget_too_low`	`max_budget_usd` is below this request's estimated cost.
`413`	The file is too large for this model.
`503 model_busy`	The model is at capacity right now. Wait for the time in the `Retry-After` header and try again; this request is not charged.
`503 model_unavailable`	The model is temporarily unavailable; nothing is charged.
`502 provider_error` / `504 provider_timeout`	The request did not complete; nothing is charged. Try again.