Reranking

Reranking evaluates a query together with a list of candidate documents and produces a relevance score for each one. Embedding search vectorizes query and document separately for speed; a reranker reads both at once and therefore produces a more accurate ordering.

The usual pattern has two stages: retrieve the top 50-100 candidates with embeddings, rerank them, and pass only the most relevant few to the language model. That improves answer quality and lowers the generation model's token cost.

Models

Model	Context	Price ($/1M tokens)
qwen/qwen3-reranker-8b	32,768	0.05
voyageai/rerank-2.5	32,768	0.05
voyageai/rerank-2.5-lite	32,768	0.02

To list every reranking model in the catalog:

curl -s "$LLMTR_BASE_URL/v1/models" \
  | jq -r '.data[] | select(.supported_operations[] == "RERANK").id'

Request

curl "$LLMTR_BASE_URL/v1/rerank" \
  -H "Authorization: Bearer llmtr-your_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen/qwen3-reranker-8b",
    "query": "When are invoices issued?",
    "documents": [
      "Invoices are issued on the first business day of each month.",
      "Shipping takes 2-3 business days.",
      "The return window is 14 days."
    ],
    "top_k": 2
  }'

Field	Required	Description
model	Yes	Identifier of the reranking model.
query	Yes	The query the documents are scored against.
documents	Yes	Candidate document list. At least 1, at most 1000 items.
top_k	No	Return only the N highest-scoring results.
top_n	No	Same as top_k; accepted for clients written against Cohere's spelling.
return_documents	No	When true each result also carries its document text. Defaults to false.
truncation	No	Voyage models only. See the note below.

Response

{
  "object": "list",
  "data": [
    { "index": 0, "relevance_score": 0.9977 },
    { "index": 2, "relevance_score": 0.3738 }
  ],
  "model": "qwen/qwen3-reranker-8b",
  "usage": { "total_tokens": 262 }
}

data is returned in descending order of relevance. index is the document's position in the documents array you sent — use it to map the ranking back to your own records. relevance_score runs from 0 to 1 and is not comparable across models; it only orders the documents within a single request relative to each other.

Billing

You are charged on total processed tokens, and usage.total_tokens in the response carries that count. Reranking is a single pass: there are no output tokens and no cache discount.

The total includes the query once per document. Reranking 50 documents in one request therefore processes the query 50 times. Ranking many short documents against a long query costs noticeably more than ranking the same documents against a short one.

No platform margin is added to model prices; the margin applies to credit top-ups only. See Billing for details.

Limits

A single request evaluates at most 1000 documents. Split larger lists into batches and merge the returned scores yourself.

For qwen/qwen3-reranker-8b there is a second limit: the query plus any one document may not exceed 131,072 characters. A request that exceeds it is rejected with a 400 naming the document at fault:

{
  "error": {
    "message": "documents[3] is too long: the query and one document may total at most 131072 characters, this pair is 140210.",
    "type": "invalid_request"
  }
}

This model does not shorten an over-long document, it refuses it. That is also why it does not accept the truncation parameter: you would believe a document was being trimmed when the request was in fact failing. Chunk your documents before sending them. The Voyage reranking models do support truncation.

Use in a RAG pipeline

import os, requests

BASE_URL = os.environ["LLMTR_BASE_URL"]
HEADERS = {"Authorization": f"Bearer {os.environ['LLMTR_API_KEY']}"}

# 1. First-pass candidates from vector search
candidates = vector_search(query, limit=50)   # your own search layer

# 2. Reorder by relevance
res = requests.post(
    f"{BASE_URL}/v1/rerank",
    headers=HEADERS,
    json={
        "model": "qwen/qwen3-reranker-8b",
        "query": query,
        "documents": [c["text"] for c in candidates],
        "top_k": 5,
    },
)
res.raise_for_status()

# 3. Send only the five most relevant chunks to the language model
context = [candidates[item["index"]]["text"] for item in res.json()["data"]]

The rerank endpoint is not part of the OpenAI SDK; call it over plain HTTP.

Choosing a model

qwen/qwen3-reranker-8b reads over 100 languages and works on Turkish query-document pairs. It is an 8-billion-parameter model, so it runs a larger network than the similarly priced alternatives, at the cost of latency that grows with the document count.

The Voyage reranking models use the same endpoint and the same request shape; voyageai/rerank-2.5-lite is the lowest unit cost reranking option in the catalog. Switching between the three requires changing only the model field.

Errors

Status	Meaning
400 invalid_request	Document count, length limit or top_k value is invalid.
404 model_not_found	The model identifier is not in the catalog.
400 unsupported_operation	The model does not support reranking. You may have sent a rerank request to a chat model.
429 rate_limit_error	The API key's request limit was exceeded.
502 provider_error	The provider could not complete the request.

See Errors for the error format.