Connecting Open WebUI to LLMTR: keeping the chat model list usable

The catalog lands in the chat list unchanged

Open WebUI is a multi-user chat interface you run on your own server. The model picker collects the model list of every configured connection and merges them into one. When a connection points straight at a model server, that list already contains only chat models and nothing goes wrong.

A connection pointing at a gateway behaves differently. A gateway publishes the catalog of every provider it fronts as one list, and that catalog holds embedding, rerank, image generation, video generation and audio models too. Open WebUI lists all of those as chat models. Select one and the request goes to the chat endpoint, which the model does not serve, so the turn fails on the first message.

The problem is not in the connection setup but in how the list is built. In a catalog of several hundred rows this decides whether the picker stays usable at all: models you can chat with sit interleaved with models you cannot.

Set up the connection, read the catalog without a key first

Two values are enough on the LLMTR side: a base URL and an API key. Keys carry the llmtr- prefix and go into the Open WebUI admin connection screen as they are. Model identifiers use the provider/model form, so you use the name exactly as the list publishes it.

You can inspect the catalog before configuring anything, because the model list endpoint is readable without a key. Each entry declares the operations it serves in its supported_operations field and the gateway routes those operations map to in supported_endpoints. The command below dumps the list with those fields, so you can see how many entries are non-chat before the connection exists.

The model list is keyless, and every entry carries its operation field

# The two values entered in the Open WebUI connection screen
# Base URL: https://llmtr.com/v1
# Key:      llmtr-your_key

curl -s https://llmtr.com/v1/models \
  | jq -r '.data[] | [.id, (.supported_operations | join("+"))] | @tsv' \
  | sort -k2

# Count only the non-chat entries:
curl -s https://llmtr.com/v1/models \
  | jq '[.data[] | select((.supported_operations | index("CHAT_COMPLETIONS")) | not)] | length'

How the fix narrows the model list

The fix lives on the fix/gateway-non-chat-model-filter branch of the knowhycodata/open-webui repository and adds a single helper named supports_text_generation to backend/open_webui/routers/openai.py. It is applied in the two places where a fetched catalog is turned into a model list: the path that aggregates every connection response and the path that returns one connection's model identifiers.

The decision comes down to one question: does this entry serve the text generation endpoint this connection uses? When the connection's api_type is responses the required operation is RESPONSES, otherwise it is CHAT_COMPLETIONS. If the entry publishes supported_operations that list is consulted directly; if it does not, the suffix of each name in supported_endpoints is checked instead.

What the patch does not do matters just as much. A provider that advertises nothing is assumed to be chat capable, so plain OpenAI, Ollama and vLLM connections behave exactly as before. Connections with a manually configured list of model identifiers are left untouched.

• On connections that publish the fields, the list narrows to entries declaring the required operation.
• When supported_operations is missing or empty, the endpoint names in supported_endpoints decide.
• A provider advertising neither field counts as chat capable, so existing installs do not break.
• Connections whose model identifiers are written by hand stay outside the filter.

Responses connections and the one-way half of the bridge

On LLMTR a model does not need a CHAT_COMPLETIONS binding to be callable from the chat endpoint: models with a RESPONSES binding work there too. The reverse does not hold, because a model with only a chat binding cannot be called through the Responses endpoint. The bridge runs one way.

The filter knows nothing about that bridge, since it decides purely on the operation an entry declares. The consequence is that a chat-type connection drops entries that declare only RESPONSES, even though the gateway would have served the request. In the catalog openai/gpt-5.6-terra and xai/grok-4.6 sit in that position. Entries carrying both bindings, such as google/gemini-3.7-flash, stay listed on either connection type.

Configure the connection as a Responses one and the equation inverts: now entries with only a chat binding, such as anthropic/claude-sonnet-5 and llmtr/gemma-4, drop out, and that is correct because the request genuinely would not be served.

• A chat-type connection can call models with either a chat or a Responses binding.
• A Responses-type connection only works with models that carry a RESPONSES binding.
• Because the filter ignores the bridge, a chat-type connection loses a handful of otherwise valid models.
• To keep such an entry listed, add it to the manual model identifier list for that connection.

Without the patch: the manual model identifier list

Open WebUI's own release does not carry this filter; what it offers instead is a per-connection list of model identifiers written by hand. When that list is filled, catalog entries are taken at face value and the picker is built only from the identifiers you wrote. The patched build leaves those connections alone as well, so the two approaches do not collide.

This route is enough for installs running a short, stable set of models. The cost is that you maintain the list as the catalog moves: a newly added model goes unnoticed, and a retired one keeps sitting in the picker. The filter, by contrast, recomputes the list on every catalog fetch.

Combining them is reasonable in practice: leave the everyday connection to the filter, and pin a special Responses connection with an explicit list.

A manual model identifier list in the connection config disables catalog narrowing

{
  "model_ids": [
    "anthropic/claude-sonnet-5",
    "google/gemini-3.7-flash",
    "llmtr/gemma-4"
  ]
}

What changes in a multi-user install

The filter operates per connection, not per user. Touch one connection's configuration and the picker changes for everyone using it, with no per-user state to maintain. You can define two connections on the same server, one chat type and one Responses type, and hand different lists to different teams.

Cost is independent of the interface. Billing is token based and draws down a credit balance, so which model gets picked drives consumption directly. The LLMTR platform margin applies only when credits are topped up and is 8 percent; no margin is added to model prices. That is why a short list is a spend predictability question as much as a usability one.

One privacy detail is worth stating plainly: user prompts and model response bodies are not written to the database. The side that stores conversation history is your own Open WebUI install.

Frequently asked questions

Do I lose models if a provider publishes no operation field?

No. Entries that advertise nothing are assumed to be chat capable and stay in the list. That is why plain OpenAI, Ollama and vLLM connections are unaffected by the patch: the filter only engages on catalogs that publish those fields.

Is this fix part of Open WebUI's own release?

No. The change sits on the fix/gateway-non-chat-model-filter branch of the knowhycodata/open-webui repository. Open WebUI's own release has no equivalent field reading; its counterpart is the per-connection list of model identifiers written by hand.

Can I still use embedding and rerank models?

The filter narrows only the chat model picker. Embedding and rerank entries keep serving requests on their own endpoints; the patch does not close those calls, it removes them from the chat list. The endpoint field in the model list shows where each entry is callable.

A model I expected is missing from the picker. Where do I look?

Read that entry's operation field from the model list endpoint first. If it says only RESPONSES and your connection is chat type, the filter caught it, and adding it to the manual model identifier list brings it back. If the field comes back empty, the entry is not callable from any published endpoint today.

Related posts

• Developer tools that work with LLMTR: how 21 integrations connect
• The models.dev LLMTR provider entry and metadata sync
• Running the Open SEO SAM agent through an LLMTR gateway

• Model catalog
• Chat Completions guide
• API keys
• Pricing