Integration guides ยท 2026-08-29
Pointing the OpenReview code review bot at LLMTR
Replacing the default gateway in the self-hosted OpenReview bot with two environment variables, choosing the reviewing model with a third, and managing per pull request review cost through that single lever.
What OpenReview does and where it connects by default
OpenReview is an open-source, self-hosted code review bot. It deploys to Vercel, attaches to a repository through a GitHub App, and runs each review as a durable workflow. Positive reactions on a review comment approve and apply its suggestions, negative ones skip them. Its instruction sets are discovered at runtime from .agents/skills/, and the agent sees only skill names and descriptions in its system prompt, loading the detail when a review calls for it.
Model calls resolve through the deployment platform's own gateway by default. The reviewing model comes from the MODEL variable, and when that is unset the id anthropic/claude-sonnet-4.6 is used. Routing lives in lib/model.ts; the agent asks for a model through a single resolve call and is unaware of which gateway answered.
The option described here lives on the openai-compatible-gateway branch of the knowhycodata/openreview repository. The change was not taken upstream, so a reader installing the official distribution will not find these variables. Take the code from that branch before starting.
The size of the change: three variables, no code
What the branch adds fits in one sentence: model calls can be routed at any OpenAI-compatible endpoint. There is no new provider class, no adapter and no configuration file. The moment MODEL_BASE_URL is set, every call goes to the chat completions path of that address.
All three variables are optional and carry independent jobs. MODEL_BASE_URL decides the route, MODEL_API_KEY carries authentication for it, and MODEL selects the row that performs the review. The key being optional is deliberate: a local server that requires no authentication works with the base URL alone. LLMTR is not in that group, since its requests are answered against a key in the authorization header.
On the deployment side these values are added to the project environment. Because the value has to be visible to the running build, confirm after adding it that the live version reads that environment.
| Variable | Required | Effect |
|---|---|---|
| MODEL_BASE_URL | No, but setting it takes the default gateway out of the path | Every model call goes to the chat completions path of this address |
| MODEL_API_KEY | No, needed when the endpoint authenticates | Fills the authorization header on requests to the base URL |
| MODEL | No, defaults to anthropic/claude-sonnet-4.6 | Selects the model id that performs the review |
| ANTHROPIC_API_KEY | Still listed in the documentation | The branch does not say it can be dropped; not verified in this version |
Configuration: base URL, key and model id
On the LLMTR side the base URL is given with its version suffix, because the client appends its paths onto that value. Keys carry a fixed prefix and go straight into MODEL_API_KEY. The repository documentation uses zai/glm-5.3 as its example model id; we confirmed that row is live in the public catalogue and carries a chat completions binding.
The three lines below are the entire configuration. Nothing else is touched, and the agent still asks for a model through the same call.
Remember that the id string is passed through as written. The MODEL value has to exist in the target catalogue exactly as spelled; a small difference becomes a model-not-found error at run time.
The configuration that replaces the default gateway
MODEL_BASE_URL=https://llmtr.com/v1
MODEL_API_KEY=llmtr-your_key
MODEL=zai/glm-5.3Choosing a model id and checking its endpoint binding
The default id anthropic/claude-sonnet-4.6 is live in the public catalogue under the same spelling. So a first move can leave MODEL untouched and change only the base URL and the key, deferring model choice to a separate decision.
When you do change it, check two things. First that the row is live in the catalogue: rows marked retired are not a starting point for a new install, so move to the successor. Second the endpoint binding. OpenReview posts to the chat completions path, which works for a row carrying a chat completions binding. The direction with no bridge is the other one, where only rows with a Responses binding can be reached through the Responses path.
The two commands below verify the swap in one pass: that the id exists in the catalogue exactly, and that the same key produces a real answer. The second is a short request, so it reports back without waiting on a review run.
Verifying the id and a one-request smoke test
# 1) Does the id exist in the catalogue exactly (no key needed)
curl -s https://llmtr.com/v1/models \
| jq -r '.data[].id' | grep -x 'zai/glm-5.3'
# 2) Does the same key produce a real answer
curl -s https://llmtr.com/v1/chat/completions \
-H "Authorization: Bearer llmtr-your_key" \
-H "Content-Type: application/json" \
-d '{"model":"zai/glm-5.3","max_tokens":16,"messages":[{"role":"user","content":"ping"}]}'Managing the cost of a review per pull request
A code review is not one question. The agent explores the repository, reads the relevant files, loads skill instructions when they apply and then writes its comments. The repository documentation states this plainly: a review can spend a large number of tokens exploring a repository, which is why self-hosted deployments may prefer a lower-cost model than the default.
Because charging is token based, the cost of a pull request is the product of three things: how much context the review reads, how long the written output is, and the rate of the selected row. The first two follow from the repository and the change itself; the only input an operator can alter without writing code is the third, MODEL. That makes model choice as much a budget decision as a quality one in this integration.
A workable arrangement looks like this: one of the catalogue's free rows for installation and smoke tests, for instance qwen/qwen3.6-27b-free; a smaller row with a chat completions binding for day-to-day reviews; and a return to the default family on critical repositories. Moving between them means editing MODEL and nothing else. On the spend side, calls stop when the credit balance runs out, so the balance is the first place to look as review volume grows.
- Token spend per review grows with how much of the repository is explored.
- The only lever that needs no code change is the MODEL value.
- Free catalogue rows are usable for smoke tests during setup.
- Model rates carry no added margin; LLMTR's 8 percent margin applies only to credit top-ups.
Verification and the failures you will meet
After the swap, opening a single pull request and watching one review through is the fastest check. If a comment arrives, the route, the key and the model id are working together. If it does not, the table below narrows down which layer stopped.
The causes listed come from behaviour the branch documents and from the endpoint contract on the LLMTR side; they are mappings, not measurements.
| Symptom | Likely cause | Check |
|---|---|---|
| Calls still go to the default gateway | MODEL_BASE_URL is missing from the running deployment's environment | Confirm the variable was added and that the live build reads it |
| Authorization fails with 401 | MODEL_API_KEY is empty or lacks the expected prefix | Enter the key with its prefix and exercise the header with the smoke test |
| A model-not-found error | The MODEL value does not exist in the target catalogue verbatim | Search the listing for an exact match; move from a retired row to its successor |
| The request path cannot be found | The base URL was given without its version suffix | Keep the suffix on the address, since the client appends paths onto it |
| A review starts and then stops midway | The credit balance may be exhausted | Check the balance and retry with a smaller model if needed |
Frequently asked questions
Is this option available in the official OpenReview release?
No. The variables described here live on the openai-compatible-gateway branch of the knowhycodata/openreview repository, and the change was not taken upstream. A reader installing the official distribution will not see these fields, so the code has to come from that branch.
Is ANTHROPIC_API_KEY still needed once MODEL_BASE_URL is set?
The repository's environment variable table continues to list it, and the branch does not say it can be dropped when routing is in place. That is not verified in this version, so test it on your own deployment before removing the field.
Does changing the model require writing code?
No. The reviewing row is selected through the MODEL variable, and routing stays inside lib/model.ts. The agent asks for a model through one resolve call and does not know which gateway answered the request.
Can I point it at an endpoint that needs no key?
Yes, because MODEL_API_KEY is optional a local server without authentication works with the base URL alone. That does not apply to LLMTR, where requests must carry a key in the authorization header.