Integration guides ยท 2026-09-22

Embeddings and rerank on the OpenAI-compatible gateway

How to connect the embeddings and rerank endpoints, beyond chat completions, to an OpenAI-compatible AI gateway, and when each one is actually needed.

Gateway architecture diagram showing the same base URL and API key connecting to chat completions, embeddings, and rerank endpoints at once.

Migration doesn't end at chat completions

The migration steps described in the OpenAI-compatible AI gateway guide focus on moving an existing chat integration by changing the base URL and model identifier. But if an app uses not only chat but a semantic layer such as search or recommendations, you can also call the `/v1/embeddings` endpoint on the same gateway; authentication and the basic request shape stay the same.

The rerank endpoint answers a different need: after embedding search returns a broad candidate set, a separate model is needed to reorder those candidates against the query. The two endpoints don't replace each other; they are used in sequence.

Which endpoint for which scenario

If you're building a purely chat-based assistant, the chat completions endpoint is enough. If you're building a semantic search layer for document search, product recommendations, or FAQ matching, you first convert your documents to vectors with the embeddings endpoint and store them in a vector database, then use the same endpoint at query time to convert the query itself.

If you want to improve search hit rate, sending the top N results from embedding search to the rerank endpoint for reordering produces a noticeable improvement, especially in sets containing many similar documents.

  • Chat completions: conversation and text generation.
  • Embeddings: document/query vectorization, the foundation of semantic search and RAG.
  • Rerank: reordering embedding search's initial results by relevance.

You can pick a different model from the same catalog

The model you use for chat does not have to come from the same provider as the model you use for embeddings or rerank. Every model in the catalog is listed with its own price and capability set; choosing three separate models for the three endpoints builds a more flexible architecture than being tied to a single provider.

Frequently asked questions

Do I need a separate API key to use the embeddings endpoint?

No. The same key and base URL you use for chat completions also work for the embeddings and rerank endpoints. Only the request path and body shape change.

Is embedding search alone enough without reranking?

Often enough for small document sets. As document count grows, or when queries match many documents with similar meaning, a rerank step noticeably improves hit rate.

Related posts