RAG and data ยท 2026-09-22

Ranking sources in a RAG pipeline with Muse Spark 1.2

After splitting the context budget, this covers what order to feed multiple retrieved document chunks to the model in, and how to keep the most relevant source from getting lost.

RAG source ranking diagram showing multiple retrieved document chunks ordered by relevance, with the most relevant one placed closest to the model.

Once the budget is split, the ordering question starts

As the Muse Spark 1.2 long-context guide explains, splitting context into input, output, a safety margin, and a total limit determines how much source text you can send. But once you fit multiple document chunks into that input budget (say, 8-10 chunks returned from a vector search), what order to give them in requires a separate decision.

Ordering can seem trivial, but it's a known behavior that a model doesn't attend equally to every region of context; information near the start or end of context can be used more consistently than information stuck in the middle.

Don't lose the most relevant source in the middle

Rather than giving results from a vector search in descending relevance order (most relevant first), placing the top two or three most relevant results at the start and end of context, with less relevant ones in the middle, reduces the risk of the model missing the most important information. This is a practical countermeasure against the behavior known in RAG literature as 'lost in the middle.'

Adding a rerank step (reordering embedding search's initial results with a separate model) lets you decide more reliably which chunks go into context; this produces a more accurate result than leaving the ordering decision to raw embedding similarity scores alone.

  • Place the most relevant results at the start and end of context, not buried in the middle.
  • Build your ordering strategy with the 'lost in the middle' behavior in mind.
  • A rerank step improves the decision of which chunks go into context.

Test ordering separately from the budget calculation

Even when the context budget is calculated correctly, a poorly ordered context can make it harder for the model to find the right answer. Testing the two variables (budget and ordering) separately makes it easier to tell which one is responsible when a problem comes up.

Frequently asked questions

Should I always order source chunks by relevance score?

Raw relevance score is a starting point, but placing the most relevant results at the start and end of context compensates for the model's tendency to use information stuck in the middle less consistently.

Can I work with embedding ranking alone, without a rerank step?

Often enough for small sets with few sources. As source count grows, or when results carry close scores, a rerank step increases the reliability of the ordering decision.

Related posts