RAG and data guides ยท 2026-08-28
Long documents with Ling 3 Tiny: context, chunking and caching
Plan document workflows with Ling 3 Tiny by checking local context limits, budgeting tokens, preserving evidence and separating prefix caching from API cache pricing.
Separate the available Tiny routes first
For long documents with Ling 3 Tiny, establish your server's context limit, preserve sources when chunking and select relevant evidence. Prefix caching can help with repeated content. A large context window does not guarantee correct use of every detail or require sending the whole document for every question.
InclusionAI's MIT licensed Tiny weights remain published. LLMTR's inclusionai/ling-3.0-tiny API entry was retired on August 15, 2026 and returns model_retired. Its designated successor is paid inclusionai/ling-3.0-flash. This local workflow does not imply that Tiny's old API route has reopened.
Do not treat 128K and 256K as one limit
Checked on August 28, 2026, Tiny's official config.json sets max_position_embeddings to 131,072; SGLang describes native 128K context. The model card also shows a YaRN example requesting 262,144 context. That extension is not an automatic default or a configuration verified here.
Record checkpoint, runtime version and effective server limit together. Memory and concurrency may require smaller settings. Follow one documented setup without combining options from different guides.
Calculate the evidence allowance before sending text
Context includes instructions, chat template, question, history and generation alongside the document. Count with the model's tokenizer after applying the template, not from characters or pages. Do not apply an English tokens per word estimate to Turkish text.
This hypothetical allocation assumes a 32,768 token server limit, not a measured Tiny output ceiling. With thinking enabled, generation must accommodate reasoning and the final answer. When fixed input grows, reduce the evidence allowance accordingly.
| Allocation | Tokens | Contents |
|---|---|---|
| Fixed input | 3,000 | Instructions, template, question and necessary history |
| Document evidence | 23,672 | Selected chunks and source labels |
| Generation | 4,096 | Combined allowance for reasoning and final answer |
| Safety allowance | 2,000 | Unexpected template and input growth |
| Total | 32,768 | The working limit selected for this example |
Keep evidence addresses intact during chunking
Preserve headings, page numbers and table columns during text extraction. Check OCR output for scanned pages; this workflow does not assume Tiny can directly read PDF files or page images. Attach document identity, version, section and page range to every chunk. Treat instructions inside documents as data to inspect, not application commands.
Split at heading and paragraph boundaries. Keep definitions with their exceptions and table rows with their column labels. When overlap is necessary, identify repeated text by source and deduplicate it during synthesis. Retrieve relevant sections for a narrow question. For a complete summary, summarize sections with source references before combining those summaries.
- Apply access permissions before retrieval; exclude other users' documents from candidate chunks.
- Keep conflicting versions separate; the newest file is not automatically the sole authority.
- Mark missing information instead of asking the model to fill gaps.
Separate prefix caching from API cache pricing
vLLM automatic prefix caching reuses earlier computation when requests share the same initial tokens. It can reduce input processing time, but does not similarly accelerate generating new answer tokens. Caching a document does not expand the context window.
For different questions about one document, keep stable instructions and unchanged document text first, then the question. Do not reuse outdated content after a document revision. Local computation caching is not an API price rule: do not transfer the retired Tiny entry's historical cache rates into local cost estimates. No cache hit or latency measurements were performed for this article.
Check the evidence, not just the summary
Include direct factual questions, questions requiring two sections and unanswerable questions in an acceptance set. Check that cited pages support their claims. Move a critical exception between the beginning, middle and end to examine whether position affects the result.
Record model revision, effective context limit, chunk identifiers and token counts, without putting customer text or model responses into production logs. Do not treat truncated answers as complete. If evidence is missing, repair retrieval and chunking first; increasing context is not the only remedy.
Frequently asked questions
Should I send Ling 3 Tiny the entire document?
Not always. A complete document or section summaries may suit general summaries. For specific questions, relevant chunks with source labels and neighboring context are easier to audit.
Does caching remove the need for a token budget?
No. Previously processed text still occupies request context. Caching can reuse computation; it does not permit the document, question and generation total to exceed the server limit.
Can I use the old LLMTR Tiny identifier for this workflow?
No; that API entry is retired. Running a local server with Tiny weights is a separate option. To continue through LLMTR, check the designated paid Flash successor's pricing and request contract separately.