Integration guides ยท 2026-08-28
Ling-3.0-Tiny download guide: choosing BF16, FP8 or INT4
Compare the official Ling-3.0-Tiny BF16, FP8 and INT4 repositories, check download sizes, pin a revision and prepare the files for local inference.
Where can you download Ling-3.0-Tiny?
Ling-3.0-Tiny weights are available from InclusionAI's official BF16, FP8 and INT4 repositories on Hugging Face, all labeled MIT. These are weights, configuration and tokenization files, not a chat application. Generating answers requires a compatible runtime and suitable hardware.
Choose a supported package before downloading. The smallest files are not guaranteed to run on every computer or generate answers fastest.
How should you choose between BF16, FP8 and INT4?
These approximate repository sizes appeared on official file pages on August 28, 2026. They are not RAM or VRAM requirements and can change with revisions. BF16 means 16-bit floating-point representation, FP8 8-bit floating point and INT4 4-bit integers. A package may contain tensors with different types.
Quantization aims to reduce weight memory by changing numerical precision. We suggest identifying supported formats first, then comparing quality and resource use on your task. The table is not a performance ranking.
| Format | Hugging Face repository ID | Approximate repository size | Selection criterion |
|---|---|---|---|
| BF16 | inclusionAI/Ling-3.0-tiny | 15.8 GB | A comparison baseline if your resource budget permits |
| FP8 | inclusionAI/Ling-3.0-tiny-fp8 | 8.42 GB | Runtime and hardware support for this FP8 package |
| INT4 | inclusionAI/Ling-3.0-tiny-int4 | 5.82 GB | Explicit support for its quantization format and task quality checks |
Disk space and working memory are different budgets
Tiny has 7.9 billion total parameters and approximately 1.3 billion active per token. The repository contains the complete model; active parameters do not determine download size.
Inference also needs intermediate computations, context state and runtime allocations. Concurrency and context length affect memory usage. File size therefore provides no universal minimum RAM figure. Leave disk space for the runtime environment, temporary files and additional revisions.
Preview the files before downloading
In a Bash session with Hugging Face CLI installed, this example previews the FP8 download without transferring weights. Set the LING_REVISION environment variable to the full commit ID copied from your selected repository's Files and versions section. An empty variable stops the command. FP8 is an example, not a hardware recommendation.
- Check that inclusionAI owns the repository and that its name matches your selected format.
- Record a full commit ID instead of the moving main branch so your team uses the same files.
- Use separate destination folders for formats and revisions; do not mix their files.
Download preview only; requires Hugging Face CLI
REPO="inclusionAI/Ling-3.0-tiny-fp8"
DEST="./models/ling-3.0-tiny-fp8"
: "${LING_REVISION:?Set LING_REVISION to the selected full commit SHA}"
hf download "$REPO" --revision "$LING_REVISION" --local-dir "$DEST" --dry-runDownload the package and check its integrity
After checking size and compatibility, use these commands in the same Bash session. The first transfers actual files and consumes bandwidth. The second compares local files with Hub checksums. Neither command was executed while preparing this article.
For an initial setup, do not select just one safetensors shard. Keep all weight shards together with the index, configuration, tokenizer and chat template. Resolve missing or mismatched files before proceeding to inference.
Download and verify the complete package after approving the transfer
hf download "$REPO" --revision "$LING_REVISION" --local-dir "$DEST"
hf cache verify "$REPO" --revision "$LING_REVISION" --local-dir "$DEST" --fail-on-missing-filesA completed download is not a running model
Correct file checksums do not establish runtime support for Ling's architecture or your chosen quantization. The INT4 card's generic launch example uses the BF16 repository ID; that example alone does not establish INT4 compatibility. Check runtime instructions for the specific package and version separately.
The repositories also contain custom Python code. If launch instructions require trusting remote code, inspect that code and your pinned revision first. Start in a separate environment without secrets and use a short sample. Loading successfully does not establish adequate quality for your actual workload; evaluate it with your own checks.
LLMTR API retirement does not prevent downloading
The LLMTR API ID inclusionai/ling-3.0-tiny was retired on August 15, 2026 and returns model_retired. This does not mean the official Tiny weights were removed. Downloading and running them locally are separate ways to use the model.
LLMTR's designated successor for that retired ID is the paid inclusionai/ling-3.0-flash model. Downloading weights neither uploads a model to LLMTR nor reactivates the old API ID. Assess hardware, electricity and operating costs for local inference separately.
Frequently asked questions
Can I convert BF16 by renaming its files to FP8?
No. Renaming files or folders does not change tensor representations. Choose the official FP8 package or follow a separate conversion process documented by your runtime; renaming does not create compatibility.
Is INT4 always faster?
No. Smaller weights alone do not guarantee faster inference. Hardware, runtime support and workload determine the result. This guide includes no speed or accuracy measurements comparing the three formats.
Do I need to download all three packages?
No. Start with one package your environment supports. For comparisons, store other formats separately and evaluate them with the same examples, thinking setting and output budget.
Do downloads require an LLMTR API key?
No. These commands fetch Hugging Face repository files, not LLMTR API responses. If Hugging Face authentication is needed, use that service's own credentials and do not send your LLMTR key.