Model comparison · 2026-08-28
Ling 3.0 Tiny architecture: total versus active parameters
Understand Ling 3.0 Tiny's 7.9B total and 1.3B active parameters, MoE expert selection, and KDA–MLA architecture without confusing computation with memory.
How many parameters does Ling 3.0 Tiny have?
Ling 3.0 Tiny is an InclusionAI MoE language model with approximately 7.9 billion total parameters; its official model card reports approximately 1.3 billion active per token. Both descriptions are valid: one counts all model weights, the other counts those participating in a token's computation.
The smaller active count does not mean the remaining weights have been removed. Different tokens can use different experts. Calling Tiny a 1.3B dense model misses this distinction, as does equating it with a 7.9B model using all its weights for every token.
How does MoE select experts for a token?
MoE means mixture of experts. These experts are feed-forward neural network components, not separate chatbots. A router computes selection scores from the token's representation at that layer. Selected expert outputs are combined using weights, and the shared expert's contribution is added. Users do not need to choose an expert by name.
| Architecture measure | Ling 3.0 Tiny | How to interpret it |
|---|---|---|
| Total parameters | Approximately 7.9 billion | All the model's weights |
| Active parameters per token | Approximately 1.3 billion | The computation path for one token |
| Routed experts in an MoE layer | 128 | The pool from which experts are selected |
| Experts selected per token | 8 routed + 1 shared | Not every expert processes the same token |
| Attention layer arrangement | 3 KDA + 1 MLA | The arrangement within each four-layer attention block |
Why combine KDA and MLA?
Tiny combines three Kimi Delta Attention layers with one Multi-head Latent Attention layer. KDA belongs to the linear attention family; MLA is a full attention approach that uses latent representations. The architecture therefore does not rely on a single attention method to process context. This is separate from MoE routing: attention design and expert selection control different parts of the computation.
The implication is that models with equal total parameter counts can have different computational structures. However, this layer ratio establishes neither a particular speedup nor a document understanding score. Using long text correctly must be evaluated; architecture cannot replace a quality measurement.
1.3B active parameters is not a memory requirement
The active-to-total ratio is approximately 16.5%. This arithmetic does not establish that Tiny will run six times faster than another model. Attention operations, routing, memory transfers, and runtime implementation also contribute to latency. Processing the input and generating new tokens should be measured separately.
Counting only the selected experts is insufficient for memory planning. Weight storage or movement, request states, caches, and runtime allocations all require resources. Concurrent requests must also be considered. Do not derive a minimum RAM or VRAM requirement from the parameter count alone.
Official BF16, FP8, and INT4 weights are available. These are numerical representation options; they do not remove the distinction between total and active parameters. Evaluate how lower precision affects quality and actual working memory in your chosen implementation.
Build the comparison around a real task
If you need to extract a topic, urgency level, and follow-up question from a support message, evaluate those fields. A larger parameter count does not show whether a model handles your labels or Turkish texts better.
The following checklist is an experiment plan, not a benchmark or model response produced for this article.
- Give every candidate the same Turkish and English examples without personal data.
- Record the weight revision, numerical precision, runtime version, and hardware.
- Keep the thinking setting and output budget comparable.
- Score correct extraction, missing information, and unnecessary guesses separately.
- Measure time to first token, completion time, peak memory, and concurrent request load together.
Architecture does not determine API availability
As of 28 August 2026, InclusionAI's official Tiny weight repositories remain accessible and their model cards identify the MIT license. The inclusionai/ling-3.0-tiny API identifier on LLMTR has been retired since 15 August 2026 and is handled with a model_retired error. This does not mean the model weights have disappeared.
LLMTR's designated successor is the paid inclusionai/ling-3.0-flash model. Local Tiny requires a separate hardware and software evaluation. This architecture article does not claim that installation or live API access was tested.
Frequently asked questions
Is Ling 3.0 Tiny a 7.9B or a 1.3B model?
Both figures describe different measures: approximately 7.9B total parameters and approximately 1.3B active per token. Read the model card's total and active counts together.
Why do eight selected experts not mean 6.25% active parameters?
8/128 is only the fraction of routed experts selected. It excludes the shared expert and other model components, so it cannot be equated with the overall active parameter ratio.
Does an MoE expert call a tool or another model?
No. Expert routing is computation inside the neural network. External tool calling is a separate workflow executed by the application.
Can the active count tell me whether Tiny runs on my computer?
No. Weight format, runtime support, available memory, context, and concurrency must be assessed together. No hardware execution test was performed for this article.