A distilled AI model is a smaller “student” model trained to copy the behavior of a larger “teacher” model. Instead of learning only from raw text, the student learns from the teacher’s answers, so it picks up much of the teacher’s skill in a fraction of the size. That’s why distilled models are popular for laptops and phones: they’re small enough to run locally and noticeably smarter than a same-sized model trained from scratch.
The name you’ve probably seen is DeepSeek-R1-Distill, a set of small models taught to reason like DeepSeek’s big R1 model. Here’s how distillation works, how it differs from the other ways models get smaller, and what a distilled model can and can’t do.
Where does the idea come from? #
The term comes from a 2015 paper by Geoffrey Hinton, Oriol Vinyals and Jeff Dean, Distilling the Knowledge in a Neural Network. They showed that the knowledge in a large, expensive model (or a group of models) could be compressed into a single smaller model that’s much easier to deploy.
The insight is that a teacher’s answers carry more information than the “correct answer” alone. When a big model reads “The capital of Australia is” and predicts the next word, it doesn’t just pick “Canberra”. It assigns probabilities to every possible word: high for Canberra, lower but not zero for Sydney, near zero for Paris. Those probabilities reveal how the teacher thinks about the problem, including which wrong answers are nearly right. A student trained to match that whole distribution learns faster and generalizes better than one trained only on right answers.
How does distillation work for language models? #
There are two common approaches today:
- Matching the teacher’s probabilities. The student sees the same text as the teacher and is trained to predict the same probability for each next token. This needs access to the teacher model itself. Google used this approach for the smaller Gemma 2 models: its technical report says the 2B and 9B versions were trained with knowledge distillation instead of plain next-token prediction.
- Training on the teacher’s outputs. The teacher writes a large set of example answers, and the student is fine-tuned on them. This only needs the teacher’s text, not its internals. DeepSeek made its R1-Distill models this way.
Either way, the student ends up imitating the teacher’s style and skills, within the limits of its smaller size.
The DeepSeek R1 distilled models #
When DeepSeek released its R1 reasoning model in January 2025, it also released six smaller distilled models. According to the official model card, they were built by fine-tuning existing open models on 800,000 samples curated with DeepSeek-R1:
| Distilled model | Built on |
|---|---|
| DeepSeek-R1-Distill-Qwen-1.5B | Qwen2.5-Math-1.5B |
| DeepSeek-R1-Distill-Qwen-7B | Qwen2.5-Math-7B |
| DeepSeek-R1-Distill-Llama-8B | Llama-3.1-8B |
| DeepSeek-R1-Distill-Qwen-14B | Qwen2.5-14B |
| DeepSeek-R1-Distill-Qwen-32B | Qwen2.5-32B |
| DeepSeek-R1-Distill-Llama-70B | Llama-3.3-70B-Instruct |
Two things follow from that table. First, a “DeepSeek R1” that runs on a phone or a normal laptop is almost always one of these distills, not the full R1, which is far too large. Second, each distill is really a Qwen or Llama model that has been taught R1’s step-by-step reasoning style. The 1.5B and 7B versions started from math-focused models, so they’re strongest at math and weaker at general conversation.
Distillation vs quantization vs pruning #
People often mix these up, because all three make models smaller or cheaper to run. They work very differently.
| Technique | What it does | Who does it | Effect on the model |
|---|---|---|---|
| Distillation | Trains a new, smaller model to imitate a bigger one | The model maker, with a lot of computing power | A different, smaller model with fewer parameters |
| Quantization | Stores the same model’s numbers with fewer bits (for example 4 bits instead of 16) | Often the community, after release | Same model, smaller file, slight quality loss |
| Pruning | Removes parts of a model judged unimportant | The model maker or researchers | Same model with pieces cut out, usually retrained afterwards |
These stack. A model can be distilled from a big teacher, then quantized into a GGUF file so it fits in a phone’s memory. Most models you run on a phone have been through at least quantization. Our guide to GGUF quantization explains names like Q4_K_M and Q8_0.
Is a distilled model as good as the original? #
No, and it isn’t meant to be. A distilled model is a trade: much smaller and faster, somewhat less capable.
What students tend to keep well:
- the teacher’s style and format, such as reasoning step by step before answering
- skills that were well represented in the training examples, such as math for the R1 distills
What they tend to lose:
- breadth of knowledge. A 1.5B model simply can’t store as many facts as a model with hundreds of billions of parameters, however well it’s taught
- reliability on unusual or complex questions
- skills the teacher’s examples didn’t cover
In practice, a good distilled model often beats a same-sized model trained the ordinary way, but it won’t beat a model several times its size. For why size matters so much on a phone, see what makes an AI model suitable for on-device use.
Can you run a distilled model on your phone? #
Yes, if it’s small enough. A 1.5B or 7B distill converted to GGUF and quantized to 4 bits can run on many recent phones, the 7B and 8B ones only on phones with plenty of RAM.
Personal LLM runs GGUF models entirely on iPhone and Android, with no account and no connection after the download. Its catalog focuses on current small models built for phones, such as Qwen 3.5 (0.8B, 4B and 9B) and Gemma 4 E2B and E4B, and it also lets you add any GGUF model by pasting its Hugging Face link. That’s how you’d try a DeepSeek R1 distill or any other distilled model on your phone. The app shows each model’s speed in tokens per second, so you can compare a distill against the catalog models on your own hardware. Our guide to adding a custom GGUF model covers finding the right file.
Check the license before you use a distill for work. The DeepSeek card says the Qwen-based distills carry Apache 2.0 from their base models, while the Llama-based ones follow the Llama 3.1 and 3.3 licenses.
How can you tell if a model is distilled? #
- The name. Many include “Distill” or “distilled”.
- The model card. Makers usually say how a model was trained, including which teacher was used.
- The base model. If a model card says “fine-tuned from Qwen2.5 on outputs of a larger model”, that’s distillation in all but name.
Not every small model is distilled, and not every distilled model advertises it. The name matters less than how the model performs on the tasks you actually care about.
Frequently asked questions #
What does “distilled” mean in AI? #
It means a smaller model was trained to imitate a larger one, by learning from the larger model’s outputs or probabilities instead of only from raw data. The result keeps much of the teacher’s ability at a much smaller size.
Is DeepSeek R1 a distilled model? #
The full DeepSeek-R1 isn’t; it’s the large teacher. The DeepSeek-R1-Distill models are the distilled versions: Qwen and Llama models from 1.5B to 70B parameters fine-tuned on 800,000 samples curated with R1.
Are distilled models worse? #
They’re less capable than their teacher, especially on broad knowledge and hard problems, but usually better than a same-sized model trained the ordinary way. Whether one is good enough depends on your task and your hardware.
Is distillation the same as quantization? #
No. Distillation trains a new, smaller model. Quantization stores an existing model’s numbers at lower precision so the file is smaller. A model you run on a phone is often both distilled and quantized.