If you take someone’s recipe and make it cheaper, faster, and just as good … are you a genius or a thief? 🤔
Replace recipe with AI model and now we are talking about AI distillation!
AI distillation is where models learn from bigger, more expensive models … with or without permission. Sound familiar with the latest buzz around DeepSeek and OpenAI?
Think of AI distillation like this …
Imagine a world-class chef, let’s call him GPT-o1 spending years perfecting a complex dish. Then, a sharp-eyed sous-chef, let’s call him DeepSeek comes in, tastes it, takes some notes, and recreates a simplified but still delicious version at a fraction of the cost. Innovative? Or intellectual property theft with extra steps?
Distillation is a common practice in AI, it’s how we get smaller, more efficient models from bigger ones … you know like the ‘mini’ models. But when you don’t own the big model and still use it as a teacher … well, thats when things can get a little murky.
On one hand, for consumers, it democratizes AI, making powerful models cheaper and more accessible.
On the other hand, it raises ethical and morality questions … should companies be allowed to “learn from” competitors without permission?
If OpenAI did the hard work, is DeepSeek just “cleverly borrowing” or straight-up pulling a Napster (Google can help understand that one for my less aged friends)?
Keep in mind, there is alway the hypocritical viewpoint when we consider the rocky roads foundational model creators like OpenAI are facing around their training data and the rights available for its use.
