First rule: the tier beats the brand
A ”better” model that does not fit your memory is a useless model: it crawls or does not run at all. The sensible choice starts from the tier — how much memory you can field — and only then the model. Here are the tiers, with concrete open-weight models for each.
Light tier (up to ~8 billion parameters)
The office and normal-PC tier. Surprising quality for summaries, extraction, drafts and light Q&A.
- Llama 3.1 8B — the dependable classic: wide tool ecosystem, predictable behaviour.
- Qwen3 8B — excellent on European languages and reasoning in its class.
- Gemma 3 12B — slightly larger, still runs on 16 GB: the best quality/load ratio of this tier.
Hardware: 16 GB RAM even without a GPU, better with 8 GB of VRAM.
Mid tier (~20-32 billion)
Where local AI stops being an experiment: work-grade quality on documents, serious RAG, more concurrent users.
- Gemma 3 27B and Mistral Small 24B — the European workhorses of the tier.
- Qwen3 32B — the strongest of the tier on structured reasoning.
- gpt-oss 20B — designed to run locally: compact for its class.
Hardware: a GPU with 16-24 GB of VRAM or a workstation with 32 GB. The calculator has the combinations already sized.
High tier (70 billion and beyond)
When quality must approach the frontier models and the hardware is a server, not a PC.
- Llama 3.3 70B — the tier's reference point, with quality that a year ago came only via API.
- gpt-oss 120B — the efficient route into the high tier: more quality per GB than many competitors.
- Qwen3 235B (MoE), DeepSeek V3, Llama 3.1 405B — infrastructure territory: they make sense in a company with serious sizing, not on the desktop.
How to choose in three questions
- What hardware do you have (or want to buy)? → it dictates the tier. Not the other way around.
- What will you do with it? Q&A over documents and drafts → light/mid tier is enough. Long analysis and agents → mid/high.
- In which languages? → in our experience Qwen3 and Gemma 3 are the most solid across European languages in the lower and mid tiers.
For installation steps there is the complete guide; for memory estimates and the buy-vs-API comparison, the calculator; and if you want the high tier without buying the server, there is the private AI plan on EU infrastructure.
Frequently asked
Are these models free?
Yes: they are open-weight under permissive licences (the Llama, Gemma, Mistral, Qwen, gpt-oss families). The cost is hardware and management, not the model.
How often should they be replaced?
Families refresh every few months. The right strategy is staying in the tier that fits your hardware and updating the model within it, without chasing every release.
Which one for questions over my documents?
With a well-built RAG, the mid tier (24-32B) is the sweet spot for company documents: sufficient quality and reasonable hardware costs. The light tier works for small archives.