The hardware tiers (the numbers that matter)
First rule: the cost depends on the model you want to run. Anyone selling you "local AI" as a single package without asking what you will do with it is not telling you everything. Realistic tiers:
| Setup | What it handles | Indicative cost |
|---|---|---|
| Used PC, 16 GB RAM, no GPU | 3–4B models: trials, tests, light loads | €300–600 |
| PC with a used 8–12 GB GPU | 7–8B models running smoothly: typical office use | €700–1,200 |
| Workstation with a 24 GB GPU | 13–30B models: higher quality, more users | €2,000–4,000 |
| Dedicated EU server (rented) | Continuous loads, no capital invested | from ~€50–150/month |
These are market tiers for orientation, not quotes: for the numbers on YOUR favourite model, the Local AI calculator estimates the required memory and hardware and compares buy-vs-API in one shot.
The lines almost everyone forgets
- Electricity. A GPU system draws a few hundred watts under load: over a year it is a real line item, not negligible for continuous workloads.
- Management time. Model and tool updates, restarts, answer-quality checks: if nobody budgets this, the system dies quietly.
- The fallback. Sooner or later you need support: knowing who to call is cheaper than standing still.
The honest comparison with APIs
The alternative to local is paying per call to cloud models. The math is simple: with few users and low volume, APIs cost less than any hardware and need no management. Local starts winning when volume grows, when the data must not leave, or when privacy is non-negotiable — and at that point the saving per call is only half the story: the other half is that your documents stay yours.
Rule of thumb: below ~50-100 questions per day, consider EU APIs. Above that, or with sensitive data, local pays for itself.
And when you want to manage nothing
Third way: managed private AI — open-weight models on European infrastructure or your own servers, sized and maintained by someone else. That is what Altovar's Private AI plan offers: costs become a predictable subscription, and the initial assessment includes an estimate for your request volume.
Frequently asked
Is a used GPU worth buying?
Often yes to get started: used 8–12 GB cards have excellent price/performance for 7–8B models. Check memory and condition before buying.
How much electricity per month?
It depends on usage: intermittent office use means a few euros a month; continuous heavy GPU loads grow the bill and belong in the buy-vs-API math.
Is there a cost for the models?
No: open-weight models (Qwen, Llama, Mistral and the like) download and run freely. The cost is hardware, energy and management.