Required for the site to work (login, preferences, security). Always on.
Which AI model can your hardware run?
Pick a model, a quantization level and how many people will use it at once — get an honest memory estimate, which GPUs or Macs actually fit, and whether buying hardware beats paying for API tokens. Nothing here is sent to a server: it all runs in your browser.
1. Configure
- Qwen3.6-35B-A3B
- Gemma 4 26B-A4B
- Qwen3.8-27B
- Gemma 4 31B
- Mistral Small 4
- Qwen3.5-122B-A10B
- Step-3.7-Flash
- DeepSeek V4-Flash
- MiMo-V2.6-Flash
- GLM-5.3-Flash
- Qwen3.5-397B-A17B
- Mistral Large 3
- Custom size…
Dense-model assumption — for your own fine-tune or a model not in the list.
2. Memory needed
100% of RTX 4060 Ti 16GB usable memory used
How we calculate this
Weights ≈ parameters × bits per weight ÷ 8 × 1.1 (a 10% allowance for runtime buffers and activations).
KV cache = bytes per token × context length × users. For the listed models the bytes per token come from the model's own configuration (only the layers that cache every token, at 16-bit), so models with compressed attention need far less; for a custom size we approximate it from a typical grouped-query architecture.
These are estimates, not guarantees. Real usage varies with the inference engine, batching strategy and prompt shape.
3. What fits
Memory fit only — we never invent tokens-per-second numbers we can't measure.
| Hardware | Memory | Fit |
|---|---|---|
| RTX 4060 Ti 16GB ★ | 16 GB | Tight |
| RTX 4090 24GB | 24 GB | Fits |
| RTX 5090 32GB | 32 GB | Fits |
| 2× RTX 4090 (48GB) | 48 GB | Fits |
| RTX 6000-class 48GB | 48 GB | Fits |
| RTX 6000-class 96GB | 96 GB | Fits |
| Apple M-series 32GB | 32 GB | Fits |
| Apple M-series 64GB | 64 GB | Fits |
| Apple M-series 128GB | 128 GB | Fits |
| NVIDIA DGX Spark-class 128GB | 128 GB | Fits |
| 2x DGX Spark (256 GB) | 256 GB | Fits |
| 4x DGX Spark (512 GB) | 512 GB | Fits |
"Fits" leaves headroom for the OS and other apps. "Tight" means it's within 15% over that headroom — a smaller context or heavier quantization would close the gap.
4. Buy hardware or pay per token?
Edit any number below — these are starting points, not our prices.
Amortized over 36 months
Assumptions
- Memory estimates assume the whole model and its KV cache sit in VRAM or unified memory at once — no offloading to system RAM.
- The 1.1× overhead factor is a rough allowance for runtime buffers and activation memory, not a measured figure for any specific inference engine.
- KV-cache size per token is taken from each listed model's own configuration, and approximated from a typical architecture for a custom size — see sources.
- Quantization below Q8 changes model behavior, not just file size — always check quality for your own use case.
- Hardware fit compares required memory to a fixed usable-memory assumption per device; actual headroom depends on your OS, drivers and what else is running.
- The cost comparison ignores maintenance, depreciation beyond 36 months, and the value of your own time.
Sources
Parameter counts are checked against each model's official card, technical report or Hugging Face configuration:
- Qwen3.6-35B-A3B — Hugging Face (Alibaba)
- Gemma 4 26B-A4B — Hugging Face (Google)
- Qwen3.8-27B — Hugging Face (Alibaba)
- Gemma 4 31B — Hugging Face (Google)
- Mistral Small 4 — Hugging Face (Mistral AI)
- Qwen3.5-122B-A10B — Hugging Face (Alibaba)
- Step-3.7-Flash — Hugging Face (StepFun)
- DeepSeek V4-Flash — Hugging Face (DeepSeek)
- MiMo-V2.6-Flash — Hugging Face (Xiaomi)
- GLM-5.3-Flash — Hugging Face (Z.ai)
- Qwen3.5-397B-A17B — Hugging Face (Alibaba)
- Mistral Large 3 — Hugging Face (Mistral AI)
Need this running in production, not just estimated?
Altovar deploys sovereign, local AI for European businesses — on your own hardware or in an EU data center, with no data leaving your infrastructure. See how Altovar deploys local AI →
New to running models locally? 3 repos every SME should know for local AI →