The one number that decides whether an AI model runs on a graphics card is its memory (VRAM). Speed only decides how fast. This page gives the rule of thumb, then the numbers for chat models, image models and training — and which card to rent for each.
The rule of thumb
A model's memory is mostly its weights:
- Full precision (16-bit): about 2 GB per billion parameters.
- 8-bit: about 1 GB per billion.
- 4-bit (Q4, the usual choice): about 0.6 GB per billion.
Then add 10–30% for the conversation (the "context" or KV cache) — more for long documents, less for short chats. So a 7B model needs about 14 GB at 16-bit, but only about 5 GB in 4-bit. 4-bit models lose very little quality for chat, which is why tools like Ollama default to them.
Chat models (LLMs)
| Parameters | 4-bit | 8-bit | 16-bit | Cheapest card that fits (4-bit) |
|---|---|---|---|---|
| 3 B | ~2.5 GB | ~4 GB | ~7 GB | any 8 GB card |
| 7–8 B | ~5 GB | ~9 GB | ~16 GB | RTX 3060 12 GB · about $0.14/h |
| 14 B | ~9 GB | ~16 GB | ~30 GB | 12 GB cards · RTX 4070 about $0.21/h |
| 20–24 B | ~14 GB | ~25 GB | ~48 GB | 16 GB cards · RTX 4060 Ti 16 GB about $0.18/h |
| 32 B | ~20 GB | ~35 GB | ~66 GB | 24 GB cards · RTX 3090 about $0.31/h |
| 70 B | ~42 GB | ~75 GB | ~145 GB | 48 GB cards · RTX 6000 Ada about $0.56/h |
| 120 B | ~70 GB | ~125 GB | ~240 GB | 80 GB cards · A100 80 GB about $1.05/h |
| 400 B+ | ~240 GB+ | — | — | several 80 GB cards together |
"MoE" models (mixture of experts, such as gpt-oss or DeepSeek's larger models) need memory for all their parameters even though only some are used per word — size them by their total count.
Long contexts cost memory. A 32B model that fits a 24 GB card for chat may run out with a 100-page document in the prompt. If you plan long inputs, take the next size up.
Image and video models
| Model | Comfortable | Minimum (with tricks) |
|---|---|---|
| Stable Diffusion 1.5 | 8 GB | 4 GB |
| SDXL | 12 GB | 8 GB |
| Flux.1 (dev / schnell) | 24 GB | 12 GB with FP8, ~8 GB with 4-bit |
| Video models (Wan, HunyuanVideo, LTX) | 24–48 GB | 16 GB, slowly |
For images, a 24 GB card such as the RTX 4090 ($0.49/h) or RTX 3090 ($0.31/h) runs everything popular at full quality. More in Make AI images on a rented GPU.
Training and fine-tuning
Training needs far more memory than running a model, because the optimizer keeps extra copies of every weight.
| What | 7–8 B model | 13–14 B | 70 B |
|---|---|---|---|
| QLoRA (4-bit base + small adapters) | ~10–16 GB | ~18–24 GB | ~48–80 GB |
| LoRA (16-bit base) | ~20–24 GB | ~35–40 GB | 2+ × 80 GB |
| Full fine-tune | ~80–120 GB | ~160 GB+ | many 80 GB cards |
QLoRA on a single 24 GB card is how most people fine-tune a 7–8B model today. See Fine-tune an LLM on a rented GPU.
When one card is not enough
Rent several machines together: a cluster gives them a private network between each other, so a model or a training run can be spread over them (vLLM, llama.cpp's RPC mode, PyTorch distributed). For very large jobs that split into independent pieces — embeddings, evaluation, rendering, data processing — a batch spreads up to 100,000 tasks over every free machine at once, through one API call.
FAQ
Does a faster GPU help if the model does not fit?
No. If a model does not fit, it either refuses to load or spills onto the computer's main memory, which makes it many times slower. Fit first, then speed.
Is 12 GB enough for AI in 2026?
For chat with 7–14B models and for SDXL images, yes. For 32B chat models and Flux at full quality, 24 GB is the comfortable size.
Does system RAM matter?
A little: the model is loaded through it, so have at least as much RAM as the model file. Every listing here shows the machine's RAM, processor cores and free disk, and you can filter by them.