Quick answer
What is the best GPU to fine-tune Llama 70B?
The best GPU to fine-tune Llama 70B depends on the method. QLoRA needs about 52.5 GB, so one 80 GB A100 or H100. LoRA on a BF16 base needs about 168 GB, so one 180 GB B200 or two H200s. Full fine-tuning needs 1,120 to 1,260 GB before activations, so eight B200s. As of 29 October 2026, GPUwithCrypto rents an A100 80GB at $0.92/hour, paid in crypto.
Which GPU for which method?
| QLoRA (4-bit base) | ≈ 52.5 GB → 1× A100 80GB ($0.92/hour) or 1× H100 ($2.32/hour) |
|---|---|
| LoRA (BF16 base) | 140 GB floor, ≈ 168 GB to plan → 1× B200 ($4.61/hour) or 2× H200 ($6.14/hour) |
| Full fine-tune, 8-bit Adam | ≈ 840 GB → 8× H200 ($24.56/hour) |
| Full fine-tune, AdamW | 1,120–1,260 GB → 8× B200 ($36.88/hour) |
| Too small | One 48 GB L40S misses the 52.5 GB QLoRA estimate; the QLoRA authors did fit a 65B model on one 48 GB GPU |
How much memory does each method need for a 70B model?
- QLoRA: the base model is loaded in 4-bit and frozen; only small adapters train. The QLoRA paper fine-tuned a 65B model on one 48 GB GPU: 48 ÷ 65 = 0.74 GB per billion parameters, rounded to 0.75. For 70B: 70 × 0.75 = 52.5 GB.
- LoRA: the base stays in BF16 and frozen: 70 × 2 bytes = 140 GB. Adapters and their optimizer add about 1%; with our 20% planning margin, 140 × 1.2 = 168 GB.
- Full fine-tuning, mixed precision with AdamW: EleutherAI counts 2 + 12 + 2 = 16 bytes per parameter (weights, optimizer with FP32 copy, gradients): 70 × 16 = 1,120 GB. Hugging Face counts 6 + 8 + 4 = 18 bytes: 1,260 GB. Activations come on top.
- Full fine-tuning with 8-bit Adam: Hugging Face puts 8-bit optimizer states at 2 bytes instead of 8, so 6 + 2 + 4 = 12 bytes: 70 × 12 = 840 GB.
These are planning rules from the cited sources, not measurements. Sequence length and batch size change the activation memory; gradient checkpointing reduces it. The same rules for other model sizes: how much VRAM do I need?
How many GPUs does each method need?
✓ = the configuration's combined VRAM holds the method's estimate. Multi-GPU rows assume the model is sharded across GPUs (FSDP or DeepSpeed ZeRO-3). Our prices per hour for the whole configuration, 29 October 2026.
| Configuration | Total VRAM | QLoRA (52.5 GB) | LoRA (168 GB) | Full, 8-bit Adam (840 GB) | Full, AdamW (1,260 GB) | GPUwithCrypto per hour |
|---|---|---|---|---|---|---|
| 1× L40S | 48 GB | ✗ | ✗ | ✗ | ✗ | $0.82/hour |
| 1× A100 80GB | 80 GB | ✓ | ✗ | ✗ | ✗ | $0.92/hour |
| 1× H100 | 80 GB | ✓ | ✗ | ✗ | ✗ | $2.32/hour |
| 1× H200 | 141 GB | ✓ | ✗ | ✗ | ✗ | $3.07/hour |
| 1× B200 | 180 GB | ✓ | ✓ | ✗ | ✗ | $4.61/hour |
| 2× H200 | 282 GB | ✓ | ✓ | ✗ | ✗ | $6.14/hour |
| 4× A100 80GB | 320 GB | ✓ | ✓ | ✗ | ✗ | $3.68/hour |
| 8× H100 | 640 GB | ✓ | ✓ | ✗ | ✗ | $18.56/hour |
| 8× H200 | 1,128 GB | ✓ | ✓ | ✓ | ✗ | $24.56/hour |
| 8× B200 | 1,440 GB | ✓ | ✓ | ✓ | ✓ | $36.88/hour |
Prices checked on
8× H200 holds the 16-byte estimate (1,120 GB) on paper but leaves 8 GB for activations, so it is marked ✗ for AdamW. 8× B200 leaves 180 GB after the 18-byte estimate. Instances have 1 to 8 GPUs depending on the host. Before you start, the dashboard shows each host’s country, GPU count, form factor (SXM or PCIe) and interconnect.
What does a Llama 70B fine-tuning run cost?
Cost = GPUs × wall-clock hours × price per GPU-hour. The hours below are assumptions chosen for illustration, not measurements: run time depends on the number of training tokens, sequence length, batch size and software. Measure your own steps per hour on a short run before you size a long one. The assumed hours match our blocks of 10, 24 and 168 GPU-hours. GPU-hours are sold in blocks of 10, 24, 168 or 720 hours for one GPU model, at the same hourly rate; instances of that GPU draw from them per minute, and unused hours never expire. Every block and GPU is listed on GPU pricing.
Run-cost estimate under assumed hours
| Method | Configuration | Assumed hours (not measured) | GPU-hours | GPUwithCrypto cost |
|---|---|---|---|---|
| QLoRA | 1× A100 80GB | 10 | 10 | $9.20 |
| QLoRA | 1× H100 | 10 | 10 | $23.20 |
| LoRA | 1× B200 | 24 | 24 | $110.64 |
| LoRA | 2× H200 | 12 | 24 | $73.68 |
| Full, 8-bit Adam | 8× H200 | 21 | 168 | $515.76 |
| Full, AdamW | 8× B200 | 21 | 168 | $774.48 |
Prices checked on
How much more does full fine-tuning cost than QLoRA?
About 40 times more per hour of hardware. According to GPUwithCrypto's price check of 29 October 2026, the lowest self-serve on-demand price for an 8× B200 instance was $43.46 per hour (Massed Compute, $5.4325 per GPU). One A100 80GB cost $1.09 per hour at Thunder Compute. The ratio is the same at our prices: $36.88/hour against $0.92/hour.
That gap is why most 70B projects start with QLoRA or LoRA. The LoRA paper reports results on par with full fine-tuning on the tasks it tested, with 10,000 times fewer trainable parameters.
Is the H100 or the A100 better for QLoRA on 70B?
Both hold the 52.5 GB estimate in 80 GB. The H100 has 3.2 times the A100's peak BF16 Tensor throughput (989.5 against 312 TFLOPS dense) and 3,350 GB/s of bandwidth (SXM) against 2,039 GB/s. These are NVIDIA's peak figures; real training speed depends on the software. The A100 costs $0.92/hour and the H100 $2.32/hour at our prices. Pick the A100 when the budget is fixed, the H100 when the deadline is. More: H100 vs A100.
Which software runs each method?
Hugging Face TRL runs QLoRA and LoRA from one command: --load_in_4bit --use_peft for QLoRA, --use_peft alone for LoRA. For multi-GPU LoRA and full fine-tuning, TRL ships ready-made Accelerate profiles such as fsdp2 and zero3 (DeepSpeed ZeRO-3), passed with --accelerate_config. The Llama 3.1 repositories on Hugging Face are gated: accept Meta's licence on the model page first. Step-by-step commands: fine-tune Llama on a rented GPU.
Llama 70B fine-tuning questions
Can I fine-tune Llama 70B on one GPU?
Yes, with QLoRA on an 80 GB GPU: the estimate is 52.5 GB. LoRA on a BF16 base also fits one GPU, but only a 180 GB B200. Full fine-tuning needs eight B200s or eight H200s with an 8-bit optimizer.
Can I fine-tune Llama 70B on an RTX 4090?
No. Even in 4-bit, the 70B weights alone take 35 GB, more than the RTX 4090's 24 GB. The smallest GPU we rent that holds a QLoRA run is an 80 GB A100.
How much VRAM do I need to fine-tune Llama 70B with LoRA?
About 140 GB for the frozen BF16 weights, and about 168 GB with a 20% margin for adapters and activations. One B200 (180 GB), two H200s (282 GB) or four 80 GB GPUs hold it.
How long does fine-tuning Llama 70B take?
It depends on your dataset's token count, sequence length and batch size, and we have not measured a run, so we publish no timing. Start with a 10-hour block, measure steps per hour on your data, then size the full run.
Can I pay for the GPUs with crypto?
Yes. GPUwithCrypto takes BTC, USDT, USDC, ETH, SOL, LTC from a $25 top-up, with no ID and no card. Unused GPU-hours never expire.
Sources (12)
- QLoRA: Efficient Finetuning of Quantized LLMs — arXiv (read 2026-10-29)
- Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA — Hugging Face (read 2026-10-29)
- LoRA: Low-Rank Adaptation of Large Language Models — arXiv (read 2026-10-29)
- Transformer Math 101 — EleutherAI (read 2026-10-29)
- Model training anatomy (GPU memory) — Hugging Face (read 2026-10-29)
- Llama 3.1 70B model card — Meta / Hugging Face (read 2026-10-29)
- TRL command line interfaces — Hugging Face (read 2026-10-29)
- TRL PEFT integration — Hugging Face (read 2026-10-29)
- NVIDIA A100 specifications — NVIDIA (read 2026-10-29)
- NVIDIA H100 specifications — NVIDIA (read 2026-10-29)
- Massed Compute pricing vm.massedcompute.com/pricing — Massed Compute (read 2026-10-29)
- Thunder Compute pricing thundercompute.com/pricing — Thunder Compute (read 2026-10-29)