# What is the best GPU for LLM inference?

> The best GPU for LLM inference depends on model size: an RTX 4090 for 8B, an L40S for 32B in FP8, an H200 for 70B in FP8. Paid in crypto, no KYC, Oct 2026.

- URL: https://gpuwithcrypto.com/best-gpu-for/llm-inference/
- Last updated: 2026-10-29
- Publisher: GPUwithCrypto (https://gpuwithcrypto.com)

Pick the cheapest GPU whose memory holds the model plus its cache. The table below does that for every common model size, with our hourly price next to the lowest on-demand price we found elsewhere.

**Quick answer — What is the best GPU for LLM inference?**

The best GPU for LLM inference is the cheapest one whose VRAM holds the model plus about 20%: an RTX 4090 (24 GB) for 7–8B models, an L40S (48 GB) for 32B in FP8 or 70B in 4-bit, an H200 (141 GB) for 70B in FP8, and 8× H200 for 671B DeepSeek. As of 29 October 2026, **GPUwithCrypto** rents the H200 at $3.07/hour, paid in crypto.

## Which GPU should I use for each model size?

VRAM needed = weights × 1.2 (see the [VRAM guide](https://gpuwithcrypto.com/how-much-vram-do-i-need/ "How much VRAM do I need to run or fine-tune an LLM?")). Our price and the lowest self-serve on-demand price per GPU-hour that we found among the providers we checked on 29 October 2026.

| Model size (example) | VRAM needed | Best GPU | Why | GPUwithCrypto | Lowest elsewhere (per GPU) |
| --- | --- | --- | --- | --- | --- |
| 7–8B in BF16 (Llama 3.1 8B, Mistral 7B) | 17.5–19.2 GB | [RTX 4090 24GB](https://gpuwithcrypto.com/gpu-prices/rtx-4090/ "How much does it cost to rent an RTX 4090 per hour?") | Holds BF16 weights plus a 32K-token cache (4.3 GB on Llama 3.1 8B); FP8 Tensor Cores | $0.28/hour | $0.33 (Salad) |
| 7–8B with a 128K-token context | 16 GB + 17.2 GB cache = 33.2 GB | [L40S 48GB](https://gpuwithcrypto.com/gpu-prices/l40s/ "How much does an L40S cost per hour?") | One 128K-token BF16 cache alone needs 17.2 GB | $0.82/hour | $0.97 (Massed Compute) |
| 30–32B in FP8 (Qwen2.5 32B) | 39 GB | [L40S 48GB](https://gpuwithcrypto.com/gpu-prices/l40s/ "How much does an L40S cost per hour?") | 48 GB with FP8 Tensor Cores (733 TFLOPS dense FP8 peak) | $0.82/hour | $0.97 (Massed Compute) |
| 30–32B in BF16 | 78 GB | [A100 80GB](https://gpuwithcrypto.com/gpu-prices/a100/ "How much does an A100 cost per hour?") | Entry 80 GB GPU in our range; no FP8 Tensor Cores, so serve BF16 or 4-bit | $0.92/hour | $1.09 (Thunder Compute) |
| 70–72B in 4-bit | 42–43.6 GB | [L40S 48GB](https://gpuwithcrypto.com/gpu-prices/l40s/ "How much does an L40S cost per hour?") | Entry GPU above 44 GB in our range | $0.82/hour | $0.97 (Massed Compute) |
| 70–72B in FP8 (Llama 3.1 70B, Qwen2.5 72B) | 84–87.2 GB | [H200 141GB](https://gpuwithcrypto.com/gpu-prices/h200/ "How much does an H200 cost per hour?") | One GPU instead of two H100s; 54–57 GB left for KV cache; 4,800 GB/s | $3.07/hour | $3.62 (Massed Compute) |
| gpt-oss-120b (MXFP4) | 78.3 GB | [H100 80GB](https://gpuwithcrypto.com/gpu-prices/h100/ "How much does it cost to rent an H100 per hour?") | Its model card targets a single 80 GB GPU; FP8 Tensor Cores, 3,350 GB/s (SXM) | $2.32/hour | $2.73 (Massed Compute) |
| DeepSeek V4 Flash (284B MoE) | 191.5 GB | 2× H200 | Its FP4/FP8 weights take 159.6 GB; SGLang documents 4× H200 | $6.14/hour | $3.62 (Massed Compute) |
| 405B in FP8 (Llama 3.1 405B) | 486 GB | 4× H200 | 564 GB on four GPUs; 8× H100 would need twice the GPU count | $12.28/hour | $3.62 (Massed Compute) |
| 671B–1.6T MoE (DeepSeek-V3/R1 FP8, V4 Pro) | 805–1,037.6 GB | 8× H200 | 1,128 GB on one host; 8× H100 holds only 640 GB | $24.56/hour | $3.62 (Massed Compute) |

*Prices checked on October 29, 2026.*

Per GPU-hour, USD, taxes excluded. Spot and marketplace prices are not counted: they can be lower, but the GPU can be taken back.

## Why does VRAM matter more than TFLOPS for inference?

VRAM decides whether the model runs at all, and memory bandwidth sets the speed limit for generating text. Each new token reads every active weight from memory once, so at small batch sizes the GPU waits on memory, not on arithmetic. Compute matters more for long prompts (prefill) and for large batches of users.

That is why an H200 and an H100 have the same peak FP8 compute (1,979 TFLOPS dense, NVIDIA figures) but the H200 holds 76% more memory and moves it 43% faster: 4,800 GB/s against 3,350 GB/s. Bandwidth and TFLOPS here are NVIDIA's peak figures, not measurements of ours.

## Which GPU fits which model? (ranked by VRAM)

**Methodology.** Published by GPUwithCrypto (SerpGoat LLC), which rents all seven GPUs. Each GPU is judged on the same three criteria. VRAM and bandwidth are NVIDIA's official figures; no speed was measured by us. Prices are ours and the lowest self-serve on-demand price we found on 29 October 2026.

*Prices checked on October 29, 2026.*

The seven GPUs we rent, listed by VRAM per GPU, largest first, because VRAM decides which model fits. Bandwidth is the second criterion, the price per hour the third.

### 1. NVIDIA B200 — Most VRAM

180 GB of HBM3e at 8,000 GB/s (per-GPU share of NVIDIA's 8-GPU HGX figures), FP8 and FP4 Tensor Cores. Holds a 70B model in BF16 (168 GB) on one GPU.

- Best for: 70B in BF16 on one GPU, FP4 inference
- Price: $4.61/hour · lowest elsewhere $5.43

### 2. NVIDIA H200 141GB — Best for 70B

141 GB of HBM3e at 4,800 GB/s. Holds a 70B model in FP8 (84 GB) with 57 GB left for the KV cache. Eight of them hold DeepSeek-V3 or V4 Pro.

- Best for: 70B in FP8, long contexts, large MoE models
- Price: $3.07/hour · lowest elsewhere $3.62

### 3. NVIDIA H100 80GB — All-round

80 GB of HBM3 at 3,350 GB/s (SXM) or 2,000 GB/s (PCIe), FP8 Tensor Cores. Holds gpt-oss-120b, or a 32B model in BF16.

- Best for: gpt-oss-120b, 32B in BF16, high-throughput 8B serving
- Price: $2.32/hour · lowest elsewhere $2.73

### 4. NVIDIA A100 80GB — Entry 80 GB

80 GB of HBM2e at 2,039 GB/s (SXM) or 1,935 GB/s (PCIe). No FP8 Tensor Cores: serve BF16 or 4-bit.

- Best for: 32B in BF16, 70B in 4-bit with long context
- Price: $0.92/hour · lowest elsewhere $1.09

### 5. NVIDIA L40S 48GB — Best value 48 GB

48 GB of GDDR6 at 864 GB/s, FP8 Tensor Cores, no NVLink. Holds 32B in FP8 or 70B in 4-bit.

- Best for: 32B in FP8, 70B in 4-bit, 8B with 128K context
- Price: $0.82/hour · lowest elsewhere $0.97

### 6. NVIDIA GeForce RTX 5090

32 GB of GDDR7 at 1,792 GB/s, FP8 and FP4 Tensor Cores. Holds a 12B model in BF16 (28.8 GB) or a 32B model in 4-bit (19.5 GB) with room for a long context.

- Best for: 12B in BF16, 32B in 4-bit with long prompts
- Price: $0.42/hour · lowest elsewhere $0.50

### 7. NVIDIA GeForce RTX 4090 — Entry GPU

24 GB of GDDR6X at 1,008 GB/s, FP8 Tensor Cores. Holds a 7–8B model in BF16.

- Best for: 7–8B models, development and testing
- Price: $0.28/hour · lowest elsewhere $0.33

*Disclosure: this page is published by GPUwithCrypto. Competitor details come from their public websites at the date shown above and may have changed since.*

## Is one H200 better value than two H100s?

For a 70B model in FP8, yes: one H200 holds it, while two H100s must split it and exchange data on every layer. According to GPUwithCrypto's price check of 29 October 2026, the median on-demand H200 across 13 providers was $5.29 per GPU-hour, against $4.16 for an H100 across 16 providers. The H200 costs 27% more per hour for 76% more VRAM, so it is 28% cheaper per GB of memory.

At our prices the choice is $3.07/hour for one H200 against $4.64/hour for two H100s. Full comparison: [H100 vs H200](https://gpuwithcrypto.com/compare/h100-vs-h200/ "H100 vs H200: is the H200 worth the extra cost?").

## What about spot and marketplace GPUs?

They cost less and can disappear mid-job. The cheapest H100 offer on Vast.ai's marketplace was $1.73 per hour on 29 October 2026 (Vast.ai public price feed), below the lowest on-demand price we found. For an inference server that must stay up, an interruption means downtime. Instances are on-demand, not spot: nobody can take your GPU back while your balance covers it, and the instance runs until you stop it. When to use each: [what is a spot GPU](https://gpuwithcrypto.com/what-is/spot-gpu/ "What is a spot GPU, and is it worth it?").

## LLM inference GPU questions

### Which GPU do I need to run Llama 70B?

One H200 in FP8 (84 GB needed, 141 GB available), one L40S in 4-bit (42 GB needed, 48 GB available) or one B200 in BF16 (168 GB needed, 180 GB available). Two H100s also hold the FP8 version, at 160 GB combined.

### What is the best GPU for DeepSeek V4?

The H200. DeepSeek V4 Flash's weights take 159.6 GB, so it needs at least two H200s, and the SGLang cookbook documents four. V4 Pro's 864.7 GB need eight H200s. Commands are in the [DeepSeek V4 guide](https://gpuwithcrypto.com/guides/run-deepseek-v4-on-cloud-gpu/ "How do I run DeepSeek V4 on a cloud GPU?").

### Is the RTX 4090 good for LLM inference?

For 7–8B models, yes: 24 GB holds Llama 3.1 8B in BF16 (19.2 GB with overhead) and it is the entry GPU of our range, with 1,008 GB/s of bandwidth. Above about 10B parameters in BF16 it runs out of memory.

### Is the A100 still good for inference in 2026?

For BF16 and 4-bit models up to 80 GB, yes, and it is the entry 80 GB GPU in our range. It has no FP8 Tensor Cores, so FP8 checkpoints run better on an L40S, H100 or H200. See [H100 vs A100](https://gpuwithcrypto.com/compare/h100-vs-a100/ "H100 vs A100: which should I rent?").

### Can I rent these GPUs with crypto?

Yes. GPUwithCrypto takes BTC, USDT, USDC, ETH, SOL, LTC, XMR from a $1 top-up, with no ID and no card. Instances are billed per minute at the hourly rate while they run, from the GPU-hours you bought for that GPU. Stopping an instance stops the charge at once.

## Sources

1. [NVIDIA H200 specifications](https://www.nvidia.com/en-us/data-center/h200/) — NVIDIA (read 2026-10-29)
2. [NVIDIA H100 specifications](https://www.nvidia.com/en-us/data-center/h100/) — NVIDIA (read 2026-10-29)
3. [NVIDIA A100 specifications](https://www.nvidia.com/en-us/data-center/a100/) — NVIDIA (read 2026-10-29)
4. [NVIDIA L40S specifications](https://www.nvidia.com/en-us/data-center/l40s/) — NVIDIA (read 2026-10-29)
5. [NVIDIA HGX platform (B200)](https://www.nvidia.com/en-us/data-center/hgx/) — NVIDIA (read 2026-10-29)
6. [NVIDIA Ada GPU architecture whitepaper (RTX 4090)](https://images.nvidia.com/aem-dam/Solutions/geforce/ada/nvidia-ada-gpu-architecture.pdf) — NVIDIA (read 2026-10-29)
7. [NVIDIA RTX Blackwell architecture whitepaper (RTX 5090)](https://images.nvidia.com/aem-dam/Solutions/geforce/blackwell/nvidia-rtx-blackwell-gpu-architecture.pdf) — NVIDIA (read 2026-10-29)
8. [Transformer Math 101](https://blog.eleuther.ai/transformer-math/) — EleutherAI (read 2026-10-29)
9. [gpt-oss-120b model card](https://huggingface.co/openai/gpt-oss-120b) — OpenAI / Hugging Face (read 2026-10-29)
10. [DeepSeek-V4 cookbook](https://docs.sglang.io/cookbook/autoregressive/DeepSeek/DeepSeek-V4) — SGLang (read 2026-10-29)
11. [Massed Compute pricing](https://vm.massedcompute.com/pricing) — Massed Compute (read 2026-10-29)
12. [SaladCloud price changes, November 2026](https://blog.salad.com/saladcloud-price-changes-november-2026) — Salad (read 2026-10-29)
13. [Thunder Compute pricing](https://www.thundercompute.com/pricing) — Thunder Compute (read 2026-10-29)
14. [Vast.ai public GPU price feed](https://storage.googleapis.com/vast-public-gpu-pricing/gpu-pricing-public.json) — Vast.ai (read 2026-10-29)

---

GPUwithCrypto: GPUwithCrypto rents cloud GPUs (NVIDIA H100, H200, B200, A100, L40S, RTX 4090 and RTX 5090) by the minute, paid in crypto with no KYC; each GPU is priced at least 15% under a named on-demand reference read on 29 October 2026.