Guide · Hugging Face TRL commands checked against its docs · prices of 29 October 2026

How do I fine-tune Llama on a rented GPU?

A LoRA or QLoRA adapter is the cheapest way to teach Llama your data. This guide picks the GPU, runs the training with Hugging Face TRL, and gets the adapter off the machine before the disk is wiped.

Last updated: October 29, 2026 · 7 min read

Quick answer

How do I fine-tune Llama on a rented GPU?

To fine-tune Llama on a rented GPU, train a LoRA or QLoRA adapter with Hugging Face TRL. Llama 3.1 8B needs about 6 GB in QLoRA and 19.2 GB in LoRA, so one 24 GB RTX 4090 is enough; Llama 3.1 70B in QLoRA needs about 52.5 GB, one A100 80GB. Copy the adapter off before deleting the instance. As of 29 October 2026, GPUwithCrypto rents the RTX 4090 at $0.28/hour, paid in crypto.

Which GPU do I need to fine-tune Llama?

Memory by the rules in our VRAM guide: QLoRA ≈ 0.75 GB per billion parameters, LoRA = 2 bytes per parameter × 1.2, full fine-tuning 16–18 bytes per parameter. Our prices of 29 October 2026.

Model	Method	VRAM (rule)	GPU	GPUwithCrypto per hour
Llama 3.1 8B	QLoRA	6 GB	1× RTX 4090 (24 GB)	$0.28/hour
Llama 3.1 8B	LoRA (BF16 base)	19.2 GB	1× RTX 4090 (24 GB)	$0.28/hour
Llama 3.1 8B	Full fine-tune	128–144 GB + activations	1× B200 (180 GB)	$4.61/hour
Llama 3.1 70B	QLoRA	52.5 GB	1× A100 80GB or 1× H100	$0.92/hour or $2.32/hour
Llama 3.1 70B	LoRA (BF16 base)	168 GB	1× B200 or 2× H200	$4.61/hour or $6.14/hour

Prices checked on October 29, 2026

Disk: the Llama 3.1 8B Instruct weights take 16.1 GB, the 70B Instruct weights 141.1 GB. Leave room for checkpoints.

How do I run a QLoRA fine-tune of Llama 3.1 8B?

Commands from the Hugging Face TRL and Hub documentation. Swap the model id for meta-llama/Llama-3.1-70B-Instruct on an 80 GB GPU to train the 70B model the same way. New to rented GPUs? How to rent a GPU for AI covers the account, SSH and a first job.

Rent the GPU
Start a 1× RTX 4090 instance for the 8B model, or a 1× A100 80GB for 70B. Each instance gives you whole GPUs, never a time-shared slice or a MIG partition. You get root SSH access to Ubuntu with the NVIDIA driver, CUDA and PyTorch installed, or start from your own Docker image.
Get access to Llama
The meta-llama repositories on Hugging Face are gated: request access on the model page and accept Meta's licence, then run hf auth login on the instance with a read token.
Install TRL, PEFT and bitsandbytes
pip install "trl[peft]" bitsandbytes. bitsandbytes provides the 4-bit loading that QLoRA needs.
Pick a dataset
Use a Hugging Face dataset id or your own file of conversations. The example uses trl-lib/Capybara, the dataset in TRL's own examples.
Train the adapter
Run the trl sft command below inside tmux. --load_in_4bit loads the base in 4-bit, --use_peft adds the LoRA adapter, and checkpoints land in --output_dir.
Test the adapter
Load the base model, attach the adapter with PEFT and compare answers with the base model on your own prompts.
Copy the adapter off, then delete
Every instance has local NVMe disk, sized for its GPU and shown before you start. The disk is wiped when you delete the instance, so copy your checkpoints out first. Copy the output folder to your computer or push it to a private Hugging Face repository, then delete the instance.

train-qlora.sh

pip install "trl[peft]" bitsandbytes
hf auth login

trl sft \
  --model_name_or_path meta-llama/Llama-3.1-8B-Instruct \
  --dataset_name trl-lib/Capybara \
  --load_in_4bit \
  --use_peft \
  --lora_r 32 \
  --lora_alpha 16 \
  --lora_target_modules q_proj k_proj v_proj o_proj gate_proj up_proj down_proj \
  --learning_rate 2.0e-4 \
  --num_train_epochs 1 \
  --per_device_train_batch_size 1 \
  --gradient_accumulation_steps 16 \
  --gradient_checkpointing \
  --save_strategy steps \
  --save_steps 200 \
  --output_dir llama-3.1-8b-qlora

What do the main training flags do?

--lora_r 32 --lora_alpha 16: the adapter's rank and scaling, the values in TRL's PEFT examples. Rank 32 on all seven linear layers of Llama 3.1 8B trains about 84 million parameters, about 1% of the model.
--learning_rate 2.0e-4: TRL's documentation recommends about 10 times the full fine-tuning rate (2.0e-5) for LoRA.
--per_device_train_batch_size 1 --gradient_accumulation_steps 16: an effective batch of 16 with the memory of a batch of 1.
--gradient_checkpointing: recomputes activations instead of storing them, trading time for memory.
--save_strategy steps --save_steps 200: writes a checkpoint every 200 steps, so a crash costs at most 200 steps.

save-adapter.sh

# from your computer: copy the adapter folder before you delete the instance
rsync -avP -e "ssh -p SSH_PORT" root@HOST:/root/llama-3.1-8b-qlora/ ./llama-3.1-8b-qlora/

# or, on the instance: create a private repo on huggingface.co first, then upload
hf upload your-name/llama-3.1-8b-qlora ./llama-3.1-8b-qlora .

# later, serve the base model with the adapter in vLLM
vllm serve meta-llama/Llama-3.1-8B-Instruct \
  --enable-lora --max-lora-rank 32 \
  --lora-modules my-adapter=./llama-3.1-8b-qlora

What does fine-tuning Llama on a rented GPU cost?

Cost = GPUs × wall-clock hours × price per GPU-hour. The hours below are assumptions for illustration, not measurements: run time depends on your dataset's tokens, sequence length and batch size. We chose hours that match our blocks of 10 and 24 GPU-hours. GPU-hours are sold in blocks of 10, 24, 168 or 720 hours for one GPU model, at the same hourly rate; instances of that GPU draw from them per minute, and unused hours never expire. Every GPU and block is listed on GPU pricing.

A small GPU is the biggest saving. According to GPUwithCrypto's price check of 29 October 2026, the lowest self-serve on-demand RTX 4090 cost $0.33 per hour (Salad) and the lowest H100 $2.73 (Massed Compute): 8.3 times more, for an 8B QLoRA job that fits in 24 GB either way. The RTX 4090 price page lists the providers we checked.

Run-cost estimate under assumed hours

Job	Configuration	Assumed hours (not measured)	GPU-hours	GPUwithCrypto cost
Llama 3.1 8B, QLoRA	1× RTX 4090	10	10	$2.80
Llama 3.1 8B, LoRA	1× RTX 4090	24	24	$6.72
Llama 3.1 70B, QLoRA	1× A100 80GB	24	24	$22.08
Llama 3.1 70B, LoRA	2× H200	12	24	$73.68

Prices checked on October 29, 2026

How do I avoid losing work on a rented GPU?

Save often, copy off early and watch the balance. Instances are on-demand, not spot: nobody can take your GPU back while your balance covers it, and the instance runs until you stop it. When the GPU-hours of a GPU run out, its instances stop and their disks are kept; a new block of hours lets you start them again. If a run stops, Hugging Face's Trainer resumes from the last checkpoint in the output folder when you call trainer.train(resume_from_checkpoint=True) from Python. Unsloth and Axolotl are two other well-known LoRA tools; this guide uses TRL because its command-line flags are documented on Hugging Face's own pages.

Llama fine-tuning questions

How much VRAM do I need to fine-tune Llama 3.1 8B?

About 6 GB with QLoRA and 19.2 GB with LoRA on a BF16 base, so a 24 GB RTX 4090 runs both. Full fine-tuning needs 128 to 144 GB before activations, one 180 GB B200.

Can I fine-tune Llama 70B on one GPU?

Yes, with QLoRA on an 80 GB A100 or H100: the estimate is 52.5 GB. LoRA on a BF16 base needs about 168 GB, one B200 or two H200s. More in the best GPU to fine-tune Llama 70B.

Should I use LoRA or QLoRA?

QLoRA when memory is the limit: it loads the base model in 4-bit and roughly divides the memory by three against LoRA. LoRA when the GPU has room: the base stays in BF16. Compare both adapters on your own evaluation set.

How long does fine-tuning Llama take?

It depends on the number of training tokens, the sequence length and the batch size, and we have not measured a run, so we give no timing. A 10-hour block is enough to measure your steps per hour before a long run.

Can I pay for the GPU with crypto?

Yes. GPUwithCrypto takes BTC, USDT, USDC, ETH, SOL, LTC from a $25 top-up, with no ID and no card. Instances are billed per minute at the hourly rate while they run, from the GPU-hours you bought for that GPU. Stopping an instance stops the charge at once.

Sources (12)

TRL command line interfaces — Hugging Face (read 2026-10-29)
TRL PEFT integration (LoRA, QLoRA, saving adapters) — Hugging Face (read 2026-10-29)
Hugging Face CLI guide (hf auth login, hf upload) — Hugging Face (read 2026-10-29)
Trainer and TrainingArguments — Hugging Face (read 2026-10-29)
Llama 3.1 8B model card — Meta / Hugging Face (read 2026-10-29)
Llama 3.1 70B model card — Meta / Hugging Face (read 2026-10-29)
The Llama 3 Herd of Models (Table 3) — arXiv (read 2026-10-29)
QLoRA: Efficient Finetuning of Quantized LLMs — arXiv (read 2026-10-29)
LoRA: Low-Rank Adaptation of Large Language Models — arXiv (read 2026-10-29)
vLLM recipes (LoRA serving flags) — vLLM project (read 2026-10-29)
SaladCloud price changes, November 2026 blog.salad.com/saladcloud-price-changes-november-2026 — Salad (read 2026-10-29)
Massed Compute pricing vm.massedcompute.com/pricing — Massed Compute (read 2026-10-29)