Generative AI
Advanced 16 min read Generative AI

Fine-Tuning & Adapters (LoRA)

Adapting pretrained weights without retraining the universe

Fine-tuning adapts weights to a task, domain or tone. LoRA freezes the base model and trains a small low-rank pair of matrices — enabling task adapters at a fraction of the memory and cost.

fine-tuningLoRAQLoRAPEFTadapterfull fine-tunecatastrophic forgetting
Prerequisites: Language Model Training RAG
Before we start

What you will learn

  • Distinguish full fine-tuning, adapters and LoRA
  • Explain how low-rank updates save memory
  • Decide fine-tune vs RAG vs prompting for a product
Introduction

What is this?

Fine-tuning continues training on task data — full fine-tuning updates all weights (costly), adapters add small per-layer modules, and LoRA freezes the base and trains a low-rank additive ΔW = B·A per weight matrix. LoRA/QLoRA (quantized base) lets a 7B model be adapted on a single GPU. The base keeps its general knowledge; the adapter encodes the task. Choice: prompting (cheap, weak) → RAG (grounding, no weights) → fine-tune (skill/tone/format at its best).

Motivation

Why do you need it?

  • Prompts hit a ceiling on style, latent-domain knowledge and consistent formatting.
  • Adapters cost pennies to train and store; a fleet of task adapters share one base model host.
  • Low-rank updates are (mostly) additive — you can hot-swap adapters without duplicating the base.
  • Handling catastrophic forgetting: a small, focused LoRA preserves general ability far better than full FT.
Think about it

Mental model

Imagine

A well-trained chef learning a new regional cuisine

Think of

LoRA/adapters on a base model

The chef doesn't relearn knife skills, stocks and dessert basics; they train a slim 'regional add-on' — a few recipes and spices layered on top. Same kitchen, same chef; swap in the new-region card for a different menu and swap it right back tomorrow.

Mechanics

How it works

  • Choose base + rank r (8-64 for LoRA); ΔW = B (d×r) · A (r×d), frozen base.
  • Insert LoRA into attention (and sometimes MLP) weight matrices only — the rest stays frozen.
  • Train with standard SFT loss on task samples, but only the adapter parameters participate.
  • QLoRA: quantize the frozen base to 4-bit + NF4, keeping gradients flowing into the adapter.
  • At inference, merge ΔW into W (or keep separate and switch adapters per request).

Low-rank adapter beside frozen weights.

Base W
ΔW = B·A
Task data
W + ΔW
Task behavior
Real world

In production

  • Domain tone (support voice, legalese) and consistent output schemas.
  • Multi-tenant: one base, dozens of per-customer adapters in RAM.
  • Edge/on-prem: small QLoRA-adapted models serve specialized tasks under compliance.
Guidelines

Best practices

  • Use QLoRA when base ≥ 7B and budget is a single GPU; r 8-16 is usually enough for style/tone.
  • Keep adapter data diverse and balanced — a LoRA on repetitive data overfits fast.
  • Eval: hold out the exact format/style cases; report adapter gain over zero-shot baseline.
  • Version adapters by task + data hash; merge carefully and validate post-merge behavior.
Watch out

Common mistakes

  • Full fine-tuning a 13B model when LoRA hits 95% of the gain at 5% of the cost.
  • Over-fitting a tiny dataset into memorized one-liners that don't generalize.
  • Tuning the base and adapter with identical LR — adapters want their own schedule.
  • Shipping a merged model without re-running the regression eval suite.
Root cause analysis

Common issues

Error
Adapter trains but no behavior change
Root cause

Target modules exclude the right matrices or adapter disabled in eval

Fix

Verify LoRA targets include attention projections; confirm adapter is enabled and unmerged.

Error
Forgetting returns despite LoRA
Root cause

Rank too high coaxing global moves, or data too monotonous

Fix

Lower r, add diverse SFT samples, train fewer steps with early stopping.

Error
OOM on QLoRA fine-tune
Root cause

Survivors of quantization + large batch + gradient checkpoint disabled

Fix

Enable gradient checkpointing, shrink batch/seqlen, or reduce r and blocks.

When things break

Debugging tips

  • Baseline first: prompt-only result is your floor; the adapter must beat it decisively.
  • Check adapter weight norms — exploding/dead adapters signal LR issues inside LoRA.
  • Turn on/off the adapter on the same prompts; the delta shows exactly what it learned.
  • Watch for overfit: eval loss diverges from train loss quickly when data is thin.
Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2
1

In LoRA, the base model weights are:

2

Adapters can be swapped at serving time without retraining the base.

Go deeper

Official documentation

LoRA paper
docs
Hugging Face PEFT
docs