Transformers
Intermediate 15 min read Transformers

Fine-Tuning Transformers

Adapting pretrained weights on a small budget

Fine-tuning takes a pretrained transformer and adapts it with task data — full fine-tune, parameter-efficient (LoRA), or frozen-feature heads. The choice tunes the compute/quality trade-off.

fine-tuningLoRAfreezeadapterdataset formatcatastrophic forgetting
Prerequisites: Encoder-Only (BERT) Decoder-Only (GPT)
Before we start

What you will learn

  • Compare full fine-tuning, feature extraction and parameter-efficient (LoRA)
  • Format task data correctly for encoder vs decoder models
  • Pick an approach from task, labels and compute
Introduction

What is this?

Fine-tuning continues training a pretrained transformer on a task dataset. Full fine-tuning updates every weight (powerful, costly, risky). LoRA freezes everything and trains small low-rank adapter matrices — a few percent of parameters, robust and cheap. Frozen feature extraction trains only an added head. All share the pretrained weights' knowledge, so fine-tuning needs far less data than training from scratch.

Motivation

Why do you need it?

  • Pretrained knowledge transfers; your labels just reshape it — thousands of examples beat millions from scratch.
  • LoRA lets you fine-tune a 7B model on one GPU, store compact adapters, and swap tasks trivially.
  • Freezing protects pretrained features from catastrophic forgetting on small datasets.
  • Adapter-side serving is cheap to A/B — switch adapters, not models.
Think about it

Mental model

Imagine

A pilot with a type-rating upgrade

Think of

Fine-tuning a pretrained model

The pilot already knows flying (pretrained); the type-rating course adapts quirk handling (fine-tune). A full type course (full fine-tune) is thorough but long; a refresher module (LoRA) updates just the checklist behaviors; flying the same airframe with a note card (frozen features) works when little changes.

Mechanics

How it works

  • Data format: encoder models take one/many sentence pairs with [CLS]; decoders take instruction-style sequences.
  • Full fine-tune: small LR (1e-5..5e-5), few epochs, watch for overfit and forgetting.
  • LoRA: inject rank-r matrices into W_q/W_v (rules often), train only those; merge/keep separate at inference.
  • Feature extraction: freeze backbone, add a classifier head, train head only (fast, robust).
  • Evaluate on a truly held-out split and compare task metrics, not loss.

Fine-tuning approaches on a pretrained stack.

Pretrained model
Full FT
LoRA
Frozen+head
Evaluate
Real world

In production

  • Chat models: instructed with LoRA on domain data for customer support voices.
  • Search/classification: fine-tune small encoders into high-value classifiers on label gold.
  • Multimodal & agents: adapter fine-tunes keep base weights stable while task skills are layered.
Guidelines

Best practices

  • Start with frozen+head or LoRA; escalate to full fine-tune only when quality demands it.
  • Hold out a real test set and evaluate task metrics — loss alone is misleading after pretraining.
  • Use 1-3 epochs and early stop; transformers adapt fast and forget quickly.
  • Merge LoRA into the base weights at export for zero-latency overhead.
Watch out

Common mistakes

  • Fine-tuning with too-high LR and physically corrupting pretrained embeddings.
  • Applying LoRA rank 128 everywhere and claiming it's 'efficient' — pick r by ablations (8-32).
  • Forgetting to set the model to eval/prompt for inference after FT.
  • Fine-tuning on shuffles that corrupt long-context instruction formatting.
Root cause analysis

Common issues

Error
Task metric improves but general answers get weird
Root cause

Catastrophic forgetting on high LR full fine-tune

Fix

Lower LR, shorter epochs, LoRA/frozen approaches, or mix in general data.

Error
LoRA barely changes behavior
Root cause

Rank too small or adapters applied to wrong layers

Fix

Target W_q/W_v and raise r; verify injected matrices in the forward path.

Error
Training is OOM despite 'efficient' adapter
Root cause

Backbone not frozen (requires_grad leaks) or huge batch

Fix

Freeze backbone params explicitly, use gradient accumulation and smaller batch.

When things break

Debugging tips

  • Overfit a 20-example microset first — plumbing bugs, not data, are the first failure.
  • During FT, if task metric improves while base knowledge degrades, add frozen checkpoints/EWC-style loss.
  • Compare LoRA-merged vs unmerged logits at inference to catch merge bugs.
  • Log adapter scale norms vs base — a wild divergence flags an LR problem.
Interviews

Interview checkpoint

Practice

Hands-on lab

Fine-Tune a Small Transformer Head
35 min · intermediate
In progress
Objectives
  • Load a small pretrained encoder and add a classification head
  • Fine-tune on a tiny synthetic sentiment set
  • Compare frozen-head vs LoRA-style-adaptation outcomes

Fine-Tune a Small Transformer Head

35 min · intermediate

0/4 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

LoRA's core idea is:

2

Fine-tuning transformers always beats feature-extraction, regardless of data size.

Go deeper

Official documentation

LoRA paper
docs