Container Storage & Volumes
Everything outside the layer is a decision
Storage in containers is layered: containers' writable layers are ephemeral; volumes persist outside the layer. Bind mounts share host paths; named volumes get lifecycle management; clouds attach EBS/PV at the orchestrator level.
What you will learn
- Explain why the container writable layer is not durable
- Choose bind mount, named volume or platform storage
- Handle volume permissions and backups
Write a file in a container without mounting → dies with the container. Named volumes persist in the storage-driver-managed area; bind mounts point at host paths; orchestrators project cloud disks (EBS/PV/Disks) into pods.
What is this?
Writable layer: fastest to write, lost on rm, COW overhead. Named volumes: docker volume-managed, survive rm/stop, stored under /var/lib/docker/volumes. Bind mounts: host path → container path; use for config/dev-code; care with permissions (UID mapping). tmpfs: in-memory, temp/secrets, lost on restart. K8s: emptyDir (same as tmpfs-ish), hostPath, PVC→PV (network/volume providers), CSI (EBS/gcePersistentDisk/azureDisk). Backups: volume snapshots/CSI snapshots, replication, and patterns (drain replicas, RPO). Rights: docker -v ownership, K8s fsGroup/securityContext.
Why do you need it?
- State outside the layer survives deploys, restarts, and platform migrations.
- Persistent volume classes decouple 'I need a disk' from 'which disk'.
- tmpfs keeps secrets/pipes out of disk at a cost (no persistence, memory only).
- Backed-up volumes turn a container crash into a warm restart, not data loss.
Mental model
A scratchpad (writable layer) vs a filing cabinet (volume)
Container ephemeral vs persistent storage
Notes you don't file are thrown away when the desk closes; the cabinet survives. You decide what's paper that matters at the start, not during cleanup.
How it works
- Create a volume/bind; the runtime mounts it into the container FS at the path.
- Writes to that path hit the volume (host or remote), bypassing the layer.
- Orchestrator provisions a cloud disk (via CSI/PV) and attaches it; it survives pod rescheduling.
- Multi-replica writes require coordination (disk locks/PVC RWO vs RWX) — plan accordingly.
- Snapshots/CSI snapshots protect volume state for restore and migration.
Where data lands.
Examples
Volume and bind mount in one command
docker volume create data-voldocker run -d --name db -v data-vol:/var/lib/postgres/data -v "$PWD/config:/etc/pg:ro" db-image:1.0# data-vol survives the container; config bind is read-onlyOne volume for state (durable, managed lifecycle is the DB's), one read-only bind for config you iterate locally.
In production
- Postgres/X on a named volume or k8s PV with snapshots scheduled.
- Static/upload content on a shared RWX volume (NFS/efs) behind multiple replicas.
- Config secrets injected at mount time via secrets/ConfigMaps rather than baked images.
Best practices
- Put state only on volumes; treat the writable layer as disposable.
- Mount secrets/tmpfs read-only and ephemeral where possible.
- Pick RWO vs RWX consciously; shared writes need a locking-aware app.
- Schedule snapshots and test restores — volume durability is a promise, not a given.
- Track volume ownership with delete-protection to prevent surprise rm -rf.
Common mistakes
- Running Postgres on the container layer and losing the DB on redeploy.
- Bind-mounting host paths into prod images with mismatched permissions.
- RWX volumes backing a binary-format DB (corruption risk, not a feature).
- No snapshot policy: 'the volume is persistent' becomes 'the data is gone'.
Debugging tips
- docker inspect -f '{{ json .Mounts }}' shows host source paths accurately.
- Permission errors mount-time → UID/fsGroup mismatch; check the image user vs volume owner.
- Container starts but DB read-only → mount mode (ro) or disk full on the PV.
- Cross-pod conflicts → verify access mode; switch RWX schemes or single-leader writes.
Interview checkpoint
Quick quiz
Which storage survives a container remove?
K8s persistent volumes are attached via: