Storage
Intermediate 12 min read Storage

Storage Tiers and Lifecycle

Pay for access, not just bytes

Storage tiers trade price for access latency. Lifecycle policies age data automatically from hot to archive and delete it when obsolete — turning 'cleanup' from a manual chore into a declarative rule.

storage classlifecycletierarchiveS3 Intelligent-TieringcoldGCS nearline
Prerequisites: Object Storage
Before we start

What you will learn

  • Explain the hot/cool/cold/archive price-latency ladder
  • Configure lifecycle transitions that age data safely
  • Compare Intelligent-Tiering, blob access tiers and GCS classes
In 60 seconds

Hot tiers cost more per GB and read instantly; archive tiers are ~10x cheaper but need hours to retrieve. Lifecycle rules move objects by age or size automatically, and delete them at expiry.

Introduction

What is this?

S3 classes: STANDARD → IA → Glacier Instant/IR → Glacier Flexible → Glacier Deep Archive; plus S3 Intelligent-Tiering (auto monitor/move). Azure Blob access tiers: Hot → Cool → Cold → Archive (rehydrate before read). GCS classes: Standard → Nearline → Coldline → Archive, with 'Autoclass' auto-tiering. Transitions charge a data-access fee + minimum 30/90-day storage: small short-lived objects in IA can cost more than STANDARD.

Motivation

Why do you need it?

  • Archives are 80-90% cheaper for the same bytes when rarely accessed.
  • Lifecycle prevents unbounded growth: expiry deletes logs/old versions automatically.
  • Tiering discipline keeps hot storage bill stable as data grows.
  • Declarative rules pass an audit: 'logs expire after 90 days', not 'someone should delete them'.
Think about it

Mental model

Imagine

Files moved from desk drawer to filing cabinet to off-site storage warehouse

Think of

Storage lifecycle tiers

Keep the report you're reading today in the drawer (hot), year-old ones in the cabinet (cold-ish), and closed case files in the warehouse (archive) — with a courier when you finally need one.

Mechanics

How it works

  • Lifecycle rule = filter (prefix/tags) + actions (transition to tier after N days, expire after N days).
  • Archive retrieval requires a restore job (Glacier/Archive rehydration) taking minutes to hours.
  • Transitions are one-way for single-tier rules; Intelligent-Tiering/Autoclass move both ways automatically.
  • Sizes below 128KB and ages under 30/90 days often not worth transitioning.
  • Versioned buckets: apply expiration to noncurrent versions to prune history.

Lifecycle aging chain.

Hot
Infrequent
Cold
Archive
Expire
Hands-on

Examples

A lifecycle policy that ages and purges data

A lifecycle policy that ages and purges data
# AWS S3 lifecycle JSON intent:
#  - move to IA after 30 days
#  - move to Glacier Deep Archive after 365 days
#  - expire (delete) after 1825 days
aws s3api put-bucket-lifecycle-configuration --bucket my-bucket \
  --lifecycle-configuration '{
    "Rules": [{
      "ID": "logs-aging",
      "Filter": {"Prefix": "logs/"},
      "Status": "Enabled",
      "Transitions": [
        {"Days": 30,  "StorageClass": "STANDARD_IA"},
        {"Days": 365, "StorageClass": "DEEP_ARCHIVE"}
      ],
      "Expiration": {"Days": 1825}
    }]
  }'

One rule ages logs cheaply and deletes them after five years — no cron cleanup job needed.

Real world

In production

  • Analytics pipelines move raw parquet to Glacier after 30 days but keep 1 year hot for dashboards.
  • Azure storage auto-tiering keeps user-upload scans in hot then archives after 90 days.
  • GCS Autoclass moves report outputs cold automatically, cutting a storage bill 35%.
Guidelines

Best practices

  • Tie expiry to compliance policy and document the retention reasons.
  • Test lifecycle on a staging prefix before production (data can only leave expensive tiers).
  • Use Intelligent-Tiering/Autoclass when access patterns are unpredictable.
  • Watch minimum-duration fees: tiny/short-lived objects should stay hot.
Watch out

Common mistakes

  • Transitioning small objects to IA and paying minimum-30-day fees that exceed hot cost.
  • Deleting versions/lifecycle accidentally: versioned buckets need noncurrent-expiration, not deletion.
  • Putting a live website on Glacier — first visitor rehydrates for 3 hours, static site 'down'.
  • Forgetting data-access fees when transitioning huge datasets.
Root cause analysis

Common issues

Error
Objects never reach archive
Root cause

Lifecycle rule filter mismatch or transition disabled

Fix

Verify prefix/tags match objects exactly and the rule State is Enabled with the right days.

Error
Archive cost climbing unexpectedly
Root cause

Objects shuffle between tiers on re-reads

Fix

Convert to Intelligent-Tiering or define clear access patterns before tiering.

When things break

Debugging tips

  • Check lifecycle policy for overlaps: two rules matching the same object cause churn.
  • Object stuck mid-tier after transition failed → inspect the bucket inventory for storage class.
  • Restore of archived object slow: request the restore and poll; Deep Archive restores take up to 12h.
  • Bill anomaly = tier piling: use cost breakdown by storage class to find it.
Interviews

Interview checkpoint

Practice

Hands-on lab

Write and Simulate a Storage Lifecycle Policy
25 min · intermediate
In progress
Objectives
  • Design a tiering+expiry rule for a real data set
  • Translate it into an S3/Blob/GCS lifecycle config
  • Weigh small-object and minimum-duration traps

Write and Simulate a Storage Lifecycle Policy

25 min · intermediate

0/4 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/1
1

Archive tiers retrieve as fast as hot tiers.

Go deeper

Official documentation

AWS — S3 lifecycles
docs
Azure — Blob access tiers
docs
GCP — Storage classes
docs