Cloud Object Storage

• Explain object storage's model (buckets, keys, objects) and its consistency guarantees
• Choose storage classes and lifecycle rules to control cost
• Secure access with IAM, bucket policies and encryption

• Cost-per-GB collapses versus a database: keep raw data indefinitely for pennies.
• Durability is engineered (11 nines S3) with automatic replication across facilities.
• Versioning and lifecycle let you keep every overwrite and auto-purge cold data without custom jobs.
• It is the interchange format: sources publish to objects, tools read them, no one owns an ETL port.

You don't organize the crates by shelf; you label each crate with a key (folder/name) and the warehouse handles durability, insurance and retrieval. Objects are just labeled crates you can fetch by key or scan with a prefix.

• Buckets are namespaces; objects are bytes + metadata + version-id, addressed by key.
• Storage classes trade access latency/cost: STANDARD, IA (infrequent), GLACIER (archive with restore time).
• Lifecycle rules can transition class automatically and expire objects on a schedule.
• Consistency: S3 now offers strong read-after-write for PUTs of new objects; beware eventual consistency windows in some paths.
• Access via IAM roles/keys, bucket policies, ACLs and server-side encryption (SSE/S3 KMS).
• Data goes in via SDK/CLI/events (S3 Event Notifications) → pipelines (Glue, Lambda, Kafka connectors) → tools (Athena, Spark, Snowflake external data).

• A startup lands every nightly export into S3 as raw/YYYY/MM/DD and runs Athena over it directly, deferring a warehouse for a year.
• A media company version-switches stored content so every publish is a new object version, with a lifecycle rule deleting versions older than 90 days.
• A bank archives 7 years of GLACIER retention for compliance and restores a single month on subpoena via a restore ticket.

• Use key partitioning (dt=, or YYYY/MM/DD) so queries and lifecycles prune cheaply.
• Turn on versioning for troubleshooting, with a lifecycle to expire old versions.
• Apply IAM least-privilege per bucket/prefix; never share long-lived access keys.
• Encrypt at rest with SSE-KMS and set a bucket deny-public-access guard.

• Treating keys as a real filesystem and doing millions of tiny LIST/HEAD calls (object latencies and throttling).
• Forgetting cross-region replication cost and latency when the warehouse lives elsewhere.
• Using STANDARD forever and watching the bill grow while data never gets queried.
• No bucket policy → a leaked role or key exposes everything.

• When a downstream job 'sees nothing', check whether the expected object prefix actually has the object (list one prefix).
• For 'impossible' missing objects, check eventual consistency windows after concurrent writes and read-after-list.
• Cost spike: query the cost breakdown by storage class and region — Glacier rehydration or IA minimums are common culprits.
• Access denied yet the key fits: diff the IAM policy, bucket policy, and any ACL/session tags all at once.