Google Cloud Platform
Intermediate 16 min read Google Cloud Platform

Cloud Storage & Databases

Google Cloud Storage is object storage: files called objects live in buckets with globally unique names, keyed by a path, with storage classes, versioning and lifecycle rules. Cloud SQL is managed relational storage; Firestore is flexible NoSQL; BigQuery is a serverless warehouse. Learn how to choose among them and operate them.

Cloud Storagebucketobjectkeystorage classversioninglifecycleCloud SQLFirestoreBigQuery
Prerequisites: GCP Platform Overview Compute Engine & IAM
Introduction

What is this?

Google Cloud Storage is object storage: files called objects live in buckets — containers with globally unique names — and are addressed by a key (a path like logs/app/2026/08/run.log). Buckets support storage classes (Standard, Nearline, Coldline, Archive), versioning, lifecycle rules and IAM-based access. Cloud SQL is a managed relational database for PostgreSQL, MySQL and SQL Server. Firestore is a serverless NoSQL document database, and BigQuery is a serverless data warehouse for analytical SQL. Choosing among them is a core architecture decision.

Motivation

Why do you need it?

  • Cloud Storage is where most GCP data actually lives — logs, backups, artifacts, Terraform state and static assets.
  • Cloud SQL removes the operational burden of running your own PostgreSQL or MySQL (patching, backups, high availability).
  • BigQuery is the standard answer for analytical workloads and doubles as a cheap way to explore data with SQL.
  • Firestore provides real-time, serverless-friendly document storage for application state.
  • Object versus relational versus document versus warehouse is one of the most common interview topics.
Think about it

Mental model

Imagine

A warehouse with a ledger office and an analyst reading room

Think of

Cloud Storage, Cloud SQL, Firestore and BigQuery

Cloud Storage is the bulk warehouse: boxes (objects) on shelves (buckets), cheap and enormous, retrieved by label (key). Cloud SQL is the ledger office: rows in strict tables with relationships, kept by a trained bookkeeper. Firestore is the card catalog: flexible index cards pulled instantly by key. BigQuery is the reading room: the analyst piles up records and runs SQL over them to find trends — charged by how many pages are read.

Mechanics

How it works

  • You create a bucket with a globally unique name and a storage class; objects are stored with keys, and both appear in the object's URL.
  • Data is redundantly stored across zones or regions depending on the bucket's location type; lifecycle rules age objects to cheaper classes and delete them on schedule.
  • Bucket IAM and object ACLs control who can read or write; Cloud SQL and Firestore enforce their own IAM plus database-level users.
  • Cloud SQL provisions an instance with automated backups, point-in-time recovery and optional high availability across zones.
  • BigQuery stores data in tables and runs massively parallel SQL queries serverlessly, billing by bytes scanned.

Storing, querying and managing data across GCP data services.

Upload objects to a bucket
Storage classes + lifecycle
Cloud SQL instance
Firestore documents
BigQuery tables
IAM + ACLs
Reference

Key commands

gcloud storage buckets create gs://my-bucket --location=us-central1

Create a bucket in a specific region.

gsutil cp file.txt gs://my-bucket/

Upload an object into a bucket by key.

gsutil ls gs://my-bucket/

List the objects in a bucket.

gcloud sql instances create my-db --database-version=POSTGRES_15 --cpu=1 --memory=4GiB --region=us-central1

Create a managed PostgreSQL instance.

gcloud sql instances list

List the Cloud SQL instances in the project.

gcloud bigquery datasets create my_dataset

Create a BigQuery dataset to hold tables.

Hands-on

Examples

Upload, read and query data like a data engineer

Upload, read and query data like a data engineer
$ gsutil cp report.csv gs://my-bucket/reports/2026/08/report.csv
Copying file://report.csv [Content-Type=text/csv]...
Operation completed over 1 objects/15.0 B.
 
$ gsutil cat gs://my-bucket/reports/2026/08/report.csv
name,status
app,healthy
 
$ bq query --use_legacy_sql=false \
    'SELECT region, COUNT(*) AS total FROM `my-project.mydataset.events` GROUP BY region'
+---------+-------+
| region  | total |
+---------+-------+
| us      |  4210 |
+---------+-------+

gsutil cp uploads an object by bucket and key; gsutil cat streams it back, proving the bytes round-tripped. bq query runs a SQL aggregation against a BigQuery table and prints the result table. The pattern is: objects for raw data, BigQuery for analytical answers.

Real world

In production

  • Store application logs, backups and Terraform state in Cloud Storage with lifecycle rules that age data to Coldline or Archive.
  • Run transactional workloads on Cloud SQL with automated backups, point-in-time recovery and deletion protection enabled.
  • Use Firestore for session state, real-time documents and serverless-friendly application data.
  • Point analytics at BigQuery so teams query data with standard SQL without provisioning servers.
  • Protect critical buckets with versioning, retention policies and IAM-scoped access.
Guidelines

Best practices

  • Choose the bucket location type deliberately: regional for one region, dual-region or multi-region for higher availability.
  • Default to versioning enabled and lifecycle rules from day one — you cannot add history after data is gone.
  • Use IAM roles like roles/storage.objectViewer instead of public ACLs.
  • Enable automated backups, high availability and deletion protection on production Cloud SQL instances.
  • Partition and cluster BigQuery tables so queries scan less data and cost less.
Watch out

Common mistakes

  • Making a bucket public for one object and accidentally exposing every object in it.
  • Choosing BigQuery for transactional data and fighting per-row cost and latency.
  • Running a database on Compute Engine without snapshots and losing data in a VM failure.
  • Forgetting that BigQuery bills by bytes scanned — full-table scans get expensive fast.
  • Using one bucket for everything, then struggling to attach per-team permissions.
Root cause analysis

Common issues

Error
403 Forbidden reading a public object
Root cause

Bucket IAM denies the read, object ACLs block it, or uniform bucket-level access is on.

Fix

Use IAM roles or object ACLs consistently and inspect bindings with gcloud storage buckets get-iam-policy.

Error
Cloud SQL connection refused
Root cause

The client IP is not authorized, or SSL is required and not configured.

Fix

Add the IP to authorized networks or use the Cloud SQL Auth Proxy, and verify SSL settings.

Error
BigQuery quota exceeded
Root cause

Too many concurrent queries, or slot consumption is above the project limit.

Fix

Check the quotas page, reduce concurrency, and rewrite long-running queries to scan less.

When things break

Debugging tips

  • Check the exact error: NOT_FOUND (missing key or bucket), ACCESS_DENIED (IAM) and QUOTA_EXCEEDED all mean different fixes.
  • Use gsutil ls gs://bucket/prefix/ to confirm an object exists before debugging permissions.
  • For Cloud SQL connection problems, test from a VM in the same VPC and check the authorized networks list.
  • For BigQuery cost surprises, check the job's bytesProcessed before re-running an expensive query.
  • Inspect bucket IAM with gcloud storage buckets get-iam-policy before changing ACLs.
Interviews

Interview checkpoint

Practice

Hands-on lab

Store, Verify and Back Up Data Like Cloud Storage
25 min · intermediate
In progress
Objectives
  • Create a bucket-like directory structure with prefixes
  • Write objects (files) and confirm them with ls and cat
  • Verify object integrity with checksums
  • Version and snapshot data the way buckets and backups do

Store, Verify and Back Up Data Like Cloud Storage

25 min · intermediate

0/6 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/3
1

Which service is a fully managed relational database on GCP?

2

BigQuery bills for the data scanned by a query, not for a fixed number of server hours.

3

You need to store and query billions of rows of ad-hoc analytics data with standard SQL, without provisioning servers. Which service fits best?

Quick reference

Cheat sheet

Storage classes
StandardHot data, frequent access
NearlineAccessed less than once a month
ColdlineAccessed less than once a quarter
ArchiveAccessed less than once a year
Lifecycle ruleAges objects between classes, then deletes
Database picker
Cloud SQLRelational OLTP — PostgreSQL, MySQL
FirestoreDocument NoSQL, real-time, serverless
BigQueryAnalytics warehouse, SQL over big data
Cloud SpannerGlobal relational with strong consistency
Go deeper

Official documentation

Cloud Storage Overview
docs
Storage Classes
docs
Cloud SQL Overview
docs
Firestore Overview
docs
BigQuery Overview
docs