Cloud Data Services
Beginner 15 min read Cloud Data Services

The Data Cloud and Managed Platforms

The platform that markets itself as easy

Modern cloud vendors (and vendors like Snowflake and Databricks) sell data platforms as a service: storage, compute, catalog, governance and integration in one product with consumption-based pricing.

data cloudSnowflakeDatabricksmanaged platformwarehouse as a serviceinterconnect
Prerequisites: Cloud Object Storage
Before we start

What you will learn

  • Describe the services a 'data cloud' bundles (storage/catalog/compute/governance)
  • Compare compute-and-storage separation vs classic warehouses
  • Evaluate vendor lock-in, egress and interoperability trade-offs
Introduction

What is this?

A 'data cloud' is a managed platform that fuses storage, compute, catalog, governance and data sharing under one control plane — snowflake, Databricks Lakehouse, Google BigQuery, Microsoft Fabric. Compute and storage are typically separated (you pay per query-hour and per byte stored independently), an elastic catalog tracks every object, and governance (access, lineage, quality) is a first-class service rather than a bolt-on.

Motivation

Why do you need it?

  • It removes the 'build five tools and glue them' tax: catalog, warehouse and governance come integrated.
  • Elastic, separated compute means you pay for what you run, not a fixed cluster that idles.
  • Data sharing across org boundaries becomes an exchange (no one emails CSVs).
  • Serverless consumption lets small teams get enterprise-grade quality without a platform team.
Think about it

Mental model

Imagine

Booking electricity instead of building a generator

Think of

A data cloud platform

You don't wire a house to your own power plant — you buy metered electricity from a grid you share. Data clouds meter queries and storage the same way and sell you a governance switchboard on top.

Mechanics

How it works

  • Storage in a vendor-native format (or open formats like Parquet via Iceberg/Delta on object storage).
  • Compute clusters spin up per query and scale horizontally; warehouses = virtual compute endpoints.
  • Catalog centralizes schemas, tables, volumes, functions and lineage across the org.
  • Governance layer unifies row/column access, masking, tagging and compliance policies.
  • Data sharing/cleanrooms allow governed access to other people's data (and your own) without copying.
  • Integrations/connectors ingest from SaaS and databases; open catalogs expose it to Spark/Athena/Trino.

Layers inside a typical data cloud.

Integrations
Catalog
Compute
Governance
Sharing
Real world

In production

  • A ten-person fintech stands up Snowflake, loads SaaS data via a connector, and serves CFO reports on day one — no platform team, no self-hosted orchestration.
  • A retailer shares a governed product catalog with suppliers via a cleanroom, so exact revenue numbers cross orgs without copying files.
  • An analytics team runs ad-hoc BigQuery queries on inbound events while a marketing warehouse auto-scales to zero at night.
Guidelines

Best practices

  • Ground decisions in total cost of ownership: storage, compute, egress and idle-cluster behavior.
  • Prefer open table formats (Iceberg/Delta) so a future exit doesn't ransom your data.
  • Standardize on one catalog for lineage and access even if compute is multi-engine.
  • Set up per-role cost allocation so each team sees its own consumption bill.
Watch out

Common mistakes

  • Assuming 'serverless' means free — runaway queries and overnight cubes burn real money.
  • Copying everything into one vendor and then discovering egress is expensive to leave.
  • Letting each team provision its own isolated workspace, fragmenting the catalog.
  • Skipping cost alerting until the monthly invoice shocks the finance team.
When things break

Debugging tips

  • Slow query → check if compute autoscale met the query or concurrency throttled it to a queue.
  • Cost spike → run the vendor usage explorer by warehouse/time-of-day; overnight backfills are the usual suspect.
  • Data invisible → the object is in storage but the catalog hasn't refreshed (external table metadata drift).
  • Access denied → check the platform's tag/masking policy layer, which can hide columns even with table grants.
Interviews

Interview checkpoint

Practice

Hands-on lab

Model Data Cloud Consumption Pricing
30 min · beginner
In progress
Objectives
  • Build a cost model separating storage and compute
  • Simulate autoscale to zero and backfill spikes
  • Derive a per-query and per-month bill comparison

Model Data Cloud Consumption Pricing

30 min · beginner

0/4 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

Which benefit best describes compute-storage separation in data clouds?

2

Open table formats are a hedge against platform lock-in.

Go deeper

Official documentation