Pandas DataFrames
The table that does everything Excel does — in code
The DataFrame is analytics' home base: labeled rows and columns that reshape, filter, group and join like a spreadsheet but at warehouse scale and in reproducible code.
What you will learn
- Build DataFrames from dicts, lists and CSV
- Work with columns, index and dtypes
- Slice, loc/iloc and reshape tables
What is this?
A pandas DataFrame is a labeled, two-dimensional table: rows share an index, columns share a name and a dtype. A Series is a single labeled column. Because operations return frames and chain, you can express whole analysis steps as one readable expression.
Why do you need it?
- Labeled data beats raw arrays: names, not row numbers, drive the analysis.
- Vectorized ops run fast even on millions of rows.
- Excel-style pivots, v-lookups and filters are one-liners.
- DataFrames hand off to statsmodels, scikit-learn and plotting with no conversion.
Mental model
A spreadsheet plus its formula engine
A DataFrame
Every cell sits at (row, column) under named headers with formulas that cascade when a source cell changes. A DataFrame is that, but the 'formulas' are methods applied to whole columns at once — and every step is recorded in code.
How it works
- pd.DataFrame(data) builds from dict-of-lists, list-of-dicts, or a numpy array.
- Columns are typed per-cell-family (dtype); object dtype signals mixed text.
- df.shape, df.columns, df.index, df.dtypes, df.info() inspect the skeleton.
- df.head()/df.tail() sample; df.describe() summarizes numeric columns.
- Selection: df[col] returns a Series; df.loc[labels], df.iloc[positions] slice precisely.
- Ordering, renaming and setting indexes make the frame talos layout to your question.
Anatomy of a DataFrame.
Examples
Build and inspect in one breath
import pandas as pd df = pd.DataFrame({ "city": ["austin", "houston", "austin"], "sales": [120, 200, 80],})print(df.shape, list(df.columns), df["sales"].dtype)print(df.loc[df["sales"] > 100])print(df.iloc[0])A three-line build turns a dict into a typed table; loc filters by condition, iloc grabs by position.
In production
- A revenue analyst merges seven source exports into one DataFrame the moment they land, then filters by segment.
- A QA script loads every day's sales CSV into a frame and asserts shape and dtype expectations.
- A migration check compares old and new tables by reading both into frames and diffing on keys.
Best practices
- Inspect shape/dtypes/info immediately after each load before any logic.
- Set a meaningful index when row identity matters; keep default RangeIndex for math.
- Chain operations with .copy() at branch points to avoid SettingWithCopy surprises.
- Name columns once at the boundary so downstream code reads cleanly.
Common mistakes
- Forgetting that df[['a', 'b']] returns a frame but df['a'] returns a Series.
- Using df.loc and believing it's row-by-row when the value you set leaks.
- Mixed dtypes silently becoming object columns that break sums.
- Mutating df inside a loop that reassigns a slice.
Debugging tips
- Stop at dtype surprises: print df.dtypes, then df[col].head() to see the first offenders.
- A SettingWithCopyWarning means a view vs copy problem; use .copy() or regroup.
- RangeIndex vs custom index: after a filter the index keeps gaps — reset_index() if that matters.
- Suspicious aggregates: re-derive on a 20-row slice before trusting the full frame.
Interview checkpoint
Hands-on lab
- Construct a DataFrame from CSV and from dicts
- Inspect shape, dtypes and summary stats
- Select rows by condition and position correctly
Build and Sanity-Check a DataFrame
25 min · beginner
Quick quiz
df[['a','b']] returns:
A Series is a single labeled column of data.