Python for Analytics
Beginner 12 min read Python for Analytics

Working with Files

Get data in, get results out, cleanly

Reading the right file type — CSV, JSON, Excel, plain text — is where raw evidence becomes analysis. pathlib and with-blocks keep file handling safe and portable.

pathlibcsvjsonopenwithparsingfile IO
Prerequisites: Python Data Types
Before we start

What you will learn

  • Read and write CSV, JSON and text files
  • Use with-blocks and pathlib for safe, portable paths
  • Parse messy raw files row by row
Introduction

What is this?

File I/O bridges the outside world: CSVs from exports, JSON from APIs, flat files from warehouses, Excel workbooks from finance. Python reads them with built-in modules — csv, json, pathlib — and the with statement guarantees the file closes even when code throws. Path handling via pathlib makes scripts work on any OS.

Motivation

Why do you need it?

  • Most 'the data is wrong' stories begin with an import that silently misread a file.
  • Safe closing (with) prevents locks, truncated writes and weird cross-platform bugs.
  • pathlib paths compose and inspect themselves (suffix, name, parent) without string surgery.
  • JSON parsing is the default bridge to APIs and modern data lakes.
Think about it

Mental model

Imagine

Handing a file to a clerk who stamps it in and out

Think of

The with block

The clerk guarantees the ledger is closed and locked whether you grab one page or had to sprint away mid-task. with open(...) as f does exactly that — enter, use, always exit and close, even on an exception.

Mechanics

How it works

  • open(path, mode) with modes 'r', 'w', 'a' and 'b' (binary); context-managed by with.
  • csv.reader/csv.DictReader parse delimited text; csv.writer writes rows consistently.
  • json.load / json.dump map files to Python objects; json also handles in-memory strings.
  • pathlib.Path builds portable paths, .read_text()/.write_text() handle string IO simply.
  • Excel needs extra libs (openpyxl/pandas), but its data lands in the same dict/list shapes.

Safe file flow.

Path
Open
Parse
Process
Write
Hands-on

Examples

Read and write with guarantees

Read and write with guarantees
import csv, json
from pathlib import Path
 
p = Path("sales.csv")
with p.open() as f:
    rows = list(csv.DictReader(f))
totals = {}
for r in rows:
    totals[r["city"]] = totals.get(r["city"], 0) + int(r["sales"])
 
with Path("totals.json").open("w") as f:
    json.dump(totals, f, indent=2)
print(p.suffix, p.name, totals)

pathlib builds the path, with guarantees closure, csv.DictReader maps headers, and JSON writes the summary.

Real world

In production

  • A weekly script reads a vendor export CSV, reconciles totals, and writes a JSON status file for the dashboard.
  • An API integration dumps pages of JSON and merges them into a single normalized CSV with csv.writer.
  • A log analyzer streams gigabytes of text line-by-line, extracting errors without loading the file.
Guidelines

Best practices

  • Wrap every open in with to guarantee close on failure.
  • Use pathlib (/) for joins instead of string concatenation — portable and safe.
  • Handle the BOM and line-ending issues explicitly for CSV exports from Excel.
  • Validate a small sample (first 3 rows, types) immediately after each read.
Watch out

Common mistakes

  • Forgetting with, leaking open handles across thousands of iterations.
  • Windows backslash paths that break on Linux; pathlib removes that hazard.
  • Assuming headers are clean — BOM, spaces and quoting mangle DictReader keys.
  • json.load on a file that holds a custom API schema, then blaming 'bad JSON'.
When things break

Debugging tips

  • A UnicodeDecodeError means specifying encoding='utf-8-sig' for Excel exports.
  • Empty output after write often means forgetting to flush/close with.
  • DictReader keys with trailing spaces surface as KeyError — print r.keys() once.
  • For multi-format files, open in text and inspect the first 200 characters before guessing the parser.
Interviews

Interview checkpoint

Practice

Hands-on lab

Reconcile Vendor CSV Exports
30 min · beginner
In progress
Objectives
  • Read CSV with DictReader and handle headers
  • Aggregate totals and validate row count
  • Write a JSON output with pathlib and with-blocks

Reconcile Vendor CSV Exports

30 min · beginner

0/4 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

Which guarantees a file is closed even on an error?

2

csv.DictReader returns rows as strings, not typed numbers.

Go deeper

Official documentation

Python docs: pathlib
docs
Python docs: csv
docs