Python
Beginner 12 min read Python

Python Environments & Dependencies

Making every run reproducible

Environment and dependency management is the most common source of 'it worked on my machine'. venv, pip pinning and lockfiles make AI services reproducible end to end.

venvpiplockfilesuvpoetryreproducibilitydependencies
Prerequisites: Python for AI Engineering
Before we start

What you will learn

  • Create and activate isolated Python environments
  • Pin dependencies with lockfiles and audit them for known vulnerabilities
  • Reproduce a service environment from scratch deterministically
Introduction

What is this?

A Python environment is the combination of interpreter, packages and versions your code runs under. `venv` isolates an interpreter per project so versions can't conflict. Production AI work adds lockfiles that pin the transitive dependency graph, so a fresh checkout runs byte-for-byte the same pipeline as CI.

Motivation

Why do you need it?

  • Unpinned dependencies mean your Monday morning code runs a different library than your Friday code.
  • Model SDKs release frequently; a background 'pip install --upgrade' can silently change tokenization or API behavior.
  • Security: lockfiles + audits catch vulnerable versions of transitive deps before they ship.
  • Reproducibility is required for evals — a changed dependency can make a golden test pass or fail without touching your code.
Think about it

Mental model

Imagine

An airline's loaded manifest

Think of

A pinned dependency environment

A plane doesn't take off without a manifest of exactly what's on board, checked before every flight. Your AI service shouldn't either: an exact list of what versions exist and why.

Mechanics

How it works

  • `python -m venv .venv` creates an isolated folder with its own interpreter and pip.
  • `pip install -r requirements.txt` installs direct requirements; a lockfile also pins every transitive dependency.
  • Tools like uv or poetry generate and maintain the lockfile deterministically.
  • The lockfile is committed; deploy and CI install from it, not from a loose spec.
  • `pip-audit` or `pip install --dry-run` surfaces known-vulnerable versions periodically.

From loose requirements to reproducible environment.

requirements.in
Resolver
lockfile
CI / deploy
Running service
Real world

In production

  • A RAG pipeline pins the embedding SDK, the vector client and the eval framework in one lockfile so offline evals replicate production behavior.
  • A deploy pipeline runs `pip install -r requirements.lock` inside a container, then `pip-audit` fails the build on a known CVE in a transitive dependency.
  • A team keeps a requirements.in of seven top-level deps while the lockfile manages hundreds of pinned transitive packages.
Guidelines

Best practices

  • Commit the lockfile and install from it everywhere, including local dev.
  • Review dependency upgrades explicitly; never `pip install --upgrade` blindly.
  • Use separate environments for dev vs CI to catch 'not installed in prod' bugs early.
  • Audit dependencies for CVEs in CI, at least on every tagged release.
Watch out

Common mistakes

  • Freezing packages with `pip freeze > requirements.txt` which includes environment-specific junk and misses hashes.
  • Forgetting to activate the venv before running a script, so import resolves to the global site-packages.
  • Two requirements files that drift apart (dev vs prod) until a bug lands only in prod.
  • Pinning model SDK versions in docs but not in code.
Root cause analysis

Common issues

Error
pip install fails with a dependency conflict
Root cause

Two packages require incompatible versions of a shared dependency

Fix

Use a resolver (uv/poetry/pip-tools) that can pick compatible versions, or bump one top-level dep.

Error
Code works locally but the API returns a 400 from the model provider in prod
Root cause

Prod installed an older/newer SDK than local; request format differs

Fix

Install from the same lockfile everywhere and pin the exact SDK version.

Error
pip-audit fails CI on a transitive package
Root cause

A vulnerable numpy/urllib3 pulled in by an indirect dependency

Fix

Upgrade the top-level dependency that pulls it in, or pin a fixed transitively-resolved version.

When things break

Debugging tips

  • 'Works locally, fails in prod' — diff the lockfiles and the interpreter versions first.
  • When a model call changes behavior, check the SDK changelog and installed version.
  • Print `pip list` in CI to prove what actually installed, then compare to your pinned set.
  • If imports resolve to the wrong package, check for a same-named local file shadowing site-packages.
Interviews

Interview checkpoint

Practice

Hands-on lab

Reproduce an Environment from a Lockfile
20 min · beginner
In progress
Objectives
  • Create a venv and install pinned dependencies
  • Generate a lockfile from a small requirements.in
  • Rebuild the environment from scratch and prove it works

Reproduce an Environment from a Lockfile

20 min · beginner

0/5 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

What is the main purpose of a lockfile?

2

A pinned environment is optional for production AI services.

Go deeper

Official documentation

venv docs
docs
uv documentation
docs