Lesson 1 of 7 in Python and scripting for ops
Bash or Python: picking the right tool
Bash is a shell language built for starting programs and connecting them with pipes. Python is a general language built for data, logic and libraries. Ops work needs both, and the skill is knowing which one fits the job in front of you.
Draft. This lesson is written and waiting for a person to run every command and review it. It is safe to learn from, but check the official documentation for anything you depend on.
Read these first: Linux and the shell
The problem this solves
Most ops automation starts as a handful of commands you type by hand. You paste them into a file and now you have a script. That works until the script needs to read a config file, loop over a list of servers, or call an API and look at the answer.
Bash can do all of that, but it gets awkward fast. Values are text, errors are easy to ignore, and one unquoted space can change what a command does. Python makes those jobs easier, but it is heavier when all you want is to run three commands in a row.
Picking the wrong tool does not stop you. It just makes the script harder to read and easier to break six months later.
How it works
- People and clients
- Runs code
- Other
An ops task goes to Bash when it is mostly running commands, or to Python when it is mostly data and logic, and both finish by reporting an exit code.
- Ops task to Bash: 1a mostly commands
- Ops task to Python: 1b mostly data
- Bash to Exit code: 2 exit 0 or not
- Python to Exit code: 2 exit 0 or not
Look at what the script spends its time doing. If it mostly starts programs and passes their output along, for example grep, sort and tar, Bash is the natural fit. The shell is already the thing that starts programs.
If it parses JSON or YAML, loops with conditions, calls an HTTP API, or needs to explain its own errors clearly, Python is the better fit. It has real data structures and a standard library for files, JSON and logging.
Whichever you pick, the script ends with an exit code. Zero means success. Anything else means failure. Cron, CI systems and other scripts only read that number, so a script that fails but exits zero is lying to everything around it.
A rough rule many teams use is to move from Bash to Python once a script grows past a screen or two, or the first time you want a data structure. It is a judgement call, not a law.
Try it yourself
Do the same small job in both languages. You need a Linux or macOS terminal with python3. Everything stays inside one throwaway folder.
- 1Make a lab folder and a sample log
mkdir -p ~/scripting-lab && cd ~/scripting-lab cat > app.log <<'EOF' INFO start ERROR disk full INFO retry ERROR disk full ERROR timeout EOF
All later concepts reuse this folder. Create it once.
- 2Bash: count the error lines
grep -c ERROR app.log
You should see
3
One command does the whole job. This is where Bash shines.
- 3Python: count each distinct error
python3 - <<'EOF' from collections import Counter from pathlib import Path lines = Path("app.log").read_text(encoding="utf-8").splitlines() counts = Counter(line.split(" ", 1)[1] for line in lines if line.startswith("ERROR")) print(counts.most_common()) EOFYou should see
[('disk full', 2), ('timeout', 1)]Grouping by message is a few lines in Python. In Bash it needs a longer pipeline of
cut,sortanduniq. - 4See that both speak the same exit code language
python3 -c "import sys; sys.exit(3)" echo $?
You should see
3
A Python script can pick its own exit code with
sys.exit. The shell sees it the same way as any other program.
What breaks, and where to look
bash: python: command not found
- WHY IT HAPPENS
- Many Linux distributions ship only
python3, not a command calledpython. - CHECK FIRST
- Run
command -v python3and use that name in your scripts and shebang lines.
script.sh: 3: [[: not found
- WHY IT HAPPENS
- You ran a Bash script with
sh script.sh. On Debian and Ubuntushis a smaller shell that does not know Bash features like[[ ]]. - CHECK FIRST
- Look at the first line of the script (
head -1 script.sh) and run it withbash script.sh, or make it executable and run./script.sh.
cp: cannot stat 'my': No such file or directory (for a file called
my report.txt)- WHY IT HAPPENS
- The variable was not quoted. Bash split
my report.txtinto two words and passed them as two arguments. - CHECK FIRST
- Quote every variable that holds a path:
cp "$file" backup/. Run the script withbash -x script.shto see the exact arguments.
When not to use this
- Do not rewrite a working five-line Bash script in Python just because Python feels more serious. If it does one thing and nobody has trouble reading it, leave it.
- Both languages are poor fits for long-running services and heavy data work. Once a script becomes a program with users and tests, treat it like software, not like a one-off.
Check yourself
What kind of job points toward Bash?
A job that is mostly starting programs and piping their output into each other, like filtering a log with grep and sort.
What kind of job points toward Python?
A job that parses JSON or YAML, loops with conditions, calls an API, or needs clear error messages.
Why does the exit code matter whichever language you choose?
Cron, CI and other scripts only read the exit code. A script that fails but exits zero hides the failure from all of them.
Keep going
Lesson 24 of 92 in the DevOps path