Linux
Intermediate 14 min read Linux

Services & systemd

A service is a background daemon that does a defined job — sshd for SSH, nginx for HTTP, cron for scheduled tasks. On modern Linux, systemd is the init system running as PID 1 that supervises every service: starting it at boot, restarting it on failure, ordering dependencies and collecting logs. Learn the systemctl and journalctl commands that control it all.

Linux servicedaemonsystemdsystemctljournalctlPID 1init systemenablenginxcron
Prerequisites: Processes & Job Control
Introduction

What is this?

A service (or daemon) is a background process that runs without a terminal and does a well-defined job: sshd answers SSH, nginx answers HTTP, cron runs scheduled commands. On modern Linux, systemd is the init system that owns all services — it boots first as PID 1, starts services in dependency order, restarts them when they fail, and captures their logs. Two commands control all of it: systemctl for service life cycle and journalctl for logs.

Motivation

Why do you need it?

  • Services are what a server actually does — every web, database, SSH and observability workload is a supervised service.
  • One mental model: systemctl start/stop/restart/status works identically for every service on the machine.
  • Boot correctness: systemd starts nginx only after the networking and filesystems it depends on are ready.
  • Self-healing and observability: Restart= policies bring crashed services back, and journalctl keeps every log line in one searchable place.
Think about it

Mental model

Imagine

A hotel front desk

Think of

systemd as PID 1

systemd is the front desk: it checks guests in at boot (start), handles complaints (restart), shows who is in-house (status, list-units) and logs every call (journal). Each guest is a service with a defined role. The front desk decides who is allowed to check in first and rings room service again when it drops the phone.

Mechanics

How it works

  • The kernel spawns systemd as the very first process (PID 1); every other process is started by systemd, which also adopts and reaps orphaned processes.
  • Each service has a unit file (/etc/systemd/system/<name>.service) declaring the command to run, its dependencies and its restart policy.
  • systemctl start <name> tells systemd to read the unit, satisfy its dependencies and launch the ExecStart command as a supervised process.
  • When a service exits, systemd applies its Restart= policy (always, on-failure, no) and can relaunch it automatically.
  • systemd-journald captures each service's stdout/stderr into the journal, which journalctl queries with -u <name> or streams with -f.

How systemd supervises a service from boot to logs.

Kernel
systemd (PID 1)
Unit file
Service process
Journal
Reference

Key commands

systemctl status nginx

Show state, enabled-at-boot flag, Main PID and the tail of the service's logs.

● nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; preset: enabled)
     Active: active (running) since Sat 2026-08-08 09:14:23 UTC; 2min 5s ago
    Main PID: 4567 (nginx)
      Tasks: 3 (limit: 4634)
       CPU: 12ms
     CGroup: /system.slice/nginx.service
             ├─4567 "nginx: master process /usr/sbin/nginx"
             ├─4568 "nginx: worker process"
             └─4569 "nginx: worker process"
systemctl start nginx

Start the service now. Success prints no output.

systemctl stop nginx

Stop the service now.

systemctl restart nginx

Stop then start again — the classic way to apply config and recover from a bad state.

systemctl enable nginx

Register the service to start automatically at boot by creating symlinks.

systemctl disable nginx

Remove the auto-start-at-boot registration. It does not stop a running service.

systemctl list-units --type=service --state=running

List every service unit that is currently running.

  UNIT           LOAD   ACTIVE SUB     DESCRIPTION
  cron.service    loaded active running Regular background program processing daemon
  nginx.service   loaded active running A high performance web server
  ssh.service     loaded active running OpenBSD Secure Shell server
journalctl -u nginx -f

Follow the nginx service's journal live, like tail -f.

journalctl -u nginx -b

Show all nginx journal entries from the current boot.

Hands-on

Examples

Start, enable and inspect the nginx service

Start, enable and inspect the nginx service
# 1. Confirm systemd really is PID 1
ps -p 1 -o comm=
 
# 2. Start nginx and check its state
systemctl start nginx
systemctl status nginx --no-pager | head -5
 
# 3. Make it start automatically at boot
systemctl enable nginx
 
# 4. Ask systemd to pick it up and put it down
systemctl restart nginx
 
# 5. Read only nginx's log lines
journalctl -u nginx --no-pager | tail -15

systemctl status shows the unit's state (active/running), whether it is enabled at boot, its Main PID and the most recent log lines. enable creates symlinks so systemd starts nginx automatically. journalctl -u filters the whole journal down to a single unit — the modern replacement for hunting through /var/log files.

Real world

In production

  • Every major distro — Ubuntu, Debian, RHEL, Fedora — runs systemd, so the same commands work across the whole fleet.
  • systemctl status is the first command in most runbooks: it reports state, boot enablement, PID and the tail of the log in one shot.
  • Restart=on-failure with a WatchdogSec keeps flaky workers alive without a human paging at 3am.
  • Containers and Kubernetes pods are supervised processes under the hood — the PID 1 and logging concepts transfer directly.
  • systemd timers replace cron for periodic jobs and get full journal integration for free.
Guidelines

Best practices

  • Learn status/start/stop/restart first; enable/disable (boot behaviour) is a separate concern that is easy to confuse.
  • Use systemctl restart instead of stop+start: it orders the transition and re-reads configuration.
  • Prefer systemctl reload for services that support live reload (nginx) to avoid downtime.
  • Scope log reads to a unit: journalctl -u <unit> -f, and -b to limit results to the current boot.
  • After editing a unit file, run systemctl daemon-reload before restart so systemd sees your changes.
Watch out

Common mistakes

  • Editing a unit file and forgetting systemctl daemon-reload, then wondering why changes never apply.
  • Using enable when you meant start: enable only configures boot behaviour, it does not launch the service now.
  • Running stop then start instead of restart, leaving the service down between the two commands.
  • Stopping at 'Active: failed' without reading the journal — status says the unit failed; the reason lives in journalctl.
Root cause analysis

Common issues

Error
System has not been booted with systemd as init system (PID 1). Can't operate.
Root cause

You are inside a container that has no systemd (the usual case with a plain ubuntu image).

Fix

Use service nginx start or the daemon's binary directly, or run a systemd-enabled image (e.g. jrei/systemd-ubuntu).

Error
Failed to connect to bus: No such file or directory
Root cause

systemd's dbus is not running, typically because systemd is not PID 1 in the container.

Fix

Use a systemd-capable container image, or fall back to service <name> start and read /var/log/<service>/ logs.

Error
Job for nginx.service failed because the control process exited with error code.
Root cause

The service binary failed at startup — often a config error or a port already in use.

Fix

Read the real error with journalctl -u nginx -b and check for a conflicting process with ss -tulpn.

Error
Unit nginx.service could not be found.
Root cause

The package is not installed or you guessed the unit name wrong.

Fix

Install it (apt install nginx / dnf install nginx) and confirm the name with systemctl list-units.

When things break

Debugging tips

  • systemctl status <unit> already shows the last few log lines — read those before anything else.
  • journalctl -u <unit> -b --no-pager | grep -i error finds the smoking gun fast.
  • systemctl --failed lists every unit that failed to start on this boot.
  • An exit code in status (Result: exit-code) means the service process itself failed, not systemd.
Interviews

Interview checkpoint

Practice

Hands-on lab

Supervise a service with systemd and read its logs
20 min · intermediate
In progress
Objectives
  • Launch a container where systemd runs as PID 1
  • Install and start nginx through systemd
  • Make the service auto-start at boot and verify its status
  • Generate traffic and follow the service's logs in journalctl
  • Restart the service and confirm it returns with a fresh PID

Supervise a service with systemd and read its logs

20 min · intermediate

0/7 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/6
1

Which process is PID 1 on a modern Linux system?

2

Which command stops a service from starting automatically at boot?

3

systemctl enable nginx starts the service immediately as well as at boot.

4

How do you follow the live logs of just the nginx service?

5

A unit shows Active: failed. Where do you look first for the reason?

6

nginx crashes every few minutes in production. Which unit setting makes systemd relaunch it automatically?

Quick reference

Cheat sheet

Lifecycle
systemctl start nginxStart the service now
systemctl stop nginxStop the service now
systemctl restart nginxStop then start again
systemctl reload nginxReload config without downtime
systemctl status nginxState, PID and log tail
systemctl is-active nginxPrint active or inactive
Boot behaviour
systemctl enable nginxAuto-start at boot
systemctl enable --now nginxEnable and start at once
systemctl disable nginxStop auto-starting at boot
systemctl is-enabled nginxPrint enabled or disabled
systemctl daemon-reloadRe-read unit files after edits
Discover units
systemctl list-unitsAll loaded units
systemctl list-units --type=service --state=runningRunning services only
systemctl list-unit-filesAll units and enable state
systemctl --failedUnits that failed this boot
Journal (logs)
journalctl -u nginxAll nginx entries
journalctl -u nginx -fFollow nginx logs live
journalctl -u nginx -bOnly the current boot
journalctl -p err -bErrors and worse, this boot
journalctl --since '1 hour ago'Entries from the last hour
Go deeper

Official documentation

systemd project homepage
docs
systemctl man page
docs
journalctl man page
docs
systemd.unit man page
docs