Lesson 2 of 8 in Linux and the shell

The Linux directory tree and where things live

Core8 min

Linux puts every file under a single tree that starts at /, called the root. Standard directories such as /etc for configuration and /var for changing data mean you can find things on an unfamiliar server.

Draft. This lesson is written and waiting for a person to run every command and review it. It is safe to learn from, but check the official documentation for anything you depend on.

Read these first: Shell basics: commands, pipes and redirection

The problem this solves

When something breaks on a server you did not build, you need to know where to look. Where is the config? Where do logs go? Which folder is filling the disk?

Linux answers this with a shared convention. Almost every distribution puts settings, program files, user files and changing data in the same few places, so the skill transfers from one machine to the next.

There are no drive letters. A second disk or network share is attached to a folder in the same tree, so a path alone does not tell you which disk it lives on.

How it works

One tree, and each top folder has a job
One tree, and each top folder has a jobEverything hangs off the root slash, with settings in /etc, logs and changing data in /var, user files in /home and installed programs in /usr.settingschanges whilerunningyoursprograms/the root of everything/etcconfiguration files/varlogs, caches, app data/homeeach user's files/usrinstalled programs
  • Other
  • Control
  • Data
  • Runs code

Everything hangs off the root slash, with settings in /etc, logs and changing data in /var, user files in /home and installed programs in /usr.

  1. / to /etc: settings
  2. / to /var: changes while running
  3. / to /home: yours
  4. / to /usr: programs
A path is read one step at a time from the root
A path is read one step at a time from the rootThe path /etc/ssh/sshd_config starts at the root, enters etc, then ssh, and ends at the file sshd_config.1 open /etc2 open ssh3 open thefile/start at the rootetcfoldersshfoldersshd_configthe file
  • Other
  • Control
  • Data

The path /etc/ssh/sshd_config starts at the root, enters etc, then ssh, and ends at the file sshd_config.

  1. / to etc: 1 open /etc
  2. etc to ssh: 2 open ssh
  3. ssh to sshd_config: 3 open the file

Start at /. A path that begins with a slash is absolute and means the same thing wherever you are. A path that does not begin with a slash is relative and is resolved from your current directory, which pwd prints.

Think of the main folders by how often they change. /etc holds configuration you edit on purpose. /usr holds programs that package managers install. /var holds data that changes while the system runs, including logs in /var/log and service data in /var/lib. /home holds each person's files, and /root is the home of the administrator account.

Some folders are not real disk storage. /proc and /sys are windows into the kernel, where reading a file asks the kernel a question. /tmp is scratch space that the system may clear at reboot or on a timer, depending on the distribution.

Different disks join the tree at mount points. df -h shows which disk backs which folder and how full it is, which is the first thing to check when a server reports it is out of space.

Try it yourself

These commands only read. Run them on a Linux machine, a VM or WSL.

  1. 1
    Look at the root
    cd /
    pwd
    ls

    You will see folders such as etc, home, usr, var and tmp.

  2. 2
    See which names are really links
    ls -ld /bin /tmp

    On most current distributions /bin is a link to usr/bin, shown with an arrow. The t at the end of the /tmp permissions is the sticky bit, which stops users deleting each other's files.

  3. 3
    Find out which system you are on
    head -n 3 /etc/os-release

    This file is the standard place for the distribution name and version.

  4. 4
    See where programs are found
    echo $PATH
    command -v ls

    PATH is the list of folders the shell searches, separated by colons. command -v prints which one matched.

  5. 5
    Check disk space two ways
    df -h /
    df -i /

    The first shows space used. The second shows inodes, the entries that track files. A disk can run out of either.

  6. 6
    Read from the kernel
    head -n 3 /proc/meminfo

    /proc/meminfo is not stored on disk. The kernel generates it each time you read it.

What breaks, and where to look

  • No space left on device, even though you only wrote a small file.

    WHY IT HAPPENS
    Either the disk is full of data or it has run out of inodes, which happens when millions of tiny files pile up. A separate disk mounted at /var can fill while / still has room.
    CHECK FIRST
    Run df -h and df -i and look for a line at or near 100 percent. Then find the folder that grew with sudo du -xh --max-depth=1 /var | sort -h | tail.
  • cat: config.yml: No such file or directory, but you can see the file in another terminal.

    WHY IT HAPPENS
    You used a relative path from a different current directory. Cron jobs and services start in a directory you did not choose.
    CHECK FIRST
    Run pwd and ls in the shell that fails. In scripts and unit files use absolute paths such as /etc/myapp/config.yml.
  • Files you saved in /tmp are gone after a reboot or a few days.

    WHY IT HAPPENS
    /tmp is scratch space. On some distributions it lives in memory and empties at reboot, and on others a timer deletes old files.
    CHECK FIRST
    Run findmnt /tmp to see whether it is a separate memory-backed mount. Put data you want to keep in your home directory or under /var/lib/<app>.

When not to use this

  • The layout is a convention, not a law. Containers, NixOS and some appliances place things differently, so check /etc/os-release and the distribution's documentation before assuming.
  • Do not edit files under /usr by hand. The package manager owns them and the next upgrade will overwrite your change. Put your changes in /etc instead.

Check yourself

Where would you look first for a service's configuration and where for its logs?

Configuration is normally under /etc, and logs are under /var/log or in the systemd journal.

What is the difference between /etc/hosts and etc/hosts?

The first is absolute and always means the same file. The second is relative to the current directory and only works if you are in /.

Why can you get "No space left on device" while df -h shows free space?

The filesystem may be out of inodes, which df -i shows, or the full disk may be a different mount than the one you checked.

Lesson 9 of 92 in the DevOps path

Search

Search cheat sheets and projects