highIntermediateUSE#file-descriptors#ulimit#open-files#emfile

Too many open files — EMFILE error on process

EMFILE: Too many open files error in application logs

Likely source:availabilityperformance

Focused incident mode — strips away everything but this runbook.

Symptoms

  • EMFILE: Too many open files error in application logs
  • accept() or open() calls fail
  • Connection refused or dropped connections under load

EMFILE is a common failure under load. The default ulimit of 1024 is too low for any network-facing service. Always set this in the service unit file, not just in the shell.

Evidence to grab

Collect these artifacts before you change anything — they prove (or rule out) each cause.

  • Current open file descriptor count

    ls /proc/<PID>/fd | wc -l

  • Process ulimit for open files

    cat /proc/<PID>/limits | grep 'open files'

Diagnostic flow

Run the steps in order, marking each claim confirmed or refuted. The evidence you gather feeds the ranked causes below.

Guided decision tree is ready.

Steps with commands, expected output, and pass/fail branches.

Ranked root causes

Evidence checklist

  • ulimit -n shows 1024 (default) for the service
  • FD count is at the limit

Confirm before fixing

cat /proc/<PID>/limits | grep 'open files'
READ

Check the hard and soft limits

Fix

ulimit -n 65535
CHANGE

Raise the open files limit (set in the service unit file or /etc/security/limits.conf)

Resolution runbook

Apply from top to bottom after the cause is confirmed. Review every command before running it — risk tags are shown on each.

1
ls /proc/<PID>/fd | wc -l
READ

Count current open FDs for the failing process

2
cat /proc/<PID>/limits | grep 'open files'
READ

Check the ulimit for open files

3
lsof -p <PID> | wc -l
READ

Detailed count of open files

4
ulimit -n 65535 && systemctl restart <service>
CHANGE

Raise the limit and restart the service

Prevention

  • Set LimitNOFILE=65535 or higher in systemd unit files for all network services.
  • Alert on FD usage exceeding 80% of the process limit.
  • Audit applications for FD leaks by monitoring lsof count over time.
  • Set fs.file-max to a high value (2097152+) at boot via sysctl.conf.

Related runbooks

Same stack, same failure family — worth a skim.

TCP TIME_WAIT socket exhaustion — ephemeral port exhaustionhigh

Cannot assign requested address errors when connecting out

LinuxAdvancedUSE2 causes
Open runbook
Deleted open file holding disk space — phantom disk usagehigh

df shows full disk but du shows less total usage

LinuxIntermediateUSE2 causes
Open runbook

Commands marked READ CHANGE DESTRUCTIVE — never run DESTRUCTIVE commands against production data you cannot restore. Copy → verify → execute.