mediumAdvancedRED#keepalive#connection-close#churn#performance#http

Keepalive close storm -- server sends Connection: close, causing connection churn

High rate of TCP TIME_WAIT connections on the server

Likely source:performancescaling

Focused incident mode — strips away everything but this runbook.

Symptoms

  • High rate of TCP TIME_WAIT connections on the server
  • HTTP response headers show Connection: close instead of keepalive
  • Throughput drops and latency increases during traffic spikes due to connection setup overhead

A close storm is usually benign but wastes resources. Focus on the root cause: disabled keepalive, an application bug, or a proxy misconfiguration.

Evidence to grab

Collect these artifacts before you change anything — they prove (or rule out) each cause.

  • Connection header in HTTP responses

    curl -I http://<HOST>/ | grep -i 'connection'

  • TIME_WAIT socket count

    ss -s | grep -i timewait

Diagnostic flow

Run the steps in order, marking each claim confirmed or refuted. The evidence you gather feeds the ranked causes below.

Guided decision tree is ready.

Steps with commands, expected output, and pass/fail branches.

Ranked root causes

Evidence checklist

  • Connection: close header in every response
  • keepalive_timeout is set to 0 or very low

Confirm before fixing

grep keepalive_timeout /etc/nginx/nginx.conf
READ

Check the keepalive timeout setting

Fix

sed -i 's/keepalive_timeout 0/keepalive_timeout 75/' /etc/nginx/nginx.conf && nginx -t && systemctl reload nginx
CHANGE

Enable keepalive with a reasonable timeout

Resolution runbook

Apply from top to bottom after the cause is confirmed. Review every command before running it — risk tags are shown on each.

1
curl -I http://<HOST>/ | grep -i 'connection'
READ

Check the Connection header

2
ss -s | grep -i timewait
READ

Count TIME_WAIT connections

3
grep -E 'keepalive_timeout|keepalive_requests' /etc/nginx/*.conf
READ

Check the keepalive configuration

4
sed -i 's/keepalive_timeout 0/keepalive_timeout 75/' /etc/nginx/nginx.conf && nginx -t && systemctl reload nginx
CHANGE

Enable keepalive if it is disabled

Prevention

  • Always enable HTTP keepalive on the server and proxy (keepalive_timeout > 0).
  • Set keepalive_requests to a high value (1000+) for high-traffic services.
  • Monitor TIME_WAIT socket count and alert if it exceeds 10,000.
  • Ensure the application supports HTTP keepalive and doesn't send Connection: close.

Related runbooks

Same stack, same failure family — worth a skim.

Idle connections silently dropped -- NAT/firewall timeout killing long-lived connectionshigh

WebSocket or long-poll connection drops after ~300s or ~3600s of inactivity

NetworkingAdvanced3 causes
Open runbook
HTTP 502 Bad Gateway -- proxy cannot reach or connect to the upstream servicecritical

curl returns HTTP 502 Bad Gateway

NetworkingIntermediateRED3 causes
Open runbook

Commands marked READ CHANGE DESTRUCTIVE — never run DESTRUCTIVE commands against production data you cannot restore. Copy → verify → execute.