Lesson 5 of 8 in Networking for DevOps
NAT and routing: how traffic leaves
Routing is how each device chooses the next hop for a packet by matching its destination against a routing table. NAT, or network address translation, rewrites the source address of outgoing traffic so many private machines can share one public address.
Draft. This lesson is written and waiting for a person to run every command and review it. It is safe to learn from, but check the official documentation for anything you depend on.
Read these first: IP addresses, CIDR and subnets
The problem this solves
Private addresses are not routed on the internet. A server at 10.0.2.5 cannot send a request to a website and get a reply unless something connects its private address to a public one.
That same mechanism explains why you cannot reach a machine behind your home router from outside, and why a cloud server in a private subnet has no internet until you add a route and a NAT gateway.
How it works
- Runs code
- Network
- Data
A private host sends a packet, the NAT device swaps in its public address and records the mapping, and the website sees only the public address.
- Private host to NAT device: 1 source is 10.0.0.7
- NAT device to Website: 2 source is public IP
- NAT device to NAT table: 3 remember the pair
- People and clients
- Control
- Runs code
- Network
A packet arrives, the router looks up the destination in its table, delivers locally on a specific match, and otherwise uses the default route.
- Packet to Routing table: 1 look up destination
- Routing table to Local subnet: 2 match: deliver here
- Routing table to Default route: 2 no match: default
- Default route to Next router: 3 pass it on
A routing table is a list of destination ranges and where to send each. The router picks the most specific range that contains the destination. The entry 0.0.0.0/0 matches everything, so it is the default route and catches whatever nothing else claims.
With NAT, the device rewrites the source address and port of each outgoing connection and remembers the pairing in a table. The website replies to the public address. The device looks up the pairing and sends the reply to the private host. Home routers, Docker's default bridge and cloud NAT gateways all work this way.
The table has an important limit. An entry only exists because a private host started the connection. A packet arriving from outside with no matching entry has nowhere to go, so NAT blocks it by accident, and that is why inbound access needs a port forward, a load balancer or a tunnel.
In a cloud network, a private subnet reaches the internet through a route that sends 0.0.0.0/0 to a NAT gateway placed in a public subnet. The names differ between providers, and the shape is the same.
Try it yourself
These commands only read your network configuration.
- 1Read your routing table
ip route
The line starting with
default viais your default route and gateway. The other lines are specific ranges that are directly attached. On macOS usenetstat -rn. - 2Ask which route a destination would use
ip route get 8.8.8.8
It prints the gateway and the source address your machine would use. No packet is sent.
- 3Compare your private and public address
ip -4 addr show | grep inet curl -s https://ifconfig.me; echo
The first command shows your private address. The second asks a third-party echo service which address it sees, which is the one your NAT device uses. They differ when you are behind NAT.
- 4Look for Docker's NAT rule (optional)
sudo iptables -t nat -S | grep -i masquerade
Read-only. If Docker is installed you will usually see a
MASQUERADErule, which is its NAT for container traffic. Some systems use nftables, so an empty result is not an error.
What breaks, and where to look
An instance in a private subnet cannot reach the internet:
curl: (28) Failed to connect ... Connection timed out, and package installs hang.- WHY IT HAPPENS
- The subnet's route table has no
0.0.0.0/0route to a NAT gateway, or the NAT gateway is itself in a subnet without a route to the internet. - CHECK FIRST
- Run
ip routeon the instance for a default route, then inspect the subnet's route table in your cloud console and follow the0.0.0.0/0target.
You can reach a machine from inside your network but a friend or a webhook cannot reach it from outside.
- WHY IT HAPPENS
- The machine has a private address behind NAT. No mapping exists for connections started from the internet.
- CHECK FIRST
- Compare
ip -4 addrwith the public address from the previous step. Use a port forward on the router, a cloud load balancer or a tunnel for inbound traffic.
curl: (7) Failed to connect ... Network is unreachable.- WHY IT HAPPENS
- The machine has no route for that destination, often because there is no default route after a network or VPN change.
- CHECK FIRST
- Run
ip routeand look for adefault vialine. If it is missing, the interface or the gateway setting is the problem.
When not to use this
- NAT hides addresses but is not a security control, and you should not rely on it for protection. Use firewall rules and security groups.
- NAT gateways in the cloud are a managed service and can add cost for data processed. If your servers only need a few cloud services, a private endpoint can be a better fit than sending traffic through NAT.
Check yourself
Which route wins when several match?
The most specific one, meaning the longest prefix. The default route 0.0.0.0/0 matches only when nothing else does.
Why can a private host reach a website but an outsider cannot reach the host?
NAT only has a mapping for connections the private host started. Unsolicited inbound packets have no entry.
What does a private subnet need to reach the internet in the cloud?
A default route that points to a NAT gateway, which has a public address.
Keep going
Lesson 20 of 92 in the DevOps path